How feedback, motor imagery, and reward influence brain self‐regulation using real‐time fMRI

Abstract

The learning process involved in achieving brain self‐regulation is presumed to be related to several factors, such as type of feedback, reward, mental imagery, duration of training, among others. Explicitly instructing participants to use mental imagery and monetary reward are common practices in real‐time fMRI (rtfMRI) neurofeedback (NF), under the assumption that they will enhance and accelerate the learning process. However, it is still not clear what the optimal strategy is for improving volitional control. We investigated the differential effect of feedback, explicit instructions and monetary reward while training healthy individuals to up‐regulate the blood‐oxygen‐level dependent (BOLD) signal in the supplementary motor area (SMA). Four groups were trained in a two‐day rtfMRI‐NF protocol: G_F with NF only, G_F,I with NF + explicit instructions (motor imagery), G_F,R with NF + monetary reward, and G_F,I,R with NF + explicit instructions (motor imagery) + monetary reward. Our results showed that G_F increased significantly their BOLD self‐regulation from day‐1 to day‐2 and G_F,R showed the highest BOLD signal amplitude in SMA during the training. The two groups who were instructed to use motor imagery did not show a significant learning effect over the 2 days. The additional factors, namely motor imagery and reward, tended to increase the intersubject variability in the SMA during the course of training. Whole brain univariate and functional connectivity analyses showed common as well as distinct patterns in the four groups, representing the varied influences of feedback, reward, and instructions on the brain. Hum Brain Mapp 37:3153–3171, 2016. © 2016 Wiley Periodicals, Inc.

INTRODUCTION

During the last decade, several studies have demonstrated that brain's metabolic signals can be voluntarily controlled by healthy individuals and patients by means of neurofeedback (NF) [Birbaumer and Cohen, 2007; Birbaumer et al., 2013]. In the NF studies using the Blood Oxygen‐level Dependent (BOLD) signal, real‐time Near Infrared Spectroscopy (rtfNIRS) [Mihara et al., 2013; Naseer and Hong, 2015; Sitaram et al., 2007b] and Functional Magnetic Resonance (rtfMRI) have been used; the latter being the most informative because of its higher spatial resolution and whole‐brain coverage [Caria et al., 2007; deCharms et al., 2005; Haller et al., 2013; Hamilton et al., 2011; Lawrence et al., 2013; Yoo et al., 2008; Young et al., 2014; Zotev et al., 2011]. Volitional control of brain metabolism can lead to behavioral changes [deCharms et al., 2005; Linden et al., 2012; Ruiz et al., 2013a; Subramanian et al., 2011; Young et al., 2014), therefore opening new opportunities for potential therapeutic and research applications [Lee et al., 2011; Ruiz et al., 2013b; Sitaram et al. 2007a; Sitaram et al. 2011; Sitaram et al. 2014; Weiskopf, 2012].

The magnitude of self‐regulation typically expressed as percentage difference in the hemodynamic signal between the regulation and baseline trials, and the learning effect, expressed as change in the magnitude of self‐regulation over time during NF training [deCharms et al., 2005], are two important measures of a participant's performance during NF training. Both the magnitude of self‐regulation and the learning effect can be influenced by several factors such as type of feedback, reward, instructed mental strategies, session duration, among others [Schwartz and Andrasik, 2003; Sulzer et al., 2013a). Volitional control depends on contingent feedback, as has been demonstrated by the inclusion of control conditions, such as, non‐contingent feedback (sham‐feedback) and mental imagery in the absence of feedback [Caria et al., 2007, 2012; deCharms et al., 2005; Hamilton et al., 2011; Rota et al., 2009; Zotev et al., 2011]. In the majority of rtfMRI‐NF studies so far participants were instructed to control the feedback related to the BOLD signal extracted from the region of interest (ROI) [Caria et al., 2007; deCharms et al., 2005; Lawrence et al., 2013; Scheinost et al., 2013; Sokunbi et al., 2014]. In these cases, contingent (visual) stimuli may intrinsically represent a reward or reinforcement since it guides the desired response based on the internal motivation of the experimental subjects [Fetz et al., 2007; Ruiz et al. 2014; Strehl, 2014].

Furthermore, instructing participants to use certain kind of mental imagery is a common practice, employed with the aim to enhance the efficiency of the learning process [Caria et al., 2007; Hwang et al., 2009; Lawrence et al., 2013; Ray et al., 2015; Rota et al., 2009; Scharnowski et al., 2012; Sitaram et al., 2011; Subramanian et al., 2011; Sulzer et al., 2013b; Yoo et al., 2008; Young et al., 2014; Zilverstand et al., 2015].

However, opposing views have been raised about the importance of these strategies, particularly in electroencephalography NF (EEG‐NF) [Kober et al., 2013; Strehl, 2014] and rtfMRI‐NF studies [Birbaumer et al., 2013; Shibata et al., 2011; Sulzer et al., 2013a]. In fact, learning to self‐regulate brain activity has been proposed as a process of operant conditioning [Birbaumer et al., 2013] since the early reports of brain signal control in non‐human animals [Carmena et al., 2003; Fetz, 1969; Fetz and Finocchio, 1971; Koralek et al., 2012; Philippens and Vanwersch, 2010; Schafer and Moore, 2011; Shinkman et al., 1974; Sterman et al., 1969]. Based on this outlook, explicit and conscious strategies may not be necessary and may even hinder efficient learning [Kober et al., 2013; Witte et al., 2013]. Recently, human studies using rtfMRI‐NF have also started to consider this aspect in their experimental design, increasing the relevance of reward in the training process. Monetary reward, proportional to the desired change in the brain signal, has been used in studies as another factor to reinforce learning [Bray et al. 2007; Buyukturkoglu et al. 2015; Shibata et al. 2011; Megumi et al., 2015], often with protocols in which the participants are not informed or aware of the meaning of the feedback signal [Bray et al., 2007; Megumi et al.,2015; Shibata et al.,2011].

Unravelling the underlying psychobiological process of learning of self‐regulation is one of the most important open issues in the field of NF and Brain‐Computer Interfaces [Emert et al., 2016; Ruiz et al., 2014; Sulzer et al. 2013a]. Finding an optimal strategy to enhance brain self‐regulation is of fundamental importance for the development of NF for clinical interventions.

For this purpose, we focused our study on testing and comparing three factors presumed to influence learning brain self‐regulation: contingent feedback, explicit instructions related to the activity of the ROI, that is, motor imagery, and monetary reward. The first aim of the current study was to compare the efficiency of motor imagery and monetary reward as complementary strategies to contingent feedback.

We trained four groups of healthy individuals using different combinations of these factors, to achieve volitional control of supplementary motor area (SMA). We chose SMA as the ROI, as it has an important role in planning and execution of motor activity and its dysfunction has been related to motor deficits observed in post stroke and movement disorders, such as, Parkinson's disease [Jahanshahi et al., 1995; Nachev et al., 2008; Radman et al., 2013; Roland et al., 1980]. Additionally, the function of SMA has been extensively studied during motor imagery [Gerardin et al., 2000; Guillot et al., 2012; Kasess et al., 2008; Lafleur et al., 2002; Lotze & Halsband, 2006] and also through fMRI‐NF studies [Scharnowski et al. 2015], particularly as a potential intervention for treating Parkinson's disease [Buyukturkoglu et al., 2013; Subramanian et al., 2011].

We evaluated the magnitude of self‐regulation (r _SMA), expressed as percentage difference in the hemodynamic signal between the regulation and baseline trials, and the learning effect (Δr _SMA), expressed as change in the magnitude of self‐regulation over time during NF training [deCharms et al., 2005]. Furthermore, we explored changes in the brain across the different experimental groups with both univariate analysis and functional connectivity (FC) analysis. First, we used univariate analysis by Statistical Parametric Mapping (SPM) to find differences in brain activations in the four experimental groups. Additionally, FC analysis was performed to compare the functional connectivity changes due to feedback, motor imagery, and monetary reward.

MATERIALS AND METHODS

Participants

Twenty male, right‐handed volunteers, aged 18–35 years (22.75 ± 1.6) and without any history of previous psychiatric or neurological disorders were included in the study. Exclusion criteria included the presence of electronic or ferromagnetic body implants and prior history of claustrophobia or panic attacks. All participants in the study were naïve to NF and fMRI experiments. The experimental protocol was approved by the ethics committee of Pontificia Universidad Católica de Chile. Each participant signed a written informed consent on each day of the study.

Participants underwent two days of NF training to achieve volitional control of SMA (ROI 1). To evaluate the effects of the different factors, namely, feedback, motor imagery and reward, on NF training, participants were randomly distributed in four groups of equal size (n = 5), matched by age.

The following were the four groups of participants:

• Group G_F: Participants of this group received only contingent feedback (F) from SMA. No further instructions or strategies to self‐regulate were given.

• Group G_F,I: In addition to contingent feedback, participants of this group were instructed that feedback was proportional to the activity of a movement related area of the brain, and hence participants could use mental imagery of movement (i.e., motor imagery, I) without performing actual movement, for self‐regulation.

• Group G_F,R: Participants were given contingent feedback and monetary reward (R) proportional to the increase in the BOLD signal in the SMA at the end of each up‐regulation block.

• Group G_F,I,R: Participants were given contingent feedback, monetary reward and instructions for motor imagery.

Every group received contingent visual feedback from SMA by means of a graphical thermometer. The bars presented in the thermometer reflected the BOLD signal‐level in SMA. The thermometer was regularly updated at intervals of 1.5 s. All participants were instructed to observe the thermometer display and to increase the thermometer bars knowing that it was related to their brain activity. Participants were additionally informed to consider the delay in the feedback signal due to the tardiness of the brain hemodynamic response as well as due to restrictions imposed by data acquisition and processing.

MR Acquisition

The rtfMRI system was implemented using a Philips Achieva 1.5T MR scanner (Philips Heathcare, Best, The Netherlands) at the Biomedical Imaging Center of the Pontificia Universidad Católica de Chile. A standard 8‐channel head coil was used. Functional image acquisition used Fast Field Echo (FFE) EPI (gradient echo planar imaging) sequence with TR/TE = 1500/45 ms, matrix size = 64 × 64, flip angle α = 70°, FOV: RL = 210 mm; AP = 210 mm; FH = 79 mm. Sixteen slices (voxel size = 3.2 × 3.3 × 4 mm³, gap= 1 mm) oriented with AC/PC alignment and 150 scans (10 dummy scans) were used in each run. Anatomical T1‐weighted (T1W) brain volumes were acquired each training day using T1W‐3D Turbo Field Echo (TFE, magnetization prepared gradient echo also known as MPRAGE) sequence with TR/TE = 7.4/3.4 ms, matrix size =208 × 227, α = 8°, 317 partitions, voxels size = 1.1 × 1.1 × 0.6 mm³, TI = 868.7ms. To prevent discomfort during MRI sessions, pads and air cushions were used to fix the head.

Real‐Time fMRI System

To implement the rtfMRI system, a typical setup used in rtfMRI‐NF experiments was assembled [Caria et al., 2012; Ruiz et al.,2014; Weiskopf et al. 2004]. At the beginning of each measurement, participants were positioned in the scanner and reference scans were acquired. Later, using an EPI sequence (see MR acquisition) functional brain volumes were generated. During image acquisition, brain volumes were transferred in real‐time directly from the scanner's image reconstruction system using the Direct Reconstructor Interface (DRIN) application (Philips Heathcare, Best, The Netherlands) to a PC (BCI PC) which had the necessary software to analyze it in real time. No modifications were done to the scanner's image reconstruction system, as described by Sitaram et al. [2011].

A standard PC running Turbo Brain Voyager 3.0 (TBV‐PC) rtfMRI software (Brain Innovations, The Netherlands) read the incoming ANALYZE (.img/.hdr) brain volumes to perform real‐time 3D motion correction and statistical analysis [Weiskopf et al., 2003]. TBV parameters were set to match parameters of the EPI acquisition and to obtain BOLD signal information coming from the two selected ROIs after each repetition time (TR) of the scans. Custom MATLAB scripts used ROI information to compute the feedback (thermometer bars) by comparing between up‐regulation and baseline blocks (details below). The feedback output was stored in a shared file (text file) in the TBV‐PC to be accessed from another computer (Presentation‐PC) in the local network. Presentation® 17.1 software (Neurobehavioral Systems, USA) read the feedback file continuously and refreshed the images on the screen, corresponding to the calculation of the thermometer bars, on a MR‐compatible visual display system (NordicNeuroLab AS, Norway).

Experimental Protocol

Training sessions

Each training session consisted of a functional localizer, four training runs and a transfer run (Table 1). Training was conducted in 2 days, with at least one day of gap (with no training) between days 1 and 2.

Table 1.

Experimental Protocol

Day 1	Explanation and instructions to participants Training Session 1 ROI localizer run Neurofeedback training runs (4) Transfer run Anatomical MRI acquisition
Day 2	Training Session 2 ROI localizer run Neurofeedback training runs (4) Transfer run Anatomical MRI acquisition
Day 3	Debriefing

ROI localizer trials

SMA (ROI 1) was delineated using both a functional localizer and anatomical references. During the functional localizer, participants performed overt motor execution. To ensure that G_F and G_F,R (groups without motor imagery) do not get any hint that the self‐regulation task that follows the localizer might be related to movement or motor activity, the functional localizer was implemented as a 2‐back task [Conway et al., 2003; Kirchner, 1958] in which participants were presented a sequence of stimuli, and the required task was to report (with a button) if the current stimulus and the stimulus observed “2” steps earlier in the sequence were the same. Although, typically the n‐back tasks are used to evaluate working memory, in this experiment we used the task to ensure frequent button presses to elicit activations in the motor areas. The localizer run included four baseline blocks and three 2‐back blocks, so that each block was presented for 30 s. For anatomical reference, the superior part of the posterior frontal lobe around the perpendicular line to anterior commissure (AC; y = 0) was used. A volume inside Broadmann Area 6 was selected, whose location was later validated in an offline analysis. ROI 1 was delineated as two contiguous slices, each one of 4 × 4 voxels in a transversal brain section covering the highest peak of activation inside Broadmann Area 6. The reference ROI (ROI 2), delineated as a single slice around anterior part of the third ventricle, was selected to cancel effects of global activation.

Neurofeedback training runs

Eight training runs were equally distributed in two scanning days (four runs per day). Each run of 150 brain volumes included first 10 dummy scans (15 s) to reach T1 steady state (which were later discarded), followed by alternating baseline (4) and up‐regulation (3) blocks (20 volumes, 30 s per block). During baseline blocks, volunteers were asked to remain in rest, and the image of the thermometer remained static. During up‐regulation blocks, contingent feedback was provided. Groups G_F,R and G_F,I,R were visually presented the value of their monetary reward, in the last 3 s (2 volumes) of the block, using an image indicating the amount of money earned corresponding to the increase in the BOLD signal in the previous up‐regulation block.

Feedback calculation

As in other previous studies [Caria et al. 2007; Lawrence et al. 2013; Ninaus et al., 2013; Ruiz et al., 2013a), graphical depiction of a thermometer was used as visual feedback of BOLD signal changes, with moving bars showing the increments (red bars rising over middle point) or decrements (blue bars under the middle point) of the BOLD signal in SMA, in comparison to the immediately preceding baseline block, using the following formula:

$$F={\left({\text{BOLD}}_{\text{Upreg}}–{\text{BOLD}}_{\text{Base}}\right)}_{\text{ROI}1}-{\left({\text{BOLD}}_{\text{Upreg}}–{\text{BOLD}}_{\text{Base}}\right)}_{\text{ROI}2}$$ (1)

where F is the feedback value, BOLD_Upreg is the BOLD signal in ROI1 or ROI2 during a moving average calculated from the last three scans during the up‐regulation block, and BOLD_Base the average BOLD signal during the entire immediately preceding baseline block. Subsequently, F was rounded‐off to the closest integer. In case of abrupt changes (considered if greater than eight points in this study) in the F‐value, potentially due to movement artifacts, swallowing, etc., an online correction was applied by replacing the spurious value by the F‐value from the previous TR.

In groups with monetary reward (G_F,R and G_F,I,R), the amount of money given to each participant was calculated in proportion to the mean increase in BOLD in the up‐regulation block in comparison to that of the baseline block. To ensure correspondence between real‐time feedback (thermometer bars) and monetary reward, the Eq. (2) was used for calculating the amount of monetary rewarded:

$$R=F^{\prime }*M$$ (2)

where R is the reward value and M the monetary units per 1 unit of F′. F′ [Eq. (2)] was computed using the same formula as F [Eq. (1)], but considering the percentage BOLD increase in all the up‐regulation block compared to the previous baseline block. The maximum permissible reward for each block was 3 USD. The total reward given to participants was the sum of the amount of money earned in each reward block with a total maximum of 60 USD for the whole training. If the calculation of the monetary reward resulted in a negative number, the value was fixed to zero.

Self‐Reports and Subjective Ratings

At the end of the two training days, participants were asked about their comfort during training, subjective appreciation about the level of control over the feedback signal and descriptions of mental strategies used to control the thermometer. The questionnaire responses were assessed using a 5‐point Likert scale (1: completely disagree/difficult; 5: fully agree/easy).

Transfer Run

At the end of both training days, an additional run was included during which participants were instructed to perform the same up‐regulation/baseline paradigm described previously, but without receiving feedback. The aim of transfer runs was to test if participants can maintain volitional control of the BOLD signal after training. A further aim is to test if magnitude of self‐regulation can be maintained in a context different from a NF experiment. The number and duration of blocks was the same as that of the NF training runs, with the only difference that the thermometer (feedback) was not shown. Unlike previous studies with transfer runs performed only once and during the last day [Caria et al., 2007; Ruiz et al., 2013a; Zotev et al., 2011], here a transfer run was included at the end of each day. The objective was to familiarize the participant with the transfer, avoiding novelty effect over the results.

Offline Processing

Preprocessing

For brain imagining analysis, a spatial pre‐processing step using SPM 8 (Wellcome Department of Imaging Neuroscience, London, UK) was performed, involving motion correction, realignment and slice‐timing correction. Functional EPI images were coregistered with the anatomical images of the same day. Normalization to Montreal Neurological Institute (MNI) space was done in anatomical and functional EPI volumes. Smoothing with a Gaussian kernel of FWHM of 8 × 8 × 8 was applied over all functional volumes.

To execute first level analysis, general linear model (GLM) was defined to evaluate regions responsive to self‐regulation, considering two conditions (up‐regulation and baseline). Convolution of the regressor with canonical hemodynamic response function (HRF) was performed. Six generated motion confounds were added to the model. After estimating the first level model, whole brain analysis was performed at group level to find other areas responsive to up‐regulation (defined contrast = 1 −1, i.e., [up‐regulation – baseline]) using second level analysis in SPM. One‐sample t‐test was performed for each group taking data from the second day only, in order to find significant brain activation of learned self‐regulation and the different factors. Whole brain map (Fig. 4) shows significant t‐values (threshold of P < 0.001 and FDR P < 0.01, cluster size = 10) and are visualized using the xjView toolbox (http://www.alivelearn.net/xjview). Brain regions defined in the AAL atlas [Tzourio‐Mazoyer et al., 2002] were used to find the location of activation. Repeated measures ANOVA was used to determine regions particularly activated for each one of the tested factors (explicit instructions [motor imagery] vs. monetary reward).

Figure 4.

Activation maps during the up‐regulation of SMA obtained from whole‐brain statistical parametric mapping (SPM) during day 2 with one‐sample t‐test, (FDR P < 0.01, cluster size = 10). SMA activity is present in all groups. [Color figure can be viewed in the online issue, which is available at http://wileyonlinelibrary.com.]

SMA‐ROI self‐regulation progress

To determine the effect of BOLD self‐regulation in the target region, smoothed normalized brain volumes were used. The analysis was performed in an 8 mm³ ROI in the SMA (MNI coordinate limits: x: −8, 8; y: −‐8, 8; z: 52, 68). This region was selected comprising a wide area in the SMA, including the regions named in the literature as pre‐SMA and SMA proper [Mayka et al., 2006]. Using mean BOLD values across each run (including training and transfer runs) the Percentage BOLD (r _SMA) was computed as below:

$$r_{S\text{MA}}=100\ast \frac{\text{Mean}\left({\text{BOLD}}_{\text{Upreg}}\right)-\text{Mean}\left({\text{BOLD}}_{\text{Base}}\right)}{\text{Mean}\left({\text{BOLD}}_{\text{Base}}\right)}$$ (3)

where BOLD_Upreg and BOLD_Base are vectors with the mean BOLD signal time series in the SMA‐ROI during up‐regulation and baseline blocks. Group comparison of self‐regulation levels was done using one‐way ANOVA. To measure the participant's learning effect (Δr _SMA) in terms of his improvement in increasing the BOLD signal in the up‐regulation condition compared to the baseline condition over the 2 days of training, we considered the difference of the mean r _SMA in the four runs of second day minus the mean r _SMA of the four runs of first day. All data were checked for normality and non‐parametrical tests were used when appropriate.

Additionally, to determine if intersubject r _SMA variability can be affected by the inclusion of training factors (i.e., feedback, motor imagery, and monetary reward) standard deviation (SD) of the group r _SMA for each run was calculated. In this case, Kruskal–Wallis test was used to check for group effect, U Mann‐Whitney test was used as post hoc and Bonferroni correction was considered. Intrasubject variability was also analyzed by group. For this purpose, r _SMA for each up‐regulation TR was calculated for each subject and the variance of r _SMA (using SD) was calculated for each run. With SD values for each run and participant, repeated measures ANOVA test was calculated. Additionally, significant run effect was tested in each group using Friedman test. A measure of functional SNR, defined as the ratio of the signal difference between the experimental conditions to their combined noise [Huettel et al., 2004], was calculated using the Fisher Score (FS) [Bishop, 1995; Lal et al., 2004; Ruiz et al., 2013a] as follows:

$$\text{FS}\hspace{0.17em}\hspace{0.17em}=\hspace{0.17em}\hspace{0.17em}\frac{{\left[\text{Mean}\left({\text{BOLD}}_{\text{Upreg}}\right)-\text{Mean}\left({\text{BOLD}}_{\text{Base}}\right)\right]}^2}{\text{Var}\left({\text{BOLD}}_{\text{Upreg}}\right)+\text{Var}\left({\text{BOLD}}_{\text{Base}}\right)}$$ (4)

FS was calculated for each subject, run, and block. Repeated measures ANOVA was used to compare the FS across groups, runs, and blocks. Significant run effect was tested in each group using Friedman test.

The relationship between intrasubject variability and delivered monetary reward was also evaluated using linear regression from the G_F,R data.

In all cases, Spearman correlation coefficient was used to test dependence of variability/FS and run progress.

Functional connectivity analysis

FC analysis was performed to recognize network changes during up‐regulation in SMA in different experimental groups. For this purpose, correlation coefficients were computed to measure the linear relationship between BOLD activity in different voxels or regions [Friston, 2011]. To perform ROI to ROI FC analysis, the CONN toolbox was used [Whitfield‐Gabrieli and Nieto‐Castanon, 2012] after the following pre‐processing steps were performed: denoising using bandpass‐filtering (0.008–0.09 Hz), inclusion of estimated head motion parameters, white matter and CSF as covariates, and linear detrending and despiking before calculating regression. Regions inside the field of view were selected from the AAL atlas [Tzourio‐Mazoyer et al., 2002] (please see Supporting Information Table S1) for grouping the brain voxels inside these areas to calculate the ROI to ROI BOLD signal correlations. Additionally, a customized ROI of Nucleus Accumbens was included in the analysis due its relevance in reward processing [Ikemoto and Panksepp, 1999; Knutson et al., 2001].

ROI to ROI bivariate correlations were calculated for up‐regulation blocks. Each ROI pair (seed‐target) was considered as independent from other pairs (i.e., calculation of the correlation coefficient [r] in isolation). Correlation coefficient was calculated according to the formula [Whitfield‐Gabrieli & Nieto‐Castanon, 2012]:

$$r={\left(x^{t}x\right)}^{-\frac{1}{2}}\left(x^{t}y\right){\left(y^{t}y\right)}^{-\frac{1}{2}}$$ (5)

where x and y are vectors of the BOLD time‐series for seed ROI and target ROI.

CONN analysis produced one FC Z matrix (51 × 51 in this case) for each study group (4) and run (8). FC Z matrix contained Fisher‐transformed correlation coefficients z _FC (i,j) (z _FC(i,j)= atan(r(i,j))) between all the i and j ROI pairs. In our analysis, z _FC was used to report the FC values between ROIs.

To assess the similarities and differences of FC in the brain among the groups, FC patterns in different groups were compared with the FC pattern of the group G_F (reference group). In the following sections, two different ways of selecting the ROIs for the above analysis are described. In the two cases, brain regions in G_F are sorted according to two different criteria, namely, (1) Mean pairwise correlation coefficients of functional connectivity (mean z _FC values), from all the training runs, between the top six regions_, and (2) Rate of change, slope, of the the z _FC values between the top six regions over the course of NF training. Additionally, four extra regions were selected for each of the two criteria described above but with restriction of SMA as the seed region (i.e., the selected region needs to be connected with SMA). Finally, right and left SMAs were also included in the analysis for both cases. Therefore, the complete analysis considered 12 regions for each criterion.

Mean FC changes among groups

As the first criterion, mean z_FC values across runs were considered to unveil functional connections that could be relevant during the NF training process. Considering the FC data of the reference group, the 3 pairs of regions with the top mean z_FC values taking in consideration all regions in FC Z matrix (hereafter called the “whole‐brain connectivity matrix”) were selected. To give relevance to SMA as the target region in our NF training, the four regions with top mean z _FC values and (right or left) SMA as seed area were additionally selected. Left and right SMAs were also included in the analysis. In total, mean z _FC analysis included 12 regions (top mean whole‐brain connectivity matrix [n = 6] + top mean SMA‐seeded [n = 4] + R&L‐SMA [n = 2]).

Plots were generated (Fig. 5) considering the 12 selected regions for each group, by presenting significant functional connections ( $\left|z_{\text{FC}}\right|$ ≥ 0.26, P < 0.001) among them. For the purpose of visualization, the thickness of the lines connecting the ROIs is represented to be proportional to the magnitude of z _FC.

Figure 5.

FC values for the selected 12 brain regions with the highest mean correlation values across all the NF training runs. The effect of the different experimental factors (feedback, motor imagery and reward) on FC patterns was analyzed in comparison to the group G_F as the reference group (see section 7.3 for details on the method). The thickness of lines is proportional to z _FC (z _FC values shown on the line). FC patterns across groups were found to be similar to each other. (Precentral L = L‐PreC; Precentral R = R‐PreC; Frontal Sup L = L‐SFG; Frontal Sup R = R‐SFG; Frontal Mid L = L‐MFG; Frontal Mid R = R‐MFG; Supp Motor Area L = L‐SMA; Supp Motor Area R = R‐SMA; Cingulum Ant L = L‐ACing; Cingulum Ant R = R‐ACing; Cingulum Mid R = R‐MCing; Paracentral Lobule L = L‐ParaC). [Color figure can be viewed in the online issue, which is available at http://wileyonlinelibrary.com.]

Slope FC changes among groups

As a second criterion for analysis we used the rate of change of z_FC values across the NF training process, that is, we considered the slope of z _FC curve across the eight training runs. Similar to the previous method used in the analysis of mean FC changes among groups (with G_F as the reference), three pairs of regions with the top z _FC slope from the whole‐brain connectivity matrix were selected. Four additional regions with the top z _FC slope, considering SMA as the seed region, were selected. Again, left and right SMAs were included in the analysis. In total, this analysis included 12 regions (top slope whole‐brain connectivity matrix [n = 6] + top slope SMA‐seeded [n = 4] + R&L‐SMA [n = 2]).

The 12 selected regions and the changes between mean $z_{\text{FC}}$ of day 2 minus the mean $z_{\text{FC}}$ of day 1 ( $\Delta z_{\text{FC}}=\left|z_{\text{FC},2}-z_{\text{FC},1}\right|$ > 0.15) for each group are presented here (Fig. 6). In this case, the thickness of line is proportional to the magnitude of $\Delta z_{\text{FC}}$. The red color of the line was chosen for positive $\Delta z_{\text{FC}}$ values and the dashed blue line for negative $\Delta z_{\text{FC}}$ values.

Figure 6.

FC increases/decreases between 12 brain regions with the highest change rate (slope) across NF. The thickness of each line is proportional to the corresponding Δz _FC for the connection (increases with red lines, decreases with dashed blue line and z _FC values shown on the line). Groups without motor imagery presented higher FC increases than imagery groups (Supp Motor Area R = R‐SMA; Supp Motor Area L = L‐SMA; Precentral R = R‐PreC; Frontal Mid L = L‐MFG; Frontal Inf Oper R = R‐FIO; Frontal Sup Medial L = L‐MSFG; Parietal Inf L = L‐IPar; Angular L = L‐ANG; Precuneus L = L‐PREC; Precuneus R = R‐PREC; Putamen R = R‐PTMN; Pallidum R = R‐PLLD). [Color figure can be viewed in the online issue, which is available at http://wileyonlinelibrary.com.]

Additionally, a correlation change index (CCI) was calculated considering the 12 selected regions (a connectivity matrix with only these regions was generated) and the “whole‐brain connectivity matrix.” CCI summarizes in one value how the brain's functional connectivity changed in the selected network.

$${\text{CCI}}_{\text{incr}}=\left[{\sum }_i{\sum }_j\right(\left|z_{(\text{FC},2)}\left(i,j\right)-z_{(\text{FC},1)}\left(i,j\right)\right|)]/N_{\text{total}}\text{with}\left[z_{(\text{FC},2)}\left(i,j\right)-z_{(\text{FC},1)}\left(i,j\right)\right]>0$$ (6)

$${\text{CCI}}_{\text{decr}}=[{\sum }_i{\sum }_j\left(\right|z_{(\text{FC},2)}\left(i,j\right)-z_{(\text{FC},1)}\left(i,j\right)\left|\right)]/N_{\text{total}}\text{with}\left[z_{(\text{FC},2)}\left(i,j\right)-z_{(\text{FC},1)}\left(i,j\right)\right]<0$$ (7)

with $z_{\text{FC},X}\left(i,j\right)$ being the mean value of $z_{\text{FC}}$ between regions i and j during day X, and N _total being the total number of possible bi‐regional functional connections (66 for the selected 12 regions, and 1275 for whole‐brain connectivity matrix). Since the FC Z matrix is symmetrical, and to avoid unnecessary duplication of the calculus, only one permutation of the pair of ROI i − j was considered in the summation [see Eqs. (6) and (7)] i.e. only half of FC Z matrix was considered to calculate CCI. Consequently, ${\text{CCI}}_{\text{incr}}$ and ${\text{CCI}}_{\text{decr}}$ express the mean increase and decrease (between training days 1 and 2), respectively, observed in the network. ${\text{CCI}}_{\text{incr}}$ and ${\text{CCI}}_{\text{decr}}$ were calculated for each group.

RESULTS

Strategies for Self‐Regulation and Self‐Reported Performance

During participants' self‐reports the strategies or methods that generated the best control of feedback signal were collected. Participants of groups G_F,I and G_F,I,R used motor imagery as expected, including running, moving hands, dancing, among others. In the case of Groups G_F and G_F,R, although participants were not instructed to use mental imagery at all, during the debriefing at the end of all the training runs, they reported the use of a variety of mental strategies for self‐regulation (meditation, relaxation, sequential thinking, focusing, etc.) but quite different from motor imagery. Please see Table 2.

Table 2.

Reported strategies during the rtfMRI‐NF training sessions

	Strategy reported
GF (Feedback)	Relaxing (nature environment related to rest: beach, forest, vacation)
	Remembering (trying to remember details of friends and trips)
	Meditation (concentration in one body point free of other thoughts)
	Positive mood (encouraging himself to increase the thermometer bars)
	Focusing (heat from a fire, movement of a flame, concentration on a white background)
GF,I (Feedback + instructed imagery)	Motor imagery (a very active rock concert)
	Motor imagery (aggressive movements to get released from the scanner)
	Motor imagery (running, fast movements in scanner)
	Motor imagery (fast and intense movements while playing basketball)
	Motor imagery (playing piano and rugby)
GF,R (Feedback + Monetary Reward)	Recalling (remembering topics and linking them to new ones)
	Sequences (repeat three words sequences, chosen at the moment, not necessarily related)
	“Speaking in his brain” (inner speaking) and recalling important autobiographical memories
	Concentrating on increasing the bars of the feedback thermometer. Thinking about videogames
	Relaxing and focusing on increasing the bars of the feedback thermometer
GF,I,R (Feedback + Monetary Reward + Instructed Imagery)	Motor imagery (funk dancing)
	Motor imagery (pumping activity or repetitive movement)
	Motor imagery (simple first person actions, i.e., move right hand to touch left elbow)
	Motor imagery (skate tricks)
	Motor imagery (swimming, running, boxing)

Similar levels of comfort were reported across groups (Kruskal Wallis non parametric test, day 1: H (3) = 1.09, P > 0.05, ns; day 2: H (3) = 3.33, P > 0.05, ns). Participants were asked to rate a Likert scale (5 top performance) their perceived success in controlling the feedback signal (i.e., thermometer) during days 1 and 2 of training. Taking all groups together, a significant increase in ratings was observed during the second day (reported day 1 Mdn = 3; day 2 Mdn = 4, paired Wilcoxon signed rank test, Z = −2.98, P < 0.01). No significant differences between groups were found in self‐report of performance (Kruskal Wallis non parametric test, day 1: H (3) = 6.55, P > 0.05; day 2: H (3) = 2.39, P > 0.05, ns).

SMA‐ROI Self‐Regulation

SMA BOLD activation levels

First, we compared self‐regulation of the BOLD signal of the SMA in the four groups of participants across the two days of training. Percent BOLD change (r _SMA) during a feedback run was used as an indicator of the amplitude of self‐regulation. All groups had a significant mean increase in BOLD activity for each day (one‐sample t‐test, compared to zero; G_{F d1}: M = 0.10, t(19) = 2.87, P = 0.01, Cohen's d = 0.64; G_{F d2}: M = 0.15, t(19) = 4.31, P < 0.001, Cohen's d = 0.97; G_{F,I d1}: M = 0.11, t(19) = 2.49, P < 0.05, Cohen's d = 0.55; G_{F,I d2}: M = 0.19, t(19) = 3.52, P < 0.01, Cohen's d = 0.78; G_{F,R d1}: M = 0.30, t(19) = 5.03, P < 0.001, Cohen's d = 1.12; G_{F,R d2}: M = 0.29, t(19) = 4.75, P < 0.001, Cohen's d = 1.06; G_{F,I,R d1}: M = 0.22, t(19) = 4.62, P < 0.001, Cohen's d = 1.03; G_{F,I,R d2}: M = 0.20, t(19)=3.37, P < 0.01, Cohen's d = 0.75). Figure 1 (and Supporting Information Table S2) shows the values for up‐regulation during the training runs of days 1 and 2. Group differences were tested using one‐way ANOVA among the total number of training runs in each one of the four groups (40 total runs by group). A significant group factor appears in this case (F _3‐156 = 4.643; P < 0.01, effect size η ² = 0.08). In Games‐Howell post hoc test for multiple comparisons G_F,R showed a significant mean difference with groups G_F and G_F,I (G_F vs. G_F,R: P < 0.01, Cohen's d = 0.779 and G_F,I vs. G_F,R: P < 0.05, Cohen's d= 0.595). Analysis by day also showed a significant group effect (F _7‐152 = 2.205; P < 0.05, effect size η ² = 0.09), although in the post hoc analysis no significant difference was found between the groups.

Figure 1.

Mean BOLD signal change (r _SMA) for the first and second day training runs in the SMA‐ROI (MNI x = 0, y = 0, z = 60) for each group. A significant difference was found between days 1 and 2 for Group G_F. From ANOVA analysis, group G_F,_R was significantly different from 1 and 2. Standard deviation bars and SMA‐ROI are shown (** = P < 0.01; * = P < 0.05). [Color figure can be viewed in the online issue, which is available at http://wileyonlinelibrary.com.]

Figure 2.

Box plot showing results of the transfer run for all groups on day 2. The y‐axis shows r _SMA values presented in the brain region centered at MNI: x = 0, y = 0, z = 60, and the x‐axis indicates the four groups. No significant differences between the groups were found. All groups presented significant increases in SMA BOLD during the up‐regulation blocks of the transfer runs. [Color figure can be viewed in the online issue, which is available at http://wileyonlinelibrary.com.]

To test if subjective perception of feedback control and magnitude of SMA self‐regulation were correlated, a comparison of self‐report ratings of day 1 and day 2 with the BOLD signal difference (r _SMA) were performed. A significant positive correlation was found between r _SMA and self‐report rating for each day (day 1: r(18) = 0.48, P < 0.05; day 2: r(18) = 0. 53, P < 0.05).

Self‐regulation learning

The learning effect in SMA self‐regulation throughout the training days was analyzed as the difference between mean values of r _SMA in days 1 and 2 (Δr _SMA). Δr _SMA was defined as our measure of learning self‐regulation of the SMA [deCharms et al 2005]. A significant difference between day 1 and day 2 was found only for G_F (r _SMA, five values one‐sample Wilcoxon signed‐rank test for median difference from 0, G_F: r _SMA Mdn = 0.064, Z = −2.023, P < 0.05, effect size r = −0.9; G_F,I: r _SMA Mdn = 0.025, Z = −0.674, P > 0.05, effect size r= − 0.3, ns; G_F,R: r _SMA Mdn = 0.016, Z = −0.135, P > 0.05, effect size r = −0.06, ns; G_F,I,R: r _SMA Mdn = −0.091, Z = −0.674, P > 0.05, effect size r= −0.3, ns). For group comparisons of Δr _SMA, non‐significant differences were found (Kruskal Wallis non‐parametric test, Mdn: G_F = 0.064, G_F,I = 0.025, G_F,R = 0.016, G_F,I,R = −0.091; H(3) = −0.58, P > 0.05, ns).

Transfer Runs

Participants capability to self‐regulate after training was evaluated in the transfer runs. During transfer runs the discriminative stimuli for up‐regulation and baseline were presented as in the training runs (white and black background respectively) but no feedback of the ROI signal was provided. One‐sample Wilcoxon signed rank test was performed to verify that participants could up‐regulate their activity during the transfer run. As shown in Figure 2, all groups presented a magnitude of self‐regulation (r _SMA) significantly different from zero during the transfer runs of day 2 (one‐sample Wilcoxon signed‐rank test for median difference from 0; Day1: G_F: Mdn = 0.055, Z = −1.214, P > 0.1, effect size r = −0.54, ns; G_F,I: Mdn = 0.13 Z = −1.753, P > 0.05, effect size r = −0.78, ns; G_F,R: Mdn = 0.241 Z = −0.944, P > 0.1, effect size r = −0.42, ns; G_F,I,R: Mdn = 0.247 Z = −2.023, P < 0.05, effect size r = −0.90; Day 2: G_F: Mdn = 0.269, G_F,I: Mdn = 0.234, G_F,R: Mdn = 0.10, G_F,I,R: Mdn = 0.15; Z = −2.023, P < 0.05, effect size r = −0.90 in all groups). The transfer r _SMA values of groups G_F and G_F,I showed a significant increase during the second day (G_F: Z = −2.023, P < 0.05; G_F,I: Z = −2.023, P < 0.05; G_F,R: Z = −0.135, P > 0.1, ns; G_F,I,R: Z = −1.214, P > 0.1, ns). No significant differences in transfer r _SMA were found among groups (Kruskal Wallis non parametric test, day 1: H(3) = 4.166, P > 0.1; day2: H(3) = 2.109, P > 0.1, ns). During the day 2, the magnitude of self‐regulation (r _SMA) in the transfer run was not significantly different from training in the 4 groups (Wilcoxon signed‐rank test, G_F: Z = −1.753, P > 0.05 (P = 0.08), ns, G_F,I: Z = −0.944, P > 0.05,ns, G_F,R: Z = −1.753, P > 0.05 (P = 0.08), ns, G_F,I,R: Z = −0.135, P > 0.05,ns).

Variability analysis

High level of variability in the data was observed in all the groups. To analyze if variability can be related to the experimental factors, namely, feedback, motor imagery or reward, we compared the intersubject variability using the standard deviation (SD) of the signals among the groups (Fig. 3). Kruskal–Wallis test reported a significant group effect among the intersubject SD of r _SMA in the training runs (Group SD G_F: Mdn = 0.157; G_F,I: Mdn = 0.224; G_F,R: Mdn = 0.263; G_F,I,R: Mdn = 0.270; H(3) = 21.463, p < 0.01). Subsequent post hoc test found significant differences, after applying Bonferroni correction, between G_F and G_F,R (corrected α = 0.5/6 = 0.0083; G_F vs. G_F,I: Z = −2.310, P = 0.021, ns; G_F vs. G_F,R: Z = −3.151, P = 0.002; G_F vs. G_F,I,R: Z = −2.415, P = 0.016, ns; G_F,I vs. G_F,R: Z = −1.365, P > 0.1, ns; G_F,I vs. G_F,I,R: Z = −1.155, P > 0.1, ns; G_F,R vs. G_F,I,R: Z = −0.525, P > 0.1, ns). Linear regression of group intersubject variability across runs showed no significant linear trend (G_F: −0,0001x + 0,1634, r _s = −0.071, P > 0.1, ns; G_F,I: y = 0,0025x + 0,2052, r _s = 0.024, P > 0.1, n.s.; G_F,R: −0,0088x + 0,321, r _s = −0.357, P > 0.1, ns; G_F,I,R: y = 0,0111x + 0,1974, r _s = 0.262, P > 0.1, ns). Analyses of intrasubject variability versus functional SNR and intrasubject variability versus reward, both by group and feedback run, showed no significant effects (please refer to Supporting Information figures S2, S3, S4, S5 and S6).

Figure 3.

Box plot showing the intersubject variability by groups. The presented distribution considers standard deviation of the mean (SD group r _SMA) for each one of the eight NF training runs by group. [Color figure can be viewed in the online issue, which is available at http://wileyonlinelibrary.com.]

Whole Brain Univariate Analysis

To examine activations in other brain regions during the up‐regulation blocks, group level, univariate, whole‐brain analysis was performed using Statistical Parametric Mapping (SPM) (Fig. 4). Only the data from the second day's training were used to focus our analysis on brain activations resulting on late stages of training. The calculations were done considering the contrasts up‐regulation > baseline blocks ([1 −1]). The results showed that SMA activation was present in all groups. Other brain regions that were consistently activated in all study groups were bilateral precentral gyrus, insula and supramarginal gyrus. From the 2‐way ANOVA (considering the factors effects, groups and runs) no major differences in activations were found except some clusters of scattered activations (e.g., group G_F,R has only a significantly increased cluster of k _E = 19 at right precentral gyrus after applying FWE a P < 0.05) (please refer to Supporting Information Table S3).

Functional Connectivity Analysis

Comparison of mean FC changes among groups

The 12 selected regions of this analysis, based on the AAL atlas [Tzourio‐Mazoyer et al., 2002], were: Precentral (precentral gyrus) L, Precentral R, Frontal Sup (superior frontal gyrus) L, Frontal Sup R, Frontal Mid (middle frontal gyrus) L, Frontal Mid R, Supp Motor Area (SMA) L, Supp Motor Area R, Cingulum Ant (anterior cingulate cortex) L, Cingulum Ant R, Cingulum Mid R, and Paracentral Lobule L. FC patterns for the 12 selected regions by group are presented in Figure 5. Despite the slight differences observed in FC patterns, a consistent pattern of correlations was found across all groups. Regions that appear with high z _FC,T (mean z _FC during the eight training runs, z _FC,T) across groups are: middle frontal gyrus and superior frontal gyrus in right and left hemispheres, left and right anterior cingulate gyrus, left and right SMA, left and right precentral gyrus, left SMA and precentral gyrus (Supporting Information Table S4). In general, two zones of correlated regions can be observed, one frontal and a posterior‐motor functional network. Only for Group G_F a path between these two zones (left superior frontal gyrus and SMA) appears with higher correlation.

Comparison of slopes of FC changes among groups

The selected 12 regions with the highest slope values were Supp Motor Area R, Supp Motor Area L, Precentral R, Frontal Mid L, Frontal Inf Oper R (inferior frontal gyrus par opercularis, Broca area BA 44), Frontal Sup Medial L (medial superior frontal gyrus, MFG), Parietal Inf L (inferior parietal excluding supramarginal and angular gyrus), Angular L (angular gyrus), Precuneus L, Precuneus R, Putamen R, and Pallidum R. Figure 6 presents the increases and decreases (red and blue respectively) in z _FC between days 1 and 2. Inspection of the Figure 6 indicates that Groups G_F and G_F,R have higher number of regions that increased their FC between days 1 and 2. Conversely, groups G_F,I and G_F,I,R have less regions with increases in FC values and more regions that display decreases in FC values. Additionally, bilateral precuneus in G_F,R increased its connections across training [Δz _FC (L‐precuneus, L‐MFG)= 0.23, Δz_FC (R‐precuneus, L‐MFG)= 0.34; z _FC(L‐precuneus, L‐MFG) = 0.38 during last training run].

Furthermore, CCI (mean FC increases (CCI_incr) or decreases (CCI_decr) in a network from day 1 to day 2) was used as complementary information to FC slope plots to express the changes in FC from day 1 to day 2. Hence, for each group CCI_incr and CCI_decr were calculated from the connectivity matrix of the 12 selected regions (Supporting Information Table S5) and the “whole‐brain connectivity matrix” (Supporting Information Table S6). As can be inferred from the connectivity plots, Group G_F and G_F,R have higher increases in z _FC (higher CCI_incr values) and lower decreases in z _FC (lower CCI_decr values) compared to other groups, that is, these groups showed more increments and less decrements in correlation values on the second day of training compared to the first day. However, considering “whole‐brain connectivity matrix,” G_F,R alone had the greatest increase in correlation coefficients (higher CCI_incr) during the second day of training. One‐way ANOVA was used to test whether this comparative increase in G_F,R was significant or not. The Δz _FC values for all the regions in the “whole brain connectivity matrix” were compared among the groups. A significant group effect was found (F _3‐5100 = 23.06; P < 0.001) and a Games‐Howell post hoc test showed that the FC increments in G_F,R were significantly higher than in the other groups (G_F < G_F,R: P < 0.001; G_F,I < G_F,R: P < 0.001; G_F,I,R < G_F,R: P < 0.001).

DISCUSSION

The first aim of the current study was to compare the effects of three different factors that are expected to influence the capability of learning volitional control of brain activity, that is, contingent feedback, motor imagery, and monetary reward. For this purpose, we trained four experimental groups using a combination of these three factors in a rtfMRI NF experiment.

We used two measures to study self‐regulation proficiency in the ROI: (1) r _SMA, which expresses the magnitude of self‐regulation of SMA during the NF training, and (2) Δr _SMA that indicates the improvement in the magnitude of self‐regulation, namely, the learning effect, through the training process.

In most of the NF studies so far participants were provided instructions to use mental imagery to control brain activity, in addition to contingent feedback [Caria et al., 2007; Hwang et al., 2009; Lawrence et al., 2013; Rota et al., 2009; Scharnowski et al., 2012; Sitaram et al., 2011; Subramanian et al., 2011; Yoo et al., 2008; Young et al., 2014; Zilverstand et al., 2015]. Although monetary reward has not been used extensively, successful self‐regulation in rtfMRI without instructing mental imagery have also been recently reported [Buyukturkoglu et al. 2015; Megumi et al., 2015; Shibata et al. 2011]. In our study, all the groups, irrespective of whether they were given instructions or not, were able to up‐regulate the BOLD signal in the SMA throughout the experiment. Interestingly, when the magnitude of up‐regulation in SMA was compared among the groups, the group in which monetary reward was given showed the highest amplitude of self‐regulation during the training period, in comparison with no‐reward groups (G_F and G_F,I). Groups in which motor imagery was included (G_F,I and G_F,I,R) did not reach a significant increase in r _SMA amplitude in comparison to G_F. Therefore, we cannot state whether the inclusion of motor imagery is an enhancing factor in self‐regulation training since we did not find significant increases in the r _SMA of these groups during training. Conversely, the significant higher amplitudes of G_F,R (in comparison to G_F and G_F,I) could be explained by a beneficial effect of monetary reward on SMA self‐regulation.

While evaluating the learning effect across training days (Δr _SMA), it is apparent that the only group in which learning was observed was the group that was given only contingent feedback (G_F). Considering that group G_F,R had the highest level of SMA up‐regulation already on day 1, it is possible that the lack of learning effect in this group could be due to a ceiling effect (achievement of a very high‐level up‐regulation already on day 1). An alternate explanation is that the learning curve for this group is more gradual and cannot be recognized clearly in 2 training days.

The results indicating that the inclusion of explicit instructions to perform motor imagery do not improve up‐regulation during training might be counterintuitive considering the widespread use of such instructions in NF experiments [Caria et al., 2007; Lawrence et al., 2013; Rota et al., 2009; Scharnowski et al., 2012; Sitaram et al., 2011; Subramanian et al., 2011; Sulzer et al., 2013b; Yoo et al., 2008; Young et al., 2014; Zilverstand et al., 2015]. For this reason, we emphasize the need to examine the mechanisms involved in learning brain self‐regulation, which are still far from being totally elucidated [Scharnowski et al., 2012]. Our results may also be construed as supporting the proposal that operant conditioning can play an important role for successful learning of brain hemodynamics control [Birbaumer et al., 2013]. In operant conditioning, desirable responses are positively reinforced and negative ones discouraged leading finally to an automatized skill achieved through a “trial‐and‐error” process [Strehl, 2014]. In the present experiment, the desirable response, that is, BOLD signal increase in SMA, is reinforced by the rising bars of the thermometer during the training runs, assuming that participants assign reward values to the thermometer bars.

However, the self‐reports of the participants at the end of NF training indicate that even when participants were not instructed any motor imagery (in groups G_F and G_F,R) they did indeed use some form of mental imagery although not always related to motor imagery. This opens an important point that NF training in humans even in the absence of explicit instructions can induce participants to incorporate some form of mental strategy to learn volitional control of their brain signals. In the group G_F,R, an additional factor, namely monetary reward was provided to the already existing feedback information, generating a stronger reinforcement with the consequential rise in brain activations.

One of the major features in this kind of learning is the secondary place of the conscious involvement of the participant in performing the requested task, that is, moving the thermometer bars [Birbaumer et al. 2013]. In fact, similar to our experiment (for groups G_F and G_F,R), Shibata et al. [2011] did not inform the participants about the exact meaning or the contingency of the delivered feedback signal, yet demonstrated learned volitional control. Some studies from EEG‐NF also support this view, leading to a speculation that use of mental imagery and conscious brain resources thereof can impair an efficient mechanism of brain control [Kober et al., 2013; Witte et al., 2013].

The “Dual process theory” proposed by Lacroix [1986] states that both “feedforward” and “feedback” processes are involved in the control of the desired signal. Feedforward processes are active when verbal instructions enable participants to retrieve existing behavioral programs to effectively perform the task, for example, a motor imagery program of moving the right hand. The aim of the NF training then is to find the program (or a combination of programs) that generates the best control of the feedback signal. Conversely, “feedback processes” are active when participants do not receive verbal instructions about the bodily signal they have to control, and consequently, need to construct a new behavioral program through determination of the properties of the system (interoception) by trial and error, based on contingent feedback. Therefore, the dual‐process theory suggests that giving explicit verbal information about the potentially relevant behavioral programs that control the selected body signal can help participants to reduce the time needed for constructing a new program. In other words, verbal instruction to use mental imagery can be seen as a “shortcut” for helping to achieve self‐regulation. Through NF training, further refinements of the selected behavioral program are achieved using the feedback signal to reach an optimal response. However, learning to self‐regulate brain signals can be impaired when: (1) the behavioral program to perform self‐regulation is not retrieved because it simply does not exist, that is, it is not in the subject's behavioral repertoire; (2) participants may possess behavioral programs that work only partially, therefore maintaining the use of an ill‐fitted strategy through the course of NF training. In these scenarios, the theory proposes that subjects end up relying on feedback processes to control the signal, presumably through operant conditioning processes.

Previous biofeedback studies evaluated the effects of reward in addition to contingent feedback [Bennett et al., 1978; Blanchard et al., 1974; Bouchard and Granger, 1980]. Blanchard et al. [1974] studied the additive effects of monetary reward and feedback to train voluntary increase of heart rate. A non‐consistent advantage of delivering monetary reward in comparison to using a feedback‐only scheme was found. In contrast, Bennett et al. [1978] found an increase in heart rate score in the groups with reward in comparison to the non‐reward group. Additionally, Bennett et al. [1978] also studied the effect of the cognitive strategies on learning to increase heart rate. Various mental strategies (e.g., frightening or sexual thoughts) were reported, indicating that a wide variety of imagery can be used to control heart rate.

In the present study, successful up‐regulation of SMA was achieved in participants of the groups G_F and G_F,R despite not using motor imagery (as usually instructed in the previous rtfMRI‐NF studies). However, the participants of the above two groups used other mental strategies even when they were not instructed (See Table 2). This outcome could be explained by the role of SMA on non‐movement related brain activity [Chung et al., 2005; Nachev et al., 2008] that might have been used by the participants included in the non‐imagery groups. Furthermore, no significant learning effect was found in the groups provided with explicit instructions of motor imagery (G_F,I and G_F,I,R). This outcome could be due to the sub‐optimal levels of BOLD increase due to the use of motor imagery while more flexible exploration of other forms of mental imagery could have produced greater signal increases in SMA. Alternately, the above outcome may also be due to the inability of the current real‐time fMRI approach to precisely localize the SMA sub‐clusters pertaining to movement of a specific body part. Future progress in fMRI signal acquisition (e.g., Multiplexed EPI sequences for sub‐second whole brain fMRI, Feinberg et al 2010] and real‐time pattern classification (e.g., Cox and Savoy, 2003; Rana et al., 2013; Sitaram et al., 2011; Zheng et al., 2013] may allow precise feedback of the brain activity pertaining to a specific brain function that is being addressed.

During the second day, transfer runs (when no contingent feedback was presented) of all experimental groups showed up‐regulation of SMA. However, it is interesting to note that the performance of G_F,R in transfer runs was similar to the other groups, and this group did not show the largest increase in self‐regulation magnitude in the transfer runs as was earlier observed during the training runs. If we additionally consider that monetary reward was not delivered during transfer runs, we could be tempted to assume that the loss of G_F,R relative increase in r _SMA during transfer may be interpreted as an extinction effect. However, this assumption is only speculative since we did not find a statistically significant decrease in BOLD ratio between training and transfer sessions in G_F,R. Further work should carefully assess the effect of different reward schedules on learning and its extinction with time.

The comparison of transfer runs showed an increase in BOLD self‐regulation from day 1 to day 2 for groups G_F and G_F,I. G_F also presented a learning effect during training, which is in line with the results found by Auer et al. [2015], related to the association between self‐regulation performance during training and BOLD level in the transfer run. However, it should be noticed that our study did not include a transfer day before the first training run. Therefore, we are not able to characterize properly the learned self‐regulation effect, although an approximation to the dynamics or changes within this process can be made using the progress in transfer runs from day 1 to day 2.

As has been discussed above, the significant improvement in the r _SMA amplitude from day 1 to day 2 of NF training was found only in G_F, in contrast with the non‐significant increases we found in motor imagery groups. However, when we consider the significant increase of the r _SMA amplitude in the transfer runs from day 1 to day 2 in both the G_F and G_F,I groups, one is left to speculate whether mental strategies may have some beneficial effect when feedback is removed in the transfer conditions. Future work may be able to resolve this issue if longer‐term effects of mental imagery‐based NF training in contrast to no imagery are compared by performing the transfer condition in the same participants several days after the training. If in such a scenario, the G_F,I group performs significantly better than the G_F group, the data may suggest the beneficial effects of mental imagery during NF training.

The large variability we observe in r _SMA values during the training runs in all the groups may be due the small group sizes in our study, but can also be explained by the large intrasubject and intersubject variability that has been generally observed in fMRI studies [Gaxiola‐Valdez and Goodyear, 2012; Kannurpatti et al., 2010; Lund et al., 2005]. Alternatively, it is possible that the large variability in the data may represent the exploratory trial and error process in which participants use different types of mental imagery to achieve self‐regulation [Galea et al., 2013; Pekny et al., 2015; Wolpert et al., 2011].

We evaluated whether the variability in the SMA signal during NF training could have been generated due to the differential effects of feedback, motor imagery and reward. We found a significant increase in intersubject variability in the group G_F,R in comparison to group G_F. The inclusion of reward as an additional factor tended to increase both the amplitude and the variability of the magnitude of self‐regulation (r _SMA). A possibility is that the inclusion of monetary reward amplifies the desired response (r _SMA)_, as observed in our results, and consequently enhances the already high intersubject variability. Previous studies on operant conditioning have shown that when the value or strength of the reinforcement is increased (e.g., giving more food pellets to a rat), the desired response tends to increase the magnitude or the speed in reaching the asymptote of the learning curve [Bower and Miller, 1960; Bower and Trapold, 1959]. However, it was also reported that learning is dependent on the maintenance of the reward and is prone to extinction when the reinforcer is taken out. Finally, the presentation of reward in G_F,R could have introduced an additional source of variability to the problem due to the individual differences in participants' response toward reward [Cohen, 2007; Peters and Büchel, 2011].

In our experiment, we assume that when participants receive only feedback (G_F), they have an intrinsic motivation to achieve volitional control. Conversely, with the inclusion of the monetary incentives, the extrinsic reward starts to have a prominent role as a motivational factor for learning self‐regulation. Previous studies have found that although extrinsic reward may generate better results initially, in the long term, the intrinsic motivation is undermined and the initial, positive results can diminish when the explicit reward is retired [Birch et al., 1984; Deci, 1971; Deci et al., 1999]. Therefore, reinforcing intrinsic motivations can be a more reliable approach for long‐term training. The inclusion of reward can be helpful, particularly during the initial stages of NF training, but the maintenance of this factor for longer duration should be carefully evaluated to maintain the beneficial effects in brain self‐regulation.

The second aim of our study was to explore neural substrates of brain hemodynamic control. For this purpose, we conducted univariate analysis and FC analysis of the whole brain. The analysis of magnitude of self‐regulation showed differences between groups. However, this difference did not persist when we looked at the spatial brain activation elicited during up‐regulation training. The results of univariate, whole brain analysis showed that brain activations were strikingly similar across the groups. Similarly, from the comparison of mean FC changes among groups, we found a core network of connections that is observed in the all four groups. This similarity in brain activations and FC patterns among the groups can be due the fact that contingent feedback was given to all the groups.

Our results indicating the common activations in insula, left supramarginal gyrus and precentral gyrus have been reported in previous NF studies. Ninaus et al. [2013] asked participants to control the feedback signal (a thermometer) during a covert sham‐feedback experiment in fMRI. The reported regions of significant activation, when participants tried to get control of thermometer bars in contrast to only watching the moving bars, were the insula, supramarginal gyrus, precentral gyrus, anterior cingulate gyrus, middle frontal gyrus, thalamus, and SMA. Another study, a meta‐analysis of 12 rtfMRI studies by Emmert et al. [2016] also found similar activations, particularly, in the anterior insula and tempo‐parietal areas along prefrontal cortex (dorsolateral and ventrolateral).

The active regions during the up‐regulation blocks in our study have been previously linked to different brain processes. Insula has been related to driving attention to inner states [Haller et al., 2013; Ninaus et al., 2013] in NF tasks. Supramarginal gyrus has been reported to participate in inner speech and language production [Geva et al., 2011; Hartwigsen et al., 2015]. SMA as part of the motor network has well‐documented connections with precentral gyrus [Kasess et al., 2008; Solodkin et al., 2004].

The analysis of mean FC values shows that the FC patterns were similar among groups, and can be roughly divided in two spatial groups: frontal and motor areas. However, this separation could be partially favored by the method used to select the regions involved in the analysis: we selected “whole‐brain connectivity matrix” and “SMA‐seeded” regions. From the FC analysis, we found a strong frontal network with connections between superior frontal gyrus and middle frontal gyrus across groups. Prefrontal involvement has been previously reported in NF studies [Emmert et al., 2016]. Superior and middle frontal regions have also been associated with attentional processes [Corbetta and Schulman, 2002] and motor imagery [Halder et al., 2011].

In contrast, from the analysis of changes (or slope) in FC through the training, we found that different factors, namely, feedback, motor imagery and reward, had different effects on the functional connections. Groups which were instructed to use motor imagery have relatively less enhancement of the connection strengths (correlation coefficients) from day 1 to day 2, due to training, in comparison to the group that included monetary reward (G_F,R) (Supporting Information Table S5 & S6, Fig. 6). FC increases were also found between precuneus and parietal regions (inferior parietal and angular gyrus) particularly in groups with contingent feedback and monetary reward (groups G_F and G_F,R). Furthermore, the highest increase in FC was observed between precuneus and middle frontal gyrus in group G_F,R. Precuneus has been linked with autobiographical memory [Eustache et al., 2004; Rauchs et al., 2013], imagery and self‐processing operations [Cavanna and Trimble, 2006].

Interestingly, the group with the highest self‐regulation amplitude, G_F,R, is also the one with the highest connectivity increases. This result suggests that successful BOLD self‐regulation is a process that could be mediated by changes in the whole brain, even when only a single ROI is targeted for NF training as in this study. Previous studies have also found connectivity changes related to self‐regulation of single ROIs. Ruiz et al. [2013a] performed an effective connectivity analysis in the NF training of insula with schizophrenia patients. They reported an increase in connectivity density of a network of emotion‐related brain regions during the final sessions of NF training when participants had learned to volitionally control the target ROI. Scharnowski et al. [2014] described specific changes in effective connectivity of a self‐regulation “learner's” group during NF training of a ROI in the visual cortex. Potentiation of top‐down connections between the visual cortex and contralateral superior parietal lobe was found at the end of training process. Haller et al. [2013] studied functional connectivity changes related to self‐regulation training in the auditory cortex. They found network variations with the target ROI appearing as hub of the connectivity changes. The study by Haller et al. [2013] proposed that single ROI self‐regulation could trigger changes in other regions of the brain network driven by the target ROI. In our study, we observe that changes in the network could also be related to the type of training (e.g., presence of reward) and/or to the success of the training approach (e.g., self‐regulation amplitude). These findings could be very important for the correction of abnormal connectivity patterns in the recovery and treatment of various neurological disorders.

Our study has a few technical and scientific limitations. High intersubject and intrasubject variability (in terms of standard deviation of the BOLD signal in the SMA) due to small group size and limited training period (2 days of four training runs per day) are two major limitations. It is still an open question as to how many days of training are required for successful learning, especially in the context of intersubject variability [Sulzer et al., 2013a). Due to restrictions of scanning time and cost, it is difficult to incorporate extensive training in order to attain clear asymptotic levels of BOLD self‐regulation. In this sense, our study can be seen as a preliminary view of how the selected factors can influence the NF performance. A recent study by Auer et al. [2015] studied rtfMRI‐NF in extended periods of training, delivering more details about the progress of the learning process. Further clarification of the use mental imagery and reward (and other factors) in longer NF training schemes are needed to understand thoroughly the dynamics of brain self‐regulation. In the present study, a significant learning effect was not attained in the group of contingent NF and reward. This may be explained by the short period of training. Furthermore, the similarity observed in brain patterns found in the univariate and FC analyses could be also due to this short period of training. Longer NF training [as in Auer et al.,2015] may enhance the differences that training factors generate in the brain activations required for a successful self‐regulation. However, it should be also noted that previous NF studies have performed training with similar durations of training [Caria et al., 2007; Caria et al., 2010; Chiew et al., 2012; deCharms et al., 2005; Haller et al., 2010; Hamilton et al., 2011; Johnson et al., 2012; Lawrence et al., 2013; Rota et al., 2011; Veit et al., 2012; Young et al., 2014; Zotev et al., 2011] and reported successful brain self‐regulation in terms of BOLD signal change during the training.

Regarding the acquisition methods, using a higher spatial resolution could have benefited the univariate and FC analyses. However, it would have increased the acquisition time of brain volumes (longer TR), therefore decreasing the number of samples available (lower temporal resolution). Due to our short training period, we preferred to increase the sampling rate to enhance the statistical power and to determine more precisely the changes in SMA self‐regulation.

In terms of generalizability of this work, it should be noted that the results of this study could at best be attributed to only the SMA, and cannot be generalized to other brain areas without further experimental testing on those regions. It is in fact possible that the volitional control of other brain regions, for example, emotion‐related regions such as insula or amygdala, could be influenced by reward and explicit mental imagery in quite different ways in comparison to the motor cortex. Additionally, we should consider that our study only included male participants in order to decrease potential variability in our sample. However, exclusion of women limits the generalization of the obtained results. Future work should investigate the effects of gender differences in self‐regulation training and the subsequent changes in brain and behavior.

CONCLUSION

The present study provides first evidence for the differential effects of three factors, namely feedback, motor imagery and monetary reward, on learning brain self‐regulation of the SMA. The status of the explicit (motor) instructions to subjects as an enhancer of self‐regulation remains uncertain since we did not find consistent significant results to support this view. In contrast, the presentation of contingent feedback alone produced a significant learning effect. Furthermore, when monetary reward was provided to the participants in proportion to their performance, a tendency for higher magnitudes of self‐regulation was observed, although no learning effect was noticed during the course of the training. Results of the univariate and functional connectivity analyses show a remarkable similarity in brain activations and functional connectivity across all groups, indicating that similar neural processes may be involved in self‐regulation despite differences in the way participants were trained. However, differences in the mean functional connectivity values and their change over time (slope) in the groups also indicate differences in the effect of feedback, motor imagery, and reward on the dynamic changes in brain during the training period.

Supporting information

Supporting Information