Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2025 Jan 21.
Published in final edited form as: Psychophysiology. 2023 Jun 16;60(11):e14367. doi: 10.1111/psyp.14367

Selecting an Optimal Realtime fMRI Neurofeedback Method for Alcohol Craving Control Training

Samantha J Fede 1,2,*, Mallory A Kisner 1, Sarah F Dean 1, Mike Kerich 1, Vinai Roopchansingh 3, Nancy Diazgranados 1, Reza Momenan 1
PMCID: PMC11748215  NIHMSID: NIHMS1952280  PMID: 37326428

Abstract

Realtime fMRI neurofeedback (rt-fMRI-NF) is a technique in which information about an individual’s neural state is given back to them, typically to enable and reinforce neuromodulation. Its clinical potential has been demonstrated in several applications, but lack of evidence on optimal parameters limits clinical utility of the technique. This study aimed to identify optimal parameters for rt-fMRI-NF aided craving regulation training in alcohol use disorder (AUD). Adults with AUD (n=30) participated in a single-session study of four runs of rt-fMRI-NF where they downregulated “craving-related” brain activity. They received one of three types of neurofeedback: multi-region of interest (ROI), support vector machine with continuous feedback (cSVM) and support vector machine with intermittent feedback (iSVM). Performance was assessed on the success rate, change in neural downregulation and change in self-reported craving for alcohol. Participants had more successful trials in run 4 versus 1, as well as improved downregulation of the insula, anterior cingulate, and dorsolateral prefrontal cortex (dlPFC). Greater downregulation of the latter two regions predicted greater reduction in craving. iSVM performed significantly worse than the other two methods. Downregulation of the striatum and dlPFC, enabled by ROI but not cSVM neurofeedback, was correlated with a greater reduction in craving. rt-fMRI-NF training for downregulation of alcohol craving in individuals with AUD shows potential for clinical use, though this pilot study should be followed with a larger randomized-control trial before clinical meaningfulness can be established. Preliminary results suggest an advantage of multi-ROI over SVM and intermittent feedback approaches.

Keywords: fMRI, neurofeedback, alcohol use disorder, craving, neuromodulation, methods

INTRODUCTION

Real time fMRI neurofeedback (rt-fMRI-NF) is a technique in which information about an individual’s neural state is given back to them, typically for the purpose of changing through reinforcing some aspect of neural function (i.e., neuromodulation). rt-fMRI-NF guided neuromodulation is supported by many proof-of-concept studies and some clinical trials, which we have previously reviewed systematically 1. For example, rt-fMRI-NF training enabled participants to upregulate regional brain activity including: the anterior insula (AI) 2,3, amygdala4, ventral tegmental area/substantia nigra (VTA/SN) 5, and inferior frontal gyrus6, as well as to downregulate subgenual anterior cingulate (ACC) activity 7. Some early work supports sustained effects of rt-fMRI-NF. A clinical trial by Young and colleagues found two sessions of rt-fMRI-NF focusing on training patients to upregulate the amygdala significantly reduced depression symptoms one week later in major depression patients 8. Also, rt-fMRI-NF guided modulation of the rostral ACC was related to reductions in levels of chronic pain 911. In some cases, training and transfer effects of neurofeedback may be sustained up to 14 months post-initial training 12.

The rt-fMRI-NF technique has previously been investigated for feasibility in substance use samples. It has enabled cocaine users to downregulate the VTA/SN 13, and in smokers, nicotine craving was reduced via downregulation of ventral ACC 14,15. Social heavy alcohol drinkers have been able to downregulate their ventral striatum responses to alcohol cues 16; in another study, individuals with alcohol use disorder (iAUDs) modulated neural response to alcohol cues in individualized ROIs (ACC, dorsolateral prefrontal cortex (dlPFC), or AI), and rt-fMRI-NF corresponded to reduced craving17. Initial evidence from conference proceedings indicates that rt-fMRI-NF may also be able to reduce short-term impulsivity in social drinkers 18.

Despite the potential of rt-fMRI-NF as a clinical intervention, evidence-based optimization of the technique itself is needed before such an application can be realized. There is significant variation in the algorithms (e.g., signal change, classification) and sources of the neurofeedback signal (i.e., what brain region(s)), as well as how the feedback is displayed to the participant (visual vs. audio, intermittently vs. continuously). These and other considerations are described at length in our previous review 1, but it remains the case that little experimental comparison of these methods have been done, particularly as they relate to clinical populations. This is a necessary step to reach the standardization necessary for clinical application, and to ensure that the true utility of the technique is being evaluated in clinical trials.

The need for such investigation is further underlined by what evidence that does exist, which often goes against the most common approach of single ROI signal change-based feedback with continuous, visual feedback display. Kim et al19 illustrated feedback of specific ROI activity was less effective than other methods in helping individuals regulate cigarette craving, though another found support vector machine (SVM) classifier feedback qualitatively comparable to ROI 20. Likewise, several studies found that intermittently displayed feedback (e.g., a summary score at the end of short regulation block) was more effective than continuously displayed feedback 2123, though a simulation study suggests that continuous feedback outperforms intermittent when learning explicit skills rather than implicit regulation24. Harmelech et al25 found that auditory feedback was more effective than visual.

With that in mind, the overall purpose of this study was to develop and pilot a neurofeedback training procedure that will later be investigated as a supplemental intervention in alcohol use disorder (AUD). We first aimed to establish a protocol for neurofeedback that works to train control over craving-related brain activity over repeated runs. Based on previous studies of fMRI neurofeedback in substance use, we hypothesized (H1a) that repeated sessions of neurofeedback would lead to increased ability to downregulate activity in the ACC, AI, dlPFC, striatum, and VTA/SN areas during craving stimulus presentation. Further, we hypothesized (H1b) that this learned downregulation of activity would transfer to a reduction in self-reported craving. We also aimed to optimize neurofeedback parameters to best aid this acquisition of craving control. Based on our previous systematic review of the neurofeedback literature 1, we hypothesized (H2) that although continuous, region of interest (ROI) based feedback have been more commonly used, whole-brain based pattern classification based approaches may have more utility in helping iAUDs learn to control alcohol craving. After initial piloting of continuous feedback only, we also came to hypothesize that intermittent feedback display would be more effective than continuous feedback.

MATERIALS AND METHOD

The procedures described herein were conducted as part of the first, pilot stage of a larger randomized controlled trial (NCT03535129). Only procedures and results related to the Stage 1 arm aim to pilot and optimize the intervention are reported here. The Stage 2 interventional arm of the study is ongoing as of March 2023. Eligibility criteria, aims, and outcome measures for the overall study are publicly available through clinicaltrials.gov. See Supplemental Materials for CRED-nf checklist26.

Participants

A total of 30 adult iAUDs (ages 21–65 years old) were recruited from the Washington DC metro area to participate in this study and undergo functional magnetic resonance imaging (MRI). Participants met criteria for the iAUD group if they were diagnosed with current moderate to severe AUD based on DSM-5 criteria. See Table 1 for demographic information.

Table 1.

Alcohol Use Disorder Sample Description

Overall ROI cSVM iSVM Test statistic p

A) F

Age 40.83 39.82 39.27 44.38 0.52 0.602
Years of Education 14.17 13.45 14.91 14.12 1.09 0.351
AUDIT 24.93 27.18 21.5 26.5 1.90 0.170
ACQ Baseline 2.82 2.45 3.35 2.58 1.20 0.316
ACQ Difference Score 0.02 −0.20 0.05 0.26 1.51 0.240
Regulation Success Rate Change 0.06 0.14 0.06 −0.06 2.75 0.082

B) Χ 2

N 30 11 11 8 0.60 0.741
Sex (%male) 60 64 46 75 1.78 0.411
Treatment Status (%inpatient) 73 91 36 100 12.34 0.002
Smoking Status (%smokers) 55 46 50 75 1.80 0.407

Race (%) 2.70 0.609
Black/African American 24 27 30 13
White 66 55 70 75
Multiracial 10 18 0 13

Ethnicity (% Hispanic or Latino) 14 18 20 0 1.78 0.410

Notes: Description of sample. A) Mean values overall and broken down by subgroups. Test of the differences between groups (ANOVA) and p-value of that test reported. B) Descriptives for categorical variables overall and broken down by subgroup. Test of the differences in frequency between groups (X2) and p-value of that test reported. All participants self-identified as male or female. ROI: multi-region of interest neurofeedback group. c/iSVM: continuous/intermittent support vector machine neurofeedback groups. ACQ: Alcohol Craving Questionnaire

Exclusion criteria included: significant history of head trauma or cranial surgery, history of neurological disease, MRI contraindications (including pregnancy, claustrophobia, and presence of ferromagnetic objects), and history of non-substance related psychosis. Participants were not scanned if they tested positive for opiates, cocaine, or methamphetamine, if they were currently experiencing withdrawal from alcohol, or if they had a positive breath alcohol measure.

Procedures and Assessments

Participants first completed general data collection measures as part of a separate screening protocol with appropriate data sharing permissions; these were used to evaluate the inclusion and exclusion criteria above. This included sociodemographic data, the Structured Clinical Interview for DSM-527 (SCID) to provide clinical diagnoses on a variety of mental health conditions (including substance use disorders), a 5-panel urine drug screen, a breathalyzer, an assessment of withdrawal, a clinical history and physical exam conducted by staff medical personnel, and an Alcohol Timeline Followback to assess drinking patterns in the last 90 days28. All research activities were conducted as approved by the NIH intramural Institutional Review Board consistent with the Declaration of Helsinki.

All subjects provided written informed consent before completing several questionnaires unrelated to this pilot aim. Then, participants spent approximately 15 minutes in a bar-like environment during which they ate a standard snack bag provided by the NIH Clinical Center. The goal of this procedure was to prime their craving. Typically, participants were scanned around lunchtime (12–2). Immediately before entering the MRI, participants completed the Alcohol Craving Questionnaire29 (ACQ) to assess pre-scan craving. These are self-report measures. Participants repeated these measures immediately after completing their MRI scan session.

Subjects received one of three versions of rt-fMRI-NF during their MRI session, designed to enable participants to downregulate craving. These versions were progressively modified for the aim of optimization, so version assignment was not prospective.

Image Acquisition and Neurofeedback Methods

Participants underwent one scan session on a 3 Tesla Siemens Prisma MRI machine. The total MRI session typically took between 1 and 1.5 hours, including both functional and structural scans. Participants were asked to complete four runs of the neurofeedback training procedures (approximately 7 minutes each); these were sequential within the same MRI scanning session. An echoplanar-imaging (EPI) pulse sequence was used (TR: 2000 msec, flip angle: 90degrees, FOV: 24×24 cm, 38 mm slice thickness, 36 slices, multislice mode: interleaved.) A MPRAGE structural scan was also acquired for use in co-registration.

All rt-fMRI-NF training procedures were done using AFNI to process the data in real-time (version 17.2.16) using a Python-based framework (custom written and publicly available at https://pypi.org/project/afniRTI/) to interface with feedback presentations built and delivered using PsychoPy2 31. Specifically, as each volume of data was collected on the MRI, DICOM files were written by the Siemens console to a local network location on a separate desktop “processing” computer. A script running on the processing computer listened for and received the DICOM data, then passed it to AFNI, where real-time processing occurred. This processing resulted in summary values for each TR, which were sent over the local network to a third “stimulus presentation” computer, where a PsychoPy task received the values and used them to modify the visual feedback presented to the participant. These processing procedures for each TR typically took between 100 and 900 msec. Three rt-fMRI-NF methods were used during the piloting phase, described in more detail below (more technical detail available in Supplemental Materials 1): multi region of interest (ROI; n = 11), support vector machine with continuous feedback (cSVM; n = 11) and support vector machine with intermittent feedback (iSVM; n = 8).

ROI neurofeedback used the realtime plugin from the AFNI toolbox to process signal change in 5 pairs of regions (left and right; ACC, AI, dlPFC, striatum, and VTA/SN) as new DICOM volumes were collected and transmitted from the scanner. The cSVM and iSVM methods used the 3dSVM plugin based on input data from the whole brain. For the SVM methods, a short “functional localizer” cue reactivity task was run prior to neurofeedback to train the classification algorithm; this contained blocks of neutral, non-craving images and blocks of images designed to evoke craving (alcohol images corresponding to the participant’s drink of choice). Incoming data during the neurofeedback task was run through this algorithm and a single value (distance to hyperplane) was produced.

All runs in all types of rt-fMRI-NF presentations followed the same general format (see Figure 1). Participants first saw two 30 second blocks of 6 pictures, displayed for 5 seconds each. The first block was preceded by the instruction “Look” and consisted of neutral object pictures. Neutral images were selected from IAPS32 and OASIS33 image sets. The second block was preceded by the instruction “Crave” and consisted of pictures of alcohol. Alcohol cue pictures were from a set used in a previous, published cue reactivity task34, supplemented by similar images from image sets like IAPS32, OASIS33, and general online image searches. During this time, participants were asked to “crave” or “want” the alcohol as much as possible to provide a baseline for their craving-related neural activity. For all methods, the “Look” and “Crave” blocks were used to determine the “thermometer” feedback range.

Figure 1.

Figure 1.

Diagram of neurofeedback tasks paradigms. A) Diagram of multi ROI and continuous SVM neurofeedback tasks. 16 total trials each consisting of stimulus presentation with continuous feedback presented for 17 seconds followed by summary feedback presented for 5 seconds. Top- Example of successful trial. Bottom- Example of unsuccessful trial. B) Diagram of intermittent SVM neurofeedback task. 18 total trials each consisting of stimulus presentation for 12 seconds followed by summary and interim feedback for 5 seconds. C) Diagram of pre-neurofeedback functional localizer and baseline cue reactivity blocks. For the functional localizer, used for SVM neurofeedback, participants saw 5 of each non-craving and craving block (5 seconds per image, 4 images per block), interleaved. A 6 second “Rest” screen was displayed between blocks. Order and images were randomized across blocks. For the baseline cue-reactivity (all neurofeedback types), one non-craving block and one craving block were shown prior to the neurofeedback trials.

Then the instruction “Control” appeared, and the feedback portion itself began. Prior to beginning the task, participants were instructed that they would see a thermometer and an image of alcohol. They were told the thermometer represented their brain activity associated with craving and that they would see a mark on the thermometer labeled “Goal”. For each trial, they were told to try to reduce their craving to that goal temperature or below. They were told, “One strategy some people find useful is to think about the FUTURE negative consequences of drinking. However, you might find another strategy more helpful.” For trials where they reached the goal, they saw summary positive feedback and earned 65 cents. For trials where they did not reach the goal, they saw summary negative feedback and did not earn any money. The goal adjusted progressively, getting harder (lower) when participants were successful and getting easier (higher) when participants were not successful. This was to encourage increased control over time. Participants were told there would be a delay between their brain changing and the thermometer changing. For ROI and cSVM methods, the thermometer was displayed alongside the alcohol cue; for iSVM, the thermometer was displayed in between cues along with the summary feedback.

Offline Image Processing and Analysis

For the purposes of summary analysis after the completion of data collection, each run of neurofeedback training for each participant was also processed separately offline using AFNI (v21.0.20) using the following pipeline: First, we shifted slice time courses so that each voxel was aligned to the same temporal origin and identified outliers in the time series. Then volumes across the time series were aligned to the first EPI volume and to the skull-stripped structural image of the subject before being transformed to standard MNI space unsing a non-linear warping procedure. Data was then smoothed using a 4 mm full-width at half-maximum Gaussian kernel and scaled to a run mean of 100 (min 0, max 200). Outlying TRs and those with motion derivatives of 0.3 mm or greater were censored from all analyses that followed.

The condition of interest (Regulation, i.e., “Control” blocks) was modeled using a block design for the basis function, with demeaned motion parameters and their derivatives included to remove variance associated with movement. One individual was removed from group analyses due to excessive motion (having a sensor fraction greater than 30%), resulting in a total n = 29 (ROI: n = 11; cSVM: n = 10; iSVM: n = 8). Average beta values were extracted for each subject from the 10 regions-of-interest described above in the ROI neurofeedback method. Group analysis was then conducted in R (vers.3.6.3).

To address hypothesis H1a, we conducted a one-sided paired t-test for each region to identify whether there was a significant reduction in brain activity during the Regulate block between run 1 and run 4 of the neurofeedback training. To address hypothesis H1b, we calculated a difference score between run 1 and run 4, then examined whether that change in brain activity during the Regulate block was correlated (using a Pearson correlation test) with a change in self-reported craving before and after the scan (as measured by the ACQ). To address hypothesis H2, we used the difference score in an ANOVA for each region to evaluate whether changes in brain activity during the Regulate block differed by neurofeedback type; for significant tests, we followed up with a Tukey’s HSD posthoc test to determine which method(s) drove this finding. We also conducted a linear model of the effect of neurofeedback type on the association of brain change with craving change.

RESULTS

Main Effects of Neurofeedback on Craving Regulation

See Table 2 for test statistics and p-values for all regions. See Table 3 for mean values, broken down by neurofeedback group.

Table 2.

Effects of neurofeedback on brain activity and associated craving in individuals with Alcohol Use Disorder

Region Laterality H1A (Main Effect of Time) H1B (Time x Craving) H2 (Type x Time) H2 (Type x Time x Craving)
T p r p F p F R2adj p
ACC L 1.73 0.048 0.42 0.025 4.35 0.023^ 1.61 0.10 0.197
R 0.18 0.428 0.35 0.066 6.84 0.004* 1.35 0.06 0.278
AI L 1.26 0.109 0.23 0.232 5.41 0.011^ 0.69 −0.06 0.636
R 2.16 0.020 0.31 0.100 1.78 0.188 1.31 0.05 0.296
DLPFC L 1.79 0.042 0.18 0.350 0.89 0.422 1.14 0.02 0.366
R 0.84 0.205 0.39 0.038 0.90 0.420 1.63 0.10 0.192
VS L 1.51 0.071 0.21 0.279 3.97 0.031^ 1.16 0.03 0.357
R 1.23 0.114 0.32 0.089 2.74 0.084 1.91 0.14 0.132
VTA/SN L 0.92 0.182 0.18 0.344 0.14 0.869 1.04 0.01 0.419
R 0.98 0.169 0.16 0.401 0.44 0.650 0.73 −0.05 0.611

Notes: Tests statistics and p-values from the primary analysis. Uncorrected p-values reported.

*

0.05 FDR threshold

^

0.1 FDR theshold.

Abbreviations- ACC: anterior cingulate; AI: anterior insula; DLPFC: dorsolateral prefrontal cortex; VS: ventral striatum; VTA/SN: ventral tegmental area / substantia nigra; L: left, R: right.

Table 3.

Descriptives of neurofeedback associated changes, broken down by neurofeedback type

Region Laterality (A) Change in Brain Activity (B) Association with Change in Craving
Overall ROI cSVM iSVM ROI cSVM iSVM
ACC L −0.17 −0.45 −0.17 0.21 0.60t −0.29 0.16
R −0.02 −0.18 −0.16 0.40 0.47 −0.23 0.29
AI L −0.14 −0.47 −0.13 0.31 0.25 −0.15 0.04
R −0.24 −0.44 −0.26 0.07 0.55t −0.10 0.10
DLPFC L −0.33 −0.33 −0.61 0.02 0.42 0.19 −0.26
R −0.08 −0.07 −0.25 0.10 0.60* 0.39 −0.03
VS L −0.22 −0.62 −0.21 0.32 0.45 −0.38 −0.05
R −0.19 −0.46 −0.33 0.35 0.69* −0.24 −0.10
VTASN L −0.47 −0.12 −0.73 −0.62 0.29 0.604t −0.04
R −0.20 −0.14 −0.45 0.03 0.26 0.18 0.08

Notes: A) Average of run 4 - run 1 betas in the region of interest during down-regulation. B) Correlation between Run 4 - Run 1 difference and Post-scan - Pre-scan ACQ craving measure.

*

p < 0.05

t

p < 0.10.

Abbreviations- ACC: anterior cingulate; AI: anterior insula; DLPFC: dorsolateral prefrontal cortex; VS: ventral striatum; VTA/SN: ventral tegmental area / substantia nigra; L: left, R: right. Neurofeedback technique abbreviations- ROI: multi region-of-interest; cSVM: continuous support vector machine; iSVM: intermittent support vector machine

(H1a). There was a significant increase in the rate of successful regulation trials between neurofeedback run 1 (mean = 51%, sd = 15%) and run 4 (mean = 57%, sd = 15%; p = 0.043; see Figure 2). For all regions, there was a qualitative decrease in brain activity during the Regulate blocks. However, the only regions with significant reduction were the right AI (Δmean = −0.24, Δsd = 0.6), the left ACC (Δmean = −0.17, Δsd = 0.53), and the left dlPFC (Δmean = −0.33, Δsd = 0.99). Table 3A for changes in brain activity broken down by neurofeedback type. See Figure 3 for plots of regulation related activity for each ROI per run.

Figure 2.

Figure 2.

Plots of behavioral measures over time. Error bars correspond to standard error. Bars represent mean. A) Plots of the rate of successful trials by run, where Success Rate is percent of neurofeedback trials where participant lowered their brain activity below the goal line. B) Plots of craving ratings on the ACQ self-report measure (where 0 is the lowest and 5 is the highest) prior to neurofeedback training and after neurofeedback training. Top (A&B): overall, collapsed across neurofeedback types. Bottom (A&B): broken down by neurofeedback type. Abbreviations: NF- neurofeedback; iSVM- intermittent support vector machine neurofeedback; cSVM- continuous support vector machine neurofeedback; ROI- multi region of interest continuous neurofeedback. Significance levels as follow: t p < 0.10; *p < 0.05; ** p < 0.01.

Figure 3.

Figure 3.

Plots of brain activity during “Control” downregulation neurofeedback trials by run. Error bars correspond to standard error. Betas from each region of interest are plotted separately. Bars represent mean brain activity. A) Overall brain activity per run, collapsed across neurofeedback types. B) Brain activity broken down by neurofeedback type. For each group, a linear fitted line is also plotted to summarize the change in activity within neurofeedback type. Abbreviations: NF- neurofeedback; iSVM- intermittent support vector machine neurofeedback; cSVM- continuous support vector machine neurofeedback; ROI- multi region of interest continuous neurofeedback.

(H1b) There were no significant decreases in self-reported craving before and after neurofeedback training. Change in brain activity in the left ACC and the right dlPFC were significantly associated with degree of reduction in self-reported craving (ACC: r = 0.415; dlPFC: r = 0.387). See Figure 4 for plots of these associations in each ROI.

Figure 4.

Figure 4.

Plots of the association between change in craving and change in brain activity during “Control” downregulation neurofeedback trials. X-axis “Change in Betas” represents the Run 1 level of brain activity during “Control” trials minus the Run 4 level of brain activity during “Control” trials. Y-axis “Change in Craving” represents the Post-scan ACQ measure of craving minus the Pre-scan ACQ measure of craving. These subtractions were chosen for the plots such that they would correspond to the intuitive interpretation (i.e., that lowering activity more during regulation corresponded to more reduction in craving); thus, a negatively sloping line on these plots corresponds to the positive correlations (i.e., that more downregulation corresponded to more reduction in craving) reported in the manuscript and in the Tables (and vice versa). Dots represent individual data points; lines are fitted linear best fit lines. Results from each region of interest are plotted separately. A) Overall association between change in brain activity and change in craving, collapsed across neurofeedback types. B) Overall association between change in brain activity and change in craving, broken down by neurofeedback type. Abbreviations: NF- neurofeedback; iSVM- intermittent support vector machine neurofeedback; cSVM- continuous support vector machine neurofeedback; ROI- multi region of interest continuous neurofeedback.

Effects of Neurofeedback Type on Learned Craving Regulation

(H2) There were no significant moderation effects of neurofeedback type on the association between change in brain activity and change in self-reported craving. There was a significant effect of neurofeedback type on change in brain activity in left and right ACC, left AI, and left striatum. Tukey’s HSD post-hoc tests revealed that this was primarily driven by a significant difference between ROI and iSVM methods (pdifACCL = 0.018; pdifACCR = 0.007; pdifAIL = 0.008; pdifStrL = 0.024). There were also significant differences between cSVM and iSVM methods in the right ACC (pdif = 0.010). Post-hoc tests did not reveal any significant differences between ROI and cSVM methods in terms of changes in learned craving regulation.

When examining the associations between changes in brain activity and in craving within each neurofeedback group, there were significant effects in several additional regions that appeared only for the ROI group. Specifically, there was a reduction in brain activity in the left AI and the left striatum in the ROI group but not in the cSVM and iSVM groups. There was also a significant association between craving change and change in brain activity in the right dlPFC and right striatum in the ROI group but not in the cSVM and iSVM groups.

There were no significant neurofeedback group effects on self-reported craving change. However, there was a marginal effect of neurofeedback type on change in regulation success. Specifically, there was a significant increase in the rate of successful regulation (Δmean = 14%, Δsd = 16%, pdif = 0.008) after neurofeedback for individuals in the ROI group, but not for the other neurofeedback types.

CONCLUSIONS

This pilot study of rt-fMRI-NF aided training of craving downregulation demonstrates promise for the technique. Not only were individuals with AUD able to learn over four sequential neurofeedback runs to downregulate activity in the ACC, AI, and dlPFC when presented with continuous feedback, the degree to which regulation was learned predicted a corresponding acute decrease in self-reported craving for alcohol. This was consistent with our first set of hypotheses.

Although there was a clear effect of neurofeedback sessions on ability to regulate brain activity, the most effective neurofeedback approach was less clear. We did not confirm our hypotheses that the intermittent feedback would be preferable to a continuous feedback approach; in fact, we found in the ACC, AI, and striatum, iSVM performed worse than either other technique. Moreover, intermittent feedback tended to be associated with an increase (albeit, non-significant) in brain activity during downregulation, suggesting this approach was potentially counterproductive. There was not a significant difference between ROI and cSVM techniques as far as improving the ability to downregulate brain activity.

We did see initial evidence to support the potential of multi-ROI neurofeedback over cSVM in clinical trial applications. There were strong correlations between change in learned down regulation of the ACC, AI, dlPFC and striatum and change in self-reported alcohol craving for the ROI neurofeedback group. On the other hand, individuals in the cSVM group had negative (albeit weak) correlations between craving change and learned downregulation in many of these regions. Individuals in the ROI group also had the greatest improvement in ability to regulate brain activity over runs (14% increase versus 6% with cSVM). Based on this evidence, we have adopted multi-ROI feedback for the Stage 2 interventional arm of this trial. We should also note that the increased performance of ROI compared to cSVM feedback might be based on the relative parsimony of that model; multi-ROI had 10 neural inputs while the SVM algorithm was based on data from the entire brain. Although our initial hypothesis of the superiority of SVM for this application was based in part on the thought that improvement in craving control and AUD recovery might involve complementary, non-craving specific brain functions that would not be adequately captured in ROI-based neurofeedback, it is conceivable that the whole-brain SVM approach overcorrects and therefore underperforms on this particular, clinically important symptom.

Our findings regarding the changes in involvement of the ACC, AI, dlPFC, and striatum are consistent with the literature on craving and on self-regulation. For example, Neurosynth meta-analysis on the term “craving” (as of April 2022) found activation of the ACC and striatum across studies (e.g., 35,36). The AI is also classically proposed to relate to craving via introspective processing of the effects of drug use 37,38, supported by findings that lesions to the AI disrupted craving and addiction 39,40. Finally, the dlPFC has a complicated role in both regulating craving and coding value signals, with causal studies illustrating that either enhancement or disruption of dlPFC function leads to reduced substance craving 41,42.

There were inconsistent effects in the VTA/SN. There was a lack of significant change in brain activity over runs. The VTA/SN has a well-demonstrated role in reward and aversion signaling 43,44, although other models suggest a specific role in positive reinforcement only 45. The operant reinforcement approach of neurofeedback training in general and the positive reinforcement focus of our technique (i.e., rewarding success with money, no loss of money on failure) could be expected to activate the VTA/SN, making downregulation difficult. Moreover, limited group differences in the VTA/SN downregulation and craving could be attributed to the group differences in number of successful trials and feedback frequency (i.e., differences in amount of positive reinforcement).

Limitations

There was a significant difference between groups in the percentage of individuals that were receiving treatment for AUD. Specifically, a lower percentage of the cSVM group were currently receiving treatment for AUD. Although there were not differences in baseline craving or in overall craving change by group, there were significant differences in self-reported induced craving by the images themselves, where the expected cue-induced craving was observed in the non-treatment seeking AUD subgroup, but not in those currently receiving treatment for AUD. These results are reported and discussed at more length in Supplemental Materials 2 and Supplemental Table 1. Regardless, given that there were a larger proportion of the non-treatment seeking individuals with AUD in the cSVM group, we re-ran our analyses using just the inpatient subset to rule out the possibility of this driving our results. The results were substantively the same (see Supplemental Table 2).

The sample size of this pilot study was small, as it was primarily designed to aid in the internal selection of optimal parameters for the follow-up randomized controlled trial (RCT). This is reflected in the few significant results despite large effect sizes, particularly in the more complex models. Similarly, we report results from our planned analyses, and within-group, without correction for multiple correction, which runs the risk of inflated false discovery rates. We believe this is justified in this specific context: First, the choice between these three techniques is forced for future clinical trials, in that neurofeedback must be done with one of these methods if the focus is to be on the effect on outcomes. Second, previous studies have demonstrated that all three of these techniques are efficacious. Finally, there is no difference in burden to the participants between the techniques. Because of these three factors, the impact of a false positive (i.e., that despite our findings, no difference between the techniques exist) on clinical trials is low; in the case of a false positive, it would simply mean that the non-used techniques were not in fact sub-optimal, and there would be no impact to patients receiving the neurofeedback intervention that was instead equivalent to other procedures.

Given that this pilot study was not placebo controlled, and was not randomized, there are potential concerns about whether the observed results are due to placebo/ practice effects or order effects only. Although a properly randomized controlled trial is a necessity prior to clinical application (and is currently ongoing), we do not believe placebo effects drive our findings here. First, we found associations between degree of brain engagement change and self-reported craving change. Although this does not entirely rule out the possibility of a placebo effect, it suggests that change above a simple binary belief in effectiveness was at work. Second, qualitative participant feedback to research staff reflected frustration that their sense of craving did not correspond to the feedback they were given. This should not be over-emphasized, as it was not systematically collected, but may be useful insight for interpretation of these results. Third, even if the H1 effects of the study were due to placebo effects, analyses related to H2 concerned between neurofeedback technique effects. There is no reason to suspect that placebo effects would be significantly different between the three techniques, and thus, we do not expect this drives our findings. We cannot rule out that order effects impacted the results, but there were no systematic differences in study procedures, staffing, or patient characteristics over time that could be expected to impact results in a systematic fashion.

Although we did see some evidence of transfer effect in our measurement of self-reported craving before and after the scanning session, this is not enough to conclude that this neurofeedback intervention is meaningful. We cannot know based on the current study neurofeedback impacts relevant clinical outcomes (such as drinking and community functioning) at more extended timepoints, nor how differences in the techniques might influence those outcomes. We also did not include a transfer neuroimaging cue-reactivity measure after the fourth neurofeedback training run. These types of markers are extremely important to investigate and are missing from many neurofeedback studies1. It is difficult to predict how the observed effects would change had we followed up with these individuals. Although effects of interventions for substance use are often smaller over time (e.g., Magill & Ray, 2009), there is evidence that neurofeedback effects are persistent 12 and can even grow larger after some consolidation period 25,47,48. Given this, we would also expect that in a transfer cue-reactivity fMRI measurement following the fourth neurofeedback run would be similar to, but more pronounced, than the cue-reactivity measure after the third run.

We are currently conducting a RCT (NCT03535129) using the multi-ROI neurofeedback method piloted here that will address many of these limitations and help us to address whether multi-ROI neurofeedback is clinically meaningful. It is a randomized, placebo-controlled study of two sessions of rt-fMRI-NF in addition to treatment as usual, with follow-ups out to 6 months post treatment. The RCT is designed to assess the efficacy of rt-fMRI-NF training for AUD. The optimal dosing and approach is a relevant question but one that would have required a significantly larger sample size and would be outside the scope of demonstration of sustained efficacy and transfer effects. For this reason, we have conducted this pilot study to inform our selection of neurofeedback approach.

Finally, although this study was designed to compare certain aspects of neurofeedback training approaches, it did not evaluate all possible techniques. The overall study design did not include an intermittent-ROI condition. Given that we hypothesized that SVM, rather than ROI, neurofeedback would perform better, we did not anticipate the need to explore intermittent feedback within the ROI condition. That being said, we do not have any reason to anticipate an interaction effect where the impact of feedback display type would vary by signal processing approach. As we discussed in our prior systematic review 1, there is little to no quantitative comparison of other elements that can vary in neurofeedback training. For example, the differences between implicit and explicit training, auditory versus visual feedback, different software, and traditional activation-based versus connectivity-based sources of feedback signal have not been explored in the literature and are not investigated here. Moreover, we cannot assume that the optimum for fMRI neurofeedback is consistent across treatment and brain area targets. Different regions, such as the amygdala, and different clinical targets, such as depression symptoms, may benefit from a different approach.

Conclusion

This pilot study provides the first quantitative comparison of rt-fMRI-NF approaches for training individuals with AUD to reduce alcohol craving. Results suggest that neurofeedback not only helps individuals to learn to downregulate brain activity in the ACC, AI, dlPFC, and striatum during alcohol cue exposure, it also supports the transfer of this learned regulation to a reduction of craving after the training, though the translation of this to clinical relevance is yet unestablished. Finally, evidence supports the use of a multi-ROI source neurofeedback source used to generate feedback, displayed continuously, for future studies of rt-fMRI-NF in AUD.

Supplementary Material

SupplementalMaterials

FUNDING AND ACKNOWLEDGEMENTS

This work was supported by the National Institutes of Health [Office of Behavioral and Social Science Research Bench-to-Bedside Award; National Institute of Alcohol Abuse and Alcoholism ZIAAA000126 and K99AA027830].

We thank our participants for their cooperation in this study and the NIH Clinical Center Alcohol Clinic staff for their hard work supporting our research.

Footnotes

DECLARATIONS OF INTEREST

None

DATA SHARING STATEMENT

The data that support the findings of this study are available on request from the corresponding author upon reasonable request. The data are not publicly available due to privacy or ethical restrictions.

REFERENCES

  • 1.Fede SJ, Dean SF, Manuweera T, Momenan R. A Guide to Literature Informed Decisions in the Design of Real Time fMRI Neurofeedback Studies: A Systematic Review. Front Hum Neurosci 2020;14(60) (Systematic Review) (In English). DOI: 10.3389/fnhum.2020.00060. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Ruiz S, Lee S, Soekadar SR, et al. Acquired self-control of insula cortex modulates emotion recognition and brain network connectivity in schizophrenia. Hum Brain Mapp 2013;34(1):200–212. DOI: 10.1002/hbm.21427. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Caria A, Sitaram R, Veit R, Begliomini C, Birbaumer N. Volitional Control of Anterior Insula Activity Modulates the Response to Aversive Stimuli. A Real-Time Functional Magnetic Resonance Imaging Study. Biological Psychiatry 2010;68(5):425–432. DOI: 10.1016/j.biopsych.2010.04.020. [DOI] [PubMed] [Google Scholar]
  • 4.Zotev V, Krueger F, Phillips R, et al. Self-Regulation of Amygdala Activation Using Real-Time fMRI Neurofeedback. PLOS ONE 2011;6(9):e24522. DOI: 10.1371/journal.pone.0024522. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Sulzer J, Sitaram R, Blefari ML, et al. Neurofeedback-mediated self-regulation of the dopaminergic midbrain. Neuroimage 2013;83:817–825. DOI: 10.1016/j.neuroimage.2013.05.115. [DOI] [PubMed] [Google Scholar]
  • 6.Rota G, Sitaram R, Veit R, et al. Self-Regulation of Regional Cortical Activity Using Real-Time fMRI: The Right Inferior Frontal Gyrus and Linguistic Processing. Hum Brain Mapp 2009;30(5):1605–1614. DOI: 10.1002/hbm.20621. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Hamilton JP, Glover GH, Hsu J-J, Johnson RF, Gotlib IH. Modulation of subgenual anterior cingulate cortex activity with real-time neurofeedback. Hum Brain Mapp 2011;32(1):22–31. DOI: 10.1002/hbm.20997. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Young KD, Siegle GJ, Zotev V, et al. Randomized Clinical Trial of Real-Time fMRI Amygdala Neurofeedback for Major Depressive Disorder: Effects on Symptoms and Autobiographical Memory Recall. Am J Psychiat 2017. DOI: 10.1176/appi.ajp.2017.16060637. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.deCharms RC, Maeda F, Glover GH, et al. Control over brain activation and pain learned by using real-time functional MRI. Proceedings of the National Academy of Sciences of the United States of America 2005;102(51):18626–18631. DOI: 10.1073/pnas.0505210102. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Emmert K, Breimhorst M, Bauermann T, Birklein F, Van De Ville D, Haller S. Comparison of anterior cingulate vs. insular cortex as targets for real-time fMRI regulation during pain stimulation. Frontiers in behavioral neuroscience 2014;8. DOI: 10.3389/fnbeh.2014.00350. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Guan M, Ma LJ, Li L, et al. Self-Regulation of Brain Activity in Patients with Postherpetic Neuralgia: A Double-Blind Randomized Study Using Real-Time fMRI Neurofeedback. Plos One 2015;10(4). DOI: 10.1371/journal.pone.0123675. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Robineau F, Meskaldji DE, Koush Y, et al. Maintenance of voluntary self-regulation learned through real-time fMRI neurofeedback. Front Hum Neurosci 2017;11. DOI: 10.3389/fnhum.2017.00131. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Kirschner M, Sladky R, Stämpfli P, et al. 634. Cognitive Neurostimulation of the Dopaminergic Midbrain with Real Time fMRI Neurofeedback Training: A Novel Treatment Approach for Cocaine Addiction? Biological Psychiatry 2017;81(10, Supplement):S257. DOI: 10.1016/j.biopsych.2017.02.504. [DOI] [Google Scholar]
  • 14.Brady KT, Hanlon CA, Hartwell K, et al. Real-time functional MRI feedback attenuates cue-induced craving in non-treatment-seeking nicotine-dependent smokers. Drug & Alcohol Dependence 2015;146:e229. DOI: 10.1016/j.drugalcdep.2014.09.090. [DOI] [Google Scholar]
  • 15.Hanlon CA, Hartwell KJ, Canterberry M, et al. Reduction of cue-induced craving through realtime neurofeedback in nicotine users: The role of region of interest selection and multiple visits. Psychiatry Research: Neuroimaging 2013;213(1):79–81. DOI: 10.1016/j.pscychresns.2013.03.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Kirsch M, Gruber I, Ruf M, Kiefer F, Kirsch P. Real-time functional magnetic resonance imaging neurofeedback can reduce striatal cue-reactivity to alcohol stimuli. Addiction Biology 2016;21(4):982–992. DOI: 10.1111/adb.12278. [DOI] [PubMed] [Google Scholar]
  • 17.Karch S, Keeser D, Hummer S, et al. Modulation of Craving Related Brain Responses Using Real-Time fMRI in Patients with Alcohol Use Disorder. Plos One 2015;10(7). DOI: 10.1371/journal.pone.0133034. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Deshpande H, Lisinski J, Snider S, Koffarnus M, Bickel W, LaConte SM. Impulsivity is reduced in social drinkers following real-time fMRI. Organization for Human Brain Mapping. Vancover, BC, Canada: 2017. [Google Scholar]
  • 19.Kim DY, Yoo SS, Tegethoff M, Meinlschmidt G, Lee JH. The Inclusion of Functional Connectivity Information into fMRI-based Neurofeedback Improves Its Efficacy in the Reduction of Cigarette Cravings. Journal of cognitive neuroscience 2015;27(8):1552–1572. DOI: 10.1162/jocn_a_00802. [DOI] [PubMed] [Google Scholar]
  • 20.Lorenzetti V, Melo B, Basílio R, et al. Emotion regulation using virtual environments and real-time fMRI neurofeedback. Frontiers in neurology 2018;9:390. DOI: 10.3389/fneur.2018.00390. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Marxen M, Jacob MJ, Muller DK, et al. Amygdala Regulation Following fMRI-Neurofeedback without Instructed Strategies. Front Hum Neurosci 2016;10. DOI: 10.3389/fnhum.2016.00183. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Hellrung L, Dietrich A, Hollmann M, et al. Intermittent compared to continuous real-time fMRI neurofeedback boosts control over amygdala activation. Neuroimage 2018;166:198–208. DOI: 10.1016/j.neuroimage.2017.10.031. [DOI] [PubMed] [Google Scholar]
  • 23.Johnson KA, Hartwell K, LeMatty T, et al. Intermittent “Real-time” fMRI Feedback Is Superior to Continuous Presentation for a Motor Imagery Task: A Pilot Study. J Neuroimaging 2012;22(1):58–66. DOI: 10.1111/j.1552-6569.2010.00529.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Oblak EF, Lewis-Peacock JA, Sulzer JS. Self-regulation strategy, feedback timing and hemodynamic properties modulate learning in a simulated fMRI neurofeedback environment. Plos Computational Biology 2017;13(7). DOI: 10.1371/journal.pcbi.1005681. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Harmelech T, Preminger S, Wertman E, Malach R. The Day-After Effect: Long Term, Hebbian-Like Restructuring of Resting-State fMRI Patterns Induced by a Single Epoch of Cortical Activation. J Neurosci 2013;33(22):9488–9497. DOI: 10.1523/jneurosci.5911-12.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Ros T, Enriquez-Geppert S, Zotev V, et al. Consensus on the reporting and experimental design of clinical and cognitive-behavioural neurofeedback studies (CRED-nf checklist). Brain 2020;143(6):1674–1685. DOI: 10.1093/brain/awaa009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.First M, Williams J, Karg R, Spitzer R. Structured clinical interview for DSM-5—Research version (SCID-5 for DSM-5, research version; SCID-5-RV). Arlington, VA: American Psychiatric Association; 2015. [Google Scholar]
  • 28.Sobell LC, Sobell MB. Timeline follow-back: A technique for assessing self-reported alcohol consumption. Measuring alcohol consumption: Psychosocial and biochemical methods 1992:41–72. DOI: 10.1007/978-1-4612-0357-5_3. [DOI] [Google Scholar]
  • 29.Higley AE, Crane NA, Spadoni AD, Quello SB, Goodell V, Mason BJ. Craving in response to stress induction in a human laboratory paradigm predicts treatment outcome in alcohol-dependent individuals. Psychopharmacology 2011;218(1):121–129. DOI: 10.1007/s00213-011-2355-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Nijs IMT, Franken IHA, Muris P. The modified Trait and State Food-Cravings Questionnaires: Development and validation of a general index of food craving. Appetite 2007;49(1):38–46. DOI: 10.1016/j.appet.2006.11.001. [DOI] [PubMed] [Google Scholar]
  • 31.Peirce JW. PsychoPy—psychophysics software in Python. J Neurosci Meth 2007;162(1–2):8–13. DOI: 10.1016/j.jneumeth.2006.11.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Lang PJ, Bradley MM, Cuthbert BN. International affective picture system (IAPS): Technical manual and affective ratings. NIMH Center for the Study of Emotion and Attention 1997:39–58. [Google Scholar]
  • 33.Kurdi B, Lozano S, Banaji MR. Introducing the Open Affective Standardized Image Set (OASIS). Behavior research methods 2017;49(2):457–470. DOI: 10.3758/s13428-016-0715-3. [DOI] [PubMed] [Google Scholar]
  • 34.Naqvi NH, Ochsner KN, Kober H, et al. Cognitive Regulation of Craving in Alcohol-Dependent and Social Drinkers. Alcoholism: Clinical and Experimental Research 2015;39(2):343–349. DOI: 10.1111/acer.12637. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Li Q, Wang Y, Zhang Y, et al. Craving correlates with mesolimbic responses to heroin-related cues in short-term abstinence from heroin: an event-related fMRI study. Brain Res 2012;1469:63–72. DOI: 10.1016/j.brainres.2012.06.024. [DOI] [PubMed] [Google Scholar]
  • 36.Smolka MN, Bühler M, Klein S, et al. Severity of nicotine dependence modulates cue-induced brain activity in regions involved in motor preparation and imagery. Psychopharmacology 2006;184(3):577–588. DOI: 10.1007/s00213-005-0080-x. [DOI] [PubMed] [Google Scholar]
  • 37.Naqvi NH, Bechara A. The hidden island of addiction: the insula. Trends in neurosciences 2009;32(1):56–67. DOI: 10.1016/j.tins.2008.09.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Garavan H. Insula and drug cravings. Brain Structure and Function 2010;214(5):593–601. DOI: 10.1007/s00429-010-0259-8. [DOI] [PubMed] [Google Scholar]
  • 39.Naqvi NH, Rudrauf D, Damasio H, Bechara A. Damage to the insula disrupts addiction to cigarette smoking. Science 2007;315(5811):531–534. (Article) (In English). DOI: 10.1126/science.1135926. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Rotge J-Y, Cocker PJ, Daniel M-L, Belin-Rauscent A, Everitt BJ, Belin D. Bidirectional regulation over the development and expression of loss of control over cocaine intake by the anterior insula. Psychopharmacology 2017;234(9):1623–1631. DOI: 10.1007/s00213-017-4593-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Hayashi T, Ko JH, Strafella AP, Dagher A. Dorsolateral prefrontal and orbitofrontal cortex interactions during self-control of cigarette craving. Proceedings of the National Academy of Sciences 2013;110(11):4422–4427. DOI: 10.1073/pnas.1212185110. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Liu Q, Shen Y, Cao X, et al. Either at left or right, both high and low frequency rTMS of dorsolateral prefrontal cortex decreases cue induced craving for methamphetamine. The American journal on addictions 2017;26(8):776–779. DOI: 10.1111/ajad.12638. [DOI] [PubMed] [Google Scholar]
  • 43.Lammel S, Lim BK, Ran C, et al. Input-specific control of reward and aversion in the ventral tegmental area. Nature 2012;491(7423):212–217. DOI: 10.1038/nature11527. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.You C, Vandegrift B, Brodie MS. Ethanol actions on the ventral tegmental area: novel potential targets on reward pathway neurons. Psychopharmacology 2018;235(6):1711–1726. DOI: 10.1007/s00213-018-4875-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.D’Ardenne K, McClure SM, Nystrom LE, Cohen JD. BOLD responses reflecting dopaminergic signals in the human ventral tegmental area. Science 2008;319(5867):1264–1267. DOI: 10.1126/science.1150605. [DOI] [PubMed] [Google Scholar]
  • 46.Magill M, Ray LA. Cognitive-Behavioral Treatment With Adult Alcohol and Illicit Drug Users: A Meta-Analysis of Randomized Controlled Trials. Journal of Studies on Alcohol and Drugs 2009;70(4):516–527. DOI: 10.15288/jsad.2009.70.516. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Rance M, Walsh C, Sukhodolsky DG, et al. Time course of clinical change following neurofeedback. Neuroimage 2018;181:807–813. DOI: 10.1016/j.neuroimage.2018.05.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Megumi F, Yamashita A, Kawato M, Imamizu H. Functional MRI neurofeedback training on connectivity between two regions induces long-lasting changes in intrinsic functional network. Front Hum Neurosci 2015;9. DOI: 10.3389/fnhum.2015.00160. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

SupplementalMaterials

Data Availability Statement

The data that support the findings of this study are available on request from the corresponding author upon reasonable request. The data are not publicly available due to privacy or ethical restrictions.

RESOURCES