Cloud-based fMRI neurofeedback to reduce the negative attentional bias in depression: a proof-of-concept study

Abstract

Depressed individuals show an attentional bias toward negatively valenced stimuli and thoughts. In this proof-of-concept study, we present a novel closed-loop neurofeedback procedure intended to remediate this bias. Internal attentional states were detected in real-time by applying machine learning techniques to fMRI data on a cloud server; these attentional states were externalized using a visual stimulus that the participant could learn to control. We trained 15 depressed (MDD) and 12 healthy control (HC) participants over three fMRI sessions. Exploratory analysis showed that MDD participants were initially more likely than HC participants to get “stuck” in negative attentional states, but this diminished with neurofeedback training relative to controls. Depression severity also decreased from pre- to post-training. These results demonstrate that our method is sensitive to the negative attentional bias in MDD and showcase the potential of this novel technique as a treatment that can be evaluated in future clinical trials.

INTRODUCTION

Depressed individuals process negative stimuli differently from healthy participants, leading to differences in attention, memory, and cognitive control (1-5). Depressed participants also tend to show larger and more prolonged neural responses to negative stimuli (6). This may manifest clinically as rumination, or the automatic replay of negative thoughts (1, 7). Given that depressed participants attend more to negative information, researchers have designed paradigms to train participants to reduce this negative bias, and ultimately, depression severity (1, 4).

One common approach is Attention Bias Modification (ABM) training, which involves learning to shift overt spatial attention away from negative stimuli and/or towards positive stimuli (8-12). Another training approach, Cognitive Bias Modification for Interpretation (CBM-I; 13), involves learning to adopt the positive interpretation of an ambiguous situation (e.g., 14).

Following these forms of training, participants typically display the reinforced behavior, for example, attending less to negative stimuli (12, 15). However, transfer to clinical measures (e.g., reduced depression severity or self-reported rumination) has been inconsistent (2, 16-18). A potential limitation of the aforementioned studies is their use of pre-programmed training schedules; recent approaches to attention training have taken a more adaptive approach, providing behavior-based real-time feedback based on mouse position (19) or eye fixation (20-21). Although such approaches have yet to be tested in clinical populations, healthy participants showed promising improvements in reappraisal (21) and rumination (19-20).

While behavioral training has been the main approach to reduce attentional biases in depression, behavioral measures such as button presses and eye movements are downstream effects of underlying neural differences. Neural feedback, such as feedback from functional magnetic resonance imaging (fMRI), allows for measures that are “closer to the source” of the biases, and thus have the potential to be more sensitive and informative. Indeed, depressed participants show neural – but not behavioral – evidence of increased processing of negative stimuli that are presented quickly (see 22). In our work, we therefore sought to combine the advantages of adaptive feedback with the potentially enhanced sensitivity of neural measurements of attention.

A previous study (23) demonstrated the potential of real-time fMRI (rt-fMRI) neurofeedback to improve sustained attention in healthy participants. Participants received visual feedback based on their brain activity during an attention task. Overlaid face and scene images with variable opacity values were shown as participants responded to a cued go category and ignored the other, un-cued no-go category. Neurofeedback was embedded in a closed-loop circuit: If the neural data indicated that participants were attending more to the incorrect category (e.g., faces), then stimuli in that category would become more opaque (e.g., faces would become more prominent and scenes would become more transparent). This served to “externalize” the participant’s bad attentional state, making the task more difficult during attentional lapses, and thus alerting them to try harder to push themselves into a better state. This procedure yielded significant improvements in attention after a single neurofeedback session. Additionally, participants receiving feedback that was veridical (based on their own brain activity) as compared to a control (someone else’s brain activity) exhibited an increased benefit, indicating that individualized feedback was advantageous.

In the current study, we adapted this closed-loop procedure to assess its potential suitability for reducing the negative attentional bias in participants with major depressive disorder (MDD), rather than improving sustained attention in general. To accomplish this goal, we modified the neurofeedback task so participants always had to ignore negative faces; when participants’ attention drifted to the negative faces, the faces were made more visible and the scenes were made less visible. In this situation, participants needed to learn to “unstick” themselves from the negative attentional state in order to make the scenes more visible, so they could continue with the instructed task of responding to the scenes. Additionally, to increase the clinical utility of this procedure, we implemented a novel open-source Python cloud-based analysis pipeline, making it possible to run our multivariate real-time fMRI procedure in the cloud regardless of the availability of local computing resources and expertise.

Given the novelty of the techniques employed here, we sought to run a “proof of concept” study to demonstrate that we met key benchmarks before undertaking a lengthy and expensive clinical trial. These benchmarks include: 1) technical feasibility – showing that we can successfully run a real-time fMRI experiment using our cloud pipeline for closed-loop neurofeedback; 2) sensitivity to negative attentional bias – identifying measures that can robustly detect the negative attentional bias in MDD participants at the start of training; 3) sensitivity to training – showing that the measured difference in negative attentional bias between MDD participants and healthy control (HC) participants decreases over the course of training. These benchmarks constitute the minimum necessary features to justify a (future) clinical trial. We also hoped to see some correlation within MDD patients between improvement on our experimental measure of negative attentional bias and improvement in clinical symptoms, but this kind of individual-differences correlation requires a much larger sample size to be adequately powered, so for our present purposes we were simply hoping to see a trend in the predicted direction.

Our approach builds on a prior pilot study using a variant of this neurofeedback task in seven depressed participants (24) in several ways: First, we used a cloud-based analysis pipeline rather than a customized local computing solution to increase accessibility (benchmark #1). Second, we included a healthy control (HC) group to assess whether our measures can detect a greater negative attentional bias in MDD vs. HC participants (benchmark #2) and how this difference can be affected by training (benchmark #3). Third, we have included a wider range of behavioral and neural measures, with the goal of identifying which of these measures are most sensitive to negative attentional bias and changes in this bias over time. Fourth, we included larger (though still modest) sample sizes to test these benchmarks more definitively.

Below, we describe how our procedure successfully meets the three benchmarks, demonstrating technical feasibility and (in an exploratory analysis) that we can both detect the negative attentional bias and reduce it through training. Although additional work is needed to show that our neurofeedback procedure is an effective treatment for MDD (see Discussion), these proof-of-concept results demonstrate the potential of our procedure for studying psychiatric disorders, and set the stage for future clinical trials.

METHODS AND MATERIALS

Participants

A total of 27 adults participated in the study, including 14 with major depressive disorder (MDD) and one participant with persistent depressive disorder (PDD); n female = 8, mean age = 27.3 years) and 12 who served as healthy controls (HC; n female = 6, mean age = 25.4 years). We had preregistered a sample size of 16 participants from each group, but we were unable to reach the target sample sizes because data collection was suspended during the COVID pandemic. (You may access the preregistration here: https://aspredicted.org/eg59c.pdf.) Twenty-three participants completed all 7 visits as planned; two participants could not complete the in-person portions of Visit 6 because of the COVID pandemic; one participant was lost to follow-up after Visit 5, and one participant was lost to follow-up after Visit 6. For all analyses, we included all data collected, regardless of the availability of follow-up data. Both groups underwent the exact same experimental procedure, differing only in initial diagnosis requirements. Participants were recruited from the University of Pennsylvania Center for Neuromodulation in Depression and Stress (CNDS) laboratory. All participants received monetary compensation for participation. The study was approved by the University of Pennsylvania Institutional Review Board, as well as the Princeton University Institutional Review Board through an IRB Authorization Agreement. See the Supplement for eligibility criteria.

Procedure

The study consisted of seven visits total per participant, as illustrated in Supplemental Table S1. The first five visits were the main study (pre-scanning, 3 neurofeedback sessions, post-scanning). Visit 6 was a behavior-only one-month follow-up. Visit 7 was a three-month follow-up phone call. For all participants, we tried to schedule the first 5 visits within 2 weeks as closely as possible.

Upon providing consent on Visit 1, participants completed the Structured Clinical Interview for DSM-5 Disorders (SCID-5; 25) to assess lifelong symptoms, current depression symptoms, and the presence of additional exclusionary conditions. Participants completed the SCID-5 on Visit 1 only. Upon completion, the MADRS structured clinical interview (26-27) was administered to assess depression symptoms specifically over the week preceding Visit 1. The MADRS was also administered on Visits 5, 6, and 7 to assess how depression severity changed over time. Participants completed additional behavioral and neural tasks before and after neurofeedback (Supplemental Table S1).

Participants completed 7–9 neurofeedback runs per visit on Visits 2-4 (we fit in as many runs as we could within each 2-hour scanning session). Each neurofeedback run contained 8 blocks: the first 4 blocks (“stable blocks”) showed only neutral stimuli with constant opacity and served as training data for the face versus scene classifier; in the last 4 blocks (except run 1), the attended category was neutral scenes and the distractor category was negative faces (Supplemental Figure S1). These blocks served as neurofeedback blocks, in which the opacity changed depending on the relative degree of neural representation of scenes versus faces indicated by a pattern classifier applied to fMRI. Participants were told that the change in opacity was determined by their brain activity rather than their button-pressing accuracy.

At the start of each block, participants were given a cue that indicated the block type (face or scene) and go category. For instance, the cue “indoor scenes” indicated that participants should press for indoor scenes (90% go trials), and refrain from pressing when seeing outdoor scenes (10% no-go trials). Additionally, while making go/no-go judgments, participants had to continuously ignore the overlaid irrelevant stimuli (e.g., faces). For each participant, the go scene category (indoor or outdoor) and go face category (male or female) were the same across all visits. Assignment of categories was counterbalanced across participants within each group. See Supplemental Figure S1 and (23) for additional task details.

Data acquisition

All scanning was acquired with a 3T MRI scanner (Siemens Prisma), using a 64-channel head coil. Sequences were matched to (23) as much as possible. During the first scanning session, we collected a high-resolution magnetization-prepared rapid acquisition gradient-echo (MPRAGE) anatomical scan to construct the whole-brain mask used in real-time and for offline registration. FSL (http://fsl.fmrib.ox.ac.uk/) was used to register the MNI152 standard-space T1-weighted average structure template (28) to each participant's brain in functional space. We used this registered whole-brain ROI as the mask for each participant. The functional scans consisted of a gradient-echo, echo-planar imaging sequence covering the whole brain (2 s repetition time, 28 ms echo time, 3 mm isotropic voxel size, 64 x 64 matrix, 192 mm field of view, 36 slices). At the end of each scanning session, a fieldmap scan was acquired for offline processing.

Real-time processing

Figure 1 provides an overview of our real-time processing system, from the initial stimulus display, to subsequent cloud analysis, and finally, stimulus update. During neurofeedback runs, each new DICOM image was motion corrected to the previous time point following Siemens' custom motion correction. After that, the DICOM file was saved onto a local Linux machine in the scanning room. Then, the data were masked and flattened into a 1D-vector, and sent to the cloud server for further preprocessing (see Supplement) and classification.

Figure 1:

Cloud-based closed-loop rt-fMRI attention training technique. (A) Participants perform a go/no-go task on overlaid face/scene stimuli, where they respond based on whether the scene image is indoor or outdoor, and thus have to constantly ignore negative faces. (B) As each new time point is acquired, the data are masked and flattened to a 1D-vector. (C) The data are sent to a cloud server for preprocessing and classification. (D) The result is sent as a text file to the local machine controlling the display. A sigmoidal transfer function converts the relative scene minus face classification evidence difference into opacity proportions, so that the attended category (as measured by the classifier) will become more visually prominent. (E) The opacity value is smoothed and updated for the next time point. As shown, when participants are in a maximally negative state, the negative faces dominate the composite image.

During neurofeedback, a multi-voxel pattern classifier (29) was used to decode the extent to which attention was directed at the task-relevant scene or the task-irrelevant face for every time point of image acquisition (TR). The difference between the amount of classifier evidence for scenes (ranging from 0 to 1) and faces (ranging from 0 to 1) was used as the output neurofeedback score. This score was saved as a text file and sent back to the local computer to influence the display during the following time point. See the Supplement for further information on cloud configuration and processing.

Neurofeedback display

The Matlab script controlling the display loaded each new text file as it was detected. The neurofeedback score was converted to an opacity value for the neutral scene using a sigmoidal transfer function (Figure 1). Then, opacity was smoothed using a moving window over the values from the previous 2 time points to ensure that changes in opacity were not abrupt (23). This smoothed value was set as the opacity for the following 3 trials (1.5 TRs) while the next time point was collected and preprocessed.

Neurofeedback performance

As a first-pass (preregistered) measure of participants’ bias towards attending to negative faces, we computed the average scene minus face classifier evidence for neurofeedback runs – the more that participants attend to faces, the more negative this score will be. Also, based on prior findings indicating that MDD participants specifically have difficulty in disengaging from negative stimuli (22, 30-31), we computed a second, disengagement-focused neural measure that tracked the probability of remaining stuck in the most negative attentional state; this measure was not preregistered. During neurofeedback runs, when the classifier detected an increase in attention to the (task-irrelevant) negative faces, this triggered an increase in the visibility of those faces, effectively punishing participants by making it even harder to see (and thus respond to) the task-relevant scenes; our second, disengagement-focused measure tracked how well participants were able to “escape” this situation (i.e., negative face attention leading to maximal negative face visibility) by disengaging from the negative face and attending to the neutral scene (see below for how we measured disengagement). To the extent that MDD participants have a disengagement deficit, we might expect this targeted measure to do a better job of detecting this deficit than simply measuring average face activity.

To estimate each participant’s attentional state at a given time, we discretized the continuous distribution of scene minus face classifier evidence [ranging from −1 to +1]. Because we used a logistic regression classifier, classification values were distributed toward the extremes (±1). We adjusted the scene minus face classification bins to roughly equate the number of classification samples in each (Figure 2A).

Figure 2:

Analysis and results. (A) Each block of continuous scene minus face classification evidence was converted into discrete attentional states (dashed lines). This resulted in a roughly equal number of observations in each state across participants. (B) Probability of staying in a particular attentional state over time, for Early and Late NF runs. (C) During Early NF, the two groups differed in their probability of staying in the most negative attentional state. This group difference was eliminated by the Late NF runs. (D) Depression severity scores significantly decreased for MDD participants over time. (E) Within the MDD group, this reduction in getting stuck in the most negative attentional state showed a trending positive correlation with the reduction in depression severity. Circles represent individual participants; bars represent group averages. Error bars represent ±1 s.e.m. ** = p < 0.01; * = p < 0.05; + = p < 0.1

We next quantified the extent to which these discretized attentional states persisted over time. We operationalized this measure as the conditional probability that the scene minus face difference remained in the same bin across time points. Because we were interested in how feedback delivered at time t affected attention, we compared the attentional state at time t to the attentional state at time t plus 5 seconds (where we would expect feedback effects on brain activity to be maximal, accounting for the hemodynamic lag). As each TR was 2 s, we separately calculated results using 2- and 3-TR shifts, and averaged the results to estimate a 5-s shift. Specifically, for each time delay, d (2 or 3 TRs), for a given attentional state bin, A, we calculated the persistence of that state by counting the number of times that the scene minus face classification value fell within A at time t + d, given it was in A at time t. We then divided this number by the total number of occurrences of state A, shown in the equation below:

$$p(A_{t+d}\mid A_t)=\frac{\sum state(t+d)=A}{\sum state(t)=A}$$

We then averaged all conditional probabilities from the 2 and 3 TR shifts over all runs considered. In this manner, we were able to compute persistence in the most negative state (the “disengagement-focused” measure described above) as well as persistence in all of the other states. To understand how attention changed from early to late in training, we isolated the initial three neurofeedback runs of the first session (Visit 2; Early NF) and the final three neurofeedback runs of the last session (Visit 4; Late NF).

RESULTS

Depression severity

As hypothesized, depression severity decreased over time for MDD participants (Figure 2D). Depression scores decreased significantly from pre-training in Visit 1 to post-training in Visit 5 (one-tailed t(14) = 3.61, p = 0.0014), to the one-month follow-up in Visit 6 (one-tailed t(13) = 2.85, p = 0.0069), and to the three-month follow-up in Visit 7 (one-tailed t(12) = 3.43, p = 0.0025).

Neurofeedback performance

We did not observe group differences in average scene minus face classification (i.e., in the amount of time that participants focused on negative faces vs neutral scenes) either at early or late neurofeedback (Supplemental Figure S3). However, we did observe group differences in persistence in specific attentional states. At the start of neurofeedback training, the largest difference between groups in the probability of remaining in the same attentional state over time occurred for the most negative state, with MDD participants showing a greater tendency to get stuck in this state (one-tailed t(24) = 2.80, p = 0.0049)¹ (Figure 2B).

At the end of neurofeedback training, MDD participants were marginally less likely to get stuck in the most negative state, compared to the start of neurofeedback training (paired one-tailed t(13) = 1.67, p = 0.059). Additionally, there was a significant interaction between group and visit, such that the MDD group decreased in their probability of getting stuck in the most negative state from early to late in training relative to the HC group (unpaired one-tailed t-test comparing change in MDD group to change in HC group, t(24) = 2.04, p = 0.026; Figure 2C).

Additionally, the reduction in the MDD group was associated (across participants) with a marginally significant reduction in depression severity (Pearson r = 0.48, p = 0.083) (Figure 2E). There was also a trending correlation in the same direction in the HC group (Pearson r = 0.51; p = 0.090). As noted in the Introduction, these individual-differences analyses are underpowered in the current study, so we were not expecting significant results; nonetheless it is promising that results are trending in the predicted direction.

DISCUSSION

Our novel closed-loop neurofeedback method successfully detected the difficulty that MDD participants have in disengaging attention from negative stimuli – this was evident in our finding that, at the outset of training, MDD (vs. HC) participants were more likely to get “stuck” in the most negative attentional state (Figures 2B, 2C). Of note, there were no significant initial differences in the average level of scene minus face classifier evidence for MDD vs. HC participants (Supplemental Figure S3) – that is, it was not the case that MDD participants simply had more brain activity related to negative faces. To expose the initial difference between groups, we relied on an exploratory measure that specifically tracked participants’ tendency to persist in a negative state; this initial difference between groups is consistent with the idea that the core deficit in MDD is related to disengaging from negative states, instead of a more general tendency toward negative affect (22, 30-31).

It is also noteworthy that there were no initial differences in behavioral performance on the go/no-go task between MDD and HC participants (Supplemental Figure S4). The detection of group differences for the go/no-go task in neural – but not behavioral – data implies that neural measures may be more sensitive for capturing attentional differences (32-33). This underscores the value of using rt-fMRI neurofeedback training to reduce negative attentional bias.

Compared to most depression studies in the rt-fMRI literature, this technique is unique in its design and analysis methods. First, we trained participants to disengage attention from negative stimuli; by contrast, most of the previous rt-fMRI studies trained depressed participants to increase neural responses to happy stimuli, such as images (34-35) or autobiographical memories (36-39). Regulating positive emotions has yielded robust benefits. For example, (35) even found unintentional clinical benefits for the control group, who imagined relaxing scenes while regulating scene-specific ROIs. Our study explored a less-common approach of training away from negative stimuli, instead of toward positive stimuli. This was based on our belief that learning to regulate negative attention may strike at the underlying dysfunction more directly. The relative efficacy of training negative vs. positive attention can be tested in future studies, e.g., by using a variant of our paradigm where participants are instructed to attend to positively valenced faces or scenes, while ignoring neutral distractors from the other category.

Our technique differs from the small number of other rt-fMRI studies using negative stimuli in that we regulated a decoded cognitive state, as opposed to mean ROI activity (40-41). For instance, (40) trained 10 depressed participants to reduce neural responses to negative images within an individualized ROI (defined based on the single voxel most sensitive to negative images within the “salience network”). In another study, (41) participants recalled negative memories while using a strategy from cognitive behavioral therapy (CBT). During CBT application, a neurofeedback signal was used to train participants to decrease anterior cingulate cortex activity. Both studies yielded promising clinical benefits specific to real-time training, in the form of decreased negative self-descriptions (40) and increased use of the trained CBT strategy after neurofeedback (41). More work is needed to assess the relative efficacy of our closed-loop attention-training procedure compared to ROI-based approaches.

Following the promising results of this initial study, future work could aim to verify that the positive clinical effects we observed are specific to the individualized nature of the neurofeedback. At this stage, we cannot rule out the possibility that nonspecific factors (e.g., time, practice, placebo effects) led to the observed changes in the MDD group. To address this, another control group of MDD participants would need to be recruited to receive feedback scores that are either yoked to a previous MDD participant's brain (23) or determined by an irrelevant ROI (36-37). If this group does not show the same improvements, it would be more certain that the improvements shown by our MDD participants relate to receiving individualized neurofeedback (vs. a more general effect of the procedure).

An important feature of the method reported here is that the real-time analysis of the imaging data was performed on the cloud. Although the sophistication and practical utility of real-time fMRI has increased over the past 10 years (42), the prevalence of its use has been hampered by hardware requirements and by the technical complexity of setting up and running an experiment. Offloading computation to the cloud should help to make this approach accessible to researchers regardless of local computing resources and local computational expertise. Furthermore, the scalable nature of cloud computing makes it easy to add computational complexity to these pipelines; for example, if one wanted to explore multiple analysis variants in parallel to optimize the performance of the classifier, one only needs to requisition more cloud computing resources. In future work, we plan to extend our framework to control both the real-time processing and neurofeedback display via a cloud-based web server, to further minimize local dependency.

In summary, this initial proof-of-concept study highlights the potential clinical benefits of real-time fMRI neurofeedback procedures that target specific cognitive states. By tracking sustained attention over time, our technique provides a face-valid way of detecting the difficulties that MDD patients experience in getting “stuck” in negative states. This was borne out in the observed sensitivity of our measure to initial differences between MDD and HC participants. By “externalizing” these internal attentional lapses (i.e., making task-irrelevant negative faces more visible as they were attended more), our technique provides rich feedback that patients can leverage to learn to control these states. This training potential is supported by our findings showing reduced sustained negative attention in MDD patients and reduced depressive symptoms. By making this technique openly accessible on the cloud, we hope to make it easier for other researchers to explore the benefits of this approach in diagnosing and treating MDD and other clinical syndromes.²