A Meta-analysis of Randomized Controlled Trials of Functional Magnetic Resonance Imaging Neurofeedback for Training of Attention and Related Cognitive Functions

Abstract

Objective

This meta-analysis investigated the effects of functional magnetic resonance imaging neurofeedback (fMRI-NF) on inattention and cognitive dysfunctions.

Methods

Using the keywords “fMRI neurofeedback” and “attention,” randomized controlled trials were identified from major electronic databases from inception to June 2024. Outcomes were expressed as standardized mean differences (SMDs) with 95% confidence intervals (CIs).

Results

Five studies including 214 participants (mean age = 16.8 years) were analyzed. The median number of fMRI-NF sessions was four (range 3−15), and the median follow-up duration was three weeks (range 1−10 weeks). Stimulation sites included the right inferior frontal gyrus (n = 2), right anterior insular cortex (n = 1), dorsal anterior cingulate cortex (n = 1), and an individualized approach (n = 1). The primary analysis showed no significant difference in inattention improvement between the fMRI-NF and control groups (SMD = 0.17, 95% CI −0.15 to 0.49, p = 0.30; four studies, 153 participants). Secondary outcomes also showed no significant intergroup differences, including vigilance (SMD = 0.08, 95% CI −0.49 to 0.65, p = 0.79), inhibition (SMD = −0.68, 95% CI −2.11 to 0.74, p = 0.35), and processing speed (SMD = −0.56, 95% CI −1.38 to 0.27, p = 0.19). Heterogeneity was greater for the secondary outcomes than for the primary outcome, with I² values of 49% for vigilance, 90% for inhibition, and 79% for processing speed, compared with 0% for the primary outcome.

Conclusion

Given the limited evidence available to date, current evidence remains insufficient to draw robust conclusions regarding the therapeutic effectiveness of fMRI-NF.

BACKGROUND

Attention, which requires proper brain functions that enable goal-directed behavior through focusing on currently relevant pieces of information and ignoring the irrelevant ones [1], involves the integration of different brain networks [2]. The behavioral and developmental importance of such brain functions is best seen in attention deficit/hyperactivity disorder (ADHD), one of the most common neurodevelopmental disorders [3]. Moreover, instead of being confined to ADHD [4], problems with attention can be quite debilitating in various neuropsychiatric disorders, such as depression [4]. Furthermore, not only is attention ability associated with academic performance in the general population but it inattention was also associated with higher odds of presenting to the emergency department including injuries from accidents and self-harm [5].

Despite well-established evidence in support of treatments for inattention including both behavioral and pharmacological interventions [6], poor medication adherence was observed, probably attributable to concerns over side effects and stigma on medication use for both caregivers of children/adolescents and adults with attention problems [7,8]. Likewise, notwithstanding the general recommendations of psychosocial interventions and environmental modifications for individuals with symptoms of inattention [6], satisfactory responses were not noted in a significant portion of cases [9]. Therefore, alternative therapies have become a popular option among those suffering from inattention as well as their caregivers [10,11].

Alternative therapies targeting ADHD encompass a wide variety of treatment approaches, ranging from nutritional supplements [10,12] to more advanced brain technologies such as brain stimulation [13]. The application of electroencephalography in neurofeedback (NF) training is among one of the most extensively investigated [14-17]. A recent meta-analysis including 21 randomized controlled trials (RCTs) with 1,261 participants reported superior therapeutic effect associated with electroencephalogram-based neurofeedback (EEG-NF) compared with waitlist/treatment as usual (TAU) but not with sham control [17]. One of the major problems with EEG-NF is the lack of preciseness in targeting the problematic region as the device collects signals from the surface of the brain which may not reflect deeper brain activities or those over specific areas [18]. Indeed, recent functional magnetic resonance imaging (fMRI) studies revealed several under-activated brain regions involved in executive control (i.e., frontostriatal and frontocerebellar systems) in individuals diagnosed with ADHD [19], suggesting potential therapeutic benefits of a targeted approach. Therefore, fMRI neurofeedback (fMRI-NF), which allows the treatment of more specific brain regions, has been used as a therapeutic alternative to EEG-NF [20].

Nevertheless, in addition to the limited number of RCTs that investigated the effectiveness of fMRI-NF in improving attention [20-24], heterogeneity in therapeutic approaches (e.g., different target locations for training) and inconsistencies in their findings further tarnished the robustness of currently available evidence. Moreover, although fMRI is superior to EEG in terms of the anatomical assessment of brain activities, the latter still has its merits including a lower cost and simpler procedures [25]. More robust evidence is needed to support either approach in clinical practice.

The present meta-analysis, therefore, aimed at investigating the therapeutic effects of fMRI-NF on inattention as well as the improvement in cognitive functions.

METHODS

Protocol and Registration

The current study, which was registered in the international prospective register of systematic reviews (PROSPERO CRD42024551166), was conducted in compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [26].

Search Strategy and Selection Criteria

Utilizing the main keywords “fMRI neurofeedback” and “attention” (Supplementary Table 1; available online), RCTs were retrieved from major electronic databases, namely PubMed, Embase, Cochrane CENTRAL, ScienceDirect, and ClinicalTrials.gov from inception to June 2024 without restrictions on geographic location and language. Details on the search strategies and keywords adopted for each database are summarized in Supplementary Table 1 (available online). To ensure thoroughness of literature search, the reference lists of the acquired articles were scrutinized for potentially eligible trials. Determination of the article eligibility was based on fulfillment of the PICO (i.e., population, intervention, comparator, and outcomes) criteria: (1) Population: RCTs of patients diagnosed with neuropsychiatric disorders or the general population, (2) Intervention: fMRI-NF, (3) Comparator: sham fMRI-NF, TAU, or controls (4) Outcome: standardized rating scales for ADHD symptom assessment or standardized tests for evaluating attention or related cognitive functions. We excluded studies that (1) did not adopt fMRI-NF as an interventional measure, (2) were non-RCTs, or (3) provided no data on inattention symptoms or results from standardized tests for attention and related cognitive functions in their outcomes.

Data Extraction and Quality Assessment

Utilizing a pre-set search strategy (Supplementary Table 1; available online), the titles and abstracts of the retrieved articles were first screened by two independent authors who were also responsible for determining study eligibility and extracting data, namely study characteristics (e.g., sample size, number of fMRI-NF sessions, and duration of follow-up) and publication-related details. Inter-rater reliability was assessed by the Kappa coefficient [27]. Electronic mails were sent to the corresponding authors of eligible studies with missing information in an attempt to acquire the original data. The quality of the eligible studies was rated with the Cochrane’s “risk of bias” assessment tool [28] which comprises six bias domains including sequence generation, allocation concealment, performance bias (blinding of participants), detection bias (blinding of assessors), attrition bias (incomplete outcome data), reporting bias (selective outcome reporting), and one additional category for scrutinizing the presence of any other notable biases. The risk of bias was rated as “low,” “unclear,” or “high.” The certainty of evidence for individual outcomes of interest was evaluated with the Grading of Recommendations Assessment, Development and Evaluation framework [29]. Disagreements on eligibility and study quality between the two reviewers were settled through discussion with a third author.

Data Synthesis and Analysis

The primary outcome was set as the improvement in attention, while the secondary outcomes were improvements in attentional or related cognitive functions including processing speed and disinhibition. For quantitative analysis, the outcomes of interest were expressed as effect size (ES) derived from standardized mean difference (SMD) and odds ratios (ORs) for continuous and categorical data, respectively, with 95% confidence interval (CI). All data analysis was conducted with Review Manager 5 (RevMan 5.4; Copenhagen: The Nordic Cochrane Center, The Cochrane Collaboration, 2014). ORs and SMD were analyzed with Mantel-Haenszel method and generic inverse-variance method, respectively. We further assessed the reliability of available outcomes using leave-one-out sensitivity analysis. I-squared statistics was used to estimate the heterogeneity across the included studies. Statistical significance was set at a pvalue below 0.05 for all outcomes. The existence of publication bias was evaluated with funnel plot inspection.

RESULTS

Study Selection and Characteristics of Included Studies

The process of the study selection was based on the PRISMA statement [26] (Fig. 1). Of the 984 articles initially retrieved from the electronic search of major and unpublished databases (e.g. ClinicalTrials.gov), 877 were deemed inappropriate following the inspection of their titles and abstracts. Of the 17 remaining studies for full-text review, five RCTs including 214 participants were considered eligible for the current meta-analysis [20-24] with the other 10 studies being excluded for various reasons (Supplementary Table 2; available online). The two reviewers unanimously agreed on the eligibility of the studies being included (Kappa coefficient = 1.0).

Fig. 1.

PRISMA diagram of identifying eligible studies.

PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses; fMRI, functional magnetic resonance imaging.

The characteristics of the included studies are summarized in Table 1. Three out of the five eligible studies included only children or adolescents (≤ 18 years) [20,22,23], while the other two recruited only adults [21,24]. The mean age of participants in the included studies was 16.80 years with the proportion of females ranging from 0 to 53.85%. The median number of repetitive transcranial magnetic stimulation treatment sessions was four (range: 3−15) with the median duration of follow-up being three weeks (range: 1−10 weeks). With regard to the targeted brain area for training, two studies focused on the right inferior frontal gyrus [20,22], one pinpointed the right anterior insular cortex (rAIC) [24], one targeted the dorsal anterior cingulate cortex, and one took an individualized approach in which the region of interest was selected during an emotional matching task [23]. Most studies allowed the stable use of psychostimulants without changing dosage during the study period. All five studies were conducted in European countries (Table 1).

Table 1.

Summary of characteristics of studies in the current meta-analysis

Study (year)	Diagnosis (criteria)	Design	Comparison	Number	Duration (wk)	Sessions	Outcome	Medications	Mean age (yr)	Female (%)	Country
Böttinger (2024) [23]	CD or ODD (DSM-5)	RCT	NF – individualized: bilateral INS or the bilateral AMG	12	10	10	ADHD symptoms = CBCL − ADHD	Allowed all medications	14.57 (12−18)	22	Europe
			TAU	13
Popovova (2024) [24]	General population	RCT	NF: right anterior insular cortex	15	1	3	Processing speed = ANT − RT	N/A	23.83	41.54	Switzerland
			Mental rehearsal training	14
Lam (2022) [20]	ADHD (DSM-5)	RCT	NF: rIFC	44	3	15	ADHD symptoms = ADHD-RS Vigilance = CPT omission Inhibition = CPT commission Processing speeda	On stable or no medication	13.2 (10−18)	0	UK
			Sham control	44
Alegria (2017) [22]	ADHD (DSM-5)	RCT	NF: rIFC	18	2	4	ADHD symptoms = ADHD-RS Vigilance = CPT omission Inhibition = CPT commission Processing speed = stop signal RT	On stable or no medication	13.9 (12−17)	0	UK
			Control: feedback from left parahippocampal gyrus	13
Zilverstand (2017) [21]	ADHD (DSM-IV-TR)	RCT	NF: dACC	7	4	4	ADHD symptoms = ADHD DSM-IV score Vigilance = SA-DOTS − vigilance Inhibition = SA-DOTS − response inhibition	On stable or no medication	36.68 (> 18)	53.85	Netherlands
			Mental rehearsal training	6

ADHD, attention-deficit/hyperactivity disorder; ADHD-RS, the ADHD rating scale; AMG, amygdala; ANT, Attention Network Test; CD, conduct disorder; CBCL, child behavior checklist; CPT, continuous performance test; dACC, dorsal anterior cingulate cortex; DSM-IV, Diagnostic and Statistical Manual of Mental Disorders, fourth edition; DSM-IV-TR, Diagnostic and Statistical Manual of Mental Disorders, fourth edition, text revision; DSM-5, Diagnostic and statistical manual of mental disorders fifth edition; INS, insula; rIFC, right inferior frontal cortex; N/A, not available; NF, neurofeedback; ODD, oppositional defiant disorder; RCT, randomized controlled trial; RT, reaction time; SA-DOTS, Sustained Attention DOTS task; TAU, treatment as usual.

^aCombined from MARS go/no-go, Simon, and CPT tasks combined.

Risk of Bias Assessment

In terms of the quality of the included studies based on the Cochrane Collaboration’s tool, two studies had unclear reporting about the randomization process (Fig. 2). Three out of the five studies were considered to have a high risk of detection and performance bias due to the lack of a double-blind sham-controlled design (Fig. 2). One study was assigned a high risk of attrition bias because its primary outcome was improvement in emotional control rather than attention (Fig. 2). Overall, an overall moderate risk of bias was assigned to the five RCTs, of which over half lacked a double-blind sham-controlled design.

Fig. 2.

Risk of bias for eligible studies.

Results of Syntheses

Primary outcome

The results of the current meta-analysis failed to demonstrate improvement in the total symptoms of ADHD in participants treated with fMRI-NF compared with the controls (SMD = 0.17, 95% CI: −0.15 to 0.49, p = 0.3, four studies with 153 participants) (Fig. 3). Robustness of the findings remained unchanged on sensitivity analysis without significant heterogeneity (I² = 0% and p = 0.96). Visual inspection of the funnel plot revealed no notable asymmetry (Supplementary Fig. 1; available online).

Fig. 3.

Forest plot of effect size for comparing the difference in the improvement of symptoms of inattention between fMRI-NF and control groups.

CI, confidence interval; fMRI-NF, functional magnetic resonance imaging neurofeedback; Std, standardized; SD, standard deviation; IV, inverse variance.

Secondary outcomes

The results of secondary outcomes also failed to demonstrate superiority of fMRI-NF over controls regarding the improvement in neurocognitive functions, including vigilance (SMD = 0.08, 95% CI: −0.49 to 0.65, p = 0.79, three studies with 132 participants), inhibition (SMD = −0.68, 95% CI: −2.11 to 0.74, p = 0.35, three studies with 132 participants) and processing speed (SMD = −0.56, 95% CI: −1.38 to 0.27, p = 0.19, three studies with 151 participants) (Supplementary Figs. 2-4; available online). The levels of heterogeneity were high for all of the three secondary outcomes (I² = 49% and p = 0.19 for vigilance, I² = 90% and p < 0.01 for inhibition and I² = 79% and p < 0.01 for processing speed). Leave-one-out sensitivity analysis revealed an association between fMRI-NF and a significant increase in processing speed after exclusion of the study by Lam et al. [20]. Visual inspection of the funnel plot showed no significant asymmetry (Supplementary Figs. 5-7; available online).

Certainty of evidence

The overall certainty of evidence for each outcome of interest is shown in Supplementary Table 3 (available online). The certainty of evidence was graded as low for the primary outcome because of the limited numbers of RCTs, relatively small sample sizes, and the high risk of bias in most studies due to the lack of a double-blind design. For all secondary outcomes, the certainty of evidence was further down-graded to very low for the high level of inconsistency.

DISCUSSION

To the best of our knowledge, this study is the first meta-analysis to investigate the therapeutic effects of fMRI-NF against attentional dysfunction. The current study, which identified five clinical trials including 214 participants, showed improvement in neither the total symptoms of inattention nor the performance in cognitive functions including vigilance, inhibition, and processing speed compared with the controls. The significance of the findings, however, was blemished by the small numbers of participants and available trials, as well as the diversity of control groups with only one study using sham controls. Therefore, the results were considered preliminary with limited robustness to rule out the possible therapeutic effects of fMRI-NF against inattention. Further large-scale clinical studies are needed to address the therapeutic and cost effectiveness of fMRI-NF in this setting.

Although EEG provides a better temporal resolution than that of fMRI, the latter offers a superior spatial resolution compared to quantitative electroencephalography [30]. Previous studies have demonstrated the therapeutic effectiveness of EEG-NF compared with the controls [15,17], but failed to show treatment advantage over sham EEG [15,17]. Therefore, fMRI-NF, which enables self-regulation of brain activation in specific regions/networks by providing real-time feedback in response to brain activity, could be a novel alternative to EEG-NF [31]. However, the results of our meta-analysis showed no significant improvement in the total symptoms of inattention in the absence of notable heterogeneity despite the wide variation in the protocols (i.e., duration of treatment or location of brain region) and control groups (i.e., sham control or TAU) across the included studies [20-24]. In contrast, a prior meta-analysis revealed a significant therapeutic benefit of EEG-NF compared with TAU but not sham EEG, suggesting the possibility of motivation enhancement due to the lack of a double-blind design rather than the true effects of EEG-NF [17]. Taking into account TAU is of minimal therapeutic advantage compared with cognitive training and sham controls, the fact that even the only study included in this meta-analysis that used TAU as control failed to show significant improvement in inattention in those undergoing fMRI-NF may further support its ineffectiveness in this setting [23]. Nevertheless, the main outcome of that study was emotional regulation rather than inattention and the targeted brain regions were associated with emotion (i.e. amygdala) rather than specific for attentional control [23]. Therefore, that study alone could not rule out the therapeutic effects of fMRI-NF compared with TAU [23]. More studies using a sham control design that focus on attentional function are required to better elucidate the cost-effectiveness of fMRI-NF.

Of the five included studies, only one using sham control [20], which is considered a better design to investigate the true effects of feedback from fMRI. The results of that study found a significant improvement in attention post-treatment even in the sham controls despite no significant difference between the active intervention and sham groups, implicating the presence of placebo training effect that may not be region-specific [20]. Consistently, a previous study demonstrated the activation of self-control regions in individuals who attempted to self-regulate and concentrate during sham EEG-NF [32], suggesting the involvement of self-referential processes including self-awareness and self-agency when the participants encounter their own physiological signals that represent considerable self-referential cognitive load. Therefore, evidence derived from the current study did not support the superiority of fMRI-NF over other treatment modalities. Nevertheless, given the limited number of included studies in which only one trial used a sham control that targeted right inferior frontal cortex, our findings were only preliminary and could not rule out the potential therapeutic benefits of fMRI-NF targeting other brain locations.

Results of our secondary outcomes also showed no therapeutic advantage of fMRI-NF regarding performance in cognitive functions including vigilance, inhibition, and processing speed compared with the control groups. Nevertheless, significant heterogeneity was noted. In particular, leave-one-out sensitivity analysis focusing on processing speed demonstrated a significant increase in processing speed associated with fMRI-NF after exclusion of the study by Lam et al. [20]. Intriguingly, of the five included studies, the one that gave the largest ES regarding the improvement in processing speed (i.e., an indicator of cognitive function) also reported significant therapeutic advantage of fMRI-NF in behavioral outcome in both active intervention (i.e., targeting frontal lobe) and control (i.e., targeting hippocampus) groups compared with their respective pretreatment status but without notable difference between the two groups after treatment in behavioral outcomes [22]. The discrepancy in findings between behavioral outcome and cognitive performance in that study may highlight the difference in their confounding factors [22]. In particular, behavioral outcome may be influenced by myriad factors, especially informant bias [33]. That study targeted the right inferior prefrontal cortex, suggesting possible specific training effects over particular brain regions [22]. Consistently, a study using fMRI showed that the cerebello-frontal network has a role to play in governing the cognitive speed [34]. Indeed, a previous study reported an association between age-related decline in processing speed and a reduction in inferior frontal cortex volume [35]. Therefore, although our primary finding demonstrated no significant improvement in behavioral symptoms of inattention which may be related to multiple neuropsychiatric functions, specific cognitive function may still be enhanced by targeting specific brain regions as reflected by the significant improvement in processing speed after removal of one study [21]. Further studies are warranted to elucidate this issue.

There are several limitations to the findings of the current study. First, our results were considered only preliminary due to the limited number of trials (i.e., only five) including a total of 214 participants. Second, despite statistically non-significant heterogeneity, the small number of included studies limited our ability to explore the potential sources of heterogeneity through subgroup or meta-regression analyses. Likewise, the absence of statistically significant heterogeneity in the primary outcome (I² = 0%) required cautious interpretation, given the limited number of studies, the reduced statistical power to detect between-study variability, as well as the wide variations in both intervention parameters (i.e., duration of treatment or location of target brain region) and the choice of control group (i.e., sham control or TAU) across the included studies. In particular, because treatment effects may vary with the selection of specific anatomical locations of the brain, our results cannot be extrapolated to target brain regions other than those included in the eligible studies. Moreover, most studies allowed a combination of fMRI-NF with medical treatment without providing detailed information about the percentage of participants under medications in the intervention and control groups, raising the issue with medication use as a potential confounder of treatment outcomes. Further clinical investigations are warranted to elucidate its impact. Third, significant heterogeneity, which was noted in our secondary results focusing on cognitive functions, was not observed in our primary findings on behavioral outcomes. This may suggest a gap between neurocognitive functions and behavioral symptoms in a real life setting. Fourth, because the raters of behavioral outcomes of the five included studies were all parents, whether assessment by teachers would give consistent results remains to be explored. Fifth, due to significant performance and detection biases in most of the included trials as only one study adopted sham controls, the level of evidence was deemed low or very low. In addition, a previous meta-analysis reported the consistent activation of several key brain regions during self-regulation, in particular the anterior insula and the basal ganglia, during real-time fMRI-NF independent of the targeted region-of-interest [36], raising concerns about whether the observed improvements in behavioral symptoms were due to actual fMRI-NF–associated regulation of the targeted brain regions or merely general training effects. Further double-blinded RCTs are required to rule out the sham effects of training. Sixth, not only did the number of sessions vary widely across the included studies (range: 3−15) but the median number of sessions (i.e., four) was also considered inadequate for behavioral training. Seventh, our leave-one-out sensitivity analysis revealed a statistically significant improvement in processing speed after exclusion of one study [20], further underscoring the relative instability of the pooled findings. Overall, our review of the available literature revealed a lack of double-blind design and standardized protocol with sufficient treatment sessions in most studies. Besides, exclusive parental assessment of behavioral outcomes and limited information on concomitant neurocognitive and behavioral measurement as well as therapeutic outcomes through targeting different brain regions presented important issues that needed to be addressed in further studies.

CONCLUSIONS

Given the limited database from which our findings were derived, current evidence remained insufficient to provide reliable conclusions about the therapeutic effectiveness of fMRI-NF. Further studies adopting sham controls with standardized protocols and broader assessments in both school and home settings using neurocognitive assessment as well as targeting different brain regions are warranted to elucidate this issue.