The climate of neurofeedback: scientific rigour and the perils of ideology

Sir,

Over the last six decades, an in-group with ideological and financial stakes has been conducting sub-par research to develop an ostensibly effective clinical intervention: EEG-neurofeedback. More recently, however, a string of independent studies featuring increased scientific rigour and tighter experimental controls has challenged the foundation on which EEG-neurofeedback stands. Earlier this year, Brain published one of the most robust EEG-neurofeedback experiments to date (Schabus et al., 2017), which sparked a flurry of correspondence concerning the therapeutic value of neurofeedback (Fovet et al., 2017; Schabus, 2017); notably, a parallel discussion continues in Lancet Psychiatry (Micoulaud-Franchi and Fovet, 2016; Thibault and Raz, 2016a; Schönenberg et al., 2017). However, to effectively interpret the pro and con viewpoints, one must appreciate the peculiar culture surrounding the field of EEG-neurofeedback. The present breezy piece provides little-discussed yet highly relevant contextual information often absent from formal papers and technical reports.

History and backdrop

EEG-neurofeedback thrives in two major arenas: commercial and academic. The commercial branch includes private practitioners, equipment manufacturers, as well as other corporate and lucrative aspects of neurofeedback. Most of the research that fuels these commercial ventures comes from vanity presses and specialty associations, including the Institute for Neurofeedback and Research (ISNR), the Biofeedback Federation of Europe (BFE), the Association for Applied Psychophysiology and Biofeedback (AAPB), and the Biofeedback Certification International Alliance (BCIA), as well as private entities (e.g. EEGInfo.com). These associations publish everything from magazines for lay audiences to methodological textbooks for practitioners and research journals that rarely seek referees beyond their clique (e.g. NeuroRegulation). The leaders of these commercial organizations often have financial and ideological stakes in EEG-neurofeedback (Thibault and Raz, 2017). Moreover, they frequently promote and offer training for neurofeedback techniques that rarely receive attention outside the commercial sphere (e.g. QEEG, z-score training, LORETA feedback). Thus, these circles rely on research that shies away from standard scientific thoroughness.

More than a decade ago, in a motion designed to separate themselves from commercial interests, the European branch of the ISNR split off to create the Society of Applied Neuroscience (SAN)—a more academic group focused on how EEG-neurofeedback works, rather than on promoting commercial products and services. We were invited to present at SAN’s most recent meeting in Greece and one of us (R.T.T.) attended. R.T.T. was surprised to hear the SAN president open the conference with a statement that the society aims to prove that neurofeedback works—a non-scientific starting point of a troublesome agenda. At the first neurofeedback workshop, a practitioner asked a volunteer to upregulate a subset of EEG waves known as the alpha band. After the practitioner displayed the data demonstrating ‘successful’ upregulation of alpha, R.T.T. pointed out an apparent contradiction: the volunteer actually misunderstood the instructions and was actively attempting to downregulate, not upregulate, alpha, but the audience seemed to neither notice nor care. The next day, when R.T.T. presented on the topic of psychosocial influences and the need for strong experimental controls, he received a response reflecting an all-too-common stance in the field: ‘Why don’t we focus on the more important issues?’ (e.g. identifying what bandwidth to train or specifying the presumed neurophysiological mechanisms).

Many members of the neurofeedback community adhere to the premise that EEG-neurofeedback alters brain activity and that this neural modulation improves behaviour. However, we have systematically analysed some of the underlying problems with this tenet and found that expectation, suggestion, motivation, and other such factors likely drive the observed behavioural outcomes (Thibault et al., 2016, 2017). Subsequently, some practitioners have responded to our critical reports with a string of passionate communications: emails, non-peer-reviewed articles (Othmer, 2017), and YouTube videos (Othmer, 2016). To paraphrase the main point, albeit specious, that they put forward: ‘placebo effects alone cannot possibly explain the benefits of EEG-nf’. But the scientific evidence shows otherwise (Thibault et al., 2015; Raz and Harris, 2016).

Registries and false positives

While EEG-neurofeedback experiments with positive findings abound, the absence of a formal registry makes it difficult, if not impossible, to identify how many negative findings fail to reach publication. In a first step to improve this state of affairs, Manuel Schabus recently set up an informal open repository for researchers to post their neurofeedback findings—be they positive or negative (http://decision.sbg.ac.at/limesurvey/index.php/778671/lang-en). This exciting development may lead to further, perhaps more formal, platforms to encourage transparent and rigorous neurofeedback research. Bear in mind, however, positive results can represent veridical effects or false positives.

The field of EEG-neurofeedback remains susceptible to a high proportion of false positives. At least six characteristics have been identified as predicting a systematic high-risk of false-positives: small sample sizes, small effect sizes, exploratory analyses, flexible research designs, ideological or financial interests, and the trendiness of a field (Ioannidis, 2005). Empirically, EEG-neurofeedback scores highly on many of these characteristics. An important next step for the field of neurofeedback, therefore, would be to pre-register clinical trials (e.g. on platforms such as www.osf.io), in line with the standard of other clinical domains.

Constructive ways to move forward

Most scientists would agree that higher quality studies are long overdue for a field spanning almost 60 years of research, with over 3000 publications, and hundreds, if not thousands, of private practitioners (Thibault and Raz, 2016b). However, unlike standard clinical researchers, many proponents of neurofeedback seem to undervalue the power of psychosocial influence and disregard the necessity for proper experimental design and control conditions. A more robust understanding of the science of placebo factors, alongside an appreciation for the value of parsimony and falsifiability (e.g. as championed by William of Ockham and Karl Popper), would surely lead neurofeedback advocates to consider alternative explanations (Thibault et al., 2017). Instead, proponents of EEG-neurofeedback continue to make unsubstantiated claims.

In a recent Letter to the Editor, Fovet et al. (2017) argue that we need ‘a deeper exploration of the neural mechanisms and methodological nuances emerging from this embryonic field—preferably before premature launches of double-blind clinical studies’. And yet, robust studies have already begun to report neural data from double-blind experiments (Schabus et al., 2017; Schönenberg et al., 2017)—precisely the type of studies that would be necessary to specify what drives the neurophysiological and behavioural outcomes. In earlier writings, Fovet et al. contradicted themselves by suggesting that ‘despite [the lack of double-blind studies], neurofeedback research is not at an embryonic stage…findings from several single randomised trials and non-randomised studies have supported the clinical efficacy of EEG-neurofeedback in ADHD’ (Micoulaud-Franchi and Fovet, 2016). Such conflicting statements—arguing that, on the one hand, the field is too nascent for a judgement call while, on the other hand, sufficient evidence supports the application of EEG-neurofeedback—present a conundrum. Moreover, Fovet et al. further argue that the correlations between EEG regulation and sleep quality in the Schabus et al. (2014) study are difficult to reduce to placebo mechanisms. Yet, placebo factors such as motivation and effort could conceivably improve both task performance and sleep quality, thus leading to a systematic correlation between these outcome measures. It seems that the power and nuance of placebo science eludes many a neurofeedback practitioner.

Researchers should be able to conduct high quality research even with financial and ideological ties. To do so, they must largely remove their partialities from the experimental process and the interpretation of results; for example, by blinding those who analyse the data and by weighing alternative explanations (Nuzzo, 2015). Unfortunately, however, such practices remain rare in EEG-neurofeedback research.

Progress in this field requires new research directions. For example, advances in machine-learning permit us to move beyond the traditional fragmentation of brain activity into five rigid bandwidths (i.e. alpha, beta, gamma, delta, theta) and to seek more nuanced and accurate neural signatures of target mental states (Meir-Hasson et al., 2014). Researchers further leverage simultaneous recording of EEG and functional MRI in an attempt to circumvent the controversies of EEG-neurofeedback by training complex electrical brain patterns that correlate with signal changes in functional MRI (Keynan et al., 2016). The jury is still out on whether or not these procedures can reliably entrain brain and behaviour, but that is a testable hypothesis—an empirical question—answerable by responsible experimentation.

In sum, ideological and financial interests dominate the field of EEG-neurofeedback. This state of affairs skews common perception with a biased literature that rests on weakly designed experiments and a systematic high-risk of false positives. Proponents of EEG-neurofeedback may continue to ‘fix neural pathways’ and ‘regulate emotional networks’ while optimistic start-ups look for better ways to ‘train the brain’; alas, the legitimacy of EEG-neurofeedback is losing buoyancy in a pool quickly filling with defiant data. If the community of neurofeedback practitioners continues to uncritically sing the praises of their intervention instead of providing solid scientific evidence to support their claims, the field, if it hasn’t already, risks reducing to a boondoggle.

Funding

Robert T. Thibault acknowledges an Alexander Graham Bell Canada Graduate Scholarship and Michael Smith Foreign Study Supplement from the Natural Sciences and Engineering Research Council of Canada (NSERC), as well as funding from the Bial Foundation. Amir Raz acknowledges funding from the NSERC Discovery grant, the Canadian Institutes of Health Research, and the Bial Foundation. We declare no conflicts of interest.