Introduction: On the Topic of Cluster Failure

Methods and procedures for acquisition, analysis and interpretation of functional MRI (fMRI) have been investigated by numerous groups; for example Hanzhang et al., 2006, Lindquist 2008, Jezzard et al., 2015, and Silva 2015. As with any complex method there is concern with the quality of practice, and Eklund et al. (2016a), after reviewing and reanalyzing a large number of fMRI data sets, indicated that there were likely problems with previously reported fMRI findings that they referred to as “Cluster failure”. The original claim was that “These results question the validity of some 40,000 fMRI studies and may have a large impact on the interpretation of neuroimaging results”. This claim was later softened removing “40,000” and changing the wording to “weakly significant neuroimaging results” (Eklund et al., 2016b). Another claim was that “It is not feasible to redo 40,000 fMRI studies, and lamentable archiving and data‐sharing practices mean most could not be reanalyzed either”. This was also changed to “Due to lamentable archiving and data‐sharing practices, it is unlikely that the problematic analyses can be redone”. These claims prompted several groups to respond (Brown & Behrmann 2017, Kessler et al., 2016, 2017) with additional comments and details regarding fMRI. Also, the OHBM posted a formal response at its website along with comments from several experts including Eklund (https://www.ohbmbrainmappingblog.com/blog/keep-calm-and-scan-on). To help clarify Eklund's original report, one co‐author, Tom Nichols, posted additional information about the processing done and suggested that the potential number of papers affected was more like 3500, along with additional details (http://blogs.warwick.ac.uk/nichols/entry/bibliometrics_of_cluster/). The intent of this special section of the journal is to clarify Eklund's claims along with discussion of each claim (Friston) and to emphasize appropriate uses of fMRI especially for dealing with a consistent concern, motion correction (Satterthwaite et al).

1. BACKGROUND

Because of the various steps involved in fMRI analyses many groups use freely distributed and validated fMRI analysis software such as AFNI (Cox 1996), SPM (Friston 2007, Ashburner 2009), and FSL (Smith et al., 2004). Some groups perform preprocessing with in‐house software and finalize using one of these freely distributed software applications. Others rely solely on freely distributed applications for fMRI analyses. Due to differences in processing with these various approaches there is potential for differences in results. Even when using the same data, preprocessing methods and processing software there is the possibility of variability in results due to differences in user‐selectable options. These considerations led us to seek out experts in the field to point out issues and solutions for fMRI studies that apply for both research and clinical purposes.

In this special section Friston reviews the report by Eklund et al, and after processing the Eklund data he provided point‐by‐point clarifications. A well‐known problem with fMRI studies is motion. Additionally, there can be variability in results due to different motion correction methods. In this special section Satterthwaite et al., review motion problems in fMRI, with emphasis on its impact in connectivity studies. They suggest methods to help manage both spatial and signal problems when performing motion correction for fMRI studies.

There are many factors to consider when planning fMRI protocols including those for experimental design, acquisition, and analysis. The following table lists some of the factors that authors should consider when designing research protocols.

Experimental Design:

1. Variables

   1. Independent vs. Dependent
   2. Categorical vs. Continuous
   3. Common Variables

      1. Age
      2. Gender
      3. Race/ Ethnicity
      4. Handedness
      5. IQ
      6. Educational achievement
      7. Psychiatric diagnoses; rating scales

2. Clinical and Research Objectives

3. Design Factors

   1. Block Design (length)
   2. Event Related (efficiency, sensitivity)
   3. Mixed
   4. Resting State
   5. Between vs. Within subjects
   6. Randomization/ Counterbalancing
4. Contrasts

   1. Experimental vs. Control
   2. Parametric vs. Subtractive
5. Response Monitoring

   1. Electrical (EEG, EOG, EMG)

Acquisition:

• motion artifacts are greater in younger individuals (especially due to motion)

• duration of studies limited by toleration (test with simulator or other methods)

• sleeping for younger subjets (parents in room)

• task‐based or resting state fMRI based on objectives of study

• RF heating

• Nerve and cardiac effects due to gradients

• parametric vs. non‐parametric designs

• differences in SNR (environment, pulse sequences, etc.)

• differences in response variables (accuracy, response bias, reaction times, etc.)

• consistent brain coverage for group studies

• consistent phase encoding direction for group studies

• consistent scan geometry (within/between slice spacing)

• consistent slice direction (anterior to posterior, left/right, superior/inferior)

• differences in eddy current correction by manufacturers

• differences in how manufacturer performs supposedly identical imaging sequences

• field strength effects on BOLD

Analysis:

• single subject study

  1. analysis software differences
• group studies

  1. age range (children, youth, adult, elderly)
  2. gender mix
  3. sociological mix (perhaps based on country)
  4. physiological mix (various disorders, normal, controls)
  5. anatomical mix (body size, brain size, body type)
  6. analysis software differences
• Software

  1. Matlab
  2. E‐Prime
  3. LabVIEW

With this special section we provide suggestions and recommendations for those with fMRI experience and those beginning to using this powerful tool.

Introduction: On the Topic of Cluster Failure. Hum Brain Mapp. 2019;40:2015–2016. 10.1002/hbm.24575