Understanding Natural Cognition in Everyday Settings: 3 Pressing Challenges

On July 2018, 156 people gathered together in Berlin to share and discuss work and ideas flourishing from one of the most exciting concepts about the mind: natural cognition (Makeig et al., 2009; Gramann et al., 2014). These researchers are looking forward to move away from good old-fashioned cognitive science (GOFACS) and transition to the next phase in the study of the biophysics of human experience. Thus, the 3rd International Mobile Brain/Body Imaging (MoBI) Conference was carried out at TU Berlin. Given each work's complexity, it would be futile to try to distinguish which or how many disciplines were represented during the conference. Talks ranged from novel software and hardware to novel experiments and analyses to clinical and artistic interventions. Thus evidencing the truly multidisciplinary scientific effort carried by the MoBI community: attendants' background ranged from physics, engineering, and neuroscience to psychology, arts and therapeutics. Furthermore, Scott Makeig's closing keynote talk provided an inspiring account on the development, current state, and future challenges of MoBI, from the point of view of the very man who has spearheaded some of the most relevant aspects of the movement (Makeig et al., 2009; Gramann et al., 2010, 2011, 2014; Gwin et al., 2011; Ojeda et al., 2014; Liao et al., 2015). The MoBI community is building a research program for the 21st century allowing the understanding of natural cognition in everyday settings. Notwithstanding relevant advancements reached during the first 10 years of its existence (Gramann et al., 2014; Ladouce et al., 2017), there still are important pitfalls to keep in mind as the field moves forward. Therefore, I would like to present the reader with what appears to me -after experiencing the meeting- three pressing short-term challenges faced by the MoBI community.

Challenge N° 1: systematically extending our laboratories

MoBI -by definition- implies extending our structured and controlled laboratory settings toward semi- or unstructured ones allowing cognitive acts to unfold naturally in all its complexity. However, there still is very little consensus on what would constitute the “standard MoBI experiment,” if there is such a thing. For some researchers the answer might lie in controlling—as carefully as possible—the type of interaction on which participants engage. Generating thus well-defined and identifiable experimental conditions embedded in more ecological settings than the laboratory (e.g., Malcolm et al., 2017; Pizzamiglio et al., 2017). These type of experiments are—to me—the epitome of laboratory extensions to the real world through translation of “classic” structured settings (and therefore “classic” effects) to the MoBI paradigm, hopefully leading toward new hypotheses—unreachable within the lab. In contrast, a more intuitive approach is generating open interactional setups, allowing for a virtually intractable number of degrees of freedom for each participant's cognitive acts (e.g., Herrera-Arcos et al., 2017). While it could be thought that unstructured settings might be the conceptual holy grail of the MoBI paradigm, it is important to clarify that semi-structured settings try to maintain the possibility of identifying experimental conditions, whilst no a priori patterns can be determined from unstructured settings. In contrast, benefits reached in terms of ecological validity are lost when decreasing degrees of freedom (Figure 1). Currently, no consensus exists on which approach should be taken and, in the long run, I think that it does not matter if consensus is ever reached. The real pitfall here is thinking that a specific position in the continuous between structured and unstructured settings is more important than others, leading to disqualifying, abandoning or ignoring progress at other points of the spectrum. These are complementary approaches that should be run in parallel as much as possible (Ladouce et al., 2017).

Figure 1.

The continuous between structured experimental designs and unstructured ones, the impact of design decisions on some dimensions such as ecological validity and cognitive degrees of freedom are depicted. MoBI experiments could be carefully designed in order to be escalable, allowing (i) the exploration of brain/body dynamics emerging from structured and controlled conditions on robust findings [e.g., N2/P3 complex (Folstein and Van Petten, 2008)], (ii) testing previous results in semi-structured complex yet fairly controlled situations [e.g. (Malcolm et al., 2017)], (iii) pushing new hypothesis into complex unstructured settings, and (iv) returning back to a more structured settings if needed. Structured experimental designs are characterized by increasing internal validity at the cost of ecological validity. Left column shows a typical structured experimental setting, where a micro-version of a complex task can be implemented. Such a structured task could range from basic attentional deployment paradigm to a fully-fledged goal-kicking/catching simulator. Furthermore, this experiment could be implemented in both one-person or hyperscanning setups. Multi-modal data acquisition is very possible and highly encouraged in structured settings. Semi-structured experimental designs are characterized by providing some control over experimental variables and more behavioral/cognitive degrees of freedom through allowing more real-life behaviors at reduced ecological validity. Middle column depicts a semi-structured interactional setting where some aspects of natural cognition are preserved. As in real-life, the behavioral goal will be to score (in the case of the kicker) and to catch the ball (in the case of the goalkeeper). Even though multi-modal data acquisition still is a complex task, given the semi-structured nature of the design (e.g., trials, experimenter-controlled timings, etc.), data analysis and interpretation might be relatively straightforward. Finally, unstructured experimental designs are (almost) real-life situations where brain/body datasets are collected. In these setups macro-cognitive states unfold where participants have large behavioral/cognitive degrees of freedom at the possible cost of internal validity and interpretability of results. Right column depicts a natural unstructured interactional setting where collective time-pressured engaged decisions are made, in this case the common goal is to score. Considering these three levels as a natural continuum provides grounds for advancing the field. EEG depictions courtesy of ANT Neuro (The Netherlands).

Personally, I would like to see escalable experimental designs (EED) becoming the standard in MoBI research, leaving completely unstructured setups as a goal for longer-term research programs or as relevant proof-of-concept for hardware/software/intervention development. EEDs are characterized by carefully designing highly structured experiments always having in mind the possibility to be later extended—on a step—wise fashion toward the real world- ultimately becoming unstructured, dynamic, and social real-world settings with solid data-driven grounds for hypothesis testing. EEDs are an epistemological stance, where reductionism is no longer a concern as the ultimate goal is laying grounds for novel hypotheses to be tested in the wild. Another relevant aspect of EEDs is that they can be thought of as emergents of a naturally distributed research program; systematically extending the laboratory toward the real world as well as connecting different research groups working the same designs at different points in the spectrum.

Challenge N°2: neurocentrism 2.0

In Stefan Debener's presentation, a clear goal was presented: developing Transparent MoBI (Debener et al., 2015; Bleichner and Debener, 2017; Debener, 2018). This is bio-behavioral measurements that do not obstruct—in any way—the agent's natural behavior. Debener's dream system would measure (neuro)physiological dynamics along with eye movements, body topology, spatial location, etc., without interfering in the agent's evolutionarily-given degrees of freedom for interaction. Unfortunately, this dream system is not yet available (Debener et al., 2015; Mihajlovic et al., 2015; Goverdovsky et al., 2016; Bleichner and Debener, 2017; Ladouce et al., 2017; Debener, 2018). However, obtrusive/bulky or not, current systems allow us to actually measure several of these signals. Within this context -although significant improvement from GOFACS- the fact that some of our work might only be interested in measuring and reporting brain dynamics in the wild might be misleading for the extended scientific community in the long run. Closing the gap between acquired mobile and laboratory signal quality is crucial at this stage of development (Mihajlovic et al., 2015; Bleichner and Debener, 2017). Thus focusing only on brain signals is highly appropriate within this context. However, acknowledging this focus as a methodological need and not as an epistemological decision does make a difference.

On the one hand, the MoBI paradigm rather explicitly positions the brain/body-in-the-world system as the new epistemological object for cognitive science (Parada and Rossi, 2018). On the other hand, neurocentrism has been so radically implanted in our mindsets (Clark, 1997, 2008) that we must consciously make efforts so it does not make its way into the MoBI community as well. In the current brain-dominated cultural landscape (Satel and Lilienfeld, 2013), we might—unknowingly—be creating a neurocentric standard for current and future MoBI experiments. Such a standard might be as difficult to deconstruct as the event-related potential (ERP) paradigm once was, where people ended up “acquiring ERP” directly from the brain. As if ERP waves were natural objects to be collected, not the mathematical outcome of lack of computers in the late 30's (Davis et al., 1939) and low computing power in the 60's (Galambos and Sheatz, 1962). Given neuroscientists' historic bias toward neurocentrism (Clark, 1997, 2008), we must be weary to not carry out this bias any further.

Personally, I think epistemologically defining cognition not only as Embodied but also as Embedded in its socio-cultural context, as well as Extended and Enacted onto the physical and social world is key (Parada and Rossi, 2018). Moreover, this 4E-cognition (Menary, 2010) approach should be accompanied with proper 4E data acquisition. Understanding natural cognition in everyday settings will inevitably induce different states in the brain/body system (Bronfenbrenner, 1977; Ladouce et al., 2017). These new states will arise from developmental, demographic, social, and life-style factors; ontogeny. Most of these data sources are usually considered as not relevant for GOFACS as we usually not even register them in our data acquisition protocols (Varela, 1996). Sometimes we even try to “control” some of these variables by providing our participants with some life-style-disrupting instructions such as “don't drink coffee before the experiment.” Some other times, large longitudinal (neuro)physiological datasets might be acquired from specific subsets of general population while only collecting basic demographic information (age, sex, etc.). I personally envision an integration between brain signals and other data sources from both micro- and macro-levels of the brain/body system, longitudinally acquired. This is, other physiological patterns (such as electromyography, electrogastrography, electrocardiography, hormonal profiles, among others; e.g., Brouwer et al., 2018 in this issue) along with demographical, lifestyle, and social information such as eating (McGarel et al., 2014; Ojha et al., 2015; Zuker, 2015) and sleeping habits (Cedernaes et al., 2015; Dashti et al., 2015; Irwin, 2015), current employment situation/satisfaction (Faragher et al., 2013; Benach et al., 2014), drug/substance usage (Volkow et al., 2016) to name a few (Mattson, 2015). After all, for better or worse MoBI is a “big data” project already. A good reason to fully accept this, can be found on how only focusing and testing isolated predictors led to a historic near-chance success rates of suicide prediction (Franklin et al., 2017). Importantly, combining hundreds of predictors leads to >80% accuracy using the predictive/prospective analytics framework including machine learning (Walsh et al., 2017). The correct implementation of such an integrated approach implies rejection of neurocentrism. Further, this is an opportunity for the unification of sociology, anthropology, and other non-brain biased social and health scientists into the MoBI paradigm.

Challenge N°3: data management, from storage to reporting

Analyzing and interpreting MoBI datasets might be as complex of a process as collecting them. As a community we should emphasize, promote, and encourage replication studies, re-analises of already published MoBI effects (e.g., adding a “replications” section at the conference), as well as collaborative data mining of existing datasets (e.g., adding a “hackaton” session at the conference). Furthermore, dataset sharing is excellent practice and will only make the MoBI community stronger (e.g., Brantley et al., 2018). As datasets increase in number and complexity, non-trivial practical challenges arise for an open-science MoBI: from simply saving data in a hard drive to how analyze, report, and interpret it.

First, current containerization initiatives (for details see Gorgolewski et al., 2016) have been developed outside the multimodal framework where MoBI operates. The fact that we might be able to still retrace most existing MoBI datasets, should motivate the community to make active efforts for standardization of containerization practices as a relevant step for facilitating data sharing and advancing the field forward.

Second, even if we could contain our data in similar formats, sharing these data will not be straightforward. There are legal issues that might directly obstruct and interfere with the open science framework. For example, in Chile there are no specific laws about research data management. Therefore the closest law must be applied, which is about patient medical records. This law declares the service provider as responsible for keeping the records confidential for at least 15 years and forbids unauthorized third-party usage of those records. In the USA, the NIH has recently redefined the meaning of “clinical trials” (Hudson et al., 2016) whilst the General Data Protection Regulation in the European Union started its regime in May 2018, with yet unknown impacts to data sharing.

Finally, when properly organized and legally distributed data is at hand we encounter analysis, interpretation and reporting decisions. Enough evidence has already been gathered about the dangers of small sample-based research and its correspondent p-value-based conclusions (Yarkoni, 2009). Large-sample studies might not be the norm in the near future for MoBI. Therefore, integrating some safe-net practices into the very core of MoBI studies might be a good idea. Such practices include: (i) refrain drawing conclusions from fixed p-value thresholds (i.e., frequentist fallacies, see Wagenmakers, 2007; Nieuwenhuis et al., 2011), (ii) avoiding scarcity of illustrations depicting data and results (usually biased only toward graphing positive results!). More and more journals, such as Frontiers, are offering a very large number of figures per publication, let's use them, and (iii) including confidence intervals in illustrations and result reports. Excellent materials on how to avoid these and other common data analysis pitfalls have been recently developed, for example see (Pernet et al., 2011, 2013; Ehinger and Dimigen, 2018; Wilcox and Rousselet, 2018) to highlight a few.

Concluding remarks

The last decade has seen the emergence of the MoBI paradigm and, after attending the 3rd International MoBI conference, we can be confident that the community stands strong toward its second decade. However, given that the field is still fledgling, here I have highlighted some pressing pitfalls. Avoiding those will mean a focus on maximizing replication both at the individual and group levels. This goal can be reached through the development of EEDs as well as implementing good practices (e.g., data sharing, specifying, and sharing parameters, exact procedures and instrumentation, etc.). Furthermore, taking into account that the brain/body system is embedded in and extended toward the world (Parada and Rossi, 2018), data acquisition will be more complex than we can imagine. When high-quality Transparent EEG is accomplished (Debener et al., 2015; Mihajlovic et al., 2015; Goverdovsky et al., 2016; Bleichner and Debener, 2017; Debener, 2018), (neuro)physiological recordings will be trivial and other data sources will be included as part of the standard MoBI setup. These data will truly help us model the unique as well as shared aspects of cognition and the biophysics of human experience.

Author contributions

The author confirms being the sole contributor of this work and has approved it for publication.

Conflict of interest statement

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.