Neuroimaging of learning and development: improving ecological validity

Abstract

Modern neuroscience research, including neuroimaging techniques such as functional magnetic resonance imaging (fMRI), has provided valuable insights that advanced our understanding of brain development and learning processes significantly. However, there is a lively discussion about whether and how these insights can be meaningful to the educational practice. One of the main challenges is the low ecological validity of neuroimaging studies, making it hard to translate neuroimaging findings to real-life learning situations. Here, we describe four approaches that increase the ecological validity of neuroimaging experiments: using more naturalistic stimuli and tasks, moving the research to more naturalistic settings by using portable neuroimaging devices, combining tightly controlled lab-based neuroimaging measurements with real-life variables and follow-up field studies, and including stakeholders from the practice at all stages of the research. We illustrate these approaches with examples and explain how these directions of research optimize the benefits of neuroimaging techniques to study learning and development. This paper provides a frontline overview of methodological approaches that can be used for future neuroimaging studies to increase their ecological validity and thereby their relevance and applicability to the learning practice.

1. Introduction

Modern neuroscience research, including non-invasive neuroimaging techniques such as functional magnetic resonance imaging (fMRI) or electro-encephalography (EEG), provides exciting new possibilities for investigating the living human brain. For example, cognitive neuroimaging research has yielded many important insights into the neurobiological basis of learning and development (e.g. Casey, Tottenham, Liston, & Durston, 2005; Crone & Elzinga, 2015). This has boosted interdisciplinary research endeavors exploring the educational relevance of these insights, by crossing boundaries between cognitive neuroscience, educational science and educational practice, giving rise to a new research field termed either educational neuroscience or mind, brain and education (Ansari, Coch, & De Smedt, 2011). Such endeavors are challenging, as applying insights gained using neuroimaging techniques is accompanied by several limitations and uncertainties (Schleim & Roiser, 2009; van Atteveldt, van Aalderen-Smeets, Jacobi, & Ruigrok, 2014). A major limitation is the low ecological validity of the performed experiments.

Two important factors compromising the ecological validity of neuroimaging studies are 1) the highly controlled stimuli and tasks used and 2) the artificial and isolated environment in which the studies take place (the setting). During a typical neuroimaging experiment, participants perform a cognitive task aimed at isolating one specific skill or cognitive process, typically consisting of highly controlled, simplified stimuli. Because of the noisy nature of neuroimaging signals, many repetitions of these simplistic stimuli are needed in the cognitive task. The setting of neuroimaging studies is far removed from a naturalistic learning situation. An MRI lab for example, is a sterile, artificial environment that can be intimidating especially for children. Participants lie on their back in a narrow scanner bore, with their heads fixated and are alone when they are being scanned. Because of the many repetitions of stimuli needed to obtain reliable signals, fMRI sessions are long and can be exhausting. This setting is incomparable to daily-life learning environments such as a classroom or the workplace where social dynamics and the learning situation are much more complex. This discrepancy introduces challenges in translating basic insights generated during neuroimaging lab experiments to real-life brain function (Kingstone, Smilek & Eastwood, 2008). Fortunately, recent advancements increasingly enable approaches to overcome the constraints that cause the low ecological validity (Matusz, Dikker, Huth & Perrodin, 2018).

The current paper focuses on how to overcome the challenges yielded by the artificial and highly controlled lab setting of standard neuroimaging research, by outlining methodological approaches to improve the ecological validity of stimuli, tasks and the measurement context (setting). Limitations and possibilities of neuroimaging research at a more technical or physiological level are discussed extensively elsewhere (Logothetis, 2008; Poldrack & Farah, 2015; Schleim & Roiser, 2009), and are included here only when they put constraints on ecological validity. There is currently a lively debate ongoing on the merits of educational neuroscience (Bowers 2016, Howard-Jones et al 2016, Gabrieli 2016). Here, we specifically address the challenges that arise from low ecological validity of neuroimaging studies rather than the challenges and added value of educational neuroscience in general. The working definition of educational neuroscience used in the current paper is: cognitive neuroimaging studies into learning and development, bringing indirectly and directly useful insights to improve learning and teaching in the practice.

In the following, we will begin with providing a brief overview of the most widely used neuroimaging techniques and their strengths and weaknesses. Next, we will illustrate what neuroimaging of learning and development can bring to the educational practice, either indirectly by informing teachers and influencing their beliefs and attitudes, or more directly by actually using a brain measure for educational purposes. It should be noted that some factors limiting ecological validity apply to laboratory experiments more generally, i.e., apply also to behavioral experiments rather than specifically to neuroimaging, such as the controlled stimulus and task designs to tap into isolated cognitive processes. However, there are several limitations that are either more extreme or unique to neuroimaging experiments. These more general versus specific limitations are explained in more detail in section 2.4 and in Figure 1.

Figure 1.

General versus more specific limitations in ecological validity in lab-based experiments

Next, we will discuss different approaches one can take to improve the ecological validity of neuroimaging, starting with ways to make stimuli and tasks more naturalistic in section 3, such as using videos or virtual reality inside the (f)MRI scanner or staging real-life social interactions during lab experiments. In section 4 we focus on the setting, by discussing developments in portable neuroimaging devices (e.g. EEG, or functional near-infrared spectroscopy or fNIRS), which enable measuring brain activity in real classrooms or other real-life learning settings. The third approach we discuss is to combine tightly controlled research in the neuroimaging lab with real-life variables (e.g., grades, risk behaviour), and with follow-up field studies (section 5). This approach exploits the technical strengths and data quality of e.g. fMRI, but still relating it to real-life learning or behavior within the same study. The final approach we discuss is to include stakeholders from the practice at all stages of the research. This is important to further improve the connection between neuroimaging studies and real-life learning, because teachers have the day-to-day experience with the chaotic and irregular teaching practice that researchers lack, and to ensure realistic expectations on what neuroimaging can and cannot offer as well as to prevent neuromyths (Howard-Jones, 2014).

2. Neuroimaging of learning and development: insights for education

Studying the human brain in neuroimaging labs has gained many neurobiological insights that changed the way we think about development and learning. Such insights can be useful to education either indirectly or more directly. An example of indirect use is that a better understanding of brain plasticity may influence teacher’s beliefs about their student’s learning potential (Dubinsky, Roehrig, & Varma, 2013). In section 2.2, we will highlight two influential examples of such more indirectly useful insights: adolescent brain development and the neurobiology of memory. Whether neuroimaging measures or insights can also be used more directly is a matter of debate (Bowers, 2016; Gabrieli, 2016; Howard-Jones et al., 2016). As mentioned above, the scope of the current paper is not to discuss the added value of educational neuroscience in general, but to illustrate how improving ecological validity of neuroimaging studies may strengthen its educational relevance. In section 2.3, we will illustrate one potentially promising example of the more direct use of neuroimaging: using a brain measure for its predictive value, also termed neuro-prediction. If adding a brain measure predicts a certain learning or intervention outcome better than behavioral tests alone, it can provide valuable information.

2.1. Basic overview neuroimaging techniques

The term neuroimaging is used for modern techniques that are able to measure signals from the brains of living (human) participants. A distinction can be made between anatomical and functional techniques. Anatomical techniques make static images of structural details of the brain; anatomical details such as volume or thickness of different tissues such as gray vs. white matter in the case of structural MRI, water diffusion orientation indicating myelinated axons and their integrity in the case of Diffusion Tensor imaging (DTI). Although insights from anatomical techniques can be relevant for educational issues, the link to learning and behavior is always indirect, as they only yield static images and no information about dynamics of brain activity. Functional techniques do capture such dynamics, by measuring time-series of brain activity while the participant is doing a task. This task is typically designed to mimic (one specific aspect of) daily-life behavior, but as we are restricted to controlled, simplistic stimuli and responses by button presses allowing only one or multiple fingers to move, often it is only loosely related to naturalistic behavior. The most commonly used functional neuroimaging techniques are fMRI and EEG. fMRI detects changes in blood oxygenation levels as a measure of neural activation, EEG measures electrical signals emitted by the brain. In Table 1, an overview of the practical set-up, nature of measurement, and the strengths and weaknesses of both techniques is provided. Other functional techniques are magnetoencephalography (MEG), which is very similar to EEG but instead measures magnetic fields produced by the electrical activity of the brain, and functional near-infrared spectroscopy (fNIRS) which is more similar to fMRI as it also detects changes in blood oxygen level, but using light in the near infrared spectrum instead of radio frequency magnetic pulses.

Table 1. Overview of the practical set-up, nature of the signal, advantages and disadvantages of the most widely used functional neuroimaging techniques fMRI and EEG.

	fMRI	EEG
Set-up: the participant…	Lies in tight magnet bore of MRI scanner Is alone, in MRI scanner in separate room Looks at screen through mirror/goggles Performs task using MR-safe button box	Wears a cap with electrodes, connected to a computer Is usually alone, in shielded room Sits behind computer Performs task using button box/keyboard
Signal: the technique measures…	Changes in oxygen level in blood vessels in the brain: assumed to be triggered by regional changes in (presynaptic) neural activity Task fMRI: relative change during experimental vs. control trials Resting-state fMRI: activity and connectivity patterns during rest	Changes in voltage: resulting from synchronized activity of parallel organized neurons, typically in cortex ERP: voltage changes time-locked to specific events (e.g. stimuli or responses) EEG: time-frequency changes over larger time periods (freq. bands have functional meaning)
Advantages	Whole-brain coverage, incl. subcortical High spatial resolution: mm range Longitudinal use possible	Time resolution: ms precision Equipment relatively mobile & cheap Longitudinal use possible Suitable for all ages
Disadvantages	Loud acoustic noise during scanning Low temporal resolution Machine is not mobile and expensive Can be intimidating and uncomfortable Indirect measure of neural activity High level of physiological noise, distortions Exclusion criteria (metal/ braces) and age restrictions	Lengthy and uncomfortable electrode preparation Low spatial resolution / localization uncertainty Only sensitive to specific neural activity (e.g. cortical) High level of physiological noise, eye movement and muscle artifacts

2.2. Indirectly influential insights from the neuroimaging lab

One of the major insights in brain development provided by neuroimaging is the protracted development of the adolescent brain. It has long been thought that the brain was mostly developed around adolescence. Although there were some indications of protracted development in certain brain regions in the late 1970’s (Huttenlocher, 1979), it wasn’t until the rise of MRI that adolescents’ brain development could be studied on a large scale in healthy, living adolescents. Moreover, as longitudinal use is possible, MRI enabled studying adolescents while they go through different stages of development, thereby revealing developmental trajectories rather than only cross-sectional age comparisons (Horga, Kaur, & Peterson, 2014). A couple of landmark studies in the late ’90 and early 2000’s have shown protracted development of certain brain regions during adolescence (Giedd et al., 1999; Lenroot et al., 2007; Paus et al., 1999; Sowell, Thompson, Holmes, Jernigan, & Toga, 1999; Sowell et al., 2004; Thompson et al., 2000). Some regions, such as temporal regions related to social cognition, show developmental changes in grey matter up to at least age 25 (Gogtay et al., 2004). White matter connections between brain areas show non-linear developmental increases in adolescence (Bava et al., 2010; Giorgio et al., 2010; Lebel & Beaulieu, 2011). The use of MRI techniques opened up the possibility of relating brain developmental to behavior. It has been shown that integrity of white matter pathways between the prefrontal cortex and striatum are positively related to increased patience in a delay discounting task (Peper et al., 2013; van den Bos, Rodriguez, Schweitzer, & McClure, 2014). The integrity of this pathway increases over adolescence and is negatively related to impulsivity in delay discounting behavior (van den Bos et al., 2014). Together, these studies show that behavioral changes in adolescence are related to developmental changes in neural architecture, and occur until a much later age than previously thought. This knowledge has important implications, especially for secondary and higher education, as brains of adolescents may work more differently from adults than we used to think. Over the last couple of years, insights about adolescent brain development has become common knowledge among parents and educators (Choudhury, McKinney, & Merten, 2012). A better understanding of protracted adolescent brain development may stimulate teachers to use certain teaching practices such as providing more guidance with planning abilities (Dekker & Jolles, 2015) or simply to be more patient and understanding (Hook & Farah, 2013).

It should be noted that it remains to be established to which degree brain development is a consequence of natural development and what the influence of experience is. This causal direction problem has consequences for, for instance, effectiveness of interventions. Some studies have aimed to disentangle the effects of natural development and experience. One example is a school cut-off design in which development in children who attend school is compared to children of the same age who did not yet attend school. Children who attended school showed higher improvement in executive function and the rate of improvement was positively correlated with increased activity in the right posterior parietal cortex, an area important for sustained attention (Brod, Bunge, & Shing, 2017), indicating that brain development is at least partly driven by experience.

Another domain in which neuroimaging research has significantly increased our understanding of learning processes is that of memory research. Research into the neurobiology of memory encompasses many different types of memory, such as working memory (i.e., a memory that is shortly kept in mind, such as when reiterating a phone number; Baddeley, 2003) and long-term memories (i.e., memories that are stored for a longer period to be retrieved again at a later point in time; Jeneson & Squire, 2012). With regard to working memory, understanding how we keep information in mind and assessing possible limits of working memory ability can be highly valuable for education (Prince & Gifford, 2016) because working memory ability is presumed to be malleable (Klingberg, 2010) and keeping track of information load can help teachers to rightly dose knowledge. However, while working memory is indeed often found to be trainable, generalization of this enhancement to other cognitive abilities is highly debated (Melby-Lervåg, Redick, & Hulme, 2016), and therefore also transfer to learning in the classroom is uncertain. With regard to long-term memory, neuroimaging research has added interesting new evidence supporting the role of prior knowledge in learning. Prior knowledge is known to strongly benefit new learning (Bartlett, 1932) and as such it has been suggested to facilitate mnemonic processing in the brain (van Kesteren, Ruiter, Fernandez, & Henson, 2012). The interplay between the Medial Temporal Lobe (MTL), with the hippocampus at its core, and the Medial Prefrontal Cortex (mPFC) is suggested to be crucial in this process. The mPFC is suggested to help integrate new memories that fit with prior knowledge directly into a prior knowledge network (or a schema), bypassing the hippocampus/MTL system, which is proposed to link to episodic details of memories (Teyler & DiScenna, 1985). This former, integrative process is proposed to yield more semantic, less detailed memories directly after learning. While episodically detailed memories might be more intense, these integrated memories are suggested to be more durable (Dudai, Karni, & Born, 2015). Moreover, they could strengthen an existing prior knowledge network such that future learning is also facilitated, which is highly important in educational settings. When understanding more about the interplay between these memory systems, teachers and students can tailor learning by optimally using their prior knowledge networks in order to either store more detailed or more integrated memories.

2.3. How to use lab-generated insights more directly: the example of neuro-prediction

Neuroimaging can be useful to the educational practice more directly if individual brain measures can be used to predict learning outcomes (Peters & Crone, 2017) or intervention effectiveness (Gabrieli, 2016). Several studies demonstrate that including a neural measure to behavioral tests, this improved the prediction of an educational or intervention outcome beyond using behavioral tests alone. For example, Hoeft and colleagues showed that adding fMRI and DTI measures to behavioral tests improved the prediction of how well dyslexic readers compensated for reading difficulty over a period of 2,5 years (Hoeft et al., 2011). This may help identifying which children would profit most from extra intervention. Another example of the predictive value of neuroimaging is a longitudinal DTI study linking white matter development and reading skill development (Yeatman, Dougherty, Ben-Shachar, & Wandell, 2012). This study showed that reading skill is predicted by how mature the relevant fiber tracts (the left arcuate and left inferior longitudinal fasciculus) are initially. Interestingly, the more mature the connections at the first measurement, the lower the reading scores. This study points out that when is the optimal timing to start learning to read may differ between children.

Studies that improve prediction of an intervention or educational outcome when a neuroimaging measure is added to behavioral tests are promising, but one extra step is still needed, similar to what has been proposed as much needed extra research to test the value of using neuroimaging markers in psychiatry (Mayberg, 2014). This step entails performing randomized controlled trials (RCT’s) to compare the effectiveness of intervention when the decision between intervention types is based on behavioral tests alone, compared to when the neuroimaging measure is additionally used for this decision.

2.4. General versus specific limitations to ecological validity (stimuli, task, setting)

The examples above highlight the potential educational relevance of neuroimaging research into learning and development. As mentioned before, actual use of such insights is limited by the low ecological validity of neuroimaging experiments. Most experimental cognitive research, including behavioral studies, has the limitation that stimuli and tasks are tightly controlled and therefore not naturalistic. However, several constraints are either more extreme for, or unique to neuroimaging studies.

When measuring signals from the brain there are extra confounding factors to take into account. One of the most pressing issues in neuroimaging is noise in the data due to head movement, eye or muscle movement, physiological measures like heart rate and breathing, and lapses in attention or fatigue due to lengthy experiments in confined spaces (Logothetis, 2008). These noise sources have implications for the stimulus and task design: they can be accounted for by adding many (preferably > 20 per condition) highly controlling experimental trials. Moreover, experimental trials should optimally be short (i.e., in the range of a few seconds) to really capture brain signals related to one specific process (Amaro & Barker, 2006; Ruiter, van Kesteren, & Fernandez, 2012). Stimuli are best presented visually (at least for fMRI because of auditory scanner noise) and should be kept simple and easy to process to extract one isolated cognitive process. For example, in a memory experiment it is better to show participants simple pictures than lengthy sentences because sentences require longer processing and eye movements, activating other neural processes. When accounting for all these factors, there is mostly only a very artificial and simplified version of real-life situations left. In addition, in fMRI studies, since it is a costly technique both financially and in time investment for participants and researchers, participants are often scanned one single time. It is currently largely unclear how variable neural measures are and how this variability relates to variability in behavior. Also, the results provided by neuroimaging are less straightforward to interpret. Hit rates or reaction times resulting from behavioral studies are relatively clear measures, but it is not as clear how the indirect hemodynamic changes measured with fMRI, or the voltage change measured at a limited number of scalp locations with EEG, should be interpreted.

In addition to the specific constraints to stimulus and task design, the setting of neuroimaging studies also poses extra challenges for ecological validity. For example, participants have to either lie in a tight fMRI-scanner with their heads fixated, or sit behind a computer in a shielded room with an EEG-cap on their heads, while moving as little as possible. The big MRI machine and narrow bore can be intimidating and in rare cases even lead to claustrophobic experiences. In developmental MRI, there are several approaches to minimize such risks and make children comfortable, such as practicing in a mock scanner (Raschle et al., 2009). In sum, the setting leads neuroimaging experiments to be very different from what happens in the educational practice, where free viewing and moving influence learning and attention, and social interactions are constantly in play. MEG and fNIRS have similar restrictions, although mobility of the equipment varies (Mehta & Parasaruman, 2013), for example, fNIRS even allows participants to walk around (Pinti et al, 2018). In Figure 1, an overview is provided of the challenges to ecological validity that apply to experimental cognitive research more generally, and which apply more severely or uniquely to neuroimaging.

3. Approach 1: Improving the ecological validity with more naturalistic stimuli and tasks

Fortunately, there are ways to circumvent or minimize the limiting factors outlined above. Here, we highlight a few of these approaches. For example, one can use stimuli that are often learned in relatively controlled educational settings, such as in vocabulary learning or other simple associations and investigate underlying neural processes that relate to language learning and forming long-term memories (Goswami, 2006). Another approach is to approximate educational stimuli in an imaging experiment (van Kesteren, Rijpkema, Ruiter, Morris, & Fernandez, 2014) or run an additional behavioral experiment in which more educationally relevant stimuli are used in a more educationally relevant setting (van Kesteren et al., 2018). This can then be compared to the behavioral results of a more strictly controlled neuroimaging study. When behavioral results are comparable for these experiments, a direct relationship between the neural processes and behavioral outcomes in the educationally relevant setup can be inferred.

Furthermore, more ecologically valid stimuli can be used and analyzed with novel, sophisticated analysis methods, such as video stimuli for memory processes (van Kesteren, Fernandez, Norris, & Hermans, 2010; Aly et al., 2018; Chen et al., 2016) or games for insight processes (Milivojevic, Vicente-Grabovetsky, & Doeller, 2015). Rather than strongly controlling and isolating stimulus trials and contrasting conditions that only differ in one respect, these analyses use network connectivity measures or inter-subject correlations during free viewing of natural stimuli (Hasson & Honey, 2012), providing the possibility to consider processes happening during longer stretches of time. This approach thus allows to ask different questions that are e.g. related to inter-subject variability or neural processes that require a more extended time period, and better resembles everyday learning than when simple stimuli are presented in isolation. In addition, new analysis techniques also include machine learning algorithms to decode brain activation patterns automatically with high accuracy (Poldrack & Farah, 2015). This offers interesting opportunities for understanding how the brain represents real-life information. For example, a recent study successfully decoded fMRI response patterns to tell which real-life sounds participants were listening to (Santoro et al., 2017). Developments in virtual reality (VR) technology for neuroimaging experiments also hold big promises for improving ecological validity of stimuli. As VR enables simulating naturalistic events, brain activity can be measured during learning situations that can closely resemble real-life situations, e.g. including social interactions. VR technology therefore combines a high degree of control with a high degree of ecological validity (Bohil, Alicea, & Biocca, 2011).

To use individual neural measures as a predictor (see section 2.3), it is crucial to have reliable neural measures on the individual level. However, the most used current neuroimaging techniques use averaged group data to show patterns of neural activation. Recent studies have started to investigate neural measures in individual participants thereby increasing specificity and detail of these measures. Since the signal-to-noise ratio in neuroimaging data is low, the same individual needs to be scanned multiple times to acquire sufficient data to characterize individual patterns of activation. It has been shown that scanning the same individual multiple times yields reliable data for this individual (Braga & Buckner, 2017; Gordon et al., 2017; Poldrack, 2017). Future work should further parse out how these individual neural patterns are related to behavior.

More naturalistic stimuli and social interactions are also of high importance for studying social processes. Human interaction in real life is complex and flexible. However, most research investigating social cognition has used static stimuli such as pictures of faces or descriptions of persons. These limitations were mainly due to technological limitations. Due to recent advances in neuroimaging technology and analysis it is now possible to test more complex social interactions in the MRI environment. One of these advances is using naturalistic videos to test which brain areas are selectively responsive to social stimuli (Iacoboni et al., 2004; Wagner, Kelley, Haxby, & Heatherton, 2016), as also described above. Other examples are real-time tracking of the updating of social reputation as a function of social interaction (Tavares et al., 2015) and staging real social interactions while being in the scanner, e.g. by having the participant in the scanner interact with persons in his or her social network (Fareri, Niznikiewicz, Lee, & Delgado, 2012), or generating a real social touch experience by a person present in the scanner room (Gazzola et al., 2012). Real social interactions can now be investigated by measuring neural activation of multiple participants simultaneously while they are interacting. This so called ‘hyperscanning’ has opened up new avenues of research and will provide more insight into the complex nature of social interaction (see Babiloni & Astolfi, 2014, for a review). Hyperscanning experiments can be situated in the lab, and developments in portable neuroimaging technology now also enables recording brain activity from multiple participants in a real-life learning setting such as a classroom (Bhattacharya, 2017), as will be showcased in section 4.

4. Approach 2: Improving the ecological validity of the setting by using portable devices

As mentioned before, a major limitation of neuroimaging techniques is their immobility. Different techniques differ in their degree of immobility, and EEG and fNIRS are relatively most mobile (Mehta & Parasuraman, 2013). This makes it possible to record EEG or fNIRS activity during real-life learning settings, such as university lectures. A recent study measured EEG while university students had to respond to visual target stimuli presented during real lectures (Ko, Komarov, Hairston, Jung, & Lin, 2017), and found a correlation between a student’s sustained attention (i.e., target detection performance) and their EEG spectra.

Although possible, the setup of regular (stationary) EEG equipment outside the lab is not straightforward. Recent developments in portable brain technology have started a new direction of increasing the ecological validity of neuroimaging, as portable equipment drastically facilitates taking the research out of the lab and into working classrooms and other real-life settings. Furthermore, portable technologies enable research in infants and young children, and in areas of the world where access to advanced research labs is limited. The main techniques in which portable variants are available are EEG and fNIRS, but recently, a mobile version of MEG has also been developed (Boto et al., 2018).

The increasing availability, quality and user-friendly nature of portable devices such as EEG improves the possibilities of measuring brain activity in real-life learning settings enormously. For example, it allows recording brain activity of multiple students in a classroom simultaneously (‘hyperscanning’ as described in section 3), while they engage in natural learning activities and social interactions. In a pioneering study, Dikker and colleagues simultaneously measured EEG activity from 12 high school students and their teacher, while they took part in different classroom activities (Dikker et al., 2017). The authors were able to establish a link between brain-to-brain synchrony and classroom engagement (Dikker et al., 2017), and in a later study, also with learning outcomes (Bevilacqua et al., 2018). Although the data quality of portable devices does not reach the same level as when using lab equipment, for example due to fewer electrodes and movement artifacts, this direction of portable neuroscience has interesting potential, especially when combined with lab-based neuroimaging to generate complementary high-quality data. Interesting future directions are investigating the role of interbrain synchrony during real-life joint attention or collaborative learning (Bhattacharya, 2017). In addition to EEG, portable fNIRS also opens many opportunities for studying brain function in freely moving participants. An advantage of fNIRS is that it is relatively insensitive to motion and muscular artifacts (Balardin et al., 2017), making this technique particularly suitable for research in infants and young children, and allows free movement of participants including walking (Pinti et al., 2018).

5. Approach 3: Linking lab-based neuroimaging to real-life behavior

5.1. Relating neuroimaging measures to separately measured real-life variables

One way to relate neural measures to real-life behavior is by measuring both separately and subsequently linking them. For example, links between the educational practice and brain processes can be made through the relationship between cognition-related brain activity and course grades (van Kesteren et al., 2014). The neural measures that link to real-life behavior such as school grades can vary in similarity to the behavior of interest and can range from neural activity during performance of highly controlled tasks, focused on one process, to more complex and ecologically valid tasks. An example of a highly controlled task is a coin flipping game in which participants were instructed to choose heads or tails, if the computer randomly selected the same side of the coin, participants won money (Braams et al, 2015). This game results in strong nucleus accumbens activation when winning compared to losing money. Nucleus accumbens activation on the task shows a positive correlation with alcohol consumption, a form of real-life risk-taking behavior. However, the real-life processes that give rise to risk-taking behavior are more complex than sensitivity to reward. A more ecologically valid task is the Balloon Analogue Risk Task (BART). In this task participants can press a button to inflate a balloon. For each puff of air they receive a small reward. Participants can choose to stop and receive the money they earned on that round, or the balloon pops and they lose the money on that round. Behavior on the BART shows relationships with real-life behavior such as alcohol use, smoking and drug use (Lejuez, Aklin, Zvolensky, & Pedulla, 2003). Groups of adolescents using alcohol, marijuana or both show differential recruitment of neural regions while playing the BART compared to adolescents who abstain from using alcohol or marijuana (Claus et al., 2017).

These studies show that current neuroimaging measures are related to real-life behavior. However, the links are often weak and it is unclear how much variance in real-life behavior is explained exactly by the neural measures. To improve the match between what we measure with neuroimaging and the real-life measure, we can either move the neuroimaging paradigms towards higher ecological validity (e.g. stimulus material as described in section 3, or tasks as described above), or we can improve the real-life measure. In many neuroimaging studies, real-life behavior is measured sparingly, by use of questionnaires or self-report behavior. A much more sensitive approach is the experience sampling method (ESM), a structured diary technique allowing for the systematic and repeated sampling of an individual’s mood, behaviour and experiences over time in relation to context (Csikszentmihalyi & Larson, 1987). ESM measures behavior in the real world without the biases associated with reflection and retrospection. These momentary experiences may indeed have different associations with neural patterns of activation, compared to retrospective reports. During scanner-based social rejection, activation in areas associated with negative affect and pain processing was associated with momentary experiences of social distress in daily life social interactions, whereas activation in areas involved in memory and self-referential processing was associated with retrospective reports of social distress (Eisenberger, Gable, & Lieberman, 2007). Using structured diary techniques, studies have also revealed longitudinal relationships between social experiences and brain activation. Spending more time with friends during adolescence was associated with reduced neural sensitivity to peer rejection two years later (Masten, Telzer, Fuligni, Lieberman, & Eisenberger, 2010); and low peer support assessed across two years prior to scanning was associated with increased neural sensitivity to peer conflict (Telzer, Fuligni, Lieberman, Miernicki, & Galván, 2014).

Another approach to a more detailed assessment of real-life behaviour is the analysis of social networks. Social network data are based on reports of groups of individuals about their mutual connections. This provides a better picture of the complexity of social groups than self-reported connections. A recent study combining structural MRI with social network analysis found that network size based on the social ties identified by a subject's social connections (indegree network), significantly correlated with a larger regional volume of the orbitofrontal cortex, dorsomedial prefrontal cortex and lingual gyrus. In contrast, no associations between brain structural volume and an individual’s self-reported ties (outdegree network) were found (Kwak, Joo, Youm, & Chey, 2018). Using logged friendship data from Facebook, another study found that connectivity in brain regions associated with pain and mentalizing during social exclusion was modulated by the density of individuals’ social networks (Schmälzle et al., 2017).

5.2. Using field studies to test real-life validity of hypotheses generated by lab research

The results of lab-based neuroimaging studies can be used to generate hypotheses regarding more practical issues, which can be subsequently tested using field research, e.g. by conducting RCT’s and intervention effect studies. To come back to the example of the insights into the protracted brain development during adolescence (section 2.2), we mentioned that this insight may have interesting implications for education. However, it is not possible to know what these implications are exactly, based on research in the artificial and highly controlled lab setting. As the brain regions that show protracted development are linked to executive functions and self-regulation, a working hypothesis could be that adolescents may need guidance in self-regulatory behavior to do well at school (i.e., resist distractions, plan school work ahead). This hypothesis can be tested in real life, by designing an intervention based on the lab research and investigating its effectiveness in a field RCT. Other interventions targeting children’s abilities to self-regulate their behavior and emotions at school have shown positive results (Eisenberg, Valiente, & Eggum, 2010), and these can be fine-tuned by neuroimaging insights.

Another example is that of memory consolidation and sleep. It has been shown, both behaviorally and using neuroimaging, that sleep is important for memory consolidation (McGaugh, 2000), a process by which memories are suggested to be strengthened to protect them from forgetting. More specifically, memories are suggested to become semanticized through consolidation (Dudai et al., 2015; Lewis & Durrant, 2011). This means they lose episodic details and integrate with related memories to form a factual knowledge network. However, also here, it is not straightforward what lab findings on the role of sleep in memory consolidation mean for real-life learning. As also raised by Sigman and colleagues (Sigman, Peña, Goldin, & Ribeiro, 2014), different questions about how to use these insights in the practice should be tested in field studies, such as whether introducing naps at school after learning something improves learning outcomes compared to using this time for extra study. This same research group recently conducted such a field study, demonstrating that post-class naps in school indeed enhance information retention (Cabral et al., 2018).

Testing the potential educational value of lab-based neuroimaging research by additional field studies is as yet uncommon, and described as ‘a nearly unexploited research frontier’ (Sigman et al., 2014). Potential reasons for the lack of integrated lab-field studies so far are the extra time investment required and the lack of best practices to integrate methodology and standards from both approaches (Howard-Jones, 2009). However, to test what lab-generated insights into learning and development really imply for the educational practice, future research should make more efforts to integrate lab and field research.

6. Approach 4: include stakeholders from the practice at all stages of the research

In addition to making stimuli, task and setting more naturalistic, the ecological validity of neuroimaging research on learning and development may further be improved by early and active inclusion of teachers and other stakeholders in the research. For example, involving teacher’s experience and perspectives when designing research may help creating more realistic research designs and selecting stimulus materials and contexts that more closely resemble real-life learning materials and circumstances. Another important aim of improving collaboration between researchers and teachers is to prevent misconceptions and unrealistic expectations (Howard-Jones, 2014), by improving translation accuracy (Shonkoff & Bales, 2011). The call for more systematic interaction between cognitive neuroscientists and stakeholders from the practice is by no means new. Examples of successful teacher-researcher collaborations are abundant (McCandliss, Kalchman & Bryant, 2003) and the dialogue between teachers and researchers has been put forward as one of the most important building blocks for educational neuroscience research (Ansari, Coch & De Smedt, 2011).

One way to increase interactions between teachers and researchers is including more neuroscience and research methodology in teacher training curricula (Ansari, Coch & De Smedt, 2011). As mentioned before, it can be very beneficial to involve teachers and other stakeholders also in the scientific research itself. A systematic approach to do this is the responsible research and innovation (RRI) framework (Owen, Macnaghten & Stilgoe, 2012), which aims to align research with values and needs of society. The RRI framework has different dimensions, such as active and early inclusion of diverse stakeholders during the entire research process, anticipation of possible future impacts, reflecting on the diverse underlying values and purposes and on whether a certain development or application is desirable, and being responsive by changing the research adaptively. Stilgoe and colleagues (2013) offer techniques to implement each of these dimensions into research. Implementing this framework to educational neuroscience could be a promising direction, as was recently also suggested for research on electrical brain stimulation for educational purposes (Schuijer, de Jong, Kupper, & van Atteveldt, 2017).

7. Conclusions

One of the main challenges causing the limited applicability of neuroimaging research on learning and development to the educational practice is the low ecological validity of lab settings, making it hard to translate neuroimaging findings to real-life learning situations. In this paper, we described four methodological approaches that increase ecological validity of neuroimaging experiments: using more naturalistic stimuli and tasks, doing neuroimaging research in more naturalistic settings using portable devices, combining tightly controlled lab-based neuroimaging with real-life variables and follow-up field studies, and including stakeholders from the practice at all stages of the research. These approaches can be taken as guidelines for future neuroimaging studies to improve their ecological validity and to maximize their educational value. Stepping away from a high level of control is becoming increasingly possible because of improved sensitivity and mobility of neuroimaging techniques, incremental insights in designs and tasks and advancements in data-analysis tools, to improve the connection between behaviors during neuroimaging and natural, real-life activities. In addition to leading to more meaningful results, this progress may also enable addressing a wider array of research questions to which cognitive neuroscience could add relevant insights. It has been argued that neuroscientific data apply mostly to low-level behaviors such as single cognitive constructs (Willingham & Loyd, 2007) but now, for example by measuring brain activity of groups of students using portable EEG devices, this scope may be widened to include more complex behaviors, such as at the classroom level.

Keypoints.

• Keypoint 1: A main challenge of applying neuroimaging research to the educational practice is the low ecological validity of lab-based neuroimaging research;
• Keypoint 2: The first approach to increase ecological validity of neuroimaging is by using more naturalistic stimuli and tasks;
• Keypoint 3: The second approach is to increase ecological validity of the setting, by using portable neuroimaging devices allowing measuring brain activity in real-life learning situations;
• Keypoint 4: The third approach to increase ecological validity of neuroimaging is to link lab-based neuroimaging measures to separately measured real-life behaviour;
• Keypoint 5: The fourth approach is involving stakeholders (teachers, students) in designing and interpreting neuroimaging research to increase the educational value of neuroimaging.