Using More Ecological Paradigms to Investigate Working Memory: Strengths, Limitations and Recommendations

Working memory (WM) is essential to daily-life activities as it allows maintaining information in the short-term while processing concurrent information (Baddeley and Hitch, 1974). For example, one must maintain which ingredient is already in the plate while following a recipe. WM is a complex cognitive function involving multiple processes (e.g., encoding, maintenance, retrieval processes). The present paper focuses on the utility of virtual reality (VR) in investigating maintenance in WM, but the relevance of VR studies also applies to other WM-related mechanisms.

Recent models proposed an attention-based mechanism supporting maintenance of domain-general information: attentional refreshing (or refreshing; Camos et al., 2009; Camos and Barrouillet, 2014; Camos, 2017). Refreshing is described as a brief thought to an information that is no longer perceptually present (Johnson, 1992) and received growing attention in both WM and episodic memory (EM). The WM field provides convincing evidence of an involvement of refreshing in maintenance of visual, spatial, verbal information, as well as in the binding between these information (Hudjetz and Oberauer, 2007; Camos et al., 2009; Vergauwe et al., 2009, 2010, 2012). Studies using delayed recall suggest that memory performance depends on the time available for refreshing (Camos and Portrat, 2015; Souza and Oberauer, 2017; Jarjat et al., 2018) and that refreshing plays a role in construction of episodic traces (Johnson et al., 2002; Loaiza and McCabe, 2013). So far, studies used very simple to-be-remembered material such as letters or spatial locations (e.g., Camos et al., 2009; Vergauwe et al., 2009, 2010; Camos and Portrat, 2015). As refreshing is involved in maintenance of domain-general information and construction of EM, it should play a significant part in maintenance and long-term retention of rich and complex information. Because WM is central in daily-life activities, future research should design more ecological experiments to better understand the role of WM and refreshing in naturalistic situations.

VR seems to be a useful tool to investigate memory functioning in daily-life-like environments. VR allows creating naturalistic situation and increasing their ecological validity as compared to classical experimental or neuropsychological tests (Plancher and Piolino, 2017). Ecological validity refers to the extent to which experimental conditions are similar to a real-world setting (Bohil et al., 2011). Accordingly, a VR experience can provide complex and rich information involving multiple senses (vision, audition, proprioception, etc.) and spatiotemporal features. VR also enables interaction with the environment, for example by controlling displacements, which increase the feeling of immersion in this environment (Mestre and Fuchs, 2006). Besides improving ecological validity, controlled environments can be created to assess multiple features of memory traces—the content of the memory trace (what) and its spatial and temporal location (where and when)—as well as the binding between these features (Plancher et al., 2010). VR is thus a good compromise between memory assessment of daily-life-like experience and experimental control.

While VR was extensively used to better understand EM (Plancher and Piolino, 2017; La Corte et al., 2019), and executive functions (Negut, 2014; Neguţ et al., 2016), only few studies investigated WM mechanisms with this method. Meilinger et al. (2008) investigated the involvement of WM in a wayfinding task. In comparison to a control condition without concurrent processing, a concurrent task (e.g., indicating the spatial location of a sound) negatively affected wayfinding of the routes previously seen. Both verbal and spatial concurrent tasks (continuously repeating a syllable sequence or tapping a spatial sequence, respectively), impaired memory performance for the landmark location and only the spatial concurrent task impaired memory performance for the route (Gras et al., 2013). More recently, Plancher et al. (2018) investigated the role of WM in construction of EM traces using a VR paradigm. While driving into a virtual town, participants had to memorize the encountered scenes as detailed as possible including the elements constituting the scene (what), the spatial location (where) and the temporal context of the scene (when). The recall of the spatial or temporal context associated to each element provided a binding score. As compared to a condition without concurrent processing, a verbal concurrent task (memorizing the number of garbage containers) only impaired memory performance of what information and a visuospatial concurrent task (memorizing the spatial position of containers) impaired memory performance of what, when, and what-where-when binding information. These results suggest that construction of memory traces rely on verbal and visuospatial maintenance mechanisms and were interpreted as reflecting an involvement of both phonological loop (i.e., verbal-specific WM mechanism, Baddeley and Hitch, 1974) and refreshing in the construction of what and an involvement of refreshing in the construction of when and binding components of EM traces.

Typically, the involvement of refreshing in maintenance is investigated using complex span tasks where to-be-processed items are interleaved in-between each to-be-memorized information (Barrouillet et al., 2004, 2007). Following the assumption that maintenance and processing compete for one limited resource (i.e., attention), increasing the amount of attentional resources required by the processing task leave less attentional resources available for attentional maintenance (i.e., refreshing, Barrouillet et al., 2007). Attentional sharing between maintenance and processing is proposed to rely on time: when time is occupied by a processing task, attentional maintenance cannot take place, and vice versa (Barrouillet et al., 2007). Varying the amount of time required for processing a concurrent task (i.e., its cognitive load) results in manipulating the amount of time available for refreshing. Poorer WM performance under higher cognitive load is taken as evidence of an involvement of refreshing in WM maintenance (Barrouillet et al., 2004, 2007; Vergauwe et al., 2009, 2010).

To understand the involvement of refreshing in maintenance of rich and complex information, we suggest adapting complex span tasks to VR paradigms. The task concurrent to maintenance should be distinct from the memorization task and allow to measure response times. Thereby, it will be possible to manipulate the cognitive load of the concurrent task and investigate whether and how refreshing is involved in maintenance of the different features of a complex memory trace (what, when, where) and the binding of these features. A passive exploration of the environment would allow determining the time-course of the task and controlling temporal parameters and require no control nor planning for traveling. Motor and planning actions required by active traveling can constitute an attentional cost (Plancher et al., 2013) and have an uncontrolled detrimental effect on WM performance. However, a passive exploration result in a simple video experience. Active navigation seems more useful to enrich the EM trace (Plancher et al., 2012, 2013; Sauzéon et al., 2012; Jebara et al., 2014). Immersion and real time interaction with the environment are necessary for self-experience and bodily representation and modulate the sense of presence in the present that is central in daily-life experience (Nash et al., 2000; Makowski et al., 2017). Self-experience and bodily representation reinforce EM performance (Bergouignan et al., 2014; Repetto et al., 2016; Tuena et al., 2017, 2019; Blanke et al., 2018) and might also influence maintenance in WM. For a more immersive experience and a better understanding of the involvement of refreshing in daily-life situations, future studies should systematically use an active condition. Contrasting different levels of immersion (from computer screens to head-mounted displays or cave automatic virtual environments) and interaction (from joystick to motion capture) with a passive condition would enable determining the minimum conditions for studying WM in an ecological context and explore how embodiment impacts refreshing.

To study WM in conditions as close as possible from real-life using VR, we suggest designing a virtual environment where the participant is freely exploring and encounter events of different nature (e.g., visual, auditory, proprioceptive, spatial, or any combination). To enhance ecological validity, events should occur at a non-isochronous pace. Temporal parameters related to to-be-memorized and to-be-processed events (number of events, presentation duration, inter-stimuli intervals) should be fixed and participants' behavior (e.g., response times) should be measured. Studies of interdependence between WM processes and other cognitive functions will also benefit from VR paradigms. Long-term memory and semantic representation seem to contribute to refreshing and WM (Loaiza et al., 2015; Loaiza and Camos, 2018). Refreshing is involved in construction of EM traces (Johnson et al., 2002; Loaiza and McCabe, 2013) and might play a part in prospective memory (Marsh and Hicks, 1998). Carefully timed VR experiments manipulating WM parameters will contribute to a detailed insight of these cognitive processes and their relationship with WM.

Developing virtual-reality-based paradigms to investigate WM would be useful to identify the neural basis of WM mechanisms in ecological situations. To date, as for behavioral studies, neurophysiological studies of maintenance in WM (Vogel and Machizawa, 2004; Guimond et al., 2011; Lefebvre et al., 2013; Grimault et al., 2014) and refreshing have used very simple stimuli (Johnson et al., 2005, 2015). To our knowledge, no study combined WM daily-life-like paradigms and neurophysiological measures. Yet, VR paradigms can be combined easily with neurophysiological recording like eye-tracking (e.g., Whitmire et al., 2016) or electrodermal and cardiac responses (e.g., Parsons et al., 2011; Armougum et al., 2019). Electroencephalography (e.g., Jaiswal et al., 2010; Bohil et al., 2011) and fMRI (e.g., Kalpouzos et al., 2010) recordings are possible during exposure in experiments with limited movements. Neurophysiological measures during retrieval depending on WM manipulation during exposure (e.g., the amount of time available for refreshing) could provide a further insight on the implication of WM processes in episodic construction.

VR is a powerful experimental tool that allow creating multimodal and naturalistic environments to assess memory for rich and complex information while keeping a strong experimental control. VR will allow testing theoretical assumptions with enhanced ecological validity and interactive fidelity and explore new hypothesis as whether and how traveling affects WM. In addition, VR allows us to examine some of the basic properties of the situated and embodied approach through exact control of methodological factors and go further into the understanding of the role of presence and consciousness in memory. Yet, using VR paradigms in WM studies require to compromise between a strong control of temporal parameters and immersion through active exploring. We recommend fixing temporal parameters of the experiment and to measure behavior, especially response times, as precisely as possible. Future studies will need to determine the minimum immersion and interaction conditions to explore WM with VR to limit the attentional cost of active navigation and facilitate the combined use of VR and neurophysiological measurements. Moreover, some other technical aspects of VR such as the risk of cybersickness, photorealistic level of environment, type of interaction and way of navigation, level and mode of the embodiment need further investigations to test their impact on WM as suggested by some authors in the domain of memory studies (Smith, 2019).

Future studies should also assess metric properties of VR-based measurements of WM to ensure a good construct validity and reliability. Besides the good equivalence between cognitive performance and physiological responses in the virtual and the real world (Sorita et al., 2013; Armougum et al., 2019), construct validity of VR-based neuropsychological assessments seem suitable (Neguţ et al., 2015), comforting the feasibility of developing VR-based assessment of cognitive functions. Given that VR is becoming more accessible at low cost, future studies will be able to multiply on a larger number of subjects to obtain reliable and reproducible data.

WM is impaired in various populations such as healthy aging, age-related dementia (e.g., Baddeley et al., 1986; Huntley and Howard, 2010) or schizophrenia (e.g., Lee and Park, 2005). In some populations, WM deficit is proposed to be due to an impairment of refreshing (e.g., Hoareau et al., 2016; Fanuel et al., 2018 in healthy aging; Grillon et al., 2013 in schizophrenia). Developing naturalistic tools to investigate WM functioning seem very useful to characterize the WM deficits in these populations. Previous studies suggest that WM training involving multi-modal stimuli, demanding high cognitive engagement and targeting WM domain-general mechanisms are more likely to yield WM and general cognitive enhancement (Morrison and Chein, 2011). VR-based trainings of WM thus seem a promising approach for enhancing both WM and broader cognitive functions. It is a crucial point nowadays to examine the VR acceptability for fragile populations.

Author Contributions

All authors conceptualized the manuscript. LF wrote the original draft. GP and PP reviewed and edited its intellectual content.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.