The Mechanism for Allocating Limited Working Memory Resources in Multitasking

Visual Working Memory and Capacity Limitation

Working memory plays an important role in many complex cognitive activities such as thinking, reasoning, decision-making, and language comprehension. Working memory refers to a mechanism involving both the temporary storage and the manipulation of information. As an important approach toward understanding the mechanisms of working memory, visual working memory has attracted attention since the 1990s. Visual working memory plays an essential role in the perception and cognition of visual information and simultaneously maintains information about multiple attributes of a visual object (e.g., colors, shapes, texture, motion direction, and spatial position) in different dimensions [1–3].

Visual working memory is one of the essential functions in daily life. To accomplish a particular goal (e.g., driving a car to a specific destination), it is usually necessary to perform multiple visual tasks and handle various types of information simultaneously (e.g., checking traffic lights and signs and watching other cars’ movements). In this situation, the cognitive system for each task must prepare a necessary amount of memory. As the number of tasks increases, the total amount of memory necessary for all tasks increases. However, the capacity of working memory is limited. A series of human behavioral studies have shown that the capacity of visual working memory is only 3–4 items [4]. Therefore, under multitasking conditions, the total amount of resources needed for all component tasks often exceeds the currently available capacity. As a result, the performance of one or more component tasks is impaired. This phenomenon is called dual-task interference. To focus discussion more on the current context, dual-task interference is confined to cognitive processes involving working memory. To understand the neural mechanisms of visual working memory and its management, it is important to understand why dual-task interference happens and how available memory is allocated to each component task depending on its demand.

Models for Flexible Allocation of Memory Resources

Based on the results of behavioral and neuroimaging studies, two models have been proposed to understand the cause of dual-task interference, namely the slot-based model and the flexible resource model. The slot-based model supposes that each item is stored in a “memory slot” and that only three or four such slots are available at one time [4, 5]. When all slots are taken by one task, additional items for other tasks cannot be stored, causing dual-task interference. Luck and Vogel (1997) used a change detection task to explore the capacity of visual working memory, in which the participants were required to answer whether two successively presented objects were the same or not [4]. They found that the participants exhibited 100% correct performance when the number of items was <3 and that their performances became significantly worse when the number of items was >4. Changing the duration of stimulus presentation did not affect these percentages. These results support the slot-based model and suggest that working memory capacity does not affect information processing.

However, the slot-based model has been challenged by studies examining how accurately memorized items are recalled. Behavioral experiments have shown that the accuracy of behavioral performance gradually decreases even if the number of objects simultaneously stored is within the capacity limit [6–8]. In addition to the number, the features binding to the stored objects such as location and orientation also affect the behavioral performance [8]. Therefore, the simple slot-based model cannot fully explain such complex behavioral effects. Thus, the flexible resource model was proposed, in which memory resources are allocated for each item flexibly depending on the importance and complexity of the items or the number of simultaneously stored items. This model has been supported by recent behavioral and neurophysiological studies [9, 10].

Baddeley’s Model of Working Memory and the Central Executive

The mechanism for allocating a limited capacity of working memory remains to be solved. Among theoretical models of working memory, Baddeley’s 4-component model is well known [11]; this is composed of one master component and three slave components including the phonological loop, the visuospatial sketchpad, and the episodic buffer. Specifically, the phonological loop stores speech-based information for speech perception and language comprehension; the visuospatial sketchpad stores information for visuospatial processing and other information that cannot be processed by language; and the episodic buffer stores integrated episodes or chunks and not only acts as a buffer between the phonological loop and the visuospatial sketchpad but also links working memory with perception and long-term memory. The master component, namely the central executive, coordinates and integrates the operations of the three slave components to achieve the current goal. The central executive is thought to be a system for allocating limited memory resources to each of the slave components depending on their demands. For example, if visuospatial processing becomes more demanding than language processing, the central executive allocates more memory to visuospatial processing. Accordingly, an appropriate control process or strategy would eventually be selected to accomplish the current goal. The functions of the central executive can be best understood using the dual-task paradigm. In a dual-task condition in which the participant is required to remember the locations and physical features of the objects simultaneously, spatial and visual working memory processes can function either independently or interactively depending on different contexts within a limited memory capacity [12]. Therefore, a system allocating memory resources to each of two processes, which corresponds to the central executive, is inevitable.

The Overlap Hypothesis and Possible Mechanism of Dual-task Interference

As Baddeley suggested, the dual-task paradigm has been thought to be an appropriate and effective method to study the mechanism for allocating working memory resources based on the demand. The memory demand for performing each task is not always fixed but changes in a flexible manner depending on the circumstances and the memory demands of other tasks. Therefore, the flexible resource model would be appropriate to explain how limited working memory resources are allocated to each task in the dual-task condition. To explain the mechanism of the flexible resource model, an “overlap hypothesis” (Fig. 1) has been proposed [13]. When two working memory tasks (A and B) are performed independently (Fig. 1A, B), electrophysiological studies show that each neuron in the prefrontal cortex, for example, usually exhibits specific stimulus or action selectivity, or exhibits a specific temporal pattern of activity, which is referred to as task-specific or task-related activity. Since each task evokes specific groups of task-related activities, the different populations of neurons would participate in the performance of each task, although some groups of neurons would participate in both tasks (shared group). When both tasks are performed simultaneously (dual-task condition, Fig. 1C), each task not only recruits its own population of neurons (task A or B specific population, Fig. 1D, E) but also tries to recruit as many neurons as possible from the shared group (Fig. 1F) by competition with the other task. This hypothesis assumes that the strength of the interference effect depends on the “functional brain distance” between the regions each of which participates in each task. If the functional distance is short for tasks A and B, both tasks A and B would have a greater degree of common functions for performing both tasks and hence have a larger population of shared group neurons. Therefore, in dual-task conditions, a strong competition would occur between two tasks to recruit as many neurons as possible from the shared group.

Fig. 1.

An example of the dual-task and an illustration of the overlap hypothesis. A A digit recall task (non-spatial visual working memory task), in which the participant has to remember the numbers during the delay period. B A visuospatial task (spatial working memory task), in which the participant has to remember the location of a visual cue during the delay period. C The dual-task condition, in which the participant performs both tasks A and B simultaneously. D, E Schematic activation area of the prefrontal cortex in the single-task condition. Dark green and magenta indicate activation areas during the performance of task A and task B, respectively. Note that each task activates not only its own cortical region but also a region shared with another task. F Schematic activation area of the prefrontal cortex in the dual-task condition. Since each task tries to recruit as much area as possible from the shared region, competition occurs between the two tasks, causing dual-task interference.

This hypothesis is supported by several imaging studies. Early studies using positron emission tomography (PET) technology indicated that while a dual-task involving both auditory and visual working memory does not activate any dual-task specific region, independent performance of each task activates largely overlapping areas in the prefrontal, parietal, and cingulate cortex [14, 15]. In a human functional magnetic resonance imaging (fMRI) study using a dual-task (auditory comprehension and visual rotation), activation in the temporal and parietal cortices during the dual-task was substantially less than the sum of the activation when the two tasks were performed independently, indicating the presence of an interference effect in the dual-task condition [16]. The degree of overlap of activated brain areas in single-task conditions correlated with the degree of the decline in dual-task performance. In neurophysiological studies using monkeys, since the dorsolateral prefrontal cortex is thought to be responsible for dual-task interference, Watanabe and Funahashi recorded prefrontal activity while the monkeys performed spatial attention and memory tasks in the dual-task condition [17]. They found that the ability of prefrontal neurons to represent task-relevant information decreased to a degree proportional to the increased demand of the counterpart task. They further proposed that the dual-task interference is caused by simultaneous and overloaded recruitment of the common neural population by the two tasks simultaneously. Thus, the overlap hypothesis considers that the memory resource for task A corresponds to a whole population of neurons that exhibit task-related activity during the performance of task A. Since the neural population for task A overlaps the population for task B, simultaneous performance of both tasks causes competition between tasks to recruit as many neurons as possible for each task, which causes normal performance for one task (winner) and impaired performance for the other task (loser), or impaired performance of both tasks, because the size of the shared neuron group is limited. Thus, the overlap hypothesis would explain how the available memory resources are flexibly and adaptively allocated to each component task in multitasking conditions and why dual-task interference occurs.

Conclusion and Perspective

Research on the neural mechanisms of dual-task interference has revealed that the memory resource corresponds to the computational capacity of a neural population associated with a particular information process and that dual-task interference is caused by simultaneous recruitment of the common neural population by concurrently performing two tasks. By a competition between two tasks to recruit as many neural populations as possible, a winner and a loser are determined. However, it is not yet clear how a winner and a loser are determined and what factor determines a winner. How much memory is allocated to each of two tasks is affected by not only the strength of memory demand of each task but also the order of the task priority or the significance of the task, or the order of task performance. It is not known which factor is most effective for determining the winner. To clarify these unsolved problems, further studies are needed, probably in non-human primates, because a variety of neuroscience methods including invasive approaches (e.g., extracellular recording, chemical/electrical stimulation, and viral-based neural tracing) are available in animal studies to understand the mechanisms of the management of working memory [18, 19].

In addition, dual-task interference can occur not only during working memory but also during other phases of perception and behavioral execution. It is not yet clear which stage of the cognitive process is more affected by dual-task interference, and how the similarity between tasks affects the interference. Using extracellular recording to reveal function-related activity (e.g. sensory-related, memory-related, and action-related) and compare the activity features between single-task and dual-task conditions, and between different pairs of dual-tasks (e.g. a pair of visual-related tasks and a pair of visual-auditory tasks), it is possible to address these questions.

On the other hand, the phenomenon that neurons exhibiting complex response profiles to different tasks can also be interpreted as “mixed selectivity” [20]. Mixed selectivity neurons encode distributed information about all task-relevant aspects. It is not clear how many kinds of information each or a population of mixed selective neurons can encode and whether the population activity of mixed selective neurons recorded during dual-task performance can decode the information necessary to perform each component task. Therefore, the capacity of information that mixed selectivity neurons can encode is not infinite but is also limited. In that sense, the concept of the overlap hypothesis seems to be similar to the concept of mixed selectivity. However, the theory of mixed selectivity focuses more on the neural dynamics in coding different features, while the overlap hypothesis talks more about neuron populations and their brain regions. Even so, there is potential that these two theories inspire each other and develop together.

Lastly, understanding how the nervous system performs memory management efficiently and in a coordinated manner to operate multiple cognitive processes may contribute to the development of a new generation of artificial intelligence (AI) [21]. To perform multiple tasks simultaneously without any disturbance from any task, AI systems must install appropriate memory management mechanisms. Understanding how a winning task is determined, what factor determines a winning task, and how the task priority, the task significance, or the order of the task performance affect which task becomes a winner in behavioral and neurophysiological studies would provide important information to avoid interference effects in multitasking conditions and contribute significantly to develop a new generation of AI systems.

Conflict of interest

The authors declare no conflict of interest.