The functional dissociation of posterior parietal regions during multimodal memory formation

Abstract

The incidental acquisition of multimodal associations is a key memory function for everyday life. While the posterior parietal cortex has been frequently shown to be involved for these memory functions, ventral and dorsal regions revealed differences in their functional recruitment and the precise difference in multimodal memory processing with respect to the associative process has not been differentiated. Using an incidental multimodal learning task, we isolated the associative process during multimodal learning and recollection. The result of the present functional magnetic resonance imaging (fMRI) study demonstrated that during both learning and recollection a clear functional differentiation between ventral and dorsal posterior parietal regions was found and can be related directly to the associative process. The recruitment of a ventral region, the angular gyrus, was specific for learning and recollection of multimodal associations. In contrast, a dorsal region, the superior parietal lobule, could be attributed to memory guided attentional processing. Independent of the memory stage, we assumed a general role for the angular gyrus in the generation of associative representations and updating of fixed association, episodic memory.

Using an incidental multimodal learning task the isolated the associative process during multimodal learning and recollection. The result of the present fMRI study demonstrated that during both learning and recollection a clear functional differentiation between ventral and dorsal posterior parietal regions was found and can be related directly to the associative process. The recruitment of a ventral region, the angular gyrus, was specific for learning and recollection of multimodal associations.

1. INTRODUCTION

The extraction and integration of contextual details and the formation of stable associations between information from different modalities is a critical human capacity to adapt to natural environmental dynamics. The basis for the formation of associative memory depends on the formation of memory for the individual stimulus but also on the emergence of an associative memory between different stimuli. Both memory traces can emerge voluntarily by the instruction to learn something or incidentally over time as a function of repetition. Both forms of learning result in explicit memory that allows fast and flexible behavior and the integration of information of meaningful associative representations (Chang et al., 2020; Hasson et al., 2015; Theeuwes, 2018, 2019). In particular, for multimodal stimuli, strong associative memory resulted in a more holistic processing for the different audio‐visual combinations and the flexible establishment of memory schemata (Tse et al., 2007). The aim of the present study is the evaluation of parietal contributions to the associative process during incidental acquisition of multimodal associations.

For associative memory processing, mainly the medial temporal structures (MTLs), including the hippocampus, are assumed to be involved in the acquisition and integration of learned and inferred details into a coherent representation (Eichenbaum et al., 2007; Eichenbaum, 2017; Ranganath, 2010; Squire et al., 2015). As a critical convergence zone among different extended large‐scale memory networks (Ritchey et al., 2015, 2020; Rugg & Vilberg, 2013), associative memory not only recruits hippocampal structures, but also involves connected posterior medial temporal and parietal regions (Gilmore et al., 2015; Kim, 2018; Ranganath & Ritchey, 2012; Ritchey et al., 2014, 2020; Spaniol et al., 2009).

Different areas like the parahippocampus, precuneus, angular gyrus (AnG), posterior cingulate, and medial prefrontal cortex (MPFC) operate at complementary timescales and are modulated by different memory functions (Aly et al., 2018; Cooper & Ritchey, 2019; Hasson et al., 2015; Ritchey et al., 2014, 2020). While MTL structures and the ventral MPFC are involved in early episodic memory processing, the AnG, precuneus, and MPFC are particularly involved in the formation and sustained maintenance of integrated, multimodal event representations (Brodt et al., 2018; Ritchey et al., 2020; Rugg & King, 2018). Interestingly, while transient hippocampal activation was related to the initial formation of a memory representation, sustained AnG activations were found during the maintenance of recollected representations (Ranganath et al., 2005; Vilberg & Rugg, 2012). In particular, the AnG was related to the formation and maintenance of an integrated, content‐specific episodic representation generalized across visual and auditory modalities (Baldassano et al., 2017; Branzi et al., 2020) with connections toward feature‐specific representations weighted by precision and confidence (Ritchey et al., 2020; Tibon et al., 2019; Yazar et al., 2014, 2017). Furthermore, greater functional involvement of the AnG was found during strong rather than weak associations (Bar et al., 2008), or during retrieval of contextual associations of a particular stimuli (Fornito et al., 2012), while being particularly sensitive to the frequencies of event occurrences (D'Acremont et al., 2013). Thus, AnG has been frequently related to the reinstatement and updating of contextual schemata and thematically relevant associations of episodic events (Humphreys et al., 2021; van der Linden et al., 2017; Ramanan et al., 2018; Rugg & King, 2018; Wagner et al., 2015), which tracks and represents contextual‐relevant details from memory during both encoding and retrieval (Baldassano et al., 2017; Favila et al., 2018; Kuhl & Chun, 2014; Lee et al., 2017). As a critical component of a context‐related, relational processing network (Binder et al., 2009; Bonnici et al., 2016; Seghier, 2013; Shimamura, 2011), the recruitment of the AnG was found to be particularly driven by past experiences or contextual factors during memory‐dependent processing (Ciaramelli et al., 2020; D'Acremont et al., 2013; Hasson et al., 2015; van der Linden et al., 2017; Ramanan et al., 2018; Ramanan & Bellana, 2019; Wagner et al., 2015). Noteworthy, the functional role of the AnG has not been exclusively related to multimodal information, but might be more sensitive to processing multimodal than unimodal information (Ben‐Zvi et al., 2015; Bonnici et al., 2016; Richter et al., 2016; Yazar et al., 2017). Thus, the functional involvement of the AnG is specifically sensitive to automatic, fixed association processing during associative memory dependent processing (Humphreys et al., 2021; Ramanan & Bellana, 2019; Rugg & King, 2018; Seghier, 2013; Shimamura, 2011).

The attention to memory (AtoM) theory outlines dissociable role for top‐down and bottom‐up processes in different parietal areas in relation to memory formation (Cabeza et al., 2008, 2011). Here, ventral parietal areas like the AnG mediates bottom–up attention processes initiated directly by memory formation, whereas dorsal regions of the PPC (dPPC) mediates top–down attention processes guided by retrieval goals.

In contrast to ventral regions of the posterior parietal cortex (vPPC), dorsal regions of the PPC (dPPC) associated with top‐down and bottom‐up attention do not overlap with regions associated with memory processing (Hutchinson et al., 2014; Sestieri et al., 2017). Influenced by variable association processing, dorsal PPC regions were frequently reported to regulate the controlled manipulation and rearrangement of perceptual‐based information in memory (Greenberg et al., 2010; Hoffman et al., 2018; Kim, 2020; Long & Kuhl, 2018; Sestieri et al., 2017; Spreng et al., 2010). Consistent with these findings, the dorsal regions of the PPC, such as the superior parietal cortex (SPL), are classified as a critical component of the dorsal attention network (DAN) specific for attentional re/‐orientation driven by internal goals and expectations (Cabeza et al., 2008; Corbetta & Shulman, 2002; D'Acremont et al., 2013; Spaniol et al., 2009). Within the ventral and dorsal PPC several sub‐regions could be differentiate according to distinct contributions toward recognition memory judgments by integrating data from resting‐state functional connectivity MRI and functional MRI (Nelson et al., 2010). In relation to attentional control in the context of memory based processing in particular a parietal region around the lateral bank of the intraparietal sulcus (IPS) was reported (Cabeza et al., 2008).

The relevance of the ventral and dorsal parietal system was demonstrated in many different memory related studies. However, for both systems it remains an open question whether the involvement in memory related processing is directly related to the associative process or is related mainly to stimulus memory. In the present study, we aim to test the hypothesis that the associative process between multimodal stimuli recruits both parietal regions differentially. To isolate the associative process independent of the memory for the single stimuli we developed a paradigm that controls for item memory but allows a systematic modulation of associative memory strength. The coordinates within the parietal systems of this associative process will be compared with known effects of different subsystems reported earlier. Further, it is unclear whether the functional dissociation of the parietal systems emerges as a function of memory strength during incidental learning situations and is transferred to a different recognition process.

During incidental learning, that is, without the intention to learn, memory is formed as a consequence of processing. Thus, this learning process is ideally suited to observe the interplay between both PPC systems and examine the functional differentiation as a consequence of memory formation.

In the present version of a paired‐associate learning task, volunteers incidentally acquired memory for new audiovisual stimulus pairs in two conditions with different associative strength but with identical item repetition. In the “fixed association”‐condition multimodal associations were constantly presented together in all repetitions assuming an increasing associative strength between stimuli. In the “variable association”‐condition the stimulus pairs are always rearranged to prevent the formation of a stable associative memory. Importantly, the amount of unimodal stimuli and related presentation was identical in both conditions, so that the memory for the individual stimuli can be regarded as identical. The direct comparison of both conditions allows the estimation of the specific memory formation and recollection for the associative process.

As a result of this differential memory encoding, we hypothesized that the vPPC activity reflect the emerging constant associations, while the dPPC is relevant for variable association processing of the variable stimulus pairs. Further, with learning the categories of both stimulus conditions should be encoded within the hippocampus over time. In the subsequent recollection test, we objectively measured the amount of explicit multimodal knowledge acquired under an incidental condition. In contrast to the learning phase, stimulus pairs were presented in a sequential order, that is, as two successive unimodal stimuli. This specific approach allowed to investigate whether the new acquired multimodal knowledge could also be accessed when only unimodal information were presented. As multimodal learning can benefit subsequent retrieval‐related performances during unimodal associative tasks (Shams & Seitz, 2008), we hypothesized a clear functional difference in PPC activation during recollection.

2. MATERIALS AND METHODS

2.1. Participants

Fifty‐six healthy individuals volunteered (age: 18–35 years, 35 females, 26 males) to participate in the functional magnetic resonance imaging (fMRI) experiment. Four participants were excluded due to medical reports or technical difficulties during fMRI measurements (n = 52 for all analyses). All participants had normal or corrected‐to‐normal vision. The study was approved by the ethics committee of the “Deutsche Gesellschaft für Psychologie” (DGPs) and present methods were conducted in accordance with the relevant ethical guidelines and regulations of the Declaration of Helsinki. All volunteers were instructed to sign an informed consent before participating in the fMRI experiment.

2.2. Stimuli

Stimuli were presented onto a white background screen controlled by a computer that ensured synchronization on a Siemens 3 Tesla Prisma MR scanner using the Presentation software package (Neurobehavioral Systems Inc., Berkeley, CA; www.neurobs.com/). For the functional MRI experiment, an LCD projector displayed the visual stimuli to the participants via a 45° mirror system to the inside of the scanner. Auditory stimuli were presented via MR‐compatible headphones. Due to scanner noise, the level of sound pressure was individually calibrated to a comfortable level for each participant. The responses of the participants were recorded using MRI‐compatible response devices (one device for each hand).

Stimuli consisted of 16 different images and 16 different sounds representing neutral object images and real life environmental sounds, respectively. Stimuli were selected from an internal database and had an unambiguous assignment to an animal or non‐animal category. Crucially, for each participant, auditory and visual stimuli were randomly assigned into novel bound audiovisual pairs which were not semantically related. For example, a picture of an owl and a sound of a car were simultaneously presented to the participants. Hence, this prevented a congruency effect within each stimulus pair (Parise & Spence, 2012) and ensured that participants acquired novel associations between arbitrary multimodal information during the experiment.

2.3. Experimental procedure

The employed experimental design consisted of two phases, the incidental paired‐associate learning phase and the explicit recollection phase, as illustrated in Figure 1.

FIGURE 1.

Schematic illustration of the experimental paradigm: the paired‐associate learning task and the completion task combined with a confidence rating. In the learning phase, participants were instructed to react to new audiovisual stimulus pairs (e.g., a picture of an owl and a sound of a car). Unbeknownst to the participants, audiovisual pairs were divided into two different conditions comprising of an equal number of auditory and visual stimuli. The conditions differed in their underlying associative regularity determining how unimodal stimuli were bound into audiovisual pairs. Unimodal stimuli of the first categorical condition were assigned into fixed audiovisual pairs, which were consistently presented together throughout learning (fixed association condition). In the second categorical condition, unimodal stimuli were continuously recombined into different audiovisual pairs (variable association condition). Importantly, each unimodal stimulus was presented an equal amount of times across conditions. For the recollection phase, we used a completion task combined with a confidence rating task as a reliable measure for the amount of acquired multimodal knowledge

2.3.1. Incidental paired‐associate learning phase

In the learning session, participants performed a paired‐associate learning task. Within each trial, an auditory and a visual stimulus were presented simultaneously to the participants and lasted for 2 s. Stimulus presentation was followed by a central fixation point with an inter‐trial interval (ITI) randomly varying between 3 and 6 s. Participants were instructed to respond to each audiovisual pair by deciding whether both the auditory and the visual stimulus were related to a living or non‐living stimulus (i.e., a picture of a fish and a sound of a dog, or a picture of a cake and a sound of a hammer), or whether both stimuli represent distinct categories (i.e., a picture of an owl and a sound of a car). Participants were instructed to response as quickly (within the ITI) and as accurately as possible.

Unbeknownst to the participants, auditory and visual stimuli were arranged into two categorical conditions: the “variable association”‐ and the “fixed association”‐condition. Each condition contained different audiovisual pairs with an equal number of single auditory and visual stimuli. The assignment of the stimuli to the conditions was counterbalanced across participants. The two conditions differed in their underlying presentation regularity. The unimodal stimuli of the “fixed association” condition were assigned into fixed audiovisual pairs which were repeatedly presented throughout learning sessions. Each “fixed association” audiovisual pair was shown 16 times in the identical pairing. In contrast, audiovisual pairs of the “variable association” condition were continuously recombined into different audiovisual pairs across learning trials and no pair was repeated.

Taken together, both conditions contained an equal number of individual unimodal stimuli, and each individual stimulus was presented an equal amount of times. Notably, the only difference between both conditions was the underlying constancy of simultaneous presentations. This specific experimental design allowed us to examine the difference in the functional involvement of the neural networks during the acquisition of multimodal representations: On the one hand, the acquisition of consistent multimodal pairs, which can be encoded more automatically with learning (“fixed association”) and are assumed to result in strong associative memory. On the other hand, the acquisition of continuously rearranged multimodal pairs, which might require an increased demand for attentional control processes for goal‐directed behavior (“variable association”) due to the fact that participants learn that the stimuli belong to the variable condition and the responses cannot be based on memory. The total experiment consisted of 256 trials, subdivided into three learning sessions (86 trials, 85 trials, and 85 trials). Further, the learning phase contained an equal number of “fixed association” and “variable association” audiovisual trials, which were randomly presented. Crucially, participants were not informed about both the different categorical conditions of the audiovisual pairs as well as the subsequent recollection phase. Before the experiment started, participants got familiar with a short multimodal practice task using different stimuli and unstructured material that followed no binding rule. Given our a priori hypotheses, the learning phase allowed to address the question of whether different parietal networks can be dissociated and whether those differed in their functional involvement during multimodal learning.

2.3.2. Recollection phase

After the learning phase, all participants were verbally asked about noticing constant pairings. The participants that did not noticed anything, did not perform the recollection phase and were thus not included within subsequent analyses. All included volunteers were able to describe the presence of two distinct conditions and then performed the recollection phase.

The recollection task was a modified version of the completion task combined with a confidence rating task (Persaud et al., 2007). A written instruction presented to the participants explicitly stated the presence of two conditions of audiovisual pairings during the learning phase, and that the task was now to decide to which category the presented stimulus pairs belonged. Hence, the aim of the recollection phase was to objectively measure the amount of explicit multimodal knowledge acquired under an incidental condition.

To measure this, we used the same multimodal pairs as in the paired‐associate learning task. However, within the recollection task, audiovisual pairs were presented in a sequential order, that is, as unimodal stimuli. Using successive unimodal stimuli rather than synchronous stimuli allowed to investigate whether the acquired knowledge about the multimodal information and the neural correlates can be transferred to unimodal conditions. To be specific, in each trial, two unimodal stimuli were first presented to the participants with a temporal delay using a jitter of 3–6 s.

With regard to the two different conditions, the eight multimodal stimulus pairs of the fixed association condition were identical to those of the learning task. Unimodal stimuli of the variable association condition, however, were now randomly assigned into eight multimodal pairs.

The participants were instructed to identify whether the two presented stimuli had been consistently paired in the previous learning task. Then, each response was followed by a confidence rating. Participants had to evaluate whether they were (a) “sure,” or (b) “unsure” about their response. Each audiovisual pair was presented four times, with the auditory and visual stimulus in first position twice, respectively. The recollection phase consisted of 64 completion task trials (32 for each condition: fixed association vs. variable association) and 64 confidence rating trials (sure vs. unsure), a total of 128 trials.

Taken together, this approach allowed us to address the questions of whether the two different parietal networks were also involved during explicit recollection under successive presentation of unimodal stimuli. Further, we tested whether the involvement of the two parietal networks were already required during the presentation of the first or second stimulus.

2.4. Statistical analyses of behavioral data

For both, the learning and the recollection phase, we only included correct responses in the subsequent fMRI analyses. The recollection phase was used to assess the amount and the status of multimodal knowledge by two criteria: the completion task performance in combination with the confidence rating, and the verbal report. First, we only included participants who performed above chance level on their correct responses. The confidence rating was used to identify whether participant's knowledge was explicit (Dienes, 2007; Dienes & Perner, 1999). Participant's knowledge was characterized as explicit if both performance scores were above 50% (at least 33 correct responses among the 64 recollection task trials) and accompanied by high confidence rating. Finally, participants were instructed to verbally describe the hidden binding regularity (fixed association vs. variable association) of the audiovisual pairs in post‐experimental interviews. Within the post‐experimental interviews, participants were explicitly instructed to describe what they have noticed during the learning phase by writing all the distinctive features they have noticed on a post‐experimental questionnaire. Only if these two criteria were met, participants were included within subsequent analyses. Participants were classified as “not explicit” if they have noticed anything or were not able to give any (correct) examples for each condition (the fixed association vs. variable association). To identify whether there is a relation between memory performance and individual event‐related blood oxygen level‐dependent (BOLD) activations in the regions of interest (ROIs), we assessed participants' memory sensitivity. To assess this, we calculated the statistic d′ for each participant. The d′ represents the standardized difference between participant's hit rate and false alarm rate. The sensitivity value of each participant was then used within subsequent fMRI analyses.

2.5. Functional MRI acquisition and data preprocessing

Functional MRI data were collected on a Siemens 3 Tesla Prisma MR system with a 32‐channel head coil. A standard gradient echo‐planar imaging (EPI) T2*‐sensitive sequence was used with parallel imaging (GRAPPA; in‐plane acceleration factor = 2) and simultaneous multi‐slice acquisitions (slice acceleration factor 2). Each functional volume contained of 54 continuous axial slices obtained with a 0.5 mm interslice gap (TR = 1636 ms, TE = 29 ms, flip angle = 70°, voxel size = 2 × 2 × 2). Finally, after functional imaging, we acquired a structural high‐resolution T1‐weighted image for each participant using a magnetization prepared rapid gradient echo (MPRAGE) sequence (voxel size = 1 × 1 × 1 mm).

For preprocessing and statistical analyses of the functional MRI data, we used the statistical parametric mapping (SPM12; http://www.fil.ion.ucl.ac.uk/spm/; Welcome Department of Imaging Neuroscience, London, UK). Using field maps, functional Images were realigned to the first volume. Then, the T1 weighted structural scans were coregistered with the functional images and segmented into the different tissue classes (gray matter, white matter, and cerebrospinal fluid). Using the DARTEL toolbox (Diffeomorphic Anatomical Registration Through Exponentiated Lie Algebra), the resulting individual subjects' tissue class images (gray matter, white matter) were applied to the structural images to create a structural group template, and to the functional images for spatial normalization. Finally, functional Images were smoothed with a 6 mm full width at half maximum (FWHM) isotropic Gaussian kernel.

2.6. Statistical analysis of functional data

2.6.1. Learning phase

The fMRI data consisted of two different experimental conditions, the fixed association versus variable association condition, subdivided into three different sessions. To allow a better time resolution of the two conditions across learning, we divided each learning session into two equally large intervals, resulting in 6 learning sub‐sessions. The fMRI data was analyzed by an estimation of the BOLD signal for each condition (stimulus condition: fixed association vs. variable association) and session (sub‐session: 1–6) and errors (incorrect responses) modeled as a hemodynamic response function. Using a general linear model, regression coefficients were obtained for each regressor (condition in session). Errors were modeled as a separate regressor. Except for the error regression coefficients, beta weights for each regression coefficients (of only included onsets of correct responses) were entered into a group analysis using a flexible factorial design.

The outcome of learning was assessed by comparing the two conditions within the final session of learning and using the memory sensitivity value (d′) as a covariate for each voxel (see Section 2.4). For a visualization of the relation between the parietal BOLD signal responses and the d′ of each participant, we used the toolbox “rfxplot” for SPM12 (Gläscher, 2009; http://rfxplot.sourceforge.net) to retrieve first level data of each participant. For each participant, we extracted the maximal signal intensity of the beta values within a sphere of 4 mm around the group peak voxel, and then fitted a linear regression between the signal intensity and the behavioral measures. This analysis is only used to visualize the relation that resulted from the whole brain covariate analysis.

Assuming that multimodal knowledge was acquired gradually over time, we were interested whether there were time course dependent changes in ventral and dorsal activity across sub‐sessions (i.e., sub‐session 1–6) and condition. In particular, we tested whether ventral and dorsal region differed in their functional involvement across time. This was statistically implemented at the level of the group analysis as an interaction contrast of condition x time separately for each assumed functions (i.e., increase in activation for (1) fixed association vs. variable association condition and (2) variable association vs. fixed association condition). This allowed to dissociate the different time courses across the learning phase between conditions. Further, for each conditions separately, we tested for an increase of activity across the six sub‐sessions. Finally, to directly test for the assumed dissociation related to ventral versus dorsal distinctions in the multimodal learning process, we further performed a repeated‐measures ANOVA outside of SPM using the following factors: region (dorsal, ventral), condition (fixed association, variable association) and sub‐session (1–6). Data for the repeated‐measures ANOVA was retrieved from first level data, using the toolbox rfxplot. For each participant, we extracted the mean signal intensity of the beta values (for condition and session) within a sphere of 4 mm around the group peak voxel.

Finally, we tested for time‐dependent changes in activation within hippocampal regions across multimodal learning sessions. We tested for an increase of neural activity in both categorical conditions, respectively, in order to examine whether hippocampal activity were found in both conditions, or whether hippocampal responses were limited to one specific condition.

2.7. Recollection phase

As in the learning session, we assumed that activations of the two different parietal networks can also be observed during successful recollection, while relying on experienced‐dependent processing (fixed association condition), or attentional control for task‐dependent processing (variable association condition) based on the formation of stable multimodal associations.

We analyzed the fMRI data by an estimation of the BOLD signal for each condition (stimulus condition: fixed association vs. variable association), event (stimulus event: first vs. second), type (stimulus type: visual vs. auditory), and errors (incorrect responses) modeled as a hemodynamic response function. Therefore, only correct remembered stimulus pairs of both conditions were used in the main regressors of all conditions (correct categorized as belonging to the variable or fixed association condition). Furthermore, since nearly 80% of all correct trials were rated as “sure” only this clear explicit trials were used and all other trials were incorporated in the error regressor. Following the same procedures as for the learning trials, contrasts for each regression coefficients (but not the error regression coefficient) were built and transmitted into a flexible‐factorial design including the inter‐subject variability as random effects, while using stimulus event, stimulus type and stimulus condition as repeated factors.

For both events, the first and second stimulus event, we examined recollection‐related effects when comparing between conditions (fixed association vs. variable association), regardless of stimulus modality (visual or auditory). This contrast allowed examining whether the same parietal networks were involved during explicit recollection of the new acquired multimodal knowledge under successive presentation of unimodal stimuli.

For both the learning and the recollection phase, we reported significant fMRI effects at a threshold of p <.05, corrected for familywise error (FWE) using whole brain analysis. ROI analysis was used to examine learning‐related activation in bilateral parietal cortex, with particular interest in the difference in neural response between ventral and dorsal regions of the bilateral parietal cortex. All ROI analyses were based on functional ROIs from an atlas defined from resting‐state connectivity (Shirer et al., 2012). Accordingly, for the left the right parietal effects and the hippocampal effects we used corresponding ROIS from the Dorsal Default Mode Network. All significant fMRI results were reported at a voxel‐level threshold of p <.05 FWE small volume corrected (SVC).

3. RESULTS

3.1. Behavioral results

Across learning trials, participants' mean overall error rates were very low (mean: 1.08%, SD: 0.6%). For each participant, mean response times (RTs) were calculated with regard to the onset of the stimulus for each single input and for each condition separately (Figure 2). A repeated‐measure ANOVA (factors session and condition [fixed association/variable association]) indicated a general decrease of RTs across sessions (F _(5,255) = 7.6, p <.001) and a difference between conditions (F _(1,51) = 79.8, p <.001). Importantly, the interaction between session and condition (F _(5,255) = 34.63, p <.001) revealed that the fixed association condition is processed faster than the variable association condition with learning. RT analysis indicated a benefit in processing stable fixed association multimodal associations.

FIGURE 2.

Behavioral results. The more pronounced decrease in reaction times (RTs) across learning sessions (divided in six sub‐sessions) for the fixed association (red) condition compared to the variable association (blue) condition demonstrates fast processing of incoming information based on stable multimodal associations. Error bars indicate standard deviations

All included participants (n = 52) reported the existence of two different conditions with different binding regularities in post‐experimental interviews. Among included volunteers, no individual participant was below chance level in correct answers. The group mean score of the completion task performance (score: 74%) was above the chance level of 50%, in which 76% of the correct answers were accompanied with high confidence ratings (total score of correct and high confidence answers: 58%). Furthermore, we used a one sample t‐test to tested whether correct responses (in percentage) were statistically significant above the chance level of 50%, supporting that a reliable multimodal knowledge was acquired (T ₍₅₁₎ = 13.32, p <.001). Among correct answers of the fixed association condition, 94% were given with high confidence indicating a great amount of explicit awareness on the newly learned information, that is, the novel acquired knowledge of the fixed multimodal associations. Thus, recollection behavioral data revealed that the fixed association audiovisual stimulus pairs were successfully learned throughout sessions. Based on participants' recollection performances, we calculated participants' performance sensitivity, d′, in order to assess participant's ability to discriminate between fixed association and variable association multimodal pairs. For the fixed association condition, we calculated a mean d′ score of 1.94, while participants' performance sensitivity varied from zero to 4.31.

3.2. Imaging results

3.2.1. Learning phase

Results indicated a clear dissociation of PPC recruitment regarding both conditions. For the fixed association condition, increased activations in the left vPPC, including the left AnG were found (MNI coordinates: x = −48, y = −70, z = 38; T = 5.31; FWE p <.05 whole brain corrected; MNI coordinates: x = −52, y = −60, z = 42; T = 5.01; FWE p <.05 whole brain corrected) contrasting both conditions within the final learning session. Additionally, effects in the right AnG (MNI coordinates: x = 56, y = −60, z = 38; T = 3.9; SVC FWE p <.05) and right hippocampus (MNI: x = 32, y = −24, z = −14; T = 3.39; SVC FWE p <.05) were specific for fixed association multimodal pairs.

On the other hand, effects for the variable association condition could be related to more dorsal regions of the PPC (MNI coordinates: x = 26, y = −66, z = −46; T = 7.19; FWE p <.05), including the bilateral SPL (MNI coordinates: x = −30, y = −58, z = 40; T = 5.27; MNI coordinates: x = 32, y = −58, z = 44; T = 4.99; SVC FWE p <.05). Furthermore, for whole brain analysis, we found activations in the occipital cortex, middle frontal gyrus, bilateral insula, cerebellum, and motor related regions (see Table 1 for full whole‐brain corrected results).

TABLE 1.

Functional magnetic resonance imaging (fMRI) effects for the two contrasts of interest during the last session (learning)

Last session: Fixed association > variable association
				MNI coordinates
Hem	Region	Cluster size	t‐value	x	y	z
L	Angular gyrus	22	5.31	−48	−70	38
			5.01	−52	−60	42
R	Hippocampus	57	3.39	32	−24	−14

Last session: Variable association > fixed association
				MNI coordinates
Hem	Region	Cluster size	t‐value	x	y	z
L	Middle occipital gyrus	371	7.19	−48	−70	38
L	Inferior frontal gyrus	438	6.61	−44	8	28
L	Posterior medial frontal gyrus	51	6.06	−6	8	52
L	Precentral gyrus	52	5.52	−30	−4	54
R	Inferior frontal gyrus	13	5.41	50	18	28
R	Cerebellum	18	5.26	6	−74	−26
R	Insula	8	5.25	34	26	2
R	Middle cingulate gyrus	5	5.13	6	14	44
L	Insula	5	5.09	−32	22	6

Note: For whole brain analysis, significant fMRI effects were reported at a threshold of p <.05, corrected for familywise error (FWE).

A positive relation between the memory sensitivity values (d′) and activation within the left (MNI coordinates: x = −52, y = −68, z = 30; T = 4.17; SVC FWE p <.05) was found. This significant relation reveals that the AnG activity was sensitive to the amount of the acquired multimodal memory (see Figure 3).

FIGURE 3.

Activation in the left angular gyrus and recollection strength of multimodal information. The correlation between the maximal signal intensity within the left angular gyrus (AnG) and the behavioral measure (d′) of each participant is plotted. It revealed a positive correlation between memory sensitivity and increased activations in the left AnG (Pearson correlation; r ∼ = .73 and p <.001). Only statistical clusters after a small volume corrected (SVC) were shown in the corresponding regions‐of‐interest (ROIs) (display voxel threshold: p <.05, SVC). Dots are single‐subject values. The dotted line represents the linear regression line, indicating the extent of correlation between the two variables on the vertical and horizontal axes

By comparing the time course of ventral and dorsal activity across learning sessions, the emergence of the functional dissociation of PPC recruitment was analyzed (interaction effects of time × condition). Across learning sessions, increased activation of fixed association multimodal stimulus pairs compared to variable association multimodal stimulus pairs was found in the left AnG (MNI: x = −48, y = −70, z = 32; T = 4.07; SVC FWE p <.05; see Figure 4). The coordinates correspond to the reported submodule around the AnG reported in earlier studies and are assumed to play a role in re‐instantiating context‐specific perceptual information (Nelson et al., 2010). To assess possible differences at the start of the experiment both conditions were compared during the first session and no reliable difference was observed supporting our interpretation that the functional differentiation emerges during learning. The reversed contrast (i.e., time‐dependent increase in activation for variable association > fixed association condition) revealed significant activation within a more dorsal region of the parietal cortex, the left (MNI coordinates: x = −32, y = −54, z = 36; T = 4.65; SVC FWE p <.05; see also Figure 4, correspond to submodule around the IPS according to [Nelson et al., 2010] involved in attentional control) and the right superior parietal cortex (MNI coordinates: x = 34, y = −56, z = 44; T = 4.09; SVC FWE p <.05). To evaluate a possible influence of RTs on the neural activity the analysis was replicated by including the individual single trial RTs in the first level GLM. The results showed no substantial changes (ANG, (MNI: x = −48, y = −70, z = 32; T = 3.94; p <.05; left parietal x = −32, y = −54, z = 36; T = 4.87; right parietal: x = 34, y = −56, z = 44; T = 4.37; p <.05). These results confirm the assumption that both parietal regions are involved in the memory processing and are not affected by the different RTs across time. The subsequent repeated‐measures ANOVA across regions revealed a significant interaction effect (region × condition × and sub‐session; F _(5,255) = 11.6; p <.001), confirming the dissociation between ventral versus dorsal regions in the multimodal learning process and showed that this functional dissociation emerged with learning.

FIGURE 4.

Different functional roles for the ventral and dorsal posterior parietal cortex during the acquisition of multimodal knowledge. We found increased activation in the left angular gyrus (AnG) for the contrast fixed association > variable association (upper left panel, activation in red), and increased activation in the left SPL for the contrast variable association > fixed association (upper right panel, activation in blue). Only statistical clusters after a SVC were shown in the corresponding ROIs (display voxel threshold: p <.05, SVC). For visualization, we extracted the mean contrast estimates at the peak coordinate in the left AnG (x = −48, y = −70, z = 32) and left SPL (x = −32, y = −54, z = 36) across session and each condition, respectively. A linear function (regression) was fitted to the extracted data to reveal the neural effects of multimodal learning across session and condition. Line graphs represent the multimodal learning‐related effects between the factors condition (red: fixed association condition, blue: variable association condition) and session. Lower left panel: For the fixed association condition, AnG activity increased during learning of fixed association multimodal material in comparison to the variable association material. Lower right panel: In contrast, fMRI signal increased within the left SPL during learning of variable association multimodal associations but decreased for fixed association associations. Error bars indicate standard error of the mean (SEM)

Furthermore, we found significant activation within the right precuneus (MNI coordinates: x = 10, y = −62, z = 22; T = 3.22; SVC FWE p <.05), the supplementary motor cortex (MNI coordinates: x = −4, y = 6, z = 52, FWE p <.05 whole brain corrected), the middle frontal gyrus left cortex (MNI coordinates: x = −40, y = 20, z = 26, FWE p <.05 whole brain corrected), and within the left anterior insular cortex (IC) (MNI coordinates: x = −34, y = 20, z = 4, FWE p <.05 whole brain corrected). The activity in the IC decreased with learning but this decrease was more pronounced for the fixed association condition.

Finally, for the additional ROI analysis, we found significant effects in bilateral hippocampus showing learning related increase in both conditions (MNI coordinates: Left hippocampus, fixed association condition: x = −26, y = −12, z = −14, T = 4.76; variable association condition: x = −28, y = −12, z = −14, T = 4.85; Right hippocampus, fixed association condition: x = 30, y = −22, z = −14, T = 4.46; variable association condition: x = 28, y = −22, z = −14, T = 4.37, SVC FWE p <.05, see Figure 5).

FIGURE 5.

Increased activity within bilateral hippocampus during multimodal learning. For both multimodal conditions, fixed association (red) and variable association (blue), neural activity increased in right and left hippocampal structures (left hippocampus: fixed association: x = −26, y = −12, z = −14; variable association: x = −28, y = −12, z = −14; right hippocampus: fixed association: x = 30, y = −22, z = −14; variable association: x = 28, y = −22, z = −14), revealing its involvement in the flexible acquisition and organization of multimodal relational information. Display voxel threshold: p <.05, SVC

3.2.2. Recollection phase

The functional dissociation of PPC activations were supported by comparable effects during the recollection phase. In particular, after the presentation of the second stimulus the fixed association condition showed significant effects within the ventral PPC, including the bilateral AnG (MNI coordinates: left: x = −40, y = −74, z = 32; T = 4.61; right: x = 52, y = −58, z = 16; T = 7.63; SVC FWE p <.05; see Figure 6, top panel). In contrast, for the variable association condition, increased activation within dorsal PPC was detected (MNI coordinates: left: x = −28, y = −52, z = 40; T = 6.57; FWE p <.05 whole brain; right: x = 48, y = −62, z = 44; T = 4.93; SVC FWE p <.05; see Figure 6 (bottom panel).

FIGURE 6.

Recollection‐related effects in ventral and dorsal regions of the parietal cortex. Upper panel (activation in red): The retrieval of fixed association multimodal associations was related to increased activity within ventral regions of the parietal cortex, in bilateral angular gyrus (AnG) (MNI coordinates: left: x = −40, y = −74, z = 32; right: x = −52, y = −58, z = 16). Lower panel (activation in blue): Increased activity within dorsal regions of the parietal cortex, in bilateral superior parietal cortex (MNI coordinates: left: x = −28, y = −52, z = 40; T = 6.57; right: x = 48, y = −62, z = 44), was associated with the recollection of variable association multimodal associations. Display voxel threshold: p <.05, small volume corrected

Furthermore, increased activations for the fixed association stimuli were found within the hippocampus (MNI coordinates: left: x = −32, y = −24, z = 12; T = 4.80; right: x = 40, y = −20, z = −16; T = 4.97; SVC FWE p <.05) and several different regions across the brain which are commonly associated with episodic memory formation (see Table 2 for full results). Interestingly, hippocampus showed increased activity for the fixed association stimulus pairs even during the presentation of the first stimulus (MNI coordinates: left: x = −18, y = −28, z = −8; T = 3.98; right: x = 18, y = −30, z = −4; T = 4.28; SVC FWE p <.05). Further, during the presentation of the first stimulus visual and frontal areas showed increased activity for the fixes associations compared to the variable associations (see Table 3).

TABLE 2.

Functional magnetic resonance imaging (fMRI) effects for the two contrast of interest during the second stimulus event (recollection)

Second stimulus event: fixed association > variable association
				MNI coordinates
Hem	Region	Cluster size	t‐value	x	y	z
L	Middle occipital gyrus	11,064	10.577	−24	−90	22
R	Precuneus	2018	9.060	8	−52	72
L	Precentral gyrus	59	6.434	−62	−2	10
R	Mid orbital gyrus	159	6.315	0	48	−6
R	Putamen	149	6.129	30	−6	−4
L	Superior frontal gyrus medial segment	160	6.310	0	48	−6
R	Putamen	149	6.130	30	−6	−4
L	Superior frontal gyrus medial segment	160	5.790	−2	56	12
R	Transverse temporal gyrus	75	5.790	50	−8	12
R	Precuneus	54	5.620	18	−54	6
R	Posterior cingular gyrus	48	5.540	4	−42	6
L	Inferior temporal gyrus	12	5.360	−44	−16	−22
R	Superior frontal gyrus medial segment	10	5.340	8	62	26
R	Precentral gyrus	18	5.280	32	−22	54
L	Precentral gyrus	14	5.260	−42	−12	36

Second stimulus event: variable association > fixed association
				MNI coordinates
Hem	Region	Cluster size	t‐value	x	y	z
R	Middle cingulate cortex	1027	8.578	6	22	46
R	Anterior insula	365	8.384	34	24	−6
L	Anterior insula	372	8.084	−32	14	8
L	Precentral gyrus	268	8.013	−52	6	20
L	Superior parietal cortex	419	6.896	−28	−52	40
L	Postcentral gyrus	75	6.608	−56	−20	42
L	Inferior frontal gyrus	64	5.984	−48	40	−4
L	Precentral gyrus	46	5.972	−38	−18	54
L	Superior frontal gyrus	22	5.607	−24	−6	50
R	middle frontal gyrus	6	5.315	50	24	28

Note: For whole brain analysis, significant fMRI effects were reported at a threshold of p <.05, corrected for familywise error (FWE).

TABLE 3.

Functional magnetic resonance imaging (fMRI) effects for the two contrasts of interest during the first stimulus event (recollection)

First stimulus event: fixed association > variable association
				MNI coordinates
Hem	Region	Cluster size	t‐value	x	y	z
R	Superior occipital gyrus	5683	8.906	24	−86	30
L	Calcarine gyrus	5683	8.641	0	−94	16
R	Cerebelum (Crus 1)	5683	7.800	46	−64	−22
L	Postcentral gyrus	308	7.206	−44	−18	62
L	Cerebelum (Crus 1)	302	6.236	−48	−64	−26
L	Inferior occipital gyrus	302	6.067	−50	−72	2
R	Precuneus	59	5.980	4	−50	72
R	Posterior medial frontal gyrus	6	5.428	2	−26	76

First stimulus event: variable association > fixed association
				MNI coordinates
Hem	Region	Cluster size	t‐value	x	y	z
R	Precentral gyrus	383	9.716	38	−16	52
L	Inferior frontal gyrus	79	6.659	−56	6	22
R	Inferior frontal gyrus	132	6.637	56	10	12
L	Insula	46	6.419	−32	12	8
L	Posterior medial frontal gyrus	38	6.318	−6	0	56
R	Insula	50	6.158	40	0	12
R	Posterior medial frontal gyrus	25	5.413	6	2	54
L	Postcentral gyrus	18	5.286	−54	−20	22

Note: For whole brain analysis, significant fMRI effects were reported at a threshold of p <.05, corrected for familywise error (FWE).

4. DISCUSSION

In the present study, we observed a clear functional dissociation of the vPPC and dPPC during memory acquisition of multimodal associations. We suggest that this functional difference can be attributed to two different processing modes related to the developed memory as outlined in previous theories (Cabeza et al., 2008, 2011). On the one hand, learning fixed audio‐visual combinations can affect neural processing based on the development of memory for distinct pairs. On the other hand, weaker memory for changing multimodal associations resulted in intentional, goal‐directed behavior requiring the serial processing of single stimuli from the different modalities. Examining the functional differentiation of the networks during incidental learning of multimodal associations, a strong relation between memory and network dynamics was revealed. Due to the experimental design the differential recruitment of the parietal areas can be attributed directly to the associative process during learning and can be compared to known effects for item and source memory tasks.

The behavioral results demonstrated a more pronounced decrease in RTs across learning sessions for fixed association compared to variable association audiovisual stimulus pairs in line with the assumption of a learning process as a function of stimulus repetition. The imaging results revealed that increased activity in the vPPC, the left AnG, was not only related to memory‐dependent multimodal memory, but also scaled with memory sensitivity (d′) assessed after learning. Further, the reliable interaction of the condition and session effect in both PPC regions showed that the functional dissociation emerges as a consequence of learning. Within the used paradigm in both conditions the same amount of individual stimuli was presented and therefore the observed effect within the AnG for the fixed association condition can be attributed to the formation of the associative pairing between multimodal stimuli. In contrast, neural activation of the dPPC decreased in response to consistently paired associations (fixed association condition), but increased for audiovisual pairs following a variable binding regularity (variable association condition). The continuous recombination of unimodal information required a constant demand for attentional mechanisms for successful task‐directed behavior, which might rely on the recruitment of the SPL. Interestingly, we observed this difference in PPC activations in both memory stages, learning and recollection. Overall, findings suggest that functional differentiation of ventral and dorsal regions emerge with learning as a function of the stability of related integrated episodic representations. The fact that the interaction of learning by condition between both areas was not affected by the different RT support the interpretation of AnG reflecting stable memory formation and is not related to the decreasing task difficulty with time.

4.1. Memory related function of ventral PPC regions

Based on present findings, AnG recruitment was related to the formation of stable memory of multimodal associations. Although some parts of the PPC seem to scale with repeated exposure of objects (Gilmore et al., 2015), the present and previous result revealed that the AnG can be functionally related to successful memory processing rather than perception (Binder et al., 2009; Daselaar et al., 2009; Favila et al., 2018; Kim, 2010, 2011). The involvement of AnG in the formation of multimodal memory related processing was reported in several studies (Aly et al., 2018; Bonnici et al., 2016; Branzi et al., 2020; van der Linden et al., 2017; Wagner et al., 2015). Across the literature, AnG activations did not only scale with memory performance (Humphreys & Ralph, 2015; Van Opstal et al., 2008; Vilberg & Rugg, 2009), but were also sensitive to experienced‐dependent learning of relational material (Aly et al., 2018; Baldassano et al., 2017; Hasson et al., 2015; van der Linden et al., 2017; Tibon et al., 2019). Further, the pattern of AnG activation was related to both the content of encoding and the related strength of the new acquired knowledge during recollection as a result of pattern reinstatement (Chen et al., 2017; Kuhl et al., 2013; Kuhl & Chun, 2014; Lee et al., 2017). The functional role of AnG is not limited to multimodal information. However, it was demonstrated that the AnG is more sensitive to the formation of multimodal compared to unimodal memory representations (Bonnici et al., 2016; Richter et al., 2016; Sestieri et al., 2012; Tibon et al., 2019), since multimodal processing requires the processing of context meaning (Branzi et al., 2019, 2020). Therefore, a multimodal setting is more sensitive for the examination of this network differentiation. However, for multimodal memory the associative binding between stimuli is important and it was unclear whether this process can be related directly to the function of the AnG. The present study clearly confirms this assumption and show that the observed effect closely correspond to previously examined submodules within the parietal memory network (Gilmore et al., 2015; Nelson et al., 2010). Memory related effects within the AnG were observed in for autobiographical events or imagined future events (Schacter et al., 2007; Thakral et al., 2017) supporting the relational processing within the AnG.

In the present study, the functional role of both PPC regions differentiates with learning but was also observed during recollection where the unimodal stimuli were presented in succession. Here the task was to identify whether those two unimodal stimuli had been consistently paired in the previous learning task, simply recognizing the identity of the two unimodal stimuli in isolation might not lead to correct answers. Thus, this task only measured associative memory of the multimodal pairs. Recollection related effects for constant stimulus pairs were observed in the AnG which is consistent with previous findings (Hutchinson et al., 2009; Uncapher & Wagner, 2009). During learning and also recollection bilateral AnG effects were observed underlying the functional diversity of the AnG (Rugg & King, 2018; Seghier, 2013). During the learning phase, however, the specific difference between conditions across learning showed reliable effects only in the left AnG, supporting its specific recruitment in memory‐dependent processing in which the acquisition of highly relational material was particularly relevant (Davis & Yee, 2019; Humphreys & Ralph, 2017; Lewis et al., 2019; Sestieri et al., 2011; Vilberg & Rugg, 2009). The recollection effect within the AnG was most pronounced after the presentation of the second stimulus regardless of modality. This further supports the importance of the AnG for the memory dependent processing of associative information. The hippocampus, frontal and occipital areas already differentiated both condition already during the presentation of the first stimulus indicating the memory access already after one stimulus. This finding indicates, that the memory is already triggered by the first stimulus, but the presentation of the associated second stimulus affected the AnG as a feedback or amplifier signal for the associative memory.

Together, AnG activation might be specific for memory‐dependent associative processing which requires the generation and updating of context meaning (Bonner et al., 2012; Branzi et al., 2019, 2020; D'Acremont et al., 2013; Wagner et al., 2015), rather than the recollection of less relational characteristics within a homogenous context (Bellana et al., 2017, ; Bonnici et al., 2016; Humphreys et al., 2021; Ramanan & Bellana, 2019; Rugg & King, 2018). Accordingly, AnG has been regarded as a necessary precursor to the formation of “situation models” (Ranganath & Ritchey, 2012) and “schemata” (van der Linden et al., 2017; Schwartz et al., 2011; Wagner et al., 2015), as those are specific for the acquisition of “thematic relationships” (Davis & Yee, 2019; Lewis et al., 2019) and “event concepts” (Binder & Desai, 2011).

Neuropsychological data from patients with lesions in the PPC (PPC patients), including the AnG, provide additional interesting findings on associative memory processing. Compared to healthy controls, PPC patients were not severely impaired when recollecting acquired memory contents, but revealed stronger restrictions when recalling their subjective experience of the learned content (Berryhill et al., 2009; Russell et al., 2019; Simons & Mayes, 2008). One study on multimodal cued recall revealed that PPC patients were significantly impaired during retrieval, in particular, when it contains contextually rich episodic memories (Ben‐Zvi et al., 2015). A more recent study on associative processing found that PPC patients were restricted in both the integration of multimodal context and subjective evaluation during memory retrieval (Ciaramelli et al., 2017). According to their interpretation, controls, but not PPC patients, were able to use the richness of the learned experience resulting in the reinstatement of related features suggested to be the critical basis for judging an item as “remembered.” Further investigations into verbal and nonverbal memory performance revealed that neurodegenerative disease patients with early PPC dysfunction were associated with significant episodic amnesia (Ramanan et al., 2020). Across several studies with PPC patients, it has been frequently emphasized that episodic memory performance relies on the structural integrity of AG‐hippocampal connections (Ramanan et al., 2020), in particular when retrieving personally‐relevant episodic details from past experiences (Ramanan et al., 2020). These observations are consistent with studies using neurostimulation reporting impaired recollection of episodic details and contextual integration after AnG stimulation (Branzi et al., 2019, 2021; Davey et al., 2015). Other studies attributed AnG effects to the reduced subjective experience of episodic memories during recollection (Koen et al., 2018; Richter et al., 2016; Sestieri et al., 2012; Thakral et al., 2017; Tibon et al., 2019; Yazar et al., 2014). However, present and previous findings suggests AnG contribution to associative memory processing which requires the conscious access to the quality of the acquired memory, which is the basis for such judgments (Rugg & King, 2018). Specifically, the AnG maintains an integrated, multimodal episodic representation, it interacts with the medial PPC regions to support fast episodic processing and related vivid imagination of internal experienced‐dependent memory representations (Arnold et al., 2018; Baldassano et al., 2017; Brodt et al., 2016, 2018; Richter et al., 2016; Ritchey et al., 2020).

In the present study, increased activations within the hippocampus during learning of both fixed association and variable association condition were observed, while, during recollection, activations were limited fixed association trials. In line with previous memory accounts (Cohen et al., 1997; Henke, 2010), hippocampal function has been interpreted as being primarily involved in the initial establishment and rapid processing of new and flexible relational representations (Eichenbaum, 2017; Henke, 2010; Ranganath et al., 2005), even under incidental learning conditions (Duss et al., 2014; Henke et al., 2013; Rose et al., 2011). Furthermore, present hippocampal activation has been related to the reactivation of learned associative representations when using associative retrieval cues (Duss et al., 2014), consistent with present recollection data. Considering the rich anatomical connections among PM regions (Rushworth et al., 2006; Seghier, 2013; Uddin et al., 2010; Vincent et al., 2006, 2008; Xu et al., 2016), a functional interaction between the hippocampus and related PPC regions during associative processing has been assumed (Cooper & Ritchey, 2019; Ramanan et al., 2018; Shimamura, 2011). Taken together, a critical functional involvement of ventral and medial PPC regions and the hippocampus in relational memory‐dependent processing of episodic information while maintaining fixed association contextual meaning can be suggested (Ranganath & Ritchey, 2012; Ritchey et al., 2020).

For the variable association condition, stable association between continuously recombined stimulus pairs could not be easily acquired. Hence, attentional mechanisms might be particularly engaged, in particular when memories are weak and information is not readily accessible (Kim, 2010; Spaniol et al., 2009). Across the literature, dorsal PPC activations were frequently related to variable association, top‐down attentional processing (Hutchinson et al., 2014; Kim, 2018; Vincent et al., 2008), particularly during effortful memory retrieval (Guerin et al., 2012; Hutchinson et al., 2014).To illustrate, SPL activations, a region of the dorsal attention network (DAN), were associated with decision uncertainty and related longer RTs (Hutchinson et al., 2014). These findings were interpreted as reflecting the sustained allocation of attentional processes in favor of memory‐guided decisions. Interestingly, invasive recordings from dorsal PPC regions further revealed higher response pattern for new compared to old stimuli (Rutishauser et al., 2018; Rutishauser, 2019), while, using fMRI, activations were pronounced during low compared to high confidence rating for both new and old stimuli (Hutchinson et al., 2014). Consistent with present findings, SPL activations were particularly related to audiovisual pairs with a variable binding (variable association), hence continuously representing a different multimodal pairing. Furthermore, activations within the SPL and the supramarginal gyrus, but not the AnG, were specifically related to goal‐relevant vs. incidental reactivation of event features (Kuhl et al., 2013). In contrast to vPPC, dPPC activations were specific for goal‐relevant compared to goal‐irrelevant feature information during retrieval (Favila et al., 2018). Interestingly, attention‐demanding tasks not only require DAN recruitment, but also deactivates regions of the vPPC, in particular the AnG (Guerin et al., 2012; Sestieri et al., 2017). With regard to the present task‐oriented use of the acquired knowledge, the sustained allocation of attentional mechanisms was required in particular during variable association trials compared to fixed association trials. Taken together, dPPC engagement, in particular the SPL, might reflect present task‐directed attentional recruitment, particularly required for goal‐directed behavioral preparation, response selection, and interference resolution (Corbetta & Shulman, 2002; Kim, 2010; Spaniol et al., 2009; Wager & Smith, 2003).

Interestingly, the IC decreased the neural activity across learning differentially between both conditions. The decrease is reliably more pronounced for the fixed association condition as expressed by the interaction effect of time by condition. The IC and its related circuitry are known to be involved in the conversion of novel to familiar stimulus for both object and taste recognition memory, mainly based on emotional salience detection (Bermudez‐Rattoni, 2014). Besides the involvement of the IC in emotional processing or affective learning the present results demonstrated that with learning the salience detection decreased but that in relation to the associative strength the variable associations recruited the IC to a larger degree.

5. CONCLUSION

Assuming that memory is an integral component of information processing, we suggest that the associative process that encodes multimodal associations can directly be addressed to the ventral parietal cortex, while the functional role of dorsal PPC regions, including the SPL, were attributed to attentional control processes related to weaker associative bindings. Present results suggest that the essential difference between the functional involvement between the vPPC and dPPC do not depend on item memory or the memory stage per se, but is a direct consequence of the amount of acquired associative memory.

CONFLICT OF INTEREST

The authors declare that they have no conflict of interests.

Jablonowski, J. , & Rose, M. (2022). The functional dissociation of posterior parietal regions during multimodal memory formation. Human Brain Mapping, 43(11), 3469–3485. 10.1002/hbm.25861

DATA AVAILABILITY STATEMENT

The datasets generated during and/or analyzed during the current study are available from the corresponding author on request.