Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2008 Jan 1.
Published in final edited form as: Neuropsychologia. 2007 Mar 12;45(11):2589–2597. doi: 10.1016/j.neuropsychologia.2007.03.006

Role of the medial temporal lobes in relational memory: Neuropsychological evidence from a cued recognition paradigm

Irene P Kan 1, Kelly S Giovanello 2, David M Schnyer 3, Nikos Makris 4,5, Mieke Verfaellie 1
PMCID: PMC1986641  NIHMSID: NIHMS25717  PMID: 17433382

Abstract

In this study, we examined the role of the hippocampus in relational memory by comparing item recognition performance in amnesic patients with medial temporal lobe (MTL) damage and their matched controls. Specifically, we investigated the contribution of associative memory to item recognition using a cued recognition paradigm. Control subjects studied cue-target pairs once, whereas amnesic patients studied cue-target pairs six times. Following study, subjects made recognition judgments about targets that were presented either alone (no cue), with the originally presented cue (same cue), or with a cue that had been presented with a different target (recombined cue). Controls had higher recognition scores in the same cue than in the recombined cue condition, indicating that they benefited from the associative information provided by the same cue. By contrast, amnesic patients did not. This was true even for a subgroup of patients whose recognition performance in the no cue condition was matched to that of the controls. These data provide further support for the idea that the hippocampus plays a critical role in relational memory, even when associative information need not be retrieved intentionally.

1. Introduction

Human neuropsychological studies and animal models have established the critical role of the hippocampus in consciously retrieving recent experiences (for a review, see Broadbent, Clark, Zola, & Squire, 2002). The cognitive processes that are mediated by the hippocampus, however, are less well defined. One prominent view states that the hippocampus is particularly important in linking previously unrelated pieces of information, such as the relationship between an item or event and the learning context, or between different elements that make up an event (Eichenbaum & Cohen, 2001; Rudy & Sutherland, 1995; Sutherland & Rudy, 1989). According to this relational view of memory, amnesic patients with hippocampal damage should be particularly impaired on explicit memory tasks that depend on memory for associations because such tasks place a high demand on relational memory processing. Furthermore, the extent of the impairment should reflect the degree to which relational processes are relied upon.

One useful distinction in characterizing the relative degree of relational demands is that between simple associations and complex associations (Isaac & Mayes, 1999a, 1999b). A simple association refers to the relation between an item and its learning context (e.g., experimental context), a relation that tends to remain constant across trials. By contrast, a complex association refers to the association among items, a relation that is variable from trial to trial. By the relational memory view, amnesic patients’ relational impairment should be evident in any task that requires the formation and retrieval of a new association, but patients should show disproportionate impairment in tasks that require retrieval of complex associations compared to tasks that require retrieval of simple associations.

One way to test this prediction is by comparing amnesic patients’ performance on item and associative recognition. In these studies, subjects are typically exposed to unrelated word pairs and are later tested on recognition memory for either single items or for associations between items (e.g., Hockley, 1991, 1992; Hockley & Consoli, 1999; Hockley & Cristi, 1996). Item memory is reflected in the correct discrimination between previously studied items (old items) and unstudied items (new items), and associative memory is reflected in the correct discrimination between pairs consisting of elements that were seen together during initial exposure (old pairs) and pairs consisting of elements that were both seen during the study phase but as part of different pairs (recombined pairs). Critically, both old and recombined pairs consist of two items that have previously been studied. Thus, successful discrimination between old and recombined pairs at test cannot be based on information about the individual items alone but, rather, must be based on information about the association between the two items studied together. According to the relational memory view described above, because relational demands are higher for associative recognition than for item recognition, amnesic patients with MTL damage are expected to be more impaired on associative recognition than on item recognition.

A few studies have reported dissociations between item and associative recognition in patients with amnesia. Giovanello, Verfaellie and Keane (2003) compared item recognition with associative recognition in a group of MTL and diencephalic amnesics and their matched controls. Subjects studied cue-target pairs and made recognition judgments about old targets, new targets, old pairs, and recombined pairs. To minimize scaling effects, item recognition performance was matched between the patient group and the control group by providing amnesic patients with additional study exposures (Giovanello & Verfaellie, 2001). Consistent with the relational memory view, amnesic patients were disproportionately impaired in associative recognition, compared to item recognition.

In another study, Turriziani, Fadda, Caltagirone, and Carlesimo (2004) evaluated associative and item recognition in a heterogeneous group of amnesic patients: amnesic patients were disproportionately worse at associative recognition (e.g., face pairs, face-occupation pairs) compared to item recognition (face only). Furthermore, based on data from a lesion analysis, the authors concluded that compared to patients with lesions elsewhere in the brain (e.g., basal forebrain), patients with restricted hippocampal damage showed a selective impairment in associative memory. These data provide further evidence for the dissociation between item and associative recognition and provide additional support for the critical role of the hippocampus in relational memory processing.

Evidence from functional neuroimaging also highlights the role of the hippomcapus in relational processing. Several studies have reported increased hippocampal involvement during encoding of relational information compared to encoding of item information (e.g., Achim & Lepage, 2005; Davachi & Wagner, 2002; Henke, Buck, Weber, & Wieser, 1997). Similarly, there is evidence that retrieval of associative information, compared to retrieval of item information, leads to increased hippocampal activity (e.g., Giovanello, Schnyer, & Verfaellie, 2004).

However, the dissociation between item recognition and associative recognition has not been consistently observed. Based on their patient studies, Stark and Squire (2002; 2003) argued that damage to the hippocampus disrupts item memory and associative memory to a similar degree. For example, Stark, Bayley and Squire (2002) used pictures of faces and houses to test item and associative memory in four patients with hippocampal damage. In the item recognition condition, subjects studied pictures of either faces or houses and were later tested with a yes/no recognition task. In the associative condition, subjects studied face-house pairs and were later asked to discriminate between old pairs and recombined pairs. The authors reported that amnesic patients were equally impaired in item and associative recognition and argued that the hippocampus mediates item and associative memory to the same extent. However, interpretation of their data is limited by several factors. First, as noted by the authors, amnesic patients performed at chance level on several tasks. Second, item recognition performance was not matched between the patient group and the control group. As such, it is difficult to interpret the numerical difference between the two conditions because baseline performance also differed between the two groups. Although item recognition was equated between the two groups in one of the experiments (Stark, Bayley, & Squire, 2002, Experiment 2), amnesic patients’ item recognition performance was at ceiling, which could obscure a disproportionate deficit in associative recognition.

An alternative explanation of the disproportionate impairment in associative memory in amnesia is that the two tasks differ in terms of task load and that the disproportionate impairment arises from this load difference. Whereas item recognition requires judgment about a single item, associative recognition requires judgment about two items. Thus, it is possible that amnesic patients’ relative impairment in associative recognition is a result of increased task load, and not increased relational demands, in associative recognition. To address this confound, Giovanello, Verfaellie, and Keane (2003) compared amnesic patients’ performance on single item recognition and two-item non-associative recognition and found no evidence of disproportionate impairment in the two-item non-associative condition in amnesic patients. These data provide preliminary support for the idea that increased task load was unlikely to be the primary reason for amnesic patients’ deficit in associative recognition. However, the load confound was not tested directly because item memory and associative memory were not directly compared under identical load conditions.

The relational memory view would be strengthened if we can find evidence for a disproportionate impairment in single item recognition in amnesia under conditions in which relational memory enhances control subjects’ performance compared to conditions in which relational memory does not support the performance of control subjects. To achieve this, we employed a cued recognition paradigm in the present experiment. Similar to item recognition, cued recognition requires discrimination between previously studied (old) and unstudied (new) items (e.g., Clark & Shiffrin, 1992). In cued recognition, subjects studied cue-target pairs and are later tested on recognition of old targets and new foils that are presented with studied cues. Previous studies have reported that neurologically intact individuals are better at discriminating studied targets when the targets are presented with the same cues with which they were studied than when targets are presented with different cues, suggesting that associative information is beneficial to item recognition in control subjects (e.g., Clark & Shiffrin, 1992). This paradigm allows us to assess both item and associative memory in the context of a single item recognition task, thus eliminating the confound of task load in associative recognition. Furthermore, unlike previous studies that required explicit retrieval of associative information, this paradigm allows us to assess subjects’ incidental use of associative information because it makes no demand on recollecting the relationships of the cue-target pairs.

We evaluated subjects’ unintentional use of associative information by manipulating the availability of associative information. We predicted that control subjects would be better at recognizing old target words when relevant associative information is available (same cue condition) than when available associative information is irrelevant (recombined condition) or when associative information is not present at all (no cue condition). By contrast, we predicted that amnesic patients would perform similarly in all three conditions because their relational memory processing impairment would impede their incidental use of associative information, and thus the presence of intact associative information would provide little or no benefit. To minimize scaling effects, we attempted to equate performance in the no cue condition in patients and controls by administering additional study presentations to the patients (six times) than to the controls (once).

2. Method

2.1 Participants

Eight amnesic patients (5 men, 3 women) with MTL lesions and two groups of healthy controls participated in this study. The patient group had a mean age of 55 years, an average of 14 years of education, and a mean verbal IQ of 102, as measured by the Wechsler Adult Intelligence Scale, Third Edition (Wechsler, 1997). Table 1 summarizes the demographic and clinical neuropsychological data for the patients.

Table 1.

Demographic and neuropsychological characteristics of amnesic patients and their matched controls.

WAIS, III
WMS, III
Patient Etiology Age Edu VIQ GM VD AD WM
P 001 Anoxia 49 14 111 59 72 52 96
P 002 Anoxia 42 16 86 49 53 52 93
P 003 Encephalitis 51 14 92 45 56 55 85
P 004 Anoxia 53 17 134 70 75 67 126
P 005 Anoxia 76 18 113 75 72 80 102
P 006 Anoxia 56 12 83 52 56 55 91
P 007 Encephalitis 62 12 106 69 68 77 111
P 008 Anoxia 47 14 90 45 53 52 93
Controls (SD) 55.1 (11.5) 14.1 (1.9) 107.5 (13.9)
Open in a new tab

Note: Age = Age in years; Edu = Education in years; WAIS, III = Wechsler Adult Intelligence Scale, III; VIQ = Verbal IQ; WMS, III = Wechsler Memory Scale, III; GM = General Memory; VD = Visual Delayed; AD = Auditory Delayed; WM = Working Memory.

One control group consisted of eight healthy individuals (1 man, 7 women) who participated in the behavioral experiment. This control group was matched to the patient group in terms of mean age (mean = 54 years, SD = 16), education (mean = 14 years, SD = 1.9), and verbal IQ (mean = 109, SD = 12.2). A second control group consisted of four men and eight women, who served as healthy controls for the volumetric analyses. Mean age for the male control subjects for P 001, P 002, and P 003 is 46 years (SD = 5.9), mean age for the female control subjects for P 007 is 61 years (SD = 2.8), and mean age for the female control subjects for P 008 is 46 years (SD = 2.9).

Six of the patients had an etiology of anoxia and the remaining two patients had an etiology of encephalitis. MRI scan could not be obtained for three of the anoxic patients because of medical contraindications (i.e., two with pacemakers and one with an aneurysm clip), but MTL pathology can be inferred on the basis of etiology. For the remaining five patients, extent of medial temporal lobe damage was evaluated based on volumetric analyses of MRI images, and each patient’s data were compared to data from four age- and gender-matched control subjects. These comparisons revealed that two of the anoxic patients (P 001 and P 008) had damage limited to the hippocampus and the two encephalitic patients (P 003 and P 007) and one anoxic patient (P 002) had damage to the hippocampus and also to the surrounding parahippocampal cortex. Table 2 summarizes these results.

Table 2.

Neuroanatomical characteristics of five amnesic patients: number of standard deviations away from control means in terms of volume (corrected for intracranial volume). Volumes that are > 2SD below control means are highlighted in bold.

Patient Left Hpp Right Hpp Bilat Hpp Left PH(a) Right PH(a) Bilat PH(a) Left PH(p) Right PH(p) Bilat PH(p)
P 001 −1.3 −2.5 −1.8 −0.7 0.4 −0.1 4.9 −0.1 2.4
P 002 −6.2 −3.4 −5.0 −2.2 0.3 −0.8 5.8 3.9 5.1
P 003 −6.4 −4.8 −5.7 −3.9 −2.3 −3.0 −2.9 0.2 −1.3
P 007 −4.5 −6.2 −5.4 −3.2 −5.8 −4.4 −1.3 −6.0 −3.0
P 008 −6.1 −6.4 −6.5 −1.8 0.8 −0.2 0.2 0.4 0.3
Open in a new tab

Note: Bilat = Bilateral; Hpp = Hippocampus; PH(a) = Anterior portion of parahippocampal cortex (entorhinal cortex, medial temporal pole, medial perirhinal cortex); PH(p) = Posterior portion of parahippocampal cortex (posterior parahippocampal gyrus).

Measurements for intracranial volume, the frontal lobes, parietal lobes, occipital lobes, and lateral temporal lobes were also made. The only significant volume reductions (> 2SD below control means) were found in the right occipital lobe (−2.4 SD) for P 003 and in the left lateral temporal lobe (−3.64 SD) for P 002. This finding for patient P 002 is consistent with the fact that he had a partial left temporal lobectomy (see section 2.2 for morphometric image analysis methods).

In accordance with the procedures of the Institutional Review Boards at Boston University and the Boston VA Healthcare System, all subjects provided informed consent.

2.2 Morphometric image analysis

The analyses were performed on magnetic resonance brain images of 17 subjects (5 patients and 12 controls). The images were acquired on a Siemens Allegra 3 Tesla scanner. The acquisition included two T1-weighted MP-RAGE series with the following parameters: TR = 2530 ms, TE = 3.25 ms, flip angle = 7 degrees, FOV = 256 mm, 128 contiguous 1.33 mm sagittal slices, matrix = 256 × 256.

Normalization routines were performed in accordance with methods outlined in Filipek (1988; 1994). Segmentation and cortical parcellation were done in a semi-automated fashion, according to the anatomic definitions of the Center for Morphometric Analysis (Filipek et al., 1994; Seidman et al., 2002). Subdivision of the cerebral cortex was accomplished with a fine-grained parcellation system that is referenced to a set of neuroanatomic landmarks that are robustly visible on MRI scans, as described in Caviness (1996) and Rademacher (1992).

With respect to medial temporal lobe structures, the amygdala and hippocampus were segmented individually as described in Seidman (2002). The parahippocampal gyrus was defined anteriorly by the isthmus of the temporal and frontal lobes, medially by the collateral fissure, laterally by the hippocampal fissure, and posteriorly by the anterior limit of the calcarine fissure. The parahippocampal gyrus was subdivided into an anterior and posterior region by a plane passing through the lateral geniculate nucleus perpendicular to the AC-PC line (Caviness et al., 1996). The anterior portion of the parahippocampal cortex consisted of entorhinal cortex, medial aspect of the temporal pole, and the medial portion of perirhinal cortex, and the posterior portion included the posterior portion of the parahippocampal gyrus.

A computer program, XVOL, was used to determine the volumes of all analytic units. On a given coronal image, the number of voxels in each parcellation unit was multiplied by the voxel volume. The volumes obtained for each unit were summated for all slices in which each unit appeared. Because volumes of some of the structures of interest (e.g. hippocampus) are affected by age (e.g., Bhatia, Bookheimer, Gaillard, & Theodore, 1993), we normalized the data for individual variation in intracranial vault volume (e.g., Free et al., 1995; Stout et al., 1999). Intracranial area was estimated using Brain Extraction Tool (BET) from the FMRIB Software Library from Oxford University (Smith, 2002).

2.3 Materials

The stimuli were 176 nouns with high frequency (mean Kucera-Francis written frequency = 84.5, SD = 27.2) and high concreteness (Coltheart, 1981 mean concreteness = 537.5, SD = 64.10) ratings. All items were between one and three syllables. From this set, 160 stimuli were randomly selected and divided into two 80-word sets that were matched on written frequency (Kucera & Francis, 1967), number of syllables, concreteness, and number of letters (all p’s > 0.10). One set served as cues and the other set served as targets. Items from the two sets were randomly paired to form 80 cue-target pairs, with the proviso that the cue and the target in each pair were not semantically related. We obtained forward and backward association ratings for each word pair (Nelson, McEvoy, & Schreiber, 1998) and found that the mean forward association and backward association ratings across all word pairs are both less than 0.0002, suggesting that the word pairs are unlikely to be associated. The remaining 16 items served as fillers for the study list.

The 80 pairs formed the original cue-target pairings. These pairs were randomly divided into two sets to create two study-test occasions. Each set of 40 pairs was divided into five sets of 8 pairs each that were used to create the five experimental conditions (old pair, recombined pair, old-new pair, old item, new item). Study lists were created by randomly selecting four of the five sets of original cue-target pairs. For one of these sets, the pairs were presented in their original pairing. For a second set, the cue-target pairs were randomly recombined by pairing the cue of one pair with the target of another pair from that set. For a third set, the cue member from each of the original pairs was presented with an unstudied filler item. These three sets formed the study conditions for the subsequent cued recognition test. The fourth set consisted of original cue-target pairs and formed the study condition for the subsequent no cue item recognition test. Pairs from the four sets were randomly intermixed in the study list, which resulted in a total of 32 study pairs.

Test lists consisted of 40 trials, which comprised 24 cue-target trials and 16 item (no cue) trials. The cue-target pairs were all presented in their original pairing. Thus, there were 8 studied (same cue) pairs, 8 recombined cue pairs, and 8 old-new pairs. The 16 no cue item trials consisted of the target members of the remaining two sets of cue-target pairs. One set had been studied and thus, these targets formed the old items, whereas the other set had not been studied, and thus, these items formed the new items. Pairs and items were randomly intermixed in the test list. The assignment of cue-target sets to conditions was counterbalanced across subjects according to a Latin square design. This resulted in five study lists and five corresponding test lists.

2.4 Procedure

Each subject was tested on two study-test lists. Whereas control subjects received both lists in one session, the amnesic patients encountered the two lists in two separate sessions, spaced approximately three months apart.

During the study phase, subjects saw word pairs presented one pair at a time in the middle of the computer screen. While subjects looked at the cue-target pair (e.g., box - brother), the experimenter read out loud a sentence that incorporated the two words and asked the subjects to make a likelihood judgment about the sentence on a 1–5 scale (e.g., Steve sent a box of chocolates to his brother for his birthday). Each cue-target pair remained on the screen until a likelihood judgment was made. In an attempt to equate performance in the no cue condition, amnesic subjects studied each set of cue-target words six times and control subjects studied each set of cue-target pairs only once.

During the test phase, either one or two words appeared on the computer screen. When one word appeared, subjects were asked to decide whether the word was “old” (i.e., encountered during study) or “new” (i.e., not encountered during study). When two words appeared, subjects were told that the word on the left was always old, meaning they had seen it during study, and that their task was to determine whether the word on the right was “old” or “new.” Subjects were also told that the reason the left word was provided was because, though not always the case, the left and right words might have occurred together during study, and that fact may help them decide the status of the word on the right. They were reminded that their decision concerned only the word on the right. Subjects were given unlimited time to make their response, and they were instructed to respond aloud with “old” or “new.” The test phase took place immediately after study.

3. Results

For each subject, hit rates, false alarm rates, discriminability (d′) scores, and corrected recognition scores were calculated (see Table 3). Corrected recognition scores for each condition (i.e., no cue, same cue, recombined cue) were calculated as the difference between the proportion of “old” responses to previously studied items (i.e., hits) and the proportion of “old” responses to new unstudied items (i.e., false alarms). The proportion of “old” responses to “old-new” pairs was used as the false alarm rate for both the same cue and recombined cue conditions, and the proportion of “old” response to “new items” was used as the false alarm rate for the no cue condition. Hits, false alarms, corrected recognition, and d′ were analyzed separately in 2 (group: amnesics, controls) × 3 (condition: no cue, same cue, recombined cue) mixed analyses of variance (ANOVAs).

Table 3.

Hit rates (endorsing studied items as “old”), false alarm rates (FA, endorsing unstudied items as “old”), corrected recognition scores (CR, hits - FA), discriminability scores (d′), and corresponding standard deviations (S.D.) in three experimental conditions (same cue, recombined cue, no cue) for all amnesic patients and matched controls.

Experimental Conditions Amnesic Patients Matched Controls
Hits FA CR d′ Hits FA CR d′
Same Cue (S.D.) 0.65 (0.14) 0.30 (0.19) 0.36 (0.27) 0.79 (0.68) 0.81 (0.13) 0.14 (0.12) 0.67 (0.10) 1.87 (0.47)
Recombined Cue (S.D.) 0.66 (0.11) 0.30 (0.19) 0.36 (0.23) 0.81 (0.57) 0.66 (0.15) 0.14 (0.12) 0.52 (0.12) 1.41 (0.42)
No Cue (S.D.) 0.71 (0.20) 0.33 (0.11) 0.38 (0.14) 0.95 (0.44) 0.72 (0.12) 0.22 (0.17) 0.50 (0.14) 1.36 (0.54)
Open in a new tab

The analysis of hit rates revealed a marginally significant group x condition interaction (F [2, 28] = 3.11, p = 0.06). Paired t-tests of control subjects’ performance revealed that their performance was higher in the presence of the same cue than in the presence of a recombined cue (t [7] = 3.17, p < 0.05), and their performance in the same cue condition was also marginally higher than when no cue was provided (t [7] = 2.17, p = 0.07). Furthermore, the presence of a recombined cue, relative to no cue, had no influence on recognition performance (t [7] = 1.46, p = 0.19). On the other hand, amnesic patients performed similarly across all three conditions, suggesting that cue availability does not affect their recognition performance (all pairwise comparisons yielded p’s > 0.26). Main effects of group and condition were not significant (all p’s > 0.10).

The false alarms analysis revealed a marginally significant group main effect (F [1, 14] = 4.39, p = 0.06), with amnesic patients making significantly more false alarms overall (mean = 0.32) than control subjects (mean = 0.18). The main effect of condition and group x condition interaction were not significant (p’s > 0.20).

The d′ analysis demonstrated a significant main effect of group (F [1, 14] = 9.39, p < 0.01), indicating that overall d′ across the three conditions was significantly higher for the control group (mean = 1.55) than for the patient group (mean = 0.85). Critically, a significant group x condition interaction was also found (F [2, 28] = 4.66, p < 0.05). Paired t-tests of control subjects’ performance revealed that their performance was higher in the presence of the same cue than in the presence of a recombined cue (t [7] = 3.45, p < 0.05), and their performance in the same cue condition was also significantly higher than when no cue was provided (t [7] = 3.24, p < 0.05). Furthermore, the presence of a recombined cue, relative to no cue, had no influence on recognition performance (t [7] < 1). On the other hand, amnesic patients performed similarly across all three conditions, suggesting that cue availability does not affect their recognition performance (all pairwise comparisons yielded p’s > 0.40). The main effect of condition was not significant (p > 0.10). Analyses with corrected recognition scores revealed largely the same pattern.

Unfortunately, our interpretation of these data is somewhat limited by the fact that we were unable to match recognition performance in the no cue condition in the amnesic subjects (mean d′ = 0.95) and the control subjects (mean d′ = 1.34), as confirmed by a marginally significant t-test (t [14] = 1.66, p = 0.12).

To better match performance in the no cue condition in the two groups, data from the worst performer in the amnesic group and the best performer in the control group were removed from subsequent analyses.1 Of the remaining seven patients, five had an etiology of anoxia and the remaining two patients had an etiology of encephalitis. Mean age, education, and VIQ were matched across these subsets of subjects (all p’s > 0.50). Mean hit rates, false alarm rates, corrected recognition scores, and discriminability scores for these selected subjects are summarized in Table 4. An unpaired t-test on discriminability scores confirmed that the two groups are matched on their performance in the no cue condition (t [12] < 1).

Table 4.

Hit rates (endorsing studied items as “old”), false alarm rates (FA, endorsing unstudied items as “old”), corrected recognition scores (CR, hits - FA), discriminability scores (d′), and corresponding standard deviations (S.D.) in three experimental conditions (same cue, recombined cue, no cue) for a subset of amnesic patients and a subset of matched controls.

Experimental Conditions Amnesic Patients Matched Controls
Hits FA CR d′ Hits FA CR d′
Same Cue (S.D.) 0.67 (0.15) 0.28 (0.20) 0.38 (0.27) 0.87 (0.69) 0.81 (0.14) 0.15 (0.13) 0.66 (0.10) 1.85 (0.50)
Recombined Cue (S.D.) 0.67 (0.12) 0.28 (0.20) 0.38 (0.24) 0.87 (0.58) 0.65 (0.16) 0.15 (0.13) 0.51 (0.13) 1.37 (0.44)
No Cue (S.D.) 0.74 (0.19) 0.34 (0.12) 0.40 (0.12) 1.01 (0.43) 0.72 (0.12) 0.24 (0.17) 0.48 (0.13) 1.27 (0.51)
Open in a new tab

Hits, false alarms, d′ and corrected recognition from the subset of subjects were analyzed separately in 2 (group: amnesics, controls) × 3 (condition: no cue, same cue, recombined cue) mixed ANOVAs.

The analysis of hit rates revealed a marginally significant group x condition interaction (F [2, 24] = 2.60, p = 0.09). Paired t-tests of control subjects’ performance revealed that their performance was higher in the presence of the same cue than in the presence of a recombined cue (t [6] = 2.80, p < 0.05). No other pairwise comparisons were significant (all p’s > 0.10). On the other hand, amnesic patients performed similarly across all three conditions, suggesting that cue availability does not affect their recognition performance (all pairwise comparisons yielded p’s > 0.15). Main effects of group and condition were not significant (all p’s > 0.10).

The false alarms analysis revealed no significant effects (all p’s > 0.10).

The d′ analysis indicated a significant main effect of group (F [1, 12] = 5.65, p < 0.05), indicating that overall d′ across the three conditions was significantly higher for the control group (mean = 1.50) than for the patient group (mean = 0.92). Critically, a significant group x condition interaction was also found (F [2, 24] = 4.18, p < 0.05). Paired t-tests of control subjects’ performance revealed that their performance was higher in the presence of the same cue than in the presence of a recombined cue (t [6] = 3.07, p < 0.05), and their performance in the same cue condition was also significantly higher than when no cue was provided (t [6] = 3.49, p < 0.05). Furthermore, the presence of a recombined cue, relative to no cue, had no influence on recognition performance (t [7] < 1). On the other hand, amnesic patients performed similarly across all three conditions, suggesting that cue availability does not affect their recognition performance (all pairwise comparisons yielded p’s > 0.50). The main effect of condition was not significant (p > 0.10). Analyses with corrected recognition scores revealed largely the same pattern.

Finally, to examine whether the hippocampus itself is critical to associative memory, we compared recognition performance between patients with lesions restricted to the hippocampus (H group, n = 2) and patients with lesions extending to the surrounding cortices (H-plus group, n = 3). If the hippocampus, and not the surrounding cortices, is critical to associative memory, we would expect a similar pattern of performance between the two sub-groups of patients. Performance in the two groups is virtually identical in the three conditions. In the same cue, recombined cue, and no cue conditions, patients in the group had mean discriminability scores of 0.48 (SD = 0.78), 0.48 (SD = 0.13), and 0.83 (SD = 0.63), respectively, and patients in the H-plus group had discriminability scores of 0.52 (SD = 0.63), 0.51 (SD = 0.36), and 0.84 (SD = 0.40). Analyses with corrected recognition revealed the same pattern. The failure to benefit from relational information in the two groups suggests that the hippocampus, and not the surrounding cortices, is critical to associative memory.

4. Discussion

In the present study, we examined the contribution of associative memory to item recognition. After studying cue-target pairs, amnesic and control subjects were tested on their recognition of single target items in the absence of a cue (no cue condition), in the presence of originally presented cues (same cue condition), and in the presence of cues that had been presented with different targets (recombined cue condition). By making previously studied associative information available at test (same cue), we explored whether amnesic individuals, like controls, will make incidental use of that information. According to the relational memory view, which states that the hippocampus is particularly important in binding previously unrelated pieces of information, we hypothesized that amnesic patients with damage to the hippocampus are unlikely to experience a same cue benefit. We found that whereas control subjects’ performance was enhanced by the presence of the same cue, amnesic subjects failed to experience a same cue benefit, providing additional evidence for the role of the hippocampus in relational memory. Our data also suggest that the failure to benefit from relational information is not modulated by the extent of medial temporal lesion, although it should be acknowledged that this particular comparison was based on a small sample size.

It is important to note that amnesic patients are severely impaired at item memory. In an effort to control scaling effects by matching item memory between the amnesic group and the control group, we provided additional presentations (six times) to patients compared to controls (once). Even under such conditions, we were not entirely successful at matching item memory performance between the two groups. This observation is consistent with the relational memory view, which states that even simple associations (i.e., item-context) require relational processing to some degree. Most importantly, however, amnesics were disproportionately impaired when the relational processing demand increased.

A question arises as to whether amnesic patients’ failure to benefit from the same cue might be due to the fact that word pairs were presented multiple times at study. In other words, could it be the case that multiple presentations actually disrupted, rather than promoted, formation of new associations? Perhaps overexposure to the stimulus pairs interfered with binding, and the same cue benefit experienced by controls might also be diminished if they had received multiple presentations. Arguing against this possibility, Chalmers and Humphreys (2003), in a study of healthy individuals, found that multiple presentations improved associative recognition of low frequency items and had no effect on associative recognition of high frequency items. Thus, amnesics’ failure to benefit from the same cue cannot be explained by a disruption of binding as a result of repeated presentations.

The same cue benefit observed in control subjects is consistent with previous reports of individuals performing better in item recognition when the target is accompanied by the same cue compared to when the target is accompanied by a different cue (Chalmers & Humphreys, 2003; Clark & Shiffrin, 1992; Humphreys, 1976, 1978). It has been suggested that the associative information available in the same cue condition serves as an alternative route to target recognition. Our findings suggest that whereas this alternative route is available to control subjects, it is not accessible to amnesic patients.

One interpretation of the differential performance of controls and patients concerns the underlying representations of the cue-target pairs formed at study. When a cue-target pair is studied, a cue representation, a target representation, and a representation of the cue-target association are formed. As such, healthy control subjects are able to rely on both the target representation and the cue-target association to support recognition judgment. Eichenbaum and colleagues (Eichenbaum & Cohen, 2001; Eichenbaum, Schoenbaum, Young, & Bunsey, 1996) have argued that the hippocampus is essential to the formation of flexible associations between items (see also Moses & Ryan, 2006). Although damage to the hippocampus has little effect on item representations, the nature of the associative representations is altered as a result of hippocampal damage. Whereas individuals without hippocampal damage can form flexible associations between items, individuals with damage to the hippocampus can only form associative representations that are fused and inflexible. Such a rigid representation can only be retrieved as a single unit and cannot be separated into its distinct elements. As such, it may be used to support recognition when a cue-target pair can be evaluated holistically, but not when a target item is probed separately.

Consistent with this idea, previous studies of implicit memory for new perceptual associations in amnesia have reported intact associative priming when, at test, studied items are re-presented and evaluated in exactly the same way as at study (Gabrieli, Keane, Zarella, & Poldrack, 1997; Goshen-Gottstein, Moscovitch, & Melo, 2000). By contrast, associative priming in amnesia is impaired when fused representations formed at study are disrupted, such as when each component of a stimulus pair is presented sequentially (e.g., Carlesimo, Perri, Costa, Serra, & Caltagirone, 2005; Paller & Mayes, 1994). Furthermore, Mayes and colleagues (2004) reported normal associative recognition and impaired cued recognition in an amnesic patient with a restricted hippocampal lesion (Y.R.). Moses and Ryan (2006) suggested that Y.R.’s normal associative recognition performance could be supported by fused representations, but such representations could not support recognition of individual items in the cued recognition task because components within the fused representations could not be separated into distinct elements. Similarly, in the context of the current experiment, the task demand of responding to the target words alone cannot be supported by fused representations. Thus, only target representations are available to support amnesic patients’ performance in the cued recognition task.

Another, not mutually exclusive, explanation of amnesic patients’ failure to benefit from the same cue focuses on the processes, rather than the representations, that are involved in recognition. Dual-process views of recognition state that recognition is mediated by two different processes: recollection and familiarity (Jacoby, 1991; Mandler, 1980). Recollection refers to the intentional retrieval of the study episode and familiarity refers to the knowledge of previous encounter because of an increased fluency in processing, but not the memory of the study episode itself. It has been demonstrated that whereas associative recognition is more reliant on recollection than on familiarity, item recognition is more reliant on familiarity than on recollection (Hockley, 1991, 1992; Hockley & Consoli, 1999).

In the context of cued recognition, the same cue benefit experienced by control subjects may be attributable to recollection of associative information. When individuals with intact memory are unable to retrieve information about a target item in the same cue condition, they may use the cue and the intact associative link to assess the status of the target, thus providing another route to successful recognition. On the other hand, in the recombined condition, recollection of associative information does not aid recognition because the cue is from a different studied pair, and thus does not inform the status of the target. In the no cue condition, associative information is simply not available. Amnesic patients, however, are unable to benefit from the same cue because their severe impairment in recollection prevents them from recollecting the associative information (Giovanello, Keane, & Verfaellie, 2006; Verfaellie & Treadwell, 1993; Yonelinas, Kroll, Dobbins, Lazzara, & Knight, 1998). As such, amnesic patients’ performance in all three conditions is primarily supported by familiarity. Their similar performance across the three conditions (same cue, recombined cue, no cue) is consistent with this view.

The present findings do not distinguish between the representation view and the process view because our data are equally compatible with both accounts. However, there is evidence that amnesic patients with damage including the hippocampus are impaired at associative recognition even when the studied pairs are evaluated together at test (Giovanello et al., 2003). These data present a challenge to the representation view because despite the presence of fused representations, amnesics’ associative recognition was impaired. Further evidence consistent with the process view is that amnesic patients’ associative recognition performance is modulated by familiarity, such that amnesic patients’ performance is improved when such recognition can be enhanced by familiarity for the association (Giovanello et al., 2006). Taken together, data from these two experiments seem to favor a process view, although future studies directly pitting the two views against each other in a single experiment will be critical to resolve this issue.

In a recent paper, Gold, Hopkins, and Squire (2006) proposed that the discrepant findings in the status of associative memory in amnesia may be a byproduct of a procedural difference between previous experiments. Whereas Stark, Bayley, and Squire (2002) employed a “separate test” procedure (i.e., item memory and associative memory tested in separate blocks), Giovanello, Verfaellie, and Keane (2003) used a “combined test” procedure (i.e., item memory trials and associative memory trials intermixed together). Gold, Hopkins, and Squire argued that since the task instructions in the combined test are more complicated than those in the “separate test,” amnesic patients may have difficulty keeping track of the tasks, which may have contributed to their disproportionate impairment in associative recognition. The authors examined this possibility by testing the same group of patients under both testing conditions and found disproportionate impairment in associative recognition in the “combined test” condition but not in the “separate test” condition (but see Turriziani et al., 2004, in which relative impairment in associative memory was found in amnesic patients even when item and associative tests were administered separately). This procedural difference is eliminated in the current study, in which task instructions and requirements remained the same for all conditions. This suggests that amnesic patients’ disproportionate impairment in associative memory is not simply an artifact of task difficulty.

Previous studies that have examined the role of the hippocampus in relational memory revealed that amnesic patients are impaired at explicit retrieval of associative information. Extending these observations, our data illustrate that amnesic patients are also impaired at incidental use of associative information. That is, the presence of associative information at test has no influence on amnesic patients’ recognition performance. These data provide converging evidence that the hippocampus plays a critical role in relational memory.

Acknowledgments

This research was supported by National Institutes of Health grant MH 57681 and the Medical Research Service of the Department of Veterans Affairs. We thank Katie Page and Elizabeth Martin for their assistance in data collection and Steve Hodge for his assistance in volumetric analyses.

Footnotes

1

Another approach to equating performance in the no cue condition between the two groups is to compare data from the highest performing amnesics with data from the lowest performing controls. We selected five individuals from each group, and the two groups are well matched on item recognition performance, with a mean difference in d′ of 0.10 (t [8] = 0.36, p = 0.73). A 2 (group) × 3 (conditions) mixed ANOVA on discriminability scores revealed the same general pattern as the analysis reported in the Results section. Critically, the condition x group interaction was also marginally significant, despite the smaller sample size (F [2, 16] = 3.23, p = 0.07). Paired t-tests of control subjects’ performance revealed that their performance was higher in the presence of the same cue than in the presence of a recombined cue (t [4] = 2.94, p < 0.05) or no cue (t [4] = 6.68, p < 0.01). Furthermore, the presence of a recombined cue, relative to no cue, had no influence on recognition performance (t [4] = 0.71, p = 0.52). On the other hand, amnesic patients performed similarly across all three conditions, suggesting that cue availability does not affect their recognition performance (all pairwise comparisons yielded p’s > 0.30).

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Achim AM, Lepage M. Neural correlates of memory for items and for associations: An event-related functional magnetic resonance imaging study. Journal of Cognitive Neuroscience. 2005;17:652–667. doi: 10.1162/0898929053467578. [DOI] [PubMed] [Google Scholar]
  2. Bhatia S, Bookheimer SY, Gaillard WD, Theodore WH. Measurement of whole temporal lobe and hippocampus for MR volumetry: Normative data. Neurology. 1993;43:2006–2010. doi: 10.1212/wnl.43.10.2006. [DOI] [PubMed] [Google Scholar]
  3. Broadbent NJ, Clark RE, Zola S, Squire LR. The medial temporal lobe and memory. In: Squire LR, Schacter DL, editors. Neuropsychology of Memory. 3. New York: Guilford Press; 2002. pp. 3–23. [Google Scholar]
  4. Carlesimo GA, Perri R, Costa A, Serra L, Caltagirone C. Priming for novel between-word associations in patients with organic amnesia. Journal of the International Neuropsychological Society. 2005;11:566–573. doi: 10.1017/S1355617705050678. [DOI] [PubMed] [Google Scholar]
  5. Caviness VS, Jr, Meyer JW, Makris N, Kennedy DN. MRI-based topographic parcellation of the human neocortex: An anatomically specified method with estimate of reliability. Journal of Cognitive Neuroscience. 1996;8:566–587. doi: 10.1162/jocn.1996.8.6.566. [DOI] [PubMed] [Google Scholar]
  6. Chalmers KA, Humphreys MS. Experimental manipulation of prior experience: Effects on item and associative recognition. Memory. 2003;11:233–246. doi: 10.1080/09658210244000009. [DOI] [PubMed] [Google Scholar]
  7. Clark SE, Shiffrin RM. Cuing effects and associative information in recognition memory. Memory & Cognition. 1992;20:580–598. doi: 10.3758/bf03199590. [DOI] [PubMed] [Google Scholar]
  8. Coltheart M. The MRC Psycholinguistic Database. Quarterly Journal of Experimental Psychology A: Human Experimental Psychology. 1981;33A:497–505. [Google Scholar]
  9. Davachi L, Wagner AD. Hippocampal contributions to episodic encoding: insights from relational and item-based learning. Journal of Neurophysiology. 2002;88:982–990. doi: 10.1152/jn.2002.88.2.982. [DOI] [PubMed] [Google Scholar]
  10. Eichenbaum H, Cohen NJ. From conditioning to conscious recollection: Memory systems of the brain. New York: Oxford University Press; 2001. [Google Scholar]
  11. Eichenbaum H, Schoenbaum G, Young B, Bunsey M. Functional organization of the hippocampal memory system. Proceedings of the National Academy of Sciences of the United States of America. 1996;93(24):13500–13507. doi: 10.1073/pnas.93.24.13500. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Filipek PA, Kennedy DN, Caviness VS., Jr A method of morphometric analysis of the human brain based upon magnetic resonance imaging. Annals of Neurology. 1988;24:356. [Google Scholar]
  13. Filipek PA, Richelme C, Kennedy DN, Caviness VS., Jr The young adult human brain: An MRI-based morphometric analysis. Cerebral Cortex. 1994;4:344–360. doi: 10.1093/cercor/4.4.344. [DOI] [PubMed] [Google Scholar]
  14. Free SL, Bergin PS, Fish DR, Cook MJ, Shorvon SD, Stevens JM. Methods for normalization of hippocampal volumes measured with MR. AJNR American Journal of Neuroradiology. 1995;16:637–643. [PMC free article] [PubMed] [Google Scholar]
  15. Gabrieli JDE, Keane MM, Zarella MM, Poldrack RA. Preservation of implicit memory for new associations in global amnesia. Psychological Science. 1997;8:326–329. [Google Scholar]
  16. Giovanello KS, Keane MM, Verfaellie M. The contribution of familiarity to associative memory in amnesia. Neuropsychologia. 2006;44:1859–1865. doi: 10.1016/j.neuropsychologia.2006.03.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Giovanello KS, Schnyer DM, Verfaellie M. A critical role for the anterior hippocampus in relational memory: Evidence from an fMRI study comparing associative and item recognition. Hippocampus. 2004;14:5–8. doi: 10.1002/hipo.10182. [DOI] [PubMed] [Google Scholar]
  18. Giovanello KS, Verfaellie M. The relationship between recall and recognition in amnesia: effects of matching recognition between patients with amnesia and controls. Neuropsychology. 2001;15:444–451. doi: 10.1037//0894-4105.15.4.444. [DOI] [PubMed] [Google Scholar]
  19. Giovanello KS, Verfaellie M, Keane MM. Disproportionate deficit in associative recognition relative to item recognition in global amnesia. Cognitive, Affective & Behavioral Neuroscience. 2003;3:186–194. doi: 10.3758/cabn.3.3.186. [DOI] [PubMed] [Google Scholar]
  20. Gold JJ, Hopkins RO, Squire LR. Single-item memory, associative memory, and the human hippocampus. Learning & Memory. 2006;13:644–649. doi: 10.1101/lm.258406. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Goshen-Gottstein Y, Moscovitch M, Melo B. Intact implicit memory for newly formed verbal associations in amnesic patients following single study trials. Neuropsychology. 2000;14:570–578. doi: 10.1037//0894-4105.14.4.570. [DOI] [PubMed] [Google Scholar]
  22. Henke K, Buck A, Weber B, Wieser HG. Human hippocampus establishes associations in memory. Hippocampus. 1997;7:249–256. doi: 10.1002/(SICI)1098-1063(1997)7:3<249::AID-HIPO1>3.0.CO;2-G. [DOI] [PubMed] [Google Scholar]
  23. Hockley WE. Recognition memory for item and associative information: A comparison of forgetting rates. In: Hockley WE, Lewandowsky S, editors. Relating theory and data: Essays on human memory in honor of Bennet B. Murdock. Hillsdale, NJ: Erlbaum; 1991. pp. 227–248. [Google Scholar]
  24. Hockley WE. Item versus associative information: Further comparisons of forgetting rates. Journal of Experimental Psychology: Learning, Memory, and Cognition. 1992;18:1321–1330. [Google Scholar]
  25. Hockley WE, Consoli A. Familiarity and recollection in item and associative recognition. Memory & Cognition. 1999;27:657–664. doi: 10.3758/bf03211559. [DOI] [PubMed] [Google Scholar]
  26. Hockley WE, Cristi C. Tests of encoding tradeoffs between item and associative information. Memory & Cognition. 1996;24:202–216. doi: 10.3758/bf03200881. [DOI] [PubMed] [Google Scholar]
  27. Humphreys MS. Relational information and the context effect in recognition memory. Memory & Cognition. 1976;4:221–232. doi: 10.3758/BF03213167. [DOI] [PubMed] [Google Scholar]
  28. Humphreys MS. Item and relational information: A case for context independent retrieval. Journal of Verbal Learning & Verbal Behavior. 1978;17:175–187. [Google Scholar]
  29. Isaac CL, Mayes AR. Rate of forgetting in amnesia: I. Recall and recognition of prose. Journal of Experimental Psychology: Learning, Memory, and Cognition. 1999a;25:942–962. doi: 10.1037//0278-7393.25.4.942. [DOI] [PubMed] [Google Scholar]
  30. Isaac CL, Mayes AR. Rate of forgetting in amnesia: II. Recall and recognition of word lists at different levels of organization. Journal of Experimental Psychology: Learning, Memory, and Cognition. 1999b;25:963–977. doi: 10.1037//0278-7393.25.4.963. [DOI] [PubMed] [Google Scholar]
  31. Jacoby LL. A process dissociation framework: Separating automatic from intentional uses of memory. Journal of Memory & Language. 1991;30:513–541. [Google Scholar]
  32. Kucera H, Francis WN. Computational analysis of present-day American English. Providence, Rhode Island: Brown University Press; 1967. [Google Scholar]
  33. Mandler G. Reocgnizing: The judgment of previous occurrence. Psychological Review. 1980;87:252–271. [Google Scholar]
  34. Mayes AR, Holdstock JS, Isaac CL, Montaldi D, Grigor J, Gummer A, et al. Associative recognition in a patient with selective hippocampal lesions and relatively normal item recognition. Hippocampus. 2004;14:763–784. doi: 10.1002/hipo.10211. [DOI] [PubMed] [Google Scholar]
  35. Moses SN, Ryan JD. A comparison and evaluation of the predictions of relational and conjunctive accounts of hippocampal function. Hippocampus. 2006;16:43–65. doi: 10.1002/hipo.20131. [DOI] [PubMed] [Google Scholar]
  36. Nelson DL, McEvoy CL, Schreiber TA. The University of South Florida word association, rhyme, and word fragment norms. 1998 doi: 10.3758/bf03195588. (Publication. Retrieved December 5, 2006: http://www.usf.edu/FreeAssociation/ [DOI] [PubMed]
  37. Paller KA, Mayes AR. New-association priming of word identification in normal and amnesic subjects. Cortex. 1994;30:53–73. doi: 10.1016/s0010-9452(13)80324-3. [DOI] [PubMed] [Google Scholar]
  38. Rademacher J, Galaburda AM, Kennedy DN, Filipek PA, Caviness VS., Jr Human cerebral cortex: Localization, parcellation and morphometry with magnetic resonance imaging. Journal of Cognitive Neuroscience. 1992;4:352–374. doi: 10.1162/jocn.1992.4.4.352. [DOI] [PubMed] [Google Scholar]
  39. Rudy JW, Sutherland RJ. Configural association theory and the hippocampal formation: An appraisal and reconfiguration. Hippocampus. 1995;5:375–389. doi: 10.1002/hipo.450050502. [DOI] [PubMed] [Google Scholar]
  40. Seidman LJ, Faraone SV, Goldstein JM, Kremen WS, Horton NJ, Makris N, et al. Left hippocampal volume as a vulnerability indicator for schizophrenia: A magnetic resonance imaging morphometric study of nonpsychotic first-degree relatives. Archives of General Psychiatry. 2002;59:839–849. doi: 10.1001/archpsyc.59.9.839. [DOI] [PubMed] [Google Scholar]
  41. Smith SM. Fast robust automated brain extraction. Human Brain Mapping. 2002;17:143–155. doi: 10.1002/hbm.10062. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Stark CEL, Bayley PJ, Squire LR. Recognition memory for single items and for associations is similarly impaired following damage to the hippocampal region. Learning & Memory. 2002;9:238–242. doi: 10.1101/lm.51802. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Stark CEL, Squire LR. Hippocampal damage equally impairs memory for single items and memory for conjunctions. Hippocampus. 2003;13:281–292. doi: 10.1002/hipo.10085. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Stout JC, Bondi MW, Jernigan TL, Archibald SL, Delis DC, Salmon DP. Regional cerebral volume loss associated with verbal learning and memory in dementia of the Alzheimer type. Neuropsychology. 1999;13:188–197. doi: 10.1037//0894-4105.13.2.188. [DOI] [PubMed] [Google Scholar]
  45. Sutherland RJ, Rudy JW. Configural association theory: The role of the hippocampal formation in learning, memory, and amnesia. Psychobiology. 1989;17:129–144. [Google Scholar]
  46. Turriziani P, Fadda L, Caltagirone C, Carlesimo GA. Recognition memory for single items and for associations in amnesic patients. Neuropsychologia. 2004;42:426–433. doi: 10.1016/j.neuropsychologia.2003.10.003. [DOI] [PubMed] [Google Scholar]
  47. Verfaellie M, Treadwell J. Status of recognition memory in amnesia. Neuropsychology. 1993;7:5–13. [Google Scholar]
  48. Wechsler S. Wechsler Adult Intelligence Scale, III. San Antonion, TX: Psychological Corporation; 1997. [Google Scholar]
  49. Yonelinas AP, Kroll NEA, Dobbins I, Lazzara M, Knight RT. Recollection and familiarity deficits in amnesia: Convergence of remember-know, process dissociation, and receiver operating characteristic data. Neuropsychology. 1998;12:323–339. doi: 10.1037//0894-4105.12.3.323. [DOI] [PubMed] [Google Scholar]

RESOURCES