Task context and organization in free recall

Abstract

Prior work on organization in free recall has focused on the ways in which semantic and temporal information determine the order in which material is retrieved from memory. Tulving’s theory of ecphory suggests that these organizational effects arise from the interaction of a retrieval cue with the contents of memory. Using the continual-distraction free-recall paradigm (Bjork & Whitten, 1974) to minimize retrieval during the study period, we show that encoding task context can organize recall, suggesting that task-related information is part of the retrieval cue. We interpret these results in terms of the Context Maintenance and Retrieval model (CMR; Polyn, Norman, & Kahana, in press), in which an internal contextual representation, containing semantic, temporal, and source-related information, serves as the retrieval cue and organizes the retrieval of information from memory. We discuss these results in terms of the guided activation theory (Miller & Cohen, 2001) of the role of prefrontal cortex in task performance, as well as the rich neuropsychological literature implicating prefrontal cortex in memory search (e.g, Schacter, 1987).

Introduction

The behavioral investigation of human memory has been of interest to neuropsychologists for several decades. Certain behavioral tests, in particular free recall, clearly display the memory deficit associated with a number of types of brain damage (Scoville & Milner, 1957; Wheeler, Stuss, & Tulving, 1995), neurological disorders (Aleman, Hijman, Haan, & Kahn, 1999; Backman, Small, & Fratiglioni, 2001; Bennett, Golob, Parker, & Starr, 2006), and healthy aging (Wingfield & Kahana, 2002; Golomb, Peelle, Addis, Kahana, & Wingfield, 2008). Free recall, in its standard form, involves learning a series of unrelated words, followed by recall of these words in any order. Neuropsychological investigations of memory performance tend to focus on the proportion of studied material recalled. However, this focus on memorability ignores the fact that the studied material is retrieved in a particular order, which reflects the underlying organization of that material in memory. A number of prominent studies have documented the organizational difficulties exhibited by various patient groups in free recall. For example, frontally damaged patients exhibit deficits in subjective organization¹ (Stuss et al., 1994; Gershberg & Shimamura, 1995), as well as organization by semantic category (Hildebrandt, Brand, & Sachsenheimer, 1998; Jetter, Poser, Freeman, & Markowitsch, 1986). This work is reviewed and simulated in a study by Becker and Lim (2003). Patients with Alzheimer’s disease show severe impairments in their ability to use category information to help them recall to-be-remembered materials, and they show almost no organization of categorized materials when it is remembered (Weingartner et al., 1981). Finally, a study by Kahana, Howard, Zaromb, and Wingfield (2002) examined differences between younger and older adults, and showed that older adults exhibit less temporal organization (the tendency to successively recall items that were contiguous on the study list) than younger adults during free recall.

In recent years, clinicians have discovered that patients who will go on to develop Alzheimer’s disease (e.g., Grober, Lipton, Hall, & Crystal, 2000) exhibit significant reductions in performance on free recall tasks (measured by proportion of items recalled) several years prior to exhibiting severe cognitive decline. While proportion recalled in free recall is a sensitive test of the state of the human memory system, there is still much room for improvement before such tests can be used to predict, in a useful way, which specific patients will go on to develop these disorders (Tian, Bucks, Haworth, & Wilcock, 2003). Clearly, increasing the sensitivity of these tests is a high priority, and this will involve both refinement of the testing procedures, and development of more sensitive analyses of the collected behavioral data, such as variants of the organizational measures introduced above. These efforts will in turn be aided by basic theoretical work aimed at elucidating the functional structure of the human memory system.

Studies of memory organization primarily examine the phenomenon of clustering, the tendency for particular studied items to be retrieved successively during memory search. Analyses of clustering during recall show that words that are similar along some dimension are recalled successively. Researchers in this area have reported organization of recalled material according to semantic similarity (Howard & Kahana, 2002b), temporal proximity (Wallace, 1970; Kahana, 1996), and several types of source characteristics, such as spatial location (Curiel & Radvansky, 1998), modality (auditory or visual, Murdock & Walker, 1969), gender of the speaker, typeface of the word (Hintzman, Block, & Inskeep, 1972; Nilsson, 1974), and judgment task used to study the word (Polyn et al., in press). The tendency for people to organize studied material by source characteristics may provide a sensitive marker of the overall health of the memory system, as this form of clustering relies on associations formed between the studied material and source characteristics during the study episode (as opposed to semantic organization, which relies on prior knowledge of the studied material).

Associative theories of human memory (e.g., Kahana, Howard, & Polyn, 2008) suggest that organizational phenomena arise from associations between an internally maintained retrieval cue, and the features of the studied material (e.g., Howard & Kahana, 2002a). Our modern understanding of the retrieval process owes much to the work of Endel Tulving (Tulving, 1962, 1972; Tulving & Donaldson, 1972; Tulving, 1974, 1979). Tulving focused theoretical attention on the importance of understanding the dynamics occurring during retrieval itself, as opposed to the tendency in the literature to focus exclusively on the encoding process (Tulving, 1983).² Tulving’s work highlighted the importance of organization in understanding memory search, and specifically, the role of the retrieval cue in determining the relative probability with which various studied items will be retrieved from memory. Free recall provides us with a paradigm where the ability of a participant to successfully retrieve studied material depends on the moment-to-moment composition of an internally maintained retrieval cue, and the associations between this cue and stored information. Tulving referred to this interaction between the retrieval cue and stored information as ecphory (Tulving, 1983; Semon, 1921), and established its theoretical importance in a series of studies (Tulving & Thompson, 1973; Wiseman & Tulving, 1976; Bartling & Thompson, 1977).

Understanding the dynamics of the retrieval process, through the development of analyses of memory organization, is essential to enhancing the sensitivity of behavioral measures used to assess free-recall performance. This effort is aided by the development of formal computational models of the memory system, which allow us to understand the processes underlying this organization. Recently, Polyn et al. (in press) introduced the Context Maintenance and Retrieval (CMR) model, a generalized version of the temporal context model (TCM) of Howard and Kahana (2002a). The CMR model proposes that organizational effects observed during memory search arise because the recall process is guided by an internally maintained context representation. This context representation contains information related to the semantic characteristics, temporal context, and source attributes (such as encoding task) of the studied material. As an item is studied, the representation of the item is associated with the current state of the context representation. Later, during memory search, the context representation can be used to reactivate the representations of studied items (causing an item to be remembered). When an item is remembered, it brings with it details of its previous occurrences, causing context to more strongly resemble the state it was in when the recalled item was originally studied. This updated context representation is then used to probe memory, causing items that are similar to the just recalled item to be recalled next. Items can be similar along a number of dimensions, including semantic (items that often appear together in text), temporal (items that were in nearby positions on the list), or source (items that were studied using the same task).

The above mentioned study by Polyn et al. (in press) showed that the source clustering by encoding task observed during an immediate free-recall paradigm was due in part to associations formed between a task representation and the representations of the studied items. However, one question that arose during the analysis of these data was to what degree this organization by source depends on processes carried out during the study period, versus processes carried out during search itself. As a number of theorists have pointed out, using data from the overt rehearsal variant of the free-recall paradigm, the pattern of item rehearsals engaged in by the participant during the study period is an important predictor of later recall organization (Rundus & Atkinson, 1970; Tan & Ward, 2000; Laming, 2006). As such, it is difficult to determine whether the organizational effects observed by Polyn et al. (in press) were influenced by covert rehearsal during the study period, or primarily by the retrieval cue during the recall period. In order to isolate the contribution of the composition of the retrieval cue to organizational phenomena, we used a variant of the continual-distraction free-recall paradigm (Bjork & Whitten, 1974), in which participants engage in distracting mental activity before and after each item on the study list. This drastically reduces the ability of participants to engage in covert rehearsal of the study material, and should afford us a more pure estimate of the contribution of the retrieval cue to the organization of studied material.

We created a set of lists in which all items were studied with the same encoding task, and another set in which the encoding task changed halfway through the list. We found that despite the addition of a distraction task during the study period, there were still strong behavioral effects of the encoding task manipulation on the memorability of items, the organization of recall, and the latency with which items were recalled. While this article does not present a detailed formal analysis of the CMR model in this domain, we discuss how these results are consistent with the CMR framework, as well as the difficulties other models might have explaining these data.

Methods

24 Undergraduates at Princeton University (17 female) participated in this experiment for course credit and payment. Participants were run in a variant of the continual-distraction free-recall paradigm (Bjork & Whitten, 1974). Each participant performed 12 trials of the paradigm (the first 4 trials were practice and were not included in the present analysis). On each trial, 12 items were presented on a computer screen. Each item was judged using one of two tasks, a pleasantness judgment (“good” or “bad”) and a size judgment (“big” or “small”, relative to a shoebox). Each study item was preceded by a task cue (for 1.5 seconds), after which the study item appeared for another 1.5 seconds. Subjects indicated their judgment with a key press. After each stimulus presentation subjects performed a distraction task for 9 seconds (counting backward by sevens from a three-digit number) before the next task cue appeared. A distraction period of the same duration preceded the list. After the final distraction period participants were given 50 seconds to verbally recall as many items from the list as they could in any order. Each trial was drawn from one of two conditions: a control condition where each item was studied with the same task, and a task-shift condition, where the shift took place between the sixth and seventh items.

Verbal responses were digitally recorded and scored for recall order and verbalization onset with software developed by the Kahana lab (pyParse; http://memory.psych.upenn.edu). A recall was classified as valid if the item recalled came from the current list. Items from previous lists, or from extra-experimental sources (intrusions) were not included in the current analyses. A given output transition between items during the recall period was considered valid if it was between two valid recalls.

Word lists were drawn from a subset of the Toronto Noun Pool (Friendly, Franklin, Hoffman, & Rubin, 1982); the subset excluded words inappropriate to the current encoding tasks (such as abstract words like “absence”). Lists were tested to ensure that the mean Ku cera-Francis frequency (Ku cera & Francis, 1967) value of the list fell within a certain range (20 to 50), and that the frequency variance fell within another range (100 to 6000). These numbers were arrived at by randomly generating a large number of sample lists, and inspecting histograms of mean and variance of frequency across these lists. The threshold values above were chosen to include the modes of the distributions, and exclude the long tails. Word-similarity scores derived with latent semantic analysis (Landauer & Dumais, 1997) were used to ensure that close semantic associates did not appear in the same list. A given list could not include two words with a similarity score exceeding 0.25.

Results

As described above, the Context Maintenance and Retrieval (CMR) model suggests that a representation of the encoding task used to study each item is maintained in an internal context representation and is then used to guide recall. A basic prediction of the theory is that the encoding task context used to process the most recent items will be active during the beginning of the recall period, increasing the likelihood that the post-shift items will be recalled. Our first analysis investigates the likelihood of recalling particular items as a function of list position (i.e. task identity, since participants shifted from one encoding task to the other halfway through the list in the task-shift condition). Since performance on the control lists for each of the two tasks was not significantly different (using a paired t-test on mean percent correct by encoding task; t(23) = −1.87; p > 0.05) all analyses of the effects of task-shift were conducted without regard to task identity.

We used an analysis of variance to compare the proportion of items recalled before and after the task shift in the task-shift condition (and from equivalent serial positions in the control condition). This two-by-two analysis of variance (list type: control or task-shift by list half: pre- or post-shift) showed no main effect of list type (F(1, 23) = 4.00; MSE = 0.21; p > 0.05), or of list half (F(1, 23) = 1.65; MSE = 0.17; p > 0.2). However, it did reveal a significant interaction between list type and list half (F(1, 23) = 4.81; MSE = 0.21; p < 0.05). A contrast investigating this interaction revealed that participants recalled significantly more post-shift items in the task-shift condition relative to the control condition (t(23) = −3.46; p < 0.05). No difference was observed between the pre-shift items in the two conditions (t(23) = 0.39; p > 0.1). In summary, items studied after the task-shift were more memorable than items from equivalent serial positions in the control lists, and there was no cost to the memorability of items studied prior to the task shift. We investigate this phenomenon further by inspecting the probability of recall by serial position (Figure 1, top plot), which shows that this advantage for the post-shift items is most pronounced for the items studied immediately after the task shift. The bottom plot in Figure 1 shows the difference in memorability for each serial position between the two conditions; the error bars are 95% confidence intervals on the subject means.

Figure 1.

The top plot shows the probability of recall by serial position for the two experimental conditions (control and task-shift). The bottom plot shows the difference in performance between the two conditions at each serial position, revealing an increased tendency to recall items following the task-shift. See text for associated statistics.

The CMR model predicts that post-shift items will not only be more memorable, but that the first recall should be a post-shift item, since the retrieval cue is more consistent with the context of these items. The probability of first recall curve shown in Figure 2 (top plot) provides insight regarding the composition of retrieval cue at the start of the recall period (Hogan, 1975; Howard & Kahana, 1999; Laming, 1999). Both conditions show a sizable long-term recency effect, as well as a primacy effect for the first list item in both conditions. Figure 2 also reveals an enhanced probability for the first post-shift item (serial position 7) to be recalled first in the task-shift condition, as compared to the control condition (t(23)=−2.46, p<0.05). Again, the bottom plot in Figure 2 shows the difference in probability of first recall for each serial position between the two conditions; the error bars are 95% confidence intervals on the subject means.

Figure 2.

The top plot shows the probability of first recall (PFR) by serial position for the two experimental conditions (control and task-shift). The bottom plot shows the difference in performance between the two conditions, and reveals an increased likelihood to first recall the first item studied after the task shift. See text for associated statistics.

The CMR model was developed to explain organizational effects in memory search. As mentioned, the core of this model is a context representation that contains information related to the encoding task used during the study period. When this same task representation is used to guide retrieval, we observe organization, or clustering, of items by task context. Task clustering can be measured by calculating the probability that a given recalled item is then followed by a same-task or a between-task transition. In order to demonstrate this organizational effect we compare the recall sequences from the task-shift condition to those from the control condition. If items were drawn from the list randomly, one would expect the probability of drawing two items from the same task successively to be distributed about 0.5. However, by the principle of contiguity (Kahana et al., 2008), items studied nearby in time tend to be recalled successively, and since by design same-task items are contiguous, this will inflate the probability of a “same-task” transition in the control lists (that is, between items that would be labeled as “same-task” in the task-shift lists). Indeed, Table 1 shows that the probability of making a same-task transition on relabeled items from the control condition is significantly above 0.5 (Wilcoxon signed rank test: z = −2.6; p < 0.01), due to the contiguity effect. Nevertheless, the organizational effect of task context in the task-shift condition leads to a significant increase of the probability of a same-task transition above this baseline (Table 1; Wilcoxon signed rank test: z = −2.86; p < 0.005)

Table 1.

	Prob. of same-task transition	Mean between-task inter-response time (msec)
Control	0.58 (0.03)	4699 (431)
Task-shift	0.70 (0.02)	6637 (666)

Note. Standard error is across subject means is reported in parentheses. See text for details of the statistics.

Besides reorganizing the contents of memory, task context also exhibits itself behaviorally in a longer inter-response time (Murdock & Okada, 1970) when a participant recalls an item from one task context followed by an item from the other task context (as compared to transitions between similar serial positions in the control condition). We assessed this difference by comparing the set of inter-response times for between-task transitions in the task-shift condition, with the set of inter-response times between equivalent serial positions in the control lists. As shown in Table 1, on average the inter-response times from the task-shift condition are longer by about 2 seconds (Wilcoxon rank sum test: z = −2.55; p < 0.05). Polyn et al. (in press) observed a similar increase in inter-response times with task shifts and interpreted it as a “recall shift cost”, analogous to the shift cost in response times seen in the task-switching literature (Allport, Styles, & Hsieh, 1994). They suggested that these increased inter-response times arose because items studied with different encoding tasks were associated with distinct retrieval cues, such that transitions between items studied with different tasks are slower because of the inconsistency in task context associated with the two items.

Inter-response times in free recall are sensitive to a number of factors; they increase exponentially with output position (Murdock & Okada, 1970), and they increase as a function of serial position lag between the two recalled items (Howard & Kahana, 2002b). Given the multiple sensitivities of inter-response times in free recall, it is important to determine whether this recall shift cost can be explained simply by a difference in mean output position or mean serial lag between the successively reported items. The mean output position for between-task transitions was 3.4 in the task-shift condition, and 3.2 for transitions between equivalent serial positions in the control condition, a difference which is not significant by a t-test across subject means (t(22) = 1.08; p > 0.2). The mean serial lag for between-task transitions was 6.2 in the task-shift condition and 6.1 in the control condition, which was also not significant by a t-test across subject means (t(22) = 0.14; p > 0.5).

A final analysis was conducted to ensure that more subtle differences in the distribution of output positions for between-task transitions in the two conditions could not be contributing to this effect. For each subject, we calculated their mean between-task inter-response time separately for each output position for the control and the task-shift condition. If a given subject had no valid inter-response times for a particular output position in one of the conditions, that output position was excluded from the analysis. Then, for each subject and output position, we calculated the difference between the mean inter-response times for the two conditions, removing the effect of output position. We were then able to calculate the mean difference score for each participant, revealing an average increase in inter-response time of 2543 msec for between-task transitions in the task-shift condition. A t-test on these mean difference scores across participants was significant (t(22) = 3.19; p < 0.005).

Discussion

We found that the shift in task context that occurred midway through the study list altered the participant’s recall behavior in a number of ways, consistent with the hypothesis that task-related information in the retrieval cue is used to guide recall. First, items studied after the task shift were more likely to be recalled than similar items in a control condition, and there was no decrease in memorability for pre-shift items (Figure 1). Polyn et al. (in press) found a similar increase in likelihood of recall for the most recently studied items in a task-shift condition. However, they also found that items from earlier list positions were less likely to be recalled. They proposed that this cost for the memorability of earlier list items in the task-shift condition was due to the disruptive nature of the process of shifting back and forth between tasks. In the current paradigm, participants shift to an unrelated distraction task between every studied item, which in effect equalizes the number of task shifts on the control and shift lists (since one must shift to the distraction task before shifting back to the encoding task for the next item). In other words, the disruption introduced by constantly shifting to the distraction task and back may eliminate the disruptive effect of shifting from one encoding task to the other.

The organizational effects of introducing a task shift can be observed immediately in the recall period, with a significantly increased likelihood of the first post-shift item coming first in the recall sequence (Figure 2). In other words, there is a primacy effect for the set of same-task items following the mid-list shift in encoding task. The CMR model, although designed to explain source-related organizational effects, does not predict this effect, mostly because it was not designed to explain the primacy effect in free recall. In a standard free-recall paradigm, one might explain this effect by suggesting that the task shift disrupted some covert rehearsal process, forcing one to restart rehearsal with the first post-shift item. However, it is unlikely that the current effect is due to covert rehearsal, as the distraction task greatly reduces the ability of participants to engage in covert rehearsal during the study period. This may be evidence for a novelty-related enhancement of learning (Von Restorff, 1933), in which the mismatch between the new encoding task representation and the representation of the previous encoding task triggers more focused attention on the first post-shift item.

Task-related organizational effects are seen throughout the recall period, as summarized by the same-task transition probabilities in Table 1. The fact that we observe a large degree of task-related clustering (Table 1) in a paradigm where rehearsal (i.e., study-period reorganization) is minimized suggests that the mechanism underlying this effect is operating primarily during the recall period. This task clustering phenomenon is accompanied by an increase in inter-response times for recall transitions between items in different task contexts. All of these results are consistent with the CMR model; same-task items are likely to be recalled successively since they are associated with similar states of the context cue, while between-task transitions are slow, since the item being transitioned to is not associated with the same task context as the just-recalled item.

Many of the results described above seem as if they may be consistent with distinctiveness models of memory, such as SIMPLE (Brown, Neath, & Chater, 2007). Distinctiveness models explain many recall phenomena in terms of the relative discriminability of items along a number of dimensions, where items that are isolated on that dimension are more discriminable and therefore more likely to be recalled (e.g. an item with a large temporal distance between it and its neighbors is temporally discriminable and is therefore more likely to be recalled). Thus, when a set of items on a list are studied with a distinct encoding task from the other items, this establishes two sets of distinct items in memory (Bird, 1980). The predictions of distinctiveness models tend to focus on the memorability of particular studied items, and predictions of these models have not been worked out with regards to recall order and recall latency. One challenge for distinctiveness models relates to their lack of a context retrieval mechanism, whereby retrieving a particular item makes it more likely that a related item will then be recalled. Such a mechanism is critical for capturing the organizational effects observed during free recall.

As mentioned in the introduction, the CMR model is designed to explain the process by which the context-based retrieval cue interacts with the contents of memory to retrieve studied material. As explored by Polyn and Kahana (2008), a context-based model such as this makes a number of predictions for researchers interested in identifying the neural substrate of such a context representation. Three hypotheses central to the model are: 1) task-related information is activated when an item is studied, and is associated with the studied item; 2) when a given item is recalled, it reinstates the same task-related activity that was present when the item was originally studied; and 3) this task-related activity is then used as part of a retrieval cue to determine which item is recalled next.

These hypotheses make a number of clear predictions regarding the dynamics of a task context representation during a memory experiment. For example, it may be possible to use pattern-classification techniques (Polyn, Natu, Cohen, & Norman, 2005; Norman, Polyn, Detre, & Haxby, 2006) to identify the pattern of neural activity associated with each encoding task during the study period. One can then look for evidence of the reinstatement of these task-related patterns during the recall period. The CMR model makes the clear prediction that if one detects a task-related pattern, this will allow one to predict the task identity of the next recalled item, and that the strength of the task-related patterns will be the greatest when the participant is recalling a cluster of same-task items (as opposed to an isolated item from a particular task).

While direct neural evidence for the context representation is scarce, rich theories have arisen from the neuropsychological literature proposing that prefrontal cortex has a central role in the maintenance and manipulation of internal representations specifying the spatiotemporal context in which a given memory occurred (Schacter, 1987; Shimamura, 1994; Norman & Schacter, 1996; Moscovitch & Winocur, 2002). An emerging framework, described by Polyn and Kahana (2008), attempts to integrate these neuropsychological theories of the role of prefrontal structures in memory with the role of prefrontal cortex in task performance. Specifically, this framework builds upon the ideas of guided activation theory (Miller & Cohen, 2001), in which prefrontal cortex maintains patterns of activation relevant to the current behavioral task, such as features of salient stimuli (Braver et al., 2001) or task representations (Cohen, Dunbar, & McClelland, 1990). This notion that task-relevant information may be maintained during study (guiding the system to respond appropriately to stimuli) as well as during memory search (guiding the system to retrieve associated memories), may be an important step in bridging neural and behavioral theories of human memory.