From humans to rats and back again: Bridging the divide between human and animal studies of recognition memory with receiver operating characteristics

Abstract

Receiver operating characteristics (ROCs) have been used extensively to study the processes underlying human recognition memory, and this method has recently been applied in studies of rats. However, the extent to which the results from human and animal studies converge is neither entirely clear, nor is it known how the different methods used to obtain ROCs in different species impact the results. A recent study used a response bias ROC manipulation with rats and demonstrated that speeding memory responses reduced the contribution of recollection, not familiarity. The current study confirms this finding in humans using a comparable response bias method. Moreover, a comparison of the response bias methods commonly used in animal studies and the confidence rating method typically employed in human studies produced similar ROC functions. The present results suggest that the analysis of recognition memory ROCs provides a fruitful method to bridge the human and animal memory literatures.

It is well established that recognition memory can be based on recollection of qualitative details about the study episode or on assessments of familiarity (Mandler 1980; Jacoby 1991; Yonelinas 2002; Yonelinas et al. 2010). Evidence for this distinction comes from studies of human participants, as well as studies of rats and nonhuman primates (Aggleton and Brown 2006; Eichenbaum et al. 2007). However, a full integration of the human and animal literatures has been limited because the paradigms typically used in studies of humans to examine these processes are not suitable for other species. For example, recollection is often indexed in humans using verbal free recall paradigms or on the basis of subjective report methods where individuals indicate when recognition is accompanied by conscious recollection or only by feelings of familiarity (Mandler 1980; Tulving 1985).

One method that holds some promise in bridging the human and animal literatures is the analysis of receiver operating characteristics (ROCs) (Macmillan and Creelman 2005). This approach involves examining recognition memory across different levels of response bias (i.e., the probability of endorsing an item as previously studied) and can be used to extract estimates of recollection and familiarity (Yonelinas 2001; Yonelinas and Parks 2007). The method has been used quite extensively in the human literature, and has been found to lead to conclusions that converge with those based on other measurement methods such as free recall and subjective reports (for review, see Yonelinas 2002).

Recently, the ROC methodology has been applied to study recognition memory in rats (for review, see Eichenbaum et al. 2010; Sauvage 2010). For example, hippocampal lesions have been found to selectively impair recollection-based recognition, but do not impact familiarity-based recognition (Fortin et al. 2004; Sauvage et al. 2008). These results are consistent with ROC studies in humans, which have also shown that hippocampal damage disrupts recollection but has little or no effect on familiarity (Yonelinas et al. 2002; Aggleton et al. 2005). These convergent results suggest that the ROC methodology may allow one to bridge the study of recognition memory across species. However, whether this type of convergence is observed for other variables is not yet clear.

In a recent study, Sauvage and colleagues (2010) examined recognition ROCs in rats under self-paced and speeded responding conditions and demonstrated that speeding recognition responses selectively reduced the contribution of recollection to recognition memory, but had no influence on the extent to which familiarity contributed to performance. This finding is in agreement with human studies suggesting that the contents of familiarity are available earlier than the contents of recollection (Gronlund and Ratcliff 1989; Johnson et al. 1994; Yonelinas and Jacoby 1994, 1996; Toth 1996; McElree et al. 1999). However, none of the studies in the human literature to date have examined how speeding responses influence estimates of recollection and familiarity derived from recognition memory ROCs. The purpose of the present experiment was to rectify this.

To obtain recognition ROCs in rats, Sauvage and colleagues (2010) used a bias manipulation procedure. Rats were exposed to odors and, after a delay, were presented with previously presented odors and new odors. The rats were trained to dig for food if the odor was new or to refrain from digging and wait at the back of the test chamber for food if the odor was previously encountered. Response bias was manipulated by varying the amount of food a rat received for a correct old or new response, as well as the difficulty of digging in the test probe cup (i.e., the height of the cup). For example, in the most conservative bias condition, the probe cup was shallow and rats received half a piece of cereal for correct old and new responses. In contrast, in the most liberal condition the probe cup was deep (i.e., the reward was harder to retrieve) and rats received three pieces of cereal for making a correct old response (i.e., refraining from digging), whereas they only received half a piece of cereal when a correct new response was made. Five levels of response bias, each containing multiple study-test cycles, were used to plot recognition ROCs for both the self-paced and speeded testing conditions. Rats were given an unlimited time to respond in the self-paced condition, whereas on the speeded test rats were required to respond within half of their average reaction time from the same bias level in the self-paced condition. The dual-process signal detection (DPSD) model, which measures recollection as the degree of asymmetry (i.e., y-intercept) and familiarity as the degree of curvilinearity, was fit to the recognition ROCs to derive estimates of recollection and familiarity (Yonelinas 1994, 1999). The results showed that speeding recognition memory responses reduced the contribution of recollection, but did not influence familiarity (see Fig. 1A,B).

Figure 1.

Aggregate ROCs (left) and average estimates of recollection and familiarity (right) for rats from Sauvage et al. (2010) and humans in the present experiment. (A,B) Estimates of recollection and familiarity derived from recognition ROCs in rats from Sauvage et al. (2010) demonstrated that speeding responses selectively reduced the contribution of recollection, but not familiarity, to recognition memory (*P < 0.05). (C,D) The results from the current experiment using a confidence bias procedure replicate the findings of Sauvage et al. (2010) by showing that estimates of recollection were significantly reduced for participants in the speeded condition (N = 23) compared to participants in the self-paced condition (N = 20), whereas estimates of familiarity remained unchanged. (E,F) Recognition ROCs generated from the confidence bias (N = 20), payoff bias (N = 22), and standard confidence rating (N = 20) conditions under self-paced responding were similar in shape with no significant differences in estimates of recollection or familiarity between the three conditions. The lines on the aggregate ROCs represent the best fitting DPSD function. Error bars represent the standard error of the mean. (Data adapted from Sauvage et al. 2010.)

The purpose of the experiment presented here was to determine how speeding responses would impact recognition ROCs in humans. ROCs are typically obtained in humans by collecting response confidence rather that manipulating response bias. That is, for each test item participants indicate how confident they are on a six-point scale (e.g., “6-sure old” to “1-sure new”), and ROCs are plotted as a function of confidence such that the most conservative point is based only on the six responses, the next point includes both 6's and 5's, and so on. This standard confidence method, however, is not appropriate in the current context because forcing subjects to respond quickly might impair a participant's ability reliably use the confidence scale.

To overcome this potential confound, an ROC procedure was developed that manipulated response bias across five different test phases so that participants only had to make binary old/new decisions. Participants were instructed to only give an old response if they had a specific level of confidence or higher. Specifically, the level of confidence required for an old response was manipulated by showing participants standard six-point confidence scale with a green box placed around the levels of confidence required to give an old response. For example, on the most conservative test, the green box was placed around the 6-sure old confidence level and participants were instructed to only give an old response on this test if they would give a “6-sure old” response and to otherwise give a new response. Likewise, on the next most conservative test, the green box was placed around both the “5-maybe old” and “6-sure old” confidence levels and participants were instructed to respond old on this test if they would give the test item a 5 or 6 confidence response and to give a new response otherwise. One group of participants completed the procedure under speeded responding instructions at retrieval, whereas another group of participants completed the tests in a self-paced fashion.

After studying a list of 300 words, participants were instructed that they would complete five recognition memory tests. The level of confidence needed to make an old response varied across the five tests (see Supplemental Material). The five confidence bias test phases occurred in a random order for each participant and each contained 60 studied and 60 lure words. Participants in the self-paced condition had an unlimited time to make their old/new decision, whereas participants in the speeded condition had 1500 msec to make their response. If a response was not given before the deadline, a buzzer sounded to indicate that participants were late with their response. Participants were instructed to enter their response as quickly as possible.² Based on the results of Sauvage et al. (2010), we expected that speeding responses would lead to a decrease in overall recognition performance, and that this decrease would arise because of a decrease in recollection but not in familiarity.

The aggregate speeded and self-paced ROCs are plotted in Figure 1C, and Figure 1D shows the average recollection and familiarity estimates across participants. Overall recognition accuracy, measured with d′, was significantly higher for participants in the self-paced condition compared to the speeded condition, t₍₄₁₎ = 2.73; P < 0.01. This was driven by a numerical tendency for the hit rate to be higher and the false alarm rate to be lower in the self-paced condition, although these differences were not statistically reliable, both t's₍₄₁₎ = 1.51; both P's = 0.14. Finally, there was no difference in the overall criterion placement (i.e., c) between the two conditions, t₍₄₁₎ = 0.10; P = 0.92 (see Table 1).

Table 1.

Mean hit rate, false alarm rate, recognition accuracy (d′), criterion placement (c), and z-transformed ROC slope and intercept for participants in the speeded (N = 23) and self-paced (N = 20) confidence bias condition, the self-paced payoff bias condition (N = 22), and the standard confidence condition (N = 20)

In addition, the aggregate ROC presented in Figure 1C suggests that the decrease in overall recognition was caused by a selective decrease in recollection, as the projected y-intercept of the aggregate ROC, which indicates the contribution of recollection, appears to be higher in the self-paced condition, whereas the degree of curve, which measures the contribution of familiarity, appears to be similar between the two conditions. To quantify this difference, each participant's ROC was fit with the DPSD model to extract estimates of recollection and familiarity (see Supplemental Material). Consistent with our prediction, estimates of recollection in the speeded condition (M = 0.18; SE = 0.05) were significantly lower than in the self-paced condition (M = 0.37; SE = 0.06), t₍₄₁₎ = 2.46; P < 0.05. Importantly, there was no significant difference in familiarity estimates between participants in the speeded (M = 0.65; SE = 0.10) and self-paced (M = 0.72; SE = 0.12) conditions, t₍₄₁₎ = 0.45; P = 0.66 (see Fig. 1D). The above-mentioned results replicate those obtained by Sauvage and colleagues (2010) and extend those results to recognition memory ROCs in humans. These results demonstrate that the decrease in recognition accuracy caused by speeding responses at retrieval is caused by a selective reduction in the contribution of recollection. The results indicate that familiarity is available earlier than recollection, but it does not imply that familiarity would not have decreased if we had used a shorter response deadline (for evidence that both recollection and familiarity decrease with extremely short deadlines, see McElree et al. 1999).

To further evaluate the shape of the ROCs we examined the slope and intercept of the z-transformed ROC (zROC). The analysis indicated that the z-intercept, which is an index of overall recognition sensitivity or accuracy, was significantly greater in the self-paced than in the speeded condition, t₍₄₁₎ = 2.41; P < 0.05, and the z-slope, was numerically, albeit not significantly, larger in the speeded condition compared to the self-paced condition, t₍₄₁₎ = 1.49; P = 0.14 (Table 1). Additionally, the average z-slope was significantly less than one in both the speeded condition, t₍₂₂₎ = 10.31; P < 0.001, and the self-paced condition, t₍₁₉₎ = 8.98; P < 0.001.

Note that in line with prior studies of item recognition, both the DPSD and classic signal detection models (i.e., single-process models) fit the data almost perfectly with both accounting for >99% of the variance in the ROCs. Although these results do not differentiate between such models of recognition memory (Yonelinas and Parks 2007), previous work has suggested that the classic signal detection model is inadequate because it is unable to account for data from recognition tasks such as associative and source recognition (Yonelinas 1997, 1999; Sauvage et al. 2008; also see Koen and Yonelinas 2010), and results from studies of hippocampal amnesia (Yonelinas et al. 2002; Fortin et al. 2004).

Although the current results are consistent with the Sauvage et al. (2010) study, one potentially important difference that may complicate such comparisons is that Sauvage and colleagues manipulated response bias by varying the reward payoff in the different conditions, whereas our method did not involve differential payoffs. Conversely, relating the current results to other human ROC studies is complicated because the bias manipulation we used is not identical to the standard confidence rating procedures that are typically used in studies of human recognition in which participants rate their confidence for every test item. Although some studies have collected both bias manipulated and standard confidence ROCs (Egan 1958; Ratcliff et al. 1992), to our knowledge, no study to date has directly compared the shapes of the recognition memory ROCs produced by these different methods.

To examine the potential effects of the different test methods, ROCs from participants in the self-paced confidence bias condition were compared to ROCs produced by an additional group of participants who completed the self-paced condition using payoff bias instructions, and the ROCs from another group of participants who completed the self-paced condition using a standard six-point confidence scale (e.g., Yonelinas 1994, 1999). Briefly, participants in the payoff bias condition were awarded points for correct responses and had points deducted for incorrect responses (e.g., Snodgrass and Corwin 1988). The points earned or lost for correct and incorrect responses were adjusted to bias participants toward responding old or new. For example, on the most liberal test participants gained 10 points for making a correct “old” response, whereas they lost only one point for making an incorrect “old” response. Similarly, they gained only one point for making a correct “new” response and lost 10 points for making an incorrect new response, whereas this point structure for old and new items was reversed on the most conservative test (see Supplemental Material).

The aggregate ROCs for the confidence bias, payoff bias, and standard confidence self-paced groups are presented in Figure 1E. There was no significant difference between any of the three self-paced conditions in the overall hit rate, false alarm rate, recognition accuracy, or criterion placement, all F's_(2,61) < 0.94; all P's > 0.39 (Table 1). Importantly, a comparison of recollection and familiarity estimates between the three ROC conditions did not reveal any significant differences, providing evidence that the shape of the ROCs were similar in the three conditions, both F's_(2,61) < 0.44; both P's > 0.65 (see Fig. 1F). Moreover, an analysis of the zROC revealed no difference in slope or intercept, which further suggests that the three methods produce similar ROCs, both F's_(2,61) < 1.53; both P's > 0.22 (Table 1). Based on these results, it appears to be the case that the different testing methods produce comparable ROCs. This finding should assuage concerns that the different methods used to obtain ROCs might produce fundamentally different patterns of results.

In sum, using comparable ROC methods, the current results with human participants replicate those observed in recognition ROC studies in rats. Thus, at least in humans and rats, it appears that speeding recognition responses leads to a reduction in recollection but not in familiarity-based recognition. Moreover, we found that the ROCs produced under the confidence bias, payoff manipulation, and confidence rating procedures were in good agreement, suggesting that the various methods used to obtain ROCs in the human and animal literatures produce comparable results. The ability to bridge the divide between studies of recognition memory in animals and humans is of critical importance in advancing our knowledge of memory, and the current results further indicate that ROCs provide a fruitful method for studying recognition memory across species. Exciting questions for future work will be to determine whether these methods can be applied to other species (e.g., non-human primates), and to further test the extent to which recollection and familiarity processes measured in animals and humans are functionally and neurally similar.