Acoustic Masking Disrupts Time-Dependent Mechanisms of Memory Encoding in Word-List Recall

Abstract

Recall of recently heard words is affected by the clarity of presentation: even if all words are presented with sufficient clarity for successful recognition, those that are more difficult to hear are less likely to be recalled. Such a result demonstrates that memory processing depends on more than whether a word is simply “recognized” versus “not-recognized”. More surprising is that when a single item in a list of spoken words is acoustically masked, prior words that were heard with full clarity are also less likely to be recalled. To account for such a phenomenon, we developed the Linking by Active Maintenance Model (LAMM). This computational model of perception and encoding predicts that these effects are time dependent. Here we challenge our model by investigating whether and how the impact of acoustic masking on memory depends on presentation rate. We find that a slower presentation rate causes a more disruptive impact of stimulus degradation on prior, clearly heard words than does a fast rate. These results are unexpected according to prior theories of effortful listening, but we demonstrate that they can be accounted for by LAMM.

When a speech signal is degraded, requiring perceptual effort for successful word recognition, recall for the speech content suffers. This is so whether perceptual effort is engendered by acoustic masking of speech stimuli for listeners with normal hearing acuity (Farley, Neath, Allbritton, & Surprenant, 2007; Murphy, Craik, Li, & Schneider, 2000; Surprenant, 1999) or by a mild-to-moderate hearing loss (McCoy et al., 2005; Rabbitt, 1990). Of importance, this negative effect on recall occurs even when the level of acoustic masking or the degree of hearing loss still allows for successful, albeit effortful, word recognition. This is an intriguing and much replicated phenomenon, but one whose source is not yet fully understood.

In the above-cited studies, masking has been applied to entire word-lists or sets of paired-associates, thus obscuring the mechanisms that might underlie this effect. In a closer examination of the effect, Piquado, Cousins, Wingfield, and Miller (2010) demonstrated that masking just a single word in a spoken word-list reduces probability of recall, not only for the masked word itself, but also for an unmasked word prior to it. These data are shown in Figure 1, demonstrating that, relative to the same word-positions in control lists, there was a significant reduction in the probability of recall for both the masked word (M) and the prior word (−1), but not for words that followed the masked word. Importantly, this effect of masking occurred even though the level of masking allowed for correct, albeit effortful, identification of the masked word.¹

Figure 1.

Lower curve shows the probability of recall of an acoustically masked word (M), the two words preceding the masked word (−1, −2), and the two words following the masked word (+1, +2). The upper curve shows the probability of recall for words in analogous positions in a control list in which no words were masked. Lists comprised 7 items, presented at a rate of 1500 ms per word, and the position of the masked word varied from list to list. Error bars are one standard error. (Data from Piquado et al., 2010).

It was argued that the effect of a masked word on prior-word recall resulted from a disrupted output pattern during recall. That is, for lists in which all of the words are presented clearly, it is typically found that participants tend to make contiguous transitions during recall. Specifically, recall of any word in a list, n, is most likely followed by recall of the next word in the presented sequence, n+1 (Kahana, 1996; Howard & Kahana, 1999). An analysis of the Piquado et al. (2010) data indicated that the occurrence of a masked word resulted in fewer of such n+1 transitions between early list items, suggesting that this may have underlay the diminished recall for these items.

In a general sense these results appear consistent with a “resource” account of the negative effects of perceptual effort on recall. This is the postulate that perceptual identification of a stimulus and higher-order processing draw on the same pool of limited resources (Murphy et al., 2000; Rabbitt, 1968, 1990; Surprenant, 1999, 2007; Rabbitt, 1990; Schneider & Pichora-Fuller, 2000). As a result, a degraded stimulus would require the dedication of more resources for item recognition, such that higher order processing—in this case encoding the stimulus in memory—would suffer (McCoy et al., 2005; Pichora-Fuller, Schneider & Daneman, 1995). This would account for the poorer recall of a word prior to a masked word, so long as the prior word is still in the process of being encoded during the masked word’s presentation. While this notion of perceptual or cognitive resources has descriptive appeal (Kahneman, 1973), a testable model of the mechanisms underlying both the general and order-specific recall effects of noise-masking has until now been lacking.

To provide such an account, we developed the Linking by Active Maintenance Model (LAMM). This model is a development of the “Linking Buffer Model” proposed in Piquado et al. (2010; see also Miller & Wingfield, 2010). LAMM is based on dual-mechanism models of free recall (Davelaar, Goshen-Gottstein, Ashkenazi, Haarmann, & Usher, 2005; Lehman & Malmberg, 2013; Raaijmakers & Shiffrin, 1981), as well as temporally proximate linking models (Howard & Kahana, 2002; Lehman & Malmberg, 2013). LAMM builds upon current process models of free recall, however, by accounting for effects of degraded input both on item recall and on recall orders.

In the next section we describe LAMM, outlining mechanisms of encoding and recall. We give special attention to how recall is instantiated within the model, and to the role of a limited capacity memory buffer. We test the model by examining the predictions it makes regarding effects of presentation rate on both recall probability and on output patterns during recall. We discuss how these predictions are fundamentally different from those made by models lacking a limited capacity buffer, even if they were modified to incorporate effects of stimulus clarity. We follow the model presentation by a behavioral experiment to test these predictions. Our goal is to present a model that makes specific behavioral predictions, is consistent with biological principals, and can account for dynamic changes in recall under different stimulus protocols.

LAMM: A dual mechanism account for effects of perceptual effort on recall.

An adequate model for the effects of a degraded but still perceptible stimulus on recall must account for three key findings: (1) reduced recall of the degraded, acoustically masked word, even though it is audible; (2) reduced recall probability for the one or two words prior to the masked word, even though these words were not masked (a prior-word effect); and (3) reduced n+1 transition probabilities in orders of recall in the presence of masking. We suggest that models of episodic recall possessing a limited capacity, short-term memory buffer with an associative-linking mechanism (Davelaar et al., 2005; Lehman & Malmberg, 2013; Raaijmakers & Shiffrin, 1981) can account for such findings, but only insofar as such models incorporate the assumption that the masking of a word increases the likelihood of buffer disruption. That is, extant models of episodic recall assume a list presented with equal clarity across items. LAMM differs from other dual store models in its focus on the case when all words are not of equal clarity, a situation not uncommon in everyday listening with intermittent or modulated background noise (Cooke, 2006; Gustafsson & Arlinger, 1994).

Computationally, we implement LAMM as a Markov chain – a stochastic model whose transitions depend on the state of the system – with two stages. During the presentation and encoding stage, the state of the system is defined by the list of words presented and the state of a capacity-limited memory buffer. During the recall stage, the system state is defined by the encoded links, the buffer occupancy, and those words already output. The model outputs simulated recall sequences, which we analyze in the same way as the behavioral data.

Hebbian plasticity strengthens synaptic connections between coactive neurons, preferentially in the direction of the activation sequence (Bi & Poo, 1998; Hebb, 1949; Pfister & Gerstner, 2006). Such strengthening of connections between neurons (Erickson, Maramara, & Lisman, 2010), can enable successive recall of items represented by activity of those neurons (Miller & Wingfield, 2010). Within our model such biological mechanisms produce associative linking by two processes.

First, direct linking occurs by the successive activation of neurons representing words as they are perceived. Considering perception as “winner-takes-all” (Pressnitzer & Hupé, 2006; Renart, Moreno-Bote, Wang, & Parga, 2007), we assume that perception of each new item causes a decay of the activity of neurons representing the prior item. Such successive activation leads to a directional “forward” link between successive items, promoting recall in serial order (Miller & Wingfield, 2010), a finding routinely observed in free recall (Howard & Kahana, 1999; Kahana, 1996).

Second, indirect linking can occur via a two-step process involving a limited capacity memory buffer. The buffer operates similarly to previous dual-mechanism models of recall (e.g., Lehman & Malmberg, 2013; Raaijmakers & Shiffrin, 1981) and can be characterized as persistent neural activity (Fuster, 1973; Goldman-Rakic, Funahashi, & Bruce, 1990) or periodic reactivations as suggested by theories of rehearsal or ‘articulatory loops’ (Baddeley & Hitch, 1974; Lisman & Idiart, 1995). Changes in buffer activity with each new word presentation can cause loss of maintained items, thus capacity is limited. In LAMM, linking occurs from neurons representing a word to buffer neurons, and from buffer neurons to neurons representing another word. This buffer-mediated process can produce stronger associative links than direct linking alone because there is a longer period of co-activation—both between item-representing neurons and buffer neurons, as well as between different buffer neurons (Miller & Wingfield, 2010). These links that have been established during encoding shape the retrieval of items during recall.

Predictions from the Model

LAMM proposes that observed transitions in recall are the manifestations of associations created between items during listening, and that the utilization of these associations aids recall and influences output order, a principle outlined in Raaijmakers and Shiffrin’s Search of Associative Memory model (SAM; Atkinson & Shiffrin, 1968; Raaijmakers & Shiffrin, 1981). Although these effects depend on the temporal relationships of presented words, unlike unitary models of episodic recall such as the Temporal Context Model (TCM; Howard & Kahana, 2002) and the Scale-independent Memory, Perception and Learning model (SIMPLE; Brown, Neath, & Chater, 2007), in LAMM these associations are not based on a representation of the list in a temporal or temporal-context space, but are mediated by associations between the items themselves. Like dual-store models (Craik, 1968; Davelaar et al., 2005; Lehman & Malmberg, 2013; Raaijmakers & Shiffrin, 1981; Usher, Davelaar, Haarmann, & Goshen-Gottstein, 2008), LAMM possesses two biophysically-based mechanisms for maintaining information–through a strengthening of associations and through active maintenance in a buffer.

LAMM reproduces the common finding that slower presentation rates yield an overall improvement in recall performance (Craik & Rabinowitz, 1985; Murdock, 1962; Murdock & Okada, 1970; Vitulli & McNeil, 1990) for reasons in common with several other buffer models (Davelaar et al., 2005; Lehman & Malmberg, 2013; Raaijmakers & Shiffrin, 1981): the longer two items are coactive in the store, the greater the strengthening of the associative connections between them. Such behavior fits current understanding of activity-dependent or spike-dependent plasticity: the longer any two groups of neurons are coactive, the more spikes their neurons exchange with one another, so the more spike-pairing events there are to trigger synaptic strengthening between the groups.

In LAMM, however, presentation of a masked word disrupts the buffer store activity, reducing this coactivity and resultant store-dependent associative strengthening. Because associations are developed only between words already heard, only prior associations are affected by such disruption. The reduction of association strengthening during list presentation leads to reduced n+1 transitions among prior words during recall, and therefore poorer memory for them. Moreover, since masking impacts only buffer-mediated indirect linking in LAMM—direct linking remains intact—it is precisely the process which improves recall with slower presentation rate that is degraded by masking. Therefore, the detrimental effect on recall for words prior to the masked word is attenuated with a faster presentation rate. Thus, unlike shared-resource models (Schneider & Pichora-Fuller, 2000), LAMM does not predict a more detrimental effect of masking on recall of the prior word when the presentation rate is faster; if anything the effect should be smaller.

Method

Participants

Participants were 43 native speakers of English aged 18-30 years. All met a criterion of age-normal hearing based on pure tone thresholds averaged across 1000, 2000, and 4000 Hz (Hall & Mueller, 1997) and speech reception thresholds (SRTs) measured by the CID W-1 list of spondees (Auditec, St. Louis, MO). Six of the 43 participants were unaffected by the level of masking used in this experiment, defined as those individuals who did not show a recall decrement for masked words relative to words in the same position in the control lists in which no words were masked, and were not included in our data analyses.²

Stimuli and Behavioral Procedures

Participants heard 60 lists of 7 words each selected randomly from the Toronto Word Pool (Friendly, Franklin, Hoffman, & Rubin, 1982). In this immediate free-recall task, a list length of seven words was used because medium length lists permit the most variability in recall strategy (Ward & Tan, 2010). Participants were instructed to listen to each list carefully, and once it ended, to recall as many of the seven words as possible in the order that they came to mind (any order was permitted). Each list was prepared under two conditions: a masked condition and a control condition. Word lists in both conditions were presented at a uniform level of 40 dB HL. Stimuli were presented binaurally over earphones in a sound-isolated testing room.

Masked Condition

In the masked condition, one of the seven words was masked by simultaneously playing 20-talker babble at 38 dB (Leigh-Paffenroth & Murnane, 2011). This represents a signal-to-noise ratio (SNR) of +2; a level allowing for generally successful word recognition albeit with perceptual effort. The position of the masked word was varied randomly across lists, as either the 3^rd, 4^th or 5^th word in the 7-item list. To reduce the acoustic contrast between the masked word and neighboring words, we smoothed the transition by ramping up the babble intensity for the 200 ms preceding the masked word and ramped the babble down for the 200 ms following the word. To further reduce a potential isolation, or von Restorff effect (Fabiani & Donchin, 1995; von Restorff, 1933), all of the other words in the list were played with a low level of babble at 32 dB. This represents a SNR of +8, a level that allows for easy audibility (Jerger & Hayes, 1977).

Control condition

Lists in the control condition had the same construction as in the masked lists, except that no single word was masked, with all seven words in the lists played with the same low-level babble (+8 SNR) as the non-masked words in the masked lists. Stimuli were presented in a mixed-list design such that participants did not know in advance whether a list would be a control list or a list with a masked word, and if the latter, which of the three possible positions would be masked. Each participant heard all 60 word-lists: half in the control condition and half in the masked condition.

Presentation rates

Each participant heard half of the lists at a rate of 1 word per second and half at a slower rate of 1 word per 2 seconds, with 15 lists at each rate heard in the masked condition and 15 in the control condition. Fast and slow presentation rate lists were blocked, with half of all participants hearing the fast lists first and half receiving the slow lists first. Lists were counterbalanced across participants such that, by the end of the experiment, each list was heard an equal number of times in the masked and control, and both rate conditions.

The acoustic properties of the control and masked conditions are shown in Figure 2. The top, black waveform illustrates the seven words in the slow-rate control condition, with continuous low-level background babble represented in lighter gray (+8 SNR). Below this is the waveform for the slow-rate masked condition, illustrated for a word-list with the masked word in the fourth position in the seven-item list. It can be seen that the level of background masking, shown in light gray, is increased to +2 SNR during presentation of the masked word. The two lower waveforms illustrate control and masked conditions for the fast presentation rate.

Figure 2.

Top waveform represents each of the words in the 7-item list (black vertical departures from the center baseline) relative to the continuous low-level background babble represented in lighter gray, for a slow presentation rate list. Below this is the slow-rate waveform for the masked condition with the masked word shown in the fourth position in the 7-item list. Words and background babble are again shown in black and light gray, respectively. The lower two waveforms show the same conditions for the fast-rate presentations.

Audibility check

To ensure that both the masked and control words were audible, upon completion of the experiment all previously presented masked words and an equal number of control words, were presented again with instructions to immediately repeat each word. Recognition accuracy was 98% for control words (SD = 2.3) and 87% for masked words (SD = 6.7; t(37) = 8.91, p <0.001). Errors in the masked condition sometimes reflected a misidentification (e.g., “folly” reported as “volley”). To reduce the effect of such recognition errors on scoring, recall in the main experiment was scored as correct if the participant recalled a word as they had reported it in the audibility check.

Methods for Modeling and Simulations

In our modeling procedures, we label words with the integers 1 to N (N=7 in the current study) according to presentation order, setting w̃ = 3,4 or 5 as the degraded word in the masked condition. Buffer occupancy is recorded as a binary vector B^(t), with B^t = 0,1, where t corresponds to the tally of words presented thus far and i denotes the word. A ‘link matrix’, J_i←j, encodes association, with the strength of link from word position j to i given by:

$${\mathrm{J}}_{i\leftarrow j}=\sum _{n=1}^{\mathrm{N}}\{\mu {\delta }_{jn-1}{\delta }_{i\mathrm{n}}+\nu {\delta }_{j\mathrm{n}}{\delta }_{in-1}+{{\mathrm{B}}_j}^{\mathrm{n}}(\gamma {\mathrm{t}}_{pres.}+\eta {\delta }_{in})\},$$

where summation over n sums contributions to the i ← j-link from the state after each word presentation, and the Dirac-delta δ_ij equals 1 if i = j, and zero otherwise. μ and ν are forward and reverse direct link strengths respectively (μ » ν), γ is the linking rate, determining strengths of bi-directional associations between items co-active in the buffer, and η is a buffer-mediated link from active buffer items to the newly entered item. Dependence on presentation rate is through buffer-mediated linking, via the term t_pres., which is the time between word presentations, our principal parameter of interest.

Limited Capacity of the Buffer

We set a ‘soft’ bound to buffer capacity by increasing the probability, P, that an item can be lost from the buffer as more items are maintained³ via a power law, P = χ k^λ, where k is buffer occupancy. The parameter χ directly impacts buffer size, while the parameter λ determines how strongly the likelihood of each word’s loss from the buffer depends on current occupancy (λ = 0 means independent of occupancy). The values of χ (0.001) and λ (3.84) obtained from fitting to recall data produced mean maximum buffer occupancies of ~ 3.5 words.

Implementation of Masking in LAMM

Three additional parameters determine how noise-masking disrupts buffer activity. First, masked words may fail to enter the buffer, with probability ζ. Second, upon presentation of a masked word, buffer disruption probability increases by an amount, ξ. Given that masking could disrupt attention, we also include a parameter, ρ, which determines a geometric decay of the effect of masking on later words and represents how rapidly buffer activity recovers following the masked word.

For both masked and non-masked words, a resistance to disruption increases while a word is maintained in the buffer (buffer potentiation, denoted b^(t)). This follows the dual-store framework (Davelaar et al., 2005; Atkinson & Shiffrin, 1968; Raaijmakers & Shiffrin, 1981; Rundus, 1971; Usher et al., 2008), where extra rehearsal or extended coactivity enhances associations with long-term memory. Buffer potentiation is proportional to the inter-word presentation time, t_pres and resistance to disruption, κ. The potentiation of word i after the presentation of the n^th word is thus:

$${{\mathrm{b}}_i}^{\mathrm{n}}=\sum _{m=1}^{\mathrm{n}}{{\mathrm{B}}_i}^{\mathrm{m}}\kappa t_{pres.}$$

Computationally, stochastic retention (F̄ = 1) or loss (F̄ = 0) of a word from the buffer is realized via comparison of each probability of loss, p, to a pseudorandom number on the unit interval, such that:

$$\stackrel{‒}{F}\left[p\right]=\{\begin{array}{c}1,p\le randomidd\in (0,1)\hfill \\ 0,p\ge randomiid\in (0,1).\hfill \end{array}\phantom{\}}$$

The buffer’s state depends iteratively on its previous state, so that following presentation of the n^th word, the presence $({{\mathrm{B}}_i}^{\mathrm{n}}=1)$ or absence $({{\mathrm{B}}_i}^{\mathrm{n}}=0)$ of the i^th word is given by

$${{\mathrm{B}}_i}^{\mathrm{n}}=({{\mathrm{B}}_i}^{n-1}+{\delta }_{i\mathrm{n}}\stackrel{‒}{\mathrm{F}}\left[\zeta {\delta }_{i\tilde{w}}\right])\stackrel{‒}{F}\left[\chi {\left(\sum _j{{\mathrm{B}}_j}^{n-1}\right)}^{\lambda }-{{\mathrm{b}}_i}^{n-1}+\zeta ({\delta }_{n\tilde{w}}+\rho {\delta }_{n-1\tilde{w}}+{\rho }^2{\delta }_{n-2\tilde{w}})\right]$$

Recall in LAMM

In the model, recall is initiated stochastically according to the behaviorally obtained probabilities of which items are recalled first, (probability of first recall; PFR). Model recall then proceeds according to either encoded item-item associations, or current buffer activity. Two final parameters, α and β, govern the tradeoff between recall via item-item associations, which we attribute to synaptic connections strengthened by buffer activity, versus recall via direct output of final buffer activity. The probability of recalling word i directly after having recalled word j is given by

$$\mathrm{P}(r_s=i\mid r_{s-1}=j)={\mathrm{J}}_{i\leftarrow j}+\beta {{\mathrm{B}}_i}^{\mathrm{s}})\alpha ∕\sum _{m\notin {{\left\{r_k\right\}}_{k=1}}^{s-1}}{\left(J_{m\leftarrow j}+\beta {B_m}^s\right)}^{\alpha },$$

which is the link strength J_i←j plus a bias (β) if word i is currently in the buffer, normalized by the sum of all remaining link strengths from j to non-recalled items. α adds a non-linearity to the conversion of link strengths to probabilities, enhancing or dampening the contrast between competing links. For high α links are followed according to the rank order of their strengths, whereas with α = 0 recall is independent of link strength.

The buffer can be disrupted during recall to produce output interference (Roediger & Schmidt, 1980):

$${{\mathrm{B}}_i}^{N+k}={{\mathrm{B}}_i}^{N+k-1}\stackrel{‒}{\mathrm{F}}\left[\phi {\left(\sum _j{{\mathrm{B}}_j}^{N+k-1}\right)}^{\lambda }-{{\mathrm{b}}_i}^{\mathrm{N}}\right]$$

where ${{\mathrm{B}}_i}^{N+k}$ is the state of the buffer after the k^th recall. Recall terminates if remaining links are weak and no words remain in the buffer, according to

$$\mathrm{P}(r_s=\varphi \mid r_{s-1}=j)=\prod _{i\notin {\left\{r_k\right\}}_{{k=1}^{s-1}}}{\left[{1-J}_{i\leftarrow j}\right]}^{+}.$$

Simulation of the recall stage may stop before all words are recalled, with the number of words recalled thus N_r ≤ N.

For each condition and parameter set used, we simulated 300k trials of the model, producing minimal error bars for model data. A summary of model parameters and their values is given in Table 1.

Table 1.

Model Parameters

Parameter		Purpose	Value
Encoding
Base linking rate /s	γ	Rate of link formation between active buffer items	.074/s
Direct link strength,  Forward	μ	Strength of direct link from one perceived item to the next	.10
Direct link strength,  Reverse	ν	Strength of direct link from one perceived item to the previous	0
Retrieval
Strong link bias	α	Non-linearity in transfer function between link strength and transition probability.	3.96
Buffer recall  preference	β	Addition to link strengths to preference retrieval of items currently maintained in buffer	0.19
Buffer
Base disruption:  coefficient	χ	Probability that items in buffer will be lost upon presentation of new item – the realization of the effortful hypothesis.	.001
Exponent	λ	P(disruption with occupancy = k) = χ kλ.	3.84
Base disruption  during recall	φ	Probability of item loss from buffer during recall (c.f. output interference)	.002
Masked buffer  disruption	ξ	Additional disruption probability when new item is masked	.21
Masked buffer  rejection	ζ	Probability that a masked item will not be stored in the buffer	.30
Disruption  persistence	ρ	Multiplier for successively attenuating the disruption probability applied to the 1st and 2nd words following the masked position	.45
Buffer potentiation  rate	κ	Reduction in disruption probability, accumulated as an item remains in buffer — reflects synaptic potentiation of afferents to neurons representing items.	.002
Extra unidirectional  direct link	η	Added direct link from current buffer items to current entrant, reflecting the causal bias in Hebbian plasticity. (c.f. Miller & Wingfield 2010).	.09

Note. All simulation results presented herein are from a model defined by the parameters listed in the table above, which were fit to the 1s behavioral data.

Optimization of the Model

We scored performance of parameter sets using a weighted sum of the squared, scaled residuals between the model and experimental recall measures. Measures were optimized for recall accuracy of the masked word and the words immediately preceding and following it, the Lag-CRP curves, and the output transition probabilities relative to a masked word. Recall accuracy (Fig. 3) was most heavily weighted, such that the main focus of this investigation – the masking-induced deficits in recall – was the dominant constraint on the model fitting.

Figure 3.

Effects of masking a stimulus word on item recall. Figure 3A shows the probability of recalling an acoustically masked word (M) and the two words preceding it (−2, −1) and following it (+1, +2) for the slow presentation rate. The upper curve shows recall for analogous positions in a control list in which none of the words was masked. Figure 3B shows the same data for the fast presentation rate. Figures 3C and 3D show simulation results for the slow and fast presentation rates respectively. Error bars represent one standard error. Error bars that do not appear were too small to plot.

A pattern-search global optimization (Hooke & Jeeves, 1961) was performed using the MATLAB (MathWorks, Natick, MA) computational suite’s Global Optimization Toolbox. This algorithm was chosen for its ability to find global minima of high-dimensional stochastic objective functions, such as our simulated recall measures. The algorithm explores multiple local minima to find a global solution via an iterative procedure. To improve the robustness of the process, in alternate iterations we employed the N+1 basis-vector Generalized Pattern Search, and the 2N basis Mesh Adaptive Direct Search algorithms.

Testing Presentation Rate in LAMM

To test whether LAMM accounts for the effects of presentation rate we fit LAMM to the 1s (faster) presentation rate data, and then validated against the 2s (slower) data set. Given that the only explicit dependence on time in LAMM is through the variable t_pres, which sets the time in seconds between word presentations (1s or 2s), these 2s model validations against the behavioral data have no free parameters: the time variable t_pres. is simply doubled.

Since LAMM uses the experimentally observed PFR to initiate model recall, it is necessary to avoid the possible confound that differences in PFR between conditions might account for any observed differences in overall recall accuracy or output ordering. Therefore, a single PFR was used for all simulated conditions: namely, the average PFR across conditions. Given such a fixed PFR, we could test whether changes within LAMM alone explain the key behavioral consequences of altered presentation rate and masking.

Results

Effects of a Masked Word on Item Recall

Behavioral Results

Table 2 shows mean recall probabilities for each serial position in control lists and for lists in which words in positions 3, 4, or 5 were masked. It can be seen that there is some non-systematic variability across the three masked positions, but with the strongest effect appearing when position 3 is masked. Figure 3 shows these recall probabilities averaged over the three word-masking positions and collapsed across positions relative to the masked word (M), along with these same relative positions in control lists. Figures 3A and 3B show these data for the slower (1 word every 2 seconds) and faster (1 word per second) presentation rates, respectively.

Table 2.

Probability of Item Recall

		Serial Position for 7-item List
		1	2	3	4	5	6	7
Slow Rate	Control Lists	0.72±0.20	0.67±0.19	0.62±0.15	0.60±0.18	0.62±0.22	0.70±0.18	0.84±0.12
	Mask Position 3	0.68±0.23	0.55±0.25	0.50±0.26	0.61±0.20	0.58±0.25	0.77±0.22	0.88±0.17
	Mask Position 4	0.71±0.22	0.65±0.25	0.55±0.21	0.52±0.27	0.56±0.23	0.70±0.24	0.85±0.19
	Mask Position 5	0.74±0.21	0.70±0.19	0.60±0.24	0.54±0.21	0.43±0.28	0.66±0.24	0.83±0.19
Fast Rate	Control Lists	0.65±0.20	0.53±0.19	0.51±0.19	0.54±0.20	0.61±0.19	0.68±0.13	0.82±0.12
	Mask Position 3	0.64±0.29	0.52±0.26	0.41±0.26	0.51±0.30	0.64±0.25	0.66±0.27	0.85±0.16
	Mask Position 4	0.59±0.24	0.51±0.26	0.45±0.24	0.46±0.24	0.67±0.23	0.77±0.22	0.88±0.16
	Mask Position 5	0.62±0.25	0.51±0.19	0.52±0.23	0.52±0.26	0.46±0.26	0.64±0.27	0.82±0.18

As would be expected from the recall literature, for control lists, in which all words were clearly audible, a comparison of Figures 3A and 3B shows better recall for the slower presentation rate than the faster rate (Murdock, 1962), with an average of 4.76 (SD = 0.64) words recalled in slower presented control lists and 4.33 words (SD = 0.63) in faster ones (p < 0.001). A 5 (Position: −2, −1, M, +1, +2) x 2 (Condition: masked, control) x 2 (Presentation rate: fast, slow) mixed-effects analysis of variance (ANOVA) showed this significant main effect of rate [F(1, 36) = 26.48, mse = 0.39, p < 0.001], interacting significantly with masking condition [Rate × Condition, F(1, 36) = 4.69, mse = 0.06, p < 0.05] just as LAMM predicts. A significant Rate × Position interaction [F(4, 144) = 5.38, mse = 0.07, p < 0.001] confirms the apparent greater effect of presentation rate on earlier than later positions in the list (Bhatarah, Ward, Smith, & Hayes, 2009; Grenfell-Essam, 2012). Consistent with prior work (Piquado et al., 2010), the masking of a single word in a list resulted in an overall reduction in recall accuracy relative to the non-masked control list, as seen by the main effect of condition [F(1, 36) = 31.61, mse = 0.27, p < 0.001]. There is also a significant Condition x Position interaction [F(4, 144) = 10.20, mse = 0.09, p < 0.001], due to the greater reduction in recall for the masked word relative to the others. Finally, a main effect of position [F(4, 144) = 26.67, mse = 0.83, p < .001] reflects an expected serial position effect, with greater recall for earliest (primacy) and latest (recency) positions.

Of primary interest to this investigation is the interaction between masking and list position. When analyzing individual positions, not only is the masked word itself less likely to be recalled than a similarly placed word in a control list at both presentation rates [slow rate: t(36) = 5.17, p < 0.001; fast rate: t(36) = 5.65, p < 0.001], but, at the slow presentation rate, the same is true for the word prior to the masked word [t(36) = 4.60, p < 0.001]. By contrast, the effect on the word prior word at the fast presentation rate fails to reach significance [t(36) = 0.94, p = 0.38]. There were no significant differences in recall probability between masked and control lists at other positions for either rate.

The prediction of LAMM tested specifically in this study is a greater impact of masking when presentation rate is slow. Indeed, the effect of masking on prior-word recall apparent in Figure 3 is significantly greater with a slow rate of presentation than with a fast rate. An additional 5 (Position: -2, -1, M, +1, +2) x 2 (Presentation rate: fast, slow) ANOVA on individual subjects’ recall differences between control and masked trials (the masking deficits) shows the effect of presentation rate on masking [F(1, 36) = 4.69, mse = 0.12, p < 0.05]. Again, the masked position shows the greatest deficit, with a significant main effect of position [F(4, 144) = 10.20, mse = 0.19, p < 0.001]. While the difference in masking deficits between presentation rates could not be attributed to any one word position (Rate × Position interaction was not significant, p = 0.85), it is revealing to note that the mean recall deficit of the masked word (M) was very similar across the two presentation rates (0.129 for the slow rate, and 0.109 for the fast rate), whereas the masking deficit for the prior word (M-1) was over three times as great with slow presentation (0.082 versus 0.026).

Model Simulations

Figures 3C and 3D show the model simulations for the slow and fast presentation rates respectively. The simulations in the control condition for both presentation rates capture the overall shapes of the serial position curves, showing both primacy and recency effects. The effects of masking produced by LAMM are also in agreement with the data. Moreover, when comparing the 2s condition with the 1s condition, the model reproduces the observed lower overall recall in control lists when presentation rate is faster. Finally, LAMM reproduces a critical component of the behavioral findings: that the recall deficit produced by masking is more pronounced with a slower presentation rate.

Lag-Conditional Response Probability (Lag-CRP)

“Lag” refers to the relative list positions of successive recall outputs, which can be analyzed to provide an important measure of output ordering. Analyses of recall order under free recall instructions show a general tendency to make output transitions among nearby list items. This conditional response probability as a function of lag (lag-CRP; Howard & Kahana, 1999; Kahana, 1996) represents the probability that an item from serial position n + lag will be recalled immediately following an item from serial position n. That is, lag-CRPs give the probabilities of making transitions during recall from any one relative word position to another. Prior studies have shown the lag-CRP to have a forward bias, illustrating the effect of positional or temporal contiguity on retrieval transitions, with the ‘+1’ lag being the most common (Howard & Kahana, 1999; Howard, Kahana & Wingfield, 2006; Kahana, 1996).

Figure 4 shows the lag-CRP curves for participants’ behavioral data and LAMM simulations for recall outputs. Lag-CRPs are calculated by considering all recalls of position n, and then computing the probability of transitioning to n+lag as mentioned above, providing that this latter position has not already been recalled. The set of all possible such transitions is then collapsed relative to the originating word’s position n (denoted “0” on the abscissa).

Figure 4.

Figures 4A and 4B show Lag-CRP functions for behavioral data for the masked and control conditions for the slower and faster presentation rates respectively. Figures 4C and 4D show simulation results for the slower and faster presentation rates respectively. Error bars represent one standard error. Error bars that do not appear were too small to plot.

The upper and lower left panels of Figure 4 show the CRP curves for the masked and control lists for the slow (Fig. 4A) and fast (Fig. 4B) presentation rates. For both presentation rates and both listening conditions, by far the single most likely transition during recall is the n+1 transition, in that participants are more than 50% likely to follow recall of a given word (“0”) with the following word (denoted as “1” on the abscissa).

According to LAMM, the superior overall recall for slower presentation rates than faster presentation rates results from the slower rate allowing more time to strengthen associations between items within the buffer. One would thus expect to see an increased n+1 transition probability for control lists presented at a slow rate compared to a faster presentation rate. Figures 4A and 4B show just that. First, for the control lists, the probability of recalling the word in relative position “1” is significantly higher for the slow presentation rate than the fast presentation rate [t(36) = 2.27, p < 0.05]. Moreover, Figures 4A and 4B show that at the slower presentation rate there were fewer n+1 transitions when a list contained a masked item than in control lists [t(36) = 3.36, p < 0.01], while there was no difference in these conditional response probabilities at the faster rate [t(36) = 0.60, n.s.]. The simulations in Figures 4C and 4D also show a decrease in n+1 transition probability for masked lists at the slow presentation rate, which, in accordance with the behavioral trend, was stronger than the decrease seen in the masked-list condition with the faster presentation rate.

Individual n+1 Transitional Probabilities

In the above analysis, the dominant n to n+1 transition was significantly affected by masking. To examine n+1 transitions more closely, Figures 5A and 5B show the effect on individual n+1 transition probabilities of masking and presentation rate plotted against position relative to the masked word (M). Changes to transition probabilities mirror the decreased recall we saw in Figure 4, with larger deficits in transition probabilities due to masking in the slow presentation rate (Figure 5A) than in the fast presentation rate (Figure 5B). A 4 (Transition: −2 to −1, −1 to M, M to 1, 1 to 2) × 2 (Condition) × 2 (Presentation rate) ANOVA conducted on the transition probabilities revealed main effects of transition [F(3, 108) = 22.33, mse = 0.92, p < 0.001] and condition [F(1, 36) = 8.82, mse = 0.23, p < 0.01], and significant Rate x Condition [F(1, 36) = 4.33, mse = 0.17, p < 0.05] and Transition × Condition [F(3, 108) = 3.78, mse = 0.10, p < 0.05] interactions. However, there was not a significant Transition x Rate x Condition interaction (p = 0.97).

Figure 5.

Proportion of n+1 transition probabilities for transitions prior to the masked word (M) and the transitions following it for masked and control conditions for the slower (Fig. 5A) and faster (Fig. 5B) presentation rates. Figures 5C and 5D show simulations for the slow and fast presentation rates respectively. Error bars represent one standard error. Error bars that do not appear were too small to plot.

It can be seen in Figure 5A that transitions following the masked word are also reduced. We thus considered the two prior transitions (-2 to -1 and -1 to M) and the following transitions (M to +1, +1 to +2) separately. Prior transitions showed significant effects of rate [F(1, 36) = 5.97, mse = 0.14, p < 0.05], condition [F(1, 36) = 23.26, p < 0.001] and transition position [F(1, 36) = 12.48, mse = 0.28, p < 0.001], as well as a Rate × Condition interaction [F(1, 36) = 5.22, mse = 0.10, p < 0.05]. Following transitions showed only a main effect of transition position (p < 0.001); neither effects of rate (p = 0.43) nor masking (p = 0.82) were significant. Corrected comparisons of all rate-condition cases of prior or following transitions confirmed that only the prior transitions in the control lists at the slow rate were significantly different from other trials (at p < 0.05). The transitions following the masked word (M to +1, +1 to +2) were not significantly different from control list transitions for either rate.

Thus, only for the slower rate of presentation were probabilities of n+1 transitions significantly reduced by masking, and these were restricted to the positions prior to the masked word: From the word two-positions-prior to the word immediately preceding the masked word [− 2 to −1; t(36) = 4.20, corrected p < 0.001], and from the word immediately preceding the masked word to the masked word [−1 to M; t(36) = 3.88, corrected p < 0.01], relative to these transitions in control lists (following transitions: p = 0.42 and 0.74). At the faster rate of presentation these effects were attenuated, and none of the contrasts with the control lists reached significance (all corrected p–values > 0.30).

The pattern of interactions between rate and condition revealed by the ANOVA is consistent with LAMM’s prediction that masking causes greater reductions in recall compared to control lists when the presentation rate is slower, and that this should be a consequence of weaker associative links between words as a result of buffer disruption. The significant reductions in the two n+1 transitions preceding the masked word, when presentation is slow, coincides with the observed reductions in recall for both the masked (M) and preceding (M-1) positions. Although the masking deficit in the M-1 position was not significantly different between the fast and slow rate lists [t(36) = 1.58, p = 0.12], the masking deficit in the dominant transition to this word (−2 to −1 transition) was significantly attenuated by the faster presentation rate [t(36) = 2.07, p < 0.05].

Simulations of LAMM are shown in Figure 5C and 5D, where they replicate the n+1 transition pattern seen in the behavioral data: compared to control lists, the transition probabilities between words prior to a masked word are reduced by a greater amount when presentation rate is slow rather than fast.

Probability of First Recall (PFR)

Figure 6 plots the probability that a participant will begin recall with words from the five masked-relative list positions (−2, −1, M, +1, +2), and words in the same relative positions in control lists (probability of first recall; PFR). Figures 6A and 6B shows the PFR for the slow and fast presentation rate, respectively, and demonstrates that PFR did not differ between the two rates for control lists in which none of the words were masked.

Figure 6.

Probabilities that subjects begin their recall with the masked word (M) or the word one (−1) or two (−2) words prior to the masked word or the word one (+1) or two (+2) words following the masked word. Data are also shown for the probability of first recall of items in the same relative positions in the control lists. Figure 6A shows these data for the slow presentation rate and Figure 6B shows these data for the fast presentation rate. Error bars represent one standard error.

As can be seen in the upper panel, when participants heard the word-lists at the slower presentation rate the presence of a masked word had a tendency to reduce the probability that recall was initiated with the word prior to the masked word (−1) when compared to that position in a control list [t(36) = 2.64, p = 0.06, Bonferroni corrected]. The data for the faster presentation rate shows an opposite, but similarly non-significant trend (corrected p > 0.10).

As we see in Figure 6, the contribution of recall initiation to the masked recall deficit in Figure 3 was found to be negligible; incorporation of such differences in PFR into LAMM, instead of LAMM’s masking mechanisms as described, was not sufficient to produce any of the observed recall differences caused by masking. This absence of a significant impact of the observed variation in PFR on word recall, combined with the significant changes in transition probabilities described in the previous section, supports the role for inter-item links in producing the recall effects of masking, independent of differences in first recall.

DISCUSSION

Over four decades have passed since Rabbitt (1968) reported an intriguing finding when he tested recall for a set of spoken 8-digit lists. The eight items were presented in two groups of four separated by a 2-second pause, and listeners were instructed to recall either the first or the second group of four digits, signaled after list presentation. In the conditions of interest, Rabbitt either presented both lists in the clear, or he acoustically masked either the first or the second set of four digits. Of special importance, he took care to ensure that the level of masking allowed for the masked digits to be correctly identified, albeit with some effort. Rabbitt found that recall of the first half of the list, even when presented clearly and without masking, was poorer when the second half of the list was heard in noise than when the second half of the list was heard in quiet. Rabbitt suggested that the increased effort required to identify the second, noise-masked digits, may have deprived the listeners of the processing resources that would ordinarily have been available to effectively encode the first digit-set in memory (Rabbitt, 1968).

The studies following Rabbitt’s seminal experiment have shown that noise-masking verbal materials produces a recall deficit relative to similar materials heard in the clear (Farley et al., 2007; Surprenant, 1999; Murphy et al., 2000). As noted, in these studies all items in the masked sets, whether word-lists or paired-associates, were uniformly masked. We subsequently found that masking just a single word in a spoken word-list was sufficient to have a detrimental effect on recall, not only for the masked word, but also on the word prior to it (Piquado et al., 2010). This is the same asymmetric effect as found by Rabbitt (1968), in that masking affected recall of what was heard prior to the masked stimuli but not those that followed it. This finding was confirmed in the present study: masking affected recall of the masked word and the word prior to it, even though the level of masking still allowed for successful, although effortful, recognition of the masked word.

Of critical interest in the current study is the finding that the detrimental, asymmetrical effect of masking is reduced when the presentation rate of the word-list is faster. This finding is in stark contrast to expectations based on notions of shared resources or an information-degradation hypothesis (Murphy et al., 2000; Schneider & Pichora-Fuller, 2000; but see also Tulving, 1969). A resource-sharing hypothesis posits that processing the sensory input and encoding what has been heard in memory compete for the same pool of limited resources (Kahneman, 1973). Thus, an impoverished input, such as an acoustically masked word, would increase the resource drain required for recognition of the masked word. The result of this increased resource drain needed for front-end perceptual operations would impair memory encoding (e.g., McCoy et al, 2005). On the other hand, a slow presentation rate would allow for a greater fraction of perceptual processing to be achieved, or to be fully completed, before the masked word arrived. This would lead to a prediction that the negative effect of masked words on prior words would be greater for a faster rate of presentation than for a slower rate, a prediction counter to our obtained findings. Further, neither a limited-resource model nor an information-degradation hypothesis would have any expectations in regard to the finding that the presence of a perceptually-difficult masked word would disrupt n+1 transitions in recall output order.

Within LAMM the buffer-induced association strength is proportional to the time the two words spend coactive in the buffer, a principle similar to the TODAM power-set model (Murdock, 1995, 2005). In the absence of any other change in the dynamics of the system, an effect of doubling the time per word is a doubling of the buffer-induced association strengths. As we saw, after optimizing the model to the 1s data the model simulations produced good fits to the 2s data with no additional parameter fitting.

The commonly encountered higher overall recall with a slower presentation rate (Murdock, 1962) follows from a greater time of coactivity in a buffer store for all words, and thus greater strengthening of associations, as demonstrated by the increased n+1 transition probabilities. The greater negative impact on the word prior to the masked word with a slower presentation rate also follows from LAMM because the encoding mechanism that increases recall for slower presentation rates is disrupted by masking. That is, the impact of masking is greater in conditions when strengthening of associations by active maintenance is greater. It is worth noting that this fundamental consequence of altered presentation rate arose in a model designed to explain the effects of stimulus degradation on recall of lists, with no thought to presentation rate (Piquado et al., 2010).

Although a number of models, such as TCM, give a good account of the dynamics of free recall under ordinary listening conditions (Howard et al., 2006; Howard & Kahana, 1999; Sederberg, Howard, & Kahana, 2008), TCM or other episodic memory models without an active store (e.g., SIMPLE; Brown et al., 2007)—even if modified to allow stimulus degradation to impact prior word recall—would also predict a greater, not lesser, impact of masking with faster relative to the slower presentation rates, an expectation contrary to our present finding. Although dual store models (Lehman & Malmberg, 2013; Raaijmakers & Shiffrin, 1981; Davelaar et al. 2005) do not consider a case where stimulus items are unequal in perceptual clarity, we suggest that such models form the best basis for such an extension.

Biological Basis for LAMM

Although LAMM is fundamentally a computational model, it was developed to be within the constraints of biological plausibility, based on the two neurobiological mechanisms for memory maintenance of synaptic plasticity and persistent activity. The two mechanisms are not independent and indeed following Hebb’s (1949) prescient hypothesis, the reverberating activity which persists after stimulus offset (Lorente de Nó, 1933), can produce the correlated spiking necessary for synaptic changes. Here, we incorporate such persistent mnemonic activity in a multi-item buffer and find that a good fit to the behavioral data appears with the postulate that such persistent activity is disrupted by the perceptual challenge induced by a degraded stimulus.

LAMM is derived from our ongoing cellular level simulations, a large-scale extension of those in Miller & Wingfield (2010), that comprise two distinct, but connected, neural circuits. One circuit maintains an active neural representation of previously heard items, allowing them to be associated with later items. Their ongoing activity is represented in LAMM by their presence in the memory buffer. Anatomically, this circuit is likely to reside in the prefrontal cortex, based on both human studies (Nyberg, 1998; Wais, Kim, & Gazzaley, 2012) and findings of mnemonic activity in primates (Funahashi, Bruce, & Goldman-Rakic, 1989; Goldman-Rakic et al., 1990). Alternatively, such buffering could also arise from repeated sequences of neural activation within the hippocampus (Fortin, Agster, & Eichenbaum, 2002; Jensen, Idiart, & Lisman, 1996; Lisman & Idiart, 1995; Winder, Reggia, Weems, & Bunting, 2009) or an interaction between the two (Jensen & Lisman, 2005).

The second circuit is the word-recognition circuit, which is winner-takes-all in nature (Wang, 2002), in which only one item is recognized/perceived at a time. Reactivation of word-representing cells in this circuit effects recall of a word. Such reactivation can arise either from previously active cells in the same circuit, or via transient associations in an active-store buffer circuit. The word-recognition circuit is most likely distributed across the core left hemisphere language area that includes inferior frontal gyrus and the posterior portion of the superior temporal gyrus (cf., Gagnepain, Henson, & Davis, 2012; Simos et al., 2009; Weems & Reggia, 2006). These two circuits produce the separate buffer-mediated contribution and the direct contribution to word associations within LAMM (see also arguments in Anderson, Qin, Sohn, Stenger, & Carter, 2003; Davelaar et al., 2005; Davelaar, Haarmann, Goshen-Gottstein, & Usher, 2006; Sohn et al., 2005).

Final Remarks

Our focus in this study was specifically on effects of degraded auditory stimuli, an issue of practical significance given reports of an increase in the incidence of hearing impairment among university-aged young adults (Shargorodsky, Curhan, Curhan, & Eavey, 2010). For this experiment we selected a single noise level relative to speech based on a pilot study. As noted, six potential participants failed to show a negative effect for the masked word itself at the masking level we employed, defined as individuals whose recall for masked words were no less accurate than recall of control words in the same position (all other positional effects, including the prior word, were not considered). Because one needed a masking effect to examine how the effects of masking on recall for non-masked words changed with rate, these participants were excluded from the participant pool prior to undertaking any data analysis. It is likely that adjusting the noise level on an individual basis would have been more effective at causing a masking effect for all listeners. The positive and negative factors thought to affect the ability to separate speech from background noise remains an active research area. Potential factors include individual differences in auditory processing not revealed by traditional pure tone audiometry (Abel, Krever, & Alberti,1990; Lorenzi, Gatehouse, & Lever, 1999; Marrone, Mason, & Kidd, 2008; Singh, Pichora-Fuller, & Schneider, 2008), individual differences in working memory capacity (Heinrich, Schneider, & Craik, 2008), and enhanced experience in isolating one sound among many, which it has been suggested, may include musicians (Parbery-Clark et al., 2011).

Despite such individual differences in the susceptibility to masking effects, the finding that listening to masked speech leads to reduced memory is a robust one (Rabbitt, 1968; Farley et al., 2007; Surprenant 1999; Murphy et al., 2000). Our behavioral results show that disrupted output during free recall significantly contributes to this reduction in memory as shown by the reduced n+1 transitions. This finding that masking a word disrupts the pattern of transitions in recall may be of some generality; serial recall studies using visual stimuli show that many perceptual factors - including word fragmentation (Serra & Nairne, 1993), perceptual interference (Mulligan, 1999, 2000), and irrelevant speech (Beaman & Jones, 1998; Neath, 2000) - can disrupt the order information for items in a list. These findings suggest that, even when perception is successful under impoverished input conditions, it comes at a cost of diminished relational or order information for items.

Disrupted relational information, however, does not always lead to poor memory. Some studies show that, if perceptual interference makes the stimulus stand out (e.g., the “von Restorff effect”; von Restorff, 1933; Rundus, 1971), memory for the item is improved, despite poor relational memory (Serra & Nairne, 1993; Mulligan, 1999). Because we took care to prevent a stand-out effect of the masked word, using a low level of background noise in both control lists and lists containing a masked word, we showed that this reduced relational information leads to poorer memory for the masked and earlier items. The combination of these findings suggests that, while order information and item specific information are distinct, they work together to aid retrieval of items during recall.

In this study, we tested the theory that the effects of masking are due to buffer disruption; when the role of the buffer is minimal, as with a fast rate of presentation, factors that would normally disrupt the buffer should no longer affect performance. In support, our behavioral results show attenuated recall and transition effects when the presentation rate was fast. Though we are specifically interested in the effects of effortful listening, we assume that there are potentially many sources of buffer disruption, including increased attentional demands (Anderson, Craik, & Naveh-Benjamin, 1998), articulatory suppression (Larsen & Baddeley, 2003), and exceeding buffer-capacity limits (Hanley & Bakopoulou, 2003). If so, such factors could lead to diminished memory effects, both for present and for prior information, similar to those we demonstrate in this paper.