A case for the role of memory consolidation in speech motor learning

Abstract

This review will explore the role of memory consolidation in speech motor learning. Existing frameworks of speech motor control account for the protracted time course of building the speech motor representation. These perspectives converge on the speech motor representation as a multi-modal unit that is comprised of auditory, motor, and linguistic information. Less is known regarding the memory mechanisms that support the emergence of a generalized speech motor unit from instances of speech production. Here, we consider the broader learning and memory consolidation literature, and how it may apply to speech motor learning. We discuss findings from relevant domains on the stabilization, enhancement, and generalization of learned information. Based on this literature, we provide our predictions for the division of labor between conscious and unconscious memory systems in speech motor learning, and the subsequent effects of time and sleep to memory consolidation. We identify both the methodological challenges, as well as the practical importance, of advancing this work empirically. This discussion provides a foundation for building a memory-based framework for speech motor learning.

Introduction

Speaking is a highly complex, time critical (Gracco & Lofqvist, 1994; Tourville, et al., 2013; Tsao & Weismer, 1997; Zemlin, 1998) motor behavior that most children acquire without conscious thought or deliberate effort (Smith & Zelaznik, 2004; Tourville, et al., 2013). The act of transferring speech motor movement into linguistic acoustic energy requires talkers to plan, program, and coordinate the crucial temporal parameters of multiple articulators with exact precision (Gracco & Lofqvist, 1994; Tourville, et al., 2013). As such, speech production is not only one of the most skilled and refined examples of human motor coordination, it is also one of the most complex skills that humans learn (Weismer & Fennell, 1985).

Current theoretical frameworks of speech motor control account for the protracted, developmental time course of speech motor learning. These frameworks specify that systematic relationships between articulator movements and the acoustic-auditory and somatosensory consequences (e.g., tactile contact sensation, proprioceptive feedback) of those movements emerge over different time scales (Callan, et al., 2000; Guenther, et al., 2006; Hewlett, et al., 1998; Kello & Plaut, 2004; Kent, 1984). As for how these relationships develop over time, there are various discussions surrounding how various teaching signals, such as other talkers in one’s social group, and one’s own expectations about an output that results from a given action (Guenther, 2016; Smith & Zelaznik, 2004), facilitate incremental adjustments that eventually lead to automatic behaviors.

By contrast, the discussion of how learning and memory processes, such as stabilization, enhancement, and generalization, that occur following the learning event apply to speech motor behavior has been limited. Insights from the wider memory literature has important applications for speech motor learning (e.g. for clinical practice in speech language pathology, see Maas et al., 2008, for review). However, observations regarding the interaction of broader learning and memory processes and their application during speech motor learning have not been unified under a wider framework of learning and memory. This examination may offer a mechanistic account of the formation of speech motor plans, and a new framework by which one can generate new hypotheses.

We propose that there are potential gains to be made in both theory and in application to consider the role of memory consolidation in learning new speech motor gestures. Consolidation is an umbrella term for the stages of processing that a memory trace undergoes between initial encoding and eventual stabilization (see Dudai, 2004, for review). This includes localized changes to synaptic strength (“synaptic consolidation”), and systemic changes to the organization of information (“systems consolidation”) that is thought to occur primarily during an offline (e.g. sleep) state. The former is thought to occur within hours (Dudai, 2004), whereas there is evidence of the latter occurring within a single day without sleep (Lindsay & Gaskell, 2013), a single period of sleep (Ellenbogen, Hulbert, et al., 2006; Fischer, et al., 2002; Walker & Stickgold, 2006), and as a protracted process taking place over days and weeks (Gais & Born, 2004; Gaskell & Dumay, 2003). Memory consolidation has been studied for decades in regard to learning language forms (vocabulary and grammar) (Davis, et al., 2009; Gomez, 2006; see Brain and Language special issue, Rasch, ed). Of relevance to speech motor learning, findings suggest that memory consolidation is important both for non-speech motor skill learning (see Walker & Stickgold, 2006, for review), and for auditory-perceptual learning of speech (Earle & Myers, 2015a, 2015b; Fenn, et al., 2003; Qin & Zhang, 2019; Xie, et al., 2018). Thus, while various adjacent literatures have examined the contribution of memory consolidation in learning, the effects of sleep and wakeful rest following speech motor practice have not yet been fully examined in speech motor learning.

The current review examines theories of speech motor learning and the neurobiological underpinnings of learning and memory consolidation, with particular attention paid to the findings of the role of rest and sleep on behavior in domains relevant to speech motor learning. Through these discussions, we call for the need to fill this critical gap in empirical research in order to advance our understanding of the memory processes involved in speech motor learning.

Theories of speech motor control and learning

The goal of speech motor learning is to acquire and produce the precise, finely coordinated movements of speech, that result in an acoustic signal that can effectively and efficiently transmit a message to a listener. Accounts of speech motor learning under consideration in the present discussion are subsumed under models of speech motor control, according to their conceptualizations of speech motor representations. This discussion will highlight the key components of a speech motor representation, that is necessary to encode, consolidate and represent through memory processes.

Models of speech motor control have developed in parallel to theories regarding non-speech motor control (Adams, 1971; Keele, 1968; Keele & Summers, 1976; Schmidt, 1975), that focus on the concept of the motor program. However, in speech motor models, the gesture (the basic unit of articulatory motor control) includes linguistic, cognitive and action information in its representation (Browman & Goldstein,1989). The coupling between gestures across different tiers of language, from syllables to prosody, comprise the hierarchical structure of speech and language (Krivokapic, 2007; Nam, et al., 2009). Thus, speech motor learning has historically involved the encoding of these context-invariant, motor-linguistic gestures, as the basic unit of this hierarchy.

According to early views, both the motor and the speech motor program were rigid and unresponsive to the changing needs of the environment, motor task, and the agent/individual (Boyce, et al., 1990; Fowler, 1977, Henry & Rodgers, 1960; Keele, 1968; Lofqvist, 1990; Ohman, 1967; Saltzman & Munhall, 1989; see Sherwood & Lee, 2003, for an excellent review). A key innovation in both models of speech and non-speech motor control was the idea that motor programs were subject to updates through the incorporation of sensory feedback (Adams, 1971; Schmidt, 1975). A well-known example is Schmidt’s (1975) cognitive theory of motor learning or Schema theory. Schema theory allowed for motor plans to be contextualized, and moreover revised and refined as a response to auditory, somatosensory, and other types of feedback. There are two primary components of motor control under Schema theory. The first is the abstracted motor plan (generalized motor plan, or GMP) that is acquired incrementally through practice (Adams 1971, 1987; Sherwood & Lee, 2003) and over different levels of conscious control (Weaver, 2015). The other primary component is parameter control. Parameter control allows the GMP to be modified and adapted to the task, the context, and to the needs of the agent. Schema theory was among the first to recognize the influence of the environment and the task at hand to motor performance. Furthermore, the acknowledgement of habitual adaptations to the GMP in order to accomplish the task at hand provided an opportunity to explore how motor plans are learned within the same framework of motor control, as opposed to as a separate, developmental, framework during acquisition.

In the speech motor literature, a similar departure from the context-invariant representation culminated in a new class of models of speech motor control, including (but not limited to) Prompts for Restructuring Oral Muscular Phonetic targets (PROMPT) (Hayden, 2008; Hayden, et al., 2010; Square, et al., 1986), Task Dynamics (TD) (Saltzman & Kelso, 1987; Saltzman & Munhall 1989), State Feedback Control (SFC) (Houde & Nagarajan, 2011), Directions Into Velocities of Articulators (DIVA) model (Guenther, 1994, 2016), and more recently, the Feedback Aware Control of Tasks in Speech (FACTS) model (Parrell, et al., 2019). Amongst this class of models, perhaps the most well-known, and most comprehensive, is the DIVA model (Guenther, 1994; 2001; Tourville & Guenther, 2011). DIVA is a neural network model of speech motor control and learning, that has been tested both computationally, through lesion studies, and through neuroimaging (Golfinopoulos, et al., 2010; Guenther, 2016). The mechanism of speech motor learning in DIVA (as well as others, e.g. SFC) reflects recent advances in the field surrounding how the striatum learns. Specifically, the basal ganglia appear to be involved in predicting the outcome to result from a preceding event or action, and in calculating the neurochemical reward to be signaled based on the relative mismatch between the two (see Ullman, et al., 2020, for recent review). The DIVA model incorporates these mechanisms in its conceptualization of how speech motor learning occurs. Building on Schmidt’s (1975) GMP, DIVA employs a speech sound map that generates both the motor plan and the expected feedback through feedforward mechanisms. Instead of parameter control, projections between the feedback control and the articulator maps allow for online modifications to the velocity and placement of the articulators, according to task and circumstantial demands (Terband, et al., 2009). Thus, while DIVA is consistent with prior frameworks of motor and speech motor learning, it is more specific on both what is learned for speech production, and the mechanism by which it is learned.

Thus, the tradition surrounding theories on speech motor learning has both paralleled and distinguished itself from the motor learning literature. Parallels are observed in the importance of an abstracted GMP. However, the speech motor literature diverges from non-speech motor literature in that the perceptual-linguistic representation, and therefore, the intention for a communicative outcome for the motor action, appears to be inseparable from the motor plan. Moreover, current models such as DIVA (Guenther, 1994), FACTS (Parrell, et al. 2019), and others, unify the mechanisms underlying speech motor adaptation with how speech motor plans are learned. Specifically, these models converge on the necessity for the integration of auditory and somatosensory feedback to be matched against predictive signals that have originated from preexisting representations.

A common limitation to these models is that the task-specific accommodations are isolated to motor performance. As such, these models do not account for differences in the circumstances surrounding the learning of the speech motor plan. Intuitively, the acquisition of sounds in infancy differs from learning a new pattern for a second language in adulthood. Similarly, correcting errored speech motor patterns, or acquiring new patterns explicitly through clinical intervention, likely differs from the conditions surrounding the implicit emergence of motor plans through typical acquisition. In addition, immersion versus classroom instruction may alter the foreign language learning process (Collentine & Freed, 2004; Freed, et al., 2004; Genesee, 1994). Of course, this is not to imply that the models do not address the development of speech motor control. For example, the implementation of the DIVA model simulates the infantile “babbling” stage of acquisition that results in the learning of the relationship between motor movements and their sensory consequences, followed by an “imitation” phase, etc. (Tourville & Guenther, 2011). To our knowledge however, there are no mechanistic distinctions specified between implicit acquisition, and the kind of learning involved in the explicit learning of a new pattern or in overriding a preexisting one. In other words, the implementation of the model suggests that speech motor learning across instances differs only in the state of underlying knowledge. However, different circumstances under which speech motor plans are learned may drive the learner to employ different strategies.

A well-known dichotomy in learning strategies is whether things are learned implicitly versus explicitly. The notion that there is a division of labor between multiple memory systems to achieve speech and language learning is not new, and has growing empirical support (Chandrasekaran, et al., 2014; Ullman, et al., 1997). These different proposals highlight in common the involvement of hippocampal/medial temporal lobe structures in explicit, fast, declarative learning, and the involvement of frontal-striatal, and cerebellar circuits in implicit, slow (trial-and-error), procedural learning. It has been observed that during the initial process of learning new information, procedural and declarative memory structures are often engaged in parallel (Rasch & Born, 2013). As such, separation of knowledge type to align with specific types of memory and their neural circuits is challenging. While there is no universal agreement for the particular demarcations across domains, disciplines, and tasks, there is growing agreement that there are different networks available for learning, and differences in the way in which things are learned may shift the division of labor between these networks (Chandrasekaran, et al., 2014; Rasch & Born, 2013; Ullman et al., 1997).

Historically, because of the implicit, trial-and-error nature of training for many types of motor learning, it has been assumed that motor learning primarily contains “procedural” skill information. However, there are emerging proposals to suggest that motor learning also involves a division of labor between memory systems. For example, Song (2009) introduces conscious versus unconscious as the key distinction in motor learning, that includes, but is broader than, the implicit versus explicit dichotomy. Within Song’s (2009) taxonomy, conscious learning involves more awareness of the goals of the task at the time of learning, and aligns more closely with the type of learning often thought to be handled by the declarative memory system (e.g. relational memory). In contrast, non-conscious learning may occur incidentally through daily, habitual interactions, and aligns with the kinds of learning thought to be handled by the procedural memory system. Such divisions allow us to consider how differences in the situational circumstances that surround speech motor learning may recruit different memory mechanisms, and how, within the same learning episode, different aspects of the speech motor learning may be encoded by different memory mechanisms.

Moreover, such distinctions in memory mechanisms may highlight different time courses of consolidation processes involved in stabilizing, enhancing, and generalizing the speech motor representation. As we discuss below, the effects of time, and sleep, may facilitate changes to speech motor performance differently for consciously and unconsciously acquired sensorimotor knowledge. These differences in how something was learned, and the events following that learning, likely inform differences in the resultant speech motor representation. These are not trivial considerations given the potential implications for clinical and educational practices. We therefore turn to a review of how time and sleep affect learning across domains relevant to speech motor learning below, followed by a discussion of how this may apply to consciously or unconsciously acquired speech motor information.

In summary, various frameworks highlight the speech motor unit as fundamentally a multi-modal representation, requiring a coordinated effort between systems that learn motoric and acoustic-phonetic sequences, and the message conveyed by the signal. Leading accounts of speech motor performance has moved away from speech motor learning as the establishment of a context-invariant representation, but rather as a dynamic, task-oriented process. Habitual encoding of sensorimotor information and the updating of speech motor representations are implied to be initial and key components of speech motor learning. Beyond the speech motor control literature however, there are emerging reports across relevant domains, such as speech, language, and motor learning, that learning occurs not through a singular memory mechanism but through a division of labor through multiple networks. By analogy, it stands to reason that situational differences in speech motor learning may similarly influence how speech motor plans are learned. Additionally, this implies that there may not be a singular time course of memory processes to be applied to the speech motor representation, as we discuss below.

Memory consolidation in speech motor learning

In the following discussion, we provide an overview of memory consolidation, paying particular attention to the controversies that illustrate the barriers to defining a consolidation framework for speech motor learning. This narrative will highlight the need for considering the discrete aspects of the speech motor representation when predicting a time course of forming new representations. We examine evidence from the domains that are relevant to these aspects of speech motor learning. In doing so, we will arrive at a framework for the time course of building new speech motor representations.

Definitions of memory processes

Memory is considered here as both the means by which we learn, and the outcome of that learning. Memory is comprised of at least three distinct operations referred to as encoding, consolidation, and retrieval (Born, et al., 2006; Gabrieli, 1998). Encoding refers to the initial capture of a memory trace (Born, et al., 2006). Consolidation is an umbrella term for the processing of the memory trace from a vulnerable, labile state to a representation that is more stable (Dudai, 2004). Retrieval refers to our ability to recall the trace, whether it be in its labile, or more permanent, state (Born, et al., 2006). Issues related to encoding and retrieval have been explored previously through models of speech motor control (see Shriberg, et al., 2012, for a memory-based discussion of speech motor learning in relation to disordered speech; Tourville & Guenther, 2011). This discussion will focus on the under-explored issue of consolidation in speech motor learning.

Memory consolidation is thought to be comprised of at least two distinct processes (Dudai, 2004; 2012). The first, synaptic consolidation, refers to localized changes to synaptic strength, that includes both changes to membrane potentials and new dendritic growth (Bramham, & Messaoudi, 2005). These changes may occur within minutes or hours after the encoding of new information or skill practice (Dudai, 2004; 2012). The second process, systems consolidation, refers to the reorganization of information from short-term to long-term store (e.g. hippocampus and cortex, respectively, Dudai, 2004; Rasch & Born, 2013). Computationally, it is posited that systems consolidation reflects the slow integration of new, episodic information, with preexisting declarative knowledge, in order to prevent the new information from overwriting the old (Complementary Systems account of Learning; McClelland, et al., 1995). Sleep is posited to promote systems consolidation by reactivating the wake-state, hippocampal memory trace, allowing for the slow, interleaved integration of knowledge that is often not practical during a conscious state (Dieklemann & Born, 2010). However, systems consolidation has been observed to occur without sleep when the circumstances surrounding learning allows for the slow interleaving of new information with the old (Lindsay & Gaskell, 2013). Thus, while sleep may not be necessary, it is thought provide the interleaving of quickly captured, wake-state information habitually, without conscious effort on the part of the learner (Dieklemann & Born, 2010; McClelland, et al., 1995). Thus, synaptic and systems consolidation, at face value, occur on different time scales, however this may depend on factors such as the structure of training, and when something is learned with respect to a period of sleep.

Such task considerations likewise influence when changes to behavior are expected to occur. To revisit, qualitative changes to behavior that are attributed to memory consolidation include stabilization, enhancement, and generalization (Walker, 2005; Gomez, et al., 2006). The time course of change associated with these behaviors depends on how each behavior is defined. Stabilization has been observed to take place both after minutes to hours following the learning event (Walker, et al., 2003), and also as a function of a period of post-training sleep (Ellenbogen, Hulbert et al., 2006; Ellenbogen, et al., 2007; 2009; see Ellenbogen, Payne, et al., 2006 for review). In the former case, the observed stabilization referred to protection against interference from preventing future enhancement of learned skill (Walker et al., 2003). In contrast, the latter referenced the role of sleep in protecting new information from decay following interfering input (Ellenbogen, Hulbert et al., 2006; Ellenbogen et al., 2009). Thus, it may be reasonable to suspect that stabilization, in the former sense, needed only to separate the target pattern from exposure to overlapping information, that the latter involved some systemic reorganization of information. Similarly, enhancement and generalization may differ in how and when these effects are observed.

Considered under these terms, we may begin to understand the discrepancy in findings in the role of sleep versus wakeful rest in the memory literature across domains. In general, context-specific changes to behavior, such as performance speed, has been observed to occur as a function of time (Robertson, et al., 2004; Song, et al., 2007). By contrast, behavioral changes that require the abstraction and integration of new information with long-term (cortical) knowledge, occurs over a relatively longer period, often inclusive of a period of sleep (Dumay & Gaskell, 2007; Takashima, et al., 2014; Tamminen, et al., 2010; c.f. Coutanche & Thompson-Schill, 2014; Lindsay & Gaskell, 2013). Despite these broad characterizations, it is important to note that behavioral effects of consolidation are not precise demarcations of the processual mechanisms that underlie these changes per se, and thus effects must be considered carefully with respect to the particular learning task.

A further complication to this narrative, particularly in the case of non-speech motor learning, is how the conscious versus unconscious learning dichotomy (Song, 2009) informs the expected time course of changes to behavior. To illustrate, when enhancement is defined as an improvement in the unconscious aspects of the task, such as in the speed of movement, such effects are observed after a period of wake (Cohen, et al., 2005; Cohen & Robertson, 2007; Song, et al., 2007). However, conscious knowledge about the task, such as the explicit awareness of the target motor sequence, has been observed to enhance motor performance following a period of sleep (see Robertson et al., 2004; Walker & Stickgold, 2006, for reviews). Furthermore, the extent to which a learner is aware of a motor sequence is subject to the individual learner’s motivation and attention to the task (Born & Wagner, 2004; Song, 2009), again illustrating the above point regarding the imprecise demarcations of consolidation afforded by behavioral outcomes.

The importance of sleep for the consolidation of consciously learned information may be further modulated by the strength of the initial memory trace. For example, various authors have found that recall performance on learned word pairs (A-B) decays following the learning of partially overlapping pairs (A-C) (Bailes, et al., 2020; Ellenbogen, et al., 2006, 2009; Sheth, et al., 2012). However, the insertion of a sleep-containing delay between the learning of A-B and A-C pairs, appears to protect recall performance on the A-B pairs (Ellenbogen, et al., 2006, 2009; Sheth, et al., 2012). Bailes, et al. (2020) recently reported a failure to replicate the protective effects of sleep as reported in Ellenbogen et al. (2009). In the Bailes et al. (2020) however, participants were not required to meet the same high threshold of learning required in the Ellenbogen et al. study, and participants were asked to complete an interview questionnaire between the learning of the A-C pairs and the subsequent testing of the A-B-C associations. In other words, the protective effect of sleep may depend both on how well the information was learned to begin with, as well as the activity that follows the learning of the partially overlapping information.

The above discussion illustrates the inherent challenges to defining a memory consolidation account of speech motor learning. To revisit, speech motor learning requires not only the learning of a new motor plan, but a complementary perceptual-linguistic representation to serve as a predictive signal for the intended outcome (i.e. goal) of the production (Parrell, et al. 2019; Tourville & Guenther, 2011). In other words, despite a common assumption that motor learning reflects procedural learning, motor performance may reflect a division of labor between procedural and declarative memory systems, as illustrated by movement information acquired unconsciously, and the goals of the movement (e.g. target sequence), that is consciously learned (Cohen, et al., 2005; Robertson et al., 2004; Song, 2009). Thus, in order to bridge the gap between speech motor learning and the memory consolidation literature, we must consider how this division of labor leads to different effects of wake versus sleep, with particular attention paid to how learning outcomes are measured. Learning outcomes that measure aspects of speech motor learning that are acquired unconsciously, may improve as a function of post-training rest. In contrast, behaviors that reflect systemic reorganization of speech motor knowledge, or abstraction of the speech motor plan, may emerge following a period of post-practice sleep. Furthermore, relational information about speech motor plans, such as the goal of the motor sequence mapped onto a semantic category, may or may not be stable following sleep, depending on how strongly that association was forged. In order to better examine consolidation effects acting upon these distinct aspects of the speech motor plan (e.g. the goal versus movement-specific information), we look to the literature in adjacent domains below.

Relevant examples of memory consolidation effects

To the point of this review, we are lacking in the direct empirical observations that may begin to address how memory consolidation may affect speech motor learning. In the absence of direct evidence from the speech motor learning literature, we will briefly summarize the evidence from relevant domains: motor, auditory-perceptual, speech-perceptual, and language learning. The following discussion will review the available evidence with respect to our predictions as stated above, and highlight the areas in need of further research.

Motor skill learning.

The stabilization and enhancement of motor skill over practice and time has been studied for decades (see Robertson et al., 2004, and Walker & Stickgold, 2006 for reviews). Periods of post-learning, wakeful rest have been observed to result in improvement in motor learning behavior (Craig, et al., 2015; Peigneux, et al., 2006; Wamsley, 2019) and changes to functional resting state connectivity in the regions involved in the learning task (Albert, et al., 2009; Vincent, 2009). Some studies suggest that post-training enhancements (i.e. increased task speed) on a motor skill are specific to an offline period spent in sleep (Desrocher, et al., 2016; Walker, et al., 2002). Others have observed that while rest and sleep may both facilitate motor memory enhancements, the magnitude of consolidation effects may be larger for a period spent in sleep over wake (Backhaus & Junghans, 2006; Walker, 2005). The magnitude of sleep effects may also interact with development and/or maturation. Specifically, task performance in younger children has been observed to improve to a greater degree following a period of sleep than in older children (Ren, et al., 2016; Yan, 2017). Post-training sleep has additionally been demonstrated to decrease motor errors and variability, which are hallmarks of increased automaticity in motor skill (Fischer, et al., 2002; Hill, et al., 2008). In summary, the exact nature of how state interacts with such changes to motor performance has not yet been clearly defined, however there appears to be broad consensus that a post-training period spent in the absence of interference (whether spent in sleep or wake) improves speed and accuracy.

Another important consideration may be in the quality of wake-state experience that follows learning. Walker et al. (2003) demonstrated that the consolidation effects following motor sequence (i.e. finger-tapping) learning differs according to the events that follow training. The authors observed that those who are trained in just one (target) sequence make gains in speed and accuracy without further training, when tested again at 24 and 48 hours. A different set of learners were given a second (interference) sequence to learn immediately after the first. While the behavior on the target sequence was unaffected when tested immediately after learning, performance enhancements at 24 hours were observed only for the second sequence. Furthermore, it was observed in another group that delaying the presentation of the second sequence by 6 hours after learning the first prevented the interference effect, suggesting that stabilization of the first sequence occurred during that time. Interestingly, presenting the second sequence immediately following a retest of the first (that is, upon reactivation of the first sequence), again resulted in a latent decline in performance instead of continued improvement on the third day. Taken together, this suggests a time course of time-dependent stabilization and sleep-dependent enhancement with periods of vulnerability immediately after learning and upon every reactivation of that learning thereafter.

There exists a clearer case for the particular role of sleep for establishing representations of motor programs, or goals, that are independent of the context in which the motor program was practiced during training (Cohen, et al., 2005; Witt, et al., 2010, see Censor, 2013, for review). Consistent with this view, sleep appears to be necessary for the improvement of performance on a skill that has not been practiced by the participant, but merely observed and performed via imitation, in both infants and adults (Konrad, et al., 2016; Konrad, et al., 2019; Seehagen, et al., 2015; Van Der Werf, et al., 2009). Such abstraction may involve sleep’s role in forgetting non-salient information (see Feld & Born, 2017, for review), that allows for only the distinctive features to be extracted and represented. Establishing a GMP for motor learning may further enhance motor performance, which may explain why the magnitude of consolidation effects are at times observed to be greater after sleep versus a comparable period of wake (Backhaus & Junghans, 2006; Walker, 2005).

The above observations are succinctly accounted for by a proposed framework on the roles of sleep and wake on motor learning by Robertson (2009). According to this framework, memory systems involved in both unconscious (i.e. movement-specific) and conscious (i.e. goal of the movement) learning share neuronal resources, and thus compete during the initial learning event. After initial learning, the framework posits that the movement components of a novel task are consolidated during time spent in a wake state, whereas the learned goal of the movement sequence is consolidated during sleep (Robertson, 2009). The model also predicts that the relative contributions of wake and sleep processes are determined by the structure of the practice (Robertson, 2009). For example, distributed practice facilitates more goal learning, and thus performance is more likely to improve following sleep. In contrast, massed practice promotes the learning of actual movements, and performance is therefore likely to improve following wake state (Robertson, 2009; Hikosaka, et al., 1999; Hikosaka, et al., 2002). In motor practice that engages both memory systems in parallel, consolidation processes are predicted to act continuously on the memory trace over the following wake and sleep states (Robertson, 2009).

In speech motor learning, we expect the same division of labor between memory systems involved in unconscious (including movement) and conscious (including acoustic-linguistic goals) learning (see Figure 1). We may see this division reflected in different time courses of change to behavior for different components of the speech motor plan, or following speech motor learning with different practice structures (see discussion below).

Figure 1. Division of labor for speech motor learning and involvement of wake versus sleep.

Through exposure and practice, individuals learn both the movement and the goal components of the speech motor plan. Learning the movement component and the goal component occurs in parallel, and yet are accomplished by memory systems that are dissociated (Robertson, 2009; Song, 2009). As such, speech motor learning is achieved through a division of labor between the unconscious and the conscious learning systems, where the former processes contain movement-based information, such as the phonetic information about placement of articulators, and the latter processes goal-based information, such as the correspondence between communicative meaning and the phonemic sequence.

Auditory learning.

Post-training wakeful rest and sleep has been shown to benefit memory consolidation of auditory perceptual information (Schroger, et al, 1992; Schroger, 1994; Schroger, 2007). For example, both sleep and restful wake were found to improve identification performance of auditory tone patterns (Gottselig, et al., 2004). Atienza, et al., (2004) observed latent gains in auditory identification performance after both wakeful rest and sleep, however, mismatch-elicited responses did not emerge until a period of sleep had taken place. This finding resonates with our suggestion above that predictive signals for error-driven learning originate from representations that are established during sleep. Other sleep-specific gains have been observed in an auditory pitch memory task (Gaab, et al., 2004) and in a complex auditory pattern discrimination task (Altienza, et al., 2002).

Speech-perceptual learning.

In the domain of speech-perceptual learning, post-training performance gains in the absence of additional practice have been observed with provision of both wakeful rest and sleep. For example, Roth, et al. (2005) demonstrated that following a single training session, gains in the ability to identify consonant-vowel syllables within background noise appeared in participants following a four-to-six hour-period of wakeful rest. A period of sleep, as opposed to a comparable period spent in wake state, has been observed to facilitate enhanced perceptual performance on nonnative contrasts (Earle & Myers, 2015b; Qin & Zhang, 2019). Parallel to the motor learning literature, a clearer case for the need for sleep appears for outcome measures that provide evidence that a context-independent representation has emerged. For example, overnight sleep has been shown to facilitate cross-talker generalization of auditory-perceptual training on nonnative speech sounds (Earle & Myers, 2015a), and foreign-accented speech (Xie, et al., 2018).

Further parallels are observed between the speech-perceptual learning literature and that of non-speech motor learning in that the quality of post-training experience appears to mediate the observed consolidation effect. For example, in Fenn, et al. (2003), it was observed that those who were trained to identify sine-wave speech in the morning appeared to decline in their performance after a 12-hour daytime interval, but that this information was recovered in the following overnight interval. By comparison, no such decline was observed in those trained in the evening, suggesting that a period of sleep soon after training protected the target information from decay. In another example by Earle and Myers (2015b), perceptual performance on a trained nonnative contrast was measured immediately after training, and again at roughly 12 and 24 hours later, in two groups of learners. The evening-trained group made continuous gains at each retest. By contrast, the morning group maintained stable performance for the first 12 hours, and then declined in their performance overnight. It was suggested that the exposure to native language immediately after training for the morning group interfered with the consolidation of the target. Subsequent experiments demonstrated that focused exposure to native language immediately after training did not affect performance on the trained contrast immediately after learning, but did prevent overnight enhancement.

Together, these studies present a time course of offline enhancement and stabilization that resonates with the motor learning literature. However, it is important to underscore the point that the ‘interference’ in these studies was not, as in Walker et al. (2002), conflicting information to the target that was actively trained. Rather, passive exposure to spoken language was considered to be in conflict with the target speech information. Therefore, we might assume that the act of using spoken language soon after speech motor learning would similarly introduce interference effects on consolidation of the target motor plan.

Language learning.

The benefits of memory consolidation to language learning have been demonstrated through two decades of research (see Schreiner & Rasch, 2017 for review), in both adults and in children. In phonology, sleep has been found to improve the application and generalization of learned phonotactic constraints in the production of syllables (Gaskell, et al., 2014). There is a robust literature describing the role of sleep in the latent emergence of lexical competition effects following the learning of a novel word form or semantic features (Davis, et al., 2009; Dumay & Gaskell, 2007; Kurdzeil, et al., 2017). The direct recall of the word form however has been observed to improve over a short (3-hour) delay (Brown, et al., 2012). Interestingly, the same study found that performance on a cued recall task (using the first syllable to predict the rest of the form) improved only after a period of sleep (Brown, et al., 2012). This may suggest that direct versus cued recall may rely on access to different stores of information.

In grammar learning, a period of sleep, but not a comparable period of wake state, has been found to facilitate retention, generalization and abstraction of grammatical forms in children and adults (Batterink & Paller, 2017; Gomez & Gerken, 1999; Gomez, et al., 2006; Simon, et al., 2017). In a recent study comparing the learning of regular and irregular grammatical forms, it was found that the tendency to apply irregular constructions to novel items increased over time spent in sleep (Mirkovic, et al., 2019). Taken together, measures of language that change over sleep appear to share in common the property that change in representational status has taken place, that is, the acquired information appear to exert new influence over task performance following sleep. Furthermore, the Brown, et al., (2012) finding resonates with the hypothesis that sleep facilitates the generation of predictive signals. By contrast, performance enhancements that require no reorganization of information appear to take place as a function of time (Brown, et al., 2012).

As mentioned in the discussion on memory above, sleep may promote, and yet not be necessary, for systems consolidation to take place. To illustrate, Lindsay and Gaskell (2013) found that, given certain conditions, individuals were able to integrate novel words into their current lexicon within a single day without sleep. The authors conducted several experiments that involved various assessments of learning following exposure to a novel word form. In a subset of the experiments, a lexical familiarity task that involved an interleaved presentation of new and pre-existing lexical forms were administered on the same day as the learning of target word forms. For these, but not the other, experiment(s), lexical competition effects were observed without a period of sleep. Thus, the authors claim that explicitly interleaving the novel word forms with the old, thereby simulating memory “replay” events that occur during sleep (see Foster, 2017 for an excellent review), may have been sufficient for systems consolidation to take place.

Similarly, Coutanche & Thompson-Schill (2014) found that through fast-mapping, novel lexical items could be integrated into established lexical networks without sleep. Fast-mapping is an implicit word learning procedure that maps a new form onto a pre-existing lexical item. In the study, fast-mapping was found to facilitate lexical competition effects following only two presentations of the items in the absence of sleep. In contrast, the explicit word learning procedure required sleep for lexical competition effects to emerge (Coutanche & Thompson-Schill, 2014). In other words, lexical competition effects appear to have emerged through the implicit modulation of pre-existing representations in the fast-mapping condition, whereas in the explicit learning condition, newly acquired word forms were integrated with the mental lexicon offline. Therefore, the mechanism underlying the emergence of lexical competition effects for fast mapping may not be the same as the narrative surrounding the sleep-mediated systems consolidation of explicit/declarative memory.

We defined memory consolidation and the different mechanisms involved in time versus sleep-mediated effects on behavior. Our discussion included observed changes to behavior following wake and sleep, and the factors that modulate the timing of when these effects are observed. We briefly reviewed the effects of post-learning time spent in rest and/or sleep in human learning for language, motor, auditory, and speech-perceptual domains. Across these literatures, there are striking similarities between these accounts. In general, a post-training offline period, spent in either wake or sleep state, appear to promote performance across domains. For example, a period of time post-training in the absence of conflicting input appears to result in a relative stabilization of information (Earle & Myers, 2015b; Ellenbogen et al., 2009; Walker et al., 2002). Moreover, improvements have been observed in performance to occur after a period of time in the absence of further training, with some advantage observed for sleep-containing periods (Backhaus & Junghans, 2006; Gaab, et al., 2004; Earle & Myers, 2015b ; Walker, 2005). There has also been reports of sleep-dependent changes to behavior that imply that qualitative changes to the representation have taken place. These changes include an integration of the new information with the old (Davis, et al., 2009; Dumay & Gaskell, 2007; Kurdzeil, et al., 2017), and also loss of fidelity to the conditions under which learning took place, allowing for broader, context-independent application of the relevant features to the task (Batterink & Paller, 2017; Earle &Myers, 2015a; Gomez & Gerken, 1999; Gomez, et al., 2006; Konrad, et al., 2016, 2019; Seehagen, et al., 2015; Simon et al. 2017; Van Der Werf, et al., 2009). These qualitative changes resonate with the kind of processes that would be required for the stable and flexible speech motor representation (e.g. GMP) required for the production of speech.

Our review of the motor learning literature included discussion on a division of labor between conscious and unconscious learning (Song, 2009), and the time course of consolidation to follow each learning type (Robertson, 2009). Accordingly, various components of the newly learned speech motor program are predicted to benefit differently from a post-practice period spent in wake versus sleep. Specifically, a period of time spent in wake may promote the memory consolidation of speech motor movements (e.g. gestures), while sleep may facilitate consolidation of the goals of that movement (e.g. sound-phoneme association, combinations of learned speech motor gestures for phonotactic patterns, suprasegmentals, and the communicative message itself) (see Figure 2).

Figure 2. Time course of memory consolidation and speech motor representation development.

It is predicted that following training in a novel speech task, different aspects of speech motor information undergo memory consolidation at different phases of the wake/sleep cycle (see Overview of memory consolidation). For speech motor information about the movement, consolidation during wake assists the speech motor gestures to become more automatic. For speech motor information concerning the goals of movement, consolidation processes during sleep facilitate the speech motor representation to move from a rigid (contextually bound) state to one that is flexible. This flexibility allows the representation to respond in an anticipatory manner to the motor conditions of the talker as well as to the environmental demands of the communicative context. Together, the long-term representation serves as a corrective (e.g. “feedforward” predictive signal in the DIVA model, Guenther, 1994) signal for the continued automatization of the motor plan.

As we have observed in the auditory learning literature, a consequence of sleep-mediated consolidation may be in facilitating the generation of predictions associated with the acoustic goal (Atienza et al., 2004). This suggests that there may be a delay between speech motor practice and the emergence, or modification of, the predictive signals that are critical to the feedforward mechanisms of speech motor performance (Guenther, 1994; Tourville & Guenther, 2006). To illustrate, the correction an errored speech motor goal (such as during speech intervention) may require a period of sleep following practice in order to establish its own predictive signal. Until then, the old (incorrect) pattern may be used as the default predictive signal, resulting in the unlearning of targeted speech motor behavior. Furthermore, this prediction is supported by the interference effects observed in the auditory-speech learning domain (e.g. Earle & Myers, 2015; Fenn et al., 2003). Parallel to auditory-speech learning, goal learning for speech motor plans might be improved by limiting exposure to old patterns between practice and sleep (Earle & Myers, 2015).

Finally, observations from language learning suggests that while sleep may facilitate the generalization and enhancement of goals for speech motor learning, that the same effects of systems consolidation may be mimicked through manipulations of practice structure. Specifically, interleaving the presentation of the corrected, along with the old (errored) speech motor plan may facilitate faster consolidation of the speech motor goal (Lindsay & Gaskell, 2013). Furthermore, engaging the unconscious learning circuit in acquiring not just the movement-specific information, but also the goal of the speech motor movement, may facilitate faster cortical integration of the speech motor goal (Coutanche & Thompson-Schill, 2014). This may allow for some flexibility in the establishment of speech motor goals, such that those with limitations in consolidation during sleep may have other means by which to accomplish this feat.

Indeed, consolidation of the speech motor representation may employ multiple memory consolidation mechanisms. Of which, the division of labor and relative contribution of these mechanisms may reflect not only how individual aspects (i.e. movements and goal) of the speech motor gesture was learned (i.e. unconsciously/implicitly versus consciously/explicitly) as well as how the gesture was practiced (e.g. massed versus distributed) (Hikosaka, et al., 2002, Robertson, 2009; Song 2009). Taken together, when learning a novel speech motor gesture, there is an intricate and complicated interaction between these learning and consolidation effects. Thus, the awareness of the learner (unconscious or conscious), the context of learning (implicit or explicit) the training specifications (massed or distributed) and the conditions of post-practice time (i.e. wake or sleep) will impact the observed speech motor behavioral outcomes (movement-based component enhancement or goal-based components enhancement). We conclude this review with a discussion of the potential implications for the field of speech motor learning below.

Discussion

Speech is one of the most intricate and refined motor skills that humans learn and perform (Weismer & Fennell, 1985). Reviewed here, current theoretical frameworks of speech motor control emphasize speech motor planning and production centered on the systematic relationships between articulator movements and their acoustic, auditory, and somatosensory consequences. We identified several gaps in current frameworks concerning how this multimodal information establishes its long-term representation in the speech motor control network. As a first step toward bridging these gaps, this review has explored how memory consolidation processes impact learning across domains that are relevant to speech motor learning. Through this review, we proposed parallel roles for post-training wakeful rest and sleep in the stabilization, enhancement, and generalization of information observed in other domains, to promote the building of new speech motor plans. Specifically, we predicted how learner state (conscious versus unconscious learning) as well as the structure of the training (massed versus distributed) affect the relative contribution and distribution of memory consolidation mechanisms within a division of labor memory framework.

This empirical gap exists with good reason. There is an inherent difficulty in decomposing the speech motor unit into discrete types of information (e.g. auditory-perceptual, linguistic, motoric), which may develop their representations along different time scales. Moreover, if we suppose that speech motor learning is subject to similar effects of rest, sleep, and interference as observed in other domains, we begin to see the challenges in the implementation of empirical work that would test these claims. For example, how might we ethically facilitate a period of speech motor rest after practice? In reality, any test of consolidation effects on speech motor learning may, by necessity, be inclusive of interfering motor patterns. Or perhaps, the interference effects may only be applicable for new information (c.f. Earle & Myers, 2015). These are challenges that will need to be addressed in order to advance this framework.

Advancing this framework however may be well worth the trouble in its potential applications. Such work may provide a new model for examining potential mechanisms of deficit, or generate recommendations for treatment, for populations with disordered speech. While habitual sleep deficits were not discussed in this review, it should be noted that disrupted and fragmented sleep architecture, including the sleep experienced by those with obstructive sleep apnea, have been demonstrated to be related to speech production difficulties and deficits, including subtle, but atypical, acoustic differences (Caspari, et al., 2008; Fiz, et al. 1993; Goldshtein, et al., 1987). Sleep disruption has also been demonstrated to be a risk factor for language learning difficulties in both children and adults (de Castro et al., 2017; Earle, et al., 2018; McGregor & Alper, 2015; Morrow & Duff, 2020) and for behavioral and/or neurocognitive dysfunction (Beebe & Gozal, 2002; Curcio, et al., 2006; Kim, et al., 2011; Landau, et al., 2012). As such, the impact of habitual sleep disturbance on speech motor learning is a relevant topic for future investigation. Moreover, understanding the memory mechanism of speech motor learning may inform treatment protocols as well as the scheduling and delivery of speech and language services. For example, scheduling interventions close to a period of sleep, or avoiding too much language practice between intervention and sleep, may facilitate faster progress toward target goals. In addition, distributed practice may stress the importance of the motor goal for subsequent periods of consolidation, whereas massed practice may facilitate the consolidation of movement-specific information. Finally, there is the potential for broader application (beyond disorders) of this framework to academic instruction, and adult learning programs, such as in foreign language training (e.g. when and how to practice pronunciation versus learning foreign language vocabulary).

It is important to acknowledge the limitations of the current review. Specifically, the scope of this paper did not allow for a discussion of reconsolidation. Consequently, our presentation may render memory consolidation as a discrete event that concludes with a memory trace becoming fixed, however there is considerable evidence to suggest that memory traces are thought to be repeatedly rendered labile through reactivation, to be once again stabilized and reconsolidated (Alberini & LeDoux, 2013; Dudai, 2012; Dudai & Eisenberg, 2004). Given the malleable and plastic nature of speech motor representations (see Theories of Speech Motor Control and Learning), the role of reconsolidation in speech motor learning is a highly compelling topic.

Another related issue that may be highly relevant to speech motor learning is the role of consolidation in forgetting (Feld & Born, 2017). For example, in potential clinical applications of this framework, the purging of incorrect speech motor information may be critical to modifying an undesired speech motor behavior. In addition, adults speaking a nonnative language may require the destabilization of a preexisting (native) speech motor pattern in order to reduce their accent. Furthermore, there may be mechanistic differences between learning a new speech motor representation and plan versus altering a pre-existing representation and/or plan. These are important questions to address empirically through future work.

This was a first attempt in addressing the role of memory consolidation in speech motor learning. Leveraging our knowledge from the broader memory and learning literatures may inform our understanding of speech motor learning processes. We examined the existing literature on memory and learning processes from relevant domains, and applied these insights to speech motor learning, in order to arrive at predictions for how humans learn to produce speech. These predictions offer exciting new directions for both theoretical and applied research, including the use of planned wakeful rest and sleep to optimize speech motor learning.

Open Practices Statement.

There was no experiment or new data reported in this manuscript.