Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 May 27.
Published in final edited form as: Psychol Res. 2014 Apr 27;79(2):318–326. doi: 10.1007/s00426-014-0568-2

Crossmodal encoding of motor sequence memories

Marianne A Stephan 1,, Brittany Heckel 1, Sunbin Song 1, Leonardo G Cohen 1
PMCID: PMC4883107  NIHMSID: NIHMS787270  PMID: 24771059

Abstract

In this study, we tested the hypothesis that exposure to specific auditory sequences could lead to the crossmodal induction of new motor memories. Twenty young, healthy participants memorized a melody without moving. Each tone in the memorized melody had previously been associated with a particular finger movement. For ten of the participants, the contour of the melody memorized was congruent to a subsequently performed, but never practiced, finger movement sequence (C group, n = 10). For the other ten participants, the melody memorized was incongruent to the subsequent finger movement sequence (InC group, n = 10). Results showed faster performance of the movement sequence in the C group than in the InC group. This difference in motor performance was most pronounced 6 h after melody learning and then dissipated over 30 days. These results provide evidence of a specific, crossmodal encoding of a movement sequence representation through an auditory sequence with the effect on motor performance lasting for several hours. The findings of this study are significant, as the formation of new motor memories through exposure to auditory stimuli may be useful in rehabilitation settings where the initial encoding of motor memories through physical training is disrupted.

Introduction

There is emerging evidence that auditory information can help people remember, produce, and learn movements. Evidence of this crossmodal effect comes from studies in both musicians who have highly-developed auditory-motor associations and non-musicians who may acquire new auditory-motor associations by for example training to perform simple melodies. For example, tones have been shown to facilitate the learning of finger movement sequences if they are mapped onto the movements in a congruent manner (i.e., movements from left to right are associated with tones of ascending pitch) (Hoffmann, Sebald, & Stöcker, 2001; Stöcker, Sebald, & Hoffmann, 2003). In a study by Rusconi, Kwan, Giordano, Umiltà, and Butterworth (2005), motor responses were more accurate and faster when a lower (or leftward) key had to be pressed in response to a low sound and an upper (or rightward) key in response to a high sound, even though pitch height was irrelevant to the task. Similarly, Keller and Koch (2008) provide evidence that action planning is faster with congruent key-tone mappings (that is, taps on top, middle, or bottom keys trigger high, medium, and low pitched tones, respectively) than with incongruent key-tone mappings. Further evidence for a close interaction between auditory and motor memories comes from a study by Bailes, Bishop, Stevens, and Dean (2012), which showed that patterns of loudness in musical scales were remembered less accurately if participants were asked to observe and remember movement sequences immediately after encoding of the loudness patterns. In addition, Brown and Palmer (2012) have shown that motor learning can enhance performers’ auditory recognition of melodies when compared to auditory learning alone. Moreover, listening to a previously practiced piano piece can lead to motor performance improvements without additional physical practice (Lahav, Boulanger, Schlaug, & Saltzman, 2005).

The beneficial effect of sound on motor performance has also inspired the idea of music-supported therapy (MST), an approach which uses sound as a neuromodulatory tool to improve motor function in patients with movement disorders (Rodriguez-Fornells et al., 2012). The idea of inducing plastic neuronal changes in the motor system by way of other sensory modalities is intriguing and could be viewed as a “backdoor” to the motor system in cases where physical motor training is difficult (Sharma, Pomeroy, & Baron, 2006). Finally, numerous studies have demonstrated an increased response and/or excitability in the primary motor cortex (M1), premotor cortex (PMC), and supplementary motor area (SMA) in musicians and, after sufficient auditory-motor training, in non-musicians during passive listening to known melodies using fMRI or TMS (Bangert et al., 2006; Baumann et al., 2007; D’Ausilio, Altenmüller, Olivetti Belardinelli, & Lotze, 2006; Chen, Penhune, & Zatorre, 2008). These close interactions between auditory and motor brain areas may represent the neural basis for the mutual influences between auditory and motor memories.

Until now, studies attempting to understand crossmodal memory formation have mainly focused on enhancing the consolidation or retention of an already existing motor memory through movement observation, motor imagery, or reactivation of the memory through the associated sounds (e.g., Antony, Gobel, O’Hare, Reber, & Paller, 2012; Lahav, Katz, Chess, & Saltzman, 2013). In contrast, the current study explored whether auditory information alone could facilitate the formation of a new motor memory and what the time course of that facilitation would be.

We thus compared the ability of non-musicians to perform a motor sequence that was either congruent (for the C group) or incongruent (for the InC group) with a melody which they listened to previously.

We hypothesized that exposure to a melody composed of movement-associated tones would lead to the encoding of motor memory traces, whose influence would be evidenced by better motor performance (faster response times) in the C group when compared to the InC group immediately following the melody memorization. An additional interest of this study was to investigate motor performance differences between groups in the hours and days after melody memorization.

Methods

Subjects

Twenty healthy adults (10 men, 10 women; age: mean (M) 27.0, standard deviation (SD) 5.44) without psychiatric or neurological disorders participated in the present study. They were all right-handed, according to the Edinburgh Handedness Inventory (Oldfield, 1971). Each participant was randomly assigned to one of two experimental groups (C group, InC group). Twelve out of 20 participants had previous musical experience (6 of the C and 6 of the InC group). There was, however, no difference in either the total number of years of musical training (C: M 4.6, median (Mdn) 2, mean rank 10.2; InC: M 2.7, Mdn 2.5, mean rank 10.9; U = 46.5, p = 0.40, r = 0.061), or in the years since last musical training (C: M 14.2, Mdn 12, mean rank 10.3; InC: M 7.83, Mdn 3, mean rank 10.8, U = 23.5, p = 0.82, r = 0.20) between groups, as revealed by two-sided Mann–Whitney U tests. All participants gave written informed consent prior to their inclusion in the study. The study conformed to the principles of the Declaration of Helsinki and was approved by the National Institute of Neurological Disorders and Stroke (NINDS) Institutional Review Board.

Tasks and procedure

Twenty young, healthy participants were instructed to memorize a short 12-note melody composed of four movement-associated tones (each tone was previously associated with a particular finger movement). The contour of each melody was either congruent (melody A, C group) or incongruent (melody B, InC group) to a subsequent motor sequence task. In this motor task, participants had to press keys as quickly and as accurately as possible in response to a repeating, 12-item sequence of visual stimuli appearing on a computer screen (motor sequence A). Motor performance was tested immediately, 6 h, 24 h, and 30 days after melody memorization (Fig. 1).

Fig. 1.

Fig. 1

Experimental design. 1 Auditory-motor association: participants learned to associate each of four tones with a particular finger movement by performing random key presses with sound feedback (“piano-playing” task). 2 Motor baseline: participants responded with key presses to circles appearing in a random order on a computer monitor. 3 Melody memorization: participants were then instructed to memorize either Melody A (congruent (C) group) or Melody B (incongruent (InC) group). 4 Motor performance: immediately after melody memorization, as well as 6 h, 24 h, and 30 days later, participants performed motor sequence A (Seq A). After the performance of Seq A 30 days after melody memorization, 7 out of 10 participants in each group also performed a motor sequence B (Seq B)

Auditory-motor association

To establish a basic association between the key press responses and the specific tones, participants performed a “piano-playing” task for about 30 min where they were required to randomly press four keys on a computer keyboard using the fingers of their right hand. Index, middle, ring finger, and pinky finger were pre-assigned to the v, b, n, and m keys. Each key produced a particular tone (v = C3 (130.81 Hz), b = E3 (164.81 Hz), n = G3 (196.00 Hz), m = C4 (261.63 Hz)). Each participant performed four runs of 360 key presses each, with short breaks in between as needed. Immediately before and after this task, participants performed a 1-min auditory-motor facilitation test in which they had to respond as quickly and as accurately as possible to circles appearing at one of four different horizontal positions on a monitor with key presses (with the m key for the most rightward circle, the v key for the most leftward circle, etc.). Simultaneously with each visual stimulus, participants heard a tone, which was irrelevant to the task. We expected that the hearing of each tone would facilitate the associated motor response, that is, hearing of the lowest tone C3 would facilitate the most leftward key press v, E3 would facilitate key press b, and so forth (see also Rusconi et al., 2005). Each combination of visual cue and tone occurred with equal chance, i.e., in each trial, the motor response cued by the visual stimulus was expected to be either facilitated by the concurrently sounding tone (in congruent combinations such as leftmost circle and tone C3) or not (in incongruent combinations such as leftmost circle and tone G3). Participants did not get any performance feedback, that is, their key presses did not evoke any tone or visual feedback. Regardless of whether their response was correct or wrong, the subsequent combination of visual cue and tone was displayed immediately after the key press. The aim of this task was to determine whether there was any difference between groups in the effect of the tones on motor responses at baseline.

Motor baseline

Thereafter, in response to circles appearing at one of four different horizontal positions on a computer monitor, participants were asked to press the corresponding keys as quickly and as accurately as possible (that is they pressed the v key with their index upon presentation of the first circle, the b key with their middle finger upon presentation of the second circle, and so forth). The circles appeared in a random order for a baseline determination of motor performance. The subsequent circle was displayed immediately after each key press, regardless of whether the response had been correct or not. The motor baseline task consisted of 192 key presses and lasted for about 2 min. No auditory input or feedback was provided.

Melody memorization

Participants were told that they were going to hear a repeating melody, and that they should try to remember this melody, since they would be asked later on to write it down and to hum it. The melody was comprised of the four movement-associated tones and was 12 tones long (Fig. 2). The melody was repeated 30 times during each of the three memorization sessions. It was thus heard a total of 90 times over the course of approximately 20 min. During the memorization sessions, participants were instructed to relax and to refrain from movement. Participants’ hands were monitored closely by visual observation. In addition, EMG activity from the first dorsal interosseus muscle was monitored to support visual motion observation, using the muscle belly tendon technique with silver surface electrodes. The melody was either congruent (melody A, C group) or incongruent (melody B, InC group) to the subsequent motor sequence performance task (Fig. 1). The two experimental melodies such as the tones used during the auditory-motor association phase were created with the ‘GarageBand’ music editing software (GarageBand 6.0.4, Apple Inc. 2011) and had a synthesized piano timber (see Supplementary Material, MelodyA.mp3, MelodyB.mp3). The duration of each of the melodies was 11 s and consisted of quarter notes only, i.e., the melodies were isochronous. There was a 2-second time interval between melody presentations. There were no pitch repetitions in either melody and each pitch was represented exactly three times. The transitions C3 to C4 and C4 to C3 (the largest pitch distance) occurred once each in both melodies.

Fig. 2.

Fig. 2

Melodies and motor sequences. Melody A was memorized by the congruent group (C), and Melody B was memorized by the incongruent group (InC). Motor sequence A was tested in both groups immediately, as well as 6 h, 24 h, and 30 days after melody memorization. Motor sequence B was tested in both groups after the performance of sequence A 30 days after melody memorization

Motor performance

The procedure for the motor performance tests was the same as the procedure used for the motor baseline task. Participants were asked to press the corresponding keys as quickly and as accurately as possible in response to circles appearing at one of four different horizontal positions on a computer monitor. Index, middle, ring finger, and pinky finger were pre-assigned to the v, b, n, and m keys, respectively. However, in contrast to the baseline task, the circles in the motor performance tests did not appear randomly, but instead appeared in a particular 12-unit sequence, referred to as sequence A, (Fig. 2) which was repeated 30 times (lasting about 4 min). Participants performed motor sequence A immediately after melody memorization. In order to determine how long the memorized melody would affect motor performance, motor performance was retested 6 h, 24 h, and 30 days after melody memorization (Fig. 1). Immediately following the final assessment 30 days after melody memorization, 7 (out of 10) participants in each group also performed motor sequence B. (Motor sequence B was congruent to melody B, and melody B was only memorized by the InC group.)

Explicit knowledge of the melody

Explicit knowledge of the melody was assessed after each of the three memorization sessions in order to keep participants focused on the melody and to determine if both groups were able to memorize the melody equally well before the participants’ motor performance was tested. First, participants were asked to hum the melody into a microphone. Recorded melodies were scored off-line by counting the longest sequence of tones with the correct contour (considering only relative changes in pitch, e.g., if the tone was higher or lower, ignoring mistakes in absolute pitch intervals). Secondly, participants were instructed to write the melody out by drawing circles on a grid with four horizontal lines. The score for the written melody was calculated by counting the longest sequence of written tones with the correct contour when considering only relative changes in pitch. The mean of the third administration of the hummed and written melody memorization assessments was used as the measure of participants’ explicit knowledge of the melody. This was done in order to test whether the ability to memorize melodies is a precondition related to the amount of auditory-motor learning. In addition, at the end of the memorization task, each participant had to indicate on a questionnaire the strategy used to memorize the melody, in order to determine whether any participant used a memorizing strategy related to finger movements.

Explicit knowledge of the motor sequence

Explicit knowledge of the motor sequence was assessed after each motor sequence performance test. The purpose of these assessments was to check whether differences in explicit knowledge of the motor sequence between groups might have influenced procedural motor performance. Participants were asked to write out the sequence of finger movements they made during the motor sequence performance test using a number to represent each finger (index finger 1; middle finger 2; ring finger 3; pinky finger 4). The score for this assessment was determined by the length of the longest correctly reported sequence. For two of our study participants, this written assessment was unfortunately only administered once, 30 days after the initial memorization of the melody.

Analysis and results

Statistical analyses were performed with R (R Core Team, 2012). For all outcome measures, normality and homogeneity of variances were tested using Shapiro–Wilk and Bartlett tests. Non-parametric tests were used for data analysis when appropriate (non-parametric permutation-based analogs of the performed mixed factorial ANOVAs confirmed the results reported below (Wheeler, 2010)). P values of motor performance and explicit motor sequence knowledge tests, assessed at multiple time points, were corrected for multiple comparisons according to Benjamini and Hochberg (‘BH’ method). Data are presented as mean (M) and standard deviation (SD), and/or median (Mdn) and mean rank.

Auditory-motor association

During the auditory-motor association task (“piano-playing” task), participants pressed all four keys and used all possible 16 key transitions (for example, pressing v and then b) roughly the same number of times, as revealed by histograms of key presses and key transitions for each participant. In order to analyze baseline auditory-motor facilitation, the first key press time was discarded in each block, as it was sometimes very prolonged due to a lack of attention when the task started. After discarding the first trial, the change in error rate (the percentage of false responses) from before to after the auditory-motor association phase was calculated for incongruent trials (delta InC) and congruent trials (delta C). Delta C was then subtracted from delta InC to get a measure of the relative increase in errors in incongruent trials when compared to congruent trials for each participant. The same measure was calculated for the mean response time. Mann–Whitney U tests revealed no difference in auditory-motor facilitation between groups at baseline (delta errors U = 34, p = 0.24, r = 0.27; delta RT U = 43, p = 0.63, r = 0.12).

Motor performance

After discarding the first key press time of each motor test, the mean response time of correct responses was calculated for each subject and session. There was no difference in motor performance between groups at baseline (Mann-Whitney U test, U = 47, p = 0.85, r = 0.051). Thereafter, mean baseline response times were subtracted from the mean response times immediately, 6 h, 24 h, and 30 days after melody memorization. These normalized response times were used for data analysis if not otherwise specified.

To determine whether there was an immediate benefit of memorizing a congruent melody and to determine how long the better motor performance (faster response times) observed in the C group would last, a mixed factorial ANOVA with the response time as the dependent variable was performed with group (C, InC) as the between-subject factor and session (immediately, 6 h, 24 h, and 30 days after melody memorization) as the within-subject factor (Pinheiro, Bates, DebRoy, Sarkar, & R Core Team, 2013). One-sided Mann–Whitney U tests were performed to test at each time point whether performance of the C group was better than performance of the InC group. Results indicated a significant difference between groups (F(1, 18) = 5.12, p = 0.036; no main effect of session, F(3, 54) = 2.17, p = 0.10; no interaction group × session, F(3, 54) = 1.02, p = 0.39). Response times were significantly faster in the C group immediately (U = 25, p = 0.032, r = 0.42), 6 h (U = 20, p = 0.012, r = 0.51), and 24 h after melody memorization (U = 27, p = 0.045, r = 0.39). There was no significant difference in response times between groups after 30 days (U = 38, p = 0.20, r = 0.20). P values after BH correction for multiple comparisons: immediately p = 0.059, 6 h: p = 0.046, 24 h: p = 0.059, 30 days: p = 0.20 (Fig. 3, for mean response times per 12 key presses see Supplemental Figure 4).

Fig. 3.

Fig. 3

Mean response times for each motor test. The triangles represent the incongruent group (InC) and the squares represent the congruent group (C). The musical notes represent the melody memorization phase of the experiment. After correction for multiple comparisons, there was a significant difference between groups at 6 h (*, p < 0.05) and a trend toward statistical significance immediately and 24 h after melody memorization ((*), p = 0.059). There was a significant increase in response time from sequence A to sequence B (Seq B) 30 days after melody memorization to an equal extent in both groups (**, p < 0.01). Error bars represent ± standard error of the mean

Moreover, for the assessment after 30 days, a mixed factorial ANOVA with group (C, InC) as the between-subject factor and sequence (A, B) as the within-subject factor was performed with the absolute mean response times as the dependent variable (Pinheiro et al., 2013). This was done to test whether sequence A was learned (determined by a significant increase in response time from sequence A to sequence B) and whether memory traces of sequence B were still present 1 month after melody memorization (determined by an interaction group × sequence). Our data show a significant increase in response time from sequence A to sequence B to an equal extent in both groups (main effect of sequence, (F(1, 12) = 13.3, p = 0.003; no interaction group × sequence (F(1, 12) = 0.007, p = 0.94; no main effect of group, F(1, 18) = 0.30, p = 0.59).

The average percentage of false responses across all motor performance tests was 2.64 percent (SD 1.61, Mdn 2.86) for the C group and 2.30 percent (SD 1.80, Mdn 1.57) for the InC group (Mann–Whitney U test: U = 55, p = 0.72, r = 0.085). These findings were not analyzed any further.

Explicit knowledge of the melody

Mann–Whitney U tests revealed a trend for a better explicit knowledge of the melody in the C group than in the InC group (C: M 10.4, SD 2.14, Mdn 11.5, mean rank 13.1; InC: M 8, SD 2.99, Mdn 7, mean rank 7.95; U = 75.5, p = 0.052, r = 0.44). However, there was no correlation between explicit knowledge of the melody and the change in response time from baseline to immediately after melody memorization in either group, as revealed by Spearman’s correlations (C: rs = −0.36, p = 0.31; InC: rs = 0.043, p = 0.91).

Participants reported using the following melody memorization strategies: (1) “Assigned each tone a number 1–4 and tried to recall what pitch went with what number/used the tones as numbers and memorized the numbers” (five participants); (2) “Painted a visual picture of the dots/visualized the four circles and what tone each made” (four participants); (3) “Visualized the placement of the tones on the keys” (three participants); (4) “Imagined sheet notes” (two participants); (5) “Imagined playing the keys” (two participants); (6) “Tried to picture a song with those tones” (one participant); (7) “Visualized the tones spatially, with each tone represented as a different height from an arbitrary baseline” (one participant); (8) “Broke the melody down into groups of tones” (one participant); (9) “Tried to memorize the tone sequence by pitch and harmony” (one participant).

Explicit knowledge of the motor sequence

No difference was detected between groups in the explicit knowledge of the motor sequence assessed after each motor test (immediately after melody memorization: U = 46, p = 0.64, r = 0.11; 6 h: U = 50, p = 0.18, r = 0.31; 24 h: U = 33, p = 0.53, r = 0.15; 30 days: U = 59, p = 0.51, r = 0.15; the p values after BH correction for multiple comparison are as follows: immediately p = 0.64, 6 h: p = 0.64, 24 h: p = 0.64, 30 days: p = 0.64).

Finally, Spearman’s correlations were used to separately test in both groups whether the number of years of musical experience correlated with the change in motor performance from baseline to immediately after melody memorization (delta baseline response time to immediate motor test). There was no correlation in either group (C: rs = −0.44, p = 0.20; InC: rs = 0.56, p = 0.089).

Discussion

The results of this experiment demonstrate that repetitive exposure to a sequence of movement-related tones can facilitate subsequent performance of a congruent motor sequence. Participants performed a motor sequence (without hearing any sound) significantly faster if they were previously instructed to memorize a melody with a congruent contour (i.e., the sequence of pitches was congruent to the sequence of finger movements) when compared to individuals who were previously instructed to memorize an incongruent melody. Performance differences between groups developed between the end of the baseline task and the beginning of the first motor sequence test. Therefore, our findings provide support for the hypothesis that listening to melodies alone without overt movement may induce motor memory traces, whose influence is evidenced by better performance of a never physically-practiced motor task.

This is in agreement with other studies showing that congruent visual information can facilitate motor learning. One such study revealed that participants who observed an actor learning to move a robotic arm to targets against a clockwise force field executed the same task more accurately when tested later on than participants who had observed a different task (Mattar & Gribble, 2005). Another study showed that participants experienced improved timing of sequential cursor movements after previous observation of cursor movements on a computer screen (Hayes, Elliott, & Bennett, 2010). In a study performed by Stefan, Classen, Celnik, and Cohen (2008), observation of thumb movements in combination with physical training significantly enhanced the process of motor memory encoding if the action observed was congruent to the action performed (thumb movement in same direction). Importantly, Stefan et al. (2005) suggest that observation alone (without physical training) may induce lasting specific motor memory traces similar to physical training. Our study extends these findings to the auditory domain indicating that motor memory traces may also be induced through auditory information.

This raises questions regarding the neuronal mechanisms of such crossmodal memory formation. Only two participants reported imagining finger movements during the melody memorization phase of our experiment. This indicates that most participants did not make an attempt to intentionally rehearse tone-finger movement associations, suggesting that motor memory formation occurred largely implicitly. Moreover, we hypothesize that during the association phase at the beginning of our experiment, participants developed an association between specific key presses and specific tones (Lahav et al., 2005; D’Ausilio et al., 2006). It has been shown previously that the retrieval of specific information reactivates brain regions that were concurrently active during the encoding of this information (Nyberg, Habib, McIntosh, & Tulving, 2000). Consequently, exposure to the melody may have reactivated the single finger movement representations associated with each tone. This repetitive activation of the same sequence of finger movements could thus have induced the encoding of the whole movement sequence.

An alternative hypothesis would be that pitch height was automatically encoded as an internal representation of space, which in turn influenced motor performance. Support for this hypothesis comes from previous studies showing that key presses are faster when they are performed in response to or concurrent with tones that are mapped onto the spatial key locations in a congruent manner (that is, low pitched tones mapped onto lower or leftward keys, high pitched tones mapped onto upper or rightward keys) (Rusconi et al., 2005; Keller & Koch, 2008; Rusconi, Kwan, Giordano, Umiltà, & Butterworth, 2006). Furthermore, tones can facilitate the learning of finger movement sequences if they are mapped onto the movements in a spatially congruent manner as compared to a spatially incongruent manner (Hoffmann et al., 2001; Stöcker et al., 2003). These findings are also in line with a recent study showing that pianists’ movements were facilitated depending on whether the direction of concurrently heard musical scales was congruent or incongruent (Taylor & Witt, 2014), suggesting that pianists automatically activate higher order spatial representations while listening to musical scales.

One could argue that in our study exposure to the congruent melody resulted in explicit knowledge of the motor sequence which in turn may have influenced procedural motor performance. However, there were no significant differences in explicit knowledge of the motor sequence between groups at any time point. This finding is in agreement with a recently published study by Antony et al. (2012), which found evidence for an effect of auditory cueing during sleep on procedural motor performance but not on explicit motor memory. Similarly, Stöcker et al. (2003) had found that tones can facilitate the learning of finger movement sequences if they are mapped onto the movements in a consistent and congruent manner. He argued that this beneficial influence on procedural motor sequence learning could not solely be attributed to an additional gain in explicit knowledge, as they found no reliable difference in explicit knowledge of the motor sequence between participants experiencing tone effects upon each key press and participants without tone effects. These findings are also in line with the results of a study by Brown and Palmer (2012), who found no beneficial effect of explicit motor learning of melodies without sound on later auditory recognition of those melodies.

An additional goal of our study was to better understand the time course of crossmodally-induced memory formation. Our results indicate that motor performance differences between groups were significant 6 h after melody memorization with this effect decreasing over time until it was extinguished 30 days later (with a trend for a faster response time in the C group immediately and 24 h after melody memorization). In addition, switching from motor sequence A to motor sequence B at the very end of the experiment revealed a significant slowing in response time to the same degree in both groups, demonstrating that motor sequence A was learned by both groups and that the previous exposure to melody B by the InC group did not lead to any advantage in performing motor sequence B 30 days later. This suggests that motor memory traces induced by the incongruent melody B had dissipated over time and were extinguished 30 days later, possibly due to interference with the physically-practiced sequence A. A study performed by Trempe, Sabourin, Rohbanfard, and Proteau (2011) is to our knowledge the only other study that has investigated interference and the time course of crossmodally-induced memories with respect to their effects on physical motor performance. Their findings suggest that the motor memory representation of a first observed sequence A had been stabilized and that it interfered with memory formation of a second sequence B observed 8 h later, leading to a less accurate performance of motor sequence B at retest compared to when sequence B was observed 5 min after sequence A (Trempe et al., 2011). Alternatively, in our study, motor performance differences between groups could have decreased due to a time-based dissipation of crossmodally-induced motor sequences B and A (the former decreasing interference, the latter decreasing facilitation). The importance of time in crossmodal consolidation paradigms was also highlighted in a study by Zhang et al. (2011) who found that movement observation can enhance the consolidation of motor memories if observation takes place immediately after physical training, as opposed to 24 h after physical training. The time course and the underlying mechanisms of auditory-motor learning could be further explored in future studies, for example with a between-subject design, in which motor performance is tested at different time points in different groups, in order to avoid repeated motor testing.

In summary, our data suggests that exposure to movement-associated tones may trigger neuronal plasticity related to procedural motor memory formation. This finding may have important implications for motor rehabilitation. The concept of encoding a new motor memory in a specific, controlled manner might be of relevance in cases where motor memories were disrupted. Rather than trying to directly influence a lesioned motor area, it may be more efficient to instead target related intact counterparts in the auditory modality. Also, patients may have problems with motor imagery, but they may still be able to focus on movement-related sound patterns. Moreover, when compared with movement observation and motor imagery, sound as a neuromodulatory tool might enable a more precise and more targeted influence of specific movement kinematics. After coupling sound features to specific movement parameters (e.g. timbre to joint angle, rhythm to movement velocity), exposure to an appropriate sound pattern might allow, for example, to specifically influence the elbow joint angle in a reaching movement. Moreover, sound, in contrast to motor imagery, is more controllable, as the sound pattern a patient is exposed to is known. Sound patterns can also be designed to address emotions and personal preferences; additionally, exposure to sound is easily incorporated into nearly every situation of daily life.

We conclude that exposure to a movement-related tone sequence can crossmodally and specifically affect subsequent performance of a new, never physically-practiced, motor sequence. A better understanding of auditory-motor system interactions might contribute to the development of new strategies using sound as a neuromodulatory tool for motor rehabilitation or other dysfunctions involving auditory-motor neuronal networks.

Supplementary Material

supplement
a
Download audio file (255KB, mp3)
b
Download audio file (255KB, mp3)
fig

Acknowledgments

This research was supported by the Swiss Foundation for Grants in Biology and Medicine (SFGBM) and the Swiss National Science Foundation (SNSF), as well as by the Intramural Research Program of the National Institute of Neurological Disorders and Stroke (NINDS) at the National Institutes of Health. We are grateful to Virginia Penhune and Rachel Brown for their helpful suggestions and discussions. Furthermore, we thank Steven Livingstone and another anonymous reviewer for their constructive comments.

Footnotes

Electronic supplementary material The online version of this article (doi:10.1007/s00426-014-0568-2) contains supplementary material, which is available to authorized users.

Conflict of interest The authors declare that there is no conflict of interest.

Contributor Information

Marianne A. Stephan, Email: marianne.stephan@alumni.ethz.ch.

Brittany Heckel, Email: blh25@georgetown.edu.

Sunbin Song, Email: songss@ninds.nih.gov.

Leonardo G. Cohen, Email: cohenl@ninds.nih.gov.

References

  1. Antony JW, Gobel EW, O’Hare JK, Reber PJ, Paller KA. Cued memory reactivation during sleep influences skill learning. Nature Neuroscience. 2012;15:1114–1116. doi: 10.1038/nn.3152. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Bailes F, Bishop L, Stevens CJ, Dean RT. Mental imagery for musical changes in loudness. Frontiers in Psychology. 2012;3:525. doi: 10.3389/fpsyg.2012.00525. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bangert M, Peschel T, Schlaug G, Rotte M, Drescher D, Hinrichs H, et al. Shared networks for auditory and motor processing in professional pianists: evidence from fMRI conjunction. Neuroimage. 2006;30:917–926. doi: 10.1016/j.neuroimage.2005.10.044. [DOI] [PubMed] [Google Scholar]
  4. Baumann S, Koeneke S, Schmidt CF, Meyer M, Lutz K, Jancke L. A network for audio-motor coordination in skilled pianists and non-musicians. Brain Research. 2007;1161:65–78. doi: 10.1016/j.brainres.2007.05.045. [DOI] [PubMed] [Google Scholar]
  5. Brown RM, Palmer C. Auditory-motor learning influences auditory memory for music. Memory and Cognition. 2012;40:567–578. doi: 10.3758/s13421-011-0177-x. [DOI] [PubMed] [Google Scholar]
  6. Chen JL, Penhune VB, Zatorre RJ. Listening to musical rhythms recruits motor regions of the brain. Cerebral Cortex. 2008;18:2844–2854. doi: 10.1093/cercor/bhn042. [DOI] [PubMed] [Google Scholar]
  7. D’Ausilio A, Altenmüller E, Olivetti Belardinelli M, Lotze M. Cross-modal plasticity of the motor cortex while listening to a rehearsed musical piece. European Journal of Neuroscience. 2006;24:955–958. doi: 10.1111/j.1460-9568.2006.04960.x. [DOI] [PubMed] [Google Scholar]
  8. Hayes SJ, Elliott D, Bennett SJ. General motor representations are developed during action-observation. Experimental Brain Research. 2010;204:199–206. doi: 10.1007/s00221-010-2303-6. [DOI] [PubMed] [Google Scholar]
  9. Hoffmann J, Sebald A, Stöcker C. Irrelevant response effects improve serial learning in serial reaction time tasks. Journal of Experimental Psychology. Learning, Memory, and Cognition. 2001;27:470–482. doi: 10.1037/0278-7393.27.2.470. [DOI] [PubMed] [Google Scholar]
  10. Keller PE, Koch I. Action planning in sequential skills: relations to music performance. The Quarterly Journal of Experimental Psychology. 2008;61:275–291. doi: 10.1080/17470210601160864. [DOI] [PubMed] [Google Scholar]
  11. Lahav A, Boulanger A, Schlaug G, Saltzman E. The power of listening: auditory-motor interactions in musical training. Annals of the New York Academy of Sciences. 2005;1060:189–194. doi: 10.1196/annals.1360.042. [DOI] [PubMed] [Google Scholar]
  12. Lahav A, Katz T, Chess R, Saltzman E. Improved motor sequence retention by motionless listening. Psychological Research. 2013;77:310–319. doi: 10.1007/s00426-012-0433-0. [DOI] [PubMed] [Google Scholar]
  13. Mattar AAG, Gribble PL. Motor learning by observing. Neuron. 2005;46:153–160. doi: 10.1016/j.neuron.2005.02.009. [DOI] [PubMed] [Google Scholar]
  14. Nyberg L, Habib R, McIntosh AR, Tulving E. Reactivation of encoding-related brain activity during memory retrieval. Proceedings of the National Academy of Sciences. 2000;97:11120–11124. doi: 10.1073/pnas.97.20.11120. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Oldfield RC. The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia. 1971;9:97–113. doi: 10.1016/0028-3932(71)90067-4. [DOI] [PubMed] [Google Scholar]
  16. Pinheiro J, Bates D, DebRoy S, Sarkar D, R Core Team nlme: linear and nonlinear mixed effects models. R package version 3.1–108 2013 [Google Scholar]
  17. Rodriguez-Fornells A, Rojo N, Amengual JL, Ripollés P, Altenmüller E, Münte TF. The involvement of audio-motor coupling in the music-supported therapy applied to stroke patients. Annals of the New York Academy of Sciences. 2012;1252:282–293. doi: 10.1111/j.1749-6632.2011.06425.x. [DOI] [PubMed] [Google Scholar]
  18. Rusconi E, Kwan B, Giordano B, Umiltà C, Butterworth B. The mental space of pitch height. Annals of the New York Academy of Sciences. 2005;1060:195–197. doi: 10.1196/annals.1360.056. [DOI] [PubMed] [Google Scholar]
  19. Rusconi E, Kwan B, Giordano BL, Umiltà C, Butterworth B. Spatial representation of pitch height: the SMARC effect. Cognition. 2006;99:113–129. doi: 10.1016/j.cognition.2005.01.004. [DOI] [PubMed] [Google Scholar]
  20. Sharma N, Pomeroy VM, Baron J-C. Motor imagery: a backdoor to the motor system after stroke? Stroke. 2006;37:1941–1952. doi: 10.1161/01.STR.0000226902.43357.fc. [DOI] [PubMed] [Google Scholar]
  21. Stefan K, Classen J, Celnik P, Cohen LG. Concurrent action observation modulates practice-induced motor memory formation. European Journal of Neuroscience. 2008;27:730–738. doi: 10.1111/j.1460-9568.2008.06035.x. [DOI] [PubMed] [Google Scholar]
  22. Stefan K, Cohen LG, Duque J, Mazzocchio R, Celnik P, Sawaki L, et al. Formation of a motor memory by action observation. Journal of Neuroscience. 2005;25:9339–9346. doi: 10.1523/JNEUROSCI.2282-05.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Stöcker C, Sebald A, Hoffmann J. The influence of response-effect compatibility in a serial reaction time task. Quarterly Journal of Experimental Psychology. A, Human Experimental Psychology. 2003;56:685–703. doi: 10.1080/02724980244000585. [DOI] [PubMed] [Google Scholar]
  24. Taylor JET, Witt JK. Listening to music primes space: pianists, but not novices, simulate heard actions. Psychol Res. 2014:1–8. doi: 10.1007/s00426-014-0544-x. [DOI] [PubMed] [Google Scholar]
  25. R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing; Vienna, Austria: 2012. http://www.R-project.org/ [Google Scholar]
  26. Trempe M, Sabourin M, Rohbanfard H, Proteau L. Observation learning versus physical practice leads to different consolidation outcomes in a movement timing task. Experimental Brain Research. 2011;209:181–192. doi: 10.1007/s00221-011-2540-3. [DOI] [PubMed] [Google Scholar]
  27. Wheeler B. lmPerm: Permutation tests for linear models. R package version 1.1–2. 2010 http://CRAN.R-project.org/package=lmPerm.
  28. Zhang X, de Beukelaar TT, Possel J, Olaerts M, Swinnen SP, Woolley DG, et al. Movement observation improves early consolidation of motor memory. Journal of Neuroscience. 2011;31:11515–11520. doi: 10.1523/JNEUROSCI.6759-10.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

supplement
a
Download audio file (255KB, mp3)
b
Download audio file (255KB, mp3)
fig

RESOURCES