Not quite there: skill consolidation in training by doing or observing

Abstract

We tested the notion that action observation engages learning processes and mnemonic representations overlapping with those engaged in actual performance. An identical number of training instances, actual performance, or observation, was afforded on a finger opposition sequence task. Both training modes resulted in immediate gains in performance, as well as in robust delayed, “off-line,” gains, indicating post-training consolidation. However, the expression of delayed gains could be blocked by the subsequent performance of a second sequence (post-training interference), but not by its observation. The mnemonic representations of “how-to” knowledge acquired from actual or observed movement may not overlap.

The observation of action can lead to subsequent specific performance gains, presumably facilitating learning processes (Heyes and Foster 2002; Brown et al. 2003; Torriero et al. 2007; Hayes et al. 2010; Zhang et al. 2011). It is not clear, however, to what degree the beneficial effects of observation reflect a direct recruitment of the motor network involved in actual movement execution; specifically, whether the recruitment of the motor network through putative systems such as the “mirror neuron system,” or an “action observation network” (AON) overlaps with the recruitment of the motor network in actual performance (Hari et al. 1998; Buccino et al. 2001; Calvo-Merino et al. 2006; Gazzola and Keysers 2009; Mukamel et al. 2010).This is a pertinent question given that there is increasing evidence that task-specific changes in the motor cortex constitute an important part of the mnemonic representation of well-trained movement sequences (Karni et al. 1998; Yang et al. 2009, 2014; Xu et al. 2010; Gabitov et al. 2014) and that along Hebbian lines, neurons involved in task execution are an integral part of the subsequent mnemonic representation. The AON presumably matches the perceived action with a neural representation within the motor repertoire of the observer. There is evidence, nevertheless, for performance gains even in observing new motor tasks, for which no representation is available in the observer's repertoire (Mattar and Gribble 2005; Stefan et al. 2005; Gatti et al. 2013).

The aim of Experiment 1 was to compare the training-related performance gains in two training modes: actual training (Act) and training by observation (Obs) given equal practice on a five-element sequence of opposition movements (Fig. 2A). Skill acquisition in different brain systems (modalities) is characterized by a distinct time-course (phases) (Karni and Sagi 1991, 1993; Karni 1996; Ari-Even Roth et al. 2005) presumably reflecting a similar repertoire of basic mechanism of plasticity and procedural memory consolidation processes (Karni and Bertini 1997; Dudai 2004; Robertson et al. 2004). Thus, a reasonable expectation would be that the acquisition of motor skills by observation would show phases of learning similar to those known to characterize learning when physical motor training is afforded. We tested not only for gains in performance occurring within the training session, as experience accumulates, but also for the evolution of between-session, delayed gains in performance. The latter are considered a hallmark of procedural (“how to”) memory consolidation processes underlying the generation of skill and leading to its retention as long-term memory (Karni et al. 1998; Krakauer and Shadmehr 2006; Song et al. 2007; Doyon et al. 2009). It is well established that physical practice in the finger-to-thumb opposition sequence (FOS) learning task (Fig. 1; Korman et al. 2003; Rozanov et al. 2010) and in other paradigms wherein specific, explicitly or implicitly instructed, finger movement sequences are repeated, can lead to delayed gains in performance; “off-line” gains can be expressed hours after the termination of the training session if sleep is afforded in the post-training interval (Maquet 2001; Fischer et al. 2002; Walker et al. 2002; Maquet et al. 2003; Korman et al. 2007; Doyon et al. 2009; Barakat et al. 2013, but see Nettersheim et al. 2015).

Figure 2.

Experiment 1: timelines of the experiment and group mean levels of performance. (A) Each participant was tested before (Test 1), immediately after (Test 2) and again at 24 h post- (Test 3_T) training. The performance of the untrained, mirror-reversed, FOS was tested immediately after Test 3_T (Test 3_U). The performance tests were composed of four successive 30 sec long blocks separated by 50-sec rest intervals. Participants were instructed to tap the assigned sequence continuously as fast and accurately as they could during the test blocks (Supplemental Experimental Procedures). Performance during the test-blocks was video recorded and two performance measures extracted: (1) the number of correctly completed sequences per block (as a measure of speed); (2) the number of incorrect sequences (movement order errors) per block (as a measure of accuracy). (B) The results of 30 healthy, right hand dominant, young university students are shown (upper panel—speed, lower panel—accuracy). Act (n = 15)—actual physical training; Obs (n = 15)—observation training. (C) Control experiment. 22 additional individuals were given an identical observation protocol as the Obs group but only tested once either immediately (Obs_Only_Imm, N = 10) or 24 h post-training (Obs_Only_24 h, N = 12). (Upper panel) timeline of the control experiment; (middle and lower panels) group mean levels of performance (averaged across the four blocks of Test), speed and accuracy, respectively. At 24 h post-training the performance of the observed sequence was superior to that of the mirror-reversed (previously unobserved) sequence in both speed (paired-t₍₁₁₎ = 8.325, P < 0.001; N = 12) and accuracy (paired-t₍₁₁₎ = 2.196; P < 0.05); but no such difference was found immediately post-training in either speed (t₍₉₎ = 1.681, P = 0.127; N = 10) or accuracy (t₍₉₎ = −0.231; P = 0.823). Bars represent standard error.

Figure 1.

The finger-to-thumb opposition task (FOS) (A) and training protocol (B). All participants were trained and tested on the finger-to-thumb opposition sequence (FOS) learning task (Korman et al. 2003). Two five-element sequences, each the reverse of the other, were used with each participant randomly assigned one of the sequences for training. Participants were instructed to oppose the fingers of the left (nondominant) hand to the thumb in a given five-element sequence, as fast and accurately as possible. All participants performed the instructed movements while lying supine. The left hand was positioned on the subject's chest with the elbow flexed, in direct view (palm-facing) of a video camera, to allow recording of all digit movements.(B) The training protocol included 160 repetitions of the assigned FOS (10 training-blocks of 16 repetitions) as in Korman et al. (2003). In the Act group participants were instructed to physically tap the sequence (FOS task) using their left hand. In the Obs group participants were instructed to observe a video presenting a person's left hand performing the to-be-learned FOS (Supplemental Experimental Procedures).

The study was approved by the University of Haifa human experimentation ethics committee (Supplemental Experimental Procedures). As can be seen in Figure 2B and Table 1, learning by observation and learning by actual performance yielded comparable gains in terms of speed and accuracy, within-session (fast learning) as well as between-sessions (“off-line” delayed gains) in the FOS task. Repeated-measures general linear model (GLM) analyses were used to compare the performance, in three time-points (Tests 1–3) as within-subject comparisons, of the two training mode groups (Act, Obs) as a between-subjects factor. There was an increase in speed as well as a reduction in the number of errors committed [F_(2,56) = 150.32, P < 0.001, η² = 0.843; F_(2,56) = 3.528, P < 0.05, η² = 0.112; speed and accuracy, respectively] indicating effective learning in both training conditions (Fig. 2B). The performance of the two training mode groups across the 24 h post-training interval (Test 2 versus Test 3) was compared with direct test for learning-related delayed gains. There were significant delayed gains in speed [F_(1,28) = 76.968, P < 0.001, η² = 0.733] with no loss of accuracy [F_(1,28) = 1.626, P = 0.213, η² = 0.055] in both groups.

Table 1.

Average performance in Tests 1–3 in Experiment 1 and the control conditions

To test for sequence-specificity in the overnight performance gains, a specificity that characterizes the gains from actual training (Korman et al. 2003; Rozanov et al. 2010), Rm-ANOVAs were run comparing the performance of the trained sequence, and a novel, untrained, sequence, in the two groups (Act, Obs) at 24 h post-training (Test 3). There was a significant main effect of sequence [F_(1,28) = 119.492, P < 0.001, η² = 0.081; F_(1,28) = 16.041, P < 0.001, η² = 0.364; for speed and accuracy, respectively], reflecting the fact that in both training modes the performance of the trained sequence was superior. There was however a trend toward a significant interaction of training condition and sequence, in performance speed though not in accuracy [F_(1,28) = 3.713, P = 0.064, η² = 0.117; F_(1,28) = 0.305, P = 0.585, η² = 0.011; for speed and accuracy, respectively] suggesting that actual training (Act) resulted in gains in speed that were more trained-sequence specific compared with training by observation (Obs) (Fig. 2B).

One should note that the training by observation condition was in fact a mixed training experience because of the repeated actual execution of the task during the performance tests. In order to test the effect of observation per se (without the experience gained in the actual performance of the sequence during the performance tests) two control conditions were run wherein the performance of the trained (observed) FOS, after the observation session, was compared with that of mirror-reversed (previously unobserved) FOS. The design of these control conditions was based on the above findings, as well as of previous studies (e.g., Karni et al. 1995; Korman et al. 2003; Dorfberger et al. 2012), that the performance gains accrued in actual physical training in the FOS task are sequence specific. In both control conditions participants underwent training identical to that afforded to the Obs group, but without the pre- and post-training tests (Test 1 and Test 2) and were tested either at 24 h or immediately after the observation session (Obs_Only_24h, Obs_Only_Imm, respectively) (Fig. 2C). Rm-ANOVAs with the performance of the two sequences (observed versus unobserved FOS) as a within-subject factor and the two time-points (immediate post-training, 24 h post-training) as between-subjects factor, showed an overall significant advantage in speed [F_(1,20) = 56.786, P < 0.001, η² = 0.74] though not in accuracy [F_(1,20) = 2.633, P = 0.12, η² = 0.116] for the observed sequence (Table 1; Fig. 2C). Moreover, there was a significant sequence × time-points interaction in speed and a trend for a significant interaction in accuracy [F_(1,20) = 33.324, P < 0.001, η² = 0.625; F_(1,20) = 3.494, P = 0.076, η² = 0.149; respectively]. Thus, the advantage in the performance of the observed sequence at 24 h post-training could be attributed to the consolidation phase, rather than to the within-session learning phase.

A landmark study (Van Der Werf et al. 2009) has shown that repeated observation of a finger (key press) movement sequence can result in significant sequence-specific performance gains if the observation training experience is closely followed by sleep. However, it is not clear whether the overnight gains reflected sleep-dependent delayed gains, as performance measurements immediately after the observation session were available only for a different group of participants; observation afforded before noon, and followed by hours in the wake condition (delayed sleep), did not result in overnight performance gains (Van Der Werf et al. 2009). The Experiment 1 and the control experiment suggest that as in actual training on the FOS task (Fischer et al. 2002; Walker et al. 2002; Korman et al. 2003, 2007), training by observation afforded many hours before a night's sleep effectively triggered the subsequent expression of sequence-specific delayed gains in the performance of a trained movement sequence.

To further test this proposal we ran Experiment 2 in which we used an interference paradigm to test whether the repeated observation of a second sequence can interfere with the consolidation of a physically trained FOS (Fig. 3A). The training protocols were identical to those used in Experiment 1. It has been previously shown that actual practice on a second FOS, up to a few hours after an effective training experience on a different FOS, can interfere with (block) the overnight expression of delayed gains for the initially trained FOS (Walker et al. 2003; Balas et al. 2007a; Korman et al. 2007).

Figure 3.

Experiment 2: timelines of the experiment, group mean levels of performance and delayed gains. (A) Performance was tested in the same time-points as in Experiment 1, however, 10 min after the termination of Test 2, a training session with the second, mirror-reversed, sequence was afforded. The interference experience comprised an additional training session on the second sequence. In the Act_int group, the experience with the second FOS was by actual performance; in the Obs_int group, by observation of a person's left hand performing the mirror-reversed FOS. (B) The results of 30 healthy, right hand dominant, young university students are shown (upper panel—speed, lower panel—accuracy). Act_int (n = 15)—interference by actual physical performance; Obs_int (n = 15)—interference by observation of performance. The Act condition of Experiment 1 was used as a control, no interference, condition (No_int). (C) Absolute delayed gains in speed expressed as the difference in performance in Test 3 compared with Test 2. There was a clear reduction in the delayed gains after actual training on the second sequence; training on the second sequence by observation as well as no interference resulted in significant delayed gains at 24 h post-training. Bars represent standard error.

Although what constitutes effective interference is not clear, a leading notion is that an overlap in some critical representation or internal model between the initially trained, to-be-interfered, task, and the interference experience is required (Shadmehr and Brashers-krug 1997; Tong et al. 2002; Balas et al. 2007a,b; but see Handa et al. 2015). Thus, given that overlapping representations underlie training by observation and physical training one would expect that training by observation would interfere with the evolution of delayed gains induced by actual training.

The results of Experiment 2, however, indicate that the overlap between procedural knowledge acquired from actual movement and observed movement may be limited (Fig. 3B). Three groups were compared: interference by actual training afforded on the second movement sequence (Act_int), interference by observation training (Obs_int), and no interference (No_int; the Act group of Experiment 1) with three Tests as a within-subject factor (Fig. 3A). There was an increase in speed as well as a reduction in the number of errors committed [F_(2,84) = 236.243, P < 0.001, η² = 0.849; F_(2,84) = 10.383, P < 0.001, η² = 0.198; speed and accuracy, respectively] with no significant main effect of group [F_(2,42) = 1.88, P = 0.164, η² = 0.083; F_(2,42) = 0.353, P = 0.704, η² = 0.017; speed and accuracy, respectively], indicating effective learning of the trained sequence in all three groups (Fig. 3B). However, there was a significant group X Test interaction in speed of performance [F_(4,84) = 9.33, P < 0.001, η² = 0.308] indicating significant differences in the learning-related gains expressed in the different phases of the experiment, by the three groups (Fig. 3B,C).

All three groups had improved performance by the end of the training session (Test 1 versus Test 2) [F_(1,42) = 278, P < 0.001, η² = 0.869 ; F_(1,42) = 6.089, P < 0.05, η² = 0.127; speed and accuracy, respectively] with no significant group effect or a group X Test interaction. However, although the performance of the three groups improved across the between-sessions consolidation phase (Test 2 versus Test 3) [F_(1,42) = 42.58, P < 0.001, η² = 0.503; F_(1,42) = 6.83, P < 0.05, η² = 0.14; speed and accuracy, respectively] there was a significant group X Test interaction for speed [F_(2,42) = 17.069, P < 0.001, η² = 0.448], and a trend for a significant interaction in accuracy [F_(2,42) = 0.558, P = 0.055, η² = 0.129]. This reflected the fact that the Act_int group failed to express significant delayed gains in speed [t₍₁₄₎ = 1.248, P = 0.232], i.e., there was significant interference. The Obs_int group, however, expressed robust delayed gains [t₍₁₄₎ = 4.566, P < 0.001]. These gains were on the order of the gains expressed by the No_int, control group, in which no experience with the second sequence was afforded after the initial training session [t₍₁₄₎ = 6.952, P < 0.001] (Fig. 3B,C). Thus, unlike actual performance, the observation of the second, reversed, sequence did not interfere with the expression of the delayed gains for the initially trained sequence.

The extent of overlap in brain regions engaged during actual performance and in action observation is not clear (Hari et al. 1998; Strafella and Paus 2000; Buccino et al. 2001; Grèzes and Decety 2001; Rizzolatti and Craighero 2004; Cross et al. 2009); some brain areas, specifically, the primary motor cortex (M1) were found to be activated more intensely (Caetano et al. 2007; Gazzola and Keysers 2009; Vigneswaran et al. 2013) or even exclusively (Frey and Gerry 2006; Cross et al. 2009) during actual practice. This is of importance because M1 is a key area for movement sequence skill memory (Karni et al. 1995, 1998; Hikosaka et al. 2002; Gabitov et al. 2014). It may be the case that participants generate minimal movements when observing; this would make the current results even more intriguing by suggesting that such minimal movements do not interact with full sequence production. Moreover, even given M1 engagement in learning novel motor tasks by observation (Brown et al. 2003; Cross et al. 2006; Tkach et al. 2007; Dushanova and Donoghue 2010; Zhang et al. 2011) this does not necessarily indicate that the same units in M1 are engaged for mnemonic representation by observation or actual training (e.g., Stefan et al. 2008, compared with Tokuno and Tanji 1993).

We propose that the current results can be tentatively interpreted in terms of the conceptual framework provided by the synaptic tagging hypothesis (STH) (Rogerson et al. 2014). The STH posits that two distinct cellular events are essential to the generation of long-lasting task-specific synaptic consolidation: the induction of plasticity related proteins (PRP) synthesis in the cell body and the “tagging” of the relevant synapses to which these PRPs are to be sequestered. Within this framework, the results of Experiment 1 as well as the Obs_Only_24 h control experiment, suggest that an observation experience can lead to the induction of PRPs as well as effective and task-specific synaptic tagging in a neuronal population engaged in observation. The STH further suggests that because different memories may be represented by shared neuronal substrates, interference effects may represent a competition for PRPs between distinct synapses within the same cells engaged in two tasks. Thus, in Experiment 2, Act_int condition, one may conjecture that in neurons engaged in the generation of the two movement sequences during training, PRPs produced in the cells’ body after the initial physical training experience may be diverted to the freshly tagged synapses engaged in the physical practice of the second movement sequence. This scenario is highly likely given that the two movement sequences were composed of the same movement elements and performed with the same hand. However, in the Obs_int condition, either because the observation of the second sequence was insufficient to tag the sequence-related synapses in M1 (assuming that physical training on the initial sequence sufficed to trigger PRPs) or because observation altogether engages a different set of neurons, no competition occurred when observation followed the actual training experience. In fact, the initially trained sequence was subsequently enhanced. Given that an identical observation training session was afforded in the two experiments, a tagging failure by observation is unlikely. Thus, the results of Experiment 2 more likely indicate that plasticity induced by observation occurs in neurons different from those engaged in memory acquired from actual physical practice.

A recent study (Trempe et al. 2011) suggests that consolidation processes induced by observation and actual training in a movement timing task may undergo different patterns of interference. However, the training did not result in delayed gains and it is not clear whether procedural memory processes were equally engaged in the two learning modes.

In conclusion, observation training was found to be highly effective in inducing procedural memory consolidation effects in young adults. However, the differential interference effects of action and observation, suggest that the overlap between the neural representation of procedural knowledge acquired from actual movement and observed movement, at a level relevant for motor memory consolidation, may be limited. On a more general, conceptual and practical level, these results indicate that practice by observation, although highly effective as a learning experience, cannot be assumed to result in the same type of procedural knowledge that would have otherwise been generated if actual physical practice was afforded instead. This is especially pertinent because observation training is increasingly advocated as a therapeutic approach in the rehabilitation of motor function in patients with neurological diseases and disorders.