Robot Reinforcement and Error-Based Movement Learning in Infants With and Without Cerebral Palsy

Abstract

Background

Prone mobility, central to development of diverse psychological and social processes that have lasting effects on life participation, is seldom attained by infants with cerebral palsy (CP) and has no tested interventions. Reinforcement learning (RL) and error-based movement learning (EBL) offer novel intervention possibilities.

Objective

This study examined movement learning strategies in infants with or at risk for CP using RL and EBL during acquisition of prone locomotion.

Design

The study was a randomized trial that used repeated measures.

Setting

The study setting was a university physical therapy clinic in the United States.

Patients

Thirty infants aged 4.5 to 6.5 months participated in the study: 24 had or were at risk for CP, and 6 were typically developing.

Intervention

Infants with and at risk for CP were randomly assigned to a combination of RL and EBL (SIPPC-RE), or RL only (SIPPC-R) conditions. Infants with typical development comprised the RL-only reference group (SIPPC-TD). Infants trained in prone locomotion with the Self-Initiated Prone Progression Crawler (SIPPC) robotic system for three 5-minute trials, twice a week for 12 weeks in their homes or child care. All training sessions were videotaped for behavioral coding.

Measurements

The SIPPC gathered robot and infant trunk/limb movement data. Randomized 2-way analysis of variance with repeated measures and Pearson r to analyze the data was used.

Results

Results included the number of arm movements and trial-and-error activity distinguished between the SIPPC-RE and SIPPC-R groups. The mean change in arm movements from baseline for the SIPPC-RE and SIPPC-R groups was 4.8 m and −7.0 m, respectively. The mean differences in rotational amplitude (trial and error) from baseline to the end of the study were 278 degrees and 501 degrees, respectively. These changes were correlated with distance traveled and goal-directed movements. The latter increased over the 12 weeks for the SIPPC-RE and SIPPC-TD groups, but not the SIPPC-R group.

Limitations

The CP groups were unequal due to reassignment and did not include a typically developing comparison group of a combination of RL and EBL.

Conclusion

These findings suggest movement learning and retention in infants with CP is differentially affected by the use of RL and EBL, with a combination of both showing more promise than RL alone. The findings also implicate cognition, type of brain insult, emergence of reaching, and muscle force production, which must be explored in future studies.

Cerebral palsy (CP) continues to be the most physically disabling condition for children in the United States.¹ Of the numerous complications often experienced by children and adults with CP, the most disabling is impaired mobility. Many of the mobility constraints that confront adults with severe CP, which in turn contribute to substantial restriction on life and education participation, can be traced back to infancy.^2,3 Not only is there a paucity of self-initiated movements critical for early locomotion in children with severe CP, but also these movements diminish as the children grow (learned nonuse/disuse).⁴ Consequently, successful efforts toward maximizing independence and social integration of children with CP must also minimize learned nonuse-disuse. Such efforts will require preventive types of movement training programs that capitalize on self-generated movements, because these are considered crucial during the early stage of development to enhance synaptic connections in the brain.⁵

Two motor learning mechanisms have been shown to improve skill learning in adults with neurological deficits: reinforcement learning (RL) and error-based learning (EBL). These mechanisms also hold promise for studying skill acquisition in infants and toddlers with CP. RL is a process by which a person or an artificial system (eg, a robot) can learn a behavior that optimizes the reception of rewards (or penalties).^6,7 In RL, one selects actions that are predicted to lead to better accumulated outcomes.^8,9 RL is uniquely suited for complex movement learning, where there is no single way of performing a skill, performance is contextual, and individuals must learn without instructional information from a teacher. In contrast, EBL, a subclass of supervised learning methods, uses perceived errors to improve performance by giving the learner information about the direction and magnitude of the error resulting from their own actions.^10,11 Errors are corrected on a trial-by-trial or movement-by-movement basis. How well the RL and EBL assumptions apply to very young infants, particularly those with brain insult such as CP, is unknown.

Evidence supports the notion that although both RL and EBL contribute to motor learning, they do so in different but complementary ways.¹² EBL tends to help a person move closer to the target action, generally requires visual feedback, and promotes faster learning and sensory remapping¹³ ––but is easier to forget.¹⁴ RL does not rely on a priori knowledge of a goal but makes use of exploration, which often results in slower learning and higher variability compared with EBL.¹¹ RL does not directly inform the person about how to improve performance but shows better retention.¹⁵ Neural correlates of RL and EBL are also different. The basal ganglia are believed to be specialized for RL and the cerebellum for EBL,^16–18 although the mechanisms are not as clear. Because the cerebellum, basal ganglia, and cortex interact, ultimately the motor cortex retains what has been learned.¹³ Fundamental to both approaches is an assumption that the individual understands the end point of the task or skill and has an intact feedback mechanism.¹¹ Overall, RL provides a mechanism to improve a performance learned from EBL, and vice versa.

With advances in technology, the potential exists for infants to learn new behaviors quickly and reliably by using robots that provide carefully selected and timed rewards to the infants. For this study we used the Self-Initiated Prone Progression Crawler (SIPPC) robotic system.¹⁹ The SIPPC system represents an integration of robotics and sensor technologies designed to capture and influence movement effort as infants learn prone locomotion (Fig. 1). The system includes a 2-wheeled platform that the infant lies on and a motion capture suit (Suit) that the infant wears. The wheels of the SIPPC can turn passively or they can also be actively driven such that the infant is carried in 1 of 4 directions: straight forward or backward movement, or left or right turn. All driven movements are short in duration and carry the infant a short distance. Due to the design of the SIPPC, the passive and active components of movement combine, enabling the infant to push harder to produce movements that are larger than the active movement working alone.

Figure 1.

The Self-Initiated Prone Progression Crawler (SIPPC) system (left); the SIPPC Suit (middle); and a child using the SIPPC system (right).

The Suit includes a set of UM6 orientation sensors and a central processing unit that are mounted in a onesie to protect the electronics and for ease of attachment to the infant (Fig. 1). In real time (50 Hz), the sensor orientations are translated into an estimate of the position of the infant's trunk, arms, and legs; this information is then used to recognize one of 20 different crawling-like gestures of the arms and feet. These movement patterns were identified from movements made by infants during the development of the SIPPC and were refined with infants with typical development (5 months of age) who were trained to use the SIPPC system for 10 to 12 weeks.²⁰

RL and EBL are the driving force behind the concept, design, and function of the SIPPC system. A common feature of the 2 SIPPC system components is reinforcement. Through the design of the interaction between the infant and the SIPPC, the goal is to trigger internally generated reinforcement and error signals to encourage the practice of crawling-like skills and to enable the infant to explore the environment. This interaction takes 3 forms. First, the wheels move passively in response to the ground reaction forces. Second, if the wheel velocity achieves a predetermined threshold, then the SIPPC will amplify this motion by executing 1 of the 4 predetermined movements. Third, if a limb/trunk movement (from the Suit) is recognized as matching one of 20 distinct patterns, then the SIPPC can also respond by producing 1 of the 4 movements.

The Suit rewards the infant for appropriate “virtual” movement efforts. However, the Suit also has an EBL component. Because it is programmed to respond to more than 20 canonical arm, leg, and trunk movement patterns, the SIPPC can respond initially by moving in directions that the infant does not intend. For example, to make progress toward reaching a toy, the SIPPC must nominally be oriented at the toy before forward movements should be triggered. When not properly oriented, the infant must engage turning movements of the correct direct magnitude in order to aim the SIPPC at the toy. Early in this learning process, it is not uncommon for the infant to select the correct turning direction but to overshoot the target orientation. This visually identified overshoot and subsequent corrective movement can act as an error signal to drive an EBL process that tunes the magnitude of the movement that resulted in the overshoot. Although the 20 canonical movements provide a wide range of movement options, infants must still learn to coordinate their movements in order to optimize the appropriate assistance from and movement of the SIPPC.

The purpose of this study was to examine the effects of robot RL and EBL on movement learning strategies used by infants with or at high risk for CP during acquisition of self-initiated prone locomotion. Prone locomotion is the earliest and, in some cases, the only form of functional mobility available to children during the first year. Although prone locomotion is estimated to occur in over 95% of developing infants,²¹ and is a near universal developmental milestone, infants with CP attain this milestone significantly later than their peers²² or not at all. We define prone locomotion as the ability to move or advance the body forward in the prone posture. Prone locomotion is also critical to the integration of other functions important for cognition and exploration.^23–26 Its development coincides with a period of highly active synaptic formation in the brain⁴ and the means-to-an-end and goal-orientation substages of Piaget's sensorimotor stage of cognitive development.²⁷ Despite the compelling evidence that supports the critical role of prone locomotion in child development, to our knowledge, no studies have investigated or attempted to promote its development in children with CP.

We hypothesized that RL and EBL will differentially affect the learning strategies and retention rate in infants with or at risk for CP:

1. Infants who are exposed to a combination of RL and EBL, supported by the SIPPC and the motion capture suit (SIPPC-RE), will show greater amounts of arm and leg movement as well as goal-directed movement than those who are exposed to RL supported by the SIPPC only (SIPPC-R).

2. The amount of arm and leg movements in the SIPPC-RE group will be highly correlated with goal-directed movement, movement error (amount of trial-by-error), and total distance traveled compared with those in the SIPPC-R group (see Fig. 3).

3. Infants exposed to the SIPPC-RE condition will demonstrate more learning success and retention, as evidenced by greater total distance traveled on the SIPPC and the large number of goal-directed movements during prone locomotion on the SIPPC, than infants in the SIPPC-R condition.

Methods

Study Design

We used a repeated measures experimental design with 30 infants, 4.5 to 6.5 months old. Infants with CP were randomly assigned to the combination of RL and EBL with the SIPPC and Suit assist (SIPPC-RE), or RL only with the SIPPC assist (SIPPC-R). Infants at low risk for CP comprised the reference group and only received RL with SIPPC assist (SIPPC-TD).

Participants

Twenty-four infants were at risk for CP and 6 were typically developing. Inclusion criteria for infants with CP were: a Test of Infant Motor Performance z score of less than −1.0, a confirmed diagnosis of CP, or a positive MRI result. The Test of Infant Motor Performance²⁸ was administered at 3 to 4 months to confirm the risk. The strong predictive psychometrics of the Test of Infant Motor Performance are reported in the literature. ^29,30 The infants were recruited before 3 to 4 months of age from the University of Oklahoma Medical Center neonatal intensive care unit, pediatric neurology clinics, and the state's SoonerStart Early Intervention program. Infants with congenital deformities of the bones and joints or uncontrollable seizures were excluded. None of the infants was diagnosed with sensorineural hearing loss. The study was approved by the University of Oklahoma Health Sciences Center IRB (IRB # 2711). Table 1 presents the participants’ demographic information.

Table 1.

Participant Demographics^a

Participants	Typically Developing (SIPPC-TD) (n = 6)	CP With Suit Assist (SIPPC-RE) (n = 14)	CP Without Suit Assist (SIPPC-R) (n = 9)
Male sex, n (%)	5 (83.3)	8 (57%)	6 (67%)
Ethnicity, n (%)
Caucasian	6 (100%)	7 (50%)	5 (55.5%)
African American	—	3 (21.4%)	1 (11.1%)
Mexican	—	1 (7.1%)	1 (11.1%)
Biracial	—	1 (7.1%)	—
Missing	—	2 (14.2%)	2 (22.2%)
Gestational age at birth, n (%)
<32 weeks	1 (16.7%)	7 (50%)	3 (33.3%)
32–37 weeks	5 (83.3%)	1 (7.1%)	3 (33.3%)
>37 weeks	—	6 (42.8%)	3 (33.3%)
Parental marital status, n (%)
Single	1 (16.7%)	3 (22.4%)	3 (33.3%)
Married	5 (83.3%)	10 (71.4%)	6 (66.6%)
Missing	—	1 (7.1%)	—

^a SIPPC = Self-Initiated Prone Progression Crawler; SIPPC-R = infants with cerebral palsy (CP) receiving reinforcement learning only; SIPPC-RE = infants with CP receiving reinforcement learning and error-based learning; SIPPC-TD = typically developing infants receiving reinforcement learning only.

Power

The sample size of 24 would have 80% power to detect a difference in the proportion of subjects who can travel (distance traveled or total linear path length) at least 6 ft of 0.20 in the RL group and 0.80 in the RL+EBL group assuming a 2-sided α level = .05. Under similar assumptions, the power decreases to 0.7 and to 0.6 if the RL+EBL proportion is 0.74 and 0.70, respectively. Formal sample size calculations are not provided for the other kinematic end points because estimates of the standardized effect sizes are not available.

Measures

SIPPC system

The SIPPC data include linear and rotational velocity of the robot, from which linear path length (distance traveled) and rotational amplitude (trial and error) were computed.

Activity recognition sensor suit (Suit)

We used UM6 orientation sensors, which are 9-degrees-of-freedom inertial measurement units, from which orientation in 3 dimensions can be estimated. The Suit is placed over the infant and secured in place using Velcro straps. The data from the Suit and the SIPPC travel via ethernet to a wi-fi hub and then to a laptop for real-time processing and logging. The Suit data, combined with a model of the infant's skeleton, enable a real-time estimate of the Cartesian locations of key points on the body, including the wrists, shoulders, ankles, and toes (relative to the infant's hips). This information is used to quantify the movement of the infant's trunk and limbs and to recognize crawling-like gestures made by the infant. For infants in the SIPPC-RE condition, recognized gestures triggered corresponding movements of the SIPPC. We measured the movement activity of the infant by computing the path length traversed by several points of the body, including the wrists, ankles, and toes. Note that the path length does not include motion of the SIPPC.

Movement Observation Coding Scheme

We used the Movement Observation Coding Scheme (MOCS) subscale 3 to code the goal-directed movement behaviors from videotaped sessions during SIPPC training sessions. The MOCS was developed to collect data about the strategies that infants use to move the SIPPC.³¹ It includes 42 items and 4 subscales: Subscale 1: Posture and Support; Subscale 2: Exploratory Selection and Progression; Subscale 3: Mastery of Propulsion; and Subscale 4: Socio-emotional Responses.

Subscale 3: Mastery of Propulsion captures the range of responses that describe the progression and mastery at moving the SIPPC in a goal-directed manner. The MOCS describes duration, number, and frequency of head, trunk, arm, and leg movements, which the coders enter into a form following specific guidelines. The internal consistency (Cronbach α = .77), construct validity, and interrater reliability (intraclass correlation coefficient = 0.87) of the MOCS were examined in infants with and without CP at different ages.³¹ We trained 3 coders, who were blinded to the infants’ group assignment, to reliably code the infant movement behavior using the MOCS. Intraclass correlation coefficients ranged from 0.87 to 0.93. The use of coded videotaped performance is relatively common in behavioral and functional crawling studies.^25,32

Procedures

Infants were enrolled in the study at 4.5 to 6 months of age following signed parental consent. We used a Family Interview Form to gather information on demographic factors believed to contribute to child outcome, such as gestational age at birth, ethnicity, medical history, parents’ age and level of education, and health services.

Before each training session, the infant donned the Suit over his or her clothing, was placed prone on the SIPPC, and was secured with straps (Fig. 1). We used the following protocol:

1. During the first minute of the session, the infant was given time to play with and get accustomed to being placed on the SIPPC and playing with toys.

2. Next, the tester or caregiver moved the SIPPC back and forth while encouraging the infant to reach for toys placed on the floor in front of the infant. Occasionally the caregiver moved the infant's arms and legs to simulate crawling to provide the infant with a sense of how to move the device.

3. Next the infant was encouraged to move the SIPPC independently toward either a toy or the caregiver.

The training sessions comprised three 5-minute trials for a total of 15 minutes and were implemented in the infant's home or child care center depending on parents’ preferences or availability of space in the home. Infants were allowed up to 1 minute rest between trials depending on their tolerance of the prone position. All movement data were collected by a laptop in real time. Sessions were implemented twice a week for up to 12 weeks and were videotaped.

Statistical Analysis

We report on 3 movement learning strategies: the amount of movement of (1) the arms, (2) the legs (measured by total path length in one 5-minute trial), and (3) the amount of trial and error (measured by degrees of rotational amplitude); and on 2 primary outcomes: (1) goal-directed movements (measured by the MOCS subscale 3), and (2) the distance traveled by the SIPPC (reported as linear path length). Data obtained from 24 training sessions for each infant were reduced to minimize serial dependency. First, we selected the most appropriate trial for each week for analysis and coding. This was judged based on the activity level of the infant, the quality of the recorded data, and whether a trial was completed. Then, we aggregated the data by grouping the trials into sets of 4 time periods (weeks 1–3, 4–6, 7–9, and 10–12).

To test hypothesis 1, we compared the composite mean changes in the amount of trial and error, total path length of arm and leg movements, and goal-directed movements within and between the groups. We used a randomized 2-way analysis of variance (ANOVA) procedure that accommodates repeated measures and censored samples to examine the variance of the measured variable as a function of group and week in study (procedure adapted from Piater and associates³³ and Cohen³⁴). Samples were censored from the analysis if the infant did not attend either session for a given week (eg, due to illness), or if there were data collection errors (eg, unnoticed sensors falling off the infants’ feet). When either a group effect or interaction effect was detected using the ANOVA, we performed several post hoc tests: a 2-sample, bootstrap randomization test of between-group means and pair-wise bootstrap randomization test of means comparing subsequent time periods (4–6 weeks, 7–9 weeks, and 10–12 weeks) for each group. Each of these tests was 2-tailed and accounted for repeated measures and censored data.

For hypothesis 2, we compared the composite mean changes of the distance traveled on the SIPPC and goal-directed movements using ANOVA with repeated measures

For hypothesis 3, we used the Pearson product moment correlation (Pearson r) to examine relationships among the 3 movement strategies and 2 outcomes.

Role of the Funding Source

This study was supported by a grant from the National Institute of Child and Human Development of the National Institutes of Health (HD061678) and by the Jill Pitman Jones Professor for Physical Therapy endowment fund. The funders played no role in the design, conduct, or reporting of this study.

Results

Table 2 reports the means and SDs for the SIPPC-derived and MOCS scores. The means for the 4 time periods are also presented in Figures 2 and 3. ANOVA results revealed a group effect for rotational amplitude, arm movements, distance traveled, and goal-directed movements, but not for leg movements.

Table 2.

Means [SDs] for the Movement Strategies and Outcomes Measures^a

Measures	Group	Weeks 1–3	Weeks 4–6	Weeks 7–9	Weeks 10–12
Rotational amplitude (°)	SIPPC-TD	641 [684]	1056 [819]	1626 [616]	1523 [502]
	SIPPC-RE	737 [371]	846 [470]	759 [400]	916 [386]
	SIPPC-R	459 [404]	537 [398]	519 [334]	415 [373]
Wrist path length, m	SIPPC-TD	26.88 [20.60]	45.09 [28.52]	51.78 [13.51]	55.78 [15.79]
	SIPPC-RE	27.25 [12.26]	29.30 [13.09]	26.75 [12.70]	32.17 [16.05]
	SIPPC-R	34.53 [18.13]	37.47 [15.96]	34.24 [11.51]	27.49 [12.55]
Foot path length, m	SIPPC-TD	23.52 [18.63]	49.43 [31.09]	51.68 [21.58]	55.12 [16.11]
	SIPPC-RE	26.48 [13.97]	34.85 [21.32]	26.62 [16.51]	33.22 [15.80]
	SIPPC-R	28.91 [16.01]	31.82 [22.81]	32.59 [21.87]	26.95 [16.28]
Linear path length, m	SIPPC-TD	1.44 [1.47]	3.63 [3.23]	5.83 [2.81]	6.46 [1.86]
	SIPPC-RE	2.77 [1.45]	3.70 [2.39]	2.98 [2.30]	3.60 [2.24]
	SIPPC-R	1.23 [0.79]	1.64 [1.23]	1.81 [1.26]	1.44 [1.19]
MOCS subscale 3—Mastery of Propulsion (goal-directed movement) raw scores	SIPPC-TD	8.67 [9.55]	21.11 [8.10]	24.40 [6.53]	30.29 [3.87]
	Range	−7 to 19	13-33	14-31	24-32
	SIPPC-RE	15.68 [7.14]	16.84 [6.00]	19.10 [5.17]	19.69 [5.94]
	Range	−6 to 21	2-23	3-24	10-29
	SIPPC-R	9.66 [8.00]	11.55 [10.65]	13.21 [9.31]	10.52 [7.69]
	Range	−2 to 20	−10 to 22	−6 to 20	1-20

^a MOCS = Movement Observation Coding Scheme; SIPPC-R = infants with or at risk for cerebral palsy (CP) who received reinforcement learning only; SIPPC-RE = infants with or at risk for CP who received reinforcement learning and error-based learning; SIPPC-TD = infants with typical development who received reinforcement learning only.

Figure 2.

Bar graphs show the mean and SE of the mean for the Self-Initiated Prone Progression Crawler (SIPPC)–derived movement strategies and outcome measures over the 12-week period. CP = cerebral palsy. Suit assist = combination of reinforcement learning (RL) and error-based movement learning (EBL); no suit assist = RL only.

Figure 3.

Mean Movement Observation Coding Scheme subscale 3 raw scores over the 12-week training period. Time 1 = 1–3 weeks; Time 2 = 4–6 weeks; Time 3 = 7–9 weeks; Time 4 = 10–12 weeks. SIPPC-R = infants with cerebral palsy (CP) receiving reinforcement learning only; SIPPC-RE = infants with CP receiving reinforcement and error-based learning; SIPPC-TD = typically developing infants receiving reinforcement learning only.

Rotational Amplitude

Table 2 presents the means and SDs of the amplitude of rotations. The randomized 2-way ANOVA with repeated measures shows that at baseline (1–3 weeks) and 10- to 12-week time periods, infants in the SIPPC-RE showed more rotations than the SIPPC-R group by 278 degrees (P = .026) and 501 degrees (P = .003), respectively. However, the rotations from weeks 1–3 to 10–12 for infants in the SIPPC-RE and SIPPC-TD groups significantly increased, by 179 degrees (P = .06) and 882 degrees (P = .0001), respectively, whereas those in the SIPPC-R did not show a statistically significant increase (−44°; P = .8).

Wrist Path Length

Table 2 presents the mean distance in meters of the total movements of the arms. The results of a repeated measures 2-way ANOVA indicated that infants in the SIPPC-RE did not show a baseline advantage over the SIPPC-R group in wrist path length and at 10- to 12-week time periods (P = .88). The path length for infants in the SIPPC-RE and SIPPC-TD groups increased by 4.8 m (P = .09) and 27.8 m (P = .008), respectively, but not in the SIPPC-R group (P = .28).

Foot Path Length

Table 2 presents the mean distance in meters of the total movements of the legs. The 2-way ANOVA with repeated measures revealed that neither infants in the SIPPC-RE nor in SIPPC-R groups showed a statistically significant change in foot path length between 1- to 3-week and 10- to 12-week time periods (P = .17 and P = .71, respectively), but the foot path length for infants in the SIPPC-TD group increased by 35.9 m (P = .0001). Hypothesis 1 was partially supported.

Linear Path Length of the SIPPC

Table 2 presents the mean distance in meters and SDs for the linear path length. A randomized 2-way ANOVA with repeated measures revealed that baseline (1–3 weeks) and 10- to 12-week time period results for the SIPPC-RE group showed an advantage (distance traveled) over the SIPPC-R group of 1.5 m and 2.2 m, respectively (P = .004). However, the infants in the SIPPC-RE group showed a gradual increase from weeks 1–3 to 10–12 of 0.87 m (P = .024) compared with those in the SIPPC-R (P = .19) group (see Fig. 3). This finding indicates that the differences were above the baseline differences. Post hoc analysis showed the largest increase in meters for the SIPPC-TD group at 4 to 6 weeks. The largest mean increase in meters for the SIPPC-RE was at 7 to 9 weeks.

Goal-Directed Movement

The MOCS mean scores are presented in Table 2. A 2-way ANOVA with repeated measures indicated that the baseline mean scores for the SIPPC-RE group were higher than the SIPPC-R group, at 16.84 and 11.55, respectively (P = .02). The mean change scores of 4.01 for the SIPPC-RE group over time were higher than 0.96 for the SIPPC-R group, (P = .016 and P = .18, respectively) (Fig. 3). The largest increase in goal-directed movements was at 4 to 6 weeks and at 10 to 12 weeks for the SIPPC-TD and SIPPC-RE groups, respectively, supporting hypothesis 2.

The Pearson r correlation coefficients and slopes between the arm and foot path lengths, rotational amplitude, distance traveled, and MOCS scores are presented in Figure 4. Coefficient ranges were r = 0.77 to 0.90 for infants in the SIPPC-TD group, r = 0.49 to 0.81 for those in the SIPPC-RE group, and r = 0.50 to 0.70 for those in the SIPPC-R group, respectively. Although all the coefficients were statistically significant, they were highest for infants in the SIPPC-TD followed by the SIPPC-RE then the SIPPC-R groups. Similarly, coefficients between rotational amplitude and goal-directed movement scores were higher in the SIPPC-RE than the SIPPC-R group (r = 0.60 and r = 0.49, respectively). However, the coefficients for infants in the SIPPC-TD were the highest. Hypothesis 3 was supported for all but rotational amplitude.

Figure 4.

Correlation between scores from measures of movement strategies (rotational amplitude degrees, foot and wrist path length meters) and outcomes (linear path length meters and MOCS raw scores) over the 12-week training period. Suit assist = (reinforcement learning [RL] + error-based learning [EBL]). No suit assist = (RL only). CP = cerebral palsy; MOCS = Movement Observation Coding Scheme.

Discussion

We examined movement learning strategies in 3 groups of infants using RL and EBL during acquisition of prone locomotion. For this article we focused on 3 movement strategies, arm and leg movements and the amount of trial and error, and 2 outcomes, goal-directed movement and distance traveled. Overall the findings support the differential effect of RL and EBL on skill learning in infants with CP. Differences were observed in both the movement strategies and outcomes. The distinguishing movement strategies for participants with CP were a gradual and greater increase in arm and trial-and-error efforts by the SIPPC-RE group compared with the SIPPC-R group. These movement strategies were related to the greater increases in the distance traveled and goal-directed movements (Figs. 2 and 3). Our results and interpretations should be viewed within the context of infant development and learning. For infants action-outcome coupling is a necessary component of learning new motor skills^23,35 and likely to require RL. Yet motor performance during the initial stages of learning is also uncoordinated and “cognitively demanding”³⁶ and likely to respond better to EBL. A combination of RL and EBL should therefore be advantageous, particularly later during development.

To our knowledge, our study is unique because previous studies with adults contrasted RL with EBL^15,37 using straightforward paradigms such as reaching.^11,15,38,39 Consequently, infants with typical development provide a good reference group for some of our interpretations. In this study, participants in the SIPPC-TD group showed the largest increase in all measures during the first 6 weeks of training, and this was maintained to the end of the study. Participants in the SIPPC-RE group also increased their scores on many metrics and the gains were sustained. Because participants in the TD group only received RL, the finding of participants in the SIPPC-RE raises the possibility that for infants with CP, introduction of RL and EBL must be strategic. In the case of complex actions that require a sequence of movements, for example, prone locomotion, the outcome is removed from the action, making EBL more challenging. However, RL is also believed to help consolidate learned motor behavior,^12,38 but is dependent on the infant's success rate. Because the SIPPC Suit provides immediate and perceptible reward and error feedback for every arm and leg movement, the SIPPC-RE group realized success earlier and for longer than the SIPPC-R group.

Participants in this study showed increasing amounts of trial and error even as goal-directed movements and distance increased. One explanation could be developmental. Motor variability is key to motor control and coordination in infants who are learning new developmental skills.⁴⁰ Variability also promotes and predicts EBL.³⁹ Although motor variability is not synonymous with EBL⁴¹ it is difficult to separate the 2 in infants. Both are regulated by feedback^39,42 and could provide an additive effect. Another explanation is the nature of the task. Unlike in studies that have examined RL in tasks such as reaching,^39,43 prone locomotion is complex. Previous studies have reported a high error rate for complex versus simple tasks and difficulty in achieving optimal feedback control.¹¹ In this study the target outcome was driving the SIPPC for a distance of 6 ft. Consequently the distance of the target (toys or caregiver) was constantly moved to encourage the participant to continue to move. This required continued coordinated movement. Any mismatch in the timing and sequence of arm and leg movements increased the rotational amplitude. Overall, it appears that increased trial and error in infants with CP could benefit motor learning.

Slow Learning and Low Retention

Our findings also revealed 2 limitations of RL and EBL when used with infants with CP. Although participants in this study responded and adapted to the assistance offered by the SIPPC system, the adapted behaviors were not readily repeated at the next sessions. Based on the results of participants in the SIPPC-TD group, this finding cannot be attributed to the age of the participants. One explanation could be the sparse encoding of information that is believed to be associated with error-based learning.¹⁴ Forgetfulness is a barrier to consistent performance and long-term learning and negatively affects entrainment of new patterns of movement during skill learning.³⁵ Another and more compelling explanation could be the type of brain insult in CP. Lack of persistence in the adapted behavior has been reported in studies of adults with stroke.^44,45 Because learning retention has been associated with memory retention, which depends on the primary motor cortex,⁴⁶ this raises questions about the role the type of brain insult in CP could play. Similarly, unlike with adults, the learning rate was slow. A possible explanation could be difficulty in estimating and processing error. The brain also produces an estimate that eventually forms a model based on the reward/error frequency.⁴⁷ Participants in this study had varying levels of early brain insult, and research indicates high prevalences of sensory deficits and sensorimotor connectivity problems in children with CP.^48,49 Another possible explanation could be that the training frequency of twice a week was low and that the training could have been underdosed. Developmentally, early movement behavior is exploratory with goal driven movement emerging with successful and rewarding attempts. Our findings suggest that for infants this transition requires increased frequency of practice.

Our findings also implicate poor muscle force generation and cognition in infants with CP. Participants in the SIPPC-R group only received reinforcement from the SIPPC if they generated a ground reaction force. The distance traveled and the frequency with which they triggered the assist function on the SIPPC confirm that they generated less arm and leg ground reaction force compared with the participants with TD. Lastly, classical RL approaches are based on the assumption that individuals have good cognition, particularly motor memory.⁷ The process of learning entails updating motor commands through repeated exposure to and gradual reduction of motor errors to refine the forward model.^10,50 In this study we only assessed cognition at the beginning and end of study thus limiting comparisons of movement learning strategy.

The role of motivation cannot be overlooked. Maier and associates⁵¹ suggest that dynamically adjusting task difficulty to achieve a prescribed range of performance in a set of memory tasks leads to improved performance over time relative to cases where task difficulty is kept constant. We view this as a means of balancing motivation in performing a task with an appropriate level of challenge to drive further learning. Although we did not measure motivation, we noticed that participants in the SIPPC-TD groups showed interest in persistently reaching for toys as early as 4 to 6 weeks compared with participants with CP. The participants with TD, similar to what has been reported in the literature,²⁵ seem to come equipped with motivation systems. Early in our study, they were driven to seek novelty in outcomes so that they could begin to build causal models between their actions and observed outcomes. In the latter stages of our study, they were driven by curiosity to obtain and interact with toys that were in view. This observation suggests the need to explore the role of motivation and mobility in infants at risk for CP.

Implications for Practice

This study targeted prone locomotion because of the pivotal role it plays in shaping children's functional independence and because infants with CP seldom attain this skill. A large body of knowledge supports the connection between prone locomotion and other domains of infants’ development that are critical for learning and education.^24,52,53 Therefore, our findings have implications for early intervention for children with CP because lack of prone locomotion during infancy can not only negatively affect later motor-related functions but also attainment of other skills necessary for successful participation. The trial-and-error findings underscore the importance of encouraging self-produced movement and allowing infants to fail. The trial-to-trial memory decay highlights the need to carefully balance RL and EBL approaches. Lastly, the SIPPC system design is unique and innovative in that it is not only an intervention device but also can be used to gather comprehensive information about infant movement learning and patterns. The control algorithms also generate data about the direction, speed, efforts expended by infants, and kinetic data that will allow therapists to begin to speculate about brain plasticity.

Limitations

Studying the mechanisms underlying movement learning in infants, let alone those with CP, is fundamentally difficult. The following summarizes some of the limitations of our study.

1. The sample size relative to the high number of repeated measures limited the number of variables we could analyze.

2. The Suit-assist function for some of the participants that were originally assigned to the reinforcement-only group was inadvertently activated resulting in unbalanced groups. Three participants were switched inadvertently from the SIPPC-R group to the SIPPC-RE group during their first sessions. The switch was due to improper configuration of the SIPPC before the sessions began. As such, information about the participants was not used in the decision to make the switch. Once discovered, we chose to keep the participants in the SIPPC-RE group, rather than switching them back to the non–SIPCC-R group. Because our statistical analyses do not rely on matched pairs of participants across the 2 groups, and unequal groups are common in clinical studies, we did not consider this a major flaw.

3. We did not have a comparison group of participants with typical development. Future research will address these limitations and also incorporate brain imaging and cognition.

Conclusions

Movement learning and retention in infants with CP is differentially affected by RL and EBL. A combination of both shows promise. Robotics and machine learning offer an opportunity not only to improve skill acquisition in this population, but also to capture and quantify performance. Our findings implicated neural representation of motor memory and cognition, which must be further explored. The results underscore the complexity of movement learning in infants with CP and the need for more innovative intervention approaches.

Author Contributions and Acknowledgments

Concept/idea/research design: T.H.A. Kolobe, A.H. Fagg

Writing: T.H.A. Kolobe, A.H. Fagg

Data collection: T.H.A. Kolobe, A.H. Fagg

Data analysis: T.H.A. Kolobe, A.H. Fagg

Project management: T.H.A. Kolobe, A.H. Fagg

Fund procurement: T.H.A. Kolobe, A.H. Fagg

Providing participants: T.H.A. Kolobe

Providing facilities/equipment: T.H.A. Kolobe, A.H. Fagg

Providing institutional liaisons: T.H.A. Kolobe

Consultation (including review of manuscript before submitting): A.H. Fagg

The authors thank Peter Pidcoe, PT, DPT, PhD, at Virginia Commonwealth University for constructing the SIPPC robots used in the data collection for this study; David Miller for helping with the maintenance of the robots; Joshua Sutherland for his contribution to the construction of the activity capture sensor suit and data collection; physical therapist data collectors (Michelle Bulanda, Laura Johnson, and Amanda Porter); the DPT graduate research assistants for coding videos; Safoah Twum-Ampofo for assisting with data checking, analyses, and graphs; and the parents and infants for their participation in the study.

Ethics Approval

The study was approved by the University of Oklahoma Health Sciences Center Institutional Review Board (IRB # 2711).

Funding

This study was supported by a grant from the National Institute of Child and Human Development of the National Institutes of Health (HD061678) and by the Jill Pitman Jones Professor for Physical Therapy endowment fund.

Clinical Trial Registration

This clinical trial was not registered. The trial was conducted soon after PTJ instituted its mandatory policy for prospective clinical trial registration (2009), and the authors were not aware of the policy. The Editor-in-Chief has allowed an exception.

Disclosures

The authors completed the ICJME Form for Disclosure of Potential Conflicts of Interest and reported no conflicts of interest.