Abstract
The purpose of the study was to develop a valid and reliable tool for use in assessing motor skills in preschool children in field based settings. The development of the CHAMPS (Children’s Activity and Movement in Preschool Study) Motor Skills Protocol (CMSP) included evidence of its reliability and validity for use in field-based environments as part of large epidemiological studies. Following pilot work, 297 children (3-5 years old) from 22 preschools were tested using the final version of the CMSP and the TGMD-2. Reliability of the CMSP and interobserver reliability were determined using intraclass correlation procedures (ICC; ANOVA). Concurrent validity was assessed using Pearson correlation coefficients to compare the CMSP to the original Test of Gross Motor Development (2nd Edition) (TGMD-2). Results indicated that test reliability, interobserver reliability and validity coefficients were all high, generally above R/r = 0.90. Significant age differences were found. Outcomes indicate that the CMSP is an appropriate tool for assessing motor development of 3-, 4-, and 5-year-old children in field-based settings that are consistent with large-scale trials.
Keywords: preschool motor skills, assessing motor skills, gross motor skills
Introduction
Assessment of motor skill development is widely used for determining the overall rate and level of growth and development in early childhood (Bushnell & Boudreau, 1993; Gallahue & Ozmun, 2006; Burton & Miller, 1998). The preschool years are a developmental period during which most children acquire the basic repertoire of locomotor (e.g., running, jumping, galloping, skipping) and object control (e.g., throwing, kicking, catching, striking) (Clark, 1994; Wickstrom, 1983; Williams, 1983) skills. They also develop goal-directed motor behaviors and learn to combine different movement patterns into sequences to accomplish different goals (Piaget, 1963; Sporns & Edelman, 1993). Motor skill development plays an integral role in the early years when growth, development, and learning frequently center around play and the physical activity associated with it (Clark, 1994; Williams & Monsma, 2006). Because even young children now are faced with increasing prevalence of overweight and inadequate levels of physical activity (Ogden et al., 2006), scientists are examining potential links between motor development and physical activity behavior (Fisher et al., 2005; Graf et al., 2004) and the significant impact the relationship between the two may have on the development of optimal health-related behaviors and future health outcomes (Williams, 2001).
Assessing the quality of the movement processes (e.g. “process characteristics”) involved in performing gross motor skills is integral to assessing the overall level of motor skill development in young children (Burton & Rodgerson, 2001; Williams et al., 2006; Wilson, 2005; Burton & Miller, 1998). Assessment of preschoolers’ gross motor skills is particularly challenging and important in large-scale studies that involve large numbers of young children. A specific aim of the Children’s Activity and Movement in Preschool Study (CHAMPS) was to examine the association between motor skills status and physical activity in children 3 to 5 years of age. A battery that focused on examining critical, qualitative characteristics of motor skill performance (e.g., how the movement is performed) was essential to the CHAMPS project. The CHAMPS Motor Skills Protocol, the focus of this article, was developed primarily as a tool for use in diverse field-based settings that are typical of large, multifaceted epidemiological studies. The purpose of this paper is to describe the development of the CHAMPS Motor Skills Protocol (CMSP) and provide evidence for its validity and reliability.
Methods
Initial Development of the CHAMPS Motor Skills Protocol (CMSP)
A team of four professionals, all with extensive experience in motor skill development in young children, was responsible for developing the CMSP. The initial step in developing the CMSP was a review and careful examination of several available and commonly-used test batteries (Wiart & Darrah, 2001). Specific tests included, among others, the Ages and Stages Questionnaire (Bricker & Squires, 1999), Movement Assessment Battery for Children (MABC; Henderson & Sugden, 1992), Peabody Developmental Motor Scales (PDMS) (Folio & Fewell, 2000), and Test of Gross Motor Development 2nd Edition (TGMD-2) (Ulrich, 2000). After careful review of these and other batteries, an expert panel decided on the TGMD-2 as the prototype for developing the CHAMPS Motor Skill Protocol, because the concepts embedded in the TGMD-2 were ones that provided a logical and appropriate approach to evaluating young children’s motor skill performance. Specifically, the TGMD-2 is one of the few established instruments that uses “process” characteristics of movement performance as a basis for assessing motor skills in young children. Other approaches that might have been used were not as feasible. For example, although the concept of “developmental sequences” was considered as an approach, it was agreed that different aspects of developmental sequences for many skills of interest were not well defined and agreed upon, and that various components of these developmental sequences were difficult to observe without using some external system to record performances for more detailed analysis. Electronic recording of performances, such as videotaping, was not possible given the context of the preschool settings in which we worked (e.g., space limitations, lack of adequate technical capabilities), as well as time constraints and feasibility associated with the large-scale size and nature of the project.
The Test of Gross Motor Development-2nd Edition was examined closely and administered to 17 preschool children. Performances of the children were observed and scored in situ and with videotape. Based on this preliminary work and the goals of the study, an expert panel determined that the TGMD-2 in its current form was not appropriate for use in the study for a variety of reasons. For example, process characteristics of several of the motor skills to be observed were not described in concrete behavioral terms, thus making it difficult to observe or determine whether or not these process characteristics were present in children’s performances. In addition the panel determined that certain important process characteristics for a number of skills not a part of the TGMD-2 needed to be included. In addition, because determining hand/foot preference was important to the study, a standard procedure for doing so was included. It was also important to develop more explicit directions for administering skills and skill sequences.
The expert panel proceeded to develop a framework for the CHAMPS Motor Skills Protocol (CMSP) through discussion and consultation with other professionals. The panel included an early childhood educator, several motor development specialists with experience in administering the TGMD-2, an expert in children’s physical activity, and measurement experts. Keeping the purpose of developing a field-based instrument for large-scale studies as a focal point, critical elements to address in developing the protocol were identified. The elements were: (1) inclusion of critical movement process characteristics to assure appropriate assessment of skills; (2) development of behavioral definitions for those process characteristics; (3) inclusion of an appropriate number of trials to assess each skill and promote reliability; (4) demonstration of skills to the children from different perspectives to support their understanding of the skill to be performed; (5) order of administration of skill sequences; (6) establishment of specific guidelines for determining hand/foot preference, and (7) inclusion of an environmental distraction score to a provide a means for describing the environmental setting in which children’s performances were observed. Details of these elements are described below.
Number and Nature of Movement Process Characteristics
The panel first identified process characteristics that experts consider integral to identifying differences in motor skill performance of children of various ages and skill levels. These movement characteristics were examined in relationship to those included in the TGMD-2 and based upon experience with the TGMD-2 decisions were made about which descriptors to include. For example, in the TGMD-2, one process characteristic for the “throw” is described as “hip and shoulder rotation” and is rated as a single action. First, shoulder rotation is difficult to observe and there were numerous instances in administering the TGMD-2 when children displayed some form of hip rotation but no obvious shoulder rotation. Because variations in “hip-trunk rotation” were more easily observed, hip-trunk rotation were included as one of the movement characteristic of the throw; additional descriptors were added to address the type of rotation occurring at the hip-trunk (e.g., differentiated, block, no rotation). Similar logic was used in defining other process characteristics for various skills. Process characteristics for locomotor skills included in the CMSP are included in Table 1 and for object control skills in Table 2.
Table 1.
Locomotor Subscale
| Locomotor | |
|---|---|
| Skill | Movement Characteristics |
| Run |
|
| Broad Jump |
|
| Slide |
|
| Gallop |
|
| Leap |
|
| Hop |
|
Table 2.
Object Control Subscale
| Object Control | |
|---|---|
| Skill | Movement Characteristics |
| Overarm Throw |
|
| Underhand Roll |
|
| Kick |
|
| Catch |
|
| Stationary Strike |
|
| Stationary Dribble |
|
Number of Trials
The number of process characteristics to be observed for each motor skill was of concern since the increased number might increase the demands placed on the assessors’ observation skills. Based on consultation with measurement specialists, four trials of each skill performance were added to avoid difficulties associated with observing a large number of process characteristics, to assure that adequate opportunities were provided for observing skill performances, and to ensure an appropriate basis for establishing reliability.
Demonstration of Skills
To ensure that children better understood what they were being asked to do, two demonstrations were given for each skill. One demonstration was administered with testers facing children; the other was performed with assessors facing the direction in which children were asked to perform the skill. The exception to this procedure was the hop in which both demonstrations were administered with testers facing and moving in the direction in which children were to perform the hop. All demonstrations were given prior to children’s performing four trials. No additional demonstrations or feedback were provided by the assessors.
Order of Skill Administration
An order for administration of locomotor and object control skills was established. Based on preliminary work with the TGMD-2, it was noted that children exhibited some confusion in moving from one skill to the next in the order suggested by the TGMD-2. To avoid or minimize this confusion, locomotor and object control skills were administered in different sequences; the following sequence for locomotor skills worked well: run, jump, slide, gallop, leap, and hop. This sequence is a commonly observed, although not universally recognized, sequence in which locomotor skills emerge or are acquired during early development (Williams et al., 2006; Wickstrom, 1983). For object control skills, the order of administration was throw, roll, kick, catch, stationary strike, and dribble. The general guiding principle was Gentile’s work (Gentile, 2000), which suggested that skills in which the object is stationary and in the possession of the individual (e.g., as in the throw or roll) tend to be less complex and thus appear earlier than skills in which the object is moving, involves an implement, and is “outside” the control of the person (e.g., as in dribbling, striking objects).
Hand/Foot Preference
The hop, kick, throw, and roll are skills which involve hand or foot preference. Because hand and foot preferences were of interest, and to assure some consistency in how preference was established, the following guidelines were used to determine preference: (1) hand preference was the hand used in at least 3 of 4 trials on both the throw and roll (Bishop, Ross, Daniels, & Bright, 1996; DeAgostini & Dellatolas, 2001; Leconte & Fagard, 2004) and (2) use of the same hand on 2 or fewer trials on both tasks was an indication of lack of preference or dominance. As with hand preference, the preferred foot was determined by the use of the lead foot in the gallop and the leg used to kick in 3 of 4 trials. Use of the same foot on 2 or fewer trials on both tasks indicated a lack of established foot dominance or preference. A specific procedure for identifying hand and foot preference would allow for the use of dominance and lack of dominance as a potential factor in motor skill development.
Environmental Distraction Score
Children from three types of school settings (i.e., Head Start programs, religious preschools, and commercial childcare centers) were involved in the CHAMPS study. Therefore, motor skills testing was performed within and during a wide range of environmental circumstances and contexts (e.g., hallways, cramped spaces, noisy gyms). Because situational inconsistencies can be an important factor in test administration and interpretation (Doty, McEwen, Parker, & Laskin, 1999; Lyman, 1997), a distraction score was developed. The score was a subjective rating of the nature of the field-based circumstances in which skill observations took place. The measure allowed for the assessment of potential effects of various environmental influences on children’s skill performances and the assessors’ ability to accurately observe children’s skill performances during routine field-based situations.
Four categories formed the basis for the distraction score: (1) noise level (e.g., noise-related disturbances from classrooms, hallways); (2) general distractions (e.g., traffic in the testing area during demonstrations, teachers watching and performing the skill); (3) temperature of the testing area; and (4) nature and adequacy of the space (e.g., inadequate floor space, inappropriate floor surface). Each category was rated on a 1 to 5 Likert-type scale with 1 being “definitely interfering” and 5 being “not interfering.” Testers’ ratings from each category were summed to create an overall distraction score that ranged from 4 to 20.
Training and Selection of Motor Development Testers
Because the number of trials and the number and nature of movement process characteristics to be observed might make the assessment process more complex, and because a protocol was being developed, it was determined that testers should have some background in motor development and, if possible, experience working with preschool children. Before administering the CMSP in the actual test setting, extensive training of two testers, who were professionals with a background in motor development, was completed. The testers completed approximately 51 hours of training using both videotape and in situ observations of developmentally diverse preschool children from a pediatric perceptual-motor development program and two private preschools. Children were observed at least twice to develop some consistency in rating process characteristics and to establish an estimate of inter- and intra-observer reliability. Informed consent was obtained from parents and a monetary incentive was provided. Six children from the university-based pediatric motor development program were observed and videotaped. Performances were scored initially while they were videotaped and were also reviewed and scored again later. Five children from a private preschool then were observed, scored and videotaped as described above. Finally, live observations of six children were conducted at a second private preschool. Two testers were involved, with one observing and scoring the CMSP while the other administered the motor skill tasks and scored the assessment. These two tester roles were reversed throughout the study and were counterbalanced across children. A professional with extensive experience in young children’s motor development also viewed and scored both in situ and videotape observations and served as the “gold standard” criterion during training. In terms of interrater reliability, the “gold standard” and testers’ scores were significantly correlated (R = 0.86 - 0.99, p < 0.001) for body control (locomotor), object control, and total scores. Further, intraclass correlation coefficients (ICC; ANOVA) revealed no significant difference among any combination of raters. Intraobserver reliability was also examined during training for data on five participants. Results of ICC analyses indicated that intraobserver reliabilities were high and positive (R = 0.56 - 0.92).
Preschool Participants
Participants for the main trial included 297 children (158 boys; 139 girls) from 22 preschools. Data were collected on 98 three-year-olds (50 boys, 48 girls), 148 four-year-olds (75 boys, 73 girls), and 51 five-year-olds (33 boys, 18 girls) over an 18-month period (August 2004 through January 2006). Preschools involved were of three types: (1) commercial (n = 11 schools; 136 children), (2) religious-based preschools (n = 7 schools; 88 children), and (3) Head Start programs (n = 4 programs; 73 children). One or more parents of 53% of the children had completed college or technical school or had higher degrees. The remaining 47% of parents either were attending high school or had a high school diploma. The racial make-up of the group was as follows: 51% African American, 40% Caucasian, and 9% Other (Hispanic, Asian American, etc.).
Procedures
The two previously-described testers with a background in motor development collected all CMSP field data. Data collection generally lasted for approximately one week per preschool, depending on the number of children from each preschool who were participating and time constraints imposed by the preschool personnel. For most of the testing, both testers were present and administered the assessment protocols. One tester demonstrated motor skills and administered the CMSP, while the other one recorded performance data and rated environmental distractions.
Children were tested individually; the time required was approximately 45 minutes per child. Testers chose the best possible circumstances for testing in each preschool. Differences within preschool types and among the 22 preschools resulted in a wide variation in testing circumstances and environments. Whenever possible, a gym or long hallway that was free of traffic and distraction was used for assessing children. In some cases, it was not possible to avoid a central location in a preschool and distractions such as teachers or children interrupting assessments occurred occasionally during testing. Children completed the CMSP and the TGMD-2 during the one week testing period. Appropriate breaks were given as needed to avoid fatigue and maintain attention. Procedures for administering the TGMD-2 were those recommended in the TGMD-2 Manual (Ulrich, 2000). There were two trials per each skill; all process characteristics were rated as 0 or 1. The testers took detailed notes of each testing; the CMSP and TGMD-2 outcome data were entered into a software package (Epi Info™) within one week of testing.
CMSP Outcome Measures
Scores on the CMSP were ratings of movement process characteristics of skills in the Locomotor subscale, Object Control subscale, and Total Test categories. Movement process characteristics were rated as “1” (present) or “0” (not present) for most skills and summed (separately for 2 and 4 trials) to arrive at a score for locomotor, object control, and total test performances. Exceptions to the rating of “1” and “0” were made for one process characteristic each of the throw, strike, and hop. For the throw and strike, “hip-trunk rotation” was scored as “2” (differentiated), “1” (block), and “0” (no rotation). For the hop, a moving hop was scored “2,” a stationary hop as “1,” and no hop as “0.” These data were used in a series of analyses to examine age differences in gross motor development and to examine validity, test reliability and interobserver reliability.
Validity
Concurrent validity
To assess the suitability of using the CMSP, performances on the CMSP test and on a criterion test, TGMD-2 (Wood, 1989) were examined. Performances on both tests were observed during the same time frame. Because the validity of the TGMD-2 has been well established and it is used frequently, the Pearson correlation coefficients were used to compare the CMSP to the TGMD-2.
Construct validity
Construct validity is the degree to which a test measures the construct it was designed to measure (Wood, 1989). Several factors may be used to demonstrate construct validity. Two constructs were examined: age differentiation and correlations among subtests. Because gross motor skill performance is developmental in nature, scores on the CMSP should be strongly associated with chronological age and should differentiate among age groups. With regard to locomotor and object control skills subscale correlations, because both sub-tests purport to measure gross motor development but from different dimensions, each subscale should have a strong positive correlation with the total test score and a more moderate correlation with each other than with the total test score. A series of analyses were performed using Proc Mixed (controlling for school as a random effect) in SAS (Version 8.2; Cary, NC) to examine age-related differences. Pearson correlations were used to examine subtest relationships.
Reliability Estimates
Reliability refers to the consistency with which assessments and assessors measure behavioral attributes (Baumgartner, 1989). Reliability of the CMSP was examined through determining (a) test reliability (also referred to as internal consistency) with and without distraction scores as a factor, and (b) interobserver reliability (also referred to as objectivity) (Baumgartner, 1989) .
Reliability of CMSP
Data for children (N = 297) from the 22 preschools were included in the reliability analysis. Reliability estimates were completed for (a) locomotor subscale, object control subscale, and total scores, (b) 2 and 4 trials, and (3) with and without distraction scores as a factor. Both Pearson correlations and intraclass correlation (ICC; one-way ANOVA) procedures were used. In estimating the effect of distraction scores on test reliability, “high” and “low” distraction categories were created using the median distraction score as the cut-off. Scores of 18 or higher were designated as “high” distraction, those less than 18 as “low” distraction. Separate analyses were run for “high” and “low” categories.
Interobserver reliability
To examine interobserver reliability, both testers scored performances on 50 children from 22 schools throughout the duration of the project. This represents approximately 11% of total participants (32 boys; 18 girls; mean age: 4.3+/−0.9 years). During reliability sessions, the testers were in close proximity; this was to ensure that they had similar perspectives for observing performance, and recorded the scores concurrently and independently. Kappa statistics were used to provide estimates of interobserver reliability; estimates of Kappa were calculated for each process characteristic using data from the first trial only. Pearson correlations, the intraclass correlation (ICC) procedure, and the Estimates of Pearson correlation and ICC were completed for (a) locomotor skills, object control, and total test items, and (b) for 2 and 4 trials.
Results
Concurrent and Construct Validity
To address concurrent validity, scores on the original TGMD-2, the CMSP with 2 trials, and the CMSP with 4 trials were compared using Pearson correlations. Coefficients ranged from 0.94 - 0.98 for all comparisons (see Table 3). With regard to construct validity and the factor of “age differentiation,” means and standard errors of measurement (SE) of 3-, 4-, and 5–year-old children are shown in Table 4, with and without control of distraction scores. There were significant differences among age groups for the locomotor subscale, the object control subscale, and total test scores for both 4 and 2 trials. Four-year olds had significantly higher scores than 3-year olds and 5-year olds had significantly higher scores than 4-year olds. Differences between 3- and 4-year olds were greater than those between 4- and 5- year olds.
Table 3.
Concurrent Validity: Coefficients Among Motor Skill Assessments
| Locomotor | Object control | Total | |
|---|---|---|---|
| TGMD-2 vs. CMSP-2 | 0.98 | 0.97 | 0.98 |
| TGMD-2 vs. CMSP-4 | 0.95 | 0.94 | 0.97 |
| CMSP-2 vs. CMSP-4 | 0.98 | 0.97 | 0.98 |
NOTE: TGMD-2 = original TGMD; CMSP-2 = CHAMPS protocol with 2 trials; CMSP-4 = CHAMPS protocol with 4 trials; p < 0.0001 for all values
Table 4.
Age Differences Among CMSP Scores for 2 and 4 Trials
|
CMSP-2 (Mean [SE])* Distraction score not accounted for |
CMSP-2 (Mean [SE])* Distraction score accounted for |
|||||
|---|---|---|---|---|---|---|
| LM | OC | Total | LM*** | OC | Total** | |
| 3 (n=98) | 31.8 (1.2) | 37.6 (1.0) | 69.5 (1.9) | 31.9 (1.1) | 37.7 (1.0) | 69.6 (1.7) |
| 4 (n=148) | 41.5 (1.0) | 47.4 (0.8) | 88.9 (1.6) | 41.5 (1.0) | 47.3 (0.8) | 88.9 (1.4) |
| 5 (n=51) | 46.7 (1.5) | 54.7 (1.4) | 101.6 (2.4) | 46.4 (1.5) | 54.4 (1.4) | 101.0 (2.3) |
|
CMSP-4 (Mean [SE])* Distraction score not accounted for |
CMSP-4 (Mean [SE])* Distraction score accounted for |
|||||
|---|---|---|---|---|---|---|
| LM | OC | Total | LM | OC** | Total** | |
| 3 (n=98) | 62.7 (2.4) | 74.8 (1.9) | 138.0 (3.8) | 62.9 (2.3) | 75.1 (1.9) | 138.2 (3.6) |
| 4 (n=148) | 82.2 (2.1) | 94.9 (1.5) | 177.2 (3.3) | 82.2 (2.0) | 94.8 (1.5) | 177.1 (3.0) |
| 5 (n=51) | 93.3 (3.0) | 109.2 (2.6) | 202.8 (4.8) | 92.7 (3.0) | 108.6 (2.6) | 201.5 (4.7) |
All age groups different for all conditions p < 0.0001
Distraction score significant p < 0.05
Distraction score significant p = 0.055
NOTE: CMSP-2 = CHAMPS protocol with 2 trials; CMSP-4 = CHAMPS protocol with 4 trials, LM = locomotor, OC = object control, Total = total score
CMDP Reliability
Reliability estimates for the CMSP are shown in Table 5. Estimates were high and ranged from R = 0.88 to 0.97. Reliability estimates when distraction scores were not accounted for were 0.90 and above and were slightly higher for four trials than for two. Reliability estimates based on distraction scores were similar for both “high” and “low” distraction categories; estimates in both categories were 0.88 or higher and again were slightly higher for four than for two trials.
Table 5.
Reliability of the CHAMPS Motor Skill Protocol (CMSP) for 2 and 4 Trials and Low and High Distraction Scores
| Without accounting for distraction score |
Accounting for distraction score | |||||
|---|---|---|---|---|---|---|
| 2 Trials | 4 Trials | 2 Trials – High |
2 Trials – Low |
4 Trials – High |
4 Trials – Low |
|
| Locomotor | 0.92 | 0.96 | 0.92 | 0.92 | 0.96 | 0.95 |
| Object Control |
0.90 | 0.92 | 0.91 | 0.88 | 0.95 | 0.94 |
| Total | 0.94 | 0.97 | 0.94 | 0.94 | 0.97 | 0.97 |
NOTE: Scores of 18 or higher were designated as “low” distraction, those less than 18 were “high” distraction
Interobserver reliability
The range of Kappa coefficients for the first trial of each skill were: run (0.80-1.00), broad jump (0.48-0.96), slide (0.48-0.88), gallop (0.71-1.00), leap (0.79-0.84), hop (0.77-0.92), overarm throw (0.50-1.00), underhand roll (0.63-1.00), kick (0.66-1.00), catch (0.51-0.92), strike (0.32-0.92), and dribble (0.74-1.00) Due to missing data, Kappa coefficients for one process characteristic of the strike and one for the underhand roll could not be calculated. It is important to note that most skills had kappa coefficients above 0.6 for all process characteristics; four skills (jump, slide, catch, and strike) had one process characteristic with a kappa coefficient less than 0.6, and one skill (overarm throw) had two coefficients less than 0.6. Interrater reliabilities for two 2 and 4 four trials are given in Table 6. Results of ICC analyses indicated that interobserver reliabilities for four trials were high for the locomotor subscale (R = 0.99), object control subscale (R = 0.98) and total test (R = 0.94). Pearson correlations were also high and significant [p=.001 (locomotor subtest: r = 0.98; object control subtest: r = 0.97; total test: r = 0.90)]. Correlations and ICCs for four trials were similar to or exactly the same as those for two trials.
Table 6.
Interobserver Reliability (Objectivity) for the CMSP
| Test Components |
R | Raters Differ p-value* |
Pearson Correlations |
|---|---|---|---|
| 2 Trials (n=50) | |||
| Locomotor | 0.99 | p=0.66 | 0.97** |
| Object Control | 0.98 | p=0.29 | 0.97** |
| Total | 0.94 | p=0.71 | 0.89** |
| 4 Trials (n=50) | |||
| Locomotor | 0.99 | p=0.57 | 0.98** |
| Object Control | 0.98 | p=0.25 | 0.97** |
| Total | 0.94 | p=0.77 | 0.90** |
p-values for difference between testers
p<0.001
The effect of “distraction score” on performances on the CMSP (4 trials) was significant for both the object control subscale and total test scores (p < 0.05); distraction scores influenced the motor skills scores of children of all ages. For 2 trials, the effect of distraction was significant for total test score only (p < 0.05). When distraction scores were controlled for, scores were slightly lower for 5-year olds and slightly higher for 3-year-olds than when they were not (see Table 6). Overall the distribution of “distraction scores,” which ranged from 4 to 20, was skewed in the direction of “non-distracting” environmental conditions (mean score = 17.0, SD = 2.8, median = 18.0, minimum = 10, maximum = 20). The high mean score suggests that in general the environments in which testing took place were not distracting.
In terms of sub-test correlations, another indicator of construct validity, relationships between subtest scores and total test score were high, positive and in the appropriate direction. For four trials, the correlation of both object control and locomotor scores with total score was R = 0.88. The correlation between locomotor and object control subscale scores was R = 0.56. Correlations for four trials were similar to those for two trials (locomotor and object control subtest scores with total test score, R = 0.88; locomotor with object control subscale scores, R = 0.55).
Discussion
Overall, data from this study indicate that the CHAMPS Motor Skill Protocol (CMSP) may be used as a reliable and valid method for assessment of preschool children’s motor skill performance in field-based settings. Specifically, test reliability, interobserver reliability, and validity coefficients were high and generally above 0.90. In terms of construct validity, evidence indicated that the CMSP differentiated among preschool age groups on locomotor, object control, and total test scores and thus provides a valid method for assessing age differences in gross motor skill performance in an important period of child growth and development. Subscale scores also correlated moderately with each other and were highly correlated with total test score, another indicator of construct validity. Concurrent validity was high and based on the positive correlation of the CMSP with the TGMD-2.
Although 4 trials were included in the protocol to provide assessors additional observations of a complex number of movement characteristics, data indicate that reliability and validity were similar for 2 and 4 four trials. This suggests that comparable results may be obtained using 2 trials on the CMSP. Given the potential complexity associated with observing motor skill performances of large numbers of young children in field-based settings, consideration might be given to including 4 trials if testers are less experienced than the two CMSP testers, who had strong backgrounds in motor skill development, several years of experience working with young children, and extensive training prior to administering the protocol in the field setting. Lack of experience in testing of motor skills of young children may be more likely for data collection staff involved in large-scale trials. In such cases, some preliminary training may be required. Another consideration may be the time required to administer 2 versus 4 trials; the time to administer 4 trials was approximately 45 minutes while the estimated time to administer 2 trials is 25-30 minutes.
The number of movement skill characteristics to be observed did not appear to be an issue. This is a strength of the CMSP protocol in that by formulating more precise, observable behavioral definitions of the movement skill characteristics to be observed, testers had clearer behavioral cues to assist them in “what to look for” in children’s performances. These “concrete cues” may have aided test administrators in processing numerous movement skill characteristics and most likely contributed to the high test reliability, interrater reliability, and validity of the CMSP. These more precise behavioral definitions allowed for greater success in training testers to use the protocol reliably.
Environmental distraction scores had no affect on reliability of the test or on interobserver reliability (objectivity). This suggests that ongoing distractions in the testing environments did not, in this study, affect reliability of the assessment of children’s performances. Data clearly were skewed toward a non-distracting environment (mean = 17); the range of scores also was limited (min = 10; max = 20; scale = 4 - 20). It is interesting that although reliability and objectivity estimates were not affected by “distraction scores,” performance outcomes on both locomotor and total test scores were. These differences in performance outcomes were small but were present for children of all ages. The value of the distraction score is that it allows investigators to describe or rate and control for, at a simple level, the potential influence of the testing environment on measurement and performance outcomes. This is an advantage of the CMSP since most protocols do not provide even minimal standardized guidelines for taking into effect environmental influences on assessment of performance outcomes.
A major strength of the CMSP protocol is that it was used with a relatively large and diverse sample of preschool children from a variety of preschool circumstances and settings. A limitation of our study may be the method by which concurrent validity was established. The recommended approach is to administer the current test and the criterion test in two different time frames. In the present investigation, we collected data on the CMSP and the TGMD-2 within the same time frame. Nevertheless, coefficients were high and suggest that the CMSP is a valid measure of gross motor performance of preschool age children.
Overall, outcomes on the CMSP indicate that the protocol is appropriate for use in large-scale research to assess preschool children’s gross motor skill performances and provides a viable alternative to existing instruments for use in epidemiological, population-based studies that involve large numbers of children. To gain a fuller picture of motor skill development of preschoolers, work is being initiated that will explore the use of developmental sequences in the protocol by examining a smaller subset of the preschool population. In addition, pending further replications and additional validation work, it is possible that the CMSP may also be a valuable assessment instrument for providing important information about children’s gross motor skill needs, for making appropriate referrals for services to enhance children’s motor skill development, and for recommending well-targeted and individualized motor skill development interventions where appropriate.
Reference List
- Baumgartner TA. Norm-referenced measurement: Reliability. In: Safrit MJ, Wood TM, editors. Measurement Concepts in Physical Education and Exercise Science. Human Kinetics; Champaign, IL: 1989. [Google Scholar]
- Bishop DV, Ross VA, Daniels MS, Bright P. The measurement of hand preference: A validation study comparing three groups of right-handers. British j Journal of Psycholpg. 1996;87(Pt 2):269–285. doi: 10.1111/j.2044-8295.1996.tb02590.x. [DOI] [PubMed] [Google Scholar]
- Bricker D, Squires J. Ages and Stages Questionnaires: A Parent-completed Child-monitoring System. 2nd Edition Paul H. Brookes Publishing Co.; Baltimore: 1999. [Google Scholar]
- Burton A, Miller D. Movement Skill Assessment. Human Kinetics; Champaign, IL: 1998. [Google Scholar]
- Burton AW, Rodgerson RW. New perspectives on the assessment of movement skills and motor abilities. Adapted Physical Activity Quarterly. 2001;18:347–365. [Google Scholar]
- Bushnell EW, Boudreau JP. Motor development and the mind: The potential role of motor abilities as a determinant of aspects of perceptual development. Child Dev. 1993;64:1005–1021. [PubMed] [Google Scholar]
- Clark J. From the beginning: A perspective on movement and mobility. Quest. 2005;57:37–45. [Google Scholar]
- Clark J. Motor development. In: Ramachandran VS, editor. Encyclopedia of Human Behavior. Academic Press; San Diego: 1994. pp. 245–255. [Google Scholar]
- DeAgostini M, Dellatolas G. Lateralities in normal children ages 3 to 8 and their role in cognitive performances. Developmental Neuropsychology. 2001;20:429–444. doi: 10.1207/S15326942DN2001_7. [DOI] [PubMed] [Google Scholar]
- Doty AK, McEwen IR, Parker D, Laskin J. Effects of testing context on ball skill performance in 5-year-old children with and without developmental delay. Physical Therapy. 1999;79:818–826. [PubMed] [Google Scholar]
- Fisher A, Reilly JJ, Kelly LA, Montgomery C, Williamson A, Paton JY, et al. Fundamental movement skills and habitual physical activity in young children. Medicine and Science in Sports and Exercise. 2005;37:684–688. doi: 10.1249/01.mss.0000159138.48107.7d. [DOI] [PubMed] [Google Scholar]
- Folio M, Fewell R. Motor Development: Peabody Developmental Motor Scales (PDMS-2) PRO-ED; Austin, TX: 2000. [Google Scholar]
- Gallahue D, Ozmun J. Understanding Motor Development: Infants, Children, Adolescents, and Adults. 6th ed McGraw-Hill; Boston: 2006. [Google Scholar]
- Gentile A. Skill acquisition: Action, movement, and neuromotor processes. In: Carr JH, Shepherd RB, editors. Movement Science: Foundations for Physical Therapy. 2nd edition Aspen; Rockville, MD: 2000. [Google Scholar]
- Gesell A. The First Five Years of Life: A Guide to the Study of the Preschool Child. Harper and Row; New York: 1973. [Google Scholar]
- Graf C, Koch B, Kretschmann-Kandel E, Falkowski G, Christ H, Coburger S, et al. Correlation between BMI, leisure habits and motor abilities in childhood (CHILT-project) International Journal of Obesity and Related Metabolic Disorders. 2004;28:22–26. doi: 10.1038/sj.ijo.0802428. [DOI] [PubMed] [Google Scholar]
- Haywood K, Getchell N. Lifespan Motor Development. 4th ed Human Kinetics; Champaign, IL: 2005. [Google Scholar]
- Henderson SE, Sugden DA. Movement Assessment Battery for Children. The Psychological Corporation, Ltd.; Sidcup, Kent: 1992. [Google Scholar]
- Leconte P, Fagard J. Influence of object spatial location and task complexity on children’s use of their preferred hand depending on their handedness consistency. Developmental Psychobiology. 2004;45:51–58. doi: 10.1002/dev.20019. [DOI] [PubMed] [Google Scholar]
- Lyman H. Test Scores and What They Mean. 6th ed Prentice-Hall; Englewood Cliffs, NJ: 1997. [Google Scholar]
- Ogden CL, Carroll MD, Curtin LR, McDowell MA, Tabak CJ, Flegal KM. Prevalence of overweight and obesity in the United States, 1999-2004. Journal of the American Medical Association. 2006;295:1549–1555. doi: 10.1001/jama.295.13.1549. [DOI] [PubMed] [Google Scholar]
- Piaget J. The Origins of Intelligence in Children. Norton; New York: 1963. [Google Scholar]
- Sporns O, Edelman GM. Solving Bernstein’s problem: A proposal for the development of coordinated movement by selection. Child Development. 1993;64:960–981. [PubMed] [Google Scholar]
- Ulrich D. Test of Gross Motor Development, 2nd edition. Examiner’s Manual. PRO-ED; Austin, Texas: 2000. [Google Scholar]
- Wiart L, Darrah J. Review of four tests of gross motor development. Developmental Medicine and Child Neurology. 2001;43:279–285. doi: 10.1017/s0012162201000536. [DOI] [PubMed] [Google Scholar]
- Wickstrom J. Fundamental Motor Patterns. 3rd ed Lea and Febiger; Philadelphia: 1983. [Google Scholar]
- Williams H. Perceptual and Motor Development in Young Children. Prentice-Hall, Inc.; Englewood Cliffs, NJ: 1983. [Google Scholar]
- Williams H, Monsma E. Assessment of gross motor development in preschool children. In: Barcken B, Nagle R, editors. The Psychoeducational Assessment of Preschool Children. Lawrence Erlbaum; Hillsdale, NJ: 2006. [Google Scholar]
- Williams PT. Physical fitness and activity as separate heart disease risk factors: A meta-analysis. Medicine and Science in Sports and Exercise. 2001;33:754–761. doi: 10.1097/00005768-200105000-00012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wilson PH. Practitioner review: Approaches to assessment and treatment of children with DCD: An evaluative review. Journal of Child Psychology and Psychiatry. 2005;46:806–823. doi: 10.1111/j.1469-7610.2005.01409.x. [DOI] [PubMed] [Google Scholar]
- Wood TM. The changing nature of norm-referenced validity. In: Safrit MJ, Wood TM, editors. Measurement Concepts in Physical Education and Exercise Science. Human Kinetics; Champaign, IL: 1989. [Google Scholar]