Performance Metrics From Product-Oriented Measures of Fundamental Motor Skills—A Comparison and Developmental Perspective

Abstract

This study (a) examined the associations among different performance metrics derived from different strategies (i.e., maximum and average scores) and trials from product-oriented measures of motor skills, and (b) explored how different performance metrics from product-oriented assessments of motor skills change in young children with typical development. Children (N = 279; 156 girls; M_age = 4.44 years) completed a battery of product-oriented assessments for throwing (in meters per second, five trials); kicking (in meters per second, five trials); jumping (in centimeters, five trials); running (in meters per second, two trials); and hopping (in meters per second, four trials—two preferred foot, two nonpreferred foot). A total of 36 performance metrics were derived—throw (n = 7), kick (n = 7), jump (n = 7), run (n = 4), and hop (n = 11). Intraclass correlations examined reliability among performance metrics for each skill; linear mixed models examined whether variations changed across early childhood. There was excellent reliability among all performance metrics for each skill (all ICC> .90). Linear mixed models revealed that children’s motor performance improved for two metrics of the throw, five variations of the jump, and three metrics of the hop (all p <.05). Researchers should be aware that some performance metrics from product-oriented assessments (e.g., maximum and average of three or five trials) are highly related and change, whereas others do not.

Fundamental motor skills are basic movement skills rooted in early movement milestones such as reaching/grasping and independent locomotion that serve as the building blocks for more sport- and context-specific skills (Burton & Miller, 1998). Competency in fundamental motor skill competency (referred to as motor skill competency) refers to a child’s degree of proficiencies in a range of motor skills as well as the underlying factors and mechanisms that support those skills (Utesch & Bardid, 2019). Generally, assessment of fundamental motor skills, and subsequent measure of motor skill competence, includes measures of an individual’s posture/balance, ball skills (i.e., propel or manipulate objects in space), or locomotor skills (i.e., propel or manipulate the body through space; Ulrich, 2019). These skills should develop in childhood, and motor skill competency is a critical component of children’s trajectories of health and well-being (Barnett et al., 2022; De Meester et al., 2020; Robinson, Stodden et al., 2015; Stodden et al., 2008; Utesch et al., 2019). Based on the importance of developing these skills to promote various developmental outcomes, validly and reliably measuring children’s motor skill competency is critical to capture its predictive utility accurately. A recent systematic review demonstrated that there is a wide variety of fundamental motor skill assessments used in the literature (Hulteen, Barnett et al., 2020), with a relative lack of agreement among assessments in their measurement of motor skill competency levels and categorizations (e.g., the risk for developmental delay; Logan et al., 2017; Palmer et al., 2021; Ré et al., 2018). Therefore, there is a need to examine motor skill assessment methodologies and protocols to determine the extent to which assessments yield unique or similar information regarding children’s motor skill competency levels.

Motor skill assessments generally adopt one of two orientations to assessment, measuring the outcome of skill performance (i.e., product-oriented such as speed, distance, or accuracy) or movement patterns (i.e., process-oriented such as scoring the presence or absence of qualitative skill criteria, and ordinal scoring of segmental component skill criteria). Within each approach, there are different performance metrics that can be derived and used as the measure or yield from the assessment. These performance metrics are particularly varied in product-oriented assessments of motor skills. For example, product-oriented assessments include scoring a range of trials (two–10 trials for each skill; Halverson et al., 1977; Palmer et al., 2021; True et al., 2017) and different strategies can be applied to derive scores from the multiple trials (e.g., an average of trials or maximum score). Halverson et al. (1977) used an average of 10 trials to report throwing performance. More recent research has also used an average of two trials to document run, throw, kick, and jump performance (Hulteen, Barnett et al., 2020). Researchers have also used maximum performance metrics across multiple trials (e.g., two, three, or five trials) to evaluate the performance of throwing, kicking, jumping, and running from product-oriented assessments (Logan et al., 2017; Rodriguez-Negro et al., 2021; Stodden et al., 2006a, 2006b, 2014). Thus, there is no consensus or best practice on which strategies and subsequent performance metrics should be pulled from product-oriented assessments of motor skills. To make an informed recommendation, it is important to understand the relationship between different strategies for evaluating product-oriented measures of performance (e.g., maximum, sum, and average of multiple trials). Information learned from evaluating relationships among these strategies could be used to inform best practices for future research from both measurement and feasibility perspectives.

Testing and analyzing competency in a wide variety of motor skills can be time- and labor-intensive, especially with large-scale data collections; therefore, research is needed to determine the number of trials needed to adequately represent children’s competency levels with product-oriented assessments. This information is particularly important as optimizing testing efficiency (e.g., minimizing the number of trials needed) could lighten the load for participants, researchers, and practitioners during assessment, as well as could allow us to potentially aggregate process- and product-oriented assessments using the same number of trials. For example, common process-oriented assessments (e.g., Test of Gross Motor Development [TGMD]; Ulrich, 2019) only use three trials divided across one practice trial and two test trials. If we could assess motor skill competency using two trials of product-oriented assessments, then we could combine data collection protocols and minimize burnout and fatigue for participants as well as maximize research staff time and effort. Further, simultaneously collecting product- and process-oriented measures of performance from the same skill trials could provide a more comprehensive evaluation of competency levels without introducing error or changes in performance based on inconsistent testing conditions such as participant affect or motivation. Therefore, there is a need to determine the associations among different performance metrics derived from different strategies (i.e., maximum and average scores) and different trial numbers. The first purpose of this study was to examine the associations among different performance metrics derived from different strategies (i.e., maximum and average scores) and trials (e.g., three trials vs. five trials) from product-oriented measures of throw, kick, jump, run, and hop.

Lastly, to truly inform practice, it would be helpful to examine these associations over development or time. Early childhood is marked by the development of movement patterns (Ulrich, 2019) and product outcomes (Roberton et al., 1979) in motor skills. There is a need to understand whether and how performance metrics derived from different strategies and trials of skill performance represent changes across early childhood (3–7 years). The secondary purpose of this study was to explore how different performance metrics from product-oriented assessments of motor skills change in young children with typical development.

Materials and Methods

Sample

This study included 279 children (146 girls, M_age at study enrollment = 4.44 ± 0.27 years) from three Head Start Centers in the United States. Parents were eligible to enroll their children in a Head Start program if their annual income was at or below the federal poverty line (i.e., <$26,200 USD for a family of four). Children between 3.5 and 5 years of age at the time of study recruitment with no cognitive or developmental disabilities on record with the school were invited to participate in the study.

Motor Skills

Children completed a battery of five motor skills—throwing, kicking, jumping, running, and hopping (see Table 1). These five skills, and the number of trials per skill, were selected for two reasons. First, these skills have been included in other research examining motor competence through product-oriented measures in similar populations and have been shown to be developmentally valid and sensitive discriminators of performance (Luz et al., 2019; Nesbitt et al., 2018; Palmer et al., 2021; Rodrigues et al., 2016; Stodden et al., 2013, 2014; True et al., 2017). Second, these skills include three of the four most common skills found on motor competence assessments: overhand throw, jump, and hop (Hulteen, Barnett et al., 2020).

Table 1.

Overview of Skills, Data Collection, Verbal Instruction, and Variations in Product-Oriented Measures

Skill	Measure	Data collection procedure	Verbal prompt	Variation in product-oriented measures
Throw	Speed (m/s)	The child threw the ball at the wall five times. Throwing speed (mph) was measured using a radar gun.	“Throw the ball as hard and fast as you can toward the wall.”	1. Max speed across all five trials 2. Avg speed across all five trials 3. Avg speed across best three trials 4. Max speed across first three trials 5. Avg speed across first three trials 6. Max speed across second and third trials 7. Avg speed across second and third trials
Kick	Speed (m/s)	The child kicked the ball toward the wall five times. Kicking speed (mph) was measured using a radar gun.	“Kick the ball as hard and fast as you can towards the wall.”	1. Max speed across all five trials 2. Avg speed across all five trials 3. Avg speed across best three trials 4. Max speed across first three trials 5. Avg speed across first three trials 6. Max speed across second and third trials 7. Avg speed across second and third trials
Jump	Distance (cm)	The child completed five standing broad jumps. Distance (cm) was measured after each jump.	“Jump as far as you can.”	1. Max distance across all five trials 2. Avg distance across all five trials 3. Avg distance across best three trials 4. Max distance across first three trials 5. Avg distance across first three trials 6. Max distance across second and third trials 7. Avg distance across second and third trials
Run	Speed (m/s)	The child ran a 7-m distance two times. The speed (m/s) of the run was assessed in the lab.	“Run as fast as you can to the cone.”	1. Speed Trial 1 2. Speed Trial 2 3. Avg speed 4. Max speed
Hop	Speed (m/s)	The child hopped across a 7-m distance four times (two times on each foot). If they were able to complete four consecutive hops, the speed (m/s) of the hop was assessed in the lab.	“Hop as fast as you can to the cone” and “Big hops!” (additional cue during skill performance)	1. Speed pref foot, Trial 1 2. Speed pref foot, Trial 2 3. Avg speed—pref foot 4. Max speed—pref foot 5. Speed nonpref foot, Trial 1 6. Speed nonpref foot, Trial 2 7. Avg speed—nonpref foot 8. Max speed—nonpref foot 9. Avg speed of max across both pref and nonpref foot 10. Avg speed of all four hop trials 11. Max speed of all four hop trials

Recorded outcomes included speed (running, hopping, throwing, and kicking) and distance (jumping). Throwing and kicking speed (in meters per second) were recorded across five skill trials using a Stalker radar gun (Stalker Radar; Palmeretal., 2021; Stodden et al., 2013, 2014; True et al., 2017). Standing long jump distance (in centimeters) on five jump trials also was assessed (Stodden et al., 2014; True et al., 2017). Running and hopping performance were video recorded, and speeds were calculated using video analysis software (30 frames per second; Dartfish Team Pro6; Palmer et al., 2021; True et al., 2017). Running speed (in meters per second) was calculated as the speed of two stride cycles during two run trials. Children were granted space for a 3-m acceleration and 3-m deceleration outside of the 7-m run. Hopping speed (in meters per second) was calculated as the speed to complete three consecutive hops (heel to heel) for two hop trials on each foot. If a child could not complete three consecutive hops, a zero was recorded and was used as the final score for that trial. Hopping speed was calculated on both the preferred and nonpreferred foot. Prior to all skill performances, children were provided identical digital demonstrations that included the same verbal prompts (Robinson, Palmer et al., 2015). For discrete skills (jump, throw, and kick), the first trial was recorded. For continuous skills (run and jump), children were allowed one practice trial. The decision to include a practice trial was based on the video analysis software used. The video software required children’s performance to be in a straight line, perpendicular to the camera. The practice trial ensured children were correctly executing skills for accurate analyses. There were short breaks between each skill (approximately 2 min) to allow researchers to set up the next skill, and children were prompted and granted short breaks (15–30 s) between skill trials if requested.

From the five skills, a total of 36 variations in skill performance were calculated. See Table 1 for a complete description of all product-oriented assessments and variations in skill performance.

Study Procedures

This study included secondary data analyses from the Promoting Activity and Developmental Trajectories of Health (PATH) study (NCT03189862). A full description of the PATH study can be found elsewhere (Robinson et al., 2020). The PATH study was a two-cohort randomized controlled trial designed to evaluate the immediate and sustained effects of a preschool motor skill intervention on children’s motor skills, physical activity, and perceived motor competence. Institutional Review Board (HUM00133319) approved all PATH study procedures and all enrolled participants provided by parental consent and personal assent before inclusion in the study. Children completed motor skill assessments in the fall (October–November) and spring (April–May) in preschool, kindergarten, and first grade. Data collection was interrupted due to the COVID-19 pandemic, so children in one cohort completed the first five time points of data collection, and children in the second cohort completed the first three time points of data collection. All children in PATH with a complete product-oriented assessment of at least one motor skill in at least one time point were included in this study.

Data Analyses

Intraclass correlation coefficients (ICCs) were calculated to determine the reliability among all performance metrics for each skill. ICCs were completed for an overall relationship (i.e., all time points combined) and at each time point individually. Using the decision tree proposed by Koo and Li (2016), ICCs were calculated as two-way mixed with absolute agreement. ICC values were interpreted as >.9, excellent reliability; .75–.90, good reliability; .5–.74, moderate reliability; and <.50, poor reliability (Koo & Li, 2016). We also used Pearson’s correlations to examine the relationship among performance metrics for each skill. Correlations were completed for an overall relationship (i.e., all time points combined) and at each time point individually.

Linear mixed models with random intercepts were fit to determine whether variations in how skill performance was assessed similarly evaluated developmental change across early childhood. In other words, these models were fit to determine how different variations of skill performance (e.g., max score vs. average scores) examined changes in skill over early childhood. Models were fit, controlling for treatment, sex, height, and weight. Linear mixed models were completed in SPSS (version 28), and alpha levels were set to .05 a priori.

Results

See Table 2 for participant demographics and Table 3 for full descriptive statistics.

Table 2.

Participant Demographics

	TMPT 1	TMPT 2	TMPT 3	TMPT 4	TMPT 5
n (% male)	284 (47%)	277 (47%)	183 (50%)	61 (48%)	66 (48%)
Age (months)	53.3 ± 3.7	59.5 ± 3.6	65.0 ± 3.6	71.6 ± 3.5	77.3 ± 3.0
Race/ethnicity (n)
African American	176	172	116	42	40
American Indian and Alaska Native	0	0	0	0	0
Arabic	7	6	4	2	3
Asian	1	1	1	0	0
Hispanic/Latinx	11	10	4	2	6
Native Hawaiian and Other Pacific Islander	0	0	0	0	0
White/Caucasian	37	36	24	8	9
Other	31	29	17	5	5
Did not report	21	23	17	2	3
Height (cm)	106.2 ± 4.8	109.8 ± 5.1	113.3 ± 5.2	116.3 ± 5.5	112.0 ± 5.1
Weight (kg)	18.8 ± 3.4	20.2 ± 3.8	21.5 ± 4.2	22.2 ± 3.3	24.6 ± 4.4

Note. TMPT = time point.

Table 3.

Mean and SDs for All Variations of Ball and Locomotor Skill Product-Oriented Measure Score Overall and at Each Time Point

	Overall		TMPT 1		TMPT 2		TMPT 3		TMPT 4		TMPT 5
	n	M ± SD	n	M ± SD	n	M ± SD	n	M ± SD	n	M ± SD	n	M ± SD
Throw (mph)
Max all five	871	9.74 ± 2.11	284	8.88 ± 1.94	277	9.64 ± 1.87	183	10.32 ± 2.09	61	10.61 ± 1.85	66	11.5 ± 2.21
Avg all five	871	8.24 ± 1.94	284	7.47 ± 1.63	277	8.13 ± 1.75	183	8.63 ± 2.03	61	9.29 ± 1.72	66	9.88 ± 2.25
Avg best three	871	9.01 ± 2.00	284	8.19 ± 1.74	277	8.9 ± 1.76	183	9.51 ± 2.04	61	10.01 ± 1.76	66	10.74 ± 2.20
Max first three	871	9.4 ± 2.12	284	8.62 ± 1.97	277	9.3 ± 1.85	183	9.84 ± 2.15	61	10.36 ± 1.99	66	11.11 ± 2.26
Avg first three	866	8.27 ± 2.00	283	7.52 ± 1.69	275	8.2 ± 1.81	182	8.67 ± 2.09	61	9.22 ± 1.88	65	9.88 ± 2.35
Max second and third trials	871	9.01 ± 2.18	284	8.16 ± 1.96	277	8.95 ± 2.01	183	9.44 ± 2.19	61	10.03 ± 1.94	66	10.78 ± 2.28
Avg second and third trials	871	8.25 ± 2.09	284	7.46 ± 1.76	277	8.21 ± 1.93	183	8.61 ± 2.19	61	9.31 ± 1.89	66	9.78 ± 2.43
Kick (mph)
Max all five	871	19.32 ± 4.12	284	7.81 ± 1.71	277	8.55 ± 1.60	183	9.22 ± 1.67	61	9.84 ± 1.55	66	10.6 ± 1.77
Avg all five	871	16.35 ± 3.86	284	6.52 ± 1.47	277	7.21 ± 1.58	183	7.86 ± 1.56	61	8.48 ± 1.60	66	9.11 ± 1.78
Avg best three	871	17.86 ± 3.92	284	7.15 ± 1.54	277	7.93 ± 1.56	183	8.53 ± 1.59	61	9.22 ± 1.52	66	9.87 ± 1.73
Max first three	871	18.73 ± 4.20	284	7.49 ± 1.73	277	8.34 ± 1.64	183	8.92 ± 1.74	61	9.60 ± 1.57	66	10.35 ± 1.76
Avg first three	869	16.45 ± 4.06	284	6.57 ± 1.58	276	7.30 ± 1.69	182	7.84 ± 1.64	61	8.54 ± 1.65	66	9.21 ± 1.87
Max second and third trials	871	17.82 ± 4.34	284	7.10 ± 1.73	277	7.93 ± 1.83	183	8.48 ± 1.70	61	9.18 ± 1.76	66	10.00 ± 1.80
Avg second and third trials	871	16.34 ± 4.23	284	6.47 ± 1.64	277	7.28 ± 1.80	183	7.81 ± 1.70	61	8.33 ± 1.91	66	9.24 ± 1.77
Jump
Max all five	875	90.44 ± 27.57	289	72.56 ± 24.30	277	91.28 ± 24.27	182	101.66 ± 22.63	61	108.7 ± 19.97	66	117.43 ± 22.01
Avg all five	875	76.02 ± 28.51	289	56.13 ± 23.30	277	77.48 ± 25.19	182	88.27 ± 23.59	61	95.77 ± 20.42	66	104.95 ± 22.24
Avg best three	875	83.50 ± 28.05	289	64.42 ± 24.28	277	84.82 ± 24.59	182	95.36 ± 22.65	61	102.51 ± 20.18	66	111.17 ± 21.42
Max first three	875	85.53 ± 28.31	289	66.86 ± 24.87	277	87.15 ± 25.65	182	96.99 ± 22.43	61	103.24 ± 20.26	66	112.56 ± 22.23
Avg first three	875	74.82 ± 29.29	289	54.92 ± 24.81	277	76.19 ± 26.38	182	87.30 ± 23.36	61	94.35 ± 20.49	66	103.73 ± 23.05
Max second and third trials	875	82.36 ± 29.15	289	63.41 ± 26.05	277	84.45 ± 25.96	182	93.98 ± 23.76	61	99.39 ± 21.59	66	108.71 ± 24.21
Avg second and third trials	875	75.45 ± 29.70	289	55.62 ± 25.62	277	77.82 ± 26.50	182	87.20 ± 24.41	61	94.53 ± 21.62	66	102.38 ± 25.32
Run
Trial 1	860	3.92 ± 0.63	288	3.65 ± 0.56	276	3.85 ± 0.59	170	4.18 ± 0.59	61	4.24 ± 0.55	65	4.45 ± 0.52
Trial 2	856	3.82 ± 0.79	288	3.54 ± 0.60	273	3.76 ± 0.61	172	4.05 ± 0.69	59	4.11 ± 0.58	64	4.42 ± 1.62
Avg	854	3.87 ± 0.65	288	3.60 ± 0.54	273	3.80 ± 0.56	170	4.12 ± 0.59	59	4.18 ± 0.53	64	4.44 ± 0.91
Max	862	4.04 ± 0.74	288	3.76 ± 0.57	276	3.95 ± 0.57	172	4.30 ± 0.58	61	4.32 ± 1.48	65	4.71 ± 1.48
Hop
Pref Foot Trial 1	859	1.2 ± 1.43	283	0.86 ± 0.78	275	1.10 ± 0.79	176	1.44 ± 0.73	61	1.99 ± 4.45	64	1.72 ± 0.75
Pref Foot Trial 2	856	1.02 ± 0.80	284	0.75 ± 0.76	273	0.95 ± 0.75	174	1.25 ± 0.79	61	1.33 ± 0.70	64	1.64 ± 0.68
Pref Foot Avg	860	1.11 ± 0.96	284	0.81 ± 0.72	275	1.03 ± 0.70	176	1.35 ± 0.68	61	1.66 ± 2.31	64	1.68 ± 0.64
Pref Foot Max	860	1.33 ± 1.41	284	0.97 ± 0.78	275	1.24 ± 0.75	176	1.56 ± 0.69	61	2.15 ± 4.41	64	1.89 ± 0.62
NonPref Foot Trial 1	851	0.96 ± 0.79	282	0.6 ± 0.70	273	1.02 ± 0.77	172	1.15 ± 0.78	60	1.21 ± 0.78	64	1.52 ± 0.71
NonPref Foot Trial 2	843	0.85 ± 0.79	281	0.55 ± 0.69	271	0.86 ± 0.75	168	1.05 ± 0.80	60	1.13 ± 0.75	63	1.40 ± 0.75
NonPref Foot Avg	851	0.91 ± 0.73	282	0.58 ± 0.64	273	0.94 ± 0.71	172	1.10 ± 0.72	60	1.17 ± 0.63	64	1.47 ± 0.68
Non Pref Foot Max	851	1.08 ± 0.79	282	0.71 ± 0.74	273	1.10 ± 0.76	172	1.30 ± 0.75	60	1.47 ± 0.63	64	1.66 ± 0.65
Max Avg	851	1.01 ± 0.72	282	0.69 ± 0.60	273	0.98 ± 0.60	172	1.23 ± 0.63	60	1.43 ± 1.26	64	1.58 ± 0.58
Overall Avg	850	1.00 ± 0.66	282	0.69 ± 0.60	273	0.98 ± 0.60	172	1.23 ± 0.63	59	1.29 ± 0.58	64	1.58 ± 0.58
Overall Max	860	1.43 ± 0.73	284	1.09 ± 0.76	275	1.45 ± 0.67	176	1.65 ± 0.64	60	1.68 ± 0.57	64	1.99 ± 0.52

Note. TMPT = time point.

ICC Among Performance Metrics of Each Skill

Ball Skills

ICCs showed excellent reliability among all performance metrics for the throw (all ICCs > .90) and kick (all ICCs > .90) (see Table 4).

Table 4.

ICCs Among Performance Metrics for Each Skill at Each Time Point

	Overall		TMPT 1		TMPT 2		TMPT 3		TMPT 4		TMPT 5
Throw	.975	(.950–.985)	.968	(.940–.981)	.971	(.941–.983)	.971	(.941–.983)	.977	(.947–.989)	.978	(.948–.989)
Kick	.974	(.950–.984)	.968	(.939–.980)	.967	(.938–.980)	.965	(.932–.980)	.961	(.921–.979)	.972	(.935–.986)
Jump	.988	(.976–.992)	.978	(.958–.987)	.985	(.971–.991)	.984	(.967–.991)	.981	(.960–.990)	.984	(.966–.991)
Run	.953	(.943–.962)	.965	(.948–.976)	.967	(.954–.975)	.958	(.938–.971)	.972	(.947–.984)	.872	(.812–.916)
Hop	.964	(.958–.970)	.962	(.952–.969)	.951	(.940–.961)	.96	(.947–.970)	.958	(.937–.973)	.955	(.934–.970)

Note. ICC = intraclass correlation coefficient; TMPT = time point. ICCs were fit as two-way mixed effects with absolute agreement.

Locomotor Skills

ICCs showed excellent reliability among all performance metrics for the jump (all ICCs > .90), run (all ICCs > .90), and hop (all ICCs > .90) (see Table 4).

Correlations Among Variations of Skill Performance

Ball Skills

Across all time points, there were significant and strong correlations among performance metrics throw (r_range = .98–.80) and kick (r_range = .97–.72) (see Table 5). See Supplementary Tables for full correlational ranges for all performance metrics of the throw (Supplementary Table S1 [available online]) and kick (Supplementary Table S2 [available online]).

Table 5.

Correlations Among Variations of Skill Performances for Ball and Locomotor Skills

	Overall	Range
	(All data combined)	(Across all five time points)
Throw	.98–.85	.98–.80
Kick	.97–.83	.97–.72
Jump	.99–.92	.99–.86
Run	.95–.67	.98–.25
Hop	.97–.26	.97–.13

Locomotor Skills

There were significant and strong correlations among all performance metrics for the jump overall and across all time points (r_range = .99–.86; see Table 5). For the run, there were significant and moderate-to-strong correlations overall (r_range = .95–.67) and significant and weak-to-strong correlations across all time points (r_range = .98–.25) (see Table 5). Performance metrics for the hop ranged in strength from strong to weak overall (r_range = .97–.26) and across all time points (r_range = .97–.26) (see Table 5). Some hop performance metrics were not significantly correlated.

See Supplementary Tables for full correlational ranges for all performance metrics of the jump (Supplementary Table S3 [available online]), run (Supplementary Table S4 [available online]), and hop (Supplementary Table S5 [available online]).

Linear Mixed Models

Ball Skills

For the throw and the kick, all linear mixed models demonstrated a positive effect of age across all performance metrics (p’s < .05; see Table 6). All models also revealed positive effects of height (p < .05) and sex, whereby boys perform better than girls (p < .05) (see Table 6).

Table 6.

Results of the Linear Mixed Models for All Skills

	Intercept			Treatment			Sex			Age			Height			Weight
	B	SE	p	B	SE	p	B	SE	p	B	SE	p	B	SE	p	B	SE	p
Throw
Max all five	−2.39	1.65	.15	−0.08	0.18	.67	−1.08	0.18	<.001	0.05	0.01	<.001	0.08	0.02	<.001	0.01	0.03	.68
Avg all five	−1.13	1.55	.47	−0.03	0.16	.84	−0.82	0.16	<.001	0.06	0.01	<.001	0.05	0.02	.01	0.01	0.03	.68
Avg best three	−1.41	1.55	.36	−0.05	0.17	.76	−0.97	0.17	<.001	0.06	0.01	<.001	0.06	0.02	<.001	0.02	0.03	.41
Max first three	−1.74	1.68	.30	−0.07	0.18	.68	−1.07	0.18	<.001	0.05	0.01	<.001	0.08	0.02	<.001	0.01	0.03	.72
Avg first three	−0.82	1.59	.61	−0.04	0.16	.79	−0.79	0.16	<.001	0.06	0.01	<.001	0.05	0.02	.01	0.01	0.03	.61
Max second and third trials	−0.94	1.76	.59	−0.13	0.18	.46	−1.02	0.18	<.001	0.06	0.01	<.001	0.06	0.02	.02	0.02	0.03	.45
Avg second and third trials	−0.70	1.68	.68	−0.15	0.17	.38	−0.79	0.17	<.001	0.06	0.01	<.001	0.05	0.02	.02	0.02	0.03	.60
Kick
Max all five	−4.11	1.34	<.001	0.05	0.14	.73	−0.99	0.14	<.001	0.07	0.01	<.001	0.08	0.02	<.001	0.04	0.02	.09
Avg all five	−4.68	1.26	<.001	−0.04	0.13	.74	−0.92	0.13	<.001	0.06	0.01	<.001	0.07	0.02	<.001	0.04	0.02	.07
Avg best three	−4.63	1.26	<.001	−0.02	0.13	.90	−0.96	0.13	<.001	0.07	0.01	<.001	0.08	0.02	<.001	0.04	0.02	.08
Max first three	−5.36	1.39	<.001	0.05	0.15	.72	−0.94	0.15	<.001	0.07	0.01	<.001	0.09	0.02	<.001	0.03	0.02	.25
Avg first three	−5.07	1.36	<.001	−0.03	0.14	.83	−0.91	0.14	<.001	0.06	0.01	<.001	0.08	0.02	<.001	0.03	0.02	.24
Max second and third trials	−5.51	1.45	<.001	0.04	0.15	.80	−0.98	0.15	<.001	0.06	0.01	<.001	0.09	0.02	<.001	0.03	0.02	.17
Avg second and third trials	−4.85	1.42	<.001	−0.03	0.14	.84	−0.96	0.14	<.001	0.06	0.01	<.001	0.07	0.02	<.001	0.04	0.02	.10
Jump
Max all five	−142.40	21.08	<.001	−6.27	2.26	.01	−4.63	2.28	.04	1.30	0.15	<.001	1.68	0.28	<.001	−1.28	0.35	<.001
Avg all five	−162.02	21.64	<.001	−6.10	2.30	.01	−3.20	2.31	.17	1.52	0.16	<.001	1.57	0.29	<.001	−1.13	0.36	<.001
Avg best three	−152.85	21.30	<.001	−6.19	2.30	.01	−3.95	2.31	.09	1.43	0.15	<.001	1.63	0.28	<.001	−1.23	0.35	<.001
Max first three	−139.63	21.88	<.001	−6.56	2.33	.01	−3.49	2.34	.14	1.39	0.16	<.001	1.53	0.29	<.001	−1.17	0.36	<.001
Avg first three	−158.46	22.46	<.001	−6.79	2.34	<.001	−2.76	2.35	.24	1.53	0.16	<.001	1.51	0.30	<.001	−1.07	0.37	<.001
Max second and third trials	−147.27	22.81	<.001	−6.43	2.37	.01	−3.06	2.38	.20	1.37	0.17	<.001	1.59	0.30	<.001	−1.17	0.38	<.001
Avg second and third trials	−149.97	23.17	<.001	−6.88	2.36	<.001	−2.65	2.37	.26	1.50	0.17	<.001	1.44	0.31	<.001	−0.96	0.38	.01
Run
Trial 1	−0.42	0.51	.41	−0.03	0.05	.53	−0.06	0.05	.25	0.02	<0.001	<.001	0.03	0.01	<.001	−0.02	0.01	.01
Trial 2	−0.21	0.63	.74	−0.08	0.06	.15	−0.04	0.06	.51	0.03	0.01	<.001	0.03	0.01	<.001	−0.02	0.01	.06
Avg	−0.34	0.54	.52	−0.05	0.05	.35	−0.05	0.05	.33	0.03	<0.001	<.001	0.03	0.01	<.001	−0.02	0.01	.01
Max	−1.62	0.59	.78	−0.03	0.54	.57	−0.07	0.06	.16	0.27	0.01	<.001	0.03	0.08	<.001	−0.2	0.01	.07
Hop
Pref Foot Trial 1	−4.80	1.40	<.001	−0.04	0.11	.72	0.09	0.11	.44	0.03	0.01	.07	0.05	0.02	<.001	−0.04	0.02	.04
Pref Foot Trial 2	−3.83	0.69	<.001	−0.09	0.07	.20	0.13	0.07	.08	0.02	<0.001	<.001	0.04	0.01	<.001	−0.03	0.01	.01
Pref Foot Avg	−3.97	0.78	<.001	−0.05	0.07	.52	0.13	0.07	.07	0.03	0.01	<.001	0.04	0.01	<.001	−0.03	0.01	.01
Pref Foot Max	−5.01	1.35	<.001	−0.07	0.11	.54	0.10	0.11	.37	0.02	0.01	.09	0.05	0.02	<.001	−0.04	0.02	.03
NonPref Foot Trial 1	−3.18	0.69	<.001	−0.06	0.07	.41	0.03	0.07	.66	0.03	<0.001	<.001	0.03	0.01	<.001	−0.04	0.01	<.001
NonPref Foot Trial 2	−3.17	0.69	<.001	−0.05	0.07	.48	0.06	0.07	.41	0.03	0.01	<.001	0.03	0.01	<.001	−0.03	0.01	<.001
NonPref Foot Avg	−3.19	0.63	<.001	−0.06	0.07	.41	0.04	0.07	.54	0.03	<0.001	<.001	0.03	0.01	<.001	−0.04	0.01	<.001
NonPref Foot Max	−3.53	0.67	<.001	−0.06	0.07	.44	0.03	0.07	.65	0.03	0.005	<.001	0.03	0.01	<.001	−0.05	0.01	<.001
Max Avg	−3.50	0.61	<.001	−0.05	0.06	.44	0.09	0.06	.16	0.03	<0.001	<.001	0.03	0.01	<.001	−0.04	0.01	<.001
Overall Avg	−3.38	0.53	<.001	−0.05	0.06	.41	0.08	0.06	.18	0.03	0.003	<.001	0.03	0.01	<.001	−0.04	0.01	<.001
Overall Max	−3.47	0.59	<.001	−0.03	0.07	.7	0.06	0.07	.38	0.03	0.004	<.001	0.04	0.01	<.001	−0.05	0.01	<.001

Locomotor Skills

For the jump and run, all linear mixed models found a positive effect of age across all performance metrics (p < .05; see Table 6). There were positive effects of age for hop performance metrics except for the preferred foot Trial 1 (p = .07) and the preferred foot max (p = .09) (see Table 6). All models also revealed a positive effect of height across all hop performance metrics (p < .05) and an inverse effect of weight (p < .05) except the run Trial 2 (p = .06). The only performance metric with a significant sex effect was the max of the five jump trials where boys outperformed girls (p < .05).

Discussion

There is a limited consensus on what performance metrics should be used from different strategies (i.e., maximum and average scores) and different trial numbers in product-oriented assessments of motor skills in young children. This study examined the associations among different performance metrics derived from different strategies (i.e., maximum and average scores) and trials (e.g., three trials vs. five trials) from product-oriented measures of throw, kick, jump, run, and hop in 3- to 7-year-old children. Results support strong reliability among all various performance metrics for a single skill (all ICC > .90 except run at Time Point 5, ICC = .87). However, when we examined the relationships between pairs of performance metrics, results demonstrated a pattern of differences between associations among performance metrics for discrete versus continuous skills.

The performance metrics from discrete motor skills (throwing, kicking, and jumping) consistently showed strong correlations across different strategies (i.e., maximal and average) and trials and over the five time points tested. Relationships among performance metrics from continuous skills (running and hopping) were generally not as strong and more variable across time points. These findings indicate metrics for evaluating product-oriented performance for discrete skills in young children may be tailored to specific research goals (e.g., using the maximum trial performance or using an average of multiple trials) without compromising comparability with other study results. In contrast, further consideration must be made to compare performance metrics of continuous skills such as running and hopping as these metrics were more variable both within trials during one data collection time point and across multiple time points from 3 to 7 years of age.

Product-Oriented Performance Metrics of Discrete Skills

In throwing (r_range = .98–.80), kicking (r_range = .97–.72), and jumping (r_range = .99–.86), strong correlations were observed for all variations of product performances at each time point, suggesting intertrial performance variability was limited at the individual level. The average of five trials, the average of the top three scores, and the maximum score of five trials consistently demonstrated the strongest correlations in all of the potential combinations of performance scoring variations (throwing r_range = .97–.91; kicking r_range = .97–.91; jumping r_range = .99–.95). Of the scoring derivatives using three or fewer trials (i.e., Max 3, Avg 3, Max 2nd and 3rd trials, and Avg 2nd and 3rd trials), maximum and average product measures using three trials were more similar and consistently demonstrated stronger correlations compared to two trial variations (i.e., second and third trials only). While these data are specific to product-oriented measurement, results support previous research that has assessed skill levels using process-oriented component developmental sequences (Roberton, 1977; Stodden et al., 2006a, 2006b), indicating the modal performance across 5–10 trials demonstrates the “most prevalent” performance across trials. Thus, as the maximum performance across five trials and different performance metrics derived from three or five trials in the discrete skills tested in this study demonstrate very strong correlations, an “average” score of multiple trials strongly aligns with maximum performance capability.

Of additional importance is the fact that, even in this early childhood sample, where skill levels are low (compared to more highly skilled performers generally tested at older ages), their performance in discrete skills seemed to be quite consistent, as evidenced by the high correlations between variations of performance metrics derived from different trials. This finding contradicts the widely accepted notion that less advanced performers with generally less experience in performing skills, specifically in this age range, are highly variable (Fitts & Posner, 1967). Our analysis of multiple subgroupings of trial combinations speaks to the relative consistency in performance across five trials.

Overall, these findings demonstrate that when assessing discrete skill performance in early childhood (i.e., 3–7 years) from a product-oriented perspective only, assessing three trials is sufficient to effectively and reliably capture their maximum performance capability at one specific time point. Using the maximum score or the average of at least three trials provides a similar estimate of performance; thus, using either score is warranted. However, as literature (Logan et al., 2017) has indicated, assessing motor performance using both process- and product-oriented measures provides a better overall estimation of motor competence, aligning the product outcomes of the trials that are associated with the trials that are used for a particular process-oriented assessment(e.g., TGMDand Get Skilled Get Active—bothusetwo trials) would be a more appropriate measurement protocol. As the ICCs were >.90 and the range of correlations between all performance metrics for discrete skills were generally above .80, the use of product scores that are aligned with the specific process-oriented trials demonstrates acceptable levels of product-oriented performance that represent maximum performance capabilities. Finally, as the ICCs and correlations among the performance metrics discrete skillswere consistent across the five time points, the validity of testing product-oriented performance outcomes in early childhood is supported.

Product-Oriented Performance Metrics of Continuous Skills

Data among performance metrics from the two continuous skills (running and hopping) had similar overall reliability (ICC > .87) but demonstrated lower and more variable correlations compared to discrete skill correlations. In general, higher speeds were achieved in the earlier trials (i.e., Trial 1) for every performance metric of continuous locomotor tasks (run, hop on dominant foot, and hop on nondominant foot; Table 3). It is likely that performances across trials were affected by two factors. The first and most likely factor responsible for the highly variable correlation ranges for hopping is related to how hopping was scored. If children could not demonstrate four consecutive hops, they were given a score of 0 for that trial. A 0 score dramatically impacted average hopping scores for many children, as 29% demonstrated at least one 0 score in their trials of hopping. As product-oriented scoring methods are generally different from dichotomous (e.g., TGMD, Get Skilled Get Active, or Children’s Activity and Movement in Preschool Study [CHAMPS]) or ordinal (Component Developmental Sequences) scales where multiple components are summed, not being able to demonstrate a successful outcome of a skill (i.e., a 0 score) is an important measurement criterion. Knowing that 29% of children in this sample scored a 0 on at least one hop trial demonstrates a specific need to intervene, so children are able to successfully participate in activities where hopping (or any other skill for that matter) is inherently integrated in an activity (i.e., hopscotch and dance).

The second factor that may have impacted performances across trials was performance fatigue, specifically in this age range. These data suggest that when assessing performance on continuous skills using only product-oriented assessments (e.g., Körperkoordinationstest Für Kinder, Motor Competence Assessment, Bruininks-Oseretsky Test of Motor Proficiency, and individual skill outcomes), the maximum performance trial would be the best scoring method if the goal of the assessment was to capture an individual’s maximum performance capability and minimize a potential fatigue effect. Acknowledging the relatively low relationship (r_range = .67–.27; Table 5; Supplementary Table S5 [available online]) between performance metrics derived from hopping on the dominant foot and those from hopping on the nondominant foot when compared to intertrial relationships within other skills. These data suggest that hopping performance on the dominant foot is not consistently generalizable to performance on the nondominant foot. Thus, it may be useful for both dominant and nondominant performances to be considered when calculating measures for hopping or other unilateral locomotor skills. Further, since the two nondominant hopping trials were performed immediately after three hopping trials on the dominant foot (one practice trial and two test trials), we cannot discount the effect of accumulative fatigue on the nondominant hopping performance in this sample.

How Performance Metrics Examine Skill Changes Over Early Childhood

Additionally, this study explored whether different performance metrics from product-oriented measures of motor skills are equally sensitive to changes in performance in typical development. As evidenced by the linear mixed models, all skill performance metrics, except the two variations of the hop (preferred foot), reflected improvement from age 3 to 7 years controlling for other individual factors such as sex, height, and weight. Understanding how different performance metrics relate to changes across developmental time is important as process-oriented assessments of individual skills may be more prone to ceiling effects as these assessments are often designed to cap out at certain ages (e.g., TGMD). Process-oriented assessments also are less sensitive to demonstrating changes in individual skill levels compared with product-oriented assessments based on the type of measurement (i.e., dichotomous or ordinal vs. continuous). These results support that different performance metrics are sensitive to changes in skills across early childhood; and therefore, product-oriented assessments may be practical for tracking changes in motor competence across developmental time and may be a good way to circumvent ceiling effects, inherently or strategically, present in other measurement orientations or approaches.

Strengths, Limitations, Conclusions, and Recommendations

The current study was limited in that it only examined performance in young children (3–7 years); thus, the findings may not be generalizable to older children, adolescents, and adults. While the consistency among performance metrics discrete skills would also be expected in older populations, a better understanding of product-oriented performance metrics of continuous skills is needed with other age groups. Additionally, performance metrics for each of the continuous locomotor tasks in this study were calculated using results from only two trials, thus making comparisons to the five trial data with discrete skills difficult to compare. However, we would not promote the use of additional trials for continuous skills due to the potential impact of fatigue that may have occurred with only two scored trials (three trials if including the practice trial; discussed in more detail below).

The results of this study support testing with only three trials as an effective and reliable method for assessing maximal effort discrete skills in young children. Assessing only three trials enhances testing feasibility in large samples of children where time constraints of testing are common (e.g., in schools with limited allocated physical education time). Although correlations between the maximum performance metrics and the average of three trials of a skill were high (r_range = .96–.87; Table 5) for the entire sample, using the maximal performance (out of three trials) is recommended when using assessments that only examine product scores only as it can control for the potential impact of random “outlier” trial performances that may significantly impact an average score of three trials. In contrast, when examining both product- and process-oriented performance in the same skills, we recommend aligning product scoring with the trials scored for process assessment.

When evaluating product-oriented performance metrics of continuous locomotor skills, such as hopping and running, the relatively high intertrial variability suggests that more than two performance trials may be warranted to produce more reliable estimates of performance in young children. However, the incorporation of more test trials would decrease the feasibility of assessing these skills and potentially induce fatigue that would negatively influence performance. We suggest the use of maximal performance outcomes with two trials as it would most adequately represent the performance capabilities of young children. Further, the use of maximal product measures would control for the potential of fatigue to act as a confounding variable on performance. To account for differences in performance on dominant and nondominant legs, we recommend the use of an average of maximal performance on each leg. One caveat to these results is that children were given one practice trial for continuous skills that was not scored. As stated, this decision was based on similar approaches for the measurement of these skills (e.g., TGMD) but also based on the need of straight trajectories for the video analyses. We do not know whether performance was meaningfully different during these practice trials as compared to the two test trials. Future research should continue to explore the effects of fatigue on performance metrics of continuous skills.

In addition, this is the first study to demonstrate a consistently significant effect of height on all performance metrics of locomotor skill product measures in this age range that was above and beyond any impact of sex, age, and weight. We recommend controlling height when assessing performance levels on locomotor skills in young children. Further, we found that on ball skills, throw and kick, boys outperformed girls. This finding is consistent with other motor skill measures in this age (Ulrich, 2019) and further supports that sex should be controlled for in motor skill research in early childhood, especially ball skills.

Overall, the results of this study provide novel comparative data on different performance metrics derived from different strategies (i.e., maximum and average scores) and different trial numbers in assessing product outcomes of discrete and locomotor skills in early childhood (Figure 1). As research on the link between motor competence levels and other important developmental domains (e.g., physical, cognitive, self-concept, and social–emotional health) continues to emerge, it is important to continuously improve the measurement of motor development to enhance the validity and reliability of data.

Figure 1 —

Final conclusions and recommendations.