Concurrent Validity of the TIMP and the Bayley III Scales at 6 Weeks Corrected Age

Abstract

Purpose

Examine agreement between the Test of Infant Motor Performance (TIMP) and the Bayley III.

Methods

One-hundred, forty-five infants born at 29-34 weeks gestation with socioenvironmental risk factors were tested on the TIMP and Bayley III at 6 weeks corrected age (CA). Scores were correlated to assess convergence/divergence of content. Decision analysis using a cutoff of the mean on the Bayley Motor Composite and -.5 and -1 SD from the mean on the TIMP assessed agreement on delay/non-delay.

Results

The TIMP-Bayley Motor Composite correlation was .546, with Cognitive was .310, and with Language was .281. 9% of infants scored below −1.0 SD on the TIMP while no child performed below -1 SD on the Bayley Motor scale (sensitivity 31%).

Conclusions

Convergent validity between the TIMP and the Bayley Motor scale was demonstrated, but no infant showed delay on any Bayley scale. The TIMP is preferred for early assessment of infants.

INTRODUCTION

Accurate assessment of the motor performance of infants born prematurely or with other perinatal complications requires use of tests with validity for identifying delay or neurologic impairment. Documentation of the construct validity of a new assessment requires research demonstrating its relationships with other existing tests intended for similar purposes. An exploration of external validity examines the congruence or divergence of the constructs underlying new and criterion tests. Of greater interest to practitioners who use these tools to diagnose delay clinically is an examination of the various cut scores for each tool and the differences in referral decisions based on having selected one tool or the other at a particular age.

The Test of Infant Motor Performance was published in 2001 for use in Neonatal Intensive Care Units (NICUs) and Early Intervention (EI) programs serving infants born prematurely and other infants at risk for developmental delay.¹ The TIMP is a test of functional motor skills with age standards for performance of infants between the ages of 34 weeks postmenstrual age (PMA) and 17 weeks corrected age (CA) based on a normative sample of 990 U.S. infants with a range of medical risk for developmental delay.² Previous research demonstrated convergent validity (r = .66) at 3 months CA with the Alberta Infant Motor Scale,^3,4 and validity at 3 months CA for predicting 1) AIMS scores at 12 months CA⁵ and 2) Peabody Developmental Motor Scales motor quotients at 4-5 years of age.⁶ In a Korean sample TIMP scores from tests performed at term-equivalent age were highly predictive of scores on the Bayley II at 6 months CA,^7,8 but the TIMP has not been compared to the Bayley III.

The Bayley Scales of Infant Development have long been standard tests for measuring motor and cognitive performance of infants.⁹ The 3^rd edition of the test, the Bayley III,¹⁰ was published in 2006 with new age standards for cognitive, language, and motor development. Separate scales for gross and fine motor performance were developed. Evidence on performance of the Bayley III in clinical situations is accumulating but presents a confusing picture of how this new version of the scales performs when compared with the Bayley II and other tests. On the one hand, Green and colleagues reported clinical results consistent with expected levels of delay at 8-12 months CA in a group of 85 infants born premature.¹¹ The average score on the motor composite was 94 (SD 17); 22% of the infants had motor composite scores < 85 (1 SD below the normative mean of 100), and 12% had significantly delayed gross motor subscale scores (< 4). On the other hand, Vohr and colleagues found large discrepancies in outcomes of a sample of more than 2600 infants with extremely low birth weight (ELBW) when results from Bayley II assessments at 18-22 months in 2006-2007 were compared with results using the Bayley III on a cohort tested in 2008-2011.¹² The neurodevelopmental impairment rate was reduced from 43% in the earlier period to 13% in 2008-2011. Anderson and colleagues also reported overestimation of performance in infants born full term.¹³

The purpose of this study is to examine the convergent validity of the TIMP and the Bayley III motor scores and the divergent validity of the TIMP and the Bayley III cognitive and language scores at 6 weeks CA in a sample of infants born preterm with dual risk for poor developmental outcome because of social-environmental home conditions. In addition to exploring the correlations among these assessments, we examine the differences in clinical decisions made using threshold cut scores for identification of delayed motor development. Both tests were standardized during approximately the same time period, but given the slightly earlier time for the TIMP (2002-2004)¹ and its more extensive research documenting predictive validity, we use the TIMP as the criterion measure.

METHODS

This report is derived from a larger randomized clinical trial designed to evaluate outcomes of a developmental intervention for infants born preterm. After baseline data collection, 198 infants born at 29-34 weeks gestational age (GA), with at least 2 social-environmental risk factors, were randomly assigned to an attention-only control group or the Hospital-Home Transition: Optimizing Prematures’ Environment (H-HOPE) intervention.^14,15 The neurodevelopment of the infants was evaluated at 6 weeks CA via both the TIMP and the Bayley III. No group differences in infant development at 6 weeks CA were identified as a result of the intervention so, for the purpose of this report, we used the entire sample to examine the convergent/divergent validity of the TIMP Z scores and the Bayley III Cognitive, Motor, and Language Composite scores at 6 weeks CA.

Setting

The research was conducted in 2 inner-city Midwestern community hospital NICUs (1 a level II intermediate care unit and the second a level III unit serving infants with special health care needs such as ventilatory support).

Sample

Infants met eligibility criteria if they were born between 29 and 34 weeks GA and had no other major health problems. Infants may have previously received ventilator support or other medical therapies for maintenance (e.g., IV therapy or oxygen therapy via nasal cannula). Their mothers met eligibility criteria if they had at least 2 social-environmental risk factors such as minority status, less than high school education, less than 18 years of age, history of current mental illness (e.g., depression), family income less than 185% of the federal poverty guidelines, more than 1 infant under 24 months, 4 or more infants under 4 years of age in the home, or residing in a disadvantaged neighborhood.

Infant exclusion criteria included the presence of congenital anomalies, necrotizing enterocolitis, brain injury, chronic lung disease, history of prenatal illicit drug exposure or positive toxicology screen. Infants were excluded if their mothers were illicit drug users, not the legal guardian of the infant, or HIV positive. Sample characteristics were obtained via medical record review, with the exception of maternal race which was self-reported by the mother at baseline.

Of the 198 mother-infant dyads enrolled in the intervention study, 149 (75.3%) were retained for the 6-week CA visit, but 3 were not assessed for infant development because the tester was not available at the time of the visit. One additional infant completed the TIMP but not the Bayley III assessment as a result of fatigue. Therefore, 145 infants have scores for the TIMP and the 3 Bayley III scales and are included in this study. Table 1 summarizes the characteristics of the subjects. The average GA of the infants at birth was 32.4 weeks (SD = 1.6). The infants had a moderate degree of medical issues (mean = 70.9) as documented with the Problem-Oriented Perinatal Risk Assessment System (POPRAS) which is used to assign points to a variety of medical conditions.^16,17 Higher scores denote the presence of more medical complications and increased risk for developmental morbidity. At the 6-week CA follow up visit, the mean chronologic age was 13.4 weeks (SD = 1.9).

Table 1.

Infant Characteristics (n = 145)

	Mean or %	SD	Minimum	Maximum
Male/Female Sex (%)	52 / 48
Latina/African-American Maternal Race/Ethnicity (%)	50/50
Multiple/Singleton Birth (%)	10 / 90
Gestational Age	32.4	1.6	29	34
Birth Weight	1834	409	1000	3146
Apgar Score 5 min	8.3	1.09	4	9
Inf Morbidity Score (POPRAS)	70.9	19.8	36	136
Chronological age at 6 week CA follow-up visit, weeks	13.4	1.9	10	18

Procedures

The research was approved by the Institutional Review Boards from the university and the 2 clinical sites. After informed consent was obtained, mothers and infants were randomly assigned to the H-HOPE or the attention-only control group.¹⁵ When the infant reached 32 weeks PMA, the intervention began in the hospital following an initial oral feeding assessment. Infant development was assessed at 6 weeks CA via the TIMP and the Bayley III. For the 6-week assessments, mothers brought their infants to an examination room in the university's College of Nursing. In most cases the sessions proceeded according to the following schedule: Infants were evaluated first with the TIMP followed by the Bayley. Because this was part of a larger study which also evaluated mother-infant feeding interactions,¹⁵ the mothers were instructed to let the researchers know when they believed that their infants were showing feeding readiness cues. In such instances, the therapist stopped the developmental assessment and resumed after the infants had completed their feedings. The majority of the time (75%) the TIMP and Bayley III were completed in their entirety and in that order before the infants were ready to feed.

Tests

The TIMP is a 42-item assessment of functional gross motor performance.¹ Item responses are related to demands for movement placed on infants in daily life interactions with caregivers,¹⁸ and Rasch psychometric analysis revealed that the items reliably separate infants into 5-6 uniquely different levels of development.¹⁹ All 42 items are administered to infants at 6 weeks CA and raw scores are compared to age norms in 2-week increments. Z scores, percentile ranks and age-equivalent scores can also be obtained.

The Bayley III is used to assess the developmental functioning of infants and toddlers 1 to 42 months of age.²⁰ The Bayley III consists of 3 administered scales: Cognitive, Language (including receptive and expressive communication subtests), and Motor (including fine and gross motor subtests). The cognitive scale is used to assess sensory-perceptual acuities, discriminations, and the ability to respond to these as well as the early acquisition of object constancy and memory, learning, and problem-solving ability. The language scale is used to evaluate receptive language capabilities, expressive vocalizations and the beginnings of verbal communication. The motor scale provides a means to assess postural control, coordination of the large muscles and finer manipulation skills of the hands and fingers; results can be reported as separate gross and fine motor scores. Scores are presented as raw scores, scaled scores, composite scores, centile ranks, age equivalents, and growth scores.^10,21

Data Analysis

For this study, we assessed inter-rater reliability for a random 25% of the TIMP and Bayley assessments from a video recording of the original administration of the tests. A physical therapist who is an expert in administering the TIMP re-rated the TIMP, and an expert in administering the Bayley test re-rated the Cognitive, Language and Motor scales of that test. Both were unaware of infant group assessment and scores obtained by the study testers. Inter-rater reliability was determined using the intraclass correlation coefficient (ICC).²²

Means, standard deviations and proportions were used to describe the sample characteristics, and the TIMP Z scores and Bayley III Composite scores for cognitive, language and motor development were calculated for test comparisons. Pearson product moment correlation coefficients were calculated for the relationship between TIMP and Bayley III Composite scores, and cross-tabulations were generated to contrast the number of infants with indication of delay in motor development according to the TIMP with the number of infants falling below the mean of the Bayley III scores.

To formally compare the difference in clinical decision making regarding motor developmental outcome that would occur with TIMP Z score cutoffs of −0.5, −1.0 and −1.5 SD below the mean and a cutoff at the mean (<100) for the Bayley Motor Composite score, sensitivity, specificity, positive predictive validity (PPV) and negative predictive validity (NPV) were calculated using the TIMP as the criterion measure or reference standard. Sensitivity was calculated as the proportion of infants with scores below the cutoff on the TIMP Z score that also scored below the cutoff on the Bayley III Motor Composite score. Specificity was calculated as the proportion of infants with scores at or above the TIMP cutoff who also had Bayley III Motor Composite scores above the cutoff. The PPV was calculated as the proportion of infants with a score below the cutoff on the Bayley Motor Composite who also had a score below the cutoff on the TIMP Z score, and the NPV was the proportion of infants with a score at or above the cutoff on the Bayley Motor Composite who also had a TIMP Z score at or above the cutoff.

RESULTS

Rater Reliability

Inter-rater reliability for the TIMP resulted in an ICC of 0.79 (95% CI = 0.60-0.90); for the Bayley Cognitive scale the ICC was 0.73(0.46, 0.86); for the Bayley Language scale the ICC was 0.75 (0.51, 0.87) and for the Bayley Motor scale the ICC was 0.75 (0.46, 0.88).

Test Scores

Table 2 presents the means, standard deviations, and ranges of raw score performance on the TIMP and Bayley III Cognitive, Language and Motor scales at 6 weeks CA, and Table 3 describes the outcomes in standard score terms (Z score for the TIMP and composite scores for the Bayley III). Sixty of the 145 infants (41.4%) scored below the recommended cutoff for delay of a Z score of −0.5 SD on the TIMP.¹ Scores on the Bayley scales were much higher with the means of all composites greater than 100. Ten infants scored below the mean on the cognitive scale (6.9%), 15 (10.3%) on the language scale, and 5 (3.4%) on the motor scale. No infant scored more than 1 SD below the mean on any Bayley III scale. As a result no infants were identified as delayed on the Bayley III scales using the typical cutoff for suspicious performance.

Table 2.

Descriptive Statistics for Raw Scores for TIMP and Bayley III Subscales at 6 Weeks Corrected Age (N = 145)

	Mean	SD	Minimum	Maximum
TIMP	78.7	9.1	56	104
Bayley Cognitive	9.52	2.15	5	24
Bayley Language
Receptive Communication	5.84	2.47	4	33
Expressive Communication	4.66	1.11	2	8
Bayley Motor
Fine Motor	6.27	1.26	3	9
Gross Motor	10.35	2.61	4	16

Table 3.

Descriptive Statistics for Performance on the TIMP and Bayley at 6 Weeks Corrected Age (N=145)

	Mean	Standard Deviation	Minimum	Maximum
TIMP Z Score	−0.33	.55	−1.60	1.12
Bayley Cognitive	109	8	85	135
Bayley Language	108	8	94	129
Bayley Motor	116	8	94	136

Correlations Among Tests

Table 4 shows the correlations among the TIMP Z scores at 6 weeks and the Bayley III Cognitive, Language, and Motor Scale Composite scores. As expected, the correlation between the TIMP and the Bayley Motor Composite was higher than those between the TIMP and the Bayley Cognitive Composite or Language Composite.

Table 4.

Correlations Among TIMP and Bayley Standard Scores at 6 Weeks Corrected Age (N = 145)

	Bayley Cognitive Composite	Bayley Language Composite	Bayley Motor Composite
TIMP Z score
Pearson correlation	.310	.281	.546
Significance	<.0001	.001	<.0001
Bayley Cognitive Composite
Pearson correlation		.236	.303
Significance		.004	<.0001
Bayley Language Composite
Pearson correlation			.245
Significance			.003

Clinical Decision Comparison

Table 5 presents a 4-fold table comparing the Bayley Motor Composite results split at the mean of the distribution to the TIMP Z score results using a cutoff of −0.5 SD to designate delay. The mean of the Bayley Motor Composite was used because only 5 infants scored below the mean. The sensitivity of the Bayley for agreement with the TIMP in identifying delay is negligible at 8.3% while the specificity is 100% because all infants above the cutoff on the TIMP scored above the mean on the Bayley Motor Composite (Table 6). The PPV is also 100% because the 5 infants scoring below the mean on the Bayley Motor Composite all had delayed scores on the TIMP. The NPV, on the other hand, was poor at 61% because the vast majority of infants tested above average on the Bayley. The overall agreement of the Bayley in reflecting results on the TIMP was 62.1%.

Table 5.

Comparison of Clinical Decisions for TIMP Cutoff of −0.5 SD and Bayley Motor Composite Cutoff of the Mean

Test	TIMP <−0.5 SD	TIMP ≥−0.5 SD	Totals
Bayley < 100	5	0	5
Bayley ≥ 100	55	85	140
Totals	60	85	145

Table 6.

Comparison of Clinical Decisions for TIMP Cutoffs of −0.5, −1.0 and −1.5 SD and Bayley Motor Composite Cutoff at the Mean

Cutoff on TIMP	Sensitivity	Specificity	Positive Predictive Validity	Negative Predictive Validity
<−0.5	.08	1.00	1.00	.61
<−1.0	.31	0.99	0.80	.94
<−1.5	.00	0.97	0.00	.99

Because the cutoff recommended for identification of delay on the TIMP was derived from data on concurrent and predictive validity for TIMP assessments at 3 months CA, the lower cutoff of −1.5 SD recently published by Korean researchers⁸ for comparing TIMP results at term age with Bayley II results at 6 months CA were next examined along with an intermediate cutoff of −1.0 SD. Table 6 shows the results. A cutoff of the mean on the Bayley Motor Composite best matches results using a cutoff on the TIMP Z score of -1.0 SD with agreement between decisions at 93%, but the sensitivity remains poor at 31%, i.e., only 4 of the 13 infants (31%) identified as delayed on the TIMP have scores below the mean on the Bayley Motor Composite. One infant scoring below the mean on the Bayley achieved a score on the TIMP > 1.0 SD, and 8.96% of the infants were identified as delayed by the TIMP.

In summary, Bayley Motor Composite scores were significantly correlated with TIMP Z scores while Cognitive and Language Composite scores were less strongly related to TIMP performance. No infant was identified as scoring below average on any of the Bayley III scales while the TIMP identified 41% of infants as delayed using a cutoff of -0.5 SD or 9% of infants as delayed using a cutoff of a Z score of −1.0 SD.

DISCUSSION

A comparison of the correlations among the various Bayley scales and the TIMP demonstrated convergence between the Bayley Motor Composite and the TIMP and divergence between the TIMP and the Bayley Cognitive and Language Composites. An examination of the items on each test supports the conclusion that the Bayley Motor Scales (gross and fine) measure a number of the same skills as the TIMP at 6 weeks CA, including head control in upright, prone and supine, head turning and visual following, reaching, and making crawling movements.^1,20 The range of possible raw scores on the Bayley gross and fine motor scales, however, is about 6-15 points, while that of the TIMP at 6 weeks CA is as much as 70-80 points, providing a greater degree of precision along with a wider range of assessed skills, including postural control in supported standing, trunk rotation, and head and trunk control during lateral tipping actions evoking vestibular responses.

The Cognitive and Language scales of the Bayley III consist primarily of items assessing visual and auditory attention and habituation compared to only 4 items on the TIMP assessing attention or visual or auditory search and no habituation items.^1,20 The relatively lower, although still statistically significant, correlations among these scales support a conclusion that they measure divergent constructs with only a small degree of overlap.

Despite a good correlation between the TIMP and the Bayley Motor Composite, the decision analysis shows that use of the 2 tests at 6 weeks CA yields widely divergent results. The Bayley Motor Composite scores averaged 116 and the lowest score obtained by any infant was 94. We are not aware of any research on the predictability of later developmental outcome from Bayley III scores, but based on the results of this study, clearly no infant would be identified as delayed or even suspicious in motor development when using the Bayley Motor scale as the basis for early identification of the need for close surveillance or intervention. On the other hand, using the recommended cutoff of −0.5 SD to identify delay using the TIMP would result in 41.4% of the infants being flagged for close surveillance or referral for intervention, depending on how low the score was. Previous research on the TIMP at 30 days as compared to preschool age outcomes showed an overall accuracy in predicting Peabody Developmental Motor Scale scores < 2 SD from the mean of 80%.⁶ The PPV of early scores was 60% such that 40% of infants who scored low at 30 days were found to have normal motor development at 4-5 years of age. PPV improved to 75% and overall accuracy to 87% if testing occurred at 90 days CA. Although one might conclude that 6 weeks CA is too early to identify delay, low scores on the TIMP reflect performance below that of a national sample of 990 infants who were stratified according to perinatal medical risk factors. Thus low scores at 6 weeks CA provide the opportunity to flag infants close to the cutoff or scoring between −.5 and −1.0 SD for intervention to help them close the gap between their performance and that of a national sample from the population of infants born premature for which the TIMP was designed. Vohr and colleagues suggest that all infants born with ELBW should be offered EI because of their high risk for cerebral palsy and other disabilities.¹² In the case of this sample of infants with moderate biological risk for delay, but socioeonomically disadvantaged backgrounds, however, frequent surveillance and instruction of parents in appropriate activities to do at home are recommended until such time as a more definitive diagnosis can be reached.

The findings of this study add to the accumulating evidence on overestimation of ability when testing using the Bayley III by providing information on a moderate-risk sample of infants born preterm and assessed at 6 weeks CA. Previous studies have shown overestimation of performance at ages from 12 months to 2 years CA for Australian infants born full-term and preterm,¹³ infants born with ELBW compared with performance of earlier cohorts on the Bayley II at 18-22 months,¹² infants under 6 months CA enrolled in EI services,²³ infants post-complex cardiac surgery,^24,25 and at 6 months CA English infants born preterm.²⁶

Why has this occurred? The most compelling argument for the overestimation of performance being widely reported is that the 2006 norms for the Bayley III scales were developed in a different manner than those for the previous editions of the scales.²⁴ In an attempt to reflect the broader population of infants in the U.S., the sample for the 2006 norms included more Latino infants and about 10% clinical cases, including infants born preterm and infants with diagnosed disabilities.¹⁰ This sampling decision lowered the overall means of the scaled scores and now boosts the apparent performance of both infants born preterm and infants that are typically developing being compared to the published norms. Moreover, studies have shown that the greatest overestimation of performance is occurring in infants born prematurely.^12,26 As a result, further research on the predictability of scores for later outcomes is critically needed, and studies using the Bayley III scales should always include a control group for comparison with the group of interest.

If the Bayley III is used to make diagnoses, clinicians should be aware that a high cut score should be used. Moore and colleagues suggest a cut score of 80,²⁶ but our results suggest that any below average score at early ages should trigger parent instruction and close surveillance with repeated assessment. We agree with the suggestion of Moore and colleagues that a consensus statement is needed on the classification of developmental impairment using the Bayley III based on the accumulating research.²⁶

Infants with biologic and socioenvironmental risk, such as those in this study, have a high incidence of poor developmental outcomes. For example, Bradley and colleagues found that only 11% of 3-year-old children born prematurely and living in poverty functioned in the normal range in all areas of growth and development.²⁷ The provision of a home exercise program for infants with low scores on the TIMP at hospital discharge has been successful in significantly raising TIMP scores at 4 months,²⁸ demonstration of the TIMP for African-American mothers with low income improved their understanding of infant motor development,²⁹ and predictability to preschool motor development is high by 3 months CA.⁶ As a result, we recommend use of the TIMP for early assessment of infants at risk for delayed motor development with the selection of a cut score being based on resources available and the philosophy of EI of the agency.

Conclusion

Although the Bayley III scales have a degree of commonality with the TIMP, no children in this moderate-risk group were identified as delayed by the Bayley III scales at 6 weeks CA. For assessment of motor performance and determination of the need for intervention at early ages in infants at risk, the TIMP is the preferred test.