Use of the Mullen Scales of Early Learning for the Assessment of Young Children with Autism Spectrum Disorders

Abstract

The psychological assessment is an important component of the diagnostic evaluation in young children suspected of having an autism spectrum disorder but can be hampered by behavioral difficulties. Overt behaviors during administration of the Mullen Scales of Early Learning were coded in 22 preschoolers with an autism spectrum disorder and 20 age-matched typically developing children. Children in the autism spectrum disorder group required less time to complete the assessment but spent proportionally more time exhibiting off task behaviors and less time engaged with the assessment. Scores obtained on the Mullen Scales were positively correlated with level of engagement and negatively correlated with off task behaviors.

Assessing young children can be difficult, particularly when one considers the type of test behavior required for a psychological evaluation (Culbertson & Willis, 1993). Young children who are referred for a developmental evaluation because they appear to have features of autism can be particularly challenging, given their limited social interaction and communication skills (Marcus, Lansing, & Schopler, 1993; National Research Council, 2001). Many features and behavior problems can further interfere with the accurate assessment of a young child’s cognitive abilities, including difficulty maintaining attention, overactivity, sensory issues, and poor compliance, particularly in an unfamiliar setting. Young children with autism can be easily frustrated, and may not appear “eager to please” due to their social difficulties or limited understanding of what is being asked of them.

It is critical to identify cognitive strengths and weaknesses and developmental delays in the young child suspected of an autism spectrum disorder (ASD) as part of the initial diagnostic workup and to assist with treatment planning. With preschoolers, IQ scores are more limited in their predictive value. It is therefore important to consider what skills may be “emerging” and that the child may be more capable than they are able to demonstrate during the assessment session. However, it is often this inconsistency in behavior that interferes with the learning process in these children. The process of administering a standardized psychological test can therefore give the clinician a great deal of information about the child’s abilities as well as their behavioral difficulties.

While experienced clinicians can identify the behaviors associated with an autism spectrum disorder in very young children, it is more difficult to make distinctions between a diagnosis of Autistic Disorder or Pervasive Developmental Disorder-Not Otherwise Specified (PDD-NOS) (Lord & Risi, 1998; National Research Council, 2001). A proportion of the children identified with ASD before age three may not meet criteria for DSM-IV Autistic Disorder at later follow-up but are highly likely to meet criteria for PDD-NOS, a less severe form of autism (Cox et al., 1999; Lord, 1995; Stone et al., 1999). Children under age four may not show significant evidence of repetitive and stereotyped behaviors, and restricted patterns of interest, as required to meet the cutoff for autism on the Autism Diagnostic Interview-Revised (ADI-R; (Lord, Rutter, & Le Couteur, 1994) or according to DSM-IV criteria (Lord, 1995). In one study of two year-olds suspected of having autism (Lord & Risi, 2000), use of one of two instruments that rely on the clinician’s ratings of the child, the Autism Diagnostic Observation Schedule (ADOS; (Lord et al., 2000) or the Childhood Autism Rating Scale (Schopler, Reichler, & Rochen Renner, 1988) instead of the ADI-R alone, allowed inclusion of children who appeared to meet the social and communication criteria for autism but did not yet show significant evidence of restricted or repetitive behaviors. A follow-up assessment confirmed the diagnosis of autistic disorder in the vast majority of these children.

Published studies of young children with ASD have used the Bayley Scales of Infant Development to assess cognitive skills (e.g., (Eaves & Ho, 2004; Lord, 1995; Stone et al., 1999). The advantage of the Bayley Scales is that they can provide an age-appropriate estimate of IQ in young children. The disadvantage is that it is not possible to obtain separate standardized scores for nonverbal and verbal skills (although one can obtain a “mental age” based on the number of items passed from the Language Mental Scale). It is common for young children suspected of having an ASD to exhibit uneven cognitive abilities, particularly when one considers that the most common initial concern for parents is limited language skills (De Giacomo & Fombonne, 1998). The Mullen Scales of Early Learning (Mullen, 1995) may serve as an alternative for evaluating young children suspected of an ASD (Filipek et al., 1999). The Mullen Scales can be administered to infants and children up to 68 months of age. T scores, percentile ranks, and age equivalents can be computed for the five scales separately (Gross Motor, Visual Reception, Fine Motor, Expressive Language, and Receptive Language). Assessment of the young child’s nonverbal ability level is important for estimating overall developmental level as well as possibly predicting outcome. Children with autism who have greater nonverbal delays are less likely to develop functional language at age 5, as are children with relatively strong nonverbal skills but significantly weak receptive and expressive language skills (for review, see Lord, Risi, & Pickles, 2004).

Although clinicians often find it difficult to evaluate IQ in young children with ASD, the assessment process has not been systematically examined. In the current research study, specific, overt behaviors during cognitive assessment were coded in young children with a diagnosis of ASD and age-matched typically developing children. The goals of the present study were (1) to assess differences in testing behaviors between the groups and (2) to examine how testing behaviors were related to the level of performance on the Mullen Scales.

Methods

Participants

The participants were 22 children with a diagnosis of ASD and 20 age-matched healthy, typically developing children (TYP). Children in both groups were tested as part of the initial assessment procedure to determine eligibility for participation in a longitudinal MRI study. The children ranged in age from 16 to 43 months (see Table 1). The groups did not differ significantly in terms of chronological age. The examiners were not blind to the identity of the children. All children in the ASD group were tested by a licensed clinical neuropsychologist experienced in assessment and diagnosis of ASD in young children (N.A.). The children in the TYP group were either tested by this examiner or a trained research assistant under her supervision.

Table 1.

Mean participant characteristics (standard deviations in parentheses).

	ASD (N=22)	TYPICAL (N=20)
Age at test (months)	29.9 (7.6)	27.5 (7.4)
Age range (months)	16 – 43	16 – 40
Gender	17M, 5F	16M, 4F
Mullen Scales (t-scores)
Visual Reception*	29.9 (11.4)	53.2 (9.8)
Fine Motor*	32.4 (12.0)	52.8 (11.4)
Receptive Language*	25.0 (10.1)	51.6 (10.7)
Expressive Language*	27.3 (11.7)	49.2 (10.7)
Mullen Scales (age equiv. scores)
Visual Reception*	20.5 (7.1)	29.6 (10.2)
Fine Motor*	21.4 (6.6)	28.4 (9.2)
Receptive Language*	15.9 (8.6)	29.5 (11.1)
Expressive Language*	15.9 (10.5)	27.9 (12.0)
Vineland Scales (standard scores)**
Communication	66.0 (8.4)	-
Socialization	66.0 (7.7)	-
Daily Living Skills	67.2 (6.6)	-
Motor Skills	85.7 (11.0)	-

^*p < .0001

^**Available for 15 participants in the ASD group Table 3. Mean results from videotape coding for both groups (standard deviations in parentheses).

Diagnosis of ASD was established using a best estimate approach. The ADI-R (Lord et al., 1994) was administered to the child’s primary caregiver. The Vineland Scales of Adaptive Behavior (Sparrow, Balla, & Cicchetti, 1984) were administered to the primary caregiver for 15 of the 22 children with ASD (Table 1). At a separate appointment, each child was administered the Mullen Scales of Early Learning (Mullen, 1995) followed by the ADOS (Lord et al., 2000). Examiner reliability in administration and scoring of the ADI-R and ADOS was established prior to initiation of the study. Children were included in this group if they met criteria for autism on the ADI-R and the criteria for autism or ASD on the ADOS. Children who had another diagnosable medical condition that might affect brain development, a visual, auditory, or motor impairment, and/or who were born preterm, were excluded from the study.

Children in the TYP group were recruited from the community. Children with a known developmental disorder or delay, medical condition, and/or who were born preterm, were excluded from the study.

Procedure

The Mullen Scales of Early Learning (Mullen, 1995) was administered in the standardized format. All was assessments were completed in the same laboratory setting and videotaped for further analysis. The child’s primary caregiver sat in a chair behind the child during the assessment. The child and the examiner sat at a child size testing table. If the child left his/her seat during the assessment, he/she was encouraged to return to his/her seat. If this proved too difficult, some of the assessment was conducted on the floor until the child eventually returned to the table.

Coding Procedures

Videotaped test sessions were coded for both off task and on task behaviors in 10-second intervals. Behaviors were counted per interval, regardless of duration or how many times they occurred per interval. For example, if a child looked away, looked back at the test item, and then looked away during the same 10-second interval, “looking away” would be counted once during that interval. If a behavior continued for more than one interval, it was scored in each interval. Coding definitions are listed in Table 2. The coder was blind to the diagnostic category of the children. The coder achieved at least 85% reliability on a subset of tapes with two research assistants who had developed the coding scheme.

Table 2.

Coding definitions

Break	Scored when the assessment was temporarily stopped so the child could take a short break.
Engaging	Scored when child actively attends to the task that is presented appropriately (e.g., answering a question, pointing to an object, placing an object in the appropriate place). Scored each time a new task is presented and child tries to engage appropriately.
Out of Position	Scored when the child leaves his/her seat or standing position at the assessment area. If child is standing, this can be coded when the child walks away from the assessment area. If child is sitting, this can be coded if the child gets up and walks or crawls away from the assessment area. If child alternates between sitting and standing, but stays in the assessment area where the examiner can administer test items, he/she is considered to be in position. Positions can vary throughout the assessment session
Whining/Crying	Scored when the child whines, whimpers, cries, or tantrums for longer than 3 seconds.
Looking Away	Scored when the child actively looks away from the test items in an attempt to not engage in the test item while still in the testing position.

Results

As expected, the children in the ASD group obtained significantly lower T-scores on all scales compared to the children in the TYP group, who on average scored within the expected range for their age (ps < .0001; see Table 1). The children in the ASD group performed relatively better on the Fine Motor scale and relatively worse on the Receptive Language scale. A T-score of 20 is the minimum score and represents a score three or more standard deviations below the mean or an IQ of approximately 55. A substantial number of participants in the ASD group (N=16 or 73%) received one or more T-scores of 20. The performance range for the children in the ASD group is therefore better demonstrated by their age-equivalent scores (see Table 1).

The length of time spent in the test session was significantly shorter for the children in the ASD group than the children in the TYP group (F (1, 40) = 13.14, p = 0.001, partial eta squared = .25; Table 3). Due to this difference, the proportion of time spent in off task and on task behaviors were computed. It was sometimes necessary for the children to take a short break during the assessment. The groups were not significantly different in terms of the proportion of total time spent on a break (see Table 3). The children in the ASD group spent a significantly smaller proportion of total time engaged in the assessment than the children in the TYP group (F (1, 40) = 7.52, p = .009, partial eta squared = .16). The children in the ASD group spent a greater proportion of time out of position than the children in the TYP group (F (1, 40) = 7.01, p = .012, partial eta squared = .15). The children in the ASD group spent a significantly greater proportion of time whining and crying (F (1, 40) = 14.67, p = .0001, partial eta squared = .27) than the children in the TYP group. As shown in Table 4, a greater proportion of time spent engaged in the assessment was significantly related to higher scores on each of the Mullen Scales. A greater proportion of time spent out of position or whining/crying was significantly correlated with lower scores on almost all of the Mullen Scales. The proportion of time spent looking away was not correlated with performance on the Mullen Scales.

Table 3.

Mean results from videotape coding for both groups (standard deviations in parentheses).

	ASD (N=22)	TYPICAL (N=20)
Time in test session (min.)*	34.42 (14.7)	53.5 (19.2)
Proportion of time on break	.11 (.13)	.10 (.08)
Proportion of time engaging*	.32 (.12)	.44 (.17)
Proportion of time out of position*	.22 (.12)	.12 (.12)
Proportion of time whining/crying*	.13 (.15)	.00 (.01)

^*significant group difference

Table 4.

Correlations between Mullen Scales (age equivalent scores) and behavioral codes (proportion of total time).

	Visual Reception	Fine Motor	Receptive Language	Expressive Language
Engaging	.56**	.61**	.65**	.62**
Out of Position	−.51**	−.43**	−.49**	−.47**
Whining/Crying	−.34*	−.29	−.49**	−.46**

^*p < .05

^**p < .01

Given that the children in the ASD group obtained significantly lower scores on the Mullen Scales, tended to spend a smaller proportion of time engaged in the assessment, and a greater proportion of time in off task behaviors, one may question the meaning of the scores obtained on the Mullen Scales for the ASD group. A series of exploratory analysis of covariance tests were conducted and the group differences in Mullen performance remained when the proportion of off-task behaviors were held constant. Correlations between the T-scores on the Mullen and standard scores on the Vineland Scales of Adaptive Behavior (available for 15 of the ASD participants) were computed (Table 5). Expressive language scores on the Mullen were strongly correlated with the Communication and Socialization scale scores on the Vineland. Receptive language scores on the Mullen were correlated with Communication scale scores on the Vineland. The relative distance from the normal mean was also similar across these standard scores (i.e., approximately 2.5 standard deviations below the mean on average).

Table 5.

Correlations between Mullen Age-Equivalent Scores and Vineland Age-Equivalent Scores (ASD Group: N=15)

	Visual Reception	Fine Motor	Receptive Language	Expressive Language
Communication	.27	.06	.53*	.78**
Socialization	.09	−.09	.20	.65**
Daily Living Skills	−.18	.01	.17	−.08
Motor	.18	.35	.35	.38

^*p < .05

^**p < .01

Discussion

The results supported the hypothesis that during a standardized cognitive assessment, young children with ASD spend proportionally more time exhibiting off task behaviors than typically developing children and proportionally less time engaged with the assessment. The assessment sessions for the children in the ASD group were significantly shorter in duration. This may have been due to the reduced number of items administered (i.e., developmental delays lead to reaching the ceiling items sooner). This difference may also reflect the increased frequency of off task behaviors exhibited by these children. When a young child is less engaged, out of position, and whining and crying, the examiner may try to move through the assessment a bit more quickly in order to complete the assessment before the child can no longer put forth adequate effort, particularly when other tests are required during the assessment appointment.

The scores obtained on the Mullen Scales were positively correlated with level of engagement and negatively correlated with off task behaviors. It is unlikely, however, that these behavioral regulation problems are entirely responsible for the observed developmental delays. Some difficulties may appear by the end of the first year of life in children later diagnosed with ASD, such as limited social attention and social behavior (Maestro et al., 2002; Osterling & Dawson, 1994; Werner, Dawson, Osterling, & Dinno, 2000). Early difficulty with joint social attention that is characteristic of children with autism is hypothesized to affect the development of their early language skills. It appears likely that those children who have more difficulty engaging with others and regulating their emotions are less likely to benefit from the early opportunities to develop language and cognitive skills.

Young children with ASD have difficulty with imitation skills that do not appear to be accounted for by lack of social cooperation or attention to task (Rogers, Hepburn, Stackhouse, & Wehner, 2003). In order to direct the child to the requested activity, some items on the Mullen require the child to imitate the examiner (e.g., stacking blocks in a particular way, sorting items that match). These items were often difficult for the younger children with ASD who had not yet developed functional language and tended to lead to more off task behaviors. Pointing for purposes other than requesting is also difficult for young children with ASD (Leekam, López, & Moore, 2000; Sigman, Mundy, Sherman, & Ungerer, 1986). This proved challenging on certain items, such as the Receptive Language scale items that require the child to point to the line drawing that goes with a word and the items on the Visual Reception scale that require pointing to two items that match. Clinicians may also wonder how to interpret T-scores that fall at or below 20. Although age-equivalent scores can be misleading and cumbersome, in the case of testing young children with ASD age-equivalent scores may be more useful for obtaining an estimate of the child’s current ability level. For low functioning children, this strategy may also prove helpful for assessing progress in six-month intervals following the introduction of early intervention programs.

Motivation can affect test results; therefore it is important to enhance motivation as much as possible in young children without affecting standardized procedures of the assessment (Koegel, Koegel, & Smith, 1997). Useful strategies include the use of reinforcement procedures, a picture schedule, scheduling the assessment at a time of day that will not interfere with the child’s nap time or meal times. When young children refuse to attempt test items it can be difficult to determine their current level of functioning. Results from a study of neuropsychological task performance among preschoolers indicated that test refusals typically reflect a child’s poor underlying skills and/or an attempt to avoid failure, rather than noncompliant or oppositional behavior (Mäntynen, Poikkeus, Ahonen, Aro, & Korkman, 2001).

When children in the present study engaged in off task behavior, the examiner redirected the child back to the test item while being careful to not break rapport with the child. In all cases, the examiner continued administration of test items in order to reach a valid ceiling level. Anecdotally, the examiner asked the parent in each case if he/she felt the child was able to adequately demonstrate his/her abilities. In some cases, the parent indicated that the child had some skills that they may not have been able to demonstrate adequately under the constraints of the task (e.g., naming real objects more easily than objects represented by line drawings). Most parents indicated that the skills assessed by the test items and their child’s performance during the assessment was consistent with their current developmental level. It was helpful to discuss the session with the parent in this manner after all of the assessment was completed, particularly in those cases where the child had difficulty maintaining attention and cooperation with the examiner.

A group of typically developing children was used for comparison purposes in the present study to provide a type of “normative” range expected for children tested under these circumstances and because they were easily available at the time of the study. Future studies may consider including a group of language delayed children who do not appear to have ASD to determine if the level of off task behavior observed in this study is specific to ASD or is more consistent with developmental delay. It is important to note that specific diagnostic certainty is not typically possible until a child is 4 or 5 years of age (Lord & Risi, 1998). However, it is expected that the majority of these children will meet criteria for autistic disorder when evaluated with the ADI-R and ADOS after age 4 and a subset will meet criteria for PDD-NOS. Future studies with larger samples and a longitudinal design may determine if certain behavioral characteristics during the psychological evaluation process may be indicators of better response to early intervention and/or better prognosis.

Some have questioned the appropriateness of traditional tests with young children (Neisworth & Bagnato, 2004). Reasons cited for young children being “untestable” include behavior that is at odds with test requirements, lack of language, poor motor skills, and a lack of attention and other self-control behaviors. However, experienced clinicians typically find that few young children with autism are untestable (Ozonoff, Goodlin-Jones, & Solomon, 2005). Alternatives to a direct assessment are parent observations and reports, curriculum-based assessments by teachers and providers, play-based assessments, observations of behavior at home or in the preschool. Among the children with ASD, expressive and receptive language scores on the Mullen Scales were significantly correlated with the Vineland Communication and Socialization Scales. It is surprising that scores on other scales were not related to scores on the Vineland. This may reflect the different skills emphasized by the tests (Magiati & Howlin, 2001), as well as the use of direct observations vs. parent report. Rather than being mutually exclusive, a combination of approaches is recommended for an accurate assessment of ASD (Filipek et al., 1999; National Research Council, 2001; Ozonoff et al., 2005).

Overall, the Mullen Scales of Early Learning provided a format conducive for evaluating motor and visuomotor skills, language comprehension, and communication skills. Children with autism typically have the greatest difficulty on tests with more social and language demands and relatively less difficulty with nonverbal tests with minimal speed and motor demands (Joseph, Tager-Flusberg, & Lord, 2002; National Research Council, 2001). Nonverbal IQ scores therefore appear more stable in school age children and may be a more accurate estimate of ability in very young children with ASD. As the young child experiences early intervention services, cognitive skills should be reevaluated to measure response to treatment and to get a better sense of the child’s level of functioning. By anticipating potential problems and preparing for the evaluation, the examiner is also more likely to complete a successful evaluation (Marcus et al., 1993).