Validity and reliability of the developmental assessment screening scale

Abstract

Objectives:

To develop and validate the 6-year Ages and Stages Questionnaire (ASQ) for school-age children.

Methods:

Parents/caregivers of children 66–78 months were recruited from 6 countries and 15 states in the United States. Similar to other ASQ intervals, the 6-year ASQ has five developmental domains targeted for children from 66 months to 78 months of age. We assessed internal consistency reliability, test-retest reliability, the scale structure, validity (correlation coefficients), and utility of the 6-year ASQ. The 6-year ASQ screens development in five domains using 30 items.

Results:

Coefficient alpha measuring internal consistency was 0.97. Test-retest reliability was estimated by having parents complete two ASQ's on the same child within a week interval. Correlations between scores of the two ASQ's reflected stability of scores. Test-retest reliability results were robust with an intraclass correlation coefficient of 0.94. Factor analysis results resulted in a five factors structure. The Pearson correlations coefficients between the latent variables were moderate to large and statistically significant, P < 0.0001. The 6-year ASQ screens development in five domains using 30 items. The administration time varied from 10 to 60 min with a mean of 15.17 min. General utility result shows that the 6-year ASQ can be used with satisfaction by caregivers, pediatricians, professionals, and parents.

Conclusions:

Preliminary data on the 6-year ASQ reflected promising results. An easy-to-administer, accurate caregiver-completed screening tool may increase the frequency of screening for school-age children.

Introduction

Early identification of developmental disabilities in school-age children in the community is essential for timely remedial intervention and often leads to early treatment and ultimately improved long-term outcomes.[1,2,3,5] Early detection of developmental disabilities is critical to the welfare of children and their families because it allows access to timely diagnosis and treatment.[4] It has been estimated that only about half of the children with developmental problems are detected before they begin the school.[6,7] Developmental screening and developmental surveillance constitute an ongoing process of monitoring the status of a child by gathering information about child's developmental status from multiple sources, including skillful direct observation from parents/caregivers and relevant professionals.[8,9,10] Parents’ reports of current attainment of developmental tasks have been shown to be accurate and reliable.[11] In keeping with recommendations from the American Academy of Pediatrics,[12] and National Screening Committee,[13] consideration should be given to the use of standardized screening measures utilizing parental reports as a part of the assessment process.

The AAP policy statement emphasized the use of standardized screening tools that are practical, easy to use, and culturally appropriate, and make use of the considerable knowledge parents have about their child.[12,13,14] A second AAP policy statement set forth screening algorithms and methods, including those that use standardized parent-completed tools such as the Ages and Stages Questionnaire^® (ASQ).[12,15]

The ASQ is a parent-completed questionnaire that may be used as a general developmental screening tool, evaluating five developmental domains: Communication, gross motor, fine motor, problem-solving, and personal adaptive skills for children from the ages of 1–66 months.[16] In most cases, these questionnaires accurately identify young children who are in need of further evaluation to determine whether they are eligible for early intervention services.[17,19] The ASQ meets the requirement of Level 1 screening stated by the AAP in terms of the comprehensiveness of ASQ results, and can be used for producing general findings of children's skills and monitoring children's developmental status. The ASQ is cost-effective and widely used in the United States and other countries.[17] It is recommended by pediatricians for early identification in the United States.

The usefulness of the ASQ has been demonstrated in many contexts, and international interest has been aroused by the demonstrated benefit of early intervention for children with developmental delays. The ASQ developers designed the 6-year ASQ to answer the need for a screening tool for children approaching school-age.[37] The 6-year ASQ evaluates developmental skills in children from the ages of 66–78 months.[18] This new interval will assist with establishing a mechanism for identifying school-age children early on and improving outcomes during the school years.

Methods

Development and design of the 6-year Ages and Stages Questionnaire^®

The 6-year ASQ was developed by ASQ authors and initially studied in four phases; the first phase included 18–26 test items/domain. Initially, test items were randomly arranged (i.e., not placed in chronological order) to decrease any possibilities of order effect and to address the item functioning questions (e.g., item difficulty test items which estimate the difficulty of each test item based on the participants’ ability to correctly respond to each test item). In the second phase, test items per domain were selected by using item response theory analysis and were investigated for technical adequacy and item functioning. In the third phase, test items and item difficulty were evaluated by a panel of 13 national and international experts with previous work experience with young children. The experts were academicians/professionals with relevant experiences between 2 and 25 years in their expertise areas. The range of experiences provided a wide and relevant perspective on the appropriateness and validity of the items to be included. In the fourth and final phase, 6 test items/domain for a total of 30 test items were included.

Population

One hundred and sixty-nine participants were recruited and written informed consent was obtained based on approved protocol from the research compliance services, University of Oregon. Subjects completed the Demographic Questionnaire (DQ), the 6-year ASQ, and Utility Questionnaire (UQ).

Procedures

Participants were parents/caregivers of children ages between 66 months and 78 months. ASQ and other research measures were completed by parents/caregivers either online or via pencil and paper.

Measures

Demographic Questionnaire

A DQ was used to collect general family information about child's gender, disability status, ethnicity, child's birth weight, mother's education, yearly family income, and the child developmental status (i.e., previously identified with disability).

The 6-year Ages and Stages Questionnaire^®

The 6-year ASQ has five developmental domains (i.e., personal social, gross motor, fine motor, problem solving, and communication) comprising thirty items for children of 66 months to 78 months of age. Three response options are included: “Yes,” “sometimes,” and “not yet” and numeric values are assigned at 10, 5, and 0, respectively. The 6-year ASQ follows the content and format from the previously developed ASQ-3™.

Utility questionnaire

A UQ was used to collect information about degree of satisfaction from parents and caregivers with the 6-year ASQ. The UQ included the length of time, understanding test items, and any assistance needed when completing the 6-year ASQ.

Statistical and psychometric analyses

The psychometric properties of the 6-year ASQ were investigated. Regarding psychometric properties, we first computed the internal consistency (Cronbach's alpha), then examined the fit between the scale structures and observed the data with confirmatory factor analysis (CFA) using Analysis of Moment Structures (AMOS) structural equation modeling. In the CFA, the models were considered to be a good fit. Results suggest that a CFA model provides more accurate results, and another important design factor is the selection of the sample size.[18] The CFA procedure is designed to determine what the common factors are that account for item variance. The 6-year ASQ interval was modeled with five latent variables, which were: (a) Communication, (b) gross motor, (c) fine motor, (d) problem solving, and (e) personal social. These five factors were labeled, as on the original ASQ: Factor 1, personal social; factor 2, problem solving; factor 3, fine motor; factor 4, gross motor; and factor 5, communication. The results of the goodness of fit model indicates were χ² =1430.04, degree of freedom was 395, probability level was 0.000, and root mean square error of approximation for CFA model was 0.12. Therefore, the implied model showed an acceptable level of fit, so its parameters were interpretable.[20] The normed fit index (NFI), comparative fit index (CFI), and Tucker-Lewis index (TLI) were 0.74, 0.80, and 0.77, respectively. The possible values of CFI and TLI range from 0 to 1, with values close to one demonstrate a good fit.[21]

Results

Participants

We recruited participants from six different countries and 15 states from the United States. Parents/caregivers of more boys (54.4%) than girls (45.6%) participated. The majority of diverse participants were highly educated. Table 1 summarizes participant demographic characteristics.

Table 1.

Demographic characteristics of parents/children

Reliability

Pearson correlations coefficients between the latent variables were moderate to large and statistically significant, P < 0.0001. Table 2 shows the bivariate correlation between the latent variables of the 6-year ASQ. Table 3 shows the CFA correlation matrix and CFA principal component analysis matrix. The Kaiser-Meyer-Olkin (KMO) measure was 0.87, which was great.[22] The value in this study was 0.87, which was into the range of great, reflecting confidence that the sample size was adequate for factor analysis. Therefore, the implied model showed high level of fit and for our data, the significant alpha was (0.00) highly significant.[23]

Table 2.

Bivariate correlation among the latent variables of the 6-year Ages and Stages Questionnaire^®

Table 3.

Confirmatory factor analysis correlation matrix and confirmatory factor analysis principal component analysis matrix

Test-retest reliability or agreement between two ASQ was measured by intraclass as well as and Pearson r correlations. Result depicted that the Cronbach's alpha value for the questionnaire was 0.97, reflecting a high level of consistency (above 0.85).[27,28] Reliability between two ASQ's was computed using Statistical Package for the Social Science 21 (SPSS 21) to compute the intraclass correlation coefficient (ICC) using a two-way mixed effects model and type consistency.[24,25] The average measure ICC was 0.94 with a 95% confidence interval from 0.91 to 0.97. The extraversion subscale consisted of 60 items (α =0.95) [Tables 2 and 3].

Validity

The 6-year ASQ scores correlated significantly with the latent variables. The NFI, CFI, and TLI were 0.74, 0.80, and 0.77, respectively. The possible values of CFI and TLI range from 0 to 1, with values close to one demonstrate a good fit.[26] The KMO measure was 0.87, which was great.[22]

Utility

Satisfaction is measured by consumers’ responses to the usefulness, time, and overall impression of questionnaires.[29,30] The 6-year ASQ parent satisfaction survey included six questions related to administration time, ease of completion, difficulty in completion, benefits from completion, and any recommended changes. Most parents said the 6-year ASQ took 10–20 min to complete and the majority (87%) said it was easy to understand. More than half (58.3%) specified that it helped them to think about their child's behavior and 33.3% responded that it was interesting to complete. None of the participants provided suggestions for making it better.

Discussion

This study examined the 6-year ASQ and is the first to investigate the psychometric properties of this interval. The 6-year ASQ was modeled with five latent variables and each measured by six items. The CFA models of the 6-year ASQ were theoretically grounded. Both goodness of fit and the item loadings supported the hypothesized structure of the 6-year ASQ with five latent variables. The items were largely loaded in their latent variables. The goodness of fit indicated that the implied model showed an acceptable level of fit.[20] Pearson correlation coefficients between the latent variables showed CFA significant. The correlation between the scores of the two ASQ's reflected stability of the 6-year ASQ. The Pearson correlations coefficients between the latent variables were moderate to large and statistically significant. In summary, the 6-year ASQ showed a well-established factor structure, which was represented by five latent variables. The item loadings demonstrated their strong relatedness to the latent variables.

Limitations

The current findings should be considered in light of several limitations. A relatively small number of parents/children participated in this research study. A larger sample size might have better results for the CFA model. A second and related limitation pertains to the size and distribution of the sample. The total sample was collected only from 15 states in the United States and from six different countries. This sample may not be representative of all of the children in the US. A larger and more representative sample might better reflect the US population.

Implications

Developmental assessment of school-age children is a challenging task. Screening tools are required to identify children for further testing, follow-up, and progress monitoring. Our research study provides initial evidence for expanding the age range of the ASQ. Early identification of developmental status in school-age children in the community is essential for remedial intervention. Pediatricians and service providers advocate a family-centered care model.[31] Early identification can be completed by pediatricians, parents, and service providers, using clinical impression and/or standardized screening tools, and studies have shown that parent-completed standardized questionnaires are at least equal or superior to the pediatrician's formal screening.[32] The ASQ appears to be a suitable screening tool for developmental delays in primary care settings and research findings support the recent AAP recommendations for routine use of the ASQ as a screening tool in primary care settings.[14,33] The ASQ has shown to be reliable and cost-effective as well as correlate well with pediatricians’ and service providers’ assessment.[34,35,36] The new age interval will assist with establishing a common mechanism for identifying children early on through school entry, perhaps improving screening rates. The results of this research study support the feasibility of appropriate and effective screening questionnaire for school-age children. To ensure the questionnaires, maintain their psychometric properties, test-retest reliability, sensitivity, and specificity, and regular research is required. Initial studies have been completed on the 6-year ASQ. This tool can be used in kindergarten as it is easy to administer and score and many children can be screened at low cost. Pediatricians, professionals, and parents can use it to learn about skills the child has mastered, skills that are emerging and skills the child is not yet doing.

Conclusions

The results of this study show initial evidence that the 6-year ASQ may be useful for the early identification of the at-risk population and used to improve the early identification of school-age children and improve outcomes before disabilities become more established. Further research is needed to confirm these initial results.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.