Use of the Ages and Stages Questionnaire and Bayley Scales of Infant Development-II in Neurodevelopmental Follow-up of Extremely Low Birth Weight Infants

Abstract

Objectives

For infants born extremely low birth weight (ELBW), we examined the 1) correlation between results on the Ages and Stages Questionnaire (ASQ), and the Bayley Scales of Infant Development II (BSID-II) at 18-22 months corrected age; 2) degree to which earlier ASQ assessments predict later BSID-II results; 3) impact of ASQ use on follow-up study return rates.

Study Design

ASQ data were collected at 4, 8, 12, and 18-22 months corrected age. The BSID-II was completed at 18-22 months corrected age. ASQ and BSID-II 18 – 22 month sensitivity and specificity were examined. Ability of earlier ASQs to predict later BSID-II scores was examined through linear regression analyses.

Results

ASQ sensitivity and specificity at 18-22 months were 73% and 65%, respectively. Moderate correlation existed between earlier ASQ and later BSID-II results.

Conclusions

For ELBW infant assessment, the ASQ cannot substitute for the BSID-II, but appears to improve tracking success.

INTRODUCTION

Long-term assessment of the outcomes of high-risk infants who received neonatal intensive care is an essential component of both understanding the implications of providing care to these patients and appropriately evaluating the results of interventions. The most commonly used neurodevelopmental follow-up protocol for high-risk infants consists of a single, professionally administered neurodevelopmental evaluation at 18-22 months corrected age and includes administration of the Bayley Scales of Infant Development.^1,2,3 This follow-up paradigm is expensive and difficult, and frequently results in significant numbers of children lost to follow-up.^4,5 If accurate information regarding developmental outcomes could be obtained from a parent-administered questionnaire, this could be both cost-effective and result in improved follow-up rates for high-risk infant follow-up studies. Studies indicate that parents can be reliable and valid sources of information regarding their child’s current developmental status, when compared to professionally administered tests. ^6,7,8 Squires and Bricker were able to determine that most mothers, including those with risk factors such as low maternal education or a history of substance abuse, were able to use a parent-report screening tool and that their responses closely matched the results obtained by professionals using a standardized assessment measure.⁹

Previous comparisons of parent-administered developmental questionnaires with professionally administered neurodevelopmental evaluations of high-risk infants have yielded varying results. ^{10, 11, 12, 13} In this prospective follow-up study of extremely low birth weight (ELBW) infants, we asked to what degree the parent-administered Ages and Stages Questionnaire (ASQ), correlated with a professionally administered BSID-II assessment at 18-22 months corrected age; whether ASQs completed at 4, 8, and 12 months corrected age would predict performance on the BSID-II at 18-22 months corrected age; and whether use of the ASQ as a tool to maintain contact with families would improve follow-up rates.

METHODS

Study Population

Infants eligible for this study were 291 ELBW infants enrolled after parental consent into the PROPHET study and who survived to hospital discharge. The PROPHET Study was a multicenter, randomized, placebo-controlled trial of early, low-dose hydrocortisone to prevent bronchopulmonary dysplasia.¹⁴ Eligibility criteria for that study included birth weight between 500-999 grams and the need for mechanical ventilation at 12-48 hours of age.

Study procedures

Neurodevelopmental evaluation of infants enrolled in the PROPHET study included administration of the BSID-II and neurologic examination by certified examiners at 18 – 22 months corrected age. ³ Families were separately consented for the ancillary ASQ study and were asked to complete an ASQ when their child was 4, 8, 12 and 18-22 months corrected age. Approximately 2 weeks prior to an infant turning 4, 8, 12 and 18-22 months corrected age, an age-appropriate ASQ form was mailed to the home. Completed ASQs were either mailed back to the center (families were provided with stamped and addressed envelopes) or the research coordinator called the family and obtained the answers to the ASQ by phone. If the family had not completed the 18 – 22 month ASQ prior to the professional neurodevelopmental evaluation, the family was asked to complete the ASQ on site.

Developmental Assessment Tools

The Bayley Scales of Infant Development¹⁵ includes Mental (MDI) and Psychomotor (PDI) Scales, as well as a Behavior Rating Scale. Raw scores on the BSID-II are converted to standardized scores with a mean of 100 and a standard deviation of 15. For this study, a standard score of 70 or below, which is 2 standard deviations below the mean, was considered a “fail” on either the Mental or Psychomotor Scale of the BSID-II. The Ages and Stages Questionnaire: A Parent-Completed, Child Monitoring System¹⁶ is a screening tool that uses parent observation to assess child development and behavior. Questionnaires are available at two-month intervals from 4-24 months and at 30, 33, 36, 48, 54, and 60 months age points. The standardized assessment window begins 1 month prior to and ends 1 month following each questionnaire age point. Questions are written at a 4^th to 6^th grade reading level so that most parents can complete them independently. The questionnaires also can be used in an interview format for those who have difficulty with reading or language. The ASQ questionnaire at each age point contains 6 questions in each of 5 domains of development-communication, fine motor, gross motor, problem-solving, and personal-social, for a total of 30 questions. Answer options for each question include “yes”, “sometimes”, or “not yet”. A “yes” response receives 10 points, “sometimes” receives 5 points, and “not yet” receives 0 points. Each of the 5 domains is scored separately. These separate scores are not added together to obtain a composite score. A score of 2 standard deviations or more below the mean in any one of the domains is considered a “fail” on the ASQ.¹⁶

Statistical Analysis

To examine the relationship between the 18-22 month ASQ and the BSID-II, a 2×2 contingency table was constructed where the rows and columns were determined by a pass or fail of the ASQ and BSID-II, respectively. From this table, the test characteristics of the ASQ were calculated, including the sensitivity, specificity, and positive and negative predictive values. The ability of the ASQ to predict performance on the Bayley Mental Scale and the Bayley Psychomotor Scale was examined using linear regression analysis without adjustment for other factors. The Spearman rank correlation coefficient was used to measure the degree of association between the percentage of subjects with two or more ASQs completed and the follow-up rate. Analyses were conducted using SAS Version 9 (SAS Institute, Inc., Cary, NC).

RESULTS

Of the 291 infants who survived to discharge, 250 completed the BSID-II at 18-22 months corrected age. Of the 250 children who completed the BSID-II, 219 also completed at least one of the possible four ASQ questionnaires. Table 1 depicts the 8 unique ASQ patterns of return across the 4 age points. Of the 228 children, a total of 78 ASQs were completed at 4 months, 95 at 8 months, 105 at 12 months, and 228 at 18-22 months. As illustrated, 9 of the children with one or more ASQs did not complete the BSID-II evaluation. Characteristics for the 228 children in this study included: mean birth weight - 738.5 g (500-997 g.); mean gestational age - 25.4 weeks (23.0-31.0 weeks); gender - males 121 (53.1%), females 107 (46.9%); ethnicity - Non-Hispanic white 117 (51.3%), Black 73 (32.0%), Hispanic 27 (11.8%), Other 11(4.8%); median total household income was $30,000 - $40,000.

Table 1. Patterns of Return of the Ages and Stages Questionnaires (ASQ).

Eight patterns of ASQ return are illustrated across 4 age points: 4, 8, 12 and 18-22 months. The 18-22 month column indicates if the BSID-II exam was completed

ASQ Completion and Return Age Points
Pattern of ASQ Return	ASQs completed at 4 mos.	ASQ completed at 8 mos.	ASQ completed at 12 mos.	ASQ completed at 18-22 mos.
				Also the BSID-II Exam	Not the BSID-II Exam
A	0	0	0	65	1
B	0	0	27	27	3
C	0	15	0	15	0
D	21	0	0	21	3
E	0	34	34	34	1
F	11	0	11	11	1
G	13	13	0	13	0
H	33	33	33	33	0
Totals	78	95	105	219	9

n = 228 children

Relationship between the BSID-II and ASQ at 18-22 months

Table 2a presents the test characteristics of the ASQ, including the sensitivity, specificity, and positive and negative predictive values. When using cutoff points of >2 SD below the mean on either the Mental Development Index (MDI) or Psychomotor Development Index (PDI) of the BSID-II as a “fail” and a score of >2 SD below the mean on any of the domains of the ASQ as a “fail”, the ASQ had a sensitivity of 73% and a specificity of 65%. The positive predictive value was 52%, and the negative predictive value was 82%. Sixteen of 59 children who scored poorly (>2 SD on either the Mental or Psychomotor Scale) on the BSID-II would have been missed by the ASQ, while 39 of 110 children would have been incorrectly identified as having a developmental delay. Thus the clinical epidemiological characteristics (sensitivity and specificity) of the ASQ in relationship to the BSID-II did not reach suggested levels of 80%. ¹⁷

Table 2a.

Sensitivity/Specificity of the 18-22 Month ASQ (>2 SD) and BSID-II (> 2 SD)

		Bayley Scales of Infant Development-II
		Pass1	Fail2	Totals
ASQ	Pass1	71	16	87
	Fail2	39	43	82
	Total	110	59	169

Sensitivity: 73% (PPV=52%); Specificity: 65% (NPV=82%)

¹≤ 2 SD below the mean

²> 2 SD below the mean

We evaluated the effect of using a “fail” cut-off on the ASQ of >1 SD below the mean, rather than >2 SD. Table 2b shows the corresponding shifts in the sensitivity, specificity, positive and negative predictive values. The shift from 2 SD to 1 SD below the mean resulted in improved sensitivity with the ASQ only “missing” 3 children with a developmental delay. On the other hand, 75 children would have been incorrectly considered as suspect for a developmental delay but were found to be above the BSID-II cut off of 2 SD below the mean, thus decreasing the specificity to 32%.

Table 2b.

Sensitivity/Specificity of the 18-22 Month ASQ (>1 SD) and BSID-II (>2 SD)

		BSID-II Scales of Infant Development
		Pass1	Fail2	Totals
ASQ	Pass3	35	3	38
	Fail4	75	56	131
	Total	110	59	169

Sensitivity: 95% (PPV=43%); Specificity: 32% (NPV=92%)

¹≤ 2 SD below the mean

²> 2 SD below the mean

³≤ 1 SD below the mean

⁴> 1 SD below the mean

We also examined the effect on sensitivity and specificity rates by shifting the standard deviation of the BSID-II to >1 SD below the mean while keeping the ASQ at >2 SD. We found that the sensitivity rate dropped to 63 % while the specificity improved slightly to 75% (see Table 2c).

Table 2c.

Sensitivity/Specificity of the 18-22 Month ASQ (> 2 SD) and BSID-II (> 1 SD)

		BSID-II Scales of Infant Development
		Pass1	Fail2	Totals
ASQ	Pass3	47	40	87
	Fail4	16	67	83
	Total	63	107	170

Sensitivity: 63% (PPV=81%); Specificity: 75% (NPV=54%)

¹≤ 1 SD below the mean

²> 2 SD below the mean

³≤ 2 SD below the mean

⁴> 2 SD below the mean

Early ASQ Prediction of 18-22 month BSID-II

As illustrated in Table 1, a total of 154 children had at least one completed ASQ at age 4, 8 and/or 12 months, plus completed the BSID-II tests at 18-22 months corrected age. Excluded from this sub-sample were the 65 children who had no ASQ results prior to the 18-22 month BSID-II evaluation and 9 children who had ASQs completed but were not brought in for the BSID-II evaluation. As illustrated in Table 1, the number of ASQs completed per child over the three age points of 4, 8 and 12 months was variable; therefore, the number of ASQs available for calculations at each age point is different.

Tables 3 and 4 present the regression results as coefficients of multiple correlation using combinations of the ASQ at 4, 8 and 12 months to predict performance on the 18-22 month Bayley Mental Scale (Table 3) and Psychomotor Scale (Table 4). Overall predictability was mild to moderate for each of the scales, indicating that the majority of the variability in the BSID-II still needs to be explained after accounting for ASQ performance. The combination of the 4 and 12 month ASQ data points was the strongest predictor of the 18-22 month BSID-II Mental Scale (R² = 0.23; R = 0.48) and Psychomotor Scale (R²-= 0.32; R = 0.57).

Table 3.

Coefficients of Multiple Correlation between ASQs and BSID-II MDI

Age at ASQ (months)	n	R	BSID-II Mean	p-value
4	73	0.41	80.55	0.002
8	93	0.45	81.72	<0.001
12	103	0.45	81.76	<0.001
4 + 8	45	0.29	83.09	0.050
4 +12	42	0.48	84.62	0.001
8+12	67	0.44	83.33	<0.001
4+8+12	33	0.47	85.70	0.006

Table 4.

Coefficients of Multiple Correlation between ASQs and BSID-II PDI

Age at ASQ (months)	n	R	BSID-II Mean	p-value
4	75	0.46	85.75	<0.001
8	93	0.36	85.97	0.002
12	104	0.44	86.21	<0.001
4+8	45	0.40	88.84	0.006
4+12	43	0.57	89.95	<0.001
8+12	67	0.41	86.91	<0.001
4+8+12	33	0.44	90.42	0.011

ASQ Use as a Population Tracking and Retention Mechanism

The rates of return for the 18-22 month follow up visit were compared with numbers of completed ASQs per participating research center. Using a Spearman correlation coefficient, a significant positive correlation (r_s =0.71) was found when 2 or more ASQs were completed per child prior to the 18-22 month evaluation as seen in Figure 1.

Figure 1.

Per center comparison between the percentage of children who had 2 or more ASQs with the percentage of BSID-II evaluations completed (r_s=0.71). Each data point represents one research center.

DISCUSSION

While the Ages and Stages Questionnaire holds promise for supporting the efforts of measuring the development of young children born ELBW, the clinical epidemiological characteristics (sensitivity and specificity) of the ASQ in relationship to the BSID-II did not reach suggested levels of 80%. ¹⁸ Although the values came close to meeting suggested guidelines for accurate classification of children, the number of children who would be correctly identified as not having a developmental delay (specificity) is not strong enough to assert that the ASQ provides accurate classification when compared to the BSID-II. Specificity values declined even further when the cutoff score for failure on the ASQ was shifted from 2 SD to 1 SD.

Skellern, et al.¹³ reported overall sensitivity and specificity rates for the ASQ of 90% and 77%, respectively. Their study used an ASQ cutoff score of >2 SD, but adjusted the cutoff score of the evaluation tools (BSID-II, Griffith Mental Development Scales, and the McCarthy Scales of Children’s Abilities General Cognitive Intelligence Scale) to >1 SD. When we shifted to a combination of ASQ cutoff scores of >2 SD and cutoff scores of >1 SD for the BSID-II, sensitivity dropped to 63%, while the specificity improved toward acceptable levels (75%). It is interesting to note that when Skellern et al. used either the Griffith Mental Development Scales or the McCarthy Scales of Children’s Abilities General Cognitive Intelligence Scale, they achieved much higher sensitivity and specificity values for the ASQ than they did when using the BSID-II as the index evaluation tool. A number of studies have attempted to determine whether scores obtained during the first 2 years of life, using professionally-administered developmental measures such as the BSID-II, can predict later outcomes. Most of these studies reveal weak correlations. ^{18, 19, 20, 21}

For this study, the Ages and Stages Questionnaires completed at 4, 8, and 12 months had mild to moderate correlations with the MDI and PDI of the BSID-II administered at the 18-22 month corrected age point. It is interesting to note that correlations with the PDI improved when using either the 4 or 12 month ASQ results, or when the 4 and 12 month ASQ scores were added in combination.

The use of the ASQ appeared to support efforts to retain infants in the PROPHET study until the time of the 18-22 month professionally administered developmental evaluation. A significant positive correlation (r_s =0.71) was found between a family’s completion of two or more ASQs and their return for the professional evaluation. The possibility exists, however, that families, who are motivated to return to a center for an evaluation of their child’s development, also would be more likely to complete and return ASQ questionnaires. It also should be noted that more frequent family contact in any form improved follow-up rates. One urban center had a high follow-up rate with few ASQs completed. Their protocol was to bring children in to the follow up clinic every three months. While frequent returns to the hospital/clinic may be physically and financially possible in more urban areas, this might not be feasible in more rural settings.

Most of the research coordinators indicated that they liked using the ASQ because it gave them a reason for contacting the families. They related that families who completed the ASQs appreciated the knowledge gained about their child’s developmental progress. Several coordinators indicated that information gathered through the ASQ process helped families better understand their child’s developmental status so that developmental findings determined at the study evaluation, were not unexpected and thus were less traumatic for the family.

One limitation of this study was the relatively small number of ASQs at each of the earlier age points. This fact might have influenced the results regarding the ASQ’s ability to predict BSID-II scores at 18-22 months. These smaller numbers also did not allow for examination of any possible influences related to sociodemographic factors.

In addition, it should be noted that since the completion of this study, the ASQ-3²² and the BSID-III²³ have been published. The ASQ-3 has additional age intervals, cutoff scores have been revised, and the scoring sheet displays the cutoff point for 1 SD below the mean. The BSID-III contains 5 subtests including motor, language, cognitive, social - emotional and adaptive behavior, which may allow for more direct comparisons between the domains of the ASQ-3 and the BSID-III in future studies.

Another relevant comparison worthy of consideration is that between the BSID-III and the BSID-II²⁴, where significant differences have been shown. Mean composite scores on the BSID-III are reported to be approximately 7 points higher than mean index scores on the BSID-II.²⁴ Depending upon one’s view of the relative strengths of each tool, either the BSID-II overestimates delays or as Anderson et al²⁵ indicate, the BSID-III “seriously underestimates” developmental delay. It appears that more work needs to be done to discover which of the available assessment tools, the BSID-II, BSID-III, ASQ, or others, best documents current developmental status and in addition is the best predictor of child outcomes.

In summary, the results of this study indicate that the ASQ does not have strong enough clinical epidemiological properties to support replacing the BSID-II for NICU follow-up studies. More positively the results of the ASQ may give researchers some indication of the child’s development if the family does not return for the professionally administered evaluation. The ASQ also helps families to feel included in the process of monitoring their child’s development and provides them with valuable developmental information. It is possible that the periodicity of the ASQ process may encourage families to continue their connection with the research center and thus may be a useful adjunct to prospective neurodevelopmental research outcome studies as an effective tracking and population-retention mechanism.

Accurate determination of developmental outcomes of newborns cared for in the NICU is critical to informing ongoing care. However, the question must be posed as to whether the current paradigm of a one-time professionally administered evaluation, prior to 2 years of age, is yielding accurate information about developmental outcomes. As previously noted, developmental evaluations performed in the early years have not been found to be predictive of later outcomes.^18,19,20,21 The question needs to be raised as to whether a periodic parent-administered measure, such as the ASQ, may yield better predictive information. Studies contrasting early developmental findings gathered through the use of professionally administered evaluation tests with results from parent-completed measures would need to be conducted to determine each method’s ability to predict long term developmental outcomes.

Abbreviations

BSID-II

Bayley Scales of Infant Development II

ASQ

Ages and Stages Questionnaire

PDI

Psychomotor Development Index

MDI

Mental Development Index

NICU

Neonatal Intensive Care Unit

PPV

Positive Predictive Value

ELBW

extremely low birth weight

NPV

Negative Predictive Value