Development of a 12 Month Screener Based on Items from the Bayley II Scales of Infant Development for use in Low Middle Income Countries

Abstract

Objectives

The purpose of the current study was to adapt the Bayley Scales of Infant Development II for use as a screening measure that could be used by health care professionals in Low Middle Income (LMI) countries with 12 month old infants to determine if they needed further assessment and early intervention.

Methods

The adaptations were made as part of a larger study of children participating in a home-based early intervention program in India, Pakistan, and Zambia. Using Item Response Theory, a brief 12 months screener, with excellent sensitivity and specificity was identified.

Results

The proposed 12 month screener contains 7 mental/cognitive items and 5 motor items. Children who cannot perform more than 3 items on the mental scale (sensitivity 79%, specificity 85%) and/or 3 items on the motor scale (sensitivity 96%, specificity 95%) should be referred for further assessment.

Conclusion

This screener can reliably be used to determine if a child needs further developmental assessment.

Infants born to families in developing countries are at greater risk for experiencing developmental delays with very few procedures in place to detect and intervene [1]. A number of factors have been associated with increased risks for developmental delays including perinatal and neonatal factors, consanguinity, seizure disorders, poor nutrition, and traumatic injuries [2]. Estimates of developmental delays vary greatly among studies due to differing definitions of delays, the lack of standardized assessments to evaluate child development across cultures, and the high rates of home birth without medical follow-up [3]. There is a significant need for objective screening measures that can be administered in a variety of cultures to children for the purpose of identifying those at-risk for delays and to initiate interventions to improve areas of delayed development. The purpose of the current study was to adapt the Bayley Scales of Infant Development for use as a screening measure with12 month old infants. The adaptations were made as part of a larger study funded by an NIH grant examining outcomes for children participating in a home-based early intervention program in India, Pakistan, and Zambia.

Rates of Developmental Delay

Estimated rates of disabilities in developing countries vary greatly depending on the method of assessment, the criteria used to determine delays, and the subgroups being studied [3]. Studies conducted in India report rates ranging from 5.8% to 12.7% [4, 5]. The lower of these two estimates was specific to children diagnosed with Mild or Severe Intellectual Disability [5], while the higher estimate was based on a study examining the effect of social status on disability rates and therefore only included two of the lowest socioeconomic classes in India. The lowest SES group had a disability rate of 17.2%, while the slightly higher SES group had an overall disability rate of 8.4%. Estimated rates of Intellectual Disabilities in Pakistan range from 3.6% to 6.2% [3]. Another study conducted in Pakistan found that almost 7% of 6-10 year-old children assessed met criteria for mild Intellectual Disability. In addition, 20% of children assessed had some form of delay (e.g., speech/language, hearing difficulty, motor delays, vision problems, or learning disorders [6]). Rates of delays and Intellectual Disabilities vary by country in Africa; however, estimates of Intellectual Disability in Zambia are approximately 3.5% [5]. An estimated 1.13% of the population in the United States has a diagnosed developmental disability with estimates of Intellectual Disability approaching .78% of non-institutionalized individuals [7].

Risk Factors for Developmental Delay

Preterm birth has been shown to result in an increased risk for delays in cognitive development, language skills, and academic achievement [8, 9]. Global estimates indicate that approximately 12.9 million births in 2005 were preterm births. 85% of preterm births were in Asia and Africa with the highest concentrations in Southern Asia and Eastern and Western Africa [10]. A study conducted in Bangladesh sought to evaluate rates of neurodevelopmental impairments (NDI) in a sample of preterm infants based on evaluations by physicians using a neurodevelopmental exam and psychologists using an adapted version of the Bayley Scales of Infant Development II [11]. The results of those evaluations indicated that mild NDIs were found in 45% of the infants and serious impairments were found in 23% of infants. Seventy-two percent of children diagnosed by a physician with an NDI, had more than one area of impairment. Birth related injuries are a second risk factor for developmental delays. In addition to preterm birth, difficulties such as asphyxia, infection, and lack of access to medical care at birth are all potential risk factors for developmental delays that place children in developing countries at higher risk for such delays [2].

In addition to perinatal and neonatal factors, other medical problems related to developmental delays may exist at higher rates in developing countries. A limited body of research suggests that seizure disorders may be present at higher rates in developing countries and that the seizures are less likely to be managed through medication [12]. Consanguinity is also present at higher rates in developing countries and has been linked to disabilities present in childhood. Poor nutrition, poverty and higher rates of traumatic injury are also related to higher risk for developmental delay [2].

Early Detection

The importance of early developmental screening has been established in the United States as an effective intervention to improve outcomes for children with delays. Early identification results in early intervention, which has been shown to improve outcomes for children with suspected developmental delays. Research has shown that surveillance alone is not the most effective method for detecting developmental delays. Only 50% of children with delays are detected by informal impressions made by physicians in the United States [13]. Parent report measures can detect 70-80% of children with delays; however, literacy and language barriers can negatively impact the accuracy of these reports [14]. For these reasons, brief standardized screening measures administered by professionals appears to be one of the most effective methods for identifying children at-risk for delays [15]. The American Academy of Pediatrics released new standards for developmental surveillance and screening in 2006 that encouraged the physicians to add manual measures of screening to their current practice for evaluating development in the United States [16].

Identifying children at-risk for delays presents a number of challenges in developing countries. First, screening measures developed locally with standardized norms are not typically available. Currently, there are a limited number of options for use in multiple cultures to assess early developmental milestones. Two such measures are the Rapid Neurodevelopmental Assessment (RNDA) developed in Bangladesh [17, 18] and the 10 questions task developed by Durkin and colleagues [12]. The 10 questions task relies on parent report of children’s abilities compared to other children. The RNDA is an objective measure made up of multiple tasks designed to detect neurodevelopmental impairments and is administered by a trained professional. Second, many children do not regularly see a medical or mental health professional in the first three years of their lives, making surveillance difficult. Finally, literacy and language barriers present additional challenges when attempting to implement parent report measures in rural settings in LMI countries. In fact, a study by Schell et. all [19] found that poor female literacy was a major contributor to infant mortality rates in LMI countries.

Current Study

The purpose of the current study was to develop a culturally sensitive screening measure that can be used with 12-month old children in a variety of LMI countries. Children identified as delayed by the screeners could then be referred for further evaluation.

METHODS

Study Design

The Brain Research to Ameliorate Impaired Neurodevelopment Home-based Intervention Trial (BRAIN-HIT, clinicaltrials.gov ID# NCT00639184) [20] parallel design randomized controlled trial was implemented in two populations, infants with birth asphyxia who required ventilation as part of their resuscitation and infants who did not require resuscitation. Infants in each cohort were randomized to one of two trial conditions (early developmental intervention plus health and safety counseling or to health and safety counseling only). Mothers in both the control and intervention groups received health and safety counseling during every two-week home visits. Among the intervention group, a home-based, parent-implemented early intervention model was selected to strengthen parent-child interaction. As part of this intervention home visitors introduced playful interactive learning activities depicted on cards given and modeled to the parents. Developmental skill areas addressed with the curriculum included cognitive and fine motor, social and self-help, gross motor, and language domains. The trial was approved by the institutional review boards at the University of Alabama at Birmingham, Research Triangle Institute (RTI) International, and each participating clinical site. Details on the trial design have been published elsewhere [20, 21].

Participants

Infants who had received bag and mask resuscitation at birth born in rural communities in three clinical sites (India, Pakistan, and Zambia) in the FIRST BREATH Trial were screened for enrollment into this trial. Infants were ineligible if they met any of the following exclusion criteria: 1) weighed < 1500 grams at birth, 2) their neurological examination at 7 days was severely abnormal (grade III by Ellis classification), 3) the mother was < 15 years of age or unable/unwilling to participate, or 4) the mother was not planning to stay in the study communities for the following three years. Infants with birth asphyxia (Resuscitated) and infants without birth asphyxia or other perinatal complications (Non-resuscitated) matched for country and chronological time were randomly selected during the first week after birth using a computer generated list from infants enrolled in the FIRST BREATH Trial. Consent was obtained during the second week after birth following the 7-day neurological assessment. Randomization was performed by the data center to assure allocation concealment using block randomization.

In addition to the above exclusion criteria, subjects included in this sub-study must have completed the 12-month evaluation between 11 and 16 months according to the child’s chronological age for term children and the corrected age for preterm children. The corrected age was calculated by subtracting the amount of time in months and days the child was premature from the child’s chronological age. Children who received a score below 50 were considered incomplete and therefore were not included in this sub-study.

Measure

The Bayley Scales of Infant Development-II (BSID-II) [22] was selected as the source of items for the screener because it has been used extensively in various low- and middle-income countries [23]. It underwent extensive pre-testing at each site to verify validity in the local context and a few items were slightly modified to make the BSID-II more culturally appropriate (e.g., images of a sandal instead of a shoe and a culturally appropriate dwelling instead of a hours, a culturally appropriate children’s book). The BSID-II was administered directly to each child in the appropriate language using standardized procedures. The BSID-II was administered as part of a larger assessment at 12 months of age and was performed by evaluators trained in a 4-day workshop prior to the 12-month evaluation. The evaluators (physicians, nurses and psychologists) were familiar with the local language and culture and were masked to the treatment assignments. During the 4 day training each evaluator learned, practiced and administered the Bayley to assure they could administer a reliable and valid assessment.

The pool of possible items for the screener included the BSID-II items equivalent to 11 to 16 months of age (items 66 to 111 on the BSID-II Mental evaluation and items 54 to 79 on the Motor evaluation). Items were dichotomized as Completed (‘credit’) and Not Completed (‘no credit’/‘refused’/‘omit’/‘caregiver report’). The children received credit for items not administered because they were from an earlier item set (e.g. if the evaluator administered the 12-month item set, the 11-month items were coded as Completed). Items not answered because they were from an item set above the administered items were counted as Not Completed (e.g. if the evaluator administered the 12-month item set, the 13 through 16 month items were coded as Not Completed).

Statistical Analyses

We randomly split the sample into a development sample including 65% of the participants and a validation sample including 35%. Using the development sample, we conducted a series of analyses to assess the psychometric properties of the BSID-II items and select the best performing and most clinically relevant items for the mental and motor screeners. First, we computed the percentage of children who performed the items correctly and estimated two-parameter logistic (2PL) item response theory (IRT) parameters for each set of items (mental and motor) [24]. Next, because the screeners should identify children who have delayed development, as measured by MDI or PDI < 85, we computed the percentage that could perform the mental items for children with MDI < 85 vs. MDI ≥ 85 and the percentage of children who could perform the motor items for children with PDI < 85 vs. PDI ≥ 85. In addition, we calculated the odds ratios of MDI < 85 for each of the mental items and of PDI < 85 for each of the motor items. We then conducted classification and regression tree (CART) analyses to identify items that were the most predictive of MDI (or PDI) < 85 and to explore whether there are any interactions among items that may suggest alternative scoring algorithms are needed for the screeners. Each model included MDI (or PDI) < 85 as the outcome with all of the mental (or motor) items as possible predictor variables.

Using the information from these analyses, as well as content and clinical considerations, we then selected the items for the screeners. Ideally, items on the screener should be able to distinguish between participants who are delayed vs. not delayed (i.e., MDI/PDI < 85 vs. ≥ 85) based on having large discrepancies in percentages correct across these two groups, large odds ratios, and high IRT discrimination parameters (> 1). In terms of content, at least one item from each skill area should be included on the screener (e.g., both language and cognitive skills for the mental screener). In addition, we considered whether the items were deemed to be clinically relevant based on clinician input, whether they were feasible to administer in developing countries with limited resources (e.g., were culturally appropriate and did not require items that may be difficult to obtain), and whether they were also included on the Bayley III. Lastly, to foster adoption and use of the screener and lower burden, we restricted the number of items to no more than 10 for the mental screener and 5 for the motor screener.

After identifying the items for the screeners, scores were computed as the total number of items on the screener that the child was able to perform. We then conducted a receiver operating characteristic (ROC) curve analysis to identify a cut point for scores on the screeners that maximized the combined value for sensitivity and specificity for identifying children with MDI (or PDI) < 85.

After developing the screeners, we tested their sensitivity and specificity for identifying children with developmental delays among the remaining 35% of the participants (the validation sample). We also assessed the item characteristics, including the IRT parameters and percentage of correct responses, among the validation sample to examine the consistency of the performance of the screeners.

RESULTS

A total of 540 births were screened for inclusion in the BRAIN-HIT. Of these, 438 were eligible and 407 (93%) consented to participate in the core intervention study. Of these, 164 were resuscitated at birth and 243 were not resuscitated. Twenty seven percent of those enrolled were from Zambia, 40% from India and 33% from Pakistan. Ninety two percent (376/407) of the children completed the 12-month BSID-II, with 323/407 (79%) completing it between 12 and 16 months. These 323 children are included in this sub-study (128 resuscitated and 195 non-resuscitated children with 27% of children from Zambia, 36% from India and 37% from Pakistan). The characteristics of the BSID-II mental and motor items for these children are shown in Tables 1 and 2.

Table 1.

Characteristics of BSID-II Mental Items: Development Sample (N=211)

Item #	Description	% Correct			Predict MDI ≥ 85		IRT Parameters
		All	MDI < 85	MDI ≥ 85	OR (95% CI)	p	Discrimination (a)	Difficulty (b)
66	Rings bell purposely	100	100	100	Non-est		Non-est
67	Lifts cup by handle	99	98	99	4.23 (0.26, 69.00)	0.312	20.68	−2.26
68	Uses gesture to make wants known	100	100	99	Non-est		1.24	−4.92
69	Looks at pictures in book	99	98	99	2.10 (0.19, 23.74)	0.549	1.28	−3.92
70	Listens selectively to two familiar words	99	100	99	Non-est		0.70	−6.99
71	Repeats vowel-consonant combination	96	95	96	1.19 (0.24, 5.97)	0.829	0.69	−4.81
72	Looks for contents of box	98	98	98	1.39 (0.14, 13.73)	0.777	1.75	−3.03
73	Turns pages of book	85	59	91	7.32 (3.23, 16.56)	<0.001	1.73	−1.49
74	Puts one cube in cup	93	85	95	3.07 (1.03, 9.17)	0.045	1.84	−2.07
75	Attempts to secure three cubes	80	59	85	4.11 (1.94, 8.72)	<0.001	1.53	−1.29
76	Jabbers expressively	77	61	81	2.76 (1.32, 5.76)	0.007	0.85	−1.65
77	Pushes car	88	63	94	9.23 (3.75, 22.73)	<0.001	1.82	−1.67
78	Vocalizes four different vowel-consonant combinations	61	34	67	3.93 (1.91, 8.07)	<0.001	1.40	−0.41
79	Fingers holes in pegboard	95	85	97	5.66 (1.63, 19.58)	0.006	0.87	−3.71
80	Removes lid from box	87	68	92	5.17 (2.20, 12.17)	<0.001	0.85	−2.55
81	Responds to spoken request	90	73	94	5.87 (2.29, 15.03)	<0.001	1.61	−1.93
82	Suspends ring by string	72	39	79	6.03 (2.91, 12.50)	<0.001	1.40	−0.90
83	Pats toy in imitation	80	61	85	3.54 (1.67, 7.53)	0.001	0.87	−1.84
84	Finds one object	72	41	79	5.25 (2.55, 10.82)	<0.001	1.48	−0.87
85	Removes pellet from bottle	87	73	90	3.30 (1.41, 7.75)	0.006	0.95	−2.30
86	Puts three cubes in cup	88	68	92	5.61 (2.35, 13.35)	<0.001	2.09	−1.55
87	Places one peg repeatedly	56	24	64	5.54 (2.54, 12.06)	<0.001	1.07	−0.28
88	Retrieves toy (clear box 1)	64	20	75	12.18 (5.23, 28.40)	<0.001	1.71	−0.50
89	Puts six beads in box	79	51	86	5.79 (2.74, 12.26)	<0.001	1.86	−1.13
90	Places one piece	24	2	29	16.20 (2.17,121.11)	0.007	0.99	1.42
91	Scribbles spontaneously	63	34	69	4.38 (2.12, 9.02)	<0.001	0.83	−0.70
92	Closes round container	61	32	68	4.63 (2.22, 9.63)	<0.001	0.86	−0.60
93	Places circle piece	12	2	14	6.57 (0.86, 50.09)	0.069	1.04	2.29
94	Imitates word	59	44	63	2.17 (1.09, 4.33)	0.028	0.36	−1.06
95	Puts nine cubes in cup	72	54	76	2.72 (1.34, 5.51)	0.006	1.15	−1.01
96	Finds toy under reversed cups	51	12	60	10.80 (4.04, 28.90)	<0.001	1.05	−0.02
97	Builds tower of two cubes	35	12	41	4.92 (1.84, 13.16)	0.002	1.03	0.75
98	Places pegs in 70 seconds	20	7	24	3.90 (1.14, 13.30)	0.030	1.46	1.31
99	Points to two pictures	20	5	24	6.20 (1.43, 26.79)	0.015	1.05	1.59
100	Uses two different words appropriately	28	5	34	10.09 (2.35, 43.27)	0.002	1.34	0.95
101	Shows shoes, other clothing, or object	19	5	22	5.61 (1.30, 24.32)	0.021	2.41	1.14
102	Retrieves toy (visible displacements)	26	0	32	Non-est		1.98	0.92
103	Imitates crayon stroke	12	2	15	6.90 (0.91, 52.45)	0.062	2.60	1.43
104	Uses rod to attain toy	32	7	38	7.84 (2.33, 26.44)	0.001	1.46	0.74
105	Retrieves toy (clear box 2)	24	2	29	16.67 (2.23,124.57)	0.006	1.13	1.28
106	Uses words to make wants known	6	0	8	Non-est		7.10	1.52
107	Follows directions (doll)	12	5	14	3.20 (0.73, 14.14)	0.124	1.99	1.56
108	Points to three of doll’s body parts	4	2	4	1.72 (0.21, 14.36)	0.618	2.63	2.06
109	Names one picture	1	0	2	Non-est		24.79	1.80
110	Names one object	2	0	2	Non-est		25.20	1.76
111	Combines word and gesture	7	2	8	3.31 (0.42, 26.03)	0.255	8.09	1.48

Note: Non-est=non-estimable; The IRT threshold (b) parameter indicates at what point along the ability continuum, there is a 50% probability that children will be able to perform the item. The IRT discrimination (a) parameter indicates how well the item distinguishes among children above vs. below the threshold.

Table 2.

Characteristics of BSID-II Motor Items: Development Sample (N=211)

Item #	Description	% Correct			Predict PDI ≥ 85		IRT Parameters
		All	PDI < 85	PDI ≥ 85	OR (95% CI)	p	Discrimination (a)	Difficulty (b)
54	Walks sideways while holding on to furniture	91	64	100	Non-est		4.92	−1.52
55	Sits down	94	74	100	Non-est		4.20	−1.74
56	Uses pad of fingertips to grasp pellet	99	94	100	Non-est		2.71	−2.68
57	Uses partial thumb opposition to grasp rod	98	94	99	10.21 (1.04,100.49)	0.046	1.85	−2.96
58	Grasps pencil at farthest end	95	92	96	2.25 (0.61, 8.30)	0.225	0.41	−7.52
59	Stands up 1	74	18	92	51.86 (20.72,129.82)	<0.001	4.54	−0.69
60	Walks with help	90	60	99	52.98 (11.77,238.61)	<0.001	4.49	−1.40
61	Stands alone	73	32	85	12.13 (5.81, 25.31)	<0.001	3.24	−0.66
62	Walks alone	53	10	66	17.83 (6.69, 47.51)	<0.001	8.34	−0.04
63	Walks alone with good coordination	42	4	54	28.22 (6.63,120.05)	<0.001	6.51	0.24
64	Throws ball	67	40	76	4.69 (2.40, 9.18)	<0.001	0.87	−0.96
65	Squats briefly	59	10	74	25.50 (9.49, 68.53)	<0.001	3.50	−0.21
66	Walks up stairs with help	58	10	73	23.93 (8.92, 64.20)	<0.001	1.85	−0.25
67	Walks backward	28	2	37	28.33 (3.81,210.47)	0.001	6.73	0.61
68	Stands up 2	49	6	63	26.37 (7.86, 88.44)	<0.001	1.79	0.04
69	Walks down stairs with help	47	8	59	16.55 (5.68, 48.17)	<0.001	1.37	0.14
70	Grasps pencil at middle	66	32	77	7.12 (3.54, 14.32)	<0.001	0.92	−0.86
71	Walks sideways	24	2	31	22.07 (2.96,164.33)	0.003	3.80	0.78
72	Stands on right foot with help	43	2	56	62.11 (8.37,460.86)	<0.001	1.08	0.33
73	Stands on left foot with help	28	0	37	Non-est		1.60	0.86
74	Uses pads of fingertips to grasp pencil	21	8	25	3.93 (1.33, 11.59)	0.013	0.88	1.71
75	Uses hand to hold paper in place	6	4	7	1.76 (0.38, 8.22)	0.472	0.72	4.09
76	Places 10 pellets in bottle in 60 seconds	29	22	31	1.60 (0.76, 3.37)	0.220	0.67	1.48
77	Runs with coordination	8	0	11	Non-est		2.06	1.83
78	Jumps off floor (both feet)	0	0	1	Non-est		1.51	4.26
79	Walks up stairs alone, placing both feet on each step	1	0	1	Non-est		2.18	3.08

Note: Non-est=non-estimable; The IRT threshold (b) parameter indicates at what point along the ability continuum, there is a 50% probability that children will be able to perform the item. The IRT discrimination (a) parameter indicates how well the item distinguishes among children above vs. below the threshold.

Mental Screener

The characteristics of the BSID-II mental items included in this study are shown in Table 1. By design of the assessment, earlier items are less difficult (i.e. have higher percentages correct) than items administered later. The goal of the screener is to identify children with developmental delays as measured by MDI scores < 85. Items with large discrepancies in percentages correct between those with MDI < 85 vs. MDI ≥ 85 would be useful for identifying delays and are candidates for inclusion on the screener. For example, 34% of children with MDI < 85 were able to perform item 78 (vocalizes four different vowel-consonant combinations) compared to 67% of those with MDI ≥ 85. The odds ratios provide similar information. For example, children who can perform item 96 (finds toy under reversed cups) have over 10 times the odds of having an MDI ≥ 85 compared to children who cannot perform this task (OR(95% CI)=10.80 (4.04, 28.90), p < 0.001).

The IRT difficulty parameter indicates where children with the corresponding ability level have a 50% probability of answering the question correctly while the discrimination parameter indicate how well the item can distinguish between children above or below this ability level. Because the screener is designed to identified children with substantial delays, candidate items should be on the lower end of the scale (i.e., difficulty levels less than 0) and have higher discrimination parameters. For example, item 77 (pushes car) has a difficulty parameter of −1.67, indicating it is below average in difficulty (0 indicates average difficulty), and a high discrimination parameter (1.82), making it a candidate for inclusion in the screener, all else being equal.

In the CART analysis, only one item, item 88 (retrieves toy - clear box 1) entered into the model, suggesting that it is the most predictive of MDI < 85 and should be included on the screener. The lack of entry of other items into the model suggests that there are no significant interactions among the items.

Incorporating the statistical results, as well as clinical considerations, we selected the following eight items for the mental screener: items 73, 77, 78, 80, 81, 84, 88, and 96 (see Table 3). Based on results among the development sample, all items selected for the screener significantly distinguished between children with MDI < vs. ≥ 85 based on odds ratios, all items except item 80 had IRT slopes greater than 1, and the items covered a range of difficulty levels (Table 1). Although item 80 had a slope below 1 (slope=0.85), this item was considered clinically relevant because it captures the concept of object permanence and therefore was included on the screener.

Table 3.

Characteristics of Items on Mental and Motor Screeners: Validation Sample (N=112)

Item #	Description	% Correct			Predict MDI/PDI ≥ 85		IRT Parameters
		All	MDI < 85	MDI ≥ 85	OR (95% CI)	p	Discrimination (a)	Difficulty (b)
Mental
73	Turns pages of book	86	79	87	1.78 (0.44, 7.26)	0.419	1.23	−1.76
77	Pushes car	92	71	95	7.44 (1.72, 32.28)	0.007	2.19	−1.71
78	Vocalizes four different vowel-consonant combinations	57	29	61	3.95 (1.16, 13.49)	0.029	1.44	−0.23
80	Removes lid from box	85	43	91	13.19 (3.74, 46.54)	<0.001	1.52	−1.49
81	Responds to spoken request	84	64	87	3.63 (1.05, 12.55)	0.041	0.65	−2.70
84	Finds one object	72	36	78	6.22 (1.89, 20.48)	0.003	1.58	−0.82
88	Retrieves toy (clear box 1)	58	29	62	4.12 (1.21, 14.09)	0.024	1.75	−0.24
96	Finds toy under reversed cups	49	7	55	15.96 (2.01, 126.76)	0.009	1.68	0.07
Motor
59	Stands up 1	78	13	95	147.00 (30.54, 707.61)	<0.001	3.64	−0.84
60	Walks with help	88	42	100	non-est		4.02	−1.33
65	Squats briefly	59	8	73	29.33 (6.41, 134.34)	<0.001	2.99	−0.21
66	Walks up stairs with help	57	0	73	non-est		2.57	−0.18
72	Stands on right foot with help	35	4	43	17.48 (2.26, 135.25)	0.006	0.88	0.84

Note: Non-est=non-estimable; The IRT threshold (b) parameter indicates at what point along the ability continuum, there is a 50% probability that children will be able to perform the item. The IRT discrimination (a) parameter indicates how well the item distinguishes among children above vs. below the threshold.

Using an ROC analysis, we selected a cut point of 4 for the screener; children with scores of 4 or less should be evaluated further. At this cut point, the screener had sensitivity of 73% and specificity of 85% in the development sample (Table 4). We then applied the screener to the validation sample. The screener performed similarly in the new sample with sensitivity of 79% and specificity of 80% (Table 4) and similar item characteristics (Table 3).

Table 4.

Sensitivity and Specificity of Mental and Motor Screeners for Identifying Delays Based on BSID-II MDI or PDI Scores < 85

Screener/Statistic	Development Sample (N=211)	Validation Sample (N=112)
Mental Screener (8 items, cut point=4)
% agreement with MDI < 85	82	79
Sensitivity	73	79
Specificity	85	80
Positive predictive value	54	35
Negative predictive value	93	96
Motor Screener (5 items, cut point=2)
% agreement with PDI < 85	87	96
Sensitivity	90	96
Specificity	86	95
Positive predictive value	67	85
Negative predictive value	97	99

Motor Screener

Characteristics of the BSID-II motor items are shown in Table 2. The motor items were generally more discriminating than the mental items. For example, children who can perform item 59 (stands up 1) had almost 52 times the odds of having a PDI ≥ 85 than those who cannot perform this item (OR (95% CI)=51.86 (20.72, 129.82), p < 0.001). Eighteen percent of children with PDI < 85 can perform this item compared to 92% with PDI ≥ 85 and the item has an IRT discrimination parameter of 4.54. Similar to the CART analyses for MDI, only one item, item 59 (stands up 1) entered into the CART model for PDI (not shown).

Based on the analysis results and content and practical considerations, we selected 5 items for the motor screener: items 59, 60, 65, 66, and 72 (Table 3). All five items met the statistical criteria for inclusion in the screener based on the results with the development sample, having high IRT slopes: large discrepancies in percentages correct between those with MDI < vs. ≥ 85, large odds ratios, and IRT discrimination parameters > 1 (Table 2). In addition, these items were considered clinically relevant and were feasible to administer in developing countries.

A cut point of 2 maximized sensitivity and specificity based on the ROC analysis of the development sample data (sensitivity=90% and specificity=86%). Children with scores of 2 or lower should be referred for further evaluation. When applying the screener to the validation sample, sensitivity and specificity increased with values of 96% and 95%, respectively.

Discussion

The primary purpose of the current study was to identify a small pool of items on the Bayley Scales of Infant Development II for use as a screening measure that could be used by health care professionals in Low Middle Income (LMI) countries with 12 month old infants to determine if they needed further assessment and early intervention. Overall, the items selected for the 12 month screener demonstrated strong psychometric properties resulting in two brief screeners, one for mental (cognitive, language, and fine motor) development (8 items) and one primarily for gross motor development (5 items). The screener was found to have good sensitivity and specificity for identifying children suspected of having mental and motor delays, indicating a full developmental assessment was appropriate. This screener reduces the number of items in the 11 to 16 month range for the mental scale from 46 to 8 and the number of items on the motor scale from 26 to 5, making the screeners more feasible for administering in clinical settings in LMI countries by primary health care professionals with limited time and resources. A child who fails to complete at least 4 of the mental items and 2 of the motor items at their one year pediatric exam should be referred for a complete assessment.

The advantage to this type of screener is that pediatric professionals can actually observe and interact with the child at risk, in lieu of just asking the caretakers to complete a questionnaire or asking them about the child’s development to determine if further assessment is necessary, which is common in other available screeners [12, 17, 18]. In addition, using existing Bayley items presents a unique method to gather developmental information on a child by administering a few items of a standardized test to determine if the child is in need of further assessment. In LMI countries this can save time and assist in identifying children in need of early intervention services. Unlike some of the other screeners for this age noted above, it is a direct administration, as opposed to a parent questionnaire, it can be completed in a very brief encounter, and it results in few false positive or negative results [12, 17, 18]. This type of screener that is directly administered to the child also addresses the issue of mother’s in LMI countries completing screeners and answering questions about developmental milestones where poor female literacy is a major contributing factor to infant mortality rates [19].

Although this screener was determined to have good sensitivity and specificity, it was only validated on a sample from three countries with LMI populations; a replication study in other LMI countries would further verify its usefulness. Finally, a newer version of the Bayley is now available; however, the items used in this screener are consistent with items in the newer Bayley. Nevertheless, a follow-up study comparing the current screener with performance on the new version of the Bayley would be beneficial.

Research Highlights.

• There is a need for objective screening measures to identify infants at-risk for developmental delays in various cultures

• Early detection of developmental delays leading to intervention can improve quality of life

• We developed a 12 month screener using items from the Bayley Scales II for use in Low Middle Income Countries

• The proposed 12 month screener contains 7 mental/cognitive items and 5 motor items

• The screener can reliably be used to determine if a child needs further developmental assessment