Monitoring psychomotor development in a resource-limited setting: an evaluation of the Kilifi Developmental Inventory

Abstract

Background

Modifications made to the Kilifi Developmental Checklist and the psychometric characteristics of the new measure (The Kilifi Developmental Inventory) which assess the psychomotor functioning of children aged 6–35 months are described.

Methods

Two groups of community children (319 rural and 104 urban dwellers) and nine children with neurodevelopmental disorders were recruited for a cross-sectional study.

Results

In both a rural and urban reference population, the inventory showed excellent internal consistency, inter-observer agreement, test-retest reliability and sensitivity to maturational changes. Children with neurodevelopmental impairment and those who were underweight had significantly lower scores than the community sample, attesting to the sensitivity of the measure. Mothers found the assessment procedures acceptable and informative.

Conclusions

The Kilifi Developmental Inventory is a culturally appropriate measure that can be used to monitor and describe the development of at-risk children in resource-limited settings in Kenya.

Introduction

An estimated 200 million children in developing countries fail to achieve their developmental and cognitive potential¹ owing to exposure to multiple risk factors such as infectious diseases, malnutrition and congenital problems.² Early identification and intervention can reduce the impact of impairment;³ however, a shortage of appropriate assessment tools hampers efforts to identify and adequately monitor at-risk children in developing countries.⁴ The current study aims to contribute towards addressing the shortage of adequate assessment measures.

While standardised tests developed in western countries provide ready-made assessment tools, the transfer of western-based tests to a non-western context is associated with significant limitations of test score interpretation.^5,6 One way of over-coming these limitations is to develop tests in situ. This report describes a study in which a locally developed instrument, the Kilifi Developmental Checklist (KDC),⁷ was refined to provide a measure of psychomotor development appropriate for children aged 6–35 months. The KDC, while including a broad range of functioning, did not include items for the infant age range (children <12 months of age), nor was performance evaluated in a reference population. The Kilifi Developmental Inventory (KDI) focuses on psychomotor assessment in children aged 6–35 months, unlike the KDC which assessed four developomental domains in children from 12 months to school age. The specific aims of the current study were (i) to evaluate the reliability, validity and acceptability of the new measure, the Kilifi Developmental Inventory (KDI); (ii) to evaluate the applicability of the KDI for use in an urban community; (iii) to develop reference tables for the KDI, and (iv) to evaluate the sensitivity and adequacy of the KDI in identifying children with developmental impairment. Each aim was addressed in separate sub-studies.

Methods

Study sites and study samples

The studies took place at two sites, the Kenya Medical Research Institute, Centre for Geographic Medicine Research (Coast), Kilifi and Kisauni Location in Mombasa district.

Kilifi site

Kilifi is a mainly rural area where more than 66% of the population live below the poverty line.⁸ The study included families living in a demarcated area in Kilifi that undergoes active 4-monthly demographic surveillance which records births, deaths and movement of individuals. Children who met the following criteria qualified for inclusion: (i) aged 6–35 months, (ii) parents spoke Kiswahili or one of the Mijikenda dialects as their primary language, (iii) children reported no chronic illness in the course of the study, and (iv) parents gave informed consent.

Stratified random sampling was used to identify and recruit study participants through five government-run clinics located across the study area. Stratification was based on age, gender and geographical area. We aimed at having 25 children for each age band, represented by an equal number of boys and girls. Age-bands were based on 3-month groupings, beginning with children aged 6–8 months. Parents were approached for consent until the target number of children was achieved for all strata.

Kisauni site

The second site was Kisauni Location in Mombasa district which is an urban setting. Kisauni Location has the second highest number of people living in poverty in Mombasa, with approximately 47% below the poverty line.⁹ Although the population consists of different ethnic and linguistic groups, Kiswahili is widely spoken as the lingua franca. A network of community representatives was used to identify and approach families with eligible children. To identify other eligible children, a snowballing method was employed whereby mothers identified other families with children in the target age-group.

Modifications to the Kilifi Developmental Checklist (KDC)

The items in the KDC were selected from several sources including the Kenyan Screening Test for Children aged 6 months to 6 years,¹⁰ the Griffiths Mental Development Scales,¹¹ the Movement Assessment Battery for Children,¹² Merrill Palmer Scales of Tests,¹³ the Wessex Revised Portage Checklist,¹⁴ and Wechsler’s Preschool and Primary Scales of Intelligence¹⁵ and tasks suggested by the Shoklo Neurodevelopmental Assessment.¹⁶ Items were selected if (i) success on the action/task was readily determined by the observer, (ii) they demonstrated within-population variance, and (iii) the behaviour of interest could be readily described in the local languages. Psychometric properties were evaluated in a group of children aged 12 months to 9 years admitted to hospital with severe malarial disease.⁷

We audited the performance of children who had completed the KDC in an earlier study⁷ on the items measuring psychomotor functioning, excluding those items which had not been successfully completed by at least one child under 36 months of age. The remaining items were supplemented with tasks suitable for children aged 6–12 months. The new items were largely drawn from the Griffiths Mental Development Scales.¹¹

Further modifications were suggested by a pilot study of the new, more focussed schedule that included 70 community children whose ages ranged from 6 to 35 months. Assessment took place in their homes. We subsequently simplified the original KDC three-point scale (0, cannot do the task; 1, skill emerging; 2, skill established) to a dichotomised scale (0, cannot perform the task; 1, can perform the task) to reduce potential ambiguity.

The Kilifi Developmental Inventory (KDI)

The inventory consists of 69 items administered by an assessor who explains and demonstrates each new task before the child attempts the activity. A summated score is calculated for two functional areas, locomotor skills and eye–hand co-ordination. A detailed instructional manual was produced through an interactive process with the assessment team, standardising the administration procedure in the language of the assessment. The manual includes templates for constructing standardised test materials. During training the assessor is taught to recognise the developmental progression of items and to assign the appropriate score to all items on the inventory, irrespective of whether they are at a simpler or more complex skill level than that demonstrated by the child. Children thereby receive a score for all items in the inventory, regardless of age. For example, if a child cannot stand without support, then all the remaining locomotor items such as walking alone are scored zero, whereas a child who is observed to walk into the assessment session with no support automatically receives a score of 1 for the simpler item, ‘stands without support’.

Procedure

The assessment team was trained by two psychologists (Abubakar and Holding) and assisted by a physiotherapist. This training involved a 2-week familiarisation and skill-training workshop followed by practice in the field. Data were collected between September 2004 and June 2005. Children were seen at home accompanied by their primary caregivers. Any child who was sick on the appointed day was given an alternative assessment date. The Kenya Medical Research Institute National Scientific and Ethical Committees approved the study. Informed consent was obtained from all families of study participants.

Analysis

Data were double-entered in FoxPro and verified before being transferred to SPSS version 12 for analysis.

Results

Study One: Evaluation of the psychometric properties of the KDI in a representative rural community sample

Sample

The rural sample consisted of 319 children (159 girls) with a mean (SD) age of 19.06 (8.46) months (range 6–35). A random sample of 34 children (18 girls), mean (SD) age 17 (8.4) months (range 6–34), was selected from the main sample to evaluate inter-observer reliability. Forty-one children (21 girls), aged 7–34 months with a mean (SD) of 24 (8.0) months, were involved in evaluating test-retest reliability. The mean test-retest interval was 4 weeks (range 3–7). Criterion validity was evaluated by comparing performance on the KDI in the youngest age band with a developmental report elicited from the children’s parents. Eighty-seven children (47 girls) with a mean age of 9 months (range 5–15) were involved. The mean interval between the parental report and KDI assessment was 1 week (range 1–3).

Measures

In addition to the KDI, the Developmental Milestones Checklist was administered to parents of children in the criterion validity arm of this study. Details of the development and psychometric properties of this checklist are reported elsewhere (Abubakar et al., submitted). A trained community health worker completed the checklist in an interview with the child’s main caregiver. Summed scores were calculated for each domain separately; in addition, an overall developmental score can be computed.

Children were weighed on a SECA digital scale. Weights were recorded following three readings that were identical up to one decimal point. Weight-for-age was computed using the WHO Anthro 2005 software and reference population.¹⁷ Being underweight was defined as having a weight-for-age score <−2.00 SD in the reference population score distribution.

Analysis

Reliability was evaluated through Cronbanch Alpha and intraclass correlation co-efficients. The acceptability of the values was judged according to criteria set by Ciccheti and colleagues,¹⁸ i.e. that values above 0.70 are acceptable while those above 0.90 are excellent, and that intraclass correlation co-efficient (ICC) values between 0.60 and 0.75 are good and higher values excellent.

Results

The mean (SD) time of test administration was 62.74 (17.32) minutes, less in younger children who performed fewer tasks. Table 1 summarises the psychometric properties of the KDI. We found acceptable levels for all aspects of reliability evaluated. Attesting to the validity of the measure are the results that show a significant association between performance level and chronological age and anthropometric status, and a lack of association with gender. A significant correlation was also found with parental report.

TABLE 1. Psychometric characteristics of the Kilifi Developmental Inventory.

Statistics	n	Locomotor	Eye–hand	Psychomotor
Maximum possible score		35	34	69
Reliability
Internal consistency (α)	319	0.92	0.93	0.96
Retest reliability (ICC)	41	0.87	0.95	0.96
Inter-observer (ICC)	34	0.92	0.94	0.98
Validity
Gender		t (319) 50.30, ns	t (319) 50.36, ns	t (319) 50.34, ns
Correlation with age	319	0.91‡	0.92‡	0.93‡
Correlation with maternal reports	87	0.84‡	0.72‡	0.80‡
Sensitivity
Neurodevelopmental disorders	113	t (113) 513.34‡	t (113) 510.66‡	t (113) 513.05‡
Underweight	319	t (319) 53.03†	t (319) 52.45*	t (319) 53.14†

SD, standard deviation; ICC, intraclass correlation co-efficient; t, t-test; ns, not significant;

^*p<0.05,

^†p<0.01,

^‡p<0.001.

The acceptability of the measure to the community was evaluated through a series of focus group discussions. Mothers felt that the measures had high face validity and that their children’s performance of the tasks adequately characterised their developmental level. Mothers indicated that they found the test of sufficient value to be willing to participate in future assessments.

Study Two: Psychometric properties of the KDI in an urban and equivalent rural community

The applicability of the KDI for use in children in an urban environment was evaluated by comparing the psychometric properties of the test in the two sub-samples. Logistical and financial constraints necessitated restriction of the age range sampled.

Sample

All children were aged between 24 and 35 months. The urban sample consisted of 104 children (53 girls) with a mean (SD) age of 29.11 (3.53) months (spread 24–35). There were 99 children (52 girls) in the rural sample within the same age range and their mean (SD) age was 29.47 (3.53) months (spread 24–35).

Materials and procedures

Children at both sites were assessed in their homes by the same assessment team. Test-retest reliability was evaluated using a randomly selected sample of 19 children visited after a 3-week interval.

Results

Results of comparison between the rural and urban populations are shown in Table 2. The psychometric characteristics were good in both populations. All reliability and validity data were within the acceptable range. There were no significant differences between urban and rural children in their performance of the test.

TABLE 2. Comparison of data from the Kilifi and Kisauni children aged 24–35 months.

Sample	Statistics	Locomotor	Eye–hand	Psychomotor
Kilifi (n=99)
	Mean (SD)	22.97 (2.91)	26.73 (3.12)	49.12 (5.11)
	α	0.74	0.77	0.84
	Age*	0.45†	0.59†	0 59†
	Gender*	0.12	0.02	0.08
	Retest (ICC)	0.64	0.74	0.85
Kisauni (n=104)
	Mean (SD)	22.16 (2.89)	25.35 (3.34)	48.07 (5.39)
	α	0.77	0.80	0.87
	Age*	0.38†	0.59†	0.53†
	Gender*	0.08	0.09	0.10
	Retest (ICC)	0.82	0.88	0.91

^*Correlation of test score with age;

^†p<0.001.

Study 3: Development of a reference table

Reference tables showing the expected range of performance in each age-group were computed. They enable practitioners to identify at-risk children by comparing the scores of an individual child with appropriate age-related performance levels.

Sample

The comparability of test performance between the urban and rural population implies that the data of both groups can be combined to create a single reference group. The reference population consisted of 423 children (212 girls), mean (SD) age 20.89 (8.73) months (range 6–35).

Analysis

A table of Developmental Age Equivalent scores was created using a procedure adapted from Bayley’s Scale of Infant Development.¹⁹ Raw scores were plotted against age. The median of each age band is then computed, and the scores falling between two adjacent medians are divided into two groups. The scores in the lower half are added to the median of the lower age band while scores in the upper half are added to the median of the upper age band to determine the upper and lower levels of adjacent age bands. When the interval difference is an odd number, the larger of the two divisions (e.g. 4 of 7) is added to the lower age band. The age-appropriate range of scores for any age band is thus defined by the median ± the difference with the adjacent medians. The Reference Table was then created using mean scores and standard deviations for each age band to classify levels of functioning in children.

Results

The correlation between age and psychomotor score was highly significant: r (423) = 0.93, p<0.001, explaining approximately 86% of the variance. Similar trends were observed in the subscales (locomotor: r (423) = 0.89, p<0.001, variance explained 79%; eye–hand: r (423) = 0.92, p<0.001, variance explained 85%). Fig. 1 and Table 4 both illustrate the increase in mean scores observed with age.

FIG. 1. Means and 95% confidence intervals for the 3-month age bands.

(a, age bands in months; b, numbers per group)

TABLE 4. Levels of performance in reference population.

Age bands	Descriptive Mean (SD)	Superior performance >+3 SD	Accelerated performance +3 to +2 SD	Normal range +2 to −2 SD	Moderate delay −2 to −3 SD	Severe delay <−3 SD
6–8	15.86 (2.97)	25	22	21–11	10	7
9–11	20.44 (2.50)	28	26	25–16	18	12
12–14	27.35 (4.56)	41	33	39–21	19	14
15–17	34.29 (5.64)	51	40	39–23	22	17
18–20	40.29 (3.09)	50	46	45–35	34	31
21–23	43.63 (2.68)	51	49	48–39	38	35
24–26	44.85 (3.84)	56	51	50–39	38	33
27–29	48.28 (4.95)	63	54	52–41	40	33
30–32	50.75 (4.95)	66	56	55–41	40	36
33–35	53.06 (4.55)	67	59	58–44	43	39

Tables 3 and 4 are provided to guide the interpretation of an individual child’s psychomotor score. As illustrated in Table 3, developmental age is expressed as the mean of the age band in which a child’s score falls, plus or minus 1 month. For instance, the developmental age for a child functioning within the scores of age band 6–8 months is 7 ± 1 month.

TABLE 3. Total scores and equivalent developmental age (mths).

Developmental age, mean (SD)	Range of scores
<6	0–16
7 (1)	17–19
10 (1)	20–24
13 (1)	25–33
16 (1)	34–38
19 (1)	39–42
22 (1)	43–44
25 (1)	45–47
28 (1)	48–50
31 (1)	51–53
34 (1	>54

The child’s performance level can be classified by locating the raw score in the appropriate age band in Table 4. A child whose total score falls within 2 SD of the mean of his/her age band is said to be performing within the normal range. A child with a score ≤−2 SD but above −3 SD is reported to be experiencing a moderate delay in performance. A child with a score ≤−3 SD is taken to be experiencing a severe delay in performance. Though not specified in the table, borderline performance can be defined as a score of <1 SD below the group mean.

Study Four: Evaluation of the KDI in identifying and describing children with neurodevelopmental impairment

The study was undertaken to (i) investigate the sensitivity of the KDI to early brain insult by evaluating its ability to identify true group differences; (ii) evaluate the ability of the KDI to identify variation in the performances of a low functioning group, and (iii) evaluate the level of agreement between the KDI and an observation schedule used by the Association for the Physically Disabled of Kenya (APDK) for identifying developmental delay.

Sample

To identify true group differences, the performance of nine children attending a community-based rehabilitation programme for developmental delay [including children with cerebral palsy (4), idiopathic psychomotor delays (3), hemiplegia (1) and hydrocephalus (1)] was compared with the reference scores provided by the 104 children in the urban sample described above. Children from the rehabilitation programme (henceforward referred to as group 1) qualified for inclusion in the study if they had no severe neurosensory impairments (e.g. visual or hearing) that restricted their ability to interact with the materials provided. The mean (SD) age of group 1 was 28 (3.5) months (range 24–34).

To measure sensitivity and specificity between the two different approaches, the performance of group 1 children was compared with that of a group of children selected (according to criteria aimed at identifying both high and low performers for inclusion) from the urban reference sample (group 2). There were 18 children in group 2 with a mean (SD) age of 29 (3.0) months (range 24–35).

Materials and procedures

The KDI was administered to children in groups 1 and 2 by the study team and the observational schedule used by the APDK was employed to identify children in need of rehabilitation. The APDK schedule consists of three parts: family background, health history and developmental skills. Data are collected through a combination of parental report and observation. An occupational therapist and a community rehabilitation fieldworker administered this latter schedule.

Results

Significantly lower performances of the KDI by children with neurodevelopmental impairment (group 1) were observed. Eight of the nine children with neurodevelopmental impairment were identified as functioning below the 10th centile on the KDI compared with only 9% (n=10) of the urban children. Furthermore, the KDI scores indicated within-group differences in the performances of group 1 children.

The age-corrected, unstandardised psychomotor scores ranged from −41.27 to −5.43 with a mean (SD) of −24.80 (11.28). This performance variation was also observed for the subscales; locomotor scores had a minimum of −18.74 and a maximum of −2.75 with a mean (SD) of −13.36 (4.79) while the eye-hand score had a minimum score of −22.5 and a maximum of −2.68 with a mean (SD) of −11.45 (7.07).

The level of agreement between the KDI and the APDK in identifying developmental delay was 89% (24 of 29 children). Two children identified by the KDI as having a developmental impairment were not identified by the APDK, whereas one child was identified by the APDK but not by the KDI.

Discussion

The purpose of this study was to evaluate a modified version of a locally developed measure of psychomotor functioning in its application to the assessment of children aged 6–35 months. We established its reliability and validity within a reference group of children from both a rural and an urban setting, as well as the sensitivity of the measure to within-population variance at all levels of functioning. Results indicate that the scale is reliable, reflects maturational changes and is able to identify children with developmental delay.^20,21 Our study shows that it is possible to develop a culturally appropriate measure of psychomotor development that has sound psychometric properties. Furthermore, the present study confirms the results of earlier applications suggesting the sensitivity of the measure to early brain insult and to variation in performance even at the lower end of the performance spectrum.^7,22,23 In the absence of a gold standard against which to evaluate criterion validity,²⁴ we compared the extent to which the KDI, completed in a single test session by a person unfamiliar with the child, correlated with parental reports of child functioning. The significant correlation between the two approaches provides initial support for criterion validity.²⁵ This evidence would not be sufficient to fully support the validity of the scale. However, given the variety and convergence of evidence presented to support the validity such as relationship with gender and age, and sensitivity to neurodevelopmental disorders and anthropometric status, we consider that the parental reports add evidence of the validity of our measure. The study also enabled the initial development of reference tables to support the clinical application of the instrument.

Problems with applying and adapting standardised instruments from western countries in Africa often begin with the prohibitively high price of western materials.²⁶ The KDI test materials are relatively cheap and easy to produce locally (less than US$100) which ensures affordability and accessibility to a wide group of researchers and professionals interested in psychological assessment in resource-poor countries.

As in most parts of Africa, we worked in an area where psychological assessment is relatively new, and care must be taken to ensure procedures are acceptable to the community. Parental evaluation of the measure during focus group discussions confirmed that the procedures described here are acceptable to the local community. Community acceptability is a rarely investigated feature of test validity but it has important implications for the recruitment and retention of study participants.

Both this study and previous applications of the KDC confirm the suitability for children in a rural African community of the tasks and materials included in the inventory.⁷ The current study also provides evidence that the test is suitable for application to other settings, in particular to economically deprived urban locations. The tasks and procedures were directly transferable. The close similarity in the distribution of scores among the rural and urban children also suggests that it will be possible to develop reference tables applicable across a wider geographical region. However, more detailed investigations would be needed to address the generalisability of the reference tables.

The absence of a sufficiently large sample to compute normative data might limit the applicability of the reference tables. Another potential limitation is the restricted age range in the urban sample. It could be that urban children develop at a different rate and that a different pattern of results would have been observed in the younger age group. However, taking into account the scarcity of data on psychological assessment from sub-Saharan Africa, this study represents a good first step in providing validated measures of childhood outcome in this region. We were also able to demonstrate that the development of the tables allowed for meaningful interpretation of data from individual children. Given the high cost of developing norms based on a representative group, future efforts could focus on collating data from more than one source to allow for the development of norm tables. Future efforts also need to investigate the use of the KDI in a wider range of clinical groups and evaluate the performance of the KDI in economically advantaged populations to allow for clarification of its applicability in such populations and add to the normative data.

The KDI focuses only on psychomotor functioning; full assessment needs to look at other aspects of functioning such as language and socio-emotional development. We are currently developing and evaluating measures that address other aspects of child functioning. We have yet to evaluate the predictive validity of KDI.

The KDI is a locally assembled and culturally appropriate measure of psychomotor functioning that can be used to identify, describe and monitor effects of biological risk in children under 3 years of age. Moreover, it appears appropriate for clinical and research purposes. The KDI can be cheaply produced, administered by assessors with a limited background in child development, and has proved to be acceptable to local communities, making it a suitable instrument for resource-limited settings.