Assessing Developmental Outcomes in Children from Kilifi, Kenya, Following Prophylaxis for Seizures in Cerebral Malaria

Abstract

The purpose of the study was to develop a culture-informed measure of developmental outcome and to apply it to detect differences in developmental level between children with cerebral malaria enrolled in a clinical trial to control seizures during the acute phase of the illness. The instrument was administered to a sample of 180 children, three and 12 months after discharge from hospital. The measure demonstrated high internal consistency, good inter-observer reliability, age sensitivity and strong relations with parental report of child functioning. No association was found between performance, or change in performance, with the prophylactic regime administered. The results suggested that the use of Phenobarbital in controlling provoked seizures has no observable effect on cognitive function.

The adequate monitoring and evaluation of disease effects, related risk factors and intervention among children in rural communities in Africa is hampered by a lack of appropriate assessment instruments (Holding et al., 2004). Instruments developed in one culture cannot be readily transferred to another culture despite extensive evidence that the structure of cognitive abilities of children and adults is invariant across cultures (Georgas, Weiss, van de Vijver, & Salkofske, 2003; van de Vijver, 1997).

Among the most widely used of the published measures of developmental outcome in the early years are the Bayley Scales of Infant Development (Bayley, 1993), the Griffiths Mental Development Scales (Griffiths, 1954) and the Denver Developmental Screening Test II (Frankenburg, Dodds, Archer, Shapiro, & Bresnick, 1992). Examples of the application of these measures in Africa are described by Sigman, Neumann, Jansen and Bwibo (1989) in Kenya and Drotar et al. (1997) in Uganda. Both studies replaced pictures and objects of the Bayley Scale of Infant Development when original items unfamiliar to the children elicited no adequate responses. Similar modifications have been required when applying tests designed for older children. Boivin et al. (1995), working in the Democratic Republic of Congo, found that activities depicted in the Photo Series of the K-ABC were so unfamiliar that even school-going children could not complete the subtest. Holding et al. (2004) replaced the coloured plastic material used in the Triangles subtest of the K-ABC with wooden sticks in an adaptation of the test for six-year-olds in Kenya, when it was found that many children refused to touch the plastic material. Once modifications to test instructions, item content and procedures are made to maintain construct validity and sensitivity to within-population variations in ability, the original standardization of the measure is then rendered invalid.

Other challenges to the application of western instruments in a non-western context arise from a lack of familiarity with test demands (e.g. responding to a strange adult in one-to-one interaction), incomparability of samples being compared (e.g. schooled versus non-schooled) and poor translation of test items (Holding et al., 2004; van de Vijver, 1997). Problems with applying and adapting standardized assessment techniques in Africa begin with the often prohibitively high price of western materials (Aina & Morakinyo, 2001) and are compounded by the shortage of trained and qualified test administrators (Haataja et al., 2002; Olness, 2003). The numerous challenges described highlight the need to develop culture-appropriate items, administration procedures and the establishment of culture-specific norms for the interpretation of score levels.

Malaria in Africa is estimated to account for ‘40% of public health expenditure, 30–50% of inpatient admissions, and up to 50% of outpatient visits in areas with high malaria transmission’ (World Health Organization, 2005). It is also the leading cause of mortality in under-fives, accounting for 20 per cent of deaths (World Health Organization, 2004). Cerebral malaria, the most severe form of malarial disease, accounts for 10 per cent of in-patient admissions in malaria-endemic regions. Over 80 per cent of children with cerebral malaria have a history of seizures and 60 per cent show seizures after the onset of treatment (Crawley et al., 1996; Lesi, Nwosu, Mafe, & Egri-Okwaji, 2005; Waruiru et al., 1996).

The occurrence of multiple and prolonged seizures in cerebral malaria has been associated with increased risk of mortality (Jaffar, van Hensbroek, Palmer, Schneider, & Greenwood, 1997) and the presence of neurological sequelae (Bondi, 1992; Holding & Snow, 2001; Molyneux, Taylor, Wirima, & Borgstein, 1989) that persist several months post-discharge (van Hensbroek, Palmer, Jaffar, Schneider, & Kwiatkowski, 1997). Hemiplegia, epilepsy, hemiparesis, ataxia, behavioural problems, visual impairment and delayed speech are commonly reported sequelae (Crawley et al., 2000; Idro, Jenkins, & Newton, 2005). While children can experience partial and even full recovery from some symptoms, including ataxia and cortical blindness, they never fully recover from others such as hemiparesis (Idro, Jenkins, & Newton, 2005).

Studies involving both human and animal models have reported an increased risk of cognitive impairment following seizure activity. General cognitive impairment (Banu et al., 2003; Strafstrom, 2002), impaired spatial learning and memory (Majak & Pitknen, 2004), increased anxiety (Sayin, Sutula, & Strafstrom, 2004), motor impairment (Idro, Carter, Fegan, Neville, & Newton, 2005), school failure, behaviour and mental health problems (Freitag & Tuxhorn, 2005) have all been associated with multiple or prolonged seizures. The association between seizure activity and subsequent impairment in cognitive performance suggests the need for treatment and control of seizures during the illness episode. However anti-epileptic drugs may themselves contribute to the development of cognitive impairments (Kaindl et al., 2004). The exact nature and extent of the problems associated with the drugs may be related to type, dosage and length of use (Aldenkamp & Bodde, 2005; Etchepareborda, 1999; Majak & Pitknen, 2004; Motamedi & Meador, 2003). Indeed, Aldenkamp and Bodde’s (2005) review of the literature highlights the dilemma involved, noting the contradictory need to control seizure activity as early as possible while acknowledging the possible detrimental effects of prolonged use of the drugs on central nervous system function.

Like many countries in the region, Kenya lacks both appropriate tools to evaluate cognitive development and adequately trained personnel to administer them. The identification and diagnosis of children with special needs in Kenya is mainly carried out under the umbrella of the Educational and Assessment Resource Service, a unit of the Ministry of Education. The unit co-ordinates 52 district centres charged with the identification and support of the approximately 1 million children in the country with special needs (Muga, 2003). Children are identified through discussion with parents, that may be supplemented by the application of a screening test designed for children aged six months to six years (Kenya Institute of Special Education, 1984). Each of five functional areas (motor, vision, hearing, speech and language and emotional problems) is represented by a limited number of items restricting the application of this procedure to the identification of gross developmental impairments. In addition to this short instrument there is a need for an instrument that can provide more detailed information about a child’s skill profile and detect more subtle impairments. The inclusion in such an instrument of developmental constructs commonly used in other settings would also allow for the comparison of outcomes across different study sites.

The aim of the present study was to develop a culture-informed measure of developmental outcome for use in resource-limited settings and to apply it to detect differences in developmental level between children with cerebral malaria enrolled in a clinical trial to control seizures during the acute phase of the illness. In this article we describe the initial development of the Kilifi Developmental Checklist (KDC) and report on the reliability and validity of the tool.

Method

Study site and sample

The development of the assessment instrument took place at the Kenya Medical Research Institute, Kilifi, Kenya. Kilifi District is a predominantly rural community that stretches between Mombasa and Malindi along the Indian Ocean. The majority of the population in the area belongs to the Mijikenda ethnic/linguistic group. Two Bantu languages are most commonly spoken in the area, namely Kigiriama (a member of the Mijikenda group of languages) and Kiswahili (which is widely spoken across Eastern Africa). The majority of families depend upon subsistence farming. Low literacy levels and high poverty levels characterize the population. A total of 66 per cent of the population live below the poverty line (Government of Kenya, 2001). Extended families live in homesteads and share the child rearing. After weaning most children spend time with older siblings and spend little time in a dyadic interaction with an adult. Systematic observations have shown us that there are almost no shop-bought play materials and most children use homemade play items, often produced by older siblings (Kendall-Taylor & Katana, 2004). Medical facilities in the district are centred upon one tertiary service, the Kilifi District Hospital, and five outlying government clinics. The district hospital also provides therapeutic services for children with disabilities in the form of a paediatric physiotherapy service and an occupational therapy department.

Eligible children were aged over nine months, and had previously been recruited for a study investigating the effectiveness of a prophylactic dose of Phenobarbital as method of seizure control in the treatment of cerebral malaria (Crawley et al., 2000). A more detailed description of the sample can be found in Crawley et al. (2000). All children had been discharged from Kilifi District Hospital between June 1995 and January 1998 following an episode of cerebral malaria, defined as unarousable coma (inability to localize a painful stimulus/Blantyre score of three or less), and the presence of P. falciparum parasites (Molyneux et al., 1989). Half of the children enrolled into the original study were randomly assigned to receive an intra-muscular dose of Phenobarbital, while the other half received a placebo. All children who subsequently had seizure activity were treated with Diazepam. Seizure activity was significantly lower in the acute phase of the illness in children in the Phenobarbital (or prophylaxis) arm of the study (11 per cent versus 27 per cent of children experience three or more seizures). However the mortality in this group was doubled. Two hundred and sixty-four children discharged alive returned for a three-month neurodevelopmental assessment. Figure 1 illustrates the retention of subjects in this section of the study.

Figure 1. Sample description.

Note: The numbers in parentheses represent the children who experienced at least three seizures during admission in each treatment group

Development of the instrument

Phase one: item selection

We employed an adaptation approach (van de Vijver & Tanzer, 2004) to item selection, using constructs and items from previously developed instruments and modifying them in order to increase their appropriateness. Items in the pilot version of the instrument were drawn from several sources. Items from the Kenyan Screening Test for Children aged six months to six years (Kenya Institute of Special Education, 1984) were supplemented by items from the Griffiths Mental Development Scales (Griffiths, 1954), the Movement Assessment Battery for Children (Henderson & Sugden, 1992), the Merrill Palmer Scales of Mental Tests (Stutsman, 1948), the Wessex Revised Portage Checklist (White & East, 1983), Wechsler’s Preschool and Primary Scales of Intelligence (Weschsler, 1989) and tasks suggested by the Shoklo Neurodevelopmental Assessment (Haataja et al., 2002). An initial pool of 101 items was created on the basis of this review. Subsets of this pool were administered to children drawn from the local community. Special attention was paid to the development of appropriate instructions, different methods of observing and recording observations and to the suitability of the materials used. Excluded from the assessment of motor development was an item about climbing stairs, as stairs are not generally present in buildings in the area. Many of the gross motor activities were observed during the course of free play with a ball. Mothers and older siblings were encouraged to join in; the difficulty of simultaneously recording and observing the child was overcome by having a separate observer and facilitator.

Phase two: tool development and evaluation

The first 80 children (39 female; mean age = 38.32 months; SD = 17.24; range 7–88 months) recruited were used to pilot an initial item list and to train a reliable assessment team. Items were selected for retention on the following criteria: (1) Clarity of observation: Success on the action/task can be readily determined by the observer; (2) Within-population variance and age appropriateness: Range of performance observed in the target age range; (3) Clarity of description: The behaviour can be easily described in the local languages.

Fifty-eight items were selected for inclusion in the final checklist. Items were grouped into four subscales (see Table 1): Locomotor; Eye–Hand Co-ordination; Hearing, Speech and Language; and Social-Emotional Development. The groupings were based upon the model provided by published developmental measures, particularly the Griffiths Mental Development Scale (Griffiths, 1954).

Table 1. Description of subscales of the Kilifi Developmental Checklist.

Name of subscale	Domain of assessment	Method of data collection	Items
Locomotor	The child’s movement in space, static and dynamic balance, and motor co-ordination	Interaction with the child	17
Eye–Hand Co-ordination	The child’s ability to manipulate objects and to co-ordinate fine motor movements	Interaction with the child	17
Hearing, Speech and Language	Expressive language, comprehension and screens the child’s hearing	Interaction with the child	9
Social-Emotional	Social functioning, adaptive behaviour and daily living skills	Interaction with the child parental interview	15

The assessment of the first 80 children also provided the opportunity to train an assessment team and develop an administration manual outlining a standard format for test administration. The assessment team consisted initially of three community nurses, a fieldworker with extensive experience in administering parental interviews and a local mother with limited formal education. The inclusion of personnel with a different range of experience was carried out to enable an evaluation of the minimum initial skill level required as a prerequisite for training. The training programme was run by one of the authors (Holding) and consisted of demonstrations, guided assessments and videotaped assessment sessions. The videos were used to provide individual and group feedback and as a tool in the assessment of inter-observer reliability. Assessment skills were trained and evaluated through close observation, and through comparison of the scoring by each team member of the taped assessment sessions. In the training phase an inter-observer agreement of more than 80 per cent was required for each item across the majority of team members. The skill level achieved by the nurses and the field worker were deemed sufficient to retain them in the project. The procedures administered by one member of the team, the mother, were found to be inconsistent. She was therefore dropped from the team.

Test evaluation: materials and procedures

The final item list was administered to the remaining 180 children (88 girls) three months post-discharge. The mean age of these children was 40 months (SD = 19.8, range: 11–109 months). Eighty-six children (43 female) had received Phenobarbital. At 12 months post-discharge, 157 (79 girls) of these children were assessed again. There was no significant difference in the attrition rate between boys and girls (14 and nine, respectively), nor between Phenobarbital and placebo children (13 and 10, respectively) (see Figure 1). All the tests were administered at the Kilifi District Hospital grounds in a room set aside for assessment. Each child was assessed in the presence of the mother/guardian. A team of two assessors administered the procedure. One of the team members took the role of observer and the other the role of facilitator. The facilitator interacted with the child, introducing the items and giving the instructions to the child. The observer sat unobtrusively at the side of the room recording the child’s responses and behaviour on the checklist. Tasks were scored on a three-point scale (0: child is unable to perform the task; 1: skills in the task are emerging; 2: child’s skills in the task are established).

Parental reports were collected on a subset of 42 children. The parental report was elicited using a questionnaire developed in Kiswahili for use in Tanzania (Stoltzfus et al., 2001). The version of the questionnaire used contained 111 items, subdivided into three subscales; Motor, Social and Emotional. Members of the assessment team translated the questionnaire into Kigiriama. In our population the full-scale questionnaire showed a value of Cronbach’s alpha of .94. Moreover two subscales also showed high internal consistency (Motor: α = .84; Social: α = .93), while the value of the Emotional scale was considerably lower (α = .58).

Data on a subset of 53 children were used to evaluate the test–retest reliability of the instrument. Five months were set aside to collect retest data, and children seen in that period were invited for a second visit, regardless of whether their original appointment was for the three or 12 months visit. The mean time between the two tasks was 22 days (SD = 5 days; range: 15–46 days).

Ethical considerations

Written informed consent was obtained from all families and guardians of study participants. For participants who were not literate the consent form was read out in the language with which they were most familiar. Assent was sought through discussion and play from all children prior to the developmental assessment. Approval for the study was obtained from the Kenya National Ethical Committee.

Data management and analysis

Data were double entered in FoxPro and verified before being transferred to SPSS (version 12) for analysis. Descriptive statistics were generated to evaluate the score distribution, including means, standard deviation and item difficulty. Scores were calculated for each of the four subscales, and a total score obtained for all 58 items combined.

Principal component analysis of the subscales was carried out to examine the structure of the instrument. Estimates of internal consistency of the subscales were computed using Cronbach’s alpha. Subsamples of children were selected at random to compute the inter-observer (n = 34) and retest (n = 53) reliabilities. Criteria set out by Cicchetti (1994; Cicchetti et al., 1992) were employed in evaluating the level of acceptability of the observed values of the reliability coefficient.

The subscale and total scores for each of the two time points, as well as the change in scores over the nine months between the two assessments (change score), were computed. Repeated Measures ANOVA was applied to evaluate the sensitivity of the tool to maturational changes. Correlations between the KDC and parental report were computed to evaluate criterion validity. Univariate and multivariate association measures were computed to assess the effects of the prophylactic regime on developmental outcomes.

Results

Psychometric properties

Descriptives and reliability

Descriptive and frequency tables were utilized to evaluate item variance in the population and to order items along a ‘developmental index’. With the possible mean score per item in the range from 0 to 2, and observed scores ranging from 0.32 to 1.90, not less than 79 per cent of the theoretical range was observed on all items. Results therefore indicate that all items showed sensitivity to within-population differences in ability. A developmental progression was also found. The proportion of children passing each item ranged from 2.8 per cent (‘Can wash themselves without supervision’) to 98.3 per cent (‘Sits with support’).

The evaluation of internal consistency was addressed in two ways. The first analysis employed raw scores, while the second analysis used scores corrected for age, to control for the possibility that the large age range included may inflate the internal consistency. Linear regression analysis was used to correct for the age differences; test scores were predicted on the basis of the child’s age. The standardized residual scores of this analysis were then used to determine the internal consistency of the measure both at the full scale and subscale level. As can be seen in Table 2, all scores showed high internal consistencies; the alpha values for the raw scores ranging from .85 to .93, and for the age-corrected scores from .76 to .86. All these values are above the lower limit of .70 (Cicchetti, 1994), which points to a good internal consistency of all KDC scales. The combination of a good internal consistency and the large range of individual differences observed provide support for the adequacy of the instrument to assess individual differences in abilities.

Table 2. Means, standard deviations and Cronbach’s alpha for the scales and each subscale.

Domain	N	M	SD	α	α a
Developmental	178	80.50	26.26	.97	.94
Locomotor	180	19.31	8.68	.93	.81
Eye–Hand Co-ordination	180	20.20	8.42	.94	.86
Hearing, Speech and Language	180	11.81	4.90	.89	.82
Social-Emotional	178	28.81	5.88	.85	.76

^aInternal consistency based on age-corrected scores

Intra-class correlation coefficients (absolute agreement) were computed to estimate inter-observer reliability for the full scale and for each subscale. The values were excellent (Developmental: .88; Locomotor: .89; Eye–Hand Co-ordination: .74; Hearing, Speech and Language: .88; and Social-Emotional: .97).

Retest reliability was estimated by computing intra-class correlation coefficients (consistency) for the full scale and for each subscale. The values from the raw score were excellent (Developmental: .94; Locomotor: .91; Eye–Hand Co-ordination: .93; Hearing, Speech and Language: .83; and Social-Emotional: .88).

The dimensionality of the age-corrected subscale scores was studied in a principal component analysis. The four subscales yielded a strong first component, which accounted for 75 per cent of the variance (eigenvalue = 3.02). Factor loadings were as follows: Locomotor: .85; Eye–Hand Co-ordination: .88; Hearing, Speech and Language: .86; and Social-Emotional: .86. These results support the use of a summated score to compute an overall index called the Developmental Score. The internal consistency of the total instrument was very high (.97 for the raw scores and .94 for the age-corrected scores).

Correlations were computed to explore the relationship between gender and performance on the total score and for each of the subscales. Results indicate that there was no relationship between gender and the Developmental Score (r(180) = .07, p < .30), nor between gender and any subscale (Locomotor: r(180) = .06, p < .37; Eye–Hand Coordination: r(180) = .05, p < .44; Hearing, Speech and Language: r (180) = .08, p < .28; Social-Emotional: r(180) = .04, p < .55). Consequently, gender was not considered in the remaining analysis. The absence of gender differences supports the validity of the instrument.

The age sensitivity of the KDC Total and Scale scores was investigated by examining their correlation with age. Very high correlations were found between age and the Developmental Score (r(180) = .81, p < .001) explaining approximately 67 per cent of the variance, also between age and the subscales (Locomotor: r(180) = .83, p < .001; Eye–Hand Coordination: r(180) = .72, p < .001; Hearing, Speech and Language: r (180) = .73, p < .001; Social-Emotional: r(180) = .71, p < .001), ranging from .71 to .83. Age explained on average 56 per cent of the variance in the subscales scores. These correlations provide strong support for the age-sensitivity of the KDC.

Scores for children who were seen both at three and 12 months post-discharge (n = 157) were used to evaluate the sensitivity of the KDC to maturational changes. The differences in scores between time 1 and time 2 are displayed in Table 3. A significant increase in scores over a nine-month period was observed for all scales (Developmental Score: F (1, 155) = 88.16, p <.01; Locomotor: F(1, 156) = 135.05, p <.01; Eye–Hand Co-ordination: F(1, 156) = 45.28, p < .01; Hearing, Speech and Language: F(1, 156) = 35.61, p < .01; Social-Emotional: F(1, 155) = 19.23, p < .01). Furthermore, there were positive and significant relationships between scores at these time points, indicating that rank order and relative distance between subjects were well maintained between the two assessments.

Table 3. Test scores at three and 12 months after discharge and their correlations.

	3 months			12 months			Correlations
Domain	N	M	SD	N	M	SD	r	pra
Developmental	156	81.4	25.6	157	92.3	26.4	.84	.69
Locomotor	157	19.7	8.1	157	24.8	9.8	.85	.61
Eye–Hand Co-ordination	157	20.2	4.7	157	23.2	7.1	.78	.63
Hearing, Speech and Language	157	12.0	4.5	157	13.5	4.3	.74	.59
Social-Emotional	156	29.1	5.4	157	30.6	5.6	.68	.54

^aPartial correlation which corrects for the effects of age

The criterion validity of the KDC was evaluated by comparing performance on the KDC with parental report. As can be seen in Table 4, there was a significant, positive relationship between most KDC subscale scores and parental report particularly for scales measuring similar constructs. The exception is the parental emotional scale, which did not have a significant relationship with two of the KDC subscales. These findings support the criterion validity of the KDC.

Table 4. Correlations between KDC subscales and parental report.

	Parental report
Subscale	Motor	Social	Emotional	Developmental
Developmental	.55**	.76**	.31*	.73**
Locomotor	.59**	.75**	.31*	.72**
Eye–hand Co-ordination	.45**	.60**	.26	.58**
Hearing, Speech and Language	.41**	.69**	.20	.62**
Social-Emotional	.57**	.78**	.36**	.75**

^*p < .05;

^**p < .01 (one-tailed)

Effects of controlling seizures on cognitive outcome

Scores for children at both three and 12 months post-discharge were used to evaluate the prophylactic regime (Phenobarbital). The means and standard deviations at both time points are displayed in Table 5. Using age-corrected scores the differences between the groups were evaluated by means of an ANOVA. There were no significant differences at three months post-discharge between the children who received the prophylaxis and those who received the placebo (Developmental Score: F(1, 175) = 0.42, p < .51; Locomotor: F(1, 177) = 0.49, p < .48; Eye–Hand Co-ordination: F(1, 177) = 1.07, p < .30, Hearing, Speech and Language: F(1, 177) = 0.12, p < .72; Social-Emotional: F(1, 175) = 0.11, p < .73). Similarly, no group differences were observed at 12 months post-discharge (Developmental Score: F(1, 151) = 0.45 p < .50; Locomotor: F(1, 153) = 1.30, p < .25; Eye–Hand Co-ordination: F(1, 151) = 0.05, p < .80; Hearing, Speech and Language: F(1, 151) = 0.45, p < .50; Social-Emotional: F(1, 151) = 0.45, p < .50). Furthermore, no significant group differences were observed at the scale and subscale level (see Table 6).

Table 5. Means and standard deviations for two treatment groups.

	3 months post-discharge				12 months post-discharge
	Placebo group		Phenobarbital		Placebo group		Phenobarbital
Scale	M	SD	M	SD	M	SD	M	SD
Developmental	81.06	26.46	79.91	26.20	92.77	26.61	92.20	25.78
Locomotor	19.47	8.79	19.21	8.63	25.36	10.75	24.42	10.24
Eye–Hand Co-ordination	20.14	8.41	20.28	8.24	23.08	6.74	23.60	7.60
Hearing, Speech and Language	11.94	4.84	11.74	4.98	13.68	4.34	13.53	4.48
Social-Emotional	28.98	5.78	28.67	6.04	30.63	6.17	30.64	5.19

Table 6. Means, standard deviations and significance of the difference score.

Domain	Placebo group Mean (SD)	Phenobarbital Mean (SD)	F	p
Developmental	11.55 (17.38)	10.32 (11.29)	1.04	.30
Locomotor	5.53 (6.33)	4.69 (4.53)	1.23	.26
Eye–Hand Co-ordination	2.67 (5.51)	2.73 (4.66)	0.58	.44
Hearing, Speech and Language	1.66 (3.54)	1.42 (3.01)	0.56	.45
Social–Emotional	1.59 (5.10)	1.47 (3.63)	0.26	.60

Additional analyses (not further documented here) showed that there were no confounding effects of pre-admission seizure activities on performance three or 12 months post-discharge. However, seizure activities after the administration of the study drug were associated with test performance. A group of children in each arm of the study trial (see Fig. 1) experienced at least three or more seizures after the administration of the study drug. We analysed the outcome in this group of children at 12 months post-discharge after adjusting for treatment group. The children who experienced multiple seizures performed significantly poorer than those who did not, F(1, 151) = 8.21, p < .01, η² = .05. Similar results were observed for each of the sub-scales (Locomotor: F(1, 151) = 5.99, p < .02, η² = .03; Eye–Hand Co-ordination: F(1, 151) = 4.71, p < .03, η ² = .03; Hearing, Speech and Language: F(1, 151) = 4.75, p < .03, η² = .03; Social-Emotional: F(1, 151) = 11.55, p < .001, η² = .07).

Given the sensitivity of the measure to maturational change an improvement in scores was expected over the nine-month follow-up period. However, as the amount of improvement may depend upon several factors such as the age of the child and the specific task, we chose to apply a conservative definition of impaired development, the decrease of test scores in at least two subscales of the KDC. Of the 157 children investigated, 20 children met this criterion for developmental impairment. No significant association was found between seizure-related activities prior, during or subsequent to the malaria episode and this developmental impairment measure.

In addition, no association was found between a clinician’s assessment of neurological impairment and the classification of developmental impairment. Only two of the 20 children so classified were also identified through the clinician’s assessment at discharge as having neurological sequelae, and only one child so identified at the neurological examination at three months post-discharge.

Discussion

Psychometric properties

The Kilifi Developmental Checklist provides a standardized and reliable instrument for monitoring psychomotor development in a resource-limited setting. Within-population variance, internal consistency and retest reliability observed in this study supported both the suitability of the items selected and the reliability of the tool in our setting. Furthermore, the selection of culturally appropriate items ensured that the instrument was sensitive to normal maturational development, as shown in the significant improvement in performance of the study children over the nine-month follow-up period. The reliability of the instrument was further supported by the consistency in the ranking of the performance level of the children over the same period.

These results provide justification for the time expended in developing this instrument in context. We did not find unexpected relationships between age and performance, as sometimes found for non-adapted instruments (Oluyomi & Houser, 2002). The increasing difference between the standardization sample and diverse cultural groups that has been recorded as children get older may even suggest, inappropriately, a regression in skill development (Gay et al., 1995), compromising study results.

The shortage of personnel with a background in child assessment within the African context has been identified as a limitation to the feasibility of tests that require prior experience and extensive training (Olness, 2003). We have demonstrated that the KDC can be administered in a reliable and standardized manner by examiners with limited prior experience of child assessment. The high inter-rater reliabilities achieved attest to this. However, the inability of the mother with limited formal education to achieve the required level of competence demonstrates the potential need for a minimum level of secondary school education as a prerequisite for training.

In the absence of already existing tests there are different approaches that have been applied to the production of an assessment measure (van de Vijver & Tanzer, 2004). Translation of already existing measures, while the most common approach, may constrict the within-population variance and mask true group differences (Connolly & Grantham-McGregor, 1993). At the other extreme, the production of a novel assessment limits the comparability of outcomes across different cultural settings (Sternberg et al., 2001). The current study used an alternative approach that may overcome these limitations. We chose to assemble a selection of activities into a new measure, and care was taken to ensure that items included were not only acceptable to the target population but also evaluated constructs common to those measured by published tests. This is potentially important as it enables the comparison of disease effects across sites and contexts (Holding & Kitsao-Wekulo, 2004).

Factor-analytic evidence suggests that the KDC is unifactorial and primarily measures general developmental functioning. The combination of seemingly unrelated skills such as cognitive, locomotor and social-emotional skills in a single factor is common to infant scales, and may be associated with the insufficient differentiation of skills in the early years to allow for the delineation of more discrete skills and functions that are observed later in life (Kline, 1993). The use of subscales does allow for the investigation of differential sensitivity of functions to early childhood disease (Boivin et al., 1995; Msellati et al., 1993). Indeed, studies carried out subsequently to the one reported here have found subscale differences in the evaluation of neonatal diseases (Barlow, Mungala-Odera, Gona, & Newton, 2001; Gordon, English, Tumaini, Karisa, & Newton, 2005), mitigating the need to differentiate component skills at certain levels of analysis.

A prerequisite for adequate criterion validation is the use of a gold standard as the criterion measure (Gregory, 1992). We did not have such a measure. Therefore, we investigated the extent to which the KDC, a primarily observational-performance measure, correlated with parental reports. The latter provided preliminary support for the criterion validity of the KDC.

The KDC uses a combination of performance-based assessment and parental report. The two assessment methods were combined to overcome the inherent limitations in each of the methods. This approach should enhance the reliability of the information collected and increase the likelihood that the performance level recorded is a true representation of the child’s ability (Carter, Briggs-Gowan, & Davis, 2004). Our experience demonstrated that some behaviours can only be assessed through parental report, notably social functioning. Parents in this study, however, were not familiar with all the concepts presented to them, and were not able to report on the full range of their child’s behaviours. Performance-based testing is required to elicit information of other functions of interest to the developmentalist. We conclude that future efforts should focus on specifying the distinction between the functional areas and evaluating the most suitable assessment approach for each of the areas.

The need to expand the assessment of children’s development post-discharge to enable the identification of children with more subtle sequelae is highlighted by the lack of overlap between the neurological and the developmental assessment procedures. Based on our definition of poor development (a loss of skills in at least two of the KDC subscales) 12.7 per cent (n = 20) of the children assessed at the second time point would need further monitoring. In total, 90 per cent of the children identified as requiring monitoring and possible educational support would not have been identified by a clinical assessment of neurological functioning alone. Our findings highlight the need to develop and validate measures that can be used to supplement clinical judgement.

Control of seizures in cerebral malaria

The purpose of the development of the tool was to investigate the possible subtle benefits of prophylaxis for seizures in cerebral malaria. Our results suggest that there were no long-term cognitive benefits to receiving a prophylactic dose of Phenobarbital. On the other hand, there was no indication that the children who received the prophylactic dose of Phenobarbital suffered any negative cognitive effects.

The lack of benefit can be explained by a number of factors. First, children in the Phenobarbital group experienced higher rates of death. This high attrition may mask the benefits of the prophylaxis, limiting the kind of conclusion that can be drawn. Second, children who experience cerebral malaria are a heterogeneous group. Clinical records show variability in signs and symptoms, and children experience a number of complicating factors that may impair brain function, such as hypoglycaemia. These factors could affect developmental outcome regardless of seizure activity, potentially masking the benefits of seizure control. A third limitation is the lack of a true placebo group. All children who developed seizure activity were treated with diazepam. A greater number of children in the placebo group developed seizures and were therefore not treatment free. There is a potential, but untested, confounding effect of this treatment on later outcome.

Because we did not find any differences between treatment groups it could be argued that the KDC was not sensitive to true group differences. We do not favour this interpretation because our data are in line with both the findings of the original study (Crawley et al., 2000) and the literature on seizure prophylaxis. The results of the neurological assessment, reported in Crawley et al. (2000), showing no effect on the presence of neurological sequelae at discharge or three months post-discharge, corroborate the results of the developmental assessment. The only significant effect of prophylaxis was on provoked seizure activity. These results are similar to those of a number of different studies reviewed by Beghi who concludes that: ‘Based on the results of randomised clinical trials and meta analysis the prophylactic use of (AEDs) is effective in reducing the risk of early post traumatic seizures, whereas late seizures, disability and or death seem unaffected by active treatment’ (2003, p. 25). Given both the evidence provided by Beghi and the sensitivity of the KDC to the effects of other neonatal complications (Barlow et al., 2001; Gordon et al., 2005), we conclude that the lack of group differences is much more likely a reflection of group characteristics than a reflection of limitations of the tool. Results suggest seizure activity may not be the mediating pathway between cerebral malaria and impaired development. However, the children who went on to experience multiple provoked seizures during admission did perform poorly on the developmental assessment at 12 months compared to those who did not, regardless of the drug received. Future investigations may need to focus on the underlying causes of the seizures, and on the control of other clinical features of malaria, such as hypoglycaemia, to identify more salient indicators of subsequent impaired growth and development.

The KDC is one of the first culturally appropriate measures for resource-limited settings available, accompanied by evidence of sound psychometric properties and affordable in the context for which it was designed. The main limitation of the current study is the absence of a normative group, consisting of healthy children. This population would have allowed for a more extensive evaluation of psychometric properties of KDC such as its sensitivity to the effects of cerebral malaria and its performance in a sample representative of the community. A study is currently underway to develop normative data for the KDC, establish its predictive validity and to expand and modifying scales within the test to allow for greater sensitivity to specific disease effects.

Author biographies

Amina Abubakar is a doctoral candidate at Tilburg University and works at KEMRI/Wellcome Trust. Her interests are in identifying and describing factors that contribute to risk and resilience among infants in sub-Saharan Africa.

Fons J. R. van de Vijver, PhD, is Professor of Cross-Cultural Psychology at Tilburg University. His interests are the methodological aspects of cross-cultural comparisons, psychological assessment in non-Western context, cross-cultural comparisons of cognitive process and acculturation.

Aadik Mithwani, MTropPaeds, MMed (Paeds), is a consultant pediatrician with KEMRI/ Wellcome Trust. His research interests are Malaria and HIV with a focus on Public health, pathophysiology and pharmacokinetics issues related to these conditions.

Elizabeth Obiero, Naomi Lewa, Simon Kenga and Khamis Katana were involved in the development and evaluation of the assessment instrument. Their interests are child health and child welfare.

Penny Holding, PhD, is a pediatric psychologist. Her interests are in the evaluation of child well-being and the promotion of applied child developmental research.