Evaluating the Validity and Reliability of PDQ-II and Comparison with DDST-II for Two Step Developmental Screening

Abstract

Objective

This research was designed to identify the validity and reliability of the Prescreening Developmental Questionnaire 2 (PDQ-II) in Tehran in comparison with the Denver Developmental Screening Test-II (DDST-II).

Methods

After translation and back translation, the final Persian version of test was verified by three pediatricians and also by reviewing relevant literature for content validity. The test was performed on 237 children ranging from 0 to 6 years old, recruited by convenient sampling, from four health care clinics in Tehran city. They were also evaluated by DDST II simultaneously. Interrater methods and Cronbach's α were used to determine reliability of the test. The Kappa agreement coefficient between PDQ and DDST II was determined. The data was analyzed by SPSS software.

Findings

All of the questions in PDQ had satisfactory content validity. The total Cronbach's α coefficient of 0–9 months, 9–24 months, 2–4 years and 4–6 years questionnaires were 0.951, 0.926, 0.950 and 0.876, respectively. The Kappa measure of agreement for interrater tests was 0.89. The estimated agreement coefficient between PDQ and DDST II was 0.383. Based on two different categorizing possibilities for questionable scores, that is, "Delayed" or "Normal", sensitivity and specificity of PDQ was determined to be 35.7–63% and 75.8–92.2%, respectively.

Conclusion

PDQ has a good content validity and reliability and moderate sensitivity and specificity in comparison with the DDST-II, but by considering their relatively weak agreement coefficient, using it along with DDST-II for a two-stage developmental screening process, remains doubtful.

Introduction

Early detection of developmental disorders has an important role in the well-being of children and their families. This is a basic responsibility for pediatric primary care providers [1]. About 16–18% of children in various populations have developmental disorders but only 20–30% of them are recognized before school entrance.

This fact shows that early detection of developmental disorders must be provided in primary child health care services. Today there is an increasing effort for detection of developmental disorders at an earlier age because intervention services are cost effective and when provided in early childhood, have greater efficacy [2]. These services improve the developmental prognosis and have short and long term benefits [3–6]. In order to detect developmental disorders at an early age, the American Academy of Pediatrics (AAP) has recommended that pediatricians use developmental screening tools at 9, 18, 24 (or 30) months child health visits [1].

Developmental screening means using a brief, valid, rapid and standardized tool in order to detect those children who are at risk for developmental disorders and to help find those children who need an extensive developmental assessment [1, 7–9].

Although there are numerous developmental screening tools, each uses different approaches. There is no unique tool that can be used for all populations and all age groups[1]. For selecting a suitable tool we must consider the age range of the desired population, the time and expenses of training test providers and of administering and interpreting the test, the developmental domains that must be assessed in the desired population, the validity, reliability, sensitivity, specificity and all other positive and negative points about the tool of the test[1, 10–12]. There are two types of developmental screening tools: 1) screening tests which examiners directly observe and interpret the child‘s behavior and 2) developmental questionnaires that are completed by the child‘s care provider.

Parental information about child developmental status has been considered reliable and useful for many years [13–15]. Research shows that if questionnaires are presented correctly to the parents, almost all parents regardless of their socioeconomic status, geographic setting or health background and child rearing experiences, can give correct information about their child and their opinions have high validity and will lead to increasing rates of early detection and intervention of developmental disorders [10, 12–18]. Furthermore, if parents have a role in the developmental screening process, they can be more effective in the detection of learning needs of their children [19]. One must keep in mind that screening tools are usually used for two step diagnosis and their suspicious or delayed results must be confirmed with diagnostic or more specialized tools that need more time and expertise to administer [7]. Research has shown that using Prescreening Developmental Questionnaire 2 (PDQ-II) decreases the use of Denver Developmental Screening Test-II (DDST-II) that needs more time, expense and expertise to administer, by 69% [20].

This study was performed from February 2008 to January 2009 in Tehran, Iran to determine validity and reliability of PDQ and to estimate its sensitivity and specificity in comparison to its origin, the DDST-II.

Subjects and Methods

At first the questionnaires were translated into Persian by two experts in English language. Then the Persian version was back translated to English by two other English experts who were unfamiliar with the original version of the test. After comparing the back-translated versions of the test with the original version, the problematic parts were identified and corrections were made in the Persian translation, after which the Persian version was finalized. Next the content validity of the final Persian version was verified by three pediatricians familiar with child development and also by reviewing relevant sources and references.

PDQ-II is a developmental prescreening tool that is derived from DDST-II. 97 of 105 items of DDST-II are changed to questions that can be answered by “YES” or “NO” by the care-giver.

They are categorized into 4 questionnaires for 0–9 month, 9–24 month, 2–4 year and 4–6 year old children. The 75^th and 95^th percentiles for each question and also the developmental domain to which it belongs are shown in front of it. Caregivers must continue answering questions until they arrive at 3 “NO” answers (it is not necessary that the NO answers be consecutive). The answer to each question can be: normal (which means the child is able to do the task), delayed (which means the child is not able to do the task that 90% of his/her age-matched children can do) and caution (which means the child is not able to do the task that 75% of his/her age matched children can do). For interpretation of the results, if the child has ≤1 delay or ≤2 cautions (considered as suspicious), developmental advices are given to parents and the child must return for retesting by the PDQ-II one month later. If the child is still in a ‘suspicious’ condition in the second visit then he/she should be referred for screening by the DDST-II. If in the first prescreening visit child has ≥2 delays or ≥3 cautions (considered as delayed), he/she should be screened by the DDST-II as soon as possible.

For performing the study, 8 examiners (with a B.S. degree in occupational therapy or clinical psychology) were trained. Convenient sampling was used and 237 children aged 0–6 years (divided to 4 age groups], without any obvious disability were tested in 4 primary health care centers situated in south, north, east and western regions of Tehran (sample size was determined by correlation coefficient formula, with 90% power and 95% confidence interval and was calculated to be 48 children in each age group). For those who were younger than 24 months and were born prematurely, we calculated and considered the corrected age. The study was approved by the research committee and thereafter by the ethical committee of the University of Social Welfare and Rehabilitation Sciences. Parents were informed about the importance of developmental screening and how the test was performed. Then their written consent was acquired. For all children, at first, age-related questionnaires were completed by caregivers and then examiners performed the DDST-II. Because we did not have accessibility to any diagnostic developmental gold standard test and also because PDQ-II is a prescreening test, we used the DDST-II as gold standard test, so that the results of the PDQ-II were cross tabulated against DDST-II to obtain its sensitivity and specificity in comparison to DDST-II and also to determine their Kappa agreement coefficient. By considering this limitation, we used co-positivity of the two tests as a substitute for sensitivity and their co-negativity, for specificity. In 10% of children in order to determine the interrater reliability of the PDQ, after parents completed the questionnaires and the DDST-II was performed, the examiners asked the parents the PDQ questions again and completed the related forms. Also, Cronbach's α was used to determine reliability of the test. Results were interpreted by researchers. Data were coded and analyzed by SPSS software. For ethical purposes, all parents whose children were detected as delayed or suspicious were referred for additional evaluations and interventions.

Findings

In the present study 237 children consisting of 108 (46%) girls and 129 (54%) boys aged 0–6 years were screened by PDQ and DDST. 96% of cases were born term. Maternal education of 84% of children was at high school level or higher. In prescreening by PDQ the 'normal', 'delayed' and 'suspect' cases were 62%, 18% and 20% respectively. In screening by DDST-II 64.5% of children (68% of girls and 62% of boys) were normal and 35.5% of them were detected as delayed. The results of the two tests are shown in Table 1.

Table 1.

Comparing the results of PDQ-II and DDST-II

	DDST-II results: No. (%)
PDQ-II results	Delayed	Normal
Delayed	30 (12.6)	12 (5)
Normal	31 (13)	116 (48.9)
Suspect	23 (9.7)	25 (10.5)

PDQ-II: Prescreening Developmental Questionnaire 2

DDST-II: Denver Developmental Screening Test-II

Using the PDQ-II, the number of children falling in the categories of “caution” and “delayed” was higher in the gross motor and language domains. Whereas with DDST-II, “delays” and “cautions” were higher in the language domain and “delays” alone, were higher in fine and gross motor domains.

For determining the measure of agreement between PDQ and DDST results, considering the fact that for cross tabulating two tests, they must have similar number of answer choices, we first considered the “suspect cases” of the PDQ as “normal” and the next time, as “delayed”. When suspect cases were considered as delayed, the kappa measure of agreement was 0.383 (P<0.001) and when considered as normal, it was 0.314 (P<0.001).

The mean sensitivity (co-positivity) and mean specificity (co-negativity) of PDQ in comparison to DDST were 35.7% and 75.8% respectively. If suspect cases were considered as normal, the sensitivity and specificity of PDQ were 35.7% and 92.2% respectively, and they changed to 63% and 75.8% when suspect cases were considered as delayed.

Cronbach's α was determined for all developmental domains in each questionnaire. As observed in Table 2, it was above 0.5 in all domains and in all age groups, except the personal-social domains at 4-6 years of age. Kappa measure of agreement for interrater reliability of the test was 0.89 (P<0.001).

Table 2.

Cronbach's α coefficient of Prescreening Developmental Questionnaire 2 in all developmental domains

Domain	0–9 months	9–24 months	2–4 years	4–6 years
Personal-social	0.673	0.846	0.765	0.295
Fine motor	0.870	0.427	0.763	0.724
Language	0.800	0.828	0.900	0.813
Gross motor	0.835	0.832	0.867	0.525
Total	0.951	0. 926	0.950	0.876

Discussion

In our study the content validity of PDQ was verified and there was no need to change any of the questions in the questionnaires.

A study was performed in Saudi Arabia on 1219 children. In order to use PDQ as a prescreening tool for their population, researchers made some changes in the contents of the questionnaires, age intervals, normal age consideration for acquisition of developmental skills and also their order of emerging. These changes were greater in the language domain and showed the importance of adapting developmental screening tools before using them in other countries [13]. Another study in India showed that because of cultural differences between USA and India, PDQ was not suitable for that country [21].

We evaluated the reliability of the test by the Cronbach's α determination. It was "very good" for the 0–9, 9–24 months and 2–4 years questionnaires and "good" for the 4–6 years questionnaire. The interrater method was also used as another way for reliability determination and it was 0.89 which was also “very good”. This study showed that the reliability of PDQ was "very good" on the whole.

In 1987 Frankenburg et al showed that the reliability of PDQ in test-retest and interrater testing (parents-teacher) were 94.1% and 83% respectively [22].

By considering suspect cases as normal, the sensitivity and specificity of the PDQ compared to the DDST-II were 35.7% and 92.2%, respectively and the referral rate (that is, the delayed cases on the PDQ who were referred for performing the DDST-II) was 12.7% and the false negative rate was high. However, if suspect cases were considered as delayed, sensitivity and specificity rose to 63% and 75.8%, respectively and the referral rate rose to 22.7%. The latter values for sensitivity and specificity are closer to the acceptable range for developmental screening tools and the referral rate is more similar to the value presented by the authors of the original test. Also because in this form, the agreement coefficient between PDQ and DDST-II is better than the previous case, it seems that for having more valid results, suspect cases must be considered as delayed. Also it is suggested that existence of ≥1 delays or ≥2 cautions be considered as referral criteria (instead of ≥2 delays or ≥3 cautions).

In India a similar study was performed using the 2–4 year-old questionnaires of PDQ and showed that if ≥1 delays were considered as abnormal, sensitivity, specificity and referral rates were 100%, 7.8% and 92.6%, respectively and thus two stage screening would not be meaningful. When they considered ≥2 delays as abnormal, sensitivity, specificity and referral rates changed to 18.2%, 42.6% and 53.9%. In this situation, sensitivity and specificity of the test were not acceptable for developmental screening. They concluded that PDQ was not suitable for use in India [21].

The present study showed that regardless of considering suspect cases as normal or delayed, the agreement coefficients of PDQ and DDST-II were weak and the 2–4 year- old questionnaire had greatest agreement with DDST-II.

Berges et al conducted a research in order to determine the agreement coefficient of PDQ, its modified version (M-PDQ) and another questionnaire named Alpern-Boil Developmental Profile-II, with DDST. They concluded that all tools had good agreement with DDST and could be used for rapid developmental screening in preschool children [23]. Another study in India showed that the 2–4 year-old questionnaire of PDQ had no good relationship with DDST [21]. In another study, term and Very low birth weight infants were screened by PDQ and the Grifffiths developmental scale at 12 months of age. The study showed that these two developmental screening tools had good agreement and the PDQ was a reliable tool for infant developmental screening [24].

There was no relationship between the presence of developmental delays and such factors as sex, place of residence and maternal education in the present study.

A study in an urban area of India showed that uneducated mothers had not good information about the developmental abilities of their children such as ability to draw a circle or a straight line. Thus due to the low educational level of mothers in that area, PDQ could not evaluate the developmental status of children correctly [21]. A study was done using the PDQ in urban and rural areas of Japan with different climates. It showed that in summer the results of children developmental screening in rural areas was lower than jn urban regions but in winter, there was no difference between them.

Researchers concluded that in summer, mothers in rural areas did not have much contact with their children because of farm work, and thus were not aware of their children's developmental status. They concluded that for using the PDQ and similar questionnaires such factors need to be considered too [25].

In the present study developmental prescreening with PDQ showed that the number of cautions and delays were greater in the gross motor and language domains whereas by using the DDST–II the total number of cautions and delays was greater in the language domain and the number of delays alone, was more in the fine and gross motor areas. A study in Shiraz, Iran showed that by using DDST-II, the developmental status of 3–6 year old children in fine and gross motor domains was lower than the Denver normative sample [26]. Other studies in England, Japan, Sweden and Israel showed that children in those countries had a slower rate of fine and gross motor development [21, 25, 27–30]. In Cardiff, children were better in some personal-social items on the whole, and children younger than 18 months were better in the language domain [27]. Also children of Tokyo were faster in some personal-social items [28]. In another study, developmental screening of children in Tehran showed that by using DDST-II and ASQ (Ages & Stages Questionnaires], developmental delays were greater in the gross and fine motor areas[31].

The present study showed that there is a relatively weak correlation between results of PDQ and DDST-II in children of Tehran. Thus using PDQ and DDST-II in order to perform two-step developmental screening is doubtful. Other studies have previously shown that questionnaires completed by parents, may not have good agreement with each other[32]. The important fact is that the results of screening tools should not be interpreted alone, but decision should be made by considering the child's total function and environmental factors. Furthermore, developmental screening tools are most useful when they are repeated periodically[9, 33, 34].

Different studies in Iran have shown the rather considerable prevalence of developmental disorders in Iranian children [4, 26, 31, 35, 36]. Thus decision makers of child health care should consider suitable strategies for control of risk factors, emphasizing on prevention and early detection of developmental disorders and providing early intervention services in the country.

This study had some limitations such as limitation of time and resources for re-evaluating those children who were detected as cautious or delayed as well as absence of a gold standard developmental diagnostic test for determining the concurrent validity of the results of prescreening and screening with the PDQ-II and DDST-II, respectively.

Conclusion

This study showed that all questionnaires of PDQ have a good content validity and reliability and moderate sensitivity and specificity and there was no need to change any of the questions. We found that in our study there was a relatively weak agreement coefficient between PDQ and DDST-II, thus using PDQ along with DDST-II for a two-stage developmental screening process, remains doubtful. We recommended that other researchers confirm this finding with further studies.

Conflict of Interest

None