Reliability and Validity of the Korean Version of the Penn State Worry Questionnaire in Primary School Children

Abstract

Worry is a generalized psychological phenomenon seen among most people. When worry is excessive and nearly uncontrollable, people usually suffer psychological pain. The Penn State Worry Questionnaire for Children (PSWQ-C) was developed to measure worry objectively. It comprises 14 items for measuring excessive, generalized, and uncontrollable worry in children. This study, conducted with a large group of elementary children (3rd through 6th graders, ages 8-12 yr; N=973), investigated the reliability and validity of the Korean version of the Penn State Worry Questionnaire for Children (PSWQ-CK). The PSWQ-CK showed high reliability by test-retest and also excellent internal consistency results. To examine the validity of the PSWQ-CK, we calculated its correlation with the Revised Children's Manifest Anxiety Scale (RCMAS). The PSWQ-CK had a higher correlation with the worry/oversensitivity factor than with other subscales of the RCMAS, and it showed no correlation with the lie factor. When 3 reversed PSWQ-CK items were eliminated, the instrument showed higher internal consistency. However, this did not improve its correlation with other anxiety-measuring tools. In conclusion, the PSWQ-CK's reliability and validity were satisfactory, and it is a useful tool for objectively measuring the worry of Korean children of this age group.

INTRODUCTION

Worry is a generalized psychological phenomenon that can be seen among most people. Worry is a chain of thoughts and images, negatively affect-laden and relatively uncontrollable. The worry process represents an attempt to engage in mental problem-solving on an issue whose outcome is uncertain but contains the possibility of one or more negative outcomes; Consequently, worry relates closely to the fear process (1). Unless it is excessive, worry has a positive function; it aids in problem-solving. However, because of their characteristics or factors in their environment, some people have more worries than they can deal with and suffer psychological pain as a result. Worry is the core feature of generalized anxiety disorder, and is an anxiety disorder subtype.

According to recent studies, worry has an important role not only in the onset and maintenance of anxiety disorder but also in some somatic health issues. For example, Brosschot et al. reported that perseverative cognition, as manifested in worry and rumination, is a common response to stress and might act directly on a somatic disease via enhanced activation of the cardiovascular, immune, endocrine, and neurovisceral systems (2).

To measure worry objectively, Meyer et al. developed the Penn State Worry Questionnaire (PSWQ) when investigating clinical and nonclinical groups of adults (3). This tool consisted of 16 Likert items and showed excellent internal consistency, test-retest reliability, and concurrent and discriminative validities. Later, in a clinical group study by Brown et al., the PSWQ again displayed high reliability and validity (4).

To apply the valuable psychometric properties of the PSWQ for adults to groups of children and youths, Chorpita et al. developed the Penn State Worry Questionnaire for Children (PSWQ-C) by making the PSWQ more understandable and simple (5). The PSWQ-C is comprised of 14 items and contains 3 reversed scored items: "I do not really worry about things", "I find it easy to stop worrying when I want", and "I never worry about anything." According to investigation of Chorpita et al. on the psychometric properties of PWSW-C, the instrument showed excellent concurrent and discriminative validities, internal consistency, and test-retest reliability for children and youths ages 6-18 yr.

In a study of 486 children ages 8-12 yr, which was conducted by Muris et al., the PSWQ-C's reliability and validity again turned out to be good. However, in contrast to Chorpita et al., Muris et al. found it preferable to discard the 3 reverse-scored questions on the PSWQ-C and use only the remaining 11 items for this age group, because of unacceptable factor loading (6).

At any rate, there had been no tool for evaluating Korean children for worry. Therefore, in this study, we developed a Korean version of the PSWQ-C (PSWQ-CK) and tested its psychometric features with a large group (N=950) of children aged 8-12 yr, focusing on the following three elements.

First, we measured the PSWQ-CK's correlation with the subscales of the Revised Children's Manifest Anxiety Scale (RCMAS), which already has a Korean version, to investigate its convergent validity. As Chorpita et al. also used the RCMAS to test the convergent validity of the PSWQ-C, we expected that the results would be similar to those of Chorpita's study determine.

Second, we investigated factor structure and reliability of the instrument for this age group and determine whether, in this Korean version, the 3 reversed items out of the PSWQ-C would show unacceptable factor loading for this age group. Muris et al. proposed that the 3 reverse items in the PSWQ-C should be removed for children ages 8-12, because such children could have difficulty dealing with the double negative and also because those items show unacceptable factor loading (6). However, even if this age group might have some limits to their reading comprehension, it seem not likely to be so severe that they would not be able to understand the 3 reversed items. Therefore, they were considered to have a role, at least to some degree, in measuring the children's worry. Finally, by presenting normative data for the PSWQ-CK, we made it possible to compare the scores of children in this age group.

MATERIALS AND METHODS

Subjects and methods

The initial participants of the study were 1,049 children in grades 3-6 at 3 elementary schools: one in Seoul City; one in Suwon City; and one in Seongnam City (Bundang District) in Gyeonggido. First, we informed the students' parents of the questionnaire survey, through the notice bulletin used for communication between teachers and parents in the elementary schools, and asked whether they would participate. The parents of 76 students did not agree to participate (rejection rate 7.2%), leaving 973 participating students; 488 boys and 485 girls. The students' mean age was 10.6 yr, with a standard deviation of 1.1 yr and age range of 8-12 yr. The age distribution was 8 yr, 3.0%; 9 yr, 13.2%; 10 yr, 29.7%; 11 yr, 29.3%; and 12 yr, 25%. The grade distribution was 3rd grade, 14.3%; 4th grade, 29.1%; 5th grade, 28.0%; and 6th grade, 28.7%. Of these, 883 students participated in the test-retest reliability measurement. The period between test and retest was 3 weeks.

The researchers explained the purpose of this study, the configuration and contents of the questionnaire, and the attentive matters to the children's teachers and asked them to give brief explanation about the questionnaire to all the participating students before carrying out the survey. The participating students completed the questionnaire in their classrooms, and allowed to ask questions if something was unclear, to hopefully assure the students' precise understanding and response. The study was approved by the institutional review board (2007.11.05), and performed in accord with currently accepted ethical and safety guidelines.

The Korean version of the penn state worry questionnaire for children (PSWQ-CK)

The PSWQ-C is a 14-item questionnaire for measuring whether children have excessive, generalized, and uncontrollable worries (5). Originally, the PSWQ for adults had a 5-point scale. However, the one for children has a 4-point scale: 0=Not at all, 1=Sometimes yes, 2=Frequently yes, 3=Always yes. The overall total point range of the PSWQ-C is 0 to 42 points, where higher points indicate a higher tendency to worry. The translation procedure that created the Korean version is shown below.

The revised children's manifest anxiety scale (RCMAS)

The RCMAS is a children's anxiety scale that was created by Reynolds et al. in 1978. It is a revision of the Children's Manifest Anxiety Scale (CMAS) developed by Castaneda et al. in 1956 (7-9). RCMAS is comprised of 37 questions, which the participant answers either "yes" or "no." Of these items, 28 are about anxiety phenomena. The remaining 9 items are the lie scale, which evaluates the tendency to conform to socially desirable responses. The RCMAS contains a worry/oversensitivity factor, a physiologic anxiety factor, and a concentration factor (9, 10). The worry/oversensitivity factor is comprised of items like "I worry a lot of time," "I worry about what other people think about me," "I worry about what is going to happen," and "I often worry about something bad happening to me." The physiological factor includes items such as "I have bad dreams" and "Often I feel sick in my stomach," while the concentration factor includes "Other children are happier than I" and "Others seems to do things easier than I can." This factor structure is used as the general scoring system. It is one of the most worldwide tools for the evaluation of children's anxiety. This study used the Korean Version of the RCMAS, translated by Choi and Cho (11).

Translation of items

The translation of the items was made under the approval of the PSWQ-C's original authors. In translating the items, we followed the procedure guidelines recommended by Guilleman et al. and Beaton et al. (12, 13). First, two different translators who could speak both English and Korean, but whose native language was Korean, translated the items from English to Korean. To enhance the quality of the translation, one of the two was first author of this study, while the other did not know the purpose of this questionnaire at all. Next, 4 medical specialists discussed difference in the two translations and determined one complete translation. At this point, we selected three 8- to 12-yr-old children, who were not participants in this research and had lived in English-speaking countries for at least 3 yr, to translate the original questionnaire. We compared their translation with the composite translation to check whether children of the target age range could understand the questionnaire. Then, in the back translation stage, two new translators translated the composite questionnaire back into English. The two back translators had neither worked in the medical field nor knew the purpose of the questionnaire; they could speak both English and Korean, but their native language was English. Next, the 4 medical specialists who had determined the composite translation discussed the back translation and analyzed it for appropriateness. Finally, after completing the final Korean version, teachers with experience of teaching 3rd to 6th grade primary school students were asked whether children would be able to understand the translated version. As the result, we regard the final version of PSWQ-CK as satisfactory.

Statistical analysis

For all the statistical analysis of this research, we used the Korean Version of the Statistical Package for Social Sciences (SPSS) version 12.0 for Windows. To determine the PSWQ-CK's reliability, we calculated the internal consistency; for the reliability of the test-retest method, we measured the correlation coefficient; to find out the convergent validity, we derived its correlation coefficient with the RCMAS. In addition, we performed t-test to investigate for any difference between boys and girls and an ANOVA to check the differences among grades. We also carried out an exploratory factor analysis and used the SAS v9.1 to analyze the Z-test for comparing the dependent correlations (14). A P value of less than 0.05 was regarded as statistically significant.

RESULTS

General characteristics of study subjects

The average PSWQ-CK score for all participants was 14.49, with a standard deviation of 8.06 points. There was no significant difference in terms of genders. By grade, we found the 6th graders scored significantly higher than 4th or 5th graders (P<0.05). Even though it was not statistically significant, the 6th graders' score was higher than that of the 3rd graders (P=0.09). Among the 6th graders, the boys' average PSWQ-CK score was 15.3, with a standard deviation of 8.1, while the average girls' score was 17.1 with a standard deviation of 8.7. Even though it was not statistically significant, the girls' score was higher than the boys' (P=0.07). There was no interaction effect of gender by grade (P=0.315). Descriptive statistical data of RCMAS are summarized in Table 1. There was no significant difference in PSWQ-CK scores among the schools (P=0.936).

Table 1.

Descriptive statistics for measures and subscales

^*The P value was determined using the Student's t-test and one-way ANOVA; ^†P<0.05 versus grade 4 and 5 by Scheffe's post hoc test after one-way ANOVA.

RCMAS, the Revised Children's Manifest Anxiety Scale; PSWQ-CK, the Korean Version of the Penn State Worry Questionnaire for Children.

Reliability by test-retest and internal consistency

We investigated the PSWQ-CK's test-retest reliability with the interval of 3 weeks in 883 students. The correlation coefficient was 0.83 (P<0.001), showing satisfactory reliability and stability. The Cronbach's alpha for internal consistency of the PSWQ-CK was 0.898. The item-total correlations are presented in Table 2. When the 3 reversed items were eliminated, all the item-total correlations increased, and the Cronbach's alpha also improved from 0.898 to 0.913.

Table 2.

Item-total correlation for the original and the shortened version of the PSWQ-CK

PSWQ-CK, the Korean Version of the Penn State Worry Questionnaire for Children.

Correlation between PSWQ-CK and RCMAS

We compared the results of the original PSWQ-CK, with the 3 reversed items, and of the short version of the PSWQ-CK, without the 3 reversed items, to that of RCMAS, to verify their correlation (Table 3). The result of a Z-test for comparing dependent correlation revealed that the original PSWQ-CK had an excellent correlation in the worry/oversensitivity subscales of RCMAS, exceeding their correlations with other subscales (P<0.01), but did not show a significant correlation with the lie scale (all Z ≥9.0, P<0.001).

Table 3.

Correlations of the original and shortened PSWQ-CK with RCMAS

^*P<0.01, two-tailed.

PSWQ-CK, the Korean Version of the Penn State Worry Questionnaire for Children; RCMAS, the Revised Children's Manifest Anxiety Scale.

In the case of the short PSWQ-CK (without the 3 reversed items), its correlation with the RCMAS subscales was a little worse than that of the original PSWQ-CK.

When 3 reversed items were analyzed separately, the correlation coefficient with RCMAS was small, but they showed a statistically significant, positive correlation with the worry/oversensitivity factor, exceeding their correlations with other subscales.

Factor analysis

For the investigating factor structure of the PSWQ-CK, we carried out factor analysis using maximum likelihood method (Table 4). We selected the vari-max method as the rotation method, drawing out the factors with eigen-values greater than 1 (i.e., 6.35, 1.26). We found 36.3% of the total variance could be explained by factor 1 and 10.7% by factor 2. In particular, factor 2 was composed of the 3 reversed items (Items 2, 7, 9). After these 3 reversed items were eliminated, we repeated the factor analysis using the same method. This time, there was only one factor with an eigen-value greater than 1 (5.95), which explained 49.6% of total variance.

Table 4.

Item-factor loadings for rotated two-factor solution using maximum likelihood factoring method (N=950)

Normative data

Table 5 shows the distribution of normal scores by grades. As there were differences in worry levels by grade, the accumulative percentage of the 6th graders whose worry score was significantly higher than those of the 3rd, 4th, and 5th graders was presented separately, along with that of the total group. Table 5 also displays 10% cut-off scores.

Table 5.

Norms for the PSWQ-CK: cumulative percentages (upper panel) and cut-off points for 10 equal group (lower panel) calculated for the total sample and for the grade 3,4,5 and the grade 6 separately

PSWQ-CK, the Korean Version of the Penn State Worry Questionnaire for Children.

DISCUSSION

Individuals' worrying topics are diverse, from health to family, financial problem, interpersonal problems, and business issues. They tend to worry more about future things than past (15). In several community studies, 80% of elementary students ages 8-12 yr worry every day about school, health, death/disease, separation from parents, etc. (16, 17). Childhood worry includes the expectation of one or more threatening events, and the children who worry express the latent negative result of such events exaggeratedly. Like adults, children who worry tend to exaggerate the danger possibility and magnitude of the threat clues and to selectively interpret vague stimulations as threatening ones (18).

Borkovec et al. reported that the children's worry was closely related to emotions such as fear and anxiety (1). According to them, there is a very close correlation between worry and trait anxiety, a type of anxiety having the character of temperament, which determines personal differences in responding to traumatic threats. Trait anxiety is maintained over one's lifetime. Anxiety due to worry has an important impact on the social and cognitive development of children. In particular, by diminishing self esteem, it has a negative influence on children's academic achievement, friendships, and social participation (19). Children with many worries do not lack knowledge of problem-solving strategies but experience difficulty in exerting their power, because of anxiety and fear from their worries. Due to this, they become inattentive, confused, and negative in their feelings (20, 21).

Worry is not only the most essential feature of generalized anxiety disorder but also an important feature in other anxiety disorders. In a study that evaluated worry of normal children, about 6.7% of the children showed levels of pathological anxiety that satisfies the DSM diagnostic criteria of overanxious disorder or generalized anxiety disorder (22). Another study reported that worry was specially related to the symptoms of generalized anxiety disorder but was also significantly related to all other anxiety disorders (6). Therefore, an understanding of children's worry is very important for the treatment of children with generalized anxiety disorder or other anxiety disorders, such as separation anxiety disorder.

Thus, purpose of this study was to develop the Korean Version of the PSWQ-C, with which worry can be objectively measured, and to investigate its psychometric features.

The PSWQ-CK showed high internal consistency, and the test-retest at an interval of 3 weeks also showed a high correlation. Thus, the instrument showed high reliability and stability. In particular, it had a markedly higher correlation with the worry/oversensitivity subscale of the RCMAS than with the other subscales, showing good convergent validity.

When the 3 reversed items were eliminated from PSWQ-CK, the reliability based on internal consistency clearly increased, but, in the validity analysis, its correlation with the RCMAS subscales decreased a little. This is somewhat different from the result of the study by Muris et al. (6). According to them, when the 3 reversed items are removed from PSWQ-C, the reliability and the validity increase because this age group is so inattentive they may not understand the terms in the questionnaire, may fail to notice whether the items are reversed, and may have difficulty dealing with the negative words of the reversed items or double negations like "not at all." In the study by Chorpita et al. (5), these 3 reversed items had no appropriate factor load. However, according to Pestle et al., who evaluated the psychometric properties of the PSWQ-C in a large clinical sample (n=491), given that the reverse-scored items make a small but significant contribution to the validity of the PSWQ-C, it is recommended that all 14 items be retained for use with clinical sample of youth (23).

In our study, the factor load and item-total correlations of the 3 reversed items was greater than that found in the studies of either Muris et al. or Chorpita et al. (all the item-total correlations were greater than 0.3) and, even if all the 3 reversed items were eliminated, the correlations with other anxiety scales did not increase (5, 6). Furthermore, when the 3 reversed items were separately compared to the RCMAS subscales, their correlation with the worry/oversensitivity factor was significantly higher than that with the other subscales. This result means that the 3 reversed items have an influence, however small, on this instrument's validity and, therefore, when the 3 reversed items were eliminated, the correlation coefficient with the RCMAS subscales decreased.

Additional studies will be necessary to find out why the result of this research was different from that of Muris et al. However, one possibility is the linguistic structure difference between English and Korean. Unlike in English, the verb comes at the end of the sentence in Korean. Therefore, the positive or negative word also comes at the end. The consequence is students may give more attention to the last part of the sentence, as usual, when reading it in Korean. Alternatively, children's reading comprehension abilities may have improved in general, thanks to precedent learning (learning the school subjects in advance through private lessons, etc.) on the part of children, which is facilitated by parents and is now common in Korean society. As the elementary schools that participated in this study are located in the relatively rich regions of Korea, the above explanation is sufficiently possible.

Additionally, we used the PSWQ-CK to investigate whether there were differences in the degree of worry according to children's gender and grade. In this study, the 6th graders' mean PSWQ-CK score was significantly higher than that of the 4th and 5th graders and tended to be higher than that of the 3rd graders, even though the latter was not statistically significant. Lim et al. reported that 6th graders mainly worried about friends, external appearance, economy, future, and especially academic scores (24). In Korea, elementary school has 6 grades. After finishing it, children enter middle school, where they are to compete with each other to enter better high schools and better universities. The 6th grade children are fully aware of this situation, their PSWQ-CK score was probably higher than those of other grade.

In the studies of Muris et al. and Chorpita et al., girls' mean PSWQ-C scores were significantly higher than those of boys (5, 6). However, this study found no such difference between girls and boys. The PSWQ-CK score of girls was higher than that of boys only in 6th graders, and then it was not statistically significant. There have been no researches about the difference between boys and girls for the PSWQ-C. When we reviewed earlier research results on sexual differences in the RCMAS, we found no significant difference in mean RCMAS scores by genders in a study of Korean elementary children by Choi et al. and in a study of Nigerian elementary children by Pela et al. However, in a study of American children by Reynold et al., girls' mean RCMAS score was significantly higher than that of boys (8, 11, 25). Pela et al. stated that these results were probably caused by cultural differences. Unlike in the precedent studies, the PSWQ-CK showed no sexual difference, with regard to the fact that, additional studies seem necessary. These should select a wider sample group in consideration of cultural backgrounds, such as social pressure or roles of men and women in a society.

In conclusion, the PSWQ-CK possessed satisfactory in reliability and validity. It can be a useful tool to objectively measure the worries of Korean children. Future investigations need to examine the reliability and validity of PSWQ-CK for youths, the difference between boys and girls, and the contents of children's worry in Korea. They should also study clinical groups having generalized anxiety disorder or other anxiety disorders.