Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2012 Feb 8.
Published in final edited form as: Psychol Sch. 2000 Mar 1;37(2):91–106. doi: 10.1002/(SICI)1520-6807(200003)37:2<91::AID-PITS1>3.0.CO;2-3

IDENTIFYING PSYCHOSOCIAL DYSFUNCTION IN SCHOOL-AGE CHILDREN: THE PEDIATRIC SYMPTOM CHECKLIST AS A SELF-REPORT MEASURE

MARIA E PAGANO 1, LINDEN J CASSIDY 1, MICHELLE LITTLE 1, J MICHAEL MURPHY 1, MICHAEL S JELLINEK 1
PMCID: PMC3274771  NIHMSID: NIHMS353059  PMID: 22328794

Abstract

The Pediatric Symptom Checklist (PSC) is a brief, well-validated parent-report questionnaire designed to detect psychosocial dysfunction in school-age children during pediatric primary care visits. This study assessed the utility of the PSC when completed by children (PSC-Y) ages 9–14 in a public school when parents are not available (n = 173). The PSC-Y identified 20% of children as having psychosocial problems, a rate similar to other low-income samples. When compared with teacher ratings of attention and behavior problems, the PSC-Y showed a sensitivity of 94% and a specificity of 88%. The PSC-Y correlated significantly with teacher and parent measures of child dysfunction, and with child-reported symptoms of depression and anxiety. Three quarters of the children identified by the PSC-Y were not identified by parents on the PSC. These children had impairment on all other measures, but fewer than one in five had received mental health services, suggesting the PSC-Y identified children with unmet mental health needs. The PSC-Y has the potential to be a rapid, easily administered tool for large-scale mental health screening in schools.

Psychosocial dysfunction in children, first recognized twenty years ago as the “new morbidity” in pediatric practice, is now the leading cause of disability in childhood and adolescence (Costello & Pantino, 1987; Costello et al., 1988a, 1988b; Kelleher & Wolraich, 1995). Epidemiological research indicates that 14 to 20% of American children have one or more psychiatric disorders in the moderate to severe range (Brandenburg, Friedman, & Silver, 1990). Poor and minority children, who are a growing proportion of our population, are at even higher risk (Bird et al., 1988; Costello et al., 1996a; Zahner, Pawelkiewicz, De francesco, & Adnopoz, 1992), and the overall prevalence of children’s emotional and behavioral problems is increasing (Achenbach & Howell, 1993). Despite this growing prevalence, it is estimated that only one sixth to one half of the children with psychiatric disorders are identified. Of these, at most one fifth receive mental health services (Burns et al., 1995; Costello et al., 1988b). Numerous studies have shown that untreated mental health problems place children at risk for school failure and serious psychiatric complications, including antisocial and self-injurious behavior (Kessler, Foster, Saunders, & Stang, 1995; Koot & Verhulst, 1992; Rutter, 1991).

Traditionally, pediatricians have been seen as primary resources for the recognition of psychosocial problems in children and adolescents (Costello & Pantino, 1987), and screening within pediatric primary care has been shown to improve recognition rates (Horwitz, Leaf, Leventhal, Forsyth, & Speechley, 1992; Stancin & Palermo, 1997). In recent years, however, much of the burden of children’s psychosocial problems has fallen to schools (Zahner et al., 1992) following federal mandates to provide special education for handicapped children and the rising difficulty and cost of accessing other avenues of care, such as pediatric subspecialties. Health benefits under managed care tend to discourage comprehensive mental health services for children, adding to the demands placed on schools (Jellinek & Hurcombe, 1993; Norquist & Wells, 1991). A recent epidemiological study showed that as many as 70–80% of children receiving mental health services do so only at school, making the education system “the de facto system of care for children with mental health problems” (Burns et al., 1995).

Given educators’ concerns about the grave impact that mental health problems have on children’s learning, schools offer ideal settings for large-scale screening of children and adolescents. Schools have well-developed standards for age-appropriate expectations for children, and they provide a longitudinal view of children’s functioning in a normative, controlled setting. For older children and adolescents, who may be asserting independence from parents and home and focusing more on peers and school activities, school-based screening and teachers’ reports may be especially important (Edelbrock, Costello, Dulcan, Kalas, & Conover, 1985). School-based screening may identify troubled youth in need of services before disciplinary measures are required, because adolescents are more likely than younger children to be suspended or expelled than to receive services (Knitzer, 1982).

Recent studies suggest that diagnostic symptom criteria alone do not accurately identify children in need of mental health services (Boyle et al., 1996). Without concomitant clinical assessment, diagnostic criteria alone provide no means by which to assess symptom-associated distress and impairment, and can thus lead to over-identification of cases. Functional impairment measures improve specificity by helping to distinguish serious emotional disturbances from mild or transient conditions (Bird & Yager, 1990; Costello et al., 1996b; Gotlib, Lewinsohn, & Seeley, 1995). A recent comparison of mental disorder rates in two large community surveys showed that differing rates of morbidity resulted from a lack of standardized impairment criteria (Regier, Kaelber, & Rae, 1998). Assessing symptoms can reveal causes of impairment and assist in prioritizing treatment needs as an additional step. This may be especially true for older children and adolescents for whom rates of diagnosable and treatable disorders, particularly depression, increase significantly (Kandel & Davies, 1982; Kashani & Sherman, 1988).

While screening tools based on teacher or parent report provide an accurate measure of both younger and older children’s impairment or disruptive symptoms, internalizing problems are best identified through youth self-report (Beitchman & Corradini, 1988; Kazdin, 1990; Reynolds & Kamphaus, 1990). Numerous studies have shown that observers are less accurate in detecting children with depression and anxiety, perhaps because their symptoms emerge differently than in adults. Some children with internalizing disorders show oppositionality or somatic concerns, while others may present with comorbid conditions such as attention deficit-hyperactivity, conduct disorder, or substance abuse (Carlson & Cantwell, 1980; Hammen & Compass, 1994). Thus, although internalizing disorders cause severe impairment, they often go undetected or are discovered late in their course, unless children and adolescents are asked about symptoms directly. This is especially true of older children and adolescents who are better able to conceptualize and report on their own feelings, and for whom parent report is less reliable (Edelbrock et al., 1985).

The ideal screening instrument for psychosocial problems in older children and adolescents would be brief, easy to administer, measure both internalizing and externalizing symptoms as relevant to dysfunction, incorporate both teacher and youth reports, and would be suitable for large-scale screening in a school-based setting. The Child Behavior Checklist and Youth Self-Report (CBCL, YSR) are well-normed and widely used in research (Achenbach, 1991), but their use in widespread screening has been hindered by their length (with over 100 items), cumbersome scoring (from scales to T-scores), and lack of overall functional scales. The Childhood Depression Inventory and Revised Children’s Manifest Anxiety Scale (Kovacs, 1982, 1992; Reynolds, 1980) have also been widely used in research to identify children with internalizing symptoms. They are not useful as diagnostic tools however, and as screening instruments their focus on only internalizing symptoms is overly narrow.

The most commonly recommended way to improve identification of psychosocial problems in children is to use brief, parent-completed screening questionnaires during routine pediatric office visits. The Pediatric Symptom Checklist (PSC) has been extensively validated in a wide range of populations over the past decade, and has been shown significantly to improve physician recognition of children’s psychosocial problems (see Jellinek, Little, Murphy, & Pagano, 1995, for a review). The PSC takes less than 5 minutes to complete and score, and reflects the parent’s impression of his or her school-aged child’s overall psychosocial functioning. Using an empirically defined cut-off point, it identifies children with functional impairment who are likely to benefit from further evaluation. Although originally designed for pediatric primary care, in a single study the PSC was shown to correlate highly with teachers’ and guidance counselors’ ratings of students’ need for services, academic failure and the PSC-Y, as completed by the junior high school students themselves (Murphy, Jellinek, & Milinsky, 1989). The PSC also identified a group of students whose difficulties were previously unknown to school staff. These results suggested that the PSC may predict dysfunction and may be a useful screening tool in adolescent students.

This study examines the screening utility of the PSC as completed by school-age children in urban public school settings. The PSC was adapted to a self-report format, the Youth-PSC (PSC-Y), by changing the relevant pronouns (Appendix). The goals of the study were: a) to identify the optimal cut-off score for the PSC-Y using the external criterion of teacher report of children’s impairment; b) to use a multi-method approach to establish preliminary concurrent validity of the PSC-Y with teacher and parent measures of child dysfunction, as well as child-reported symptoms on the Children’s Depression Inventory and Revised Children’s Manifest Anxiety Scale; c) to evaluate whether the PSC-Y identified additional children not already recognized by parents; d) to estimate unmet need by determining whether children identified by the PSC-Y were receiving mental health services; and e) to assess the reliability of the PSC-Y over time.

Appendix.

Pediatric Symptom Checklist –Youth Report (PSC-Y)

Please mark under the heading that best fits you:
Never Sometimes Often
1. Complain of aches or pains . . . . . . . . . . . . . . . . . .
2. Spend more time alone . . . . . . . . . . . . . . . . . . . . .
3. Tire easily, little energy . . . . . . . . . . . . . . . . . . . . .
4. Fidgety, unable to sit still . . . . . . . . . . . . . . . . . . .
5. Have trouble with teacher . . . . . . . . . . . . . . . . . . .
6. Less interested in school . . . . . . . . . . . . . . . . . . . .
7. Act as if driven by motor . . . . . . . . . . . . . . . . . . .
8. Daydream too much . . . . . . . . . . . . . . . . . . . . . . .
9. Distract easily . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10. Are afraid of new situations . . . . . . . . . . . . . . . . .
11. Feel sad, unhappy . . . . . . . . . . . . . . . . . . . . . . . . .
12. Are irritable, angry . . . . . . . . . . . . . . . . . . . . . . . .
13. Feel hopeless . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14. Have trouble concentrating . . . . . . . . . . . . . . . . . .
15. Less interested in friends . . . . . . . . . . . . . . . . . . . .
16. Fight with other children . . . . . . . . . . . . . . . . . . . .
17. Absent from school . . . . . . . . . . . . . . . . . . . . . . . .
18. School grades dropping . . . . . . . . . . . . . . . . . . . . .
19. Down on yourself . . . . . . . . . . . . . . . . . . . . . . . . .
20. Visit doctor with doctor finding nothing wrong. . .
21. Have trouble sleeping . . . . . . . . . . . . . . . . . . . . . .
22. Worry a lot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
23. Want to be with parent more than before . . . . . . .
24. Feel that you are bad . . . . . . . . . . . . . . . . . . . . . . .
25. Take unnecessary risks . . . . . . . . . . . . . . . . . . . . .
26. Get hurt frequently . . . . . . . . . . . . . . . . . . . . . . . .
27. Seem to be having less fun . . . . . . . . . . . . . . . . . .
28. Act younger than children your age . . . . . . . . . . .
29. Do not listen to rules . . . . . . . . . . . . . . . . . . . . . . .
30. Do not show feelings . . . . . . . . . . . . . . . . . . . . . . .
31. Do not understand other people’s feelings . . . . . .
32. Tease others . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
33. Blame others for your troubles . . . . . . . . . . . . . . .
34. Take things that do not belong to you . . . . . . . . . .
35. Refuse to share . . . . . . . . . . . . . . . . . . . . . . . . . . .
Open in a new tab

Methods

Sample

Data for the current analyses came from a collaborative study of a free breakfast program in the Philadelphia and Baltimore Public Schools. In four Philadelphia and Baltimore schools, the parents of all 679 students in grades 3–8 were invited to participate in the study through letters sent home with students. After an additional invitation letter and follow-up phone calls, 274 parents (40%) agreed to participate. When interviews were scheduled, 70 of the initially agreeing parents could not be scheduled, resulting in 204 complete parent/child interviews (74% of the agreeing sample and 30% of the total sample). Teachers completed ratings of attentional and/or behavioral problems on 85% of these children, resulting in a final initial sample of 173 children.

Procedures

Students and their parents in four elementary and middle schools (two in Philadelphia and two in Baltimore) were assessed on a battery of psychosocial and academic measures before the start of a free breakfast program in the schools. Independent of battery scores, teachers of all the students in the study were asked to rate the presence of attentional and/or behavioral problems. Students and their parents were interviewed in late January or early February prior to the start of “Universal Feeding” and then again in late May-early June to assess the effects of the school breakfast program on students’ psychosocial functioning.

Semi-structured interviews were conducted separately with each parent and child by a masters’-level research assistant and a lay interviewer from the community. The 90-minute structured interviews were conducted with parents and children and consisted of standardized measures. Interviewers completed sections of the interview with parents and children (such as the clinical ratings), and guided the parents and children through each of the self-completed questionnaires. Lay interviewers were part- or fulltime employees of the school and had associates’ or bachelors’ degrees. They were trained in the administration of the measures prior to the start of the interviews by a master’s-level research assistant. Interviewers were blind to the Pediatric Symptom Checklist completed by parents or children and to teachers’ ratings of impairment. Parents and children were blind to each others’ scores on interview measures. Teachers were blind to all interview measurement scores. The study was approved by the Human Studies Subcommittee at the Massachusetts General Hospital and by the research committees of the Philadelphia and Baltimore Public Schools.

Measures

Background Data

For the current study, the children’s grade level, gender, ethnicity, history of mental health services, and parental marital status were collected from the parent interview.

Teacher Report of Impairment

Teachers were asked to report whether an attentional and/or behavioral problem was “never”, “sometimes”, or “often” present in the children enrolled in the study. Children with teacher ratings of “often” in either category were considered to have an attentional and/or behavioral problem. The teachers’ reports of either attention or behavioral problems were combined to identify children with overall psychosocial functional impairment.

Parent-Report of Impairment

Pediatric Symptom Checklist (PSC)

The Pediatric Symptom Checklist (PSC) is a brief, widely used, parent-completed questionnaire that has been validated as a screening measure to identify children with psychosocial problems (Jellinek, Murphy, & Burns, 1986; Jellinek & Murphy,1990; Jellinek et al., 1995; Little & Murphy, 1994; Murphy & Jellinek, 1988, 1990; Murphy, Reede, Jellinek, & Bishop, 1992; Murphy et al., 1996; Rauch et al., 1991; Walker, La Grone, & Atkinson, 1989). The PSC surveys major areas of the child’s functioning and screens for problems with attention, hyperactivity, depression, conduct disorder, anxiety, etc. The PSC consists of 35 items that are rated as “never,” “sometimes,” or “often” present and scored 0, 1, or 2, respectively. A total score is obtained by adding the scores for each of the items, and impairment is defined as a total score of 28 or higher.

Overall functioning

Parents were asked, “Overall, how is your child functioning?” with possible ratings of “very well,” “well,” “so so,” and “poorly.”

Child Report of Depression and Anxiety Symptoms

Children’s Depression Inventory (CDI)

The CDI is perhaps the most widely used and certainly the most widely researched depression scale for children. Twenty-seven scale items, arranged in a three-choice format (scored to identify normalcy, some pathology, and clinically significant pathology) assess depressive symptoms over the previous 2 weeks. The CDI has been normed on at least two samples of preadolescent children (Finch, Saylor, & Edwards, 1985; Smucker, Craighead, Craighead, & Green, 1986). Although age/sex cut-off scores have been developed, the lower single cut-off of 12 recommended for research purposes was used in this study (Kovacs, 1992).

Revised Children’s Manifest Anxiety Scale (RCMAS)

The RCMAS is a 37-item checklist designed to assess the trait of anxiety in children ages 6 to 19. It has been well validated, used over a broad range of socioeconomic levels, and normative data are available by age and sex (Reynolds 1980; Reynolds & Paget, 1983; Reynolds & Richmond, 1979). A Lie subscale detects social desirability effects and deliberate faking of responses. The cut-off score indicating a clinical level of anxiety (18 or higher) was used in this study.

Receiver Operator Characteristic (ROC) Curve Analysis

To identify the optimal cut-off score for the PSC-Y, an ROC curve was constructed based on the PSC-Y’s performance compared with teacher identification of impairment. Because teachers can provide an accurate assessment of the functioning of school-age children, especially for older children and adolescents for whom parent report becomes less reliable (Edelbrock et al., 1985), the teacher report of attentional and/or behavioral problems was chosen as an external criterion for validity of the PSC-Y. The ROC curve plots the true-positive rate (sensitivity) on the y-axis against the false-positive rate (1–specificity) on the x-axis as the definition of a positive PSC-Y screening scores changes. The area under the curve represents the test’s ability to discriminate between “disease” and “nondisease.” A perfect test gives an area of 1.0 while a worthless test gives an area of 0.5 (Centor & Kneightley, 1989; Centor, 1991; Hsiao, Bartko, & Potter, 1989). The sensitivity, specificity, false-positive rate, Pearson r correlation, degree of concordance (κ), and prevalence rate were calculated for PSC-Y cut-off scores from 24–39.

Results

Sample

Our sample was derived from a predominantly low-income population. Eighty-five percent (543/679) of the children from the four schools were qualified to receive a free or reduced breakfast based on income eligibility requirements of less than $14,319 for a family with one child or $19,166 with two children (Murphy et al., 1998; School District of Philadelphia, 1996). Two schools were kindergarten through 6th grade, the third was kindergarten through 8th grade, and the fourth was 5th through 8th grade. Seventy-nine percent of the children were from non-White backgrounds (134/173 African American, 3/173 Asian). Forty-two percent (73/173) of the children resided in Philadelphia and 58% lived in Baltimore. The two groups were comparable in history of mental health services (χ2 = .48, p < .49) and parental marital status (χ2 = .03, p < .85). All the children from White backgrounds (36/36), and most of the adolescents (30/32) were recruited from Baltimore. There were no statistically significant differences between children from the two cities in teacher, parent, or child report of symptoms. Although no data is available about families who refused to participate in the study, the mean grade point and average school attendance of our sample did not differ significantly from the mean grade point and average attendance at the schools in our study.

The study design called for only half of the initial sample of 204 children to be re-interviewed at the end of the school year in late May, 4 months after the initial interviews. The parents of 106 children who had participated in the initial school breakfast interview were sent invitation letters for another interview. After follow-up phone calls, 96 (91%) were scheduled for interviews. Of these 96 subjects, 94% had complete child, parent, and teacher ratings at the initial interview, leaving a re-interview sample of 90 children.

Selection of PSC-Y Screening Cut-Off Score and Criterion Validity of the PSC-Y

The first goal of the study was to establish the optimal cut-off score for the PSC-Y using the Receiver Operator Characteristics curve. Figure 1 shows the ROC curve for the PSC-Y against the external criterion of teacher report of attentional and/or behavioral problems. The mean PSC-Y score for the children aged 9–14 in this study was 22.2 (SD =8.8; range 1 to 47). The area under the curve (AUC) equals 0.66 with a standard error of 0.039. The AUC for the PSC-Y as evaluated against teacher identification of impairment compares favorably with the AUC data for other psychiatric-screening tests (Weinstein, Berwick, Goldman, Murphy, & Barsky, 1989; Wyshak, Barsky, & Klerman, 1991). Table 1 provides a comparison of different PSC-Y screening scores with teacher report of an attentional and/or behavioral problem and summarizes data on PSC-Y screening scores of 24 to 39 including sensitivity, specificity, 1–specificity, Phi coefficient, degree of concordance (kappa), and prevalence of positive PSC-Ys. As Table 1 indicates, a cut-off score of 30 or higher provides the highest average of sensitivity and specificity (SN = 94; SP = 88), the highest Phi coefficient (φ=.62), and the highest degree of concordance (κ = .58) against the external criterion of teacher report of impairment. A cut-off score of 30 on the PSC-Y was also in the highest degree of concordance with case/non case classification on the CDI (κ = .47) and RCMAS (κ = .42). Using this cutoff score, 20% of the children were identified with psychosocial problems on the PSC-Y, higher than the PSC positive rate of 13% found in the current study and other studies (Jellinek et al., 1986, 1988; Murphy & Jellinek et al., 1988; Murphy et al., 1992), but similar to rates found in other low-income populations and to rates found for child-report measures (Bird, Gould, Rubio-Stipec, Staghezza, & Canino, 1991; Costello et al., 1996b; Stavrakaki & Williams, 1991).

Figure 1.

Figure 1

Open in a new tab

Receiver-operator characteristics (ROC) curve of PSC-Y scores versus teacher-report.

Table 1.

Comparison of Different Youth Pediatric Symptom Checklist (PSC-Y) Screening Scores Using Teacher Identification of Impairment as a Criterion (n = 173)a

PSC-Y score SN SP 1–SP Φ κ Prevalence
% N
24 94 62 39 .35 .23 44 76
25 94 68 32 .39 .29 38 66
26 94 74 26 .45 .35 33 57
27 94 77 23 .48 .39 30 52
28 94 79 21 .50 .42 28 49
29 94 84 16 .57 .50 24 41
30 94 88 12 .63 .58 20 35
31 78 90 10 .54 .52 17 30
32 72 92 8 .55 .53 15 26
33 67 94 6 .57 .57 12 21
34 56 94 6 .49 .49 11 19
35 50 96 4 .48 .48 9 16
36 39 96 4 .41 .40 8 13
37 39 98 2 .48 .46 6 10
38 39 98 2 .48 .46 6 10
39 33 99 1 .51 .45 4 7
Open in a new tab
a

The dashed line indicates the cutting score providing the maximum sensitivity, specificity, Phi coefficient, and kappa.

SN = sensitivity (true-positive rate), SP = specificity (true-negative rate) and 1–SP = 1.0 minus the specificity, or false-positive rate.

PSC-Y Scores and Background Characteristics

A breakdown of the sample’s background characteristics is presented in Table 2. The mean age of the sample was 10 years (SD = 1.5) and the mean grade was 5 (SD = 1.4). Eighty-two percent (141/173) of the sample were from elementary grades 3 to 5 and 18% (32/173) were from middle-school-grades 7 and 8. Approximately half the sample (54%; 94/173) were female and half were male (46%; 79/173). Twelve percent (20/173) of the children had a history of mental health services. Forty-seven percent (82/173) of the children were from single-parent households. There were no significant differences between PSC-Y case classifications and age, grade level, gender, ethnicity/racial groups, or history of mental health services. Children with PSC-Y positive scores were significantly more likely to come from single-parent families (χ2 = 4.2, df = 1, p < .05).

Table 2.

Background Characteristics and Youth Pediatric Symptom Checklist Scores

Characteristic PSC-Yscores
Total Non-case (<30) Case (30+)
173 (100) 138 (80) 35 (20)
Age of childa
 Preadolescent (7–11 years) 141 (82) 109 (79) 32 (21)
 Adolescent (12–14 years) 32 (18) 29 (91) 3 (9)
Gender Male 79 (46) 63 (46) 16 (46)
Female 94 (54) 75 (54) 19 (54)
Ethnicity Non-minority 36 (21) 26 (19) 10 (29)
Minority 137 (79) 112 (81) 25 (71)
History of mental health services No 153 (88) 125 (91) 28 (80)
Yes 20 (12) 13 (9) 7 (20)
Single parent* No 91 (53) 78 (56) 13 (37)
Yes 82 (47) 60 (43) 22 (63)
Open in a new tab
a

Trend.

*

p< .05.

Concurrent Validity of the PSC-Y with Teacher and Parent Report of Impairment and Child-Reported Symptoms

The second goal of the study was to use a multimethod approach to establish preliminary concurrent validity of the PSC-Y. The results of the PSC-Y were compared to teacher and parent report measures of impairment and to child-report measures of symptoms of depression and anxiety. The results are shown in Table 3.

Table 3.

Child Adjustment Scores and Youth Pediatric Symptom Checklist Scores

Measure Total PSC-Y-scores
Positive predictive value (%)
Non-case (< 30) Case (30+)
Teacher report 173 (100) 138 (80) 35 (20)
 Attention/behavior problem*** no 155 (90) 137 (99) 18 (51)
yes 18 (10) 1 (1) 17 (49) 49
Parent Reporta
 PSC case* no 151 (87) 126 (91) 25 (71)
yes 22 (13) 12 (9) 10 (29) 29
 PSC total score* mean 15.8 14.8 19.7
 Overall functioning** very well 69 (40) 63 (91) 6 (9%)
well 66 (39) 52 (79) 14 (21%)
so so 29 (17) 19 (66) 10 (34%)
poor 7 (4) 2 (29) 5 (71%)
Child reportb
 CDI case*** no 135 (78) 122 (88) 13 (37)
yes 38 (22) 16 (12) 22 (63) 63
 CDI total score*** mean 7.2 5.4 14.1
 RCMAS case*** no 141 (81) 124 (90) 17 (49)
yes 32 (19) 14 (10) 18 (51) 51
 RCMAS total score*** mean 11.0 9.7 16.1
 CDI or RCMAS case*** no 121 (70%) 111 (92) 10 (8)
yes 52 (30%) 27 (52) 25 (48) 71
Open in a new tab
a

PSC = Pediatric Symptom Checklist.

b

CDI = Children’s Depression Inventory/ RCMAS = Revised Children’s Manifest Anxiety Scale.

*

p < .01.

**

p < .001.

***

p < .0001.

PSC-Y scores and teacher-report of attentional/behavioral problems

Ten percent (18/173) of the children were rated by their teachers as often having an attentional and/or behavioral problem. Children with PSC-Y positive scores were significantly more likely to be identified by their teachers as having attentional and/or behavioral problems than were children with PSC-Y negative scores (49% vs. <1%; χ2 = 68.6, df = 1, p < .0001). The agreement in case/non case classification between the PSC-Y and teacher report of attentional and/or behavioral problems was 89% (154/173; SP = 88, SN = 94; κ = .58). Test–retest reliability for the teachers’ ratings were calculated on 71 children at a 4-month interval. The Pearson r was 0.61 for the correlation between the teachers’ cases (rated as “often” on either behavior or attention) at the 2 times. Teachers’ reports of either attention or behavioral problems were highly correlated (Pearson’s r = 0.56).

PSC-Y scores and parent-report

Thirteen percent (22/173) of the total sample were identified with parent-reported psychosocial problems on the PSC. As Table 3 shows, children with PSC-Y positive scores were significantly more likely to be cases on the PSC than were children with PSC-Y negative scores (29% vs. 9%; χ2 = 9.9, df = 1, p < .01). The mean PSC score for children with positive PSC-Y scores was significantly higher (19.7) than for children with negative PSC-Y scores (14.8; F = 6.5, df = 1, p < .01). The agreement in case/non case classification between the PSC-Y and PSC was 79% (136/173; SP = 83, SN = 46; κ= .23).

Four percent of the total sample was identified by their parent as poorly functioning overall. As Table 3 shows, this parent report of children’s day-to-day functioning was significantly associated with positive PSC-Y scores (χ2 = 20.56, df = 3, p < .001).

PSC-Y scores and child-report measures of depression and anxiety

Twenty-two percent (38/ 173) of the children were identified with depression on the CDI and 19% (32/173) of the children were identified with anxiety on the RCMAS. Thirty percent (52/173) of the children were CDI or RCMAS cases. The agreement in case/noncase classification between the PSC-Y and CDI was 84% (144/173; SP = 90, SN = 58, κ = 0.50). As Table 3 shows, children with PSC-Y positive scores were 5 times more likely to score positive on the CDI compared to children who were PSC-Y negative (63% vs. 12%; χ2 = 42.8, df = 1, p < .0001). Children who scored positive on the PSC-Y also had significantly higher mean CDI scores than children with PSC-Y negative scores (14.1 vs. 5.4; F = 58.2, df = 1, p < .0001).

Similarly, children with PSC-Y positive scores were significantly more likely to be cases on the RCMAS than children who were PSC-Y negative (51% vs. 10%; χ2 = 31.6, df = 1, p < .0001). The mean RCMAS score for children with positive PSC-Y scores was significantly higher (16.1) than for children with negative PSC-Y scores (9.7; F = 28.9, df = 1, p < .0001). The agreement in case/non case classification between the PSC-Y and RCMAS was 83% (142/173; SP = 88, SN = 56, κ = 0.43).

Characteristics of Children Not Identified by the PSC (Parent Report)

The third goal of the study was to evaluate whether the PSC-Y identified additional children not already recognized by parents. While there was a strong association between PSC-Y scores and parent report of child-psychosocial problems, the overlap was by no means complete. Only 10 of the 35 children identified by the PSC-Y were identified by their parents on the PSC, and only 10 of the children identified by parents on the PSC were identified by the PSC-Y. Of the 25 children identified by the PSC-Y but not by parents, only 7 were not identified as at risk by their teacher, CDI, or RCMAS. Compared with children who were identified by neither the PSC-Y nor the PSC, children identified by the PSC-Y but not by their parents on the PSC were significantly more likely to be rated by their teachers as having attentional or behavioral problems (44% vs. 0%; χ2 = 59.8, df = 1, p < .0001), and/or to be cases on the CDI (56% vs. 10%; χ2 = 29.6, df = 1, p < .0001) and RCMAS (52% vs. 9%; χ2 = 29.2, df = 1, p < .0001).

Agreement of Identified Psychosocial Problems by PSC-Y and Child Report of Impairment

The utility of the PSC-Y in identifying children who reported depression on the CDI and/or anxiety on the RCMAS was examined. Figure 2 illustrates the agreement of children identified by the PSC-Y and those identified by the CDI and/or RCMAS. The agreement between the PSC-Y and the two child-self-report measures was considerable, with 71% (25/35) of the children with positive PSC-Y scores identified by either the CDI or RCMAS (κ = 0.44, SP = 48, SN = 92). Conversely, 48% (25/52) of children identified by CDI and/or RCMAS were identified by the PSC-Y.

Figure 2.

Figure 2

Open in a new tab

Agreement of child self-report measures. PSC-Y, Pediatric Symptom Checklist-Youth Report; CDI, Children’s Depression Inventory; RCMAS, Revised Children’s Manifest Anxiety Scale.

Agreement of Identified Psychosocial Problems by PSC-Y, Teacher and Parent Reports of Impairment

Figure 3 illustrates the agreement of children identified with the PSC-Y and those identified by parent report of psychosocial problems on the PSC and teacher report of attentional and/or behavioral problems. Thirty-six percent (62/173) of the children were classified as cases on the PSC-Y, CDI, or RCMAS measures; 4% (6/173) were cases on all three indices of impairment. While the PSC-Y identified 94% (17/18) of the children identified by their teacher as having attentional and/ or behavioral problems, the PSC-Y identified less than half (10/22) of the children whose parents reported problems on the PSC.

Figure 3.

Figure 3

Open in a new tab

Agreement of child self-report, parent-report, and teacher-report measures. PSC-Y; Teacher, Attentional and/or Behavioral Problem; PSC, Pediatric Symptom Checklist (Parent report).

PSC-Y Scores and Mental Health Services

The fourth goal of the study was to estimate unmet need by determining whether children identified by the PSC-Y were receiving mental health services. Twelve percent (20/173) of the children had a past or current history of mental health services. The children identified by the PSC-Y but not PSC were less likely to have a past or current history of mental health services than children identified by the PSC. Twenty percent (7/35) of the children identified by the PSC-Y had a past or current history of mental health services in comparison to 41% (9/22) of children identified by the PSC. The PSC-Y identified 25 children with emotional and behavioral problems who were not flagged by the PSC. Only 4 of these children (19%) had a past or current history of mental health services, suggesting that children identified by the PSC-Y but not the PSC are at risk for untreated psychosocial problems.

Reliability

The final goal of the study was to assess reliability of the PSC-Y. Test–retest reliability was calculated on a subsample of 90 children who were asked to complete a second PSC-Y approximately 4 months after filling out the first. The Pearson r was 0.45 for the correlation between the total PSC-Y scores at the two times. Categorical agreement showed a kappa of 0.50, suggesting acceptable test–retest reliability for the PSC-Y in classifying cases.

Discussion

A primary question in assessing the clinical utility of a new screening tool such as the PSC-Y is whether it improves recognition of children with psychosocial dysfunction. When compared against the external criterion of teacher report of children’s attentional and/or behavioral problems, the PSC-Y showed acceptable sensitivity and specificity in identifying children at psychosocial risk. In this inner-city, low-income sample, adding the PSC-Y to teachers’ ratings of attention or behavior problems doubled the recognition rate of children at risk from 10% to 20%. This 20% rate is higher than has been found in middle-class samples, but is similar to rates found in lower socioeconomic groups (Bird et al., 1988; Jellinek et al., 1995). As expected, children from single-parent families were identified as at higher risk than children from two-parent households.

In light of the difficulty identifying a “gold standard” of psychosocial dysfunction in this population, we used several criterion variables in a multi-method assessment approach to help establish preliminary concurrent validity of the PSC-Y. The PSC-Y showed strong associations with teacher-reported attentional and behavioral problems, parent-reported child dysfunction, and child-reported depression and anxiety. The PSC-Y agreed with 89% of the teachers in classifying children with externalizing problems, indicating that the psychosocial problems detected by the PSC-Y were evident in the day-to-day functioning of the child in school. In addition, the PSC-Y was strongly associated with parent report of overall functioning in the child.

In this study the PSC-Y appeared to improve our understanding of students’ needs for mental health services. Beyond corroborating teachers’ selection of children with externalizing symptoms, the PSC-Y identified children with internalizing symptoms who were missed by parents, consistent with the superiority of self-report as a measure of internalizing disorders. One out of eight children reported psychosocial problems that were not being recognized by their parents. As anticipated, these children were significantly more likely to have symptoms of depression and anxiety, and to be rated by teachers as having attention or behavior problems. Despite their impairment, these children were half as likely to have a past or current history of mental health services compared with children who were recognized by their parents, suggesting that the PSC-Y does identify a new group of children not already recognized as in need of further evaluation and treatment.

Findings also indicated acceptable test–retest reliability for the PSC-Y. Because PSC-Y scores were collected after a 4-month interval during which a universally-free school breakfast program was implemented, the reliability of the PSC-Y may be even stronger within a shorter test–retest time frame and without effects from interventions.

A number of limitations must be kept in mind when interpreting the results of this study. Sampling bias may have occurred because only about one third of the parents of eligible children agreed to participate in the study. Parents of more dysfunctional children may be more or less likely to participate. However, as noted, the rate of parent report of dysfunction by PSC in this study was similar to rates found in other low-income samples. The PSC-Y is subject to the limitations of other self-report measures for children and adolescents: Questions can be misunderstood (especially by children with learning and/or attentional problems), and children may hide or exaggerate symptoms. Teachers were asked to identify children with attentional or behavioral problems. These relatively nonspecific instructions may have led to underreporting by teachers. Although half (17/35) of the PSC-Y positive children and adolescents were identified by their teachers with comorbid problems in attention and/or behavior, the overlap between teacher ratings and child-report measures may be even stronger with the inclusion of more specific teacher-report measures. Support for the PSC-Y as a screen for externalizing as well as internalizing problems would be enhanced by more detailed teacher- and parent-report measures.

There is a growing consensus that the key to effective treatment of childhood and adolescent mental health problems is to identify and treat dysfunctional children as early as possible, before impairment and illness cause lower self-esteem, resistance to learning, and school failure. Earlier screening efforts have been hampered by lengthy, difficult-to-score instruments, often focused on symptoms rather than dysfunction. The PSC-Y assesses symptoms relevant to children’s functioning, that may improve accuracy in identifying children in most need of services. In this study, the PSC-Y detected children with functional impairment, including those with internalizing symptoms, many of whom were missed by their parents. The PSC-Y has particular application in schools where children are accessible and expected to function on a day-to-day basis.

A possible model for implementing the PSC-Y might include obtaining parental consent for its use at the time of registration for school. Students would complete the screen in 3 to 4 minutes, and aides or teachers could easily score classes’ responses. Positive screens would then trigger communication between school staff and parents, and referrals for full evaluation of the most impaired children. Future research directions for the PSC-Y should include controlled studies in clinical populations and testing parent and teacher acceptability through pilot implementation programs in schools.

Table 4.

Child Adjustment Scores for Children Not Recognized by the Pediatric Symptom Checklist

Measure Total PSC-Yscore
Non-case (<30) Case (30+)
151 (100%) 126 (83%) 25 (17%)
Teacher report
 Attention/behavior problem* no 140 (93) 126 (100) 14 (56)
yes 11 (7) 0 (0) 11 (44)
Child reporta
 CDI case* no 124 (82) 113 (91) 11 (44)
yes 27 (18) 13 (9) 14 (56)
 CDI total score* mean 6.7 5.3 13.7
 RCMAS case* no 127 (84) 115 (91) 12 (48)
yes 24 (16) 11 (9) 13 (52)
 RCMAS total score* mean 10.6 9.5 16.3
 CDI or RCMAS case* no 113 (75) 105 (93) 8 (32)
yes 38 (25) 21 (7) 17 (68)
Open in a new tab
a

CDI = Children’s Depression Inventory/ RCMAS = Revised Children’s Manifest Anxiety Scale.

*

p < .0001.

Acknowledgments

This study was made possible through grants from the Mid-Atlantic Milk Marketing Association and the Kellogg Corporation.

The generous cooperation of the Maryland Food Committee and the participating teachers, students, and parents at the Rhodes, Lingelbach, Lakeland, and Charles Carrol, Barrister Schools are gratefully acknowledged. The authors wish to thank Lisa Dutra for her assistance with the figures.

References

  1. Achenbach TM. Integrative guide for the 1991 CBCL/4-18, YSR and TRF profiles. Burlington, VT: University of Vermont Department of Psychiatry; 1991. [Google Scholar]
  2. Achenbach TM, Howell CT. Are American children’s problems getting worse? A 13-year comparison. Journal of the American Academy of Child & Adolescent Psychiatry. 1993;32(6):1145–1154. doi: 10.1097/00004583-199311000-00006. [DOI] [PubMed] [Google Scholar]
  3. Beitchman JH, Corradini A. Self-report measures for use with children: A review and comment. Journal of Clinical Psychology. 1988;44(4):477–490. doi: 10.1002/1097-4679(198807)44:4<477::aid-jclp2270440402>3.0.co;2-k. [DOI] [PubMed] [Google Scholar]
  4. Bird HR, Canino G, Rubio-Stipec M, Gould MS, Ribera J, Sesman M, Woodbury M, Huertas-Goldman S, Pagan A, Sanchez-Lacay A, Moscoso M. Estimates of the prevalence of childhood maladjustment in a community survey in Puerto Rico. Archives of General Psychiatry. 1988;45:1120–1126. doi: 10.1001/archpsyc.1988.01800360068010. [DOI] [PubMed] [Google Scholar]
  5. Bird HR, Gould MS, Rubio-Stipec M, Staghezza BM, Canino G. Screening for childhood psychopathology in the community using the Child Behavior Checklist. Journal of the American Academy of Child & Adolescent Psychiatry. 1991;30(1):116–123. doi: 10.1097/00004583-199101000-00018. [DOI] [PubMed] [Google Scholar]
  6. Bird HR, Yager TJ. Impairment in the epidemiological measurement of childhood psychopathology in the community. Journal of the American Academy of Child & Adolescent Psychiatry. 1990;29(5):796–803. doi: 10.1097/00004583-199009000-00020. [DOI] [PubMed] [Google Scholar]
  7. Boyle MH, Offord DR, Racien Y, Szatmari P, Fleming JE, Sanford M. Identifying thresholds for classifying childhood psychiatric disorder: Issues and prospects. Journal of the American Academy of Child & Adolescent Psychiatry. 1996;35(11):1440–1448. doi: 10.1097/00004583-199611000-00012. [DOI] [PubMed] [Google Scholar]
  8. Brandenburg NA, Friedman RM, Silver SE. The epidemiology of childhood psychiatric disorders: Prevalence findings from recent studies. Journal of the American Academy of Child & Adolescent Psychiatry. 1990;29(1):76–83. doi: 10.1097/00004583-199001000-00013. [DOI] [PubMed] [Google Scholar]
  9. Burns BJ, Costello EJ, Angold A, Tweed D, Stangl D, Farmer EMZ, Erkanli A. Children’s mental health service use across service sectors. Health Affairs. 1995;14(3):147–159. doi: 10.1377/hlthaff.14.3.147. [DOI] [PubMed] [Google Scholar]
  10. Carlson GA, Cantwell DP. Unmasking masked depression in children and adolescents. American Journal of Psychiatry. 1980;137(4):445–449. doi: 10.1176/ajp.137.4.445. [DOI] [PubMed] [Google Scholar]
  11. Centor RM. Signal detectability: The use of ROC curves and their analyses. Medical Decision Making. 1991;11:102–106. doi: 10.1177/0272989X9101100205. [DOI] [PubMed] [Google Scholar]
  12. Centor RM, Kneightley GE. Receiver operator characteristic curve analysis using the ROC Analyzer. SCAMC Proceedings. 1989;13:222–226. [Google Scholar]
  13. Costello EJ, Angold A, Burns BJ, Stangl D, Tweed DL, Erkanli A, Worthman CM. The Great Smoky Mountains study of youth: goals, design, methods, and the prevalence of DSM-III-R disorders. Archives of General Psychiatry. 1996a;53:1129–1136. doi: 10.1001/archpsyc.1996.01830120067012. [DOI] [PubMed] [Google Scholar]
  14. Costello EJ, Angold A, Burns BJ, Erkanli A, Stangl DK, Tweed DL. The Great Smoky Mountain study of youth: functional impairment and serious emotional disturbance. Archives of General Psychiatry. 1996b;53:1137–1143. doi: 10.1001/archpsyc.1996.01830120077013. [DOI] [PubMed] [Google Scholar]
  15. Costello EJ, Costello AJ, Edelbrock C, Burns BJ, Dulcan MK, Brent D, Janiszewski S. Psychiatric disorders in pediatric primary care. Archives of General Psychiatry. 1988a;45:1120–1127. doi: 10.1001/archpsyc.1988.01800360055008. [DOI] [PubMed] [Google Scholar]
  16. Costello EJ, Edelbrock C, Costello AJ, Dulcan MK, Burns BJ, Brent D. Psychopathology in pediatric primary care: The new hidden morbidity. Pediatrics. 1988b;82:415–424. [PubMed] [Google Scholar]
  17. Costello EJ, Pantino T. The new morbidity: Who should treat it? Developmental & Behavioral Pediatrics. 1987;8(5):288–291. [PubMed] [Google Scholar]
  18. Edelbrock C, Costello AJ, Dulcan MK, Kalas R, Conover NC. Age differences in the reliability of the psychiatric interview of the child. Child Development. 1985;56:265–275. [PubMed] [Google Scholar]
  19. Finch AJ, Saylor CF, Edwards GL. Children’s Depression Inventory: Sex and grade norms for normal children. Journal of Consulting & Clinical Psychology. 1985;53(3):424–425. doi: 10.1037//0022-006x.53.3.424. [DOI] [PubMed] [Google Scholar]
  20. Gotlib IH, Lewinsohn PM, Seeley JR. Symptoms versus a diagnosis of depression: Differences in psychosocial functioning. Journal of Consulting & Clinical Psychology. 1995;63(1):90–100. doi: 10.1037//0022-006x.63.1.90. [DOI] [PubMed] [Google Scholar]
  21. Hammen C, Compass DE. Unmasking masked depression in children and adolescents: the problem of comorbidity. Clinical Psychology Review. 1994;14:585–603. [Google Scholar]
  22. Horwitz SM, Leaf PJ, Leventhal JM, Forsyth B, Speechley KN. Identification and management of psychosocial and developmental problems in community-based, primary care pediatrics. Pediatrics. 1992;89(3):480–485. [PubMed] [Google Scholar]
  23. Hsiao JK, Bartko JJ, Potter WZ. Diagnosing diagnoses. Archives of General Psychiatry. 1989;46:664–667. doi: 10.1001/archpsyc.1989.01810070090014. [DOI] [PubMed] [Google Scholar]
  24. Jellinek MS, Hurcombe B. Two wrongs don’t make a right: Managed care, mental health, and the marketplace. Commentary. Journal of the American Medical Association. 1993;270:1737–1739. [PubMed] [Google Scholar]
  25. Jellinek M, Little M, Murphy JM, Pagano M. The pediatric symptom checklist: Support for a role in a managed care environment. Archives of Pediatric & Adolescent Medicine. 1995;149:740–746. doi: 10.1001/archpedi.1995.02170200030004. [DOI] [PubMed] [Google Scholar]
  26. Jellinek MS, Murphy JM. Use of the pediatric symptom checklist in outpatient practice. Current Problems in Pediatrics, October. 1990;20:602–609. [Google Scholar]
  27. Jellinek MS, Murphy JM, Burns BJ. Brief psychosocial screening in outpatient pediatric practice. Journal of Pediatrics. 1986;109:371–378. doi: 10.1016/s0022-3476(86)80408-5. [DOI] [PubMed] [Google Scholar]
  28. Jellinek MS, Murphy JM, Robinson J, Feins A, Lamb S, Fenton T. Pediatric symptom checklist: screening school-age children for psychosocial dysfunction. Journal of Pediatrics. 1988;112(2):201–209. doi: 10.1016/s0022-3476(88)80056-8. [DOI] [PubMed] [Google Scholar]
  29. Kandel DB, Davies M. Epidemiology of depressive mood in adolescents; an empirical study. Archives of General Psychiatry. 1982;39:1205–1212. doi: 10.1001/archpsyc.1982.04290100065011. [DOI] [PubMed] [Google Scholar]
  30. Kashani JH, Sherman DD. Childhood depression: Epidemiology, etiological models, and treatment implications. Integrative Psychiatry. 1988;6:1–8. [Google Scholar]
  31. Kazdin AE. Assessment of childhood depression. In: La Greca AM, editor. Through the eyes of a child: Obtaining self-reports from children and adolescents. Boston, MA: Allyn & Bacon; 1990. pp. 189–233. [Google Scholar]
  32. Kelleher KJ, Wolraich ML. Diagnosing psychosocial problems. Pediatrics. 1995;95:899–901. [PubMed] [Google Scholar]
  33. Kessler RC, Foster CL, Saunders WB, Stang PE. Social consequences of psychiatric disorders, I: Educational attainment. American Journal of Psychiatry. 1995;152(7):1026–1032. doi: 10.1176/ajp.152.7.1026. [DOI] [PubMed] [Google Scholar]
  34. Knitzer J. Unclaimed children: The failure of public responsibility to children and adolescents in need of mental health services. Washington, DC: Children’s Defense Fund; 1982. [Google Scholar]
  35. Koot HM, Verhulst FC. Prediction of children’s referral to mental health and special education services from earlier adjustment. Journal of Child Psychology & Psychiatry. 1992;33(4):717–729. doi: 10.1111/j.1469-7610.1992.tb00907.x. [DOI] [PubMed] [Google Scholar]
  36. Kovacs M. Unpublished manuscript. University of Pittsburgh; 1982. The Children’s Depression Inventory. [Google Scholar]
  37. Kovacs M. CDI Manual. North Tonawonda, NY: Multi-Health Systems; 1992. [Google Scholar]
  38. Little M, Murphy JM. Screening 4- and 5-year-old children for psychosocial dysfunction: A preliminary study with the pediatric symptom checklist. Developmental & Behavioral Pediatrics. 1994;15(3):191–197. [Google Scholar]
  39. Murphy JM, Ichinose C, Hicks RC, Kingdon D, Crist-Whitzel J, Jordan P, Feldman G, Jellinek MS. Screening for psychosocial problems during periodic EPSDT examinations: Preliminary results from a Mexican-American sample. Journal of Pediatrics. 1996;129:864–869. doi: 10.1016/s0022-3476(96)70030-6. [DOI] [PubMed] [Google Scholar]
  40. Murphy JM, Jellinek M. Screening for psychosocial dysfunction in economically disadvantaged and minority group children: Further validation of the pediatric symptom checklist. American Journal of Orthopsychiatry. 1988;58(3):450–456. doi: 10.1111/j.1939-0025.1988.tb01605.x. [DOI] [PubMed] [Google Scholar]
  41. Murphy JM, Jellinek M. The recognition of psychosocial disorders in pediatric office practice: The current status of the pediatric symptom checklist. Developmental & Behavioral Pediatrics. 1990;11(5):273–278. [PubMed] [Google Scholar]
  42. Murphy JM, Jellinek M, Milinsky S. The pediatric symptom checklist: validation in the real world of middle school. Journal of Pediatric Psychology. 1989;14(4):629–639. doi: 10.1093/jpepsy/14.4.629. [DOI] [PubMed] [Google Scholar]
  43. Murphy JM, Wehler CA, Pagano ME, Little M, Kleinman RE, Jellinek MS. Relationship between hunger and psychosocial functioning in low-income American children. Journal of American Academy of Child and Adolescent Psychiatry. 1998;37(2):163–170. doi: 10.1097/00004583-199802000-00008. [DOI] [PubMed] [Google Scholar]
  44. Murphy JM, Reede J, Jellinek MS, Bishop SJ. Screening for psychosocial dysfunction in inner-city children: Further validation of the pediatric symptom checklist. Journal of the American Academy of Child & Adolescent Psychiatry. 1992;31(6):1105–1111. doi: 10.1097/00004583-199211000-00019. [DOI] [PubMed] [Google Scholar]
  45. Norquist GS, Wells KB. How do HMOs reduce outpatient mental health care costs? American Journal of Psychiatry. 1991;148(1):96–101. doi: 10.1176/ajp.148.1.96. [DOI] [PubMed] [Google Scholar]
  46. Rauch PK, Jellinek MS, Murphy JM, Schachner L, Hansen R, Esterly NB, Prendiville J, Bishop SJ, Goshko M. Screening for psychosocial dysfunction in pediatric dermatology practice. Clinica Pediatrica. 1991;30:493–497. doi: 10.1177/000992289103000807. [DOI] [PubMed] [Google Scholar]
  47. Regier DA, Kaelber CT, Rae DS. Limitations of diagnostic criteria and assessment instruments for mental disorders. Archives of General Psychiatry. 1998;55(2):109–115. doi: 10.1001/archpsyc.55.2.109. [DOI] [PubMed] [Google Scholar]
  48. Reynolds CR. Concurrent vValidity of What I Think and Feel: The Revised Children’s Manifest Anxiety Scale. Journal of Consulting & Clinical Psychology. 1980;48(6):774–775. doi: 10.1037//0022-006x.48.6.774. [DOI] [PubMed] [Google Scholar]
  49. Reynolds CR, Kamphaus RW, editors. Handbook of psychological and educational assessment of children: Personality, behavior and context. New York: Guilford Press; 1990. [Google Scholar]
  50. Reynolds CR, Paget KD. National normative and reliability data for the revised children’s manifest anxiety scale. School Psychology Review. 1983;12:324–336. [Google Scholar]
  51. Reynolds CR, Richmond BO. Factor structure and construct validity of “What I Think and Feel”; The revised children’s manifest anxiety scale. Journal of Personality Assessment. 1979;43:281–283. doi: 10.1207/s15327752jpa4303_9. [DOI] [PubMed] [Google Scholar]
  52. Rutter M. The childhood environment and adult disease, Ciba Foundation Symposium. Vol. 156. Chichester: Wiley; 1991. Childhood experiences and adult psychosocial functioning; pp. 189–208. [DOI] [PubMed] [Google Scholar]
  53. School District of Philadelphia. Management information systems participation summary of the weekly operating report 1996 [Google Scholar]
  54. Smucker MR, Craighead WE, Craighead LW, Green BJ. Normative and reliability data for the children’s depression inventory. Journal of Abnormal Child Psychology. 1986;14(1):25–39. doi: 10.1007/BF00917219. [DOI] [PubMed] [Google Scholar]
  55. Stancin T, Palermo TM. A review of behavioral screening practices in pediatric settings: Do they pass the test? Journal of Developmental & Behavioral Pediatrics. 1997;18:183–194. doi: 10.1097/00004703-199706000-00011. [DOI] [PubMed] [Google Scholar]
  56. Stavrakaki C, Williams EC, Walker S, Roberts N, Kotsopoulos S. Pilot study of anxiety and depression in prepubertal children. Canadian Journal of Psychiatry. 1991;36:332–338. doi: 10.1177/070674379103600503. [DOI] [PubMed] [Google Scholar]
  57. Walker WO, LaGrone RG, Atkinson AW. Psychosocial screening in pediatric practice: Identifying high-risk children. Developmental & Behavioral Pediatrics. 1989;10(3):134–138. [PubMed] [Google Scholar]
  58. Weinstein MC, Berwick DM, Goldman PA, Murphy JM, Barsky AJ. A comparison of three psychiatric screening tests using receiver operator characteristics (ROC) analysis. Medical Care. 1989;27:593. doi: 10.1097/00005650-198906000-00003. [DOI] [PubMed] [Google Scholar]
  59. Wyshak G, Barsky AJ, Klerman GL. Comparison of psychiatric screening tests in a general medical setting using ROC analysis. Medical Care. 1991;29:775–785. doi: 10.1097/00005650-199108000-00010. [DOI] [PubMed] [Google Scholar]
  60. Zahner GEP, Pawelkiewicz W, DeFrancesco JJ, Adnopoz J. Children’s mental health service needs and utilization patterns in an urban community: an epidemiological assessment. Journal of the American Academy of Child & Adolescent Psychiatry. 1992;31(5):951–960. doi: 10.1097/00004583-199209000-00025. [DOI] [PubMed] [Google Scholar]

RESOURCES