Universal Emotional Health Screening at the Middle School Transition

Abstract

This article describes the implementation of the Developmental Pathways Screening Program (DPSP) and an evaluation of program feasibility, acceptability, and yield. Using the Mood and Feelings Questionnaire (MFQ) and externalizing questions from the Youth Self Report (YSR; Achenbach, 2001), universal classroom-based emotional health screening was implemented with students as they began middle school. Of all sixth graders enrolled in four participating Seattle schools, 861 (83%) were screened. Students who screened positive for emotional distress (15% of students screened) received onsite structured clinical evaluations with children's mental health professionals. Seventy-one percent of students who were evaluated were found to be experiencing significant emotional distress, with 59% warranting referral to academic tutoring, school counselor, and/or community mental health services. Successful implementation of in-class screening was facilitated by strong collaboration between DPSP and school staff. Limitations of emotional health screening and the DPSP are discussed, and future steps are outlined.

Recent research reveals that as many as 1 in 10 children suffer from a mental, behavioral, or learning problem that interferes with their ability to function effectively in school or in the community. Despite the evidence showing how critical good emotional health is to a child's development and academic success (Bearman, Jones, & Urdry 2003; Glied & Pine, 2002; Vander Stoep et al., 2000), a very low proportion of children with emotional disorders are identified and treated (Costello et al., 1996; Leaf et al., 1996). The President's New Freedom Commission Report promotes early screening for mental health problems and reduction of disparities in mental health services (National Mental Health Information Center, 2002).

Schools have been called upon to play an aggressive role in early detection (Duncan, Forness, & Hartsough, 1995). Universal screening carried out in public school settings can reach large segments of the child population, including those who do not have health insurance or coverage for mental health treatment. Low-income and minority children are likely to attend public schools, are at increased risk of mental health conditions (McLeod & Shanahan, 1996; McLoyd, 1990), and may have little access to early identification and treatment (Surgeon General, 2000). Furthermore, adolescents are more prone to seek health care at school than in clinical settings (Adelman & Taylor, 1991), and the school environment is perceived by youth to be supportive of, and conducive to, frank disclosure and discussion (Shaffer & Gould, 2000). According to Healthy Youth: An Investment in Our Nation's Future (Centers for Disease Control and Prevention, 2003), “School health programs can be an effective means of improving educational achievement. Young people who are hungry, ill, depressed, or injured are less likely to learn” (p. 2).

Some progress has been made in the development of school-based programs to detect and address emotional health problems (Reynolds & Coats, 1986). At the high school level, Signs of Suicide (Aseltine & DeMartino, 2004) and TeenScreen (Columbia University, 2003) have been used to detect suicide risk. The Vanderbilt School-Based Counseling Program determines need for services by gathering mental health assessment data from teachers on all second- through fifth-grade students at participating Nashville elementary schools (Catron & Weiss, 1994). A similar elementary school–based behavioral health program in Florida utilizes teacher ratings of all kindergarten students to assess children's school-related problem behaviors and competencies and their need for behavioral health care (Spielberger, Haywood, Schuerman, & Richman, 2004). Each of these school-based programs incorporates a screening approach that is in keeping with its target population and target goals. To date, the most complete description of the implementation and evaluation of a school-based universal emotional health screening initiative can be found in a report from the program in Florida. More systematic and detailed descriptions of how screening programs are implemented would help guide future program development.

In this article we describe the implementation of a middle school–based program, the Developmental Pathways Screening Program (DPSP). Data on program acceptability and feasibility are presented as well as information on the number, characteristics, and needs of the students identified as distressed. The DPSP was designed to identify students experiencing emotional distress as they enter their sixth-grade school year and to link these students to supports that will enhance their chances for a healthy, successful start to middle school. The program targets the critical institutional and developmental transition from elementary to middle school, a time of vulnerability that can adversely affect a child's self-esteem, social engagement, and scholastic performance (Seidman, Aber, Allen, & French, 1996; Simmons, Burgeson, Carlton-Ford, & Blyth, 1987). DPSP identifies students at this time of transition when the risk of emotional distress is heightened and when early signs of distress may be below a diagnostic threshold yet portend adverse outcomes, if not addressed. The program is implemented universally, since some cognitive, emotional, and somatic manifestations of distress are easily overlooked by parents, teachers, and even children themselves, and since help seeking varies tremendously by gender and ethnicity (Cauce et al., 2002). Using a universal approach gives all students an opportunity to be identified and maximizes the ability to broadly recognize and address emotional health needs within the school setting. Thus, the goal of this article is to describe the implementation of the DPSP, to evaluate its acceptability and feasibility, and to assess its ability to successfully identify youth who are experiencing significant emotional distress and are in need of supportive services.

Method

Participants and Setting

Sixth-grade students were eligible to participate in screening if they were enrolled in school on screening day, their parent had received notification and had not declined, and they could complete the self-administered questionnaire alone or with limited assistance. Students with very limited English proficiency, severe developmental disabilities, or other handicapping conditions were excluded. Four middle schools located in four demographically distinct areas of Seattle and together enrolling approximately 1,100 sixth graders annually implemented the DPSP program.

Procedures

Procedures were approved by the University of Washington Human Subjects Review Board and by the Seattle Public Schools Office of Research, Evaluation and Assessment. In the early fall, 2 weeks prior to screening, parents of all sixth graders enrolled in the four participating schools received a letter from the school principal with information describing the DPSP. A postcard was enclosed for parents to return should they decline to have their child participate. Instructional assistants who spoke the predominant non-English languages in each school (e.g., Spanish, Somali, Vietnamese, Tagalog) helped communicate program information by phone to parents with limited English proficiency.

DPSP staff visited schools to describe the upcoming screening activities to all sixth graders in homeroom classes or at sixth-grade assemblies. At these events, students received an informational flyer and had an opportunity to ask questions. Colorful posters announcing the program were mounted in school hallways and screening classrooms.

Program Features

Key features of the DPSP are as follows:

1. universal screening targeting all students, not only those determined to be “at risk”;

2. procedures accommodating students with special needs;

3. a screening questionnaire that assesses both internalizing and externalizing symptom dimensions to determine students' levels of emotional distress;

4. clinical follow-up at school with students who screen positive for emotional distress; and

5. facilitation of linkages to a variety of academic, social, and mental health supports.

Teams of three trained DPSP field staff administered a brief questionnaire to eligible students during one classroom period. One staff person gave students instructions for completing the assent form and the screening questionnaire. A second organized the distribution and retrieval of screening materials to ensure that each student received and returned the questionnaire linked to his or her ID number. The third staff person provided general support and troubleshooting. All staff roamed the room to answer questions while students completed their questionnaires. Teachers remained in their classrooms to address administrative and disciplinary issues, but did not participate in administration of the questionnaire.

Nonparticipants, including students who were ineligible or who declined participation, were given a packet that contained a word puzzle and were instructed to work quietly at their desks. A DPSP staff person checked each completed questionnaire for unintentionally skipped questions and invalid responses. All students were given a healthy snack and a pencil and, in addition, screening participants received a $1 snack coupon. Research staff made two additional attempts to screen students who had been absent on screening day.

Screening Questionnaire

The Developmental Pathways Screening Questionnaire (DPSQ) is designed to tap both internalizing and externalizing manifestations of distress. The DPSQ consists of 6 demographic items, 30 items from the Mood and Feelings Questionnaire (MFQ; Angold & Costello, 1987) and the 30 externalizing items from the Youth Self Report (YSR; Achenbach, 2001). The MFQ was developed for epidemiological studies of 8- to 18-year-olds. MFQ items are derived from the fourth edition of the Diagnostic and Statistical Manual of Mental Disorders (American Psychiatric Association, 1994) criteria for major depression (Angold et al., 1995) and dysthymia (Costello & Angold, 1988). High test-retest reliability and good convergent validity with a diagnosis of major depressive disorder were demonstrated in a recent study of young Norwegian adolescents (Sund, Larsson, & Wichstrom, 2001). A brief 13-item version (SMFQ) is used to assess core depressive symptomatology and to screen for depression in epidemiological studies (Angold et al., 1995). The YSR is used with children ages 11 to 18 years (Achenbach, 2001). Content, criterion-related, and construct validity of the YSR has been supported by 4 decades of research (Achenbach & Rescorla, 2001). The MFQ and YSR were chosen because they were conducive to administration to sixth graders in the classroom setting and provided norm-referenced scores that have been established to provide clinical guidance. Questionnaires were electronically scanned, and total depression and disruptive behavior scores were calculated. Missing items were assigned imputed values.

Accommodations for Students With Special Needs

All students were allowed extra time to complete the questionnaire and were given 1:1 help, if needed. Deaf and hard-of-hearing students were screened with interpreters or using sound-enhancing devices. Extra screening staff was assigned to classrooms for students with learning disabilities, and 1:1 administration with scheduled positive reinforcement was carried out with students who had serious behavioral disturbances. Instructional assistants were trained to help English Language Learners (ELL students) understand potentially unfamiliar vocabulary and concepts on the screening questionnaire.

Follow-up Procedures

Students were considered to have screened positive for emotional distress if their MFQ score was 20 or higher or if their YSR score was 25 or higher. The decision was made to use a lower standard score on the depression dimension and a higher standard score on the externalizing behavior dimension because within the school setting, internalizing symptoms were less likely than externalizing symptoms to have been identified and treated (Bramlett, Murphy, Johnson, Wallingsford, & Hall, 2002).

All students who screened positive received a 20- to 30-minute clinical assessment at school within 2 weeks of screening. DPSP hired experienced, multiethnic master's-level children's mental health professionals (ChMHPs) as temporary, part-time employees to conduct follow-up assessments. ChMHPs were trained to navigate within school settings; to complete the standardized assessment instrument; to maintain confidentiality; and to enlist students, parents, and school counselors as partners in addressing student needs.

Follow-up Evaluation Measure

To determine whether the student was experiencing a significant level of emotional distress and to inventory sources of external support and internal resources for dealing with distress, the ChMHP administered the impairment scales from the Diagnostic Interview Schedule for Children–IV (DISC-IV; Columbia University DISC Development Group, 1998) and the stress, social support, and coping sections of Thompson and Eggert's (1999) Suicide Risk Screen (SRS). Students were determined to be experiencing significant emotional distress if, according to the results of the follow-up evaluation and the clinical judgment of the ChMHP, the symptoms they endorsed on the screening questionnaire were adversely affecting their functioning. Students experiencing distress were determined to need support if they had insufficient resources to cope with their distress.

Referrals

On the basis of information gathered about the nature and severity of the student's distress and the supports currently in place, the ChMHP developed an action plan. Service recommendations were differentiated to address the array of developmental challenges (e.g., academic, social, emotional) faced by children entering middle school. Students who expressed significant concerns about school performance or difficulties adjusting to the academic demands of middle school were referred to school-based homework clubs or to tutors. Students who appeared isolated or distressed about their social adjustment were referred to an after-school activity program and/or were introduced to the school counselor. Homework clubs, after-school programs, and school counselors were available in every Seattle public middle school. Referrals to community mental health resources were made if a student's identified mental health issues extended beyond middle school adjustment, for example, exposure to recent trauma or stressful life events (e.g., death, divorce, domestic violence) or longstanding unattended conditions or situations. For any given child, referrals could be made to one or more services.

After each assessment, the ChMHP made a phone call to inform the student's parent or guardian of the meeting and its outcome. If a community-based mental health referral was recommended, parents were given referrals and a pamphlet listing nearby community mental health agencies, or students were referred directly to school-based mental health services. Staff also assisted parents in accessing academic support services and after-school activities available at school or in the student's neighborhood. The lead ChMHP gave each school counselor a brief disposition summary of each follow-up assessment.

Results

Program acceptability, yield, and feasibility were evaluated. Acceptability was measured by the overall proportion of students, as well as the proportion in gender, race, school, and school program (e.g., special education, ELL, general education, gifted) subgroups who participated in screening. The proportion of participating students who screened high on internalizing and/or externalizing dimensions was ascertained. Odds ratios and 95% confidence intervals were calculated to compare gender and racial subgroups of students with regard to their likelihood of screening positive and their likelihood of being determined by a clinician to be distressed and in need of additional support.

The DPSQ was then evaluated for its yield, or positive predictive value (PPV). The initial PPV (PPV1) was calculated as the proportion of students who screened positive who were then determined by the ChMHPs to be distressed. A more conservative calculation of PPV (PPV2) that added the criterion of “distressed and in need of support” was then applied. PPV2s were assessed at different cutoff values of the DPSQ. Feasibility was evaluated in terms of ease of implementation of screening and follow-up procedures, maintenance of confidentiality, and program costs.

Acceptability

A total of 1,034 sixth-grade students were enrolled at the four participating schools on screening day. Twenty-three (2.2%) of these students were ineligible to participate due to severely disabling conditions, 70 (6.8%) of the students' parents declined to allow their children to participate, and 11(1.0%) of the parents' mailings were returned undeliverable. Thus, a total of 930 students were eligible for screening. Of the eligible students, 861 (92.6%) participated (83.3% of students enrolled). A t test and chi-square statistics indicated no significant age, gender, or school differences in eligibility or participation. As shown in Table 1, participation was more than 81 % for each of the three predominant racial subgroups within the district, and over 89% of students in special needs groups (e.g., special education and ELL) participated. Of the 69 eligible students who did not participate, 58 declined assent, and 11 were absent for screening and all makeup days.

Table 1.

Participation in Screening by Race, Gender, and School

	Enrolled	Eligiblea	Total screenedb
Variable	n	n (%)	n	(% of enrolled)	(% of eligible)
Total	1,034	930 (89.9)	861	(83.3)	(92.6)
Race
African American	282	254 (90.1)	232	(82.3)	(91.3)
Asian American	263	233 (88.6)	214	(81.4)	(91.8)
European American	438	386 (90.4)	363	(85.0)	(94.0)
Native American	68	57 (91.9)	52	(83.9)	(91.2)
Gender
Male	558	500 (89.6)	467	(83.7)	(93.4)
Female	476	430 (90.3)	394	(82.8)	(91.6)
School
1	239	211 (88.3)	188	(78.7)	(89.1)
2	411	376 (91.5)	350	(85.2)	(93.1)
3	151	134 (88.7)	126	(83.4)	(94.0)
4	233	209 (89.7)	197	(84.5)	(94.3)

^aDoes not include students whose parent declined or could not be contacted or students who couldn't understand the screen.

^bDoes not include students who declined or who could not be screened due to multiple absences.

Yield

The DPP ChMHPs followed up with 131 students who screened positive for emotional distress. Of the students screened, 120 (13.9%) scored 20 or higher on the MFQ, and 35 (4.1 %) scored 25 or higher on the externalizing scale of the YSR (Table 2). A total of 24 (2.8%) students screened positive on both the MFQ and the YSR, and 11 (1.3%) screened positive on the YSR only. Table 3 shows that the proportion of students who screened positive differed by race but not by gender. Compared to European American students, more African American (Odds Ratio [OR] = 2.29, 95% Confidence Interval [CI] = 1.45–3.63) and Asian American (OR = 1.85,95% CI = 1.14–3.00) students screened positive for emotional distress.

Table 2.

Screening Results

	MFQ
	≥ 20		< 20
YSR	n	(%)	n	(%)
≥ 25	24	2.8	11	1.3
< 25	96	11.1	730	84.8

Note. MFQ = Mood and Feelings Questionnaire, depression (Angold & Costello, 1987); YSR = Youth Self Report, externalizing (Achenbach, 2001).

Table 3.

Referrals Made by Child Mental Health Professionals for Students Who Screened Positive

		Screened positive		Any referral madea
			ORb		ORc
Variable	Screened	n (%)	(95% CI)	n (%)d	(95% CI)
Gender
Male	467	70 (15.0)	1.0	37 (52.9)	1.0
Female	394	61 (15.5)	1.04 (.72–1.51)	40 (65.6)	1.70 (.84–3.44)
Race
African American	232	49 (21.1)	2.29 (1.45–3.63)	26 (53.1)	.92 (.39–2.14)
Asian American	214	38 (17.8)	1.85 (1.14–3.00)	27 (71.1)	1.99 (.77–5.13)
European American	363	38 (10.5)	1.0	21 (55.3)	1.0
Native American	52	6 (11.5)	1.12 (.45–2.78)	3 (50.0)	.81 (.14–4.54)
Total	861	131 (15.2)		77 (58.8)

^aIncludes referrals for school counselor, academic services, organized social activities, or mental health services.

^bOdds ratio is the ratio of the odds of a positive screen among screened students in a given gender or racial group relative to the odds of a positive screen among screened students in the reference group (males, European Americans).

^cOdds ratio is the ratio of the odds of having a referral made among students screening positive in a given gender or racial group relative to the odds of having a referral made among students screening positive in the reference group (males, European Americans).

^dPercentage of students screening positive.

Of the 131 students who were evaluated by a ChMHP, 93 were determined to be experiencing significant emotional distress (i.e., PPV1 = 71% at MFQ ≥ 20 and/or YSR ≥ 25), while 38 students (29%) screened positive, but upon clinical examination were determined to have low levels of emotional distress. These students were considered “false positives.” Sixteen distressed students were already receiving support services, including eight who were receiving specialty mental health care. Thus, of students in distress, 77 were found to be in need of support that was not currently in place (i.e., PPV2 = 59%).

The 77 students who were determined to be in need of support received referrals to one or more resources: 32 (24.4%) were referred for academic assistance; 61 (46.6%) were referred to the school counselor for adult support and guidance, 17 (13.0%) were referred to a community mental health agency; and 11 (14.3%) were referred to other services, including after-school activities. As shown in Table 3, our findings suggest that compared to European American students, a higher proportion of Asian American students who screened high were found to need additional support (OR = 1.99, 95% CI = .77–5.13).

Table 4 depicts the predictive values of a positive screen (PPV2 = students who were experiencing significant distress and who needed support/students who screened positive) with the MFQ cutoff values raised to 22 and 25. At a cutoff value of 22, 11 (15%) students who needed support would have been missed (false negatives), but follow-up would not have been carried out with 9 (17%) of the 54 students who were “false positives” at the lower cutoff of 20. With regard to students who were determined to need a referral to a community mental health agency, 1 (5.9%) would have been missed by raising the cutoff from 20 to 22. Raising the cutoff to 25 would have excluded 27 (35.1%) students in need of support, including 7 (43.8%) who were determined to need specialty mental health services.

Table 4.

Ability of a Positive MFQ Score to Predict “Distressed and in Need of Support”

			Distressed and in need of support		No support needed
Referral	MFQ screen cutoffa	Positive screens	#	Effect of raising cutoff (FN)	#	Effect of raising cutoff (FP)	PPV2
Any referral needed	≥ 20	131	77		54		.59
	≥ 22	111	66	11 FN missed	45	9 fewer FP	.59
	≥ 25	87	50	27 FN missed	37	18 fewer FP	.57
Mental health referral needed	≥ 20	131	17		114		.13
	≥ 22	111	16	1 FN missed	96	20 fewer FP	.14
	≥ 25	87	10	7 FN missed	77	45 fewer FP	.11

Note. MFQ = Mood and Feelings Questionnaire (Angold & Costello, 1987); FN = false negative; FP = false positive; PPV2 = students who were experiencing significant distress and who needed support/students who screened positive.

^aYouth Self Report (Achenbach, 2001) screen cutoff is constant at ≥ 25.

Our assessment of yield for the 13-item SMFQ was restricted to the 131 students who screened positive on the MFQ or YSR. Of those students, 118 (90%) scored 7 or higher on the SMFQ. As shown in Table 5, an SMFQ cutoff of ≥ 8 has a similar yield and number of false negatives for both “distressed and in need of support” and “mental health referral needed,” the same pattern as that observed using the 30-item MFQ at a cutoff of ≥ 22 (see Table 4).

Table 5.

Ability of a Positive SMFQ Score to Predict “Distressed and in Need of Support”

			Distressed and in need of support		No support needed
Referral	SMFQ screen cutoff	Positive screens	n (TP)	Effect of raising cutoff (FN)	n (FP)	Effect of raising cutoff (FP)	PPV2
Any referral needed	≥ 7	118	71	6 FN misseda	47		.60
	≥ 8	109	66	11 FN missed	43	4 fewer FP	.61
	≥ 9	95	58	19 FN missed	37	10 fewer FP	.61
	≥ 10	81	51	26 FN missed	30	17 fewer FP	.63
	≥ 11	75	48	29 FN missed	29	18 fewer FP	.64
Mental health referral needed	≥ 7	118	16	1 TP missed	102		.14
	≥ 8	109	16	1 TP missed	93	9 fewer FP	.15
	≥ 9	95	14	3 TP missed	81	21 fewer FP	.15
	≥ 10	81	11	6 TP missed	70	32 fewer FP	.14
	≥ 11	75	10	7 TP missed	65	37 fewer FP	.13

Note. SMFQ = Short Mood and Feelings Questionnaire; PPV2 = positive predictive value; FN = false negative; FP = false positive; TP = true positive.

^aOf these six FN who are missed, three had Mood and Feelings Questionnaire (Angold & Costello, 1987) scores ≥ 20 and three had Youth Self Report (Achenbach, 2001) scores ≥ 25.

Feasibility

Screening was easily accomplished within a 50-minute class period, with the first 25 minutes devoted to organizational tasks such as administration of assent and instructions. Depending upon the size of the school, a team of 9 to 12 DPSP staff was able to complete screening for all sixth graders in one school in 1 to 1.5 days.

Protecting confidentiality was a key factor in program feasibility, in terms of ensuring both the integrity and credibility of the program and the validity of student responses. Due to a number of actions taken to maintain confidentiality, no breaches were reported. All study personnel received training in human participant protection and signed oaths of confidentiality. Parents, teachers, and counselors were informed that students' responses and numeric scores would be protected so that students could feel more comfortable reporting personal information. To increase privacy, students sitting in close proximity to others were directed to move to a more private area. Teachers were asked to refrain from interacting with students while the screening questionnaires were being administered. During the follow-up phase of the program, disclosure of disposition information was accomplished within the guidelines of student assent. Information was shared with parents only after an agreement between the ChMHP and the student had been reached about how the student's status would be described and what type of support was being recommended. Likewise, school counselors or other service providers were contacted after an agreement was reached with the parent regarding what content would be shared. For 14 of the students who required follow-up and whose parents/guardians were not proficient in English, translators assisted the ChMHPs with phone calls.

Screening program expenses included personnel costs as well as costs associated with hiring translators, questionnaire printing and scoring, recruitment mailings and posters, supplies, participant incentives, and thank-you gifts for school personnel. Screening was implemented with reliance on full-time DPSP study staff, supplemented by part-time temporary staff in the roles of screening monitors and ChMHPs. The costs of implementing classroom-based screening and 1:1 follow-up was estimated to be $9 to $15 per enrolled student, varying as a function of school size (with larger size being more efficient) and the prevalence of positive screens (higher prevalence of positive screens requires more follow-ups) within the school.

Discussion

We have presented a detailed account of our experience implementing school-based universal emotional health screening with a diverse population of sixth graders attending urban public schools and of the acceptability, yield, and feasibility of implementation of the DPSP program. In the first year of DPSP program implementation, 83% of sixth graders participated in screening; 15% screened positive and underwent a follow-up assessment. Of those who received an evaluation, 71 % were experiencing significant distress, and 59% were determined to need additional support. Participation was high among students in four schools, including those who required special accommodations. Follow-up assessments were completed within the school setting by ChMHPs with 100% of students who screened positive. The majority of students scoring within the top 15th percentile of the DPSQ were evaluated as being distressed and in need of support. ChMHPs worked with families to link students to school counselors, tutors, after-school activities, and mental health agencies. In all, approximately 9% of the sixth graders who were screened were referred to resources early in the school year. These early DPSP evaluation results indicate that universal screening for emotional distress can be carried out successfully within the middle school setting.

In DPSP a disproportionately high number of African American and Asian American students screened positive, and among Asian Americans, students who screened positive were more likely to actually be distressed and in need of support. Implementing universal emotional health screening within a school setting has obvious benefits in that important health concerns can be addressed in populations that may have restricted access to early identification and intervention services. However, when considering whether to implement a screening program, a school district or public health authority will need to evaluate possible benefits in light of potential risks.

Schools or communities may be concerned that universal screening could yield so many children identified as needing services that existing resources would be overwhelmed. For individual children and families this problem translates into the risk of identifying a need, but not addressing it. For the student who is a “false positive,” screening may cause unnecessary concern, while a “false negative” screening result could cause unwarranted reassurance when a child may actually need further support or professional attention. In some settings, screening positive for emotional distress could lead to stigmatization.

The DPSP experience sheds light on some of these issues. With regard to the concern about high yield, among DPSP participants, 15% screened positive for emotional distress. The proportion of middle school students who screened positive in DPSP is considerably lower than the 41% to 51% of kindergartners determined by teachers to need support for social, emotional, or behavioral health problems in Palm Beach County, Florida (Spielberger et al., 2004). On the other hand, 15% is considerably higher than point prevalence estimates for major depressive disorder in early adolescence (Doi, Roberts, Takeuchi, & Suzuki, 1991). It is likely, however, that many students experiencing some symptoms of depression and moderate levels of distress in early adolescence will eventually join the ranks of the 20% of children and adolescents who experience an episode of major depressive disorder by the age of 18 years (Lewinsohn, Hops, Roberts, Seeley, & Andrews, 1993). Prior community-based studies have shown that adolescents who score above a depression screening cutoff but do not meet diagnostic criteria for major depressive disorder manifest higher levels of current and future psychopathology than do true negatives. Such youths are more likely to meet criteria for a depressive disorder within the ensuing year than are true negatives (Compas, Connor, & Hinden, 1998; Gotlib, Lewinsohn, & Seeley, 1995). In these studies, false positives did not differ significantly from true positives on a wide range of measures of psychosocial dysfunction. Additionally, Hammen and Rudolph (1996) have argued that nonclinical levels of depression are especially detrimental for adolescents because they may interfere with important developmental processes that alter a youth's trajectory and lead to a cascade of adjustment difficulties.

Besides a manageable yield, the problem of a screening program's potential to overwhelm limited resources was minimized by DPSP providing direct onsite clinical follow-up and referral for students who screened positive. Currently, the U.S. Preventive Services Task Force recommends screening adults for depression in medical practices if there are systems in place to ensure accurate diagnosis, effective treatment, and follow-up (U.S. Preventive Services Task Force, 2002). In the Vanderbilt School-Based Counseling Program, 98% of children who were referred to school-based mental health services actually received services, compared to 17% of children referred to traditional clinic-based services (Catron & Weiss, 1994).

On-site follow-up also helped to address the risk of unnecessary worry associated with a false positive screen. ChMHPs informed students of their screening status as they were accompanying them to the clinical evaluation and could quickly allay fears at the end of the 20- to 30-minute follow-up session. Risk of stigmatization was reduced through the procedures used to ensure confidentiality. Clinical evaluations carried out with students with low screening scores would be needed to address the issue of the risks inherent in being a false negative.

Other issues that a school or district interested in screening would need to address include the fundamental questions of “Who?” “When?” and “What?” The answer to the question of who will be screened will depend upon identified program goals and available resources. Although universal screening has the advantage of being amenable to implementation according to the intrinsic organization of the school (i.e., classrooms), costs of implementing universal screening may be greater than the cost of screening a selected subset of the school population. Choosing a targeted screening approach would mean identifying and screening students in a high-risk group, such as those who undergo a loss of support or status, for example, students whose parents have recently divorced or died or those who experience a precipitous drop in grade-point average. Such a screening program would include only students in whom signs of emotional distress had already been detected.

When to screen is a closely related question. Different mental health conditions affect children at different times in their development, which should be a consideration in determining when and for what conditions during a student's academic career a screen should be administered. Attention-deficit/hyperactivity disorder (ADHD), for example, commonly manifests as children first enter school. Depression typically emerges later, during adolescence, and often only after exposure to stressful circumstances. Thus, ideal timing of ADHD screening might be entry to kindergarten, while depression screening might be more appropriately implemented at a later developmental stage. Beyond the age or developmental stage of the child, potentially stressful times, such as during the transition to middle school or high school or in the wake of a natural disaster, might pose fruitful opportunities for depression screening.

The question of what problem to address should be answered on the basis of a theory of change about how to most effectively improve mental health among school-age children. A unique feature of the DPSP is the nature of the target condition. Most health screening programs, and even most mental health screening programs, target a specific disease or health outcome. In DPSP, depression and disruptive behavior scales were used to detect a high level of general emotional distress, rather than, for example, undiagnosed depressive disorder, suicide risk, or substance abuse. Follow-up was directed toward assessing impairment in functioning associated with emotional distress and need for support, rather than toward more specific diagnostic testing or need for specialty mental health services. The recommendations made to students and families were more commonly for enhanced academic and/or social support than for mental health treatment. The theory underlying our approach follows the suggestion by Weist (2003) that bolstering support at a critical developmental transition can lead directly to reduction in distress and indirectly to reduction in depression. By enhancing the emotional health of students starting middle school and preventing escalation to psychiatric disorder, we hoped to remove impediments to academic success.

Two related questions are whether depression screens such as the MFQ or the SMFQ are appropriate tools and whether the externalizing behavior screen contributed anything additional toward these goals. For adults there is a long historical precedent for using depression screening scales as a nonspecific indicator of psychopathology (Murphy, 1986). Recently, Kessler and colleagues (Kessler et al., 2002; Kessler et al., 2003) developed and validated a 6-item scale composed of 5 depression items and 1 anxiety item to screen for serious mental illness in the general adult population. With regard to the YSR, we found that it did not add appreciably to yield. At a cutoff of 25, only 11 students screened positive who did not also screen positive on the MFQ; of those 11, only 5 needed additional services. Using the 13-item SMFQ and eliminating the YSR would decrease average administration time by about 12 min.

The DPSP program used federal funding and employed university staff to carry out universal screening in middle schools. Activities such as establishing partnerships with schools and relationships with school personnel, laying groundwork prior to the implementation of screening and recruiting, and organizing and supervising teams of field staff to work in complex school milieus added significantly to screening costs. Theoretically, these costs might be reduced were a school district to decide to implement a screening program utilizing their own staff and resources while using the university to train and consult.

Successful implementation of a screening program will also depend upon a careful reading of the community's readiness to apply public health solutions to mental health problems. Considerable backlash has been generated by recommendations from the New Freedom Commission to implement universal mental health screening in public schools. In 2004 the U.S. Congress proposed an amendment to an appropriations bill that would have prohibited funding for such programs (Paul, 2004). Although the bill did not pass, some members of the media, the public, and Congress decried universal screening as coercive, potentially stigmatizing, and tied to the shadowy agenda of the pharmaceutical industry (Conservative Caucus, 2004; Eakman, 2004). Recent reports of the upsurge in use of psychotropic medications such as Ritalin® among school-aged children (Dunn, 2002) and of drug companies and even the U.S. Food and Drug Administration failing to notify the public of evidence regarding harmful side effects of antidepressants for treating adolescents have fueled apprehension (CBS News, 2004; CNN, 2004; Elias, 2004; Harris, 2004). Questions regarding the voluntary or mandatory nature of screening programs, the dissemination and use of screening results, and the interests and agendas of proponents and funders are legitimate and should be addressed openly.

Limitations and Future Directions

This description of DPSP implementation has several limitations. We did not have the resources to ascertain whether students who screened positive and were given suggestions or referrals for supportive services actually connected to the recommended resources. In addition, financial considerations allowed us to follow up only with students who screened positive, and therefore we could not evaluate the sensitivity or specificity of the screening tool. Due to these constraints, we may have missed lower scoring students who could have benefited from support.

Further, our universal screening program was not completely “universal.” A small proportion of enrolled students (2%) were excluded from screening due to severely disabling conditions. An additional 2% did not participate because of multiple absences or our inability to reach their parents. Eleven percent did not participate because either the parent or student declined. These particular subgroups of children may have been at heightened risk of experiencing emotional health problems.

Another concern is that for those who participate in screening, barriers may arise to reduce the chances that a child who needs services will get them. For example, the decision to seek mental health services is influenced by the family's culture and the cultural competence of those offering services. Implementation of a successful mental health program in a diverse school setting will require cultural competence in both service promotion and delivery.

We have learned a great deal from our early implementation experience that we hope will be of value to the development of expanded school mental health programs (Weist, 1997). We plan to build on our base of collaboration to conduct further evaluation and refinement that will increase program benefits to middle school students. Future program goals include a more extensive evaluation of DPSP, in which we plan to introduce and test a motivational enhancement component to increase the likelihood that students, parents, and school counselors will take the actions recommended. Future DPSP evaluations will be designed to enable us to track whether linkages are accomplished and to describe general and culture-specific barriers. Finally, we will design studies to address the question of whether participation in our universal screening program does, in fact, help students make a more successful adjustment from elementary to middle school.

Practical Implications

Growing recognition of the toll that emotional distress takes on school performance (Vander Stoep, Weiss, Kuo, Cheney, & Cohen, 2003) and a focus among policy makers on “leaving no child behind” (No Child Left Behind Act of 2001) indicate that the public interest and technological expertise needed to address mental health concerns in the school setting are increasing. As Hoagwood (2003) has written, “Given the serious risks that children with mental health problems face in remaining intellectually and emotionally engaged in learning, the integration of [mental health and educational] perspectives” is warranted (p. 95). With further evaluation and refinement, universal screening programs, such as the DPSP, can be used at high-risk developmental periods as a means of detecting early precursors to mental health problems that, if left unattended, are likely to interfere with the cognitive, emotional, and social development of young adolescents and to reduce their chances for academic success.