The Effectiveness of an After-school Program Targeting Urban African American Youth

Abstract

The present study reports on the effectiveness at one-year follow-up of an after-school prevention program targeting 6^th grade African American youth residing in high-risk urban areas. The program, conducted on-site over the school-year period, involved a group mentoring approach emphasizing remedial education and an appreciation of African American cultural heritage in promoting school bonding, social skills development, and greater academic achievement. Behavioral and adjustment outcome data were obtained from two participating middle-school sites (intervention and comparison, involving 237 and 241 students, respectively) serving essentially equivalent urban communities. Results of the study revealed significant effects for academic achievement and behavior in terms of grade point average and teacher ratings that favored students at the intervention site. At this site, greater participation of parents in the intervention program was found to be positively related to improvement of the children in grade point average. No differential site-related changes in negative behavior were observed.

During the past twenty years, researchers have documented the disproportionately high rates of poverty, unemployment, crime, substance abuse, and other indicators of social disorganization that are becoming increasingly concentrated in America’s cities (Jargowsky, 1997; Massey & Denton, 1993; Wilson, 1987). African American and other youth living in such environments, characterized by both social disorder and urban blight, are exposed to a wide variety and increasing number of serious social and environmental risk factors that may severely undermine their life-goals potential and increase the likelihood that they will become involved in deviant activities (Bell & Jenkins, 1993; Grant et al., 2000; Huston et al., 1994; Harrell & Peterson, 1992; O’Donnell et al., 2001; Sullivan & Farrell, 1999). Studies have shown that exposure to these types of adverse conditions often contribute to psychological and behavioral problems, including stress, depression, delinquency, substance abuse and other risky problem behaviors (Freudenberg et al., 1999; Garbarino, 1995; Grant et al., 2000; Rankin & Quane, 2002).

The increased risk of urban youth becoming involved in deviant behavior at an early age is particularly troubling, considering that one of the most consistent findings in the literature is the strong association between the age of onset of deviant behavior and the extent of later deviant activity (Chaiken & Chaiken, 1990; Loeber & LeBlanc, 1990; Vega et al., 1993). Research has clearly indicated that the emergence and continued occurrence of such problem behaviors among youth have been found to be predictive of future involvement in an increasing degree of intense, persistent, and severe problem behavior. Left unchecked, these negative pathways to deviance are likely to continue into adulthood, producing life-long detrimental consequences on physical health and general behavioral well-being (Gruber & Machamer, 2000; Jessor & Jessor, 1977; Newcomb et al., 1986; OJJDP, 2000). For urban African American youth, in particular, who are overly represented among individuals exposed to stressful life conditions associated with disadvantaged urban communities, continuation of such behaviors is especially likely to place them at high risk for adverse consequences throughout their lifetime (Dembo et al., 1991; Grant et al., 2000; Huston et al., 1994; Sutton et al., 1999).

Considering the negative influences characterizing high-risk urban environments there is an urgent need for quality preventive interventions designed for youth exposed to such influences, particularly those who are already at risk for other personal or social reasons. In order for such programs to be maximally effective, they must focus on reducing risk factors and enhancing protective factors not only among the individuals themselves but also within the environments in which they live (Botvin, 1990; OJJDP, 2000). Because of the complexity of the origins of adolescent problem behaviors, early intervention programs should ideally involve comprehensive efforts that consider the influence of family, peers, schools, and community (US DHHS, 2001; Botvin, 1990). Understandably, schools offer an ideal context for the implementation of such efforts.

After-school programs appear especially appropriate in prevention efforts since research has documented that many youth are unsupervised during after-school hours (James & Jurich, 1999; Miller et al., 1997), placing them at significantly greater risk of engaging in deviant and other high-risk behaviors than similar youth engaged in constructive activities supervised by responsible adults (Dwyer et al., 1990; Chaiken, 2000; Petit et al., 1997; Sickmund et al., 1997; Snyder & Sickmund, 1999). Research has consistently demonstrated that after-school programs that enable students to experience new and unique experiences through interaction with caring and concerned adults help to mitigate the impact of negative social environments (Freedman, 1993; Katz, 1997; McLaughlin et al., 1994).

Prominent among available evidence-based school drug prevention approaches is the Life Skills Training (LST) program (Botvin, 1990), involving a structured curriculum administered by the school’s teaching staff in sessions designed to inculcate anti-drug norms and to impart drug refusal, personal management, and general social skills. Although originally developed and tested on White students in suburban schools, LST has been successfully administered to minority (mostly African American and Hispanic) urban middle school students whose demographic characteristics closely parallel those of the presently targeted African American sample. In comparison trials involving student self-reports, the LST prevention approach has been found to be associated with lowered self-reported levels of both early substance abuse initiation and subsequent substance abuse involvement through the high-school years (Griffin et al., 2003).

Gottfredson and associates (2004) conducted a multi-site evaluation of the effects on delinquent behavior of after-school program participation in Maryland during the 1999-2000 school year. The after-school programs examined targeted elementary- and middle-school children who reported that they were unsupervised during after school hours and who were therefore considered to be particularly vulnerable to problem behavior at these times. All of the programs involved in the study were required to provide structured after-school services in three specific areas: academic assistance, social skills and character development, and recreational/leisure activities. In addition to determining the extent to which program participation reduced delinquent behavior, the research undertaken was designed to provide information on intervention process mechanisms associated with favorable outcome. To further contribute to the meaningfulness of their findings, the investigators used a comparison group of non-participants and employed statistical controls for preexisting between-group differences. Results of the evaluation indicated that after-school participation reduced delinquent behavior for middle-school but not for elementary-school, aged youth. For the middle-school, aged youth, the effect of participation was partially mediated through increases in positive peer associations and intentions not to use drugs. Programs that included a high emphasis on social skills instruction and practice were the most effective in altering these mediating factors and reducing delinquent behavior.

The present after-school prevention program had its origin in the findings and conclusions of a long-term investigation of risk and protective factors underlying vulnerability to narcotic addiction conducted at the Social Research Center (Hanlon, Bateman, Simon, O’Grady, & Carswell, 2002; Nurco, Kinlock, & Hanlon, 1994; Nurco, Kinlock, O’Grady, & Hanlon, 1996). This vulnerability program of research involved an examination of the retrospective self-reports of Baltimore City male narcotic addicts and non-addicted controls covering their early developmental history and experiences, with particular emphasis on the adolescent years 11 through 14. The findings from this program of research, which were consistent with key concepts and conclusions of prospective studies of the trajectory of deviant activity (Catalano & Hawkins, 1995; Howell, 1995; Loeber & Farrington, 1998; Thornberry et al., 1995), indicated that individuals who ultimately became addicted were much more deviant during their early developmental years than those who did not. The prevention approach that evolved from this research, the Village Model of Care, was subsequently found to be effective when applied to the treatment of urban African American youth manifesting early stage problematic school and/or delinquent behavior (Hanlon et al, 2002).

Method

Overview

The purpose of the present study was to determine the effectiveness of the Village Model of Care as an after-school intervention program targeting African American youth entering an urban middle-school environment. Controlling for initial dispositions and baseline characteristics and employing a comparison sample, the study was designed to test the assumption that, compared to no after-school intervention, the Village Model of Care after-school program would be associated with more favorable outcome at one-year follow-up in terms of generally accepted operational definitions of behavioral adjustment and academic performance. An examination of process information was also undertaken to determine whether positive outcome for the intervention was related to the degree of program participation on the part of both students and their parents.

Study Design

Male and female students enrolled in the sixth grade of two urban middle schools, all but a small percentage of whom were African Americans, served as participants in the after-school program evaluation. Recruitment of participants at the two schools was conducted over consecutive years of a four-year intake period. Following routine school orientation procedures, recruitment of participants was undertaken during the initial months of the school years and involved obtaining the informed assent of the child and informed consent of the parent (caretaker) following an explanation of the study, which included a description of the extent of participant burden and potential risks and benefits of participation.

Determined on a chance basis, one of the schools involved in the study served as the intervention and the other as the comparison site. In terms of the characteristics of the geographic areas and general populations served, the two sites were equally representative of higher-risk urban settings. Indicative of the comparability of sites, an unpublished administrative report containing 1997-1998 data compiled by the City Family League and Health Administration revealed that both school districts were in the highest category of the After-School Risk Index for the City, an index determined on the basis of juvenile arrests, overall school achievement, school truancy, and teen birth rates.

On entry to the study, participants at both sites were administered a baseline assessment that included structured interview and standard behavior inventory information with normative data applicable to an African American sample. These instruments were completed by both students and their caregivers, and also contained teacher-reported information with regard to the childrens’ school achievement, present functioning, and adjustment. Students at the intervention site were then enrolled in the Village Model of Care after-school prevention program, described below, for the remainder of the school year. Final assessments involving the same student and caregiver measures administered at baseline were conducted at both sites at the end of a 12-month (from baseline) follow-up period. During the 12-month evaluation period, no after-school intervention was conducted at the comparison school site. However, students at both sites were the recipients of routine substance abuse prevention procedures conducted periodically during regular school hours by the City School System and all students had access to guidance services available at the schools.

The Intervention Program

The preventive intervention program was guided by the tenants of the social developmental model of Hawkins, Catalano, and Miller (1992), specifying the role that risk and protective factors play in the development of deviant behavior. It was specifically designed to prevent both the initiation to, and escalation of, alcohol, tobacco and other drug (ATOD) use; to avert participation in violent behaviors; to delay initiation of sexual activity; and to improve social skills. In view of the relationship found between academic achievement and a reduction in risk for subsequent problem behavior (Gottfredson et al. 2004; Tarter, Sambrano, & Dunn, 2002), it was also designed to improve academic performance. Developed by African American professionals and employing culturally sensitive principles and methods, the day-to-day operation of the intervention involved a group process reflecting the strong sense of ethnic identity and group affiliation generally found in African American communities (Carswell & Carswell, 2007; Struchen & Porta, 1997, citing Parker, 1995). It also focused on issues and topics bearing on distinguishing features of African American heritage, culture, commonly encountered experiences, and ways of coping with negative environmental influences. With regard to the parents of the children, it was guided by the teachings of an ancient African proverb that states “It Takes a Whole Village to Raise a Child,” which highlights the importance of extended family networks and community supports in assisting parents to raise positive-minded, healthy children (Daley et al., 1995; Dickerson, 1995; Randolph, 1995).

The preventive intervention program administered was an after-school approach that included the following components: 1) Structured group mentoring; 2) Parental empowerment and support services; and 3) Community outreach services. It was implemented with youth and their primary caregiver(s) over a sixth-grade school-year period and involved after-school programming four days per week, scheduled gatherings of students with their families, and organized field trips. For description and evaluation purposes, assessment measures, including the collection of information regarding students’ risk and protective factors, were administered to all study participants at baseline and at one-year follow-up.

A core component of the after-school program, structured group mentoring, involved two adult role models working with groups of approximately 20 students. Functioning in a dual capacity as educators and advisors, mentors were role models from the community (generally African American college students or recent college graduates), who, in addition to educational assistance, provided the students guidance, companionship, and emotional support in group mentoring sessions conducted 4 days per week for 2½ to 3 hours each day. Mentored group activities involved remedial education, including study-skills exercises and assistance with homework assignments; discussion of self-control topics (such as problem solving, coping with stress, conflict resolution and anger management); consideration of career opportunities; the fostering of an appreciation of African American cultural heritage; and the provision of recreational/social activities designed to increase social skills and creative/artistic expression. Additional discussions, presentations, and didactic interactive activities involved the provision of information on health promotion and risk reduction strategies regarding ATOD use, premature sexual activity, violence, and other health-compromising behaviors.

Throughout all of the topics considered in the mentoring sessions, the importance of education to life’s goals and objectives was discussed. Thus, the group mentoring process was undertaken within the context of a program having a strong educational emphasis that was designed to have a positive influence on youths’ general orientation toward learning and life in general (Haensly & Parsons, 1993). In accord with the CDC Best Practices of Youth Violence Prevention Sourcebook (Thornton et al., 2000), which suggests that mentoring programs work best when augmented by parental support and community partnerships, the program also focused on parental involvement and community support.

Parents were engaged in program activities as a function of their lifestyle, work schedule, and family support needs. Scheduled parental involvement included voluntary participation in the following activities: 1) Family gatherings; 2) Teleconferencing (mentor feedback regarding youth participation in program activities); and 3) Community outreach services. In addition, parents were notified of program activities through the use of monthly newsletters and calendars, which were disseminated to parents/caregivers, students, and school and project staff.

Besides informal and individual case problem-solving contacts, the principal way parents were involved in the prevention program was through 2-hour family gatherings which provided program staff an opportunity to more directly determine the needs and aspirations of family participants. The gatherings, which were held several times during the school year, were also designed to provide parents an opportunity to network with each other regarding common issues; bond with school administrators, staff, and teachers; and obtain guidance and information from project staff regarding their children, as well as more general information from community representatives having expertise relevant to urban family life and child development.

Community outreach services were provided to students and their families that were designed to strengthen relationships among youth, their parents/caregivers, and the broader community by engaging community volunteers in the program and by promoting the involvement of the youth and their parents in community activities and the use of community resources and services. This component of the program was also responsible for supplementing program services with field trips, conducted at least twice during the school year, that were both educational and recreational in nature. Besides providing cultural enrichment opportunities for the youth, the field trips were intended to strengthen mentor/mentee relationships, provide a variety of opportunities for mentors to model appropriate behavior in community settings, and provide both mentors and mentees with opportunities to interact with participating parents and community volunteers.

Study Evaluation Assessments

Youth Questionnaire

Self-report information on the youth at baseline and follow-up was obtained by trained interviewers using a structured interview schedule derived from our initial work on vulnerability to narcotic addiction and from our subsequent research on the family background and early developmental experiences of substance abusing mothers and their children (Hanlon et al., 2002, 2004). In addition to demographic information, this questionnaire examined self-reported characteristics and experiences during adolescence previously found to be predictive of drug use and criminal activity in adults, including Likert-type scores for such relevant areas as family stability and functioning, school interest and performance, the deviance of peer associates, and personal characteristics and behavior (including deviant activity). At baseline, the time periods covered by the questionnaire were lifetime and previous 6-months. At 12-month follow-up, only past 6-months’ self-report information was obtained. Those responsible for the care of the children were also administered a companion interview-based questionnaire, the Caregiver Questionnaire, which provided information on the behavior and adjustment of the children from the caregivers’ perspective.

The Child Behavior Checklist (CBCL)

Completed by the caregivers at baseline and follow-up, the CBCL (Achenbach, 1991a) is designed to assess in a standardized format the behavior/emotional problems of youth ages 4-18, as reported by caregivers. The CBCL, which, in addition to social, thought, and attention problems, yields scores for both internalizing (withdrawn, somatic complaints, and anxious depression) behavior and externalizing (delinquent and aggressive) behavior, has been widely used as an evaluation research instrument to assess children. There is considerable published data establishing its reliability and validity.

Teachers Report Form (TRF)

Also completed at baseline and follow-up, the TRF is designed to provide teachers’ reports of their pupils’ adaptive functioning and behavioral problems in a standardized format (Achenbach, 1991b). Modeled on the CBCL and demonstrating sound psychometric properties, it provides an efficient and economical means of comparing a particular child’s school functioning with that of a normative sample of peers. Reflecting the CBCL, the TRF yields scores for the same primary and secondary dimensions.

Conners’ Rating Scales-Revised (CRS-R)

The short version of this instrument was used to assess the conduct and emotional problems of the children from the separate viewpoints of their parents (CPRS-R) and teachers (CTRS-R). The scales, which measure oppositional and cognitive problems as well as Hyperactivity and ADHD manifestations over the previous month, provide pertinent adjustment information for treatment evaluation purposes. Having demonstrated satisfactory reliability and validity in numerous studies (Wainwright et al., 1996), the short version of the scales is recommended for pre- and post-treatment research study designs.

Multidimensional Self-Concept Scale (MSCS)

Based on a large (N = 2,501), nationally representative normative sample of youth aged 9 to 19, the MSCS (Bracken, 1992) provides an overall assessment of self-concept, along with six individually scaled dimensions (i.e., social, competence, affect, academic, family, and physical). Research application of this self-report instrument is appropriate in academic settings where multidisciplinary assessments are conducted to explore a wide range of possibilities for poor academic performance or unacceptable school behavior. It is also useful in programmatic evaluations in which improvement in self-concept, as one aspect of social/emotional adjustment, is a principal goal.

School Records and Process Information

Because remedial education was a strong component of our after-school intervention program, in addition to self-, caregiver-, and teacher-reported information, the school records of intervention and comparison students were examined for quarterly grade point average information during the school year. For the intervention group, process information included number of days of program attendance over the course of the study and intervention staff’s jointly determined ratings of the level of parental participation, which was measured on a 4-point scale as either poor (1), fair (2), good (3), or excellent (4).

Statistical Analyses

For the total sample, repeated-measures MANOVA was the primary statistical method used to determine the extent of differential changes in outcome scores from baseline to one-year follow-up associated with the intervention conditions (O’Brien & Kaiser, 1985). In all cases, the independent variables in these analyses were intervention condition (intervention school vs. comparison school), gender, and age. Exploratory analyses involving both youth and caregiver participants in the after-school program were also conducted to determine relationships between the extent of participation in the program and outcome results. The level of significance for all statistical analyses undertaken was set at .05 (.10 for indication of a tendency).

Results

The Study Sample

Over a four-year intake period, 532 students entered the study and received a baseline assessment. Of these, 478 (237 at the intervention and 241 at the comparison school) were available for follow-up testing at the end of a one-year period and were thus included in the study sample. Of the 54 dropouts, 21 were due to school transfers unrelated to the study and 33 to refusals of follow-up testing, with nearly equal representation of refusals within the two study conditions.

Youth Questionnaire Information

Of the 478 student participants, almost all of whom were African American (97.91%), 217 (45.40%) were male and 261 (54.60%) female. The age range of the sample at baseline interview was from 11 to 15 years (only three individuals were over 13), and the mean age was 11.12 years, SD = .53. Reflecting similarities in the environments of the schools, baseline demographic characteristics, living arrangements, and family background circumstances of study participants at the two sites were essentially the same.

Eighty-three percent of the children reported that they had been raised by their birth mothers and 65% reported that their birth fathers were also their primary caregivers throughout most of their lives. In spite of high rates of parental separation, few of the children were currently in foster care. Most were currently under the care of one or both of their birth parents, and in a large percentage of the remaining cases, family members, primarily grandparents, had assumed parental responsibilities. As a result, satisfaction of the children with the upbringing and care they were receiving tended to be uniformly high.

All of the children were at risk of being negatively impacted by adverse and disadvantaged circumstances that characterized their neighborhoods, including relatively high levels of poverty, violence, criminal activity, and drug abuse. With regard to personal characteristics and behaviors ordinarily associated with higher risk, most of the children denied involvement in serious types of deviant behavior. At baseline on a scale measuring types of deviant activity ever engaged in that ranged from 0 to 25, the mean score for the total sample was only 1.7. Reported negative acting-out behavior primarily involved occasional misdemeanors or school-related misbehaviors that in many cases resulted in suspension from school (43% lifetime incidence, 24% more than once). Association of the children with close friends who were involved in deviant behavior, including drug abuse, was relatively high. Fourteen percent reported that their close friends had been expelled from school, 26% that their friends had engaged in shoplifting, and 19% that their friends had at one time or another been arrested for criminal activity. Although only 8 of the 478 children (2%) admitted ever having used drugs themselves, 32 (7%) reported that their friends had used drugs.

There were no significant differences between the intervention and comparison samples in interview questionnaire information relating to youth deviant activity provided by both students and caregivers from baseline to follow-up assessment. For the entire sample, over the course of the study observation period, there was little to no initiation or increased involvement in tobacco, alcohol, illicit drug use, or risky sexual behavior. Low levels of deviant activity continued to characterize the students at follow-up, with school problem behaviors and associations with deviant peers during the year having progressed little beyond initial levels. A generally held interest in school was maintained, as well as an expressed appreciation of the importance of avoiding drug use. At follow-up, almost all (90%) of the students indicated that they were optimistic about reaching their personal goals in the future.

Standardized Inventory Measures

At baseline, internalizing (psychological problems) and externalizing (conduct problems) mean scores of 5.74, SD 35.86 and 9.47, SD 8.45 for the total sample on the Child Behavioral Checklist (CBCL) were, respectively, slightly below and slightly above those for the normative sample. Total mean raw scores on the Teacher Rating Form (TRF) for the children also indicated lower internalizing and higher externalizing scores for our total sample (3.78, SD 5.13 and 9.38, SD 12.16, respectively).

As indicated, the repeated-measures MANOVA model was used to examine the extent of differential change in outcome measures from baseline to follow-up for the intervention and comparison samples, controlling for the effects of gender and age. Results of these analyses for the CBCL and TRF checklist internalizing and externalizing scores revealed no differential change favoring the intervention condition. Contrary to expectations, results for the CBCL externalizing dimension, based on parents’ ratings of the children, significantly favored the comparison group, due principally to the fact that the comparison group had an exceptionally high initial score on this measure at baseline. Although the mean externalizing problem score for the comparison group was significantly higher than that for the intervention group at baseline, the problem levels for the two groups were nearly identical at follow-up [Ms = 8.68 (SD 8.18) and 8.80 (SD 8.52)]. Of the two subscales underlying the externalizing dimension, aggressive and delinquent behavior, only the former was associated with differential change. Displaying no interaction with intervention condition, the only other significant effect associated with the CBCL involved gender, with females compared to males in the total sample demonstrating a significantly greater reduction in the internalizing problem dimension from baseline to follow-up.

Baseline mean raw scores for Conners’ Parent Scale (CPRS-R) and Teacher Rating Scale (CTRS-R) scores of the students in the sample were similar over all dimensions to those obtained for same-aged African American youth in the normative data. Baseline to follow-up results for the total study sample on the CPRS-R revealed significant reductions for all four of the CPRS-R dimensions: Oppositional (p = .022); Cognitive Problems (p<.001); Hyperactivity (p = .017); and ADHD (p<.000) - none of which indicated the occurrence of an interaction with the intervention conditions. For the CTRS-R, or teacher version, there was a reduction of Cognitive Problems (p = .046) favoring the intervention over the comparison group, along with tendencies for differential study condition-related reductions in Hyperactivity (p = .075) and ADHD (p = .093) also favoring the intervention group.

Based on a frequency distribution with a mean of 100 and a standard deviation of 15 in the normative sample, total sample mean scores for the six subscales of the Multidimensional Self-Concept Scale (MSCS) were found to be in the 95 to 103 range, indicating comparability with respect to the MSCS normative sample, as well as little variation among the subscales. At follow-up, there was a slight increase in total sample mean self-esteem scores for all subscales, with no indication of significant intervention–related differences. The only analysis of a differential MSCS effect approaching significance was that for Academic Self Esteem (p = .106) favoring the intervention group, a finding complemented by grade point average results presented below.

Grade Point Average

Grade point average changes from the beginning (first quarter) of the sixth-grade to the end of the school year (fourth quarter) for the intervention vs. comparison groups revealed a significant differential effect favoring the intervention group (p<.001), with no interaction of intervention condition with either age or gender. This differential effect favoring the intervention group was consistent with the ratings of teachers on the CTRS subscale, Cognitive Problems, noted above, and to a more limited extent was reflected in CTRS results for Hyperactivity and ADHD, both of which were based on teachers’ observations of the students’ classroom and general school behavior reflecting greater self-control.

Dosage

In order to determine the relevance of dosage-related process information to the grade point average change noted in the intervention sample, both student attendance and quality of parent participation in the program were examined. For those 237 intervention school students who received a follow-up assessment, the number of days of program attendance ranged from 2 to 126, with a mean attendance of 77 days. The mean number of days attended represented approximately 60% of sessions initially scheduled for the school year. Three-fourths of the intervention sample (N = 178) attended at least half of the program’s scheduled sessions. For these students, there was a tendency for a greater increase in grade point average from the first to fourth quarter than that for those 56 students failing to attend at least half of the scheduled sessions [Ms = 3.58 (SD 5.37) vs. 2.17 (SD 4.78); p = .08)].

Of those 233 intervention school parents whose program participation was jointly assessed by program staff (who were aware of program attendance by the students but unaware of grade point average findings), the quality of the participation of 88 (slightly over one-third) was judged to be “good to excellent” and that of the remaining 145 parents “fair to poor.” For children of the parents in these two groups, the respective increases in mean grade point averages from baseline to follow-up were 4.38 ± 5.08 and 2.53 ± 5.27, a difference that was significant at p = .009. In a related finding, the 88 intervention school parents who were viewed by staff as being more involved with the program had a significantly higher mean level of education than that for the 145 parents who were judged to be less involved (p = .04), but the two groups of parents did not differ significantly with respect to self-reported household income.

Discussion

Except for teacher observation and grade point average change scores, evidence of differential effects favoring the intervention condition in this study was negligible. However, those differential effects noted were reflective of the relatively high amount of time the after-school program intervention devoted to educational activities. With regard to the potential impact of an increase in grade level, Tarter et al. (2002) and, more recently, Gottfredson et al. (2004) have specifically noted that interventions that promote academic achievement have the potential to reduce the risk for subsequent problem behavior, including substance abuse. Also, in their recently reported developmental research, Werner and Johnson (2004) have provided evidence of an association between early academic performance and longer-term adaptability, reporting that scholastic competence at 10 years of age was positively linked to a sense of self-efficacy at age of 18, which in turn was linked to less distress and emotionality at age 32.

In another recent report involving a meta-analysis of the relationships among three major risk factors (deficiencies in academic performance, bonding to school, and social competence) and after-school prevention program outcome, Najaka et al. (2002) found that the most convincing evidence of a relationship between risk and problem behavior was found for school bonding on the part of the child. In those studies examined, positive changes in school bonding resulting from prevention efforts were consistently accompanied by improvement in problem behavior. And, more directly related to present findings, these investigators note that preventive interventions leading to improved academic performance also tended to yield moderate improvements in problem behavior. In discussing the results of their analyses, these authors argued for a causal, rather than correlational, relationship between changes in academic performance and problem behavior.

Although, we did not find a relationship between an increase in academic achievement, defined in terms of grade point average, and improvement in problem behavior, this result could have conceivably been due to the low levels of both initial manifestations and change in problem behavior subsequently detected in both our intervention and comparison samples in spite of a continued exposure of the children to multiple at-risk circumstances in the community. Our result is consistent with that of Baker and Witt (1996), who found that following after-school program involvement, there were significant differences between participants and non-participants in academic scores but not in terms of problem behavior. Although McCord et al. (1994) initially found no behavioral change associated with a school-based preventive intervention promoting self-control and social skills, these investigators eventually found improvement in terms of both academic performance and delinquency three to five years later.

On the basis of present findings, we are unable to determine whether positive change in grade point average among our intervention participants was simply due to supplemental educational efforts or to an increase in motivation and commitment resulting from the positive atmosphere we attempted to generate through our group mentoring process. Our finding that greater parental involvement in the program was related to increased grade point average scores at follow-up is consistent with the general finding that parental practices have an impact on the academic achievement of children (Tarter et al., 2002). Except for tracking the possible mediating effect of parental involvement on changes in grade point average, we did not undertake a separate examination of the effects on outcome of other major components of the intervention. This was due, in part, to the limited extent intervention-related changes were discernible.

The differential result involving parents’ ratings of the problem behavior of their children that favored the comparison group was not only unexpected but difficult to explain in terms of a school-based experimental condition effect inasmuch as there was no corresponding differential improvement in teacher ratings of problem behavior at the comparison school. Catalano, commenting on a discrepancy between parent and teacher assessments in his research in a recent NIDA Notes article by Farrer (2004), makes the point that a lack of parent-teacher agreement in assessments of participant children is not an unusual finding in prevention research. Because parents are generally less aware than are teachers of how their children interact in structured environments and in social behavioral contexts involving interactions with peers, they are typically less able to be objective in determining how well their children are functioning under these conditions.

During the present study, intervention staff frequently remarked about significant behavioral and attitudinal changes that appeared to be occurring among some of the intervention participants. However, determining whether such changes were intervention-related, subjective bias, or simply natural developmental occurrences would have required systematic observations of such changes in both our intervention and comparison groups to be considered other than anecdotal information. Although possibly reflected in favorable treatment outcome in ways not routinely assessed in normative research, these “exceptional cases” had little discernible impact in terms of the behavioral outcome measures examined.

The paucity of differential behavioral effects favoring the intervention condition in the present study is disappointing but not surprising considering the difficulties involved in convincingly demonstrating intervention-related changes in the problematic behavior of adolescent youth over a brief period of observation, particularly in prevention research involving a universal sample with little initial evidence of deviant activity. Because individual differences in the delinquent behavior of such individuals are more likely to emerge during later years, the field of prevention has tended to focus on changes in circumstances and behaviors that place youth at risk for the development of deviant lifestyles rather than on deviance itself. For participants in the intervention group, all but a few of whom were obviously non-delinquent, there was evidence of improvement in their school adaptation and achievement, two interrelated protective factors generally found to be negatively associated with the subsequent development of delinquent behavior. As is typical of all evaluations of short-term prevention efforts, additional research is needed to determine the extent to which the differential academic achievement presently noted persists, the extent to which further treatment-related changes occur over time, and whether the initial effects observed in this study are associated with later favorable adjustment status.

Study Limitations

A recognized limitation of the study is the fact that it did not employ random assignment of participants to the two experimental conditions and therefore is not strictly a controlled participant-based evaluation. However, the two study sites were randomly chosen as the experimental and comparison locations and were selected for outcome evaluation purposes because they were comparable with respect to the environmental characteristics of the urban area and the type of students served. As an additional precaution, initial sample differences having a bearing on study outcome criteria were statistically controlled in the analyses undertaken. However, randomization by site instead of by participant, a common practice in prevention research, has recently come under criticism from methodologists because of the inherent limitation of this procedure in establishing a cause-effect relationship between the experimental conditions and the outcomes examined (Murray et al., 2004). Another limitation of the present prevention-focused study has to do with the involvement of participants who, as a group, were at relatively low risk of serious delinquent behavior in spite of unfavorable environmental and economic circumstances. Coupled with the study’s relatively brief intervention and observation periods, this fact reduced the likelihood that differential intervention-related changes in negative behavior would be observed.

Implications for Future Studies

There are implications in this report that have a bearing on research on the effectiveness of preventive interventions targeting at-risk urban youth. Prominent among the more immediate, practical of these are two that relate to future research practice and the preventive intervention evaluation process. With regard to the former, it is important that researchers bear in mind that when it comes to assessing their children’s behavior, parents are generally not impartial observers. This is particularly the case in obtaining judgments on the extent of improvement in a child’s behavior during a study’s observation period. Besides having a greater opportunity to observe the competitiveness and social competence of the children in a peer-group environment, teachers are more likely to give a less biased and, consequently, more meaningful account of changes that occur in a child’s social and goal-oriented behavior over the course of a study. With regard to the evaluation process itself, in studies involving both parents and students, it would be informative from an intervention strategy standpoint to determine whether the intervention increases the bonding of both parents and their children to the school and to relate these findings to the quality of the child’s participation in the intervention and to the extent of change occurring in academic performance.

More generally, results of the present study highlight the importance of extended follow-up in prevention studies targeting middle school students in disadvantaged urban communities. Despite unfavorable environmental circumstances that place them at risk for the adoption of negative behavioral patterns, most of these youth do not manifest a disproportional amount of deviant behavior at this stage of their lives. Results obtained from a preventive intervention with respect to this domain are, therefore, not likely to be immediately discernable. In view of this, it is important, when feasible, to extend the post-intervention observation period of prevention studies to allow for the development of greater individual differences in both positive and negative behavior as students become older adolescents. Such an extension would also allow determination of the longer-term effects of changes in mediators (academic achievement, social skills, etc.) of resilience to deviance that may have occurred as a result of the intervention.