Programs to improve adolescent sexual and reproductive health in the US: a review of the evidence

Abstract

Background

US adolescents have high rates of teen pregnancy, childbearing, and sexually transmitted infections (STIs), highlighting the need to identify and implement effective programs that will help improve teen sexual and reproductive health.

Materials and methods

This review identified 103 random-assignment evaluations of 85 programs that incorporated intent-to-treat analyses and assessed impacts on pregnancy, childbearing, STIs, and their key determinants – sexual activity, number of sexual partners, condom use, and other contraceptive use – among teens. This review describes the evidence base for five broad program approaches, including abstinence education, comprehensive sex education, clinic-based programs, youth development programs, and parent–youth relationship programs. We also describe programs with impacts on key outcomes, including pregnancy/childbearing, STIs, and those that found impacts on both sexual activity and contraceptive use.

Results

Our review identified 52 effective programs: 38 with consistent impacts on reproductive health outcomes, and 14 with mixed findings (across subpopulations, follow-ups, or multiple measures of a single outcome). We found that a variety of program approaches produced impacts on sexual and reproductive health outcomes. Parent–youth relationship programs and clinic-based program evaluations more frequently showed impacts than other program approaches, although we also identified a number of abstinence-education, comprehensive sex education, and youth-development programs with impacts on sexual and reproductive health outcomes. Overall, we identified nine program evaluations with impacts on teen pregnancies or births, five with impacts on reducing STIs, and 15 with impacts on both delaying/reducing sexual activity and increasing contraceptive use (including condom use).

Conclusion

Future efforts should conduct replications of existing program evaluations, identify implementation components linked to impacts, rigorously evaluate programs that appear promising, and expand the evidence base on programs that impact hormonal and long-acting contraceptive method use.

Introduction

US teen-birth rates have declined across several decades, and the 2013 birth rate of 26.5 births per 1,000 teens ages 15–19 years was the lowest recorded birth rate.1 However, US teen-birth rates remain higher than those in other developed countries, and are 1.5 times the rate in the UK, more than two times the rate in Canada, and more than five times the rate in Sweden.2 Currently, an estimated one in nine teens in the US will have a birth before they reach age 20 years (Child Trends, unpublished data, 2014). Notably, the majority of teen births (77%) are unintended, including 58% that occurred sooner than the teen planned and 19% that were unwanted.3 Teen-birth rates are especially high among racial and ethnic (Child Trends, unpublished data, 2014); black and Hispanic teens have birth rates that are double the rate of whites in the US, although many of these differences reflect the socioeconomic disadvantage of these populations.4

Teen childbearing is linked to a host of negative outcomes among teen parents, their children, and society as a whole. Despite the fact that many teen parents were disadvantaged before they became parents, researchers have found associations between early parenthood and lower educational attainment and higher poverty rates among mothers5 and poorer academic and behavioral well-being among their children.6 Moreover, some research has found links between delaying childbearing in one generation (with an accompanying increase in educational attainment) and greater economic well-being of the next generation.7 Also, a substantial proportion of teen births (17% in 2013) are higher-order births,8 which are linked to even greater disadvantage among teen parents and their children.9 Moreover, teen childbearing costs US taxpayers billions of dollars through public assistance payments and through social services, such as health care and child welfare.10

US adolescents also have high rates of sexually transmitted infections (STIs). Almost half of the 20 million STIs that are reported in the US each year occur in adolescents and young adults.11 One study of sexually active teen females (ages 14–19 years) found that almost four in ten had at least one of five common STIs.12 STIs are an ongoing threat to adolescent health and well-being, and if left untreated they can lead to infertility, pregnancy complications, organ damage, and even death.13 These factors highlight the need to identify a variety of effective evidence-based programs that have found impacts on improving adolescent reproductive health through rigorous evaluations.

Key determinants of teen pregnancy and/or STIs include the timing and frequency of sexual activity, the number of sexual partners, and the consistent use of condoms and other effective methods of contraception. Almost half of high school teens have ever had sexual intercourse, including nearly two-thirds of 12th graders.14 Teens who are younger at first sex are less likely to use contraception, and are at a greater subsequent risk of a teen birth or STI.6 Although many US teens report using contraception at last sex (86% of 15- to 19-year-old females and 93% of 15- to 19-year-old males in 2006–2010), less than half of female teens use hormonal methods, and even fewer use the most effective, long-acting methods, such as intrauterine devices or implants.6 The lower use of these highly effective methods contributes to the higher teen-birth rates in the US compared with other industrialized countries.15 In addition, many teens use condoms inconsistently.16 Therefore, programs that help teens delay the timing of first sex or increase condom use or other contraceptive use can help reduce high rates of teen pregnancy and STIs. A recent study found that implementing evidence-based teen-pregnancy prevention programs is a cost-effective strategy to save taxpayer dollars.17

Polling data suggest that there is continued broad public support for sexual and reproductive health education for adolescents,18 and the US government has several funding initiatives to implement and scale up evidence-based approaches to teen-pregnancy prevention.19 However, the current list of evidence-based teen-pregnancy programs that are approved for replication is still fairly limited,20 and providers describe the need for on-the-ground adaptations of existing programs to make them more relevant for diverse teen populations.21,22 As the US population, particularly the youth population, continues to become increasingly diverse, it is vital that communities have a variety of program approaches tailored to the needs of their target populations. This highlights the need to identify a variety of rigorously evaluated program approaches and settings with impacts on adolescent reproductive health outcomes.

Materials and methods

Review criteria

This study examined evaluations of reproductive health programs that incorporated random-assignment and intent-to-treat analyses (including data for all subjects who were randomly assigned in the analysis, regardless of their participation in the study). Evaluations were identified through peer-review articles and published (or unpublished) evaluation reports. We did not set a requirement for publishing date, so articles were identified from 1990 to 2014. All of these evaluations are included in Child Trends’ What Works/LINKS (Lifecourse Interventions to Nurture Kids Successfully) database,23 a database of over 700 social intervention programs to assess child or youth outcomes related to education, life skills, and social/emotional, mental, physical, behavioral, or reproductive health. LINKS includes evaluations of programs that do and that do not have positive impacts, allowing us to assess approaches that appear to be more and less effective, based on the rigorous evaluations available to date. We limited the review to evaluations of programs that were implemented primarily with adolescents under the age of 18 years, that did not target expectant and parenting adolescents, and assessed impacts on reproductive health outcomes and their key behavioral determinants. These included: sexual behaviors (sexual initiation or the percentage of teens who had ever had sex, frequency or recency of sex, number of sexual partners, anal/oral sex, sex under the influence of drugs or alcohol); condom use or contraceptive use (any contraceptive/condom use, consistency of use, hormonal method use, use of long-acting reversible methods, such as intrauterine devices and implants); STIs (including self-reports and -testing); and pregnancies or births (based on self-reports).

This review identified 103 evaluations of 85 reproductive health program models that were used for this study. There are more evaluations than programs, because 15 program models were evaluated two or more times and differed from the original implementation based on population, setting, or program components. These programs were implemented in a variety of settings (with several that were implemented in two or more settings), including 54 that were implemented in schools, 26 in clinics, 20 in community-based organizations, and 18 in other settings, such as the home and juvenile justice facilities.

To help organize our findings, we divided our review into five broad categories of sexual and reproductive health programs: 1) abstinence-only or abstinence-based education programs (14 evaluations), 2) comprehensive sex-education programs (47 evaluations), 3) clinic-based programs (14 evaluations), 4) youth-development programs (17 evaluations), and 5) parent–youth relationship programs (eleven evaluations). We also describe evaluations – across program categories – that found impacts on key outcomes, including pregnancy or childbearing, STIs, and those that found impacts on both sex and contraception.

This review does not focus on the magnitude of the impacts found, but rather the number of statistically significant impacts found. For example, some program evaluations assessed impacts on multiple measures of a single outcome (such as condom use at last sex, condom use in the past 3 months, and consistent condom use), on more than one subpopulation (such as males and females; whenever provided, we examined impacts based on analyses of the full sample; we only took into account subpopulation analyses [such as by sex] if an evaluation did not report analyses for the full sample) or on multiple follow-ups (such as at posttest and first and second follow-ups). In these cases, our review assessed the level of evidence across these measures, populations, or follow-ups. The impacts of the program outcomes reviewed for this brief are reported in the following three categories: Found to work, Mixed findings, and Not found to work, all of which are described in the following sections. All of the tables in this report are organized by these categories. At least two researchers reviewed each evaluation to confirm the coding of program impacts.

Found to work

Programs in this category found positive and statistically significant impacts on the majority of measures assessed within an outcome or the majority of subpopulations (when full sample results were unreported) or follow-ups. For example, a program that found an impact on condom use at last sex and consistent condom use but did not find an impact on condom use in the last 3 months would be categorized as “found to work” for condom use, because it found impacts on the majority of measures of condom use. Programs were also considered to have worked for an outcome if the impact was delayed, such as a program that found no impact on pregnancy at posttest, but found a positive impact at follow-up.

Mixed findings

Programs in this category found varied impacts on particular outcomes, either at different follow-ups, for different subgroups, or on different measures. For example, a program that found significant reductions in the initiation of sex at posttest but had no impact at follow-up would receive a “mixed” coding.

Not found to work

Programs in this category had nonsignificant, marginally significant, or negative impacts on the majority of measures assessed. Note that some programs in this category have shown some impacts, but not for the majority of subgroups, follow-ups, or measures of the same outcome. For example, a program that found an impact on sexual initiation at posttest but not at two other follow-ups would receive a “not found to work” coding for sexual initiation because there were impacts for only one of three time points, and impacts diminished over time.

Effective programs

We describe programs as “effective” if they worked or had mixed findings for at least one outcome. The tables are separated by effectiveness. All programs that were not found to work for any outcome are included in Table S1, while those that worked or had mixed findings (effective programs) are included in Tables 1–5.

Table 1.

Abstinence-based/abstinence-focused programs

Program	Sexual initiation	Frequency/recency of sex	Number of partners	Anal/oral sex	Sex under the influence	Condom use	Any contraception use	Contracting STIs	Pregnancies and births	Comments
Choosing the Best24 Classroom-based abstinence curriculum providing developmentally phased messages • Mixed race/ethnicity and sex, ninth-grade students, school setting, more than 6 months, less than 10 hours	M	N								• Mixed impact on initiation of sex; positive impact at posttest, no impact at follow-up. • No impact on amount of time since last sex.
Making a Difference!25 Abstinence-based approach to HIV/AIDS and teen-pregnancy prevention • African-American mixed-sex sixth- and seventh-grade students, school setting, 3–6 months, less than 10 hours	Y	Y	N			N				• Positive impact on initiation of sex and on frequency of sexual intercourse in the past 3 months. • No impact on having multiple partners in the past 3 months, consistent condom use, or rates of unprotected intercourse.
Positive Prevention26 Abstinence-focused, HIV/STI-prevention education for high school students • Mixed-race/ethnicity and -sex ninth-grade students, school setting, less than 3 months, less than 10 hours	Y	N				N				• Positive impact on initiation of sex. • No impact on frequency of sex or frequency of condom use among students who were sexually active.
TeenSTAR27 Abstinence-focused year-long pregnancy prevention program • Hispanic females aged 15–16, school setting, more than 6 months, 10–19 hours									Y	• Positive impact on reducing pregnancy.
TeenSTAR28 Abstinence-focused year-long pregnancy-prevention program • Hispanic mixed-sex youth aged 12–18 years, school setting, more than 6 months, 10–19 hours	Y	Y								• Positive impact on sexual initiation. • Positive impact on sexual discontinuation (if participants went more than 3 months without having sex).

Notes: Y, found to work: positive and statistically significant impacts on the majority of measures assessed within the outcome. M, mixed findings: varied impacts on the outcome; impact varied based on time, subgroup (when full sample analyses unreported), or on different measures. N, not found to work: nonsignificant, marginally significant, or negative impacts on the majority of measures assessed within the outcome.

Abbreviations: STIs, sexually transmitted infections; HIV, human immunodeficiency virus; AIDS, acquired immunodeficiency syndrome.

Table 5.

Parent–youth relationship programs

Program	Sexual initiation	Frequency/recency of sex	Number of partners	Anal/oral sex	Sex under the influence	Condom use	Any contraception use	Contracting STIs	Pregnancies and births	Comments
Familias Unidas70 Program aimed to increase parents’ involvement in adolescent’s home and school life • Hispanic, mixed-sex eighth-grade students, primarily home setting (family visits), more than 6 months, 20 or more hours		N				Y				• Positive impact on frequency of condom use in the past 90 days. • No impact on frequency of sexual activity.
Familias Unidas71 Program aimed to increase parents’ involvement in adolescent’s home and school life • Hispanic, mixed-sex adolescents aged 12–17 years, primarily home setting (family visits), 3–6 months, 20 or more hours	N		Y	N	Y	Y		N		• Positive impact on the mean number of sexual partners in the past 90 days. • Positive impact on consistent condom use during vaginal and anal sex in past 90 days and condom use at last anal sex. No impact on condom use at last vaginal sex. • Positive impact on frequency of unprotected sex while under the influence in the past 90 days. • No impact on initiation of vaginal or anal sex or contracting STIs.
Familias Unidas and Parent-Preadolescent Training for HIV Prevention (PATH)72 Parent-centered intervention to prevent adolescent substance use and unsafe sexual behavior • Hispanic, mixed-sex eighth-grade students, primarily home setting (family visits), more than 6 months, 20 or more Hours					N	Y		Y		• Positive impact on condom use at last sex. • Positive impact on STIs. • No impact on drug or alcohol use before last sex.
Families Talking Together74 Parent-based intervention program focused on improving parent–child communication and parental monitoring • Hispanic, mixed-sex mother/adolescent dyads, adolescents mean age 12.9 years, clinic setting, less than 3 months, less than 10 hours	Y	Y		N						• Positive impact on initiation of sex and frequency of sex in the past 90 days. • No impact on giving or receiving oral sex.
Keepin’ It REAL77 HIV-prevention programs for African-American adolescents and their mothers • African-American mixed-sex youth, mean age 12.2 years, community-based setting, 3–6 months, 10–19 hours	N					Y				• Positive impact on condom use at last sex. • No impact on initiation of sex.
New Beginnings75 Group-therapy intervention developed for custodial mothers of children ages 9–12 years who have recently experienced divorce • Two treatment groups, white mixed-sex children aged 9–12 years, clinic setting, duration not reported, 10–19 hours			M							• Mixed impact on number of sexual partners; positive impact in version of program implemented with both mothers and children, no impact in version implemented only with mothers.
Project TALC (Teens and Adults Learning to Communicate)73 Intervention designed to improve behavior and mental health outcomes among parents with AIDS and their children • Mixed-race/ethnicity and -sex adolescents aged 11–18 years, clinic setting, 3–6 months, contact hours not reported									Y	• Positive impact on teenage parenthood.
REAL Men76 Program to provide fathers with HIV communication skills to talk with their sons • African-American males aged 11–14 years and their fathers, mean 12.8 years, community-based setting, duration not reported, less than 10 hours of sessions with the youth, additional hours with just the fathers	N					Y				• Positive, delayed impact on ever having sex without a condom, no impact 3- or 6-month follow-ups, but positive impact at 12-month follow-up. • No impact on initiation of sex.
STRIVE (Support to Reunite, Involve and Value Each Other)78 Short family-based program to reduce risk behaviors among homeless/runaway youth • Mixed-race/ethnicity and -sex adolescents aged 12–17 years, setting chosen by family of participants, less than 3 months, less than 10 hours		N	Y			N				• Positive impact on number of sexual partners in the past 3 months. • No impact on having any sex (vaginal or anal) in the past 3 months, the number of times had sex in the past 3 months, or unprotected sex in the past 3 months.

Notes: Y, found to work: positive and statistically significant impacts on the majority of measures assessed within the outcome. M, mixed findings: varied impacts on the outcome; impact varied based on time, subgroup (when full sample analyses unreported), or on different measures. N, not found to work: nonsignificant, marginally significant, or negative impacts on the majority of measures assessed within the outcome.

Abbreviations: STIs, sexually transmitted infections; HIV, human immunodeficiency virus; AIDS, acquired immunodeficiency syndrome.

Results

Overall, we identified 52 program evaluations out of 103 reviewed that were effective for sexual and reproductive health outcomes, including 38 that worked for at least one outcome and 14 with mixed impacts across follow-ups, subpopulations, or separate measures of a single outcome. These 52 programs had impacts across a range of program approaches and for a variety of teen populations, program durations, and settings (as highlighted in Tables 1–5).

Overall, 31 program evaluations were effective for some measure of sexual activity, including 14 program evaluations that delayed sexual initiation, 15 that reduced frequency of sexual activity, nine that reduced sexual partners, three that delayed or reduced oral or anal sex, and two that reduced sex under the influence. We also identified 31 program evaluations that were effective for condom use or contraceptive use, including 25 that increased condom use and nine that increased contraceptive use more generally. Five programs were effective for reducing STIs, and nine reduced pregnancies or births.

In the following sections, we discuss the findings for the five program approaches, including abstinence-based education, comprehensive sex education, clinic-based approaches, youth-development approaches, and parent–youth education programs. We also discuss findings for programs that found impacts on pregnancy or births and STIs, and those that found impacts on both sexual activity and contraceptive use. Our review describes the number and percentage of evaluated programs that were effective for each category/outcome, as well as program and implementation characteristics of effective programs.

Abstinence-based education programs

Abstinence-based programs, which promote abstinence above all other approaches, were sometimes found to be effective. Of the 14 identified evaluations of abstinence-education programs, only about a third of (five total) were effective by our definition. These programs are described in Table 1; abstinence-education programs that did not work are described in Table S1. All of the effective abstinence-based program evaluations were school-based programs with multiple 45-minute to 1-hour sessions spread out over several months or multiple years.24–28 These programs emphasized goals and dreams while promoting abstinence beliefs and attitudes. Although all five of the effective programs mentioned contraceptive use, only one incorporated condom-use skill building.26

Four of the effective abstinence-based programs worked for at least one outcome and one had mixed findings. As shown in Table 1, four programs delayed sexual initiation (including three that found consistent impacts25,26,28 and one that found a mixed impact, on some but not all follow-ups)24 and two reduced sexual frequency.25,28 One program reduced pregnancies or births.27 However, none of the abstinence programs reduced the number of sexual partners, increased condom or contraceptive use, or reduced STIs. None of these program evaluations measured anal sex, oral sex, or sex under the influence of alcohol or drugs.

Comprehensive sex-education programs

Comprehensive sex-education programs focus on improving reproductive health outcomes (eg, preventing pregnancy, increasing STI/human immunodeficiency virus [HIV] knowledge) and in general promote both abstinence and contraceptive/condom use. Overall, less than half of the evaluations of comprehensive programs were effective (21 of 47 worked or had mixed findings for at least one outcome; Table 2; comprehensive sex-education programs that did not work are in Table S1), and they varied by length and setting. Eight of the effective programs were full school-year or multiyear programs that were implemented in school29–37 and community-based settings.38 Several of these longer-duration programs included homework assignments,31–33,35,36 a parent/family component (such as parent–youth dialog or a parent session),31–33,35–38 or peer-to-peer interaction/mentoring.34–36,38 The remaining 13 effective comprehensive sex-education programs lasted less than 3 months or included less than 10 hours of programming focused on education and skill training.25,39–47 Their program settings varied, with several that were implemented in schools25,46,48–50 and community-based organizations,40–42,44,48,51 some that were implemented in clinic settings,39,44–46 one that was implemented partially in the home,42 and two in juvenile detention or drug centers.43,47 Most (seven) of these shorter-duration programs targeted reducing the risk of STIs/HIV,39,44,45 including three evaluations of Be Proud! Be Responsible!49–51 and one evaluation of ¡Cuídate!48 (an adaptation of Be Proud! Be Responsible! for Hispanics).

Table 2.

Comprehensive sex-education programs

Program	Sexual initiation	Frequency/recency of sex	Number of partners	Anal/oral sex	Sex under the influence	Condom use	Any contraception use	Contracting STIs	Pregnancies and births	Comments
Assisting in Rehabilitating Kids (ARK)47 Information and skill-based safer-sex training plus risk sensitization for substance-dependent youth • Mixed race/ethnicity and sex, mean age 16 years, residential drug-treatment setting, less than 3 months, 10–19 hours		Y	N			Y				• Positive impact on percentage abstained from sex, frequency of unprotected vaginal sex, and frequency of condom-protected intercourse in the past 3 months. • No overall impact on number of sexual partners in the past 3 months; positive impact at posttest, no impact at 6- and 12-month follow-ups.
Be Proud! Be Responsible!49 HIV-education and skill-training program for African-American adolescents • African-American males, mean age 14.6 years, school setting, less than 3 months, less than 10 hours		M	Y	Y		Y				• Positive impact on number of sexual partners, frequency of unprotected sex, and anal sex. • Mixed impact on frequency of sex; no impact on remaining abstinent in past 3 months, but positive impact on number of days had sex in the past 3 months.
Be Proud! Be Responsible!50 HIV-education and skill-training program for African-American adolescents • African-American mixed-sex youth, mean age 13 years, school setting, less than 3 months, less than 10 hours		N	N	Y		Y				• Positive impact on frequency of unprotected sex and anal sex. • No impact on abstinence in the past 3 months or number of partners.
Be Proud! Be Responsible!51 HIV-education and skill-training program for African-American adolescents • African-American mixed-sex youth aged 13–18 years, community-based settings, less than 3 months, less than 10 hours		N				Y				• Overall positive impact on condom use. Positive impact on consistent condom use, proportion of condom-protected intercourse, frequency of condom use, and condom use at last sex. • No impact on frequency of sex in the past 3 months.
Becoming a Responsible Teen (BART)39 HIV-education and skill-training program • African-American mixed-sex high school students, clinic setting, less than 3 months, 20 or more hours		Y	N			Y				• Positive impact on being sexually active. • Overall positive impact on condom use (across six measures). • No impact on number of sexual partners.
¡Cuídate!48 Program to reduce rates of sexual risk behaviors • Hispanic mixed-sex youth aged 13–18 years, school- and community-based settings, less than 3 months, less than 10 hours		Y	Y			M				• Positive impact on ever having sex and having multiple partners in the past 3 months. • Overall, mixed impact on condom use. Positive impact on consistent condom use and frequency of unprotected sex in past 3 months, but no impact on condom use at last sex or days of protected sex.
Draw the Line/Respect the Line29 STI- and pregnancy-prevention program for middle school students • Mixed race/ethnicity and sex, sixth-grade students, school setting, more than 6 months, 20 or more hours	M	N	N			N				• Mixed impact on initiation of sex; positive impact for males, no impact for females.
										• Overall, no impact on frequency of sex. No impact on the number of times had sex in the past 12 months for females, positive impact for males only at year 2 follow-up. • Mixed impact on had any sex in past 12 months; positive impact on males and no impact for females. • Overall, no impact on number of partners (positive impact at year 2 follow-up for males, no impact for females; no impact for either males or females at any other follow-up). • No impact on condom use.
Focus on Youth40,41 AIDS-prevention program for low-income African-American youth • African-American mixed-sex adolescents aged 9–15 years, community-based setting, less than 3 months, 10–19 hours						M	M			• Mixed impact on condom use at last sex; positive impact at 6-month follow-up, no impact at 12-month follow-up (18-month follow-up unreported). • Mixed impact on contraceptive use; positive impact on effectiveness of contraception at last sex; positive impact at 6- and 18-month follow-ups, no impact at 12-month follow-up. No impact on dual-method use.
Focus on Youth plus imPACT42 Two-part program involving the original FOY skill-building curriculum and an additional parental component • African-American mixed-sex adolescents aged 12–16 years, community-based and home settings, less than 3 months, 10–19 hours		M				M				• Mixed impact on condom use at last sex; positive impact at 6-month follow-up, no impact at 12-month follow-up. • Mixed impact on frequency of sex; positive impact at 6-month follow-up, no impact at 12-month follow-up.
Get Real37 Three-year comprehensive sex-education program to delay initiation for middle school students with a family component • Mixed-race/ethnicity and -sex middle school students, school settings, more than 6 months, 20 or more hours	Y									• Positive impact on becoming sexually active by the end of eighth grade.
HIV-infection prevention in Mexican schools30 Intervention in Mexican high schools based on UN program on HIV/AIDS • Hispanic mixed-sex youth, mean age 16.7 years at 1-year follow-up, school setting, more than 6 months, 20 or more hours						N	M			• Overall, mixed impact on use of emergency contraception. Positive impact on one variation of the program (which included an emergency-contraception promotion component), no impact on the other variation of the program (which did not promote emergency-contraception use). • No impact on condom use at first or last sex.
HIV prevention for adolescents in low-income housing developments38 HIV interventions targeted at the normative social and peer environments of at-risk adolescents • Mixed-race/ethnicity and -sex adolescents aged 12–17 years, community-based setting, more than 6 months, less than 10 hours	Y					Y				• Delayed positive impact on abstinence; no impact at short-term follow-up, positive impact at long-term follow-up. • Positive impact on condom use.
It’s Your Game: Keep it Real31 School-based HIV-, STI-, and pregnancy-prevention program that targets middle school students • African-American and Hispanic mixed-sex sixth-to eighth-grade students, school setting, more than 6 months, 10–19 hours	M	Y	N	Y	N	N	N			• Positive impact on oral sex, anal sex, and frequency of vaginal sex in past 3 months. • Mixed impact on initiation; no impact on initiation of vaginal sex, positive impact on initiation of any sex (oral, anal, vaginal). • No impact on lifetime or past 3-month number of sexual partners, sex under the influence, condom use at last sex, frequency of sex without a condom in past 3 months, and frequency of sex without contraception in past 3 months.
It’s Your Game: Keep it Real32,33 School-based HIV-, STI-, and pregnancy-prevention program that targets middle school students • Two treatment groups, African-American and Hispanic mixed-sex seventh- and eighth-grade students, school setting, more than 6 months, 20 or more hours	N	N	N	N		M				• Mixed impact on condom use; positive impact on unprotected last vaginal sex at first follow-up, positive at second follow-up for one treatment group. Positive impact on frequency of past 3-month unprotected vaginal sex only for one treatment group at first follow-up. Positive impact on frequency of past 3-month unprotected anal sex only at second follow-up for one treatment. • No impact on initiation, however, positive impact on initiation of vaginal sex or any sex (vaginal, anal, oral) for one treatment group at first follow-up. • Overall no impact on frequency of vaginal sex; however, positive impact for one treatment group at first follow-up. • No impact on lifetime sexual partners and negative impact on partners in the past 3 months (higher number of partners). • Overall, no impact on anal or oral sex; however, positive impact on initiation of anal sex at first follow-up and frequency of anal sex at both follow-ups for one group.
Postponing sexual involvement34 Middle school program that focuses on delaying sexual activity • African-American mixed-sex seventh-grade students, school setting, more than 6 months, less than 10 hours	M						M			• Mixed impact on initiation of sex; positive impact on females at the majority of follow-ups, no impact on males. • Mixed impact on using contraception at last sex; positive impact for females, no impact for males.
Promoting Health Among Teens (PHAT): comprehensive abstinence and safer sex intervention25 Program encouraging condom use to reduce risks in African-American youth • African-American mixed-sex sixth- and seventh-grade students, school setting, less than 3 months, less than 10 hours	N	N	Y			N				• Positive impact on having multiple partners in the past 3 months. • No impact on initiation of sex, frequency of sex in the past 3 months, consistent condom use or unprotected intercourse.
Safer Choices35,36 Sexuality-education program for high school students • Mixed-race/ethnicity and -sex ninth-grade students, school setting, more than 6 months, 20 or more hours	N	N	N		N	Y	Y			• Overall, positive impact on use of protection against pregnancy at last sex; positive impact at 7- and 31-month follow-ups, but no impact at 19-month follow-up. • Overall, positive impact on condom use across four measures; overall condom use was mixed at 7-month follow-up, did not work at 19-month follow-up, and worked at 31-month follow-up. • No impact on initiation, frequency of sex, number of partners, or sex under the influence in past 3 months.
SHARP: Sexual Health and Adolescent Risk Prevention43 Single-session HIV-prevention intervention for adolescentsin juvenile justice facilities • Two treatment groups, mixed-race/ethnicity and -sex adolescents, mean age 15.8 years, juvenile justice setting, less than 3 months, less than 10 hours					N	M				• Mixed impact on consistency of condom use; no impact at 3-month follow-up, positive impact at 6-month follow-up, positive impact at 9- and 12-month follow-ups only among one treatment group. • No impact on sex under the influence.
Sistering, Informing, Healing, Loving, and Empowering (SiHLE)44 HIV-prevention intervention for African-American female adolescents • African-American females aged 14–18 years, clinic- and community-based settings, less than 3 months, 10–19 hours			Y			Y		N	M	• Positive impact on having a new sex partner in the past 30 days. • Overall, positive impact on condom use across six measures. • Mixed impact on self-reported pregnancy; positive at 6-month follow-up, no impact at 12-month follow-up. • Overall, no impact on STI infections; positive impact on chlamydia, no impact on trichomonas or gonorrhea.
Sisters Saving Sisters45 HIV-prevention intervention for African-American female adolescents • African-American and Hispanic females aged 12–19 years, clinic setting, less than 3 months, 10–19 hours			Y		Y	Y		Y		• Delayed positive impact on number of partners and multiple partners, number of days of unprotected sex, and number of days of unprotected sex while under the influence in the past 3 months; no impact at 3- and 6-month follow-ups, but positive impact at 12-month follow-up. • Delayed positive impact on testing positive for an STI, no impact at 6-month follow-up, positive impact at 12-month follow-up.
Teen Talk46 Pregnancy-prevention program based on the health-belief model and on social learning theory • Mixed-race/ethnicity and -sex youth aged 13–19 years, school and clinic settings, less than 3 months, 10–19 hours	M						N			• Mixed impact on initiation of sex; positive impact for males, no impact for females. • No impact on effective contraceptive use at last sex.

Notes: Y, found to work: positive and statistically significant impacts on the majority of measures assessed within the outcome. M, mixed findings: varied impacts on the outcome; impact varied based on time, subgroup (when full sample analyses unreported), or on different measures. N, not found to work: nonsignificant, marginally significant, or negative impacts on the majority of measures assessed within the outcome.

Abbreviations: STIs, sexually transmitted infections; HIV, human immunodeficiency virus; AIDS, acquired immunodeficiency syndrome; FOY, Focus on Youth; UN, United Nations.

Of the 21 effective comprehensive sex-education evaluations (including both shorter- and longer-duration programs), 13 worked for at least one outcome, and eight had mixed findings based on varied impacts across measures, subpopulations, treatment groups, or follow-ups. As shown in Table 2, almost all of the effective comprehensive sex-education program evaluations measured impacts on both sexual activity and contraceptive use. Sixteen of these evaluations were effective for at least some measure of sexual activity, including six that were effective for delaying sexual initiation (including two with consistent impacts37,38 and four with mixed impacts),29,31,34,46 six for reducing sexual frequency (including four with consistent impacts31,39,47,48 and two with mixed impacts),42,49 five for reducing the number of sexual partners,25,44,45,48,49 three that were effective for decreasing anal or oral sex,31,49,50 and one for reducing sex under the influence.45 Another 14 comprehensive sex-education programs found impacts on condom use (including nine with consistent impacts35,36,38,39,44,45,47,49–51 and five with mixed impacts).32,33,40–43,48 Four programs were effective for increasing contraceptive use (including one with consistent impacts35,36 and three with mixed findings).30,34,40,41 In addition, one program was effective at reducing laboratory-tested STIs,45 and one found a mixed impact on pregnancies or births.44

Clinic-based programs

Clinic-based programs were designed for implementation in a clinic or were implemented by clinic staff or physicians. While several other types of programs, including comprehensive sex-education programs, incorporated a clinic component, clinic-based programs were generally designed exclusively for use in a clinic setting and targeted adolescents seeking clinical services. Overall, these programs were frequently found to be effective; nine clinic-based program evaluations (of 14 total) were effective for at least one outcome or population (Table 3; clinic-based programs that did not work are listed in Table S1). All of the effective clinic-based programs were implemented with youths aged 14–18 years, and most lasted less than 3 months or had less than 10 contact hours with participants (although one program incorporated an 18-month intervention that combined case management, peer leadership, and service learning).52,53 Notably, seven of these programs were implemented with a female-only population,52–59 one was implemented only with males,60 and one was with mixed sexes.61 Clinic-based programs frequently incorporated one-on-one sessions,52–54,56,57,60,61 sometimes in conjunction with interactive group sessions.52,53,56 A few clinic-based programs used only group sessions.55,58 One effective clinic-based program implemented a four-session video-only intervention,59 and two others incorporated videos in supplement to individual counseling.57,60

Table 3.

Clinic-based programs

Program	Sexual initiation	Frequency/recency of sex	Number of partners	Anal/oral sex	Sex under the influence	Condom use	Any contraception use	Contracting STIs	Pregnancies and births	Comments
Health-belief model intervention to increase condom use among high-risk female adolescents54 Clinic intervention based on the health-belief model for high-risk females • Mixed-race/ethnicity females aged 15–19 years, clinic setting, less than 3 months, less than 10 hours						M		N		• Positive impact on frequency of condom use, but no impact on condom use at last sex. • No impact on the incidence rate of infection or reinfection.
Health Improvement Project (HIP) for Teens58 Clinic-based prevention program targeting black female adolescents • African-American females aged 15–19 years, clinic setting, less than 3 months, 10–19 hours	N	Y	N			Y		N	Y	• Positive impact on reducing pregnancy, frequency of sex in the past 3 months, on frequency of unprotected sex (positive impact at 3- and 12-month follow-ups, no impact at 6-month follow-up). • No impact on STIs, remaining abstinent, or number of partners (however, abstinence and partners measures revealed a positive impact at 6-month follow-up, no impact at 3- or 12-month follow-ups).
Horizons55 Clinic-based STI/HIV-prevention program for sexually active African-American female adolescents • African-American females aged 15–21 years, clinic setting, less than 3 months, less than 10 hours						Y		N		• Positive impact on condom use in past 14 days, past 60 days, and at last sex. • Overall, no impact on STI rates; positive impact on chlamydia, no impact on trichomonas or gonorrhea.
Prime Time52,53 Clinic-based intervention for girls at high risk for pregnancy • Mixed-race/ethnicity females aged 13–17 years, clinic setting, more than 6 months, contact hours not reported		Y	N			Y	Y			• Positive impact on abstaining from sex in the past 6 months and condom use with the most recent partner. • Overall positive impact on contraception; positive impact on hormonal use at 12-month follow-up, no impact at 30-month follow-up. Positive impact on consistent dual-method use. • No impact on number of partners in the past 6 months.
Project IMAGE56 Clinic-based behavioral intervention for minority females with a history of abuse and STIs • African-American and Hispanic females aged 14–18 years, clinic setting, less than 3 months, less than 10 hours								Y		• Positive impact on STI infections.
Project RESPECT61 Clinic-based counseling program for teens and adults to prevent STIs and HIV through condom use • Mixed-race/ethnicity and -sex youth aged 14 years and older, clinic setting, less than 3 months, less than 10 hours			M			M		Y		• Mixed impact on reporting no casual partners and no new partners; positive impact at 3- and 6-month follow-ups, no impact at 9- or 12-month follow-ups. • Mixed impact on condom use; positive impact at 3- and 6-month follow-ups, no impact at 9- or 12-month follow-ups. • Positive impact on acquiring new STIs.
Project RESPECT57 Clinic-based counseling program for teens and adults to prevent STIs and HIV through condom use, incorporating a video • Mixed-race/ethnicity females aged 15–21 years, clinic setting, less than 3 months, less than 10 hours			N			M		N		• Mixed impact on condom use at last sex with main partner; positive impact at 3-month follow-up, no impact at 12-month follow-up. • No impact on number of casual partners or self-report STIs or positive chlamydia tests.
Reproductive health counseling for young men60 Reproductive health intervention combining a highly explicit half-hour slide-tape with a personal health consultation • White males aged 15–18 years, clinic setting, less than 3 months, less than 10 hours	N						Y			• Overall, positive impact on contraception; positive impact on use of pill at last sex, effectiveness of main method at last sex, and effectiveness of main method used in past year, no impact on methods used in past year or frequency of method use. • No impact on initiation of sex.
Seventeen Days (formerly What Could You Do?)59 Clinic-based interactive video intervention to increase young women’s ability to make less risky sexual health decisions • Mixed-race/ethnicity females aged 14–18 years, clinic setting, 3–6 months, less than 10 hours		M				N		M		• Mixed impact on frequency of sex; positive impact at 3-month follow-up, no impact at 6-month follow-up. • Mixed impact on STIs; positive impact on self-reported STI acquisition, no impact on clinically tested infection. • No impact on frequency of condom use in the past 3 months.

Notes: Y, found to work: positive and statistically significant impacts on the majority of measures assessed within the outcome. M, mixed findings: varied impacts on the outcome; impact varied based on time, subgroup (when full sample analyses unreported), or on different measures. N, not found to work: nonsignificant, marginally significant, or negative impacts on the majority of measures assessed within the outcome.

Abbreviations: STIs, sexually transmitted infections; HIV, human immunodeficiency virus.

Of the nine effective clinic-based program evaluations, six found consistent impacts for at least one outcome, and three had mixed findings. As shown in Table 3, four clinic-based program evaluations were effective for some measure of sexual activity; three were effective at reducing sexual frequency (including two that found consistent impacts52,53,58 and one that found a mixed impact across follow-ups),59 and one that found a mixed impact on reducing the number of partners (across follow-ups).61 None of these program evaluations found consistent impacts for sexual initiation, and none measured anal or oral sex or sex under the influence. Six program evaluations were effective at increasing condom use (including three with consistent findings52,53,55,58 and three that reported mixed findings, based on various measures of condom use in one study,54 and varying across follow-ups for two others).57,61 Two program evaluations were effective at increasing contraceptive use.52,53,60 In addition, three program evaluations were effective at reducing STIs (including two with consistent impacts56,61 and one with a mixed impact with a positive impact on self-reported STIs, but no impact on clinically tested infection),59 and one that was effective at reducing pregnancies or births.58

Youth-development programs

Youth-development programs focused on school achievement or health outcomes (many in combination with reproductive health outcomes) or were designed to increase prosocial behaviors, such as cooperation. Overall, about half of the evaluated youth-development programs were effective: eight (of 17 total) worked or found mixed impacts for at least one outcome (Table 4; Table S1 lists youth-development programs that were not found to work). All of these effective programs were longer in length, reporting a duration of more than 6 months or more than 20 contact hours with participants. Three programs – including two implementations of the Children’s Aid Society (CAS)–Carrera Program – were intensive, multiyear, and multicomponent interventions.62–64 Four programs, including two evaluations of the Teen Outreach Program, included service-learning components that combined community volunteering with classroom-based discussions.64–67 One program included individualized clinical services addressing life-planning, goal-setting, and coping skills, in addition to medical and reproductive health clinical services.68 Also, one program worked with fifth graders to promote social and character development and positive action.69 Most of the youth-development programs with impacts included some type of sex education in combination with other activities.

Table 4.

Youth-development programs

Program	Sexual initiation	Frequency/recency of sex	Number of partners	Anal/oral sex	Sex under the influence	Condom use	Any contraception use	Contracting STIs	Pregnancies and births	Comments
Children’s Aid Society (CAS)-Carrera62 Intensive, year-round, multiyear, and multicomponent after-school program designed to promote positive youth development and positive reproductive health • Mixed race/ethnicity and sex, aged 13–15 years, clinic- and community-based settings, more than 6 months, 20 or more hours	N					N	M		M	• Mixed impact on contraceptive use; positive impact on use of Depo-Provera at last sex, no impact on dual-method use at last sex. • Mixed impact on pregnancies and births; positive impact on becoming pregnant or causing a pregnancy, but no impact on two measures of births/becoming a father. • No impact on ever having sex and condom use at last sex.
Children’s Aid Society (CAS)-Carrera63 Intensive, year-round, multiyear, and multicomponent after-school program designed to promote positive youth development and positive reproductive health • Mixed race/ethnicity and sex, aged 13–15 years, clinic- and community-based settings, more than 6 months, 20 or more hours	Y	M				N	M		M	• Positive impact on sexual initiation for the full sample (however, subgroup analyses revealed a positive impact for females and no impact for males). • Mixed impact on being currently sexually active; positive impact for females, no impact for males (full sample unreported, only measured sexually experienced). • Mixed impact on dual-method use; positive impact for females, no impact for males (full sample unreported). • Mixed impact on pregnancies and births; mixed impact on pregnancy (positive for females, no impact for males), no impact on births for full sample (however, subanalyses revealed a positive impact on females, no impact on males). • No impact on condom use at last sex (only measured sexually experienced).
Positive Action Program69 School-based program focused on social and character development • Mixed-race/ethnicity and -sex fifth-grade students, school setting, more than 6 months, 20 or more hours	Y									• Positive impact on ever having sex.
Quantum Opportunities Program64 Intensive, multicomponent intervention for disadvantaged high school students • Mixed-race/ethnicity and -sex students entering ninth grade, school setting, more than 6 months, 20 or more hours									Y	• Positive impact on likelihood of having children for overall score across sites, but analyses by site did not reveal any significant findings.
Reach for Health Service Learning Program67 Intervention to help youth make positive health choices and avoid high-risk behaviors • Mixed-race/ethnicity and -sex seventh- to tenth-grade students, school setting, more than 6 months, 20 or more hours	Y	Y								• Positive impact on initiation of sex. • Positive impact on sex in the past 3 months.
Teen Outreach Program (TOP)66 Program to prevent problem behaviors by enhancing normative processes of social development • Mixed-race/ethnicity and -sex adolescents aged 11–21 years, school setting, more than 6 months, 20 or more hours							Y		Y	• Positive impact on regular use of contraception. • Positive impact on pregnancy.
Teen Outreach Program (TOP)65 Program to prevent problem behaviors by enhancing normative processes of social development • Mixed-race/ethnicity and -sex ninth-to 12th-grade students, school setting, more than 6 months, 20 or more hours									Y	• Positive impact on pregnancy.
Washington State client-centered pregnancy-prevention programs68 Seven “client-centered” pregnancy-prevention programs combining education and individualized services • Mixed-race/ethnicity and -sex youth aged 9–17 years, clinic setting, more than 6 months, contact hours varied by site	N	Y					N			• Positive impact on sex in the past month. • No impact on sexual initiation or on always sing contraception, contraception use in the last month, contraception use at last sex.

Notes: Y, found to work: positive and statistically significant impacts on the majority of measures assessed within the outcome. M, mixed findings: varied impacts on the outcome; impact varied based on time, subgroup (when full sample analyses unreported), or on different measures. N, not found to work: nonsignificant, marginally significant, or negative impacts on the majority of measures assessed within the outcome.

Abbreviation: STIs, sexually transmitted infections.

Of the eight effective youth-development program evaluations, seven worked for at least one outcome, and one had mixed findings. As shown in Table 4, four youth-development program evaluations were effective at reducing some measure of sexual activity, including three that were effective at delaying sexual initiation63,67,69 and three that were effective at reducing sexual frequency (including two with consistent impacts67,68 and one that found a mixed impact by sex).63 None of these program evaluations measured impacts on the number of partners, oral or anal sex, or sex under the influence. No youth-development program evaluations were effective at increasing condom use, but three were effective at increasing contraceptive use (including one with consistent impacts66 and two with a mixed impact, one by sex,63 and another with an impact on injectable contraceptive methods but not on dual-method use).62 In addition, no youth-development program evaluations measured STIs, but three found consistent impacts on pregnancies or births,64–66 and two had mixed findings.62,63

Parent–youth relationship programs

This category of programs aimed to improve parent–youth relationships, with a particular focus on communication about sexual behavior and romantic relationships. Overall, these programs were frequently effective. Nine parent–youth relationship-program evaluations (of eleven total) worked or had mixed findings for at least one outcome or population, and they are described in Table 5 (programs that did not work are included in Table S1). These parent–youth relationship programs varied in their implementation approaches. For example, three evaluations found behavioral impacts of the program Familias Unidas, a home-based sex-education intervention focused on increasing parental involvement in Hispanic families, including two that were implemented on their own with different age-groups70,71 and one that was implemented in combination with PATH (Parent-Preadolescent Training for HIV Prevention),72 an HIV-prevention curriculum to train parents to become effective HIV educators for their children. Another three parent–youth relationship programs that found impacts were implemented in clinic settings, including one that focused on parents with acquired immunodeficiency syndrome,73 one with group sessions for mothers and additional youth modules and parent–child homework assignments,74 and a third that was implemented with a sample of divorced mothers and their children.75 Another two programs were implemented in community settings, including a father–son HIV-prevention program focused on African-American families76 and an HIV-prevention program for African-American mothers and their children.77 The final parent–youth relationship program was for homeless and runaway teens and their parents, and was implemented in the setting of their choosing.78 Parent–youth relationship programs were generally implemented with younger youth: six of the nine programs were implemented primarily with youth aged 13 years or younger,70,72,74–77 and three were implemented with a broader age range.71,73,78

Of the nine effective parent–youth relationship-program evaluations, eight found consistent impacts for at least one outcome, and one had mixed findings. As shown in Table 5, four of these program evaluations were effective for reducing some measure of sexual activity, including one that found consistent impacts on delaying sexual initiation,74 one that was effective at reducing sexual frequency,74 three that were effective at reducing the number of partners (including two that found consistent impacts71,78 and one that found a mixed impact across program implementations),75 and one that found impacts on sex under the influence71 (no parent–youth programs were effective for reducing anal or oral sex). Another five parent–youth relationship-program evaluations were effective at increasing condom use,70–72,76,77 but no evaluations measured impacts on contraceptive use. In addition, one effective parent–youth relationship-program evaluation found consistent impacts for self-reported STIs,72 and another reduced pregnancies or births.73

Programs with impacts on key outcomes

This section highlights programs that were effective at improving key sexual and reproductive health outcomes, including those that were effective at reducing teen pregnancies, births, or STIs. We also describe programs that found impacts on both sexual and contraceptive or condom-use behaviors.

Programs that were effective at reducing teen pregnancies, births, or STIs

Our review highlights several program evaluations with impacts on teen pregnancies, births, or STIs. Of the 33 program evaluations that measured pregnancies or births, nine found an impact (including six that found consistent impacts28,58,64–66,73 and three with mixed findings across follow-ups).44,62,63 These nine program evaluations represent all program approaches: one was abstinence education,28 one was comprehensive sex education,44 one was clinic-based approaches,58 five were youth development,62–66 and one was a parent–youth relationship program. Three of these programs were implemented only with females,28,44,58 and six with mixed sexes.62–66,73 Most were implemented in schools (four)28,64–66 or clinics (two),58,73 and three were implemented in both a clinic and a community-based organization.44,62,63 While it appears that youth-development approaches are especially likely to be effective at reducing teen pregnancies, given the diversity of effective approaches, it is possible that the effective programs share one or more subtle similarities not visible in evaluation reports.

Of the 23 program evaluations that measured STIs, five were effective, including four that found consistent impacts45,56,61,72 and one that had mixed findings.59 Most of the programs with impacts on STIs were based on testing (three),45,56,61 although one was based on self-report,72 and one used both.59 Of these five effective programs, three were clinic-based interventions,56,59,61 one was comprehensive sex education,45 and one was a parent–youth relationship program.72 Three were implemented with only females,45,56,59 and two with mixed sexes.61,72 Four of these programs were implemented in a clinic setting,45,56,59,61 and one was implemented primarily in the home.72 Clearly more work is needed to develop effective STI-prevention strategies.

Programs with impacts on sexual activity and contraceptive use

Many more program evaluations measured impacts on key sexual or contraceptive-use behavioral determinants of teen pregnancies or STIs than of teen births or STIs. Programs that have impacts on both sexual activity and some measure of contraceptive use (including condom use) may be particularly effective at preventing early pregnancies and/or STIs. We examined which programs measured and had an impact on both a sexual activity outcome and a contraception outcome. Our review identified 72 program evaluations that measured both sexual activity and contraceptive or condom use; of these, 35 had consistent impacts or mixed findings on either sexual activity, contraception, or both. Of these, 15 program evaluations showed effectiveness at reducing at least one sexual activity outcome and at increasing at least one condom- or contraceptive-use outcome (including ten that had consistent impacts for both categories of outcomes38,39,44,45,47,49,50,52,53,58,71 and five that were mixed for one or both outcomes).34,42,48,61,63 Interestingly, two of these 15 programs were also effective at reducing pregnancies or births (including one with consistent impacts58 and one that had mixed findings across follow-ups),44 and two found consistent impacts on STIs.45,61 The majority (ten) of these 15 programs were comprehensive sex-education programs,34,38,39,42,44,45,47–50 but there were also three clinic-based programs,52,53,58,61 one parent–youth relationship program,71 and one youth-development program.63 These programs were implemented in school,34,48–50 community,39,44,45,52,53,58,61,63 clinic,38,42,44,48,63 home,42,71 and juvenile drug-center settings,47 and with a variety of age-groups. They were mostly shorter in duration, with eleven lasting less than 6 months and four longer than 6 months. Ten of these 15 effective programs were implemented with both sexes, one was male only, and four were female only. Again, we need to learn more about the less transparent elements of programs to understand the common elements that result in effectiveness across such a diverse body of interventions.

Discussion

We identified more than 100 rigorous random-assignment evaluations of sexual and reproductive health programs for this study, including a variety of program approaches, settings, and target populations. Two of the five programs approaches that we reviewed – parent–youth relationship programs and clinic-based approaches – were particularly effective at influencing the sexual and reproductive health outcomes that they targeted among teens. For example, more than three-quarters (nine of eleven) evaluated parent–teen relationship programs were effective for at least one outcome or population (described as “found to work” or “mixed”). In addition, several other effective programs incorporated parent-involvement components. These findings reflect the important role that parent–teen relationships, parental monitoring, and parent–teen communication play in influencing adolescent sexual and reproductive health behaviors.79

As a whole, clinic-based program approaches were also effective. Almost two-thirds of these program evaluations (nine of 14 reviewed program evaluations) were effective for at least one outcome or population. Several of these programs demonstrate the effectiveness of incorporating one-on-one components (often in combination with group-based or video sessions), particularly for increasing contraceptive use. Future research should assess the role of clinic-based approaches for reaching older teens, especially because more than two-thirds of teen births occur to women aged 18–19 years, most of whom are not connected to schools.

Although somewhat less consistently effective than the two aforementioned types of programs, about half of the youth-development programs were identified as effective. Those that were effective highlight the potential role of community service-learning approaches and often multicomponent programs that focus not only on adolescent reproductive health but other measures of well-being, including educational outcomes.80

Finally, though less effective than other types of programs, this review also identified a number of abstinence- and comprehensive sex-education programs that had impacts on sexual and reproductive health behaviors or outcomes. About a third of abstinence-based program evaluations found behavioral impacts (five of 14 studies reviewed received a “worked” or “mixed” categorization). Although low, this represents an increase from previous studies that documented limited or no effective abstinence-education programs.81,82 Not surprisingly, the effective abstinence-education programs were most likely to impact sexual initiation or activity.24–26,28 However, our review also identified other programs that were effective at delaying sexual initiation (ten total), including six comprehensive sex-education programs (that often focus on both abstinence and condom use or contraceptive use),29,31,34,37,38,46 three youth-development programs,63,67,69 and one parent–child relationship program.74 Notably, although many more effective comprehensive sex-education programs were identified than abstinence-only programs, their rate of effectiveness (21 of 47 comprehensive sex-education evaluations reviewed) was similar to that of abstinence-education programs. These findings highlight the need to identify key implementation components that are particularly effective for improving sexual and reproductive health outcomes across program approaches. In fact, many of the programs that were not effective (listed in Table S1) included components that were similar to the effective programs (in Tables 1–5), which suggests that program implementation may be as important as program content for improving outcomes.

Some research has identified important characteristics of program implementation that are particularly effective for sexual and reproductive health programs. Kirby82 identified several key characteristics of effective curriculum teen-pregnancy prevention programs, including clearly identifying health goals and the behavioral and psychosocial risk and protective factors leading to those goals during the program-development stage; incorporating activities, instructional methods and messages that are relevant to the youth’s culture, age, and sexual behavior; securing community support for the program; effectively training, monitoring, and supporting facilitators; and implementing programs with fidelity.

Recommendations for the future

Although this review identified 103 rigorous evaluations of sexual and reproductive health programs, future research and implementation evaluations can expand the evidence base in this area. First, replication is needed. Ideally, programs would impact behavior across replications in order to be designated as evidence-based; however, relatively few programs have been replicated and tested. Fifteen of the programs included in this review were evaluated more than once (this includes five evaluations of the comprehensive sex education program Be Proud! Be Responsible!). However, many of these were implemented with a different target population, setting, or location, so they did not necessarily confirm or contradict earlier findings. Programs that directly impact teen pregnancies and STIs – and their key determinants (including those with impacts on both sexual activity and contraceptive use) – are particularly important for further replications. Our review particularly highlights these programs.

Second, programs need to be updated. Several of the evaluations were of programs that were designed and implemented more than a decade ago. As a result, the activities and messages may be somewhat outdated for current populations of teens,22 and may require on-the-ground adaptations to make them relevant for race and ethnic minority populations.21 Third, we need a better understanding of how effective programs work. Since many of the effective programs had components similar to those with no impacts, future research can help identify what aspects of programs – such as implementation quality or staffing – are linked to stronger program impacts. Fourth, rigorous evaluation of promising programs is needed, including those that have been evaluated with quasiexperimental approaches and pre/posttest studies. Of particular interest are programs with impacts on contraceptive outcomes and those that are effective with older teens, who have the highest rates of teen pregnancy and STIs. Practitioner groups have highlighted the importance of hormonal and long-acting methods for pregnancy prevention and dual-method use for preventing pregnancy and STIs, but we found very few evaluations of these outcomes. Finally, although we include strong review criteria (requiring a random-assignment evaluation design with an intent-to-treat approach), we did not rate programs based on the quality of the evaluations. Future research should consider characteristics of evaluation quality, including baseline equivalence and attrition.20

Conclusion

This review examined rigorous evaluations of sexual and reproductive health programs, and identified programs with impacts using a variety of approaches. Current federal initiatives to scale up evidence-based programs can help increase the likelihood that high-risk youth will receive programming that will help them reduce high rates of pregnancy and STIs in the US. Meanwhile, new programs and approaches should be evaluated to continue to expand the evidence base into the future.

Supplementary material

Table S1.

Programs that did not work for any measured outcome

Program	Sexual initiation	Frequency/recency of sex	Number of partners	Anal/oral sex	Sex under the influence	Condom use	Any contraception use	Contracting STIs	Pregnancies and births	Comments
Abstinence-based/abstinence-focused programs
Afrocentric peer counseling1 Abstinence-focused program targeting African-American females • African-American females aged 12–16 years, community-based setting, less than 3 months, 10–19 hours		N					N		N	• No impact on past-month sexual activity, contraceptive use, or frequency of pregnancy
Families United to Prevent Teen Pregnancy (FUPTP)2,3 Abstinence-focused after-school program • Mixed-race/ethnicity and mixed-sex adolescents aged 8–13 years, school setting, more than 6 months, 20 or more hours	N	N	N			N	N	N	N	• No impact on rate of always remaining abstinent, age of first sex, rate of remaining abstinent in the past 12 months, number of sexual partners, unprotected first sex, frequency of condom use in past 12 months, use of birth control at first sex, frequency of birth control use in past 12 months, ever having an STI, having ever been pregnant, or ever having a baby.
Families United to Prevent Teen Pregnancy (FUPTP)4 Abstinence-focused after-school program • Mixed-race/ethnicity and mixed-sex mother/adolescent dyads, adolescents aged 11–14 years, clinic setting, 20 or more hours, duration varied based on wave	N									• No impact on initiation of sex.
Heritage Keepers Life Skills Education5 Character-based abstinence program • Mixed-race/ethnicity and -sex sixth-to 12th-grade students, school setting, more than 6 months, 20 or more hours	N		N			N	N	N	N	• No impact on remaining abstinent or age of first sex, number of partners, condom or contraception use at first sex or in past 12 months, ever having an STI, or pregnancy or birth rates.
Making a Difference!6 Abstinence-based approach to HIV/AIDS and teen-pregnancy prevention • African-American mixed-sex sixth- and seventh-grade students, school setting, less than 3 months, less than 10 hours		N				N				• Overall, no impact on sexual activity in the past 3 months; positive impact on percentage of participants who had sex at 3-month follow-up, but no impact at 6- or 12-month follow-ups. No impact on frequency of sex. • Overall, no impact on condom use; positive impact on frequency of condom use at 12-month follow-up, but no impact at 3- or 6-month follow-ups. No impact on consistent condom use, percentage reporting unprotected sex, or frequency of unprotected sex.
My Choice, My Future!2,3 Three-year abstinence-focused program • White mixed-sex eighth- to tenth-grade students, school setting, more than 6 months, 20 or more hours	N	N	N			N	N	N	N	• No impact on sexual activity, condom use, contraception use, reporting STIs, or ever being pregnant or having a birth.
Project Taking Charge7 Abstinence-focused sex and vocational education program for seventh-grade students in high pregnancy-risk areas • Mixed-race/ethnicity and -sex seventh-grade students, school setting, less than 3 months, contact hours not reported	N									• No impact on initiation.
ReCapturing the Vision2,3 Abstinence-based 1-year program for middle school females • Mixed-race/ethnicity females in seventh and eighth grade, school setting, more than 6 months, 20 or more hours	N	N	N			N	N	N	N	• No impact on rate of always remaining abstinent, age of first sex, rate of remaining abstinent in the past 12 months, number of partners, unprotected first sex, frequency of condom use in the past 12 months, use of contraception at first sex, frequency of contraceptive use in the past 12 months, having a reported STI, ever being pregnant, or ever having a baby.
Teens in Control3 Two-year abstinence-education program designed for fifth and sixth-grade students • Mixed-race/ethnicity and -sex fifth-grade students, school setting, more than 6 months, 20 or more hours	N	N	N			N	N	N	N	• No impact on rate of always remaining abstinent, age of first sex, rate of remaining abstinent in the past 12 months, number of sexual partners, unprotected first sex, frequency of condom use in the past 12 months, birth control use at first sex, frequency of birth control use in past 12 months, ever having a reported STI, ever being pregnant, or ever having a baby.
Clinic-based programs
African-American STI/HIV education8 STI/AIDS clinic intervention to influence male condom use • African-American males aged 15–19 years, clinic setting, less than 3 months, less than 10 hours		N	N			N				• No impact on frequency of sex with steady or casual partner, number of partners, or condom use in the past month.
AIDS education and counseling in an office setting9 Physician-delivered HIV-counseling program • African-American mixed-sex youth, mean age 17.6 years, clinic setting, less than 3 months, less than 10 hours		N	N			N		N		• No impact on frequency of sex, number of partners, condom use, or newly diagnosed STIs.
ASSESS10 Physician-administered safe-sex education program • Mixed-race/ethnicity and -sex youth aged 12–16 years, clinic setting, less than 3 months, less than 10 hours		N	N			N		N	N	• No impact on frequency of sex in the past 3 months, lifetime or past 3-month partners, pregnancy, or doctor diagnosed STI. • Overall, no impact on condom use at last sex; positive impact on condom use at 3-month follow-up, no impact at posttest or 6-month follow-up.
Culturally appropriate STI/AIDS education in a clinic setting8 STI/AIDS-education interventions in a clinic • African-American males aged 15–19 years, clinic setting, less than 3 months, less than 10 hours		N	N			N				• No impact on frequency of sex with steady and casual partners, or on condom use with steady or casual partners. • Negative impact on number of partners (increased sexual partners).
Safer-sex intervention for high-risk adolescent girls11 Individualized, clinic-based intervention for girls who are diagnosed with an STI • Mixed-race/ethnicity females aged 13–22 years, clinic setting, less than 3 months, less than 10 hours		N	N			N		N		• No impact on being sexually active, condom use, or having an STI since enrollment. • No impact on number of partners at 1-, 3-, or 12-month follow-ups, positive impact at 6-month follow-up.
Comprehensive sex-education programs
AIDS-preventive intervention12 Didactic and a discussion-based AIDS-prevention program • Two treatment groups, mixed-race/ethnicity and -sex, aged 12–18 years, residential home settings, less than 3 months, 10–19 hours					N	N				• No impact on condom use for vaginal or anal sex. • No impact on sex under the influence of alcohol or drugs.
All4You!13 Program for students in alternative high schools with classroom curriculum and service-learning activities • Mixed-race/ethnicity and -sex high school students, school setting, less than 3 months, 20 or more hours	N	N	N			N	N		N	• No impact on initiation of sex, number of sexual partners, use of contraception at last sex, or pregnancy since baseline. • No overall impact on condom use or frequency of sex; however, positive impact on frequency of unprotected sex, condom use at last sex, and frequency of sex in the past 3 months at 6-month follow-up, no impact at 12- or 18-month follow-ups.
All4You2!14 Program for students in alternative high schools with classroom curriculum and service-learning activities • Mixed-race/ethnicity and -sex high school students, school setting, duration not reported, 20 or more hours	N	N				N	N			• No impact on initiation of sex, frequency of sex in the past 3 months, frequency of unprotected sex, number of unprotected sex partners, condom use at last sex, and use of contraception at last sex.
ARREST15 Program to reduce the risk of AIDS among inner-city, minority adolescents • Mixed-race/ethnicity and-sex youth aged 12–16 years, community-based setting, less than 3 months, less than 10 hours		N	N			N				• No impact on number of sexual encounters, number of partners, or condom use.
Be Proud! Be Responsible!16 HIV-education and skill-training program for African-American adolescents • Mixed-race/ethnicity and -sex adolescents in five cities, mean age across five sites was 14–16.5 years, community-based settings, including detention centers, less than 3 months, less than 10 hours		N				N				• No impact on frequency of sex, having any sex, or frequency of unprotected sex in the past 30 days.
Be Proud! Be Responsible! Suburban replication17 HIV-prevention program originally developed for Black urban male youth replicated with a diverse population in a suburban setting • Mixed-race/ethnicity and -sex, tenth-grade students, school setting, less than 3 months, less than 10 hours	N	N			N	N				• No impact on ever having sex, sex in the past 3 months, consistent condom use, frequency of unprotected sex, or sex under the influence of drugs or alcohol.
Circle of Life18 HIV-preventive intervention developed for American-Indian and Alaska Native middle school youth • American-Indian/Alaska Native mixed-sex middle school students, school setting, less than 3 months, 20 or more hours	N	N				N				• No impact on ever having sex in the past 12 months or condom use at last sex. • Overall, no impact on initiation of sex for the full group and by age; however, separate analyses suggest that those who began the program at an earlier age had lower transitions to sexual initiation than those who began the program at later ages.
FOCUS19 Cognitive behavioral intervention to prevent STIs and unplanned pregnancies for female Marine recruits during training • Mixed-race/ethnicity females aged 17–22 years, military setting, 3–6 months, less than 10 hours			N			N		N	N	• No impact on inconsistent condom use or unintended pregnancies. • Overall, no impact on having multiple partners; positive impact for one subgroup, no impact for three other subgroups. • Overall, no impact on STIs, positive impact for one subgroup, no impact for three other subgroups.
Focus on Youth20 AIDS prevention program for low-income African-American youth • White mixed-sex adolescents aged 12–16 years, school-and community-based settings, less than 3 months, 10–19 hours	N	N				N	N			• No impact on initiation of sex, frequency of sex in past 6 months, condom use at last sex, frequency of condom use in past 6 months, or dual-method use at last sex.
Focus on Youth plus imPACT21 Two-part program involving the original FOY skill-building curriculum and an additional parental component • African-American mixed-sex adolescents aged 13–16 years, community-based and home settings, more than 6 months, 10–19 hours		N		N		N	N		N	• No impact on sex in the past 6 months, condom use at last sex, use of birth control at last sex, or if participants had been pregnant or gotten a girl pregnant in the past 6 months.
Healthy for Life Project (HFL)22 Program to prevent risky behavior, including risky sexual behavior White mixed-sex sixth- to eighth-grade students, school setting, more than 6 months, 20 or more hours	N	N				N				• No impact on ever having sex, frequency of sex in the past month, or consistent condom use.
Information-based HIV/STI risk-reduction intervention for adolescent girls23 Comprehensive program to reduce sexual risk among sexually experienced Hispanic and African-American females • Hispanic and African-American females aged 12–19 years, clinic setting, less than 3 months, 10–19 hours			N		N	N		N		• No impact on number of sexual partners in the past 3 months, percentage reporting multiple partners in the past 3 months, sex while under the influence, number of days of unprotected sex in the past 3 months, or on the percentage testing positive for an STI.
Making Proud Choices!24 Safer-sex approach to HIV/AIDS and teen-pregnancy prevention • African-American mixed-sex youth aged 10–15 years, school setting, less than 3 months, less than 10 hours	N	N	N			N				• No impact on initiation of sex, sex in the past 3 months, having multiple partners in the past 3 months, consistent condom use, or any unprotected intercourse.
Making Proud Choices!6 Safer-sex approach to HIV/AIDS and teen-pregnancy prevention • African-American mixed-sex sixth- and seventh-grade students, school setting, less than 3 months, less than 10 hours		N				N				• No impact on any sex or frequency of sex in the past 3 months. • Overall, no impact on condom use; positive impact at 3-month follow-up across four measures, but no impact at 6- and 12-month follow-ups.
The McMaster Teen Program25 Pregnancy-prevention program for middle school students • Mixed-race/ethnicity and -sex seventh-and eighth-grade students, school setting, less than 3 months, 10–19 hours	N						N		N	• No impact on sexual initiation for females, negative impact on males. • No impact on consistency of contraceptive use or if females have ever been pregnant.
Parents and Teens for Health (PATH)26 Group-based program to teach youth affect-management skills to reduce HIV risk • Mixed-race/ethnicity males aged 12–18 years juvenile drug court setting, less than 3 months, 10–19 hours	N	N	N		N	N				• No impact on lifetime sexual activity, sexual activity in the past 90 days, number of partners in past 90 days, sex under the influence, or condom use at last sex.
Peer-Led Sex Education (RIPPLE study)27,28 School-based intervention with older high school studentsleading class sessions on sex education for younger students • White mixed-sex adolescents aged 13–14 years, school setting, less than 3 months, less than 10 hours	N					N	N	N	N	• No impact on unprotected first sex, contraceptive use at first or last sex, or self-reported STIs. • Overall, no impact on initiation of sex; no impact on initiation by age 18 years, positive impact on initiation by age 16 years for females only. • Overall, no impact on pregnancies and births; no impact on unintended pregnancies or live births, but a positive impact on self-reported pregnancy by age 18 years.
Postponing Sexual Involvement29 Middle school program that focuses on delaying sexual activity • Mixed-race/ethnicity and -sex seventh- and eighth-grade students, school- and community-based settings, duration not reported, less than 10 hours	N	N	N			N	N	N	N	• No impact on initiation, frequency, or number of partners ever. No impact on condom use or oral contraceptive use. No impact on ever had an STI. • No impact on pregnancy in adult-led intervention groups; negative impact in youth-led groups (higher pregnancy).
Project iMPPACS30 Mass-media program designed to enhance HIV-preventive beliefs and behavior of high-risk black adolescents • African-American mixed-sex adolescents aged 14–17 years, television and radio ads setting, more than 6 months, less than 10 hours			N			N				• No impact on number of partners in the past 3 months, no impact on unprotected sexual contact for the full sample (subgroup analyses revealed a positive impact on older adolescents).
Project SNAPP31 AIDS- and pregnancy-prevention program for middle school students • Mixed-race/ethnicity and -sex seventh-grade students, school setting, less than 3 months, less than 10 hours	N	N	N		N	N	N	N	N	• No impact on sexual initiation, frequency of sex in the past 3 months, number of partners in past 12 months, under the influence at last sex, condom use at last sex, reported STIs, reported pregnancy, or sex under the influence. • No impact on birth-control pill use at last sex at 5-month follow-up, negative impact (lower pill use) at 17-month follow-up
Safer Choices32 Sexuality-education program for high school students • Mixed-race/ethnicity and -sex ninth- to 12th-grade students, school setting, more than 6 months, 20 or more hoursNote: Same implementation as above version of Safer Choices, but entire high school, regardless of participation in program, was evaluated in this version	N	N	N		N	N	N			• No impact on sexual initiation, frequency in the past 3 months, number of partners in the past 3 months, sex under the influence, or use of contraception. • Overall, no impact on condom use; however, positive impact on frequency of unprotected sex at 19-month follow-up and number of unprotected partners at 31-month follow-up.
Skills-based intervention on condom use33 Program to increase condom use among teens at high risk of getting STIs • Mixed-race/ethnicity and -sex adolescents aged 14–19 years, clinic and juvenile detention settings, less than 3 months, less than 10 hours			N			N				• No impact on number of partners in the past 3 months or condom use with a steady or casual partner.
Theory-driven intervention to improve contraceptive use34 Theory-driven classroom-based intervention • Mixed-race/ethnicity and -sex adolescents aged 14–19 years, school setting, less than 3 months, less than 10 hours							N			• No impact on use of contraception in the past 6 months.
Group psychosocial intervention plus group motivational enhancement therapy to prevent risky sexual behavior35 Intervention that combines a theory-based sexual risk reduction intervention with a group-based alcohol risk-reduction motivational enhancement therapy • Mixed-race/ethnicity and -sex detained youth, mean age 15.8 years, juvenile detention setting, less than 3 months, less than 10 hours					N					• No impact on sex while under the influence in the past 3 months.
Group psychosocial intervention to prevent risky sexual behavior35 Theory-based sexual risk-reduction intervention that aims to increase safe-sex practices through impacting condom-use attitudes, perceived norms of condom use, and self-efficacy • Mixed-race/ethnicity and -sex detained youth, mean age 15.8 years, juvenile detention setting, less than 3 months, less than 10 hours					N					• No impact on sex while under the influence in the past 3 months.
Group psychosocial intervention to prevent risky sexual behavior35 Theory-based sexual risk-reduction intervention that aims to increase safe-sex practices through impacting condom-use attitudes, perceived norms of condom use, and self-efficacy • Mixed-race/ethnicity and -sex detained youth, mean age 15.8 years, juvenile detention setting, less than 3 months, less than 10 hours					N					• No impact on sex while under the influence in the past 3 months.
Youth AIDS Prevention Project (YAPP)36,37 School-based prevention program designed to prevent HIV/AIDS and STIs among seventh and eighth graders • Mixed-race/ethnicity and -sex seventh- and eighth-grade students, school setting, more than 6 months, 10–19 hours	N	N	N			N	N			• No impact on initiation, frequency of sex, mean number of partners in the past 12 months, ever using condoms, or ever using condoms with foam.
Parent–youth relationship programs
Enhanced Managing Pressures Before Marriage Program (MPM)38 Series of homework assignments for middle school students to complete with their parents • White, mixed-sex eighth-grade students, school setting, less than 3 months, less than 10 hours	N	N								• No impact on initiation of sex or frequency of sex in the past 3 months.
Staying Connected with Your Teen39 Family-based program to prevent maladaptive behaviors in youth • Two treatment groups, African-American and white mixed sex eighth-grade students, school and other setting, less than 3 months, 10–19 hours	N									• Overall, no impact on initiation of sex. Positive impact in the parent-administered treatment group among African-Americans only; no impact among white participants. No impact on initiation in the self-administered treatment group on either population.
Youth-development programs
Early intervention program40 Daily skill-building class for middle school students targeting youth development • Mixed race/ethnicity and sex, sixth- to eighth-grade students, school setting, more than 6 months, 20 or more hours									N	• Negative impact (higher pregnancy) on pregnancy at year 2 follow-up, no impact at year 3 follow-up.
Flowers with Care40 GED program with an intensive counseling component for youth development • Mixed race/ethnicity and mixed sex, ninth- to 12th-grade students, school setting, more than 6 months, 20 or more hours									N	• No impact on pregnancy in the previous year.
Job Corps41,42 Social development program for disadvantaged youth and young adults • Mixed-race/ethnicity and -sex adolescents aged 16–24 years (majority 18 years or younger), community-based and residential settings, time in program varied by individual									N	• No impact on births.
JOBSTART43,44 Program for high school dropouts targeting social development • Mixed-race/ethnicity and -sex adolescents aged 17–21 years, school- and community-based settings, more than 6 months, 20 or more hours									N	• No impact on pregnancy.
Quantum Opportunities Program45–47 Intensive, multicomponent intervention for disadvantaged high school students • Mixed-race/ethnicity and -sex ninth-grade students, school setting, more than 6 months, 20 or more hours	N					N			N	• No impact on sexual initiation, condom use at last sex, or having first child before age 18 years.
Student Training and Reentry (STAR)40 Skill-building program with counseling and social service components • Mixed-race/ethnicity and -sex ninth- to 12th-grade students, school setting, more than 6 months, 20 or more hours									N	• No impact on pregnancy in the previous year.
Summer Training and Education Program (STEP)48,49 Program to reduce academic loss over the summer and reduce teen pregnancy and school dropout • Mixed-race/ethnicity and -sex youth aged 14–15 years, school- and community-based setting, more than 6 months, 20 or more hours							N		N	• No impact on contraceptive use or pregnancy rates. • No impact on sexual activity (no definition provided).
Twelve Together40 Peer-support and mentoring program offering weekly after-school discussion groups led by trained adult volunteers • Mixed-race/ethnicity and -sex youth, mean age 14 years, school setting, more than 6 months, 20 or more hours									N	• No impact on female pregnancy in the past year.
Up with Literacy40 In-class and after-school tutoring program for youth with low standardized test scores • Mixed-race/ethnicity and -sex sixth- to eighth-grade students, school setting, more than 6 months, 20 or more hours									N	• No impact on pregnancy in the previous year.

Notes: Y, found to work: positive and statistically significant impacts on the majority of measures assessed within the outcome. M, mixed findings: varied impacts on the outcome; impact varied based on time, subgroup (when full sample analyses unreported), or on different measures. N, not found to work: nonsignificant, marginally significant, or negative impacts on the majority of measures assessed within the outcome.

Abbreviations: STIs, sexually transmitted infections; HIV, human immunodeficiency virus; FOY, Focus on Youth; AIDS, acquired immunodeficiency syndrome; GED, General Educational Development.