Building an Evidence Base to Inform Interventions for Pregnant and Parenting Adolescents: A Call for Rigorous Evaluation

Abstract

Adolescent parents and their children are at increased risk for adverse short- and long-term health and social outcomes. Effective interventions are needed to support these young families.

We studied the evidence base and found a dearth of rigorously evaluated programs. Strategies from successful interventions are needed to inform both intervention design and policies affecting these adolescents. The lack of rigorous evaluations may be attributable to inadequate emphasis on and sufficient funding for evaluation, as well as to challenges encountered by program evaluators working with this population.

More rigorous program evaluations are urgently needed to provide scientifically sound guidance for programming and policy decisions. Evaluation lessons learned have implications for other vulnerable populations.

THE UNITED STATES HAS THE highest adolescent birth rate in the industrialized world.^1–5 Pregnancies and resulting births to adolescent mothers create serious public health challenges with both short- and longer-term health and social consequences. Although recent data suggest that birth rates for US adolescents aged 15 to 19 years have declined for the past 3 years, hitting a record low of 34.3 births per 1000 adolescents in 2010,⁶ focused efforts are still needed to provide pregnant and parenting adolescents with appropriate, high-quality services that seek to minimize adverse outcomes for adolescents and their children. Unfortunately, empirical evidence to inform interventions for pregnant and parenting adolescents is limited because relatively few rigorous program evaluations and replications have been conducted. The current socioeconomic environment, marked by increasingly limited budgets for social and public health programs, underscores the need for empirical evidence to inform programmatic decision-making and for increased efficiency in evaluations that are conducted.

We have provided (1) a brief overview of the consequences of adolescent pregnancy and parenting, (2) an examination of rigorous evaluations of programs for adolescent parents in the recent literature, and (3) recommendations for conducting more rigorous evaluations to address the urgent need for scientifically sound information to guide programming and policy decisions.

CONSEQUENCES

Adolescent pregnancy and childbearing have consequences at multiple levels. From a national perspective, adolescent childbearing in the United States is estimated to cost taxpayers almost $11 billion each year.⁷ The average annual taxpayer cost associated with a child born to an adolescent mother is estimated to be $1647. Many of these costs are associated with negative consequences for the children of adolescent parents, such as the expense of foster care, more-than-usual health care, and decreased tax revenue.⁷

The social, educational, and economic consequences of childbearing to the adolescent mothers themselves can be stark, although researchers disagree about the types and extent of negative effects related to adolescent parenting.^8,9 From a health perspective, adolescent mothers have higher rates of poor obstetric and neonatal outcomes than do mothers older than 20 years, such as increased risks for preterm delivery, lower birth weights, and higher neonatal mortality rates.¹⁰ Nearly half of all adolescent mothers do not earn a high school diploma, and roughly a third never obtain their general equivalency diploma.^9,11 On average, adolescent mothers are estimated to depend on various forms of public assistance for about one third of their parenting years.⁹

As others have noted,^12,13 adolescent mothers continue to lead high-risk lifestyles after the birth of their children and are at risk for rapid repeat pregnancies (defined as a second pregnancy within 24 months of the first pregnancy) that further exacerbate the risk for adverse outcomes, with almost one quarter having a second child before the age of 20 years.^8,12 Such closely spaced births further exacerbate negative consequences for these young families. Adolescent mothers with repeat pregnancies are even less likely to receive prenatal care, complete school, and maintain economic self-sufficiency and are more likely to experience preterm delivery, receive welfare, and have children with emotional and behavioral problems.⁸ Adolescent fathers also face educational and economic consequences, such as fewer years of schooling, lower odds of earning a high school diploma,¹⁴ and lower lifetime earnings than men whose children were born to mothers aged 20 years or older.¹⁵

Although the consequences of adolescent childbearing occur at multiple levels, research suggests that it is the children of adolescent parents who are most adversely affected.^15,16 The consequences for these children are apparent from birth and continue throughout their lives, even after adjustment for other socioeconomic factors. Babies of adolescent mothers are more likely to be underweight, to be premature, and to have lower health assessments and Apgar scores.^10,16–18 Cognitive and academic differences from children of older mothers are apparent; deficits demonstrated among kindergarteners¹⁸ continue into adolescence.^16,19 The tendency of children of adolescent parents to have poorer social and academic outcomes also affects their longer-term economic productivity.^9,20 Children of adolescent parents are more likely to be subjected to abuse and neglect and are more likely to enter the foster care system.²¹ They are themselves more likely to become adolescent parents,²² thus creating a multigenerational cycle of increased risk for adverse outcomes. A particularly noteworthy adverse outcome for sons of adolescent parents is their increased risk for incarceration. Extending earlier work, data indicate that delaying the mother's age at first birth would decrease the probability of her son's incarceration by almost 33% and his years in jail by 38%.^23,24

Although some analyses suggest that preexisting variables often associated with adolescent childbearing (e.g., low socioeconomic status and education levels) may account for some of these long-term adverse outcomes,^9,25,26 the results are mixed. Limitations related to data and study design affect the ability to determine causality because adolescent childbearing occurs within a complex, multidimensional environment. Social science will likely never be able to prove the specific causality of the consequences of adolescent childbearing. Nonetheless, it is generally agreed that adolescent parents are at increased risk and in need of intervention.

For many underserved adolescents, the failure of protective systems to intervene during their childhoods means that pregnancy may be their first opportunity to enter into a comprehensive system of supportive care that can address their multiple needs and risk factors across 3 major domains: health care, education, and social services. If this comprehensive and coordinated system responds appropriately, the pregnancy and associated care can provide adolescents with assistance with the potential to prevent myriad adverse outcomes. Sound and effective services from each of these system domains working in concert could break the multigenerational cycle of adolescent parenthood. By capitalizing on this transitional period in a young family's life, programs and the broader society have the opportunity to positively affect the life trajectory of all its members. However, these various system domains are often fragmented, or “siloed,” and difficult to navigate, particularly for an inexperienced adolescent parent. To mitigate the costs and consequences associated with adolescent pregnancy and childbearing for these parents, their children, and society as a whole, evidence-based interventions are needed to provide pregnant and parenting adolescents with effective services delivered in cost-effective ways.

EVALUATION CHALLENGES

Although a variety of strategies (e.g., educational parenting curricula, case management, home visitation, and clinic-based approaches) have been implemented alone or in various combinations to facilitate knowledge acquisition, behavior change, and access to resources for adolescent parents and their babies, little consensus exists regarding which strategies are most effective in helping adolescent mothers delay future pregnancies, complete their educations, and achieve their employment goals.²⁷ Program developers have tended to rely on a “black box” approach, in which multiple strategies are combined in a comprehensive program without the use of an evidence-based theory of change. Even when logic models are used, many programs are not able to distill which program components will produce the behavioral changes they wish to promote.

Evaluations

To characterize the rigorous evidence base on interventions for pregnant and parenting adolescents, we conducted a search of the peer-reviewed literature to identify rigorous quantitative assessments of various program strategies targeting this population. Focusing on articles published between 1996 and 2011, we searched 5 academic databases (PubMed, PsycINFO, Academic Search Premier, ERIC, and Social Work Abstracts) with search terms capturing the program models and settings (e.g., clinic-based intervention, home visiting program) in combination with search terms related to adolescent pregnancy and parenting (e.g., pregnancy in adolescence, teen parents, adolescent parenting).

Two coders first screened the 570 articles returned from the literature search to identify eligible studies that were conducted in the United States and focused on adolescent parents. A second round of screening narrowed the focus to articles with a program evaluation component; 47 articles were eligible. The coders then read each article and coded for study design, sample size, outcomes, and significance levels. To be as comprehensive as possible, coders also reviewed reference lists from eligible articles to find additional studies of interest. This technique identified 15 additional articles, bringing the total sample to 62 articles. To define high-quality studies, we then used and expanded on the inclusion criteria employed by Klerman in her 2004 review of repeat pregnancy interventions.¹² These criteria are shown in Box 1.

Inclusion Criteria for Final Sample of Articles Evaluating Interventions for Adolescent Parents

Country	United States
Target population	Pregnant or parenting youths aged ≤ 19 y
Program date	1990a and later
Publication date	1996–May 2011
Study design	Randomized or quasi-experimental
Sample size	≥ 50 in each treatment group at enrollment
Follow-up period	≥ 12 mo from baseline
Data analysis	Report significance levelsa
	Describe between-group differencesa
	Conduct separate data analyses for adolescents (if included in samples with adult participants)

^aExpanded criteria.

We excluded studies that used preexperimental designs or had small sample sizes. We did not systematically identify articles or reports that were not published or indexed by the databases we searched (e.g., gray literature) or studies that did not include our search terms. We excluded study samples that comprised both adolescent parents and older women (> 20 years) unless separate analyses were conducted for the adolescents in the sample. At least 2 studies that are generally regarded as high quality, but that combined adolescent and adult parents, were therefore excluded from our sample, although they likely have promising implications for adolescent parent interventions.^28,29

Of the 62 studies that contained an evaluation component, we found 14 with sufficient study quality, as defined by our inclusion criteria; both lead authors (C. R. L. and B. B. B.) abstracted these and coded them for the critical study elements (shown in Table A, available as a supplement to the online version of this article at http://www.ajph.org). Interrater agreement was approximately 90%. Table 1 summarizes the characteristics of these 14 studies, and Table A provides the individual study-level details, grouped by study design.

TABLE 1—

Characteristics of 14 Evaluation Studies of Interventions for Adolescent Parents: 1996–2011

Characteristic	Randomized Controlled Trials (n = 8)	Quasi-experimental Studies (n = 6)
Demographics and setting	Mean age = 16–17 y	Mean age = 15–16 y
	Enrolled majority African Americans (n = 4)	Enrolled majority African Americans (n = 6)
	Enrolled pregnant adolescents (n = 2); enrolled parenting adolescents (n = 3); enrolled both (n = 3)	Enrolled pregnant adolescents (n = 2), enrolled both (n = 2), not described (n = 2)
	Urban settings (n = 7)	Urban settings (n = 4), statewide samples (n = 2)
Incorporated theory or model	8	0
Program characteristics	Paraprofessional staff (n = 5)	Professional staff (n = 6)
	Program duration 24 mo (n = 6); 12 mo (n = 2)	Program duration unclear or not described (n = 5)
Program strategies	Home visiting (n = 6)	Case management (n = 5)
	Group classes or meeting (n = 4)	Clinical care (n = 4)
	Case management (n = 3)	Group meetings (n = 3)
		Home visiting (n = 2)
Sample size at baseline	> 200 (n = 4); < 200 (n = 4)	Treatment group > 300 (n = 4), < 100 (n = 2); comparison group > 250 (n = 6)
Attrition	> 40% (n = 3); differential (n = 4)	NA (n = 5)
Outcomes	Repeat pregnancy or birth (n = 6)	Repeat birth (n = 3)
	Social support (n = 2)	Healthy births (n = 2)
	Parenting behaviors/skills (n = 2)	Prenatal care receipt (n = 2)
	Child abuse, neglect, injury (n = 2)	Welfare receipt (n = 1)
	Child health and development (n = 2)	Educational progress (n = 1)
	Educational progress (n = 2)
Reported effect sizes	4	4
Limitations	Program dosage varied between groups or significantly less dosage was delivered (n = 4)	Nonrandomized (n = 6)
		Selection bias (n = 6)
	Demographics varied between treatment groups or between participants who completed program and those who dropped out (n = 3)	Demographic differences between participants in each treatment group (n = 5)
	Limited generalizability (n = 4)	Limited by use of historical record data (n = 4)
		Limited generalizability (n = 6)

Note. NA = not applicable.

Eight of the studies were randomized controlled trials, and 6 were quasi-experimental; 5 of the latter were retrospective cohort studies. All of the controlled trials and none of the quasi-experimental studies incorporated a conceptual model or theory into their study design. Both types of studies engaged similar populations of adolescent mothers, enrolling mostly African Americans living in urban settings who had a mean age between 15 and 17 years; however, 4 randomized controlled trials engaged more racially and ethnically diverse samples. Most studies enrolled adolescents who were either pregnant or parenting, but some focused exclusively on pregnant adolescents.

In regard to program staff, paraprofessionals were more likely to be described by the randomized controlled trials as the primary program implementers, and licensed professionals (e.g., clinicians, social workers) were more likely to deliver the programs described in the quasi-experimental studies. Home visiting and case management were the prevailing strategies, often used in combination with one another and with other approaches, such as parenting education, support groups, and clinical care. Sample size at baseline averaged around 200 participants for 7 of the randomized controlled trials, but varied widely in the quasi-experimental studies, often because of their different sources of comparison participants. Almost all of the randomized controlled trials experienced challenges related to attrition and variable dosage; all of the quasi-experimental studies were affected by selection bias. Ten studies stated that their findings had limited generalizability.

The studies examined a variety of outcomes. Rapid repeat pregnancy or birth was the only outcome examined by a majority of studies; 6 found significant positive effects,^30–35 and 3 reported equivocal effects.^36–38 Two studies that reported positive effects for this outcome found them only among a portion of the treatment group (e.g., participants aged 12–16 years).^30,31 Effect sizes for repeat pregnancy or birth varied between the studies, with some reporting odds ratios (e.g., control participants were 2.45 times as likely as intervention recipients to have a repeat birth³¹) and others percentage reductions (20%,³⁹ 44%,³⁰ or 50%³³ less likely).

Three studies examined variables related to educational progress.^32,38,40 Two reported conflicting results for grade-level expectation and graduation attainment,^32,40 and 1 showed positive effects for attendance and dropout only during pregnancy and not the postpartum year.⁴⁰ Effect sizes for this outcome were not reported. The remaining outcomes were examined by only 1 or 2 studies and did not use comparable metrics. These outcomes focused on the adolescent mother (e.g., parenting behaviors and skills, social support, substance abuse, health care utilization, welfare receipt) or on her infant (e.g., healthy birth weight, incidence of abuse or neglect, immunization, development). Unfortunately, because so few studies examined each of these outcomes, we were unable to analyze program effects across studies.

Generally, urban African American participants were overrepresented in the studies in our review, limiting the generalizability of their findings to more diverse populations of adolescent mothers. Furthermore, although our literature search returned a large body of descriptive literature on the issues faced by young fathers, we identified no evaluations of programs serving adolescent fathers that met our inclusion criteria. This is an obvious population in need of intervention because they make up half of the pregnancy and parenting equation.

We also found methodological issues: many of the studies were hampered by small sample sizes and high or differential attrition, and few of the studies gave a good description of the services received by comparison group participants. Finally, none of the studies in our sample reported standardized effect sizes (e.g., Pearson r correlation or Cohen's d), and only some reported odds ratios or an effect as a percentage reduction. This lack of consistency made it difficult to compare the effects from study to study without the aid of sophisticated meta-analytic techniques requiring an in-depth analysis beyond the scope of this article.

Challenges

A possible explanation for why the body of evidence is so limited is that funding from federal, state, and foundation sources over the past 40 years was largely provided to support program services for pregnant and parenting adolescents. Funders did not typically dedicate funds for or mandate rigorous evaluation of program effectiveness. As Card describes, acceptance and promotion of evaluation took a long time to evolve in the adolescent pregnancy program funder and provider communities.⁴¹ Evaluation was often viewed as diverting money and resources from essential services or as a minor add-on that had potentially risky implications if the results were negative.⁴¹

At the federal level, various political trends and legislative forces also affected what evaluation could be conducted and the results it yielded.⁴² Evaluation activities that were not federally mandated simply did not take place because programs could not afford to perform rigorous effectiveness studies without support.⁴³ On the other hand, sometimes legislative mandates had unintended effects. For example, in one federal demonstration program, the Title XX Adolescent Family Life Program, the funding legislation mandated that specific core services be provided to all program recipients regardless of their evaluation treatment group status.⁴⁴ Such mandates often made it difficult to demonstrate an intervention's effect because participants in both intervention and comparison groups were required to receive some level of services, thus prohibiting a true usual care control group. Finally, even once evaluation was embraced, it was often more logistically convenient and politically palatable to perform preexperimental studies that did not use control groups, producing findings of limited use.⁴¹

In addition to the lack of dedicated evaluation funds, programs serving pregnant and parenting adolescents have faced, and will continue to encounter, significant methodological and practical challenges to evaluation. Although these challenges are not limited to this population, they nevertheless plague evaluation efforts and can contribute to the perception that evaluation is not worth the effort and resources it requires.

Often in these studies, a domino effect occurs wherein characteristics of the pregnant and parenting adolescent target population create challenges related to the implementation of the program, which in turn lead to methodological issues for the evaluation. The 2 most challenging characteristics of this population for program evaluators are that they constitute only a small proportion of adolescents overall and that they are often transient and hard to reach or retain in programming. Most program evaluations target a single treatment program or site that typically entails a relatively small participant pool. Consequently, many program evaluations begin with small samples at baseline. Transient intervention participants are more likely to receive less than the intended program dosage or to drop out.

Small baseline or intake samples with unequal dosage and high attrition rates create issues for evaluation analyses and conclusions. These include power to detect between-group differences in the analysis and concerns about establishing causality (internal validity) and generalization (external validity). In addition to these concerns, which center on the treatment group, the comparisons often present difficulties. Appropriate comparison group participants are often difficult to identify (especially in the case of high-risk subpopulations such as pregnant, homeless adolescents) and even more challenging to retain, so the evaluation is vulnerable to threats to validity such as differential attrition and selection bias.

On the program delivery side, challenges exist at both the staff and the larger organizational levels. The magnitude of client needs and the intensive efforts required of staff to track and retain clients can contribute to high staff turnover rates, which can adversely affect program dosage and fidelity. At the organizational level, program providers' inexperience with evaluation puts the onus on evaluators to obtain staff buy-in regarding the utility of evaluation and the importance of, for example, randomized assignment, which program staff may perceive as denying a potentially effective intervention to adolescents in need. Another practical challenge faced by community-based organizations is that they usually exist in communities well outside the realm of academia; therefore, these organizations may be ill-equipped to find a qualified evaluator to work with or they may experience difficulty complying with the requirements of human participant research.

SOLUTIONS

Despite the many challenges of evaluating programs that serve pregnant and parenting adolescents, the need is evident. The effectiveness of an approach is an empirical question that must be rigorously tested. These young families deserve programs whose components have been informed by the best possible empirical evidence. The benefits to society from successful interventions proven to promote longer-term well-being and productivity for these adolescents and their children would be expected to far outweigh the short-term investments in rigorous evaluations.

Although they are sorely needed, conducting rigorous evaluations of adolescent parent interventions can be difficult. Disentangling complex programs to better understand the contributions from various components is not a simple task. Evaluation funding levels commensurate with these challenges are necessary to provide for the evaluation rigor required. Policymakers and funders should carefully mandate rigorous program evaluation to ensure that funds are properly allocated and used.

Funding alone, however, will not be sufficient to address the evaluation challenges inherent in programs serving pregnant and parenting adolescents. With increasingly constrained budgets, overcoming these challenges will require creative, comprehensive, and coordinated solutions. As a first step, interventions being tested and implemented should build from well-articulated theories of change. Similarly, evaluations must use techniques such as mediation analysis to unravel the intervention black box and determine which components contribute to the program's success or, alternatively, suggest reasons why programs do not show effects.⁴⁵ For example, although home visiting showed promise in several studies in our review, this strategy did not consistently reduce repeat pregnancies, suggesting that other variables related to home visiting may be at play. A more in-depth exploration of variables that may be mediating the success of this approach is needed to improve the outcomes for home visiting. Other research, for example, suggests that the effects of long-acting contraception may be a key variable that needs to be teased out from the overall home-visiting effect.⁴⁶ Evaluations are needed to better inform program planners about what components are effective under what conditions.

With the exception of repeat pregnancies, our analysis showed few commonalities in the outcomes measured. Furthermore, all of the strategies used by the interventions would benefit from additional investigation and replication to build the evidence base and explore the generalizability of findings. Encouraging use of common outcome measures and metrics, along with reporting standard effect sizes across studies, will be important to enable cross-study comparisons. Consistent measurement strategies would facilitate meta-analyses, which could help overcome some of the challenges of the small sample sizes that often plague work with this population. The complexities inherent in studies with adolescent parents require that multiple replications that yield similar results occur before we can have confidence that sufficient evidence justifies recommending a specific component or process. New and innovative intervention strategies are also needed, but the innovations must be based on strong theoretical frameworks and should be held to appropriate evaluation standards. Creative methodological solutions include developing cross-cutting partnerships to allow for pooling of clients to obtain sufficient sample sizes and standardizing approaches to both program components and measurement. Furthermore, a venue for disseminating negative or nonsignificant evaluation findings from well-designed studies is needed to provide an opportunity to learn about unsuccessful strategies.

The Adolescent Family Life program made commendable strides in building evaluation capacity along with an evidence base by requiring rigorous evaluations of the adolescent parenting demonstration projects it funded. Recent budget-driven programmatic restructuring, however, resulted in discontinuation of funding for this program in fiscal year 2012. In light of this, and the need to fill Adolescent Family Life's role in encouraging evaluation rigor—coupled with the broader effects of the recent economic downtown and budget cuts that are affecting all federal, state, and foundation resources—policymakers, public health researchers, and practitioners must be encouraged to work together cooperatively and strategically to achieve more with less. Policymakers and funders should be urged to provide dedicated evaluation funds with stringent research requirements to increase the number of evidence-based interventions targeting adolescent parents and their children.

Funders must be mindful of the challenges of conducting rigorous evaluation in this population and provide sufficient resources and assistance along with facilitating strategies for data sharing across studies. Strong conceptual models that allow for the testing of effectiveness, and relative effectiveness, of various strategies should be a requirement for funding, with fidelity required throughout implementation. Careful process evaluation and documentation will facilitate understanding of what has occurred, as well as subsequent replication and testing by others. Federal agencies can lead by ensuring that evaluations are coordinated across studies, for example, by arranging for multisite evaluation trials. Funders and policymakers can also emphasize broad dissemination of evaluation results to ensure that the contributions of the public health community can shape future policies for adolescent parents and their children.

CONCLUSIONS

The lessons learned from working with pregnant and parenting adolescents have broad applications to the field of public health in serving other vulnerable groups. The overarching mission of program evaluation should be to accurately inform public health practice and policies to achieve positive health and social effects for vulnerable populations through a vision that extends beyond the need to inform any particular program or strategy.

Adolescent pregnancy and childbearing require effective programmatic solutions to address multigenerational costs and consequences. Interventions that have been demonstrated to be effective by scientifically rigorous evaluation methods are needed to support these young families over the long term, with the aim of averting the harmful and costly public health and socioeconomic consequences these adolescents experience and often pass on to subsequent generations. Relying on empirically derived, effective program models offers a key for breaking the cycle stemming from adolescent pregnancy to ensure brighter futures for these young parents and their children.