Abstract
Despite high rates of intimate partner violence (IPV) among teens who are pregnant or parenting, the field is lacking evidence-based prevention programs designed for this population. The purpose of this study was to comprehensively adapt the evidence-based Safe Dates IPV prevention program and conduct a pilot study of the adapted program with female teens who were pregnant or parenting. We completed formative research including a literature review, focus groups, and pre-testing of adapted content to inform the revised curriculum. We then conducted a randomized controlled trial with 32 teens (average age = 17) to compare the adapted program to the original program on implementation characteristics and outcomes, including IPV perpetration and victimization. Directions of effect favored the adapted program over the original program for 18 out of 21 implementation outcomes for which models could be estimated and for 11 of 12 participant outcomes. The strongest effects, all favoring the adapted program, were found for observer-reported adherence to the curriculum, participant ratings of the relevance of characters and scenarios, participant knowledge of ways to get help for abuse, and attitudes toward IPV. These findings demonstrate the feasibility and acceptability of the adapted program for this target population and suggest that the program may be efficacious when evaluated in a larger study.
Keywords: Intimate partner violence, pregnant and parenting teens, prevention, adaptation, pilot study
In the United States, rates of intimate partner violence (IPV) are especially high among teens who are pregnant or involved in a pregnancy (Roberts, Auinger, & Klein, 2006; Silverman, Raj, & Clements, 2004). Physical IPV perpetration and victimization rates are as high as 62% among teens who are pregnant or parenting; psychological abuse is even more prevalent (Harrykissoon, Rickert, & Wiemann, 2002; Milan, Lewis, Ethier, Kershaw, & Ickovics, 2005; Newman & Campbell, 2011). IPV among teen parents may have physical, psychological, and sexual dimensions, and mutual perpetration of IPV between partners is common (Milan et al., 2005; Newman & Campbell, 2011). Furthermore, teens who are pregnant or parenting experience unique forms of abuse, including reproductive coercion, sabotage of education or employment, and control or manipulation related to child rearing (Born, 2012; Kennedy & Bennett, 2006; Raphael, 2006). Our own work suggests that electronic aggression, including the use of technology to control or monitor a partner, is also common in this population (Herrman, 2013; Herrman, Palen, Kan, Feinberg, Magee, & Haigh, 2019).
The consequences of IPV can endanger not only teen parents’ own health but also the well-being of their children. Pregnant and parenting female teens who are involved in IPV, as compared to those not involved, have poorer prenatal health and birth outcomes and are at increased risk for substance use and depressive symptoms (Covington, Justason, & Wright, 2001; Lindhorst & Oxford, 2008; Parker, MacFarlane, & Soeken, 1994). Maternal substance use and depression, in turn, have long-term impacts on child well-being (Black et al., 2002; Conners et al., 2003). Exposure to IPV may impede academic achievement, thereby perpetuating economic dependence and ongoing poverty (Kennedy & Bennett, 2006). In addition, teen parents in violent relationships may be at increased risk for perpetrating child abuse (Moore & Florsheim, 2008; Taylor, Guteman, Lee, & Rathouz, 2009), and children who witness IPV are at heightened risk for psychological, social, and academic problems (Kitzmann, Gaylord, Holt, & Kenny, 2003), including increased risk of IPV in their own relationships (Ehrensaft et al., 2003).
Risk factors for IPV among teens include attitudes accepting violent behavior and gender stereotypes; relationship conflict; and poor behavioral control, often operationalized as anger or general aggression (Capaldi, Knoble, Shortt, & Kim, 2012; Stith, Smith, Penn, Ward, & Tritt, 2004; Vagi et al., 2013). These risk factors may be particularly salient among teen mothers, who may experience high rates of violence in their families and communities (Kennedy, 2006) and therefore may have few models of healthy relationships. Gender stereotypes around parenting roles may be linked to partners’ attempts to limit teen mothers’ educational and employment opportunities (Herrman et al., 2019). Normative stressors at the transition to parenthood, which often strain the relationships of even well-adjusted adults (Doss, Rhoades, Stanley, & Markman, 2009), may be especially difficult for teens to navigate as they are also contending with the challenges of adolescence and are still developing communication skills and impulse control (Herrman, 2013). Qualitative research has found that teen mothers often accept violent relationships as normative and do not take violence seriously (Herrman, 2013); our own formative research suggested that even when teen mothers recognize the need for help, they have limited access to affordable support services (Herrman et al., 2019). Therefore, protective factors for this population may include the belief that teens in violent relationships need help as well as awareness of programs and resources that teens can turn to for help with their relationships.
No IPV prevention programs appear to fully address the unique needs of pregnant and parenting female teens. Group-based programs can be effective in preventing teen dating violence but have only been tested in universal school-based or child welfare populations, or with male high school athletes (Foshee & Reyes, 2009; Niolon et al., 2017). These programs may not be engaging for pregnant and parenting teens because of the lack of attention to the unique contexts and stressors in their lives, such as prior victimization and trauma, reproductive health decisions, impacts of violence on coparenting and parenting, and barriers to seeking help for or ending an unhealthy relationship. Interventions that have specifically addressed the needs of pregnant and parenting teens to prevent IPV have relied on intensive clinical couples therapy models (Florsheim, McArthur, Hudak, Heavin, & Burrow-Sanchez, 2011) or have been developed on the basis of general risk and protective factors for violence (Langhinrichsen-Rohling & Turner, 2012).
The group-based IPV prevention program Safe Dates was efficacious in preventing psychological abuse perpetration, moderate physical violence perpetration and victimization, and sexual violence perpetration in a general population of teens (Foshee et al., 2005). Effects of the program did not differ by sex or race/ethnicity, suggesting that the program is relevant for diverse youth. Safe Dates addresses norms about dating violence acceptability and gender roles, and beliefs in the need for help and the importance of awareness of services based on precaution–adoption theory (Weinstein, 1988). The program includes interactive and engaging skills-building opportunities for teens, including modeling of skills by implementers and opportunities for teens to practice and receive feedback on skills. Such rehearsals are direct preparation for actual behaviors and are meant to simulate interactions teens will encounter (Fixsen, Naoom, Blasé, Friedman, & Wallace, 2005). That Safe Dates addresses both perpetration and victimization is important given the high rates of mutual violence among teen mothers and their partners (Newman & Campbell, 2011). As a group-based program, Safe Dates also avoids the challenges of implementing couples interventions given the instability of many teen parents’ relationships (Bunting & McAuley, 2004) and enables teens to develop strong positive relationships with peers (Nation et al., 2003).
A previous study made minor revisions to Safe Dates for teen mothers and tested the feasibility of the adapted program in a single-group evaluation with 41 female teens who were receiving pregnancy or parenting services (Herrman & Waterhouse, 2014). Adaptations were based on a literature review and focus groups with teen mothers and included broadening unit-based objectives; revising scenarios and exercises to include situations that relate to pregnancy and parenting; providing background for facilitators about IPV among pregnant and parenting teens; adding selected information about the intersection of pregnancy, parenting, and IPV; and adding discussion of safety for participants and their children. There were statistically significant declines from the first to last day of the program in reported perpetration of psychological abuse and gender stereotyping and increased knowledge of community resources for IPV victims. Although anecdotal evidence suggests that the program was well received by participants, this study did not include a process evaluation, so it did not establish the feasibility, acceptability, or fidelity of the program.
Motivated by these promising, preliminary findings and the critical need for an evidence-based IPV prevention program designed specifically for this population, the current study involved a comprehensive and theoretically informed adaptation of Safe Dates for female teens who are pregnant or parenting. We then conducted a pilot randomized controlled trial (RCT) to compare the adapted program to the original program on both implementation characteristics and participant outcomes. We predicted that Safe Dates could be adapted to address the unique needs of pregnant and parenting female teens, and that the adapted program would be more feasible to implement with fidelity and more acceptable to participants than the original. We also expected that differences in participant outcomes would positively favor the group that received the adapted program.
Method
Participants
We recruited study participants from a state-funded program for pregnant and parenting teens in a southeastern state in the United States. The research team worked with state program leadership to identify and recruit four local sites for inclusion. State program staff recommended and made initial outreach to selected sites. A series of telephone calls with interested sites enabled the research team to identify those that would be able to meet the study requirements. Sites were prioritized if they had an adequate program size, participant engagement in group education sessions, availability of different staff to facilitate two versions of the program, and capacity for participation in an RCT. Staff from these local programs were responsible for recruiting their clients to the study.
Local program staff advertised the study to their clients in person or via Facebook group posts, Facebook messaging, text messaging, or telephone calls. They then screened interested individuals for the study. Clients were eligible to participate if they were females between the ages of 14 and 19, were pregnant or already had a child, were fluent in English, were free of severe mental health conditions or cognitive impairment, and (for minors) had a parent who was able to provide consent in English or Spanish. Local program staff obtained written participant consent for those aged 18 or older or participant assent and parental consent for minors. Parents provided consent by phone and returned a signed consent form to program staff or study staff.
Out of an estimated 47 eligible clients, local staff enrolled 32 in the study. On average, participants were 17 years old at baseline. There were fairly equal numbers of Hispanic (31%), non-Hispanic white (31%), and non-Hispanic black (28%) participants, with remaining participants being multiracial (9%). At baseline, most participants (78%) were parenting at least one child and about a third (31%) were pregnant. Most participants (69%) were enrolled in a high school or GED program at baseline, with the remainder already graduated from high school (22%) or not in school (9%). Demographic and baseline characteristics of the sample are shown in Table 1.
Table 1.
Baseline sample characteristics and group differences (N = 32)
| Total | Comparison | Treatment | |
|---|---|---|---|
| M (SD) | M (SD) | M (SD) | |
| Age | 17.3 (1.3) | 17.2 (1.4) | 17.5 (1.3) |
| N (%) | N (%) | N (%) | |
| Parenting | 25 (78.1) | 12 (75.0) | 13 (81.3) |
| School Status | |||
| In High School/GED Program | 22 (68.8) | 11 (68.8) | 11 (68.8) |
| Completed High School/GED | 7 (21.9) | 3 (18.8) | 4 (25.0) |
| Not in School | 3 (9.4) | 2 (12.5) | 1 (6.3) |
| Race/Ethnicity | |||
| Hispanic | 10 (31.3) | 6 (37.5) | 4 (25.0) |
| Non-Hispanic White | 10 (31.3) | 4 (25.0) | 6 (37.5) |
| Non-Hispanic Black | 9 (28.1) | 6 (37.5) | 3 (18.8) |
| Other | 3 (9.4) | 0 (0.0) | 3 (18.8) |
| Household Composition | |||
| Lives with Husband/Partner | 10 (32.3) | 4 (26.7) | 6 (37.5) |
| Lives with One/Both Parents | 24 (77.4) | 10 (66.7) | 14 (87.5) |
| In a Committed Relationship with Child’s Father | 19 (63.3) | 11 (68.8) | 8 (57.1) |
| Past Year IPV | |||
| Any Perpetration | 21 (65.6) | 10 (62.5) | 11 (68.8) |
| Any Victimization | 20 (62.5) | 9 (56.3) | 11 (68.8) |
| Psychological Perpetration | 18 (56.3) | 7 (43.8) | 11 (68.8) |
| Psychological Victimization | 20 (62.5) | 9 (56.3) | 11 (68.8) |
| Physical Perpetration | 5 (15.6)1 | 5 (31.3) | 0 (0.0) |
| Physical Victimization | 6 (18.8) | 3 (18.8) | 3 (18.8) |
| Sexual Perpetration | 1 (3.2) | 1 (6.7) | 0 (0.0) |
| Sexual Victimization | 4 (12.5) | 2 (12.5) | 2 (12.5) |
| Electronic Perpetration | 7 (21.9) | 4 (25.0) | 3 (18.8) |
| Electronic Victimization | 6 (18.8) | 2 (12.5) | 4 (25.0) |
| Birth Control Sabotage | 3 (9.4) | 1 (6.3) | 2 (12.5) |
| Pregnancy Coercion | 3 (9.4) | 0 (0.0) | 3 (18.8) |
| Work/Education Sabotage | 2 (6.3) | 0 (0.0) | 2 (12.5) |
Group difference p = .04; not statistically significant after correction for multiple comparisons.
Procedure
Institutional review boards at both the state’s Department of Health and Human Services and at RTI International approved all study materials and procedures. Prior to the baseline survey, study staff reviewed the consent or assent form individually with each teen (in a private room, if the teen desired) using a script. After the baseline survey was complete, study staff randomly assigned each participant to the comparison or treatment condition. Comparison group participants received the original Safe Dates curriculum whereas treatment group participants took part in the experimental adapted version, Safe Dates: Supporting Healthy Relationships Among Teen Mothers.
Baseline data collection occurred from August to December 2016. Participants completed the baseline survey up to 8 weeks before the program implementation with almost half of participants completing the survey on the same day as the first program session. Twenty-eight participants (88%) completed the follow-up survey from February to June 2017, approximately 6 months after baseline. Data were collected using tablet computers at local parenting program offices, high school libraries, high school conference rooms, or in participants’ homes. Participants completed the self-administered surveys individually or in small groups. Twenty-nine (91%) baseline participants and 18 (64%) follow-up participants completed the survey at the program offices. Participants received a $20 gift card to a local retail store after completing each survey.
Intervention: Safe Dates: Supporting Healthy Relationships Among Teen Mothers.
For our comprehensive adaptation, we used key elements of the ADAPT-ITT model for adapting evidence-based programs (Wingood & DiClemente, 2008). We based initial adaptations to Safe Dates on a literature review and five formative focus groups. The focus groups were conducted with pregnant and parenting teen females as well as their male partners in separate groups. The purpose of the focus groups was to learn about the specific dynamics of pregnant and parenting teens’ relationships with their partners, including language used, stressors and sources of conflict, experiences of violence, and resources for getting help. Key themes that were identified included that relationships between teen parents take many forms and that teen parents often remain with a partner because of anticipated challenges in leaving the relationship or in finding a new partner; that social isolation, challenges with co-parenting, a lack of communication skills, competing responsibilities, financial strains, and conflicts with their own parents or their partners’ parents cause stress and relationship conflict that may be linked to IPV; that body shaming, limiting contact with children, limiting access to school and work, sexual coercion, and reproductive coercion are types of abuse that teen parents may uniquely experience; and that teen parents face barriers to accessing IPV resources and help from professionals, and may instead lean on friends, family, and other teen parents for support (Herrman et al., 2019). The first phase of adaptations incorporated much of the teens’ experiences into the curriculum scenarios and activities. For example, overcoming barriers to getting help for IPV was noted as a major problem for the teens, because many of their barriers were structural as well as psychological. The resulting activity in the adapted program focused more than in the original program on how to overcome barriers to getting help, and we replaced role plays to practice helping a friend with role plays to practice asking for help.
The second phase of adaptation involved multiple iterative rounds of pretesting based on the concept of “theater testing” in the ADAPT-ITT model (Wingood & DiClemente, 2008). Study staff implemented parts of the curriculum with pregnant and parenting female teens, pausing after each section to ask the teens for their reactions and suggestions for improvement. This feedback focused on the relatability of characters, realism of the scenarios, and understanding of the activities and take-home messages. Some of the curriculum sections were implemented and revised multiple times with different teens for further refinement. Input was also gathered from local program staff who observed the pretest sessions. Adaptations after pretesting focused on refining the activities to be better suited for smaller groups, changing instructions to be more understandable, and adding some scenarios to more fully capture teens’ lived experiences.
The resulting program, Safe Dates: Supporting Healthy Relationships Among Teen Mothers, was a 10-session, group-based program designed around the original Safe Dates conceptual model (Foshee et al., 1996). Sessions ranged from 34 to 54 minutes long, with most running approximately 50 minutes, and included interactive discussions, role-plays, and brainstorming. While the core conceptual model did not change, the risk factors and contextual issues targeted were those identified in formative research.
Before implementation, local parenting program staff from our partner sites were trained to facilitate either the original or adapted program during a workshop that lasted 2½ days. The local program staff generally had college degrees, and some had advanced degrees in areas such as social work, education, or marriage and family therapy. The staff had at least 2 years (but many had more than 10 years) of experience serving vulnerable children and teens, including teens who are pregnant or parenting. The staff all had previous experience facilitating other programs with the teens in the study (including group-based programs or evidence-based home-visiting curricula) or otherwise were familiar with the teens receiving services from their agencies. The original and adapted Safe Dates programs were implemented simultaneously by two different staff members at each program site. Each site scheduled the sessions across 2 or 3 days, blocking the sessions together. Sessions were generally implemented during school breaks or in the late afternoon to evening, after school. Staff incentivized attendance through various methods chosen by each site to be most effective for their participants—food, door prizes, transportation, and gift cards were all used.
Facilitators took attendance at each program session. Across conditions, participants attended an average of 8 sessions (SD = 2.8); 78% attended at least 7 sessions, and 53% attended all 10 sessions. Facilitators completed session logs after each session or at the end of each day of the program. The session logs assessed adherence to each part of each session and participant engagement in each part, as well as any intentional changes facilitators made to the program during implementation. Study staff completed observation forms for 46% of the sessions to provide another source of data about implementation fidelity and quality. Average adherence to the program across both conditions was high (94.7% based on facilitator reports and 82.1% based on observer reports). Participants were also asked to complete an anonymous feedback form at the end of the program. A total of 28 participants completed feedback forms, but some of the forms for one site were improperly labeled by condition; therefore, analyses include data from three sites (n = 21). Across conditions, 75% of participants reported the program was “very interesting,” 89% reported the facilitator did a “very good” job, 71% reported they were “very likely” to use what they learned, and 82% reported they “definitely would” tell others that they should come to this program.
After follow-up data collection was completed, study staff conducted semi-structured telephone interviews with each of the program facilitators. These interviews covered topics such as challenges teaching the program, relevance and age appropriateness of the scenarios and activities, and aspects of implementation such as the length of the program and preparation required to facilitate the program. These interviews gave us further insight into how facilitators prepared the sessions, whether the adapted program was easier to implement or resonated better with the target audience, and suggestions for future implementation.
Measures
Observer measures.
Observers completed a checklist of whether each session activity was completed, not completed, or completed with changes. Adherence was calculated as the proportion of activities in each session that were completed without changes. Observers assessed quality with a single item per session: “Overall, how would you rate the facilitator’s performance?” Response options ranged from 1 (outstanding) to 6 (very poor), and the item was reverse coded for analysis. For each part of each session, observers responded to the question, “In general, how much did the teens participate (either actively participating or listening attentively) in this activity?” Response options ranged from 1 (they did not participate at all) to 4 (they participated a lot). Participation was calculated as the mean of these items in each session.
Facilitator measures.
As with observers, adherence was measured using a checklist for each session activity. Participant engagement, participant understanding, and quality were each measured with a single item per session (e.g., “How engaged were the teens in the session?”) and rated on a 4-point scale. Facilitators marked whether any of seven challenges (e.g., not enough time, teens were distracted, they or teens were uncomfortable discussing topics) interfered with their ability to implement the session. The number of barriers was summed for each session.
Participant measures.
The participant feedback form used closed- and open-ended items to assess participants’ reaction to and retained knowledge from the original and adapted programs. Measures of participant feedback developed for the study were assessed on a 0 to 4 Likert-type scale and included single items about how interesting the program was, likelihood of using what was learned, facilitator quality, relevance of characters and scenarios, and the extent to which they recommended the program to others. Knowledge domains included facts about IPV, abusive behaviors, red flags for abuse, ways to get help, gender stereotypes, calming strategies, SAFE communication skills, and sexual assault. These were each measured with brief sets of items developed and scored by the research team; for example, four yes/no questions assessed participants’ understanding of behaviors that are defined as abusive, and two open-ended items asked participants for examples of ways to calm down when someone is angry.
The baseline and follow-up participant surveys included measures of intermediate outcomes (e.g., attitudes, skills, relationship behaviors, self-efficacy) and IPV/abuse perpetration and victimization, many of which were adapted from the original evaluation of Safe Dates. A full list of these measures and details about each appear in Table 2. Adaptations were made to shorten scales, simplify language, and make terminology relevant to teens who are pregnant or parenting. Given the small sample size and low base rates of some types of IPV, we combined the IPV types into a single measure of any IPV for analysis. IPV measures were assessed in the past year at baseline and in the past 6 months at follow-up; other types of abuse (such as birth control sabotage) were assessed for lifetime prevalence at baseline and in the past 6 months at follow-up.
Table 2.
Outcome measures from baseline and follow-up surveys
| Construct | Source | # items | Example item |
|---|---|---|---|
| Intermediate outcomes (Mean of items taken unless otherwise noted) | |||
| Acceptance of IPV | Prescribed and proscribed norms (Foshee et al., 1998) | 10 | Hitting a partner is never OK. |
| Conflict management skills | Constructive communication skills (Foshee et al., 1998) | 7 | Tried to calm down before I talked to them |
| Belief in the need for help | Foshee et al., 1998 | 2 | Teens who are violent to their partners need to get help from others. |
| Gender stereotypes | Attitudes Toward Women Scale for Adolescents (Galambos, Petersen, Richards, & Gitelson, 1985) | 12 | Girls are always trying to manipulate boys. |
| Emotion regulation problems | Inventory of Altered Self Capacities (emotion dysregulation scale; Briere & Runtz, 2002) |
6 | Not being able to control your anger |
| Confident could get help/tell | Self-efficacy to deal with violence (Van Camp, Hébert, Guidi, Lavoie, & Blais, 2014) | 4 | If you were being abused by a romantic partner, how confident are you that you would be able to tell someone you trust? |
| Relationship self-efficacy | Emotional control subscale (Lopez, Morúa, & Rice, 2007) | 4 | How sure are you that you can…stay calm when you and your partner are having a serious argument? |
| Awareness of services for victims (Yes/No) | Foshee et al., 1998 | 1 | Are there any services in your area for helping teenagers who are victims in abusive and violent romantic relationships? |
| Awareness of services for perpetrators (Yes/No) | Foshee et al., 1998 | 1 | Are there any services in your area for helping teenagers who are abusive and violent toward their partners? |
| Talked with someone re: abuse (Yes/No) | Foshee et al., 1998 | 1 | In the past 6 months, have you talked with anyone about abusive or unhealthy behaviors in your romantic relationship? |
| IPV/abuse outcomes (> Never or Yes to any item = 1; else = 0) | |||
| Psychological IPV | Safe Dates Scale (Foshee et al., 1998) and Abusive Behavior Inventory (Shepard & Campbell, 1992) | 13 (perp), 13 (vic) | Damaged something that belonged to me |
| Physical IPV | Safe Dates Scale (Foshee et al., 1998) | 8 (perp), 8 (vic) | Pushed, grabbed, or shoved me |
| Sexual IPV | Safe Dates Scale (Foshee et al., 1998) and CTS-2 (Straus, Hamby, Boney-McCoy, & Sugarman, 1996) | 5 (perp), 5 (vic) | Used threats to make me have sex or do other sexual things that I did not want to do |
| Electronic aggression | Developed for study; informed by Picard, 2007 | 3 (perp), 3 (vic) | Used social media apps or Websites, texting, a cellphone, or email to keep track of or to control what you were saying or doing |
| Birth control sabotage | Clark, Allen, Goyal, Raker, & Gottlieb, 2014 | 5 | Taken off a condom while you were having sex so that you would get pregnant |
| Pregnancy coercion | Clark et al., 2014 | 7 | Told you not to use any birth control (like the pill, shot, ring, patch etc. |
| Education or work sabotage | Riger, Ahrens, & Blickenstaff, 2000 | 6 | Threatened you to keep you away from work or school |
Data Analyses
We first conducted descriptive analyses by examining means and frequency distributions of demographic, implementation, and outcome variables. To test for baseline differences between the original and adapted conditions, we examined t-tests and cross-tabulations using Fisher’s exact test (for continuous and dichotomous variables, respectively). Next, we examined a series of linear and logistic regression models to test for differences between the original and adapted conditions related to implementation and outcome variables at follow-up (which were dependent variables in separate models). Site was included as a control variable, so study condition and site were entered into each model as independent variables. For most of the outcome variables, which were measured at both baseline and follow-up, the baseline value on the outcome of interest was also included as an independent variable, so that the analyses tested group differences in change over time.
Given that each set of analyses involved several variables, we adjusted significance levels using a Bonferroni correction for multiple comparisons. In addition, given the small sample size for this study, tests of statistical significance may not give a complete picture of the magnitude of any group differences. Therefore, we also calculated effect sizes for all implementation and outcome variables. Means and standard deviations or estimates of group differences from the regression models (t values or odds ratios) were used to calculate effect sizes (Cohen’s d) for each variable (Cohen, 1988).
Results
Table 1 shows that the treatment and comparison group samples were demographically comparable at baseline. IPV prevalence rates varied from 3.2% for sexual abuse perpetration to 62.5% for psychological abuse victimization. Rates of unique forms of abuse (birth control sabotage, pregnancy coercion, and work or education sabotage) were low in this sample. At baseline, the comparison group participants reported a higher rate of past year physical violence perpetration, but this difference was not statistically significant after correcting for multiple comparisons.
Tables 3 and 4 present differences, between conditions, related to implementation outcomes and teen outcomes. Directions of effect favored the adapted program over the original program for 18 out of 21 implementation outcomes (for which models could be estimated) and 11 of 12 teen outcomes. Effect sizes ranged from .10 to more than 1 for favorable effects and from .03 to .51 for unfavorable effects; most of the effect sizes for implementation outcomes were small to medium, and the effect sizes for teen outcomes were fairly evenly distributed between small, medium, and large. Only the effects for three implementation outcomes and one teen outcome were large enough to be statistically significant using a traditional criterion. Observer-reported adherence, participant ratings of the relevance of characters and scenarios, and participant knowledge of ways to get help for abuse were higher for the treatment group (adapted program) than for the comparison group (original program). There were larger reductions in acceptance of IPV for the treatment group than for the comparison group from baseline to follow-up. However, these differences were not statistically significant after we applied corrections for multiple comparisons.
Table 3.
Group differences in program implementation and participant learning
| Comparison Group | Treatment Group | ||||
|---|---|---|---|---|---|
| M (SD) | M (SD) | t | df | Cohen’s d | |
| Attendance (% sessions) | 7.56 (2.68) | 8.38 (2.85) | −0.68 | 27 | .30 |
| Facilitator report | |||||
| Adherence | 93.53 (6.65) | 95.96 (5.91) | −1.80† | 75 | .39 |
| Number of barriers to adherence | 0.48 (0.57) | 0.39 (0.67) | 1.68† | 57 | .14 |
| Participant engagement | 3.36 (0.58) | 3.55 (0.64) | −1.64 | 74 | .31 |
| Participant understanding | 3.54 (0.55) | 3.65 (0.53) | −0.93 | 74 | .20 |
| Implementation quality | 3.47 (0.57) | 3.56 (0.72) | −0.58 | 73 | .14 |
| Observer report | |||||
| Adherence | 77.53 (15.02) | 85.55 (10.62) | −2.74* | 32 | .62 |
| Participation | 3.39 (0.69) | 3.52 (0.53) | −0.92 | 32 | .21 |
| Facilitator quality | 4.56 (1.41) | 5.06 (0.64) | −1.73† | 29 | .45 |
| Participant feedback | |||||
| How interesting | 3.64 (0.50) | 3.70 (0.67) | −0.21 | 17 | .10 |
| Likely to use what learned | 3.63 (0.50) | 3.80 (0.42) | −0.72 | 17 | .35 |
| Teacher did a good job | 3.91 (0.30) | 3.90 (0.32) | 0.00 | 17 | .03 |
| Relevance of characters/scenarios | 2.32 (1.15) | 3.45 (0.76) | −2.46* | 17 | 1.16 |
| Would tell others to come | 3.73 (0.47) | 3.90 (0.32) | −1.22 | 17 | .42 |
| Participant knowledge | |||||
| Facts about IPV | 5.18 (0.75) | 5.50 (0.53) | −1.08 | 17 | .49 |
| Abusive behaviors | 3.27 (0.65) | 3.10 (0.88) | 0.51 | 17 | .22 |
| Red flags for abuse | 5.82 (1.25) | 5.60 (2.01) | 0.45 | 17 | .13 |
| Ways to get help | 2.27 (1.68) | 4.00 (1.83) | −2.51* | 17 | .98 |
| Gender stereotypes | 2.64 (1.36) | 3.20 (0.92) | −1.05 | 17 | .48 |
| Calming strategies | 3.45 (0.82) | 3.80 (0.63) | −1.09 | 17 | .48 |
| N (%) | N (%) | OR | Cohen’s d | ||
| SAFE communication skills | 8 (72.7) | 9 (90.0) | 6.00 | .67 | |
| Sexual assault | 8 (72.7) | 10 (100) | ---- | ---- |
p < .10;
p < .05; none of the differences were statistically significant after correction for multiple comparisons.
Note. Blank OR and d values could not be estimated because there was no variability in treatment group responses.
Table 4.
Group differences in participant outcomes
| Comparison Group | Treatment Group | ||||||
|---|---|---|---|---|---|---|---|
| Baseline (N = 16) | Follow-up (N = 13) | Baseline (N = 16) | Follow-up (N = 15) | ||||
| M (SD) | M (SD) | M (SD) | M (SD) | t | df | Cohen’s d | |
| Acceptance of IPV | 0.03 (0.06) | 0.30 (0.44) | 0.32 (0.59) | 0.10 (0.31) | 2.75* | 20 | 1.23 |
| Conflict management skills | 2.44 (0.51) | 2.31 (0.67) | 2.15 (0.51) | 2.36 (0.54) | −1.06 | 21 | .46 |
| Belief in the need for help | 2.61 (0.60) | 2.50 (1.06) | 2.60 (0.89) | 2.77 (0.78) | −0.78 | 18 | .37 |
| Gender stereotypes | 1.46 (0.44) | 1.44 (0.63) | 1.76 (0.47) | 1.48 (0.45) | 0.66 | 21 | .29 |
| Emotion regulation problems | 2.07 (1.03) | 1.87 (0.97) | 2.33 (1.20) | 1.79 (0.97) | 0.86 | 21 | .38 |
| Confident could get help/tell | NA | 3.59 (0.87) | NA | 3.78 (0.58) | −0.68 | 22 | .26 |
| Relationship self-efficacy | NA | 2.25 (0.50) | NA | 2.31 (0.48) | −0.69 | 16 | .12 |
| N (%) | N (%) | N (%) | N (%) | OR | Cohen’s d | ||
| Awareness of services for victims | 8 (61.5) | 8 (72.7) | 4 (26.7) | 11 (73.3) | 2.54 | .52 | |
| Awareness of services for perpetrators | 6 (46.2) | 8 (66.7) | 6 (40.0) | 10 (66.7) | 1.28 | .14 | |
| Talked with someone about abuse | NA | 2 (18.2) | NA | 1 (7.7) | 0.40 | .51 | |
| Any IPV perpetration | 8 (61.5) | 6 (50.0) | 10 (66.7) | 7 (46.7) | 0.70 | .20 | |
| Psychological perpetration | 6 (46.2) | 6 (50.0) | 10 (66.7) | 6 (40.0) | 0.71 | .19 | |
| Any IPV victimization | 7 (53.9) | 9 (69.2) | 10 (66.7) | 8 (53.3) | 0.40 | .51 | |
| Psychological victimization | 7 (53.9) | 9 (75.0) | 10 (66.7) | 6 (40.0) | 0.16† | 1.01 | |
p = .08;
p = .01; not statistically significant after correction for multiple comparisons.
Note. NA = not assessed.
Facilitators reported very positive experiences with both the original and adapted programs. They found the program easy to facilitate and age-appropriate and noted that the participants enjoyed the discussions and activities. They noted the challenge of getting participants to attend multiple sessions and indicated that attendance would have been much worse if they had spread the program out longer. Facilitators of the adapted program emphasized its relevance and participants’ resonance with the scenarios, whereas facilitators of the original program noted that some elements could have been better tailored to the lives of teen mothers.
Discussion
In this study we adapted the evidence-based Safe Dates IPV prevention program to increase relevance for female teens who are pregnant or parenting and conducted a pilot RCT to assess the program’s feasibility and acceptability. Safe Dates: Supporting Healthy Relationships Among Teen Mothers seeks to fill a gap in the available practices to prevent and reduce IPV and addresses the specific contexts, risk and protective factors, and types of abuse that teen mothers may experience. Findings from the pilot study indicate that the adapted program is feasible to implement and acceptable to participants, and provide initial support for the potential of the program to prevent or reduce IPV in this high-risk population.
Implementation data from facilitators, observers, and participants supported our hypothesis that the adapted version of the program was easier to implement with fidelity than the original. Interviews with facilitators of the original program revealed that participants frequently discussed ways in which the concepts applied to parenting situations, even though this was not part of the curriculum. Drift from the original program may also have been necessary because the sessions were implemented outside of school and in a condensed manner; for instance, homework assignments (which are a feature of the original program but were eliminated in the adaptation) were not feasible. Participants in the adapted program rated the scenarios and characters as more relevant than participants in the original program. This increased resonance may have facilitated learning; participants’ understanding of key program concepts was generally better in the treatment group than in the comparison group, and effect sizes were largest for concepts that were the focus of more adaptation (e.g., ways to get help for abuse). In contrast, the treatment group showed less ability to identify abusive behaviors and red flags for abuse than the comparison group. These effects were small in magnitude and the brief measurement of these areas of knowledge may not have adequately captured participants’ understanding of the concepts. Nonetheless, given that teens would need to be able to identify red flags or abusive behaviors before enacting help-seeking strategies, it may be necessary to revise the program to more effectively teach these concepts. For example, additional examples of abusive behaviors or red flags may need to be discussed through scenarios, and more time may need to be devoted to review of these topics throughout the program. Guidance for facilitators could also be added to ensure that participants are given accurate information about behaviors that are common among teens but may be abusive if repeated in a pattern (such as controlling or monitoring behaviors).
Although the small sample size and large number of comparisons limited our power to detect significant differences between groups, the direction of effects favored the treatment group for nearly all outcomes. We expected that differences between the study conditions would be small given that teens in both conditions received a version of the program. Indeed, in post-hoc analyses not shown, we found strong improvements in the full sample from pre-test to post-test with respect to endorsement of gender stereotypes, emotion regulation, and awareness of services for IPV victims and perpetrators. Still, several effect sizes for group differences in the outcomes were medium (around .5) or larger. There were large effects favoring the treatment group for acceptance of IPV and psychological abuse victimization, and medium effects for conflict management skills, awareness of services for IPV victims, and any IPV victimization. Norms about IPV and awareness of services were significant mediators in the evaluation of the original Safe Dates program (Foshee et al., 2005); although conflict management skills were not a significant mediator, the salience of conflict in teen parents’ lives and the tendency for conflict to escalate to violence in this population (Herrman, 2013; Herrman et al., 2019) suggest that this may be an important mechanism of effects for the adapted program. The only outcome for which the direction of effects favored the comparison group was whether participants had talked with anyone about abuse in their relationship. It is possible that talking with someone about abuse is driven more by whether abuse has occurred and whether there are trustworthy others available than by motivation to talk with someone. Indeed, effects for confidence in the ability to get help for abuse favored the treatment group.
Limitations
The sample size precluded testing models for physical and sexual violence and other types of abuse. In general, the small and selective nature of the sample (i.e., that participants were already connected with programming serving pregnant or parenting teens and were largely in school) limits the conclusions that can be drawn from this study. The sample size was smaller than anticipated because of declining caseloads in the participating sites’ programs. Additional research with a larger sample will be needed to test the efficacy of the adapted program.
Other design limitations of this study include that all of the outcomes were self-reported by teen participants, and the short-term nature of the follow-up duration. Teens who participate in programming designed to reduce IPV may provide more socially desirable answers to items about attitudes and behaviors related to violence and conflict, though provision of the original program to the comparison group mitigated this bias. Future research should include more rigorous methods in order to limit self-report biases, collect additional detail about skills acquisition as a result of program participation, and test longer-term effects of the program on the well-being of teens and their children.
All of the sites chose to condense implementation of the program into two or three days in order to maximize participant retention, and it is unclear if this implementation structure impacted the ability of participants to retain the concepts they learned and to practice the acquired skills. In the original evaluation of Safe Dates in schools, the program was implemented over the course of 5 days to 10 weeks, depending on school practices (Foshee et al., 1996). Future evaluation of the adapted program will ideally include similar variation in the duration of implementation in order to test potential impacts on implementation quality, attendance, participant learning, and behavioral outcomes.
This study represents an important first step toward expanding the evidence base for IPV prevention programs that can be implemented efficiently with teens who are pregnant or parenting. Effective interventions could be adopted by practitioners, schools, and other organizations serving pregnant and parenting teens. A comprehensive adaptation of an evidence-based program yielded high implementation quality and acceptability for both facilitators and diverse teen participants. By comparing the adapted program to the original, we have demonstrated the relative advantages of the adaptation in terms of implementation and participant learning. Group differences in outcomes also suggest that adapting the program to address the unique needs and experiences of teen mothers may increase its efficacy in this vulnerable population. If found to be effective in future research, this program has the potential to improve the health and well-being of not only teen mothers, but also their partners and children.
Highlights.
Intimate partner violence (IPV) is common among teens who are pregnant or parenting, but IPV prevention programs have not been tailored for this population.
This study adapted and pilot tested an IPV prevention program for teen mothers using a randomized controlled trial.
Feasibility, acceptability, and participant outcomes were better for the adapted program than for the original program.
Results suggest advantages in carefully adapting IPV prevention programs to address unique needs of teen mothers.
Footnotes
The authors declare that they have no conflict of interest.
Ethical Approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the RTI International research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Contributor Information
Judith W. Herrman, University of Delaware, Newark, DE
Jessica D. Williams, RTI International, Research Triangle Park, NC
Mark E. Feinberg, Pennsylvania State University, University Park, PA
References
- Black MM, Papas MA, Hussey JM, Dubowitz H, Kotch JB, & Starr RH Jr. (2002). Behavior problems among preschool children born to adolescent mothers: Effects of maternal depression and perceptions of partner relationships. Journal of Clinical Child and Adolescent Psychology, 31, 16–26. doi: 10.1207/S15374424JCCP3101_04 [DOI] [PubMed] [Google Scholar]
- Born A (2012). Relationship violence and teenage parents. Journal of Infant, Child & Adolescent Psychotherapy, 11, 368–375. [Google Scholar]
- Briere J, & Runtz M (2002). The Inventory of Altered Self-Capacities (IASC): A standardized measure of identity, affect regulation, and relationship disturbance. Assessment, 9, 230–239. [DOI] [PubMed] [Google Scholar]
- Bunting L, & McAuley C (2004). Research review: Teenage pregnancy and parenthood: The role of fathers. Child and Family Social Work, 9, 295–303. [Google Scholar]
- Capaldi DM, Knoble NB, Shortt JW, & Kim HK (2012). A systematic review of risk factors for intimate partner violence. Partner Abuse, 3, 231–80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Clark LE, Allen RH, Goyal V, Raker C, & Gottlieb AS (2014). Reproductive coercion and co-occurring intimate partner violence in obstetrics and gynecology patients. American Journal of Obstetrics and Gynecology, 210, e1–e8. [DOI] [PubMed] [Google Scholar]
- Cohen J (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates. [Google Scholar]
- Conners NA, Bradley RH, Mansell LW, Liu JY, Roberts TJ, Burgdorf K, & Herrell JM (2003). Children of mothers with serious substance abuse problems: An accumulation of risks. American Journal of Drug and Alcohol Abuse, 29, 743–758. [DOI] [PubMed] [Google Scholar]
- Covington DL, Justason BJ, & Wright LN (2001). Severity, manifestations, and consequences of violence among pregnant adolescents. Journal of Adolescent Health, 28, 55–61. [DOI] [PubMed] [Google Scholar]
- Doss BD, Rhoades GK, Stanley SM, & Markman HJ (2009). The effect of the transition to parenthood on relationship quality: An 8-year prospective study. Journal of Personality and Social Psychology, 96, 601–619. doi: 10.1037/a0013969 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ehrensaft MK, Cohen P, Brown J, Smailes E, Chen H, & Johnson JG (2003). Intergenerational transmission of partner violence: A 20-year prospective study. Journal of Consulting and Clinical Psychology, 71, 741–753. [DOI] [PubMed] [Google Scholar]
- Fixsen DL, Naoom SF, Blasé KA, Friedman RM, & Wallace F (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute. [Google Scholar]
- Florsheim P, McArthur L, Hudak C, Heavin S, & Burrow-Sanchez J (2011). The Young Parenthood Program: Preventing intimate partner violence between adolescent mothers and young fathers. Journal of Couple Relationship Therapy, 10, 117–134. [Google Scholar]
- Foshee VA, Bauman KE, Arriaga XB, Helms RW, Koch GG, & Linder GF (1998). An evaluation of Safe Dates, an adolescent dating violence prevention program. American Journal of Public Health, 88, 45–50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Foshee V, Bauman KE, Ennett ST, Suchindran C, Benefield T, & Linder GF (2005). Assessing the effects of the dating violence prevention program “Safe Dates” using random coefficient regression modeling. Prevention Science, 6, 245–258. [DOI] [PubMed] [Google Scholar]
- Foshee VA, Linder GF, Bauman KE, Langwick SA, Arriaga XB, Heath JL, McMahon PM, & Bangdiwala S (1996). The Safe Dates Project: Theoretical basis, evaluation design, and selected baseline findings. American Journal of Preventive Medicine, 12, 39–47. [PubMed] [Google Scholar]
- Foshee V, & Reyes HLM (2009). Primary prevention of adolescent dating abuse: When to begin, whom to target, and how to do it. In Whitaker DJ & Lutzker JR (Eds.), Preventing partner violence: Research and evidence-based intervention strategies (pp. 141–168). Washington, DC: American Psychological Association. [Google Scholar]
- Galambos NL, Petersen AC, Richards M, & Gitelson IB (1985). The Attitudes Towards Women Scale for Adolescents (AWSA): A study of reliability and validity. Sex Roles, 13, 343–356. [Google Scholar]
- Harrykissoon SD, Rickert VI, & Wiemann CM (2002). Prevalence and patterns of intimate partner violence among adolescent mothers during the postpartum period. Archives of Pediatric and Adolescent Medicine, 156, 325–330. [DOI] [PubMed] [Google Scholar]
- Herrman JW (2013). How teen mothers describe dating violence. Journal of Obstetric, Gynecologic, and Neonatal Nursing, 42, 462–470. [DOI] [PubMed] [Google Scholar]
- Herrman JW, Palen L-A, Kan M, Feinberg M, Hill J, Magee E, & Haigh KM (2019). Young mothers’ and fathers’ perceptions of relationship violence: a focus group study. Violence Against Women, 25(3), 274–296. 10.1177/1077801218780356. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Herrman JW, & Waterhouse JK (2014). A feasibility study to assess the effectiveness of Safe Dates for teen mothers. Journal of Obstetric, Gynecologic, & Neonatal Nursing, 43, 695–709. [DOI] [PubMed] [Google Scholar]
- Kennedy AC (2006). Urban adolescent mothers exposed to community, family, and partner violence: Prevalence, outcomes, and welfare policy implications. American Journal of Orthopsychiatry, 76, 44–54. [DOI] [PubMed] [Google Scholar]
- Kennedy AC, & Bennett L (2006). Urban adolescent mothers exposed to community, family, and partner violence: Is cumulative violence exposure a barrier to school performance and participation? Journal of Interpersonal Violence, 21, 750–773. doi: 10.1177/0886260506287314 [DOI] [PubMed] [Google Scholar]
- Kitzmann KM, Gaylord NK, Holt AR, & Kenny ED (2003). Child witnesses to domestic violence: A meta-analytic review. Journal of Consulting and Clinical Psychology, 71, 339–352. [DOI] [PubMed] [Google Scholar]
- Langhinrichsen-Rohling J, & Turner LA (2012). The efficacy of an intimate partner violence prevention program with high-risk adolescent girls: A preliminary test. Prevention Science, 13, 384–394. [DOI] [PubMed] [Google Scholar]
- Lindhorst T, & Oxford M (2008). The long-term effects of intimate partner violence on adolescent mothers’ depressive symptoms. Social Science and Medicine, 66, 1322–1333. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lopez FG, Morúa W, & Rice KG (2007). Factor structure, stability, and predictive validity of college students’ relationship self-efficacy beliefs. Measurement and Evaluation in Counseling and Development, 40, 80–96. [Google Scholar]
- Milan SJ, & Lewis JB, Ethier K, Kershaw T, & Ickovics J (2005). Relationship violence among adolescent mothers: Frequency, dyadic nature, and implications for relationship dissolution and mental health. Psychology of Women Quarterly, 29, 302–312. [Google Scholar]
- Moore DR, & Florsheim P (2008). Interpartner conflict and child abuse risk among African American and Latino adolescent parenting couples. Child Abuse and Neglect, 32, 463–475. [DOI] [PubMed] [Google Scholar]
- Nation M, Crusto C, Wandersman A, Kumpfer KL, Seybolt D, Morrissey-Kane E, & Davino K (2003). What works in prevention. Principles of effective prevention programs. American Psychologist, 58, 449–456. [DOI] [PubMed] [Google Scholar]
- Newman BS & Campbell C (2011). Intimate partner violence among pregnant and parenting Latina adolescents. Journal of Interpersonal Violence, 26, 2635–2657. [DOI] [PubMed] [Google Scholar]
- Niolon PH, Kearns M, Dills J, Rambo K, Irving S, Armstead T, & Gilbert L (2017). Preventing intimate partner violence across the lifespan: A technical package of programs, policies, and practices. Atlanta, GA: National Center for Injury Prevention and Control, Centers for Disease Control and Prevention. [Google Scholar]
- Parker B, MacFarlane J, & Soeken K (1994). Abuse during pregnancy: Effects on maternal complications and birth weight in adult and teenage women. Obstetrics and Gynecology, 84, 323–328. [PubMed] [Google Scholar]
- Picard P (2007). Tech Abuse in Teen Relationships Study. Retrieved from http://www.loveisrespect.org/wp-content/uploads/2009/03/liz-claiborne-2007-tech-relationship-abuse.pdf
- Raphael J (2006). Teens having babies: The unexplored role of domestic violence. The Prevention Researcher, 12, 15–17. [PubMed] [Google Scholar]
- Riger S, Ahrens C, & Blickenstaff A (2000). Measuring interference with employment and education reported by women with abusive partners: Preliminary data. Violence and Victims, 15, 161–172. [PubMed] [Google Scholar]
- Roberts TA, Auinger P, & Klein JD (2006). Predictors of partner abuse in a nationally representative sample of adolescents involved in heterosexual dating relationships. Violence and Victims, 21, 81–90. [PubMed] [Google Scholar]
- Shepard MF, & Campbell JA (1992). The Abusive Behavior Inventory: A measure of psychological and physical abuse. Journal of Interpersonal Violence, 7, 291–305. [Google Scholar]
- Silverman JG, Raj A, & Clements K (2004). Dating violence and associated sexual risk and pregnancy among adolescent girls in the United States. Pediatrics, 114(2), e220–e225. [DOI] [PubMed] [Google Scholar]
- Stith SM, Smith DB, Penn CE, Ward DB, & Tritt D (2004). Intimate partner physical abuse perpetration and victimization risk factors: a meta-analytic review. Aggression and Violent Behavior, 10, 65–98. [Google Scholar]
- Straus MA, Hamby SL, Boney-McCoy S, & Sugarman DB (1996). The Revised Conflict Tactics Scales (CTS2): Development and preliminary psychometric data. Journal of Family Issues, 17, 283–316. doi: 10.1177/019251396017003001 [DOI] [Google Scholar]
- Taylor CA, Guteman NB, Lee SJ, & Rathouz JO (2009). Intimate partner violence, maternal stress, nativity, and risk for maternal maltreatment of young children. American Journal of Public Health, 99, 175–183. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Vagi KJ, Rothman EF, Latzman NE, Tharp AT, Hall DM, & Breiding MJ (2013). Beyond correlates: A review of risk and protective factors for adolescent dating violence perpetration. Journal of Youth and Adolescence, 42, 633–649. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Van Camp T, Hébert M, Guidi E, Lavoie F, & Blais M (2014). Teens’ self-efficacy to deal with dating violence as victim, perpetrator or bystander. International Review of Victimology, 20, 289–303. doi: 10.1177/0269758014521741 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Weinstein N (1988). Effects of personal experience on self-protective behavior. Psychological Bulletin, 105, 31–50. [DOI] [PubMed] [Google Scholar]
- Wingood GM, & DiClemente RJ (2008). The ADAPT-ITT model: A novel method of adapting evidence-based HIV interventions. Journal of Acquired Immune Deficiency Syndromes, 47(Suppl 1), S40–S46. doi: 10.1097/QAI.0b013e3181605df1 [DOI] [PubMed] [Google Scholar]
