Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Jan 1.
Published in final edited form as: Infant Ment Health J. 2018 Dec 21;40(1):54–66. doi: 10.1002/imhj.21758

Developmental Origins Perspective on the Emergence of Violent Behavior in Males with Prenatal Substance Exposure

Sarah Terrell 1, Elisabeth Conradt 1, Lynn Dansereau 2, Linda LaGasse 2, Barry Lester 2
PMCID: PMC6335163  NIHMSID: NIHMS965544  PMID: 30576590

Abstract

Children with prenatal substance exposure are at increased risk for externalizing behavior problems and violence. However, the contribution of early life experiences for placing these individuals at risk is not well understood. Utilizing a sample of 1,388 children with prenatal substance exposure from the Maternal Lifestyle Study, we attempt to shed light on these contributing factors by examining the impact of infant temperament, maternal sensitivity, and early life stress on the expression of violent behavior at ages 12 through 14. Males may be more at risk for increases in violent behavior in early adolescence through a number of early life experiences, such as variability in responses to maternal flexibility and engagement related to individual differences in temperament, as well as exposure to early adversity. Comparing two prevailing developmental theoretical frameworks, deficit models and differential susceptibility, we aim to understand the developmental origins of violent behavior in males by identifying children who may be most susceptible to early caregiving experiences.

With more than 1.2 million violent crimes occurring in the United States in 2016 alone (Federal Bureau of Investigation [FBI], 2017), violent behavior poses a significant problem for society. Efforts towards understanding and treating violence have largely been geared towards adolescent and adult criminal populations, at a time when behavior may be more resistant to treatment. Studying the early-in-life origins of violence may provide insight into developmental factors that contribute to violent proclivities and pave the way for research on higher efficacy prevention and intervention programs. One population that may be at an elevated risk for violence, in part due to their exposure to high levels of early life stress, is children with prenatal substance exposure. However, these children may differ in their susceptibility to early life experiences. Considering biological predispositions in combination with the early environment may provide more insight into developmental pathways that lead to violence among children with prenatal substance exposure.

Prenatal substance exposure and violent behavior outcomes

Prenatal substance exposure is associated with detrimental effects on brain development (e.g., Thompson, Levitt, & Stanwood, 2009; Rivkin et al., 2008), which may be one mechanism underlying risk for later behavior problems, including aggression. Another contributing factor may be that these children are also exposed to high levels of early life stress, which is by itself a risk factor for maladaptive outcomes (e.g., Shonkoff et al., 2012). Children with prenatal substance exposure are disproportionately more likely to have a caretaker who is depressed, be exposed to a violent home and neighborhood, and experience changes in caregiving (Bada et al., 2011). The combination of prenatal substance exposure and early adversity amplifies the likelihood of exhibiting increases in behavioral dysregulation over time, including increases in internalizing and externalizing symptoms and delinquent acts (Fisher et al., 2011). There are two dominant theoretical frameworks that have been used to examine how prenatal substance exposure could lead to risk for a wide range of poor outcomes: the deficit model and the differential susceptibility hypothesis.

Deficit model

Deficit models (e.g., cumulative risk: Evans, Li, & Whipple, 2013; Lupien et al., 2006; McEwen, 1998; McEwen & Stellar, 1993) have been the prevailing framework for studying the development of psychopathology. These theoretical approaches examine the effects of stress from the perspective that conditions of adversity make one vulnerable to disease-based outcomes, such as deficits in cognition and behavior. Deficit models infer that treatment methods should mend resulting impairments and intervention techniques should strive to improve damaging environmental conditions.

There is an abundant literature demonstrating that children with prenatal substance exposure are more at risk for later behavior problems and deficits in cognitive functioning, language development, and academic achievement (for extensive reviews, see Behnke & Smith, 2013; Lester & LaGasse, 2010). Extant studies tend to examine prenatal substance exposure from a deficit model, considering prenatal substance exposure as a vulnerability that would add to the developing child’s risk of negative outcomes, such as cognitive and emotional developmental impairments. For example, prenatal substance exposure has been predictive of impaired newborn neurobehavioral outcomes, lower IQ, and less adaptive behavior (Liu et al., 2010). Many studies have linked prenatal substance exposure to problem behavior such as impulsivity, inattention, and hyperactivity (Goldschmidt, Day, & Richardson, 2000; Nanson & Hiscock, 2003; Rosen & Johnson, 1985; Thapar et al., 2003) as well as externalizing behavior, disrupted school experiences, delinquent and criminal behavior, and high rates of substance abuse (Bada et al. 2007; Streissguth et al., 2004). Prenatal exposure to alcohol has been related to externalizing, aggressive, and delinquent behavior (Sood et al., 2001), whereas tobacco exposure has been connected to conduct disorders and violent offending (Brennan, Grekin, & Mednick, 1999; Olds et al., 1997).

This model has been beneficial in helping to chart the behavioral and cognitive trajectories of children exposed to substances prenatally, as well as understanding deleterious effects on the brain. This model has also been useful as a framework in research that aims to identify characteristics of resilient individuals, or those seemingly less impacted by environmental stressors, with hopes of enhancing these protective factors in vulnerable persons. However, a newer theoretical framework suggests that there may be more complexity in understanding how individuals respond to environmental conditions.

Differential susceptibility hypothesis

The differential susceptibility hypothesis purports that biological factors (i.e., temperament, genes, physiological reactivity) predispose individuals to be more susceptible to their environment, “for better or for worse” (Belsky, Bakermans-Kranenburg, & van Ijzendoorn, 2007). In other words, there may be some children that are more susceptible to early life stress or prenatal substance exposure than others. It therefore may be better to think of children previously viewed as “vulnerable” as “susceptible,” or displaying higher degrees of plasticity (Ellis et al., 2011). These children may be more responsive to their environments, showing the for worse outcomes when exposed to early adversity and the for better outcomes when in supportive environments. A separate distinct group of children seems insensitive to environmental stress and has similar behavioral outcomes regardless of rearing conditions (Belsky & Pluess, 2009; Conradt, Measelle, & Ablow, 2013).

Studies have begun to explore differences in susceptibility in children with prenatal substance exposure (Conradt et al., 2016; Derauf et al., 2011). We know that children with prenatal substance exposure are more likely to be born into adverse environments with higher levels of early life stress compared to their non-exposed counterparts; understanding how children differ in their responses to these stressors, for better or for worse, may provide insights into why some children are at a higher risk for developing violent propensities. There are three main indices of susceptibility: genetic susceptibility, behavioral susceptibility (e.g., temperament), and physiological susceptibility (Belsky & Pluess, 2009). For example, Conradt and colleagues, in a study from this current sample, examined the physiological index of susceptibility in infants at one month of age. They found that infants who exhibited more respiratory sinus arrhythmia reactivity and who were raised by a caregiver who experienced high levels of stress and psychopathology exhibited more behavior dysregulation at age 3. In comparison, infants who were less physiologically reactive and who had less stressed caregivers exhibited less behavior dysregulation at age 3 (Conradt et al., 2016).

There have been few tests of behavioral indices of susceptibility in children with prenatal substance exposure. This is surprising, given that one of the most replicated susceptibility factors is a difficult, or negative, infant temperament (Belsky & Pluess, 2009). When children with difficult temperaments are raised in supportive, nurturing environments they sometimes have the best outcomes, surpassing the outcomes of children with easy temperaments reared in similar conditions (Belsky & Pluess, 2009). However, infants with difficult temperaments reared in high stress conditions have been shown to have the worst outcomes and have been shown to be at higher risk for developing deficits in academic and social competencies, teacher-child relations, peer status, and externalizing behavior (Belsky & Pluess, 2009; Bradley & Corwyn, 2008; Dopkins-Stright, Gallagher, & Kelley, 2008; Pluess & Belsky, 2009; van Aken et al., 2007). For instance, van Aken et al. (2007) found that 16 to 19-month-old boys who were temperamentally difficult were more susceptible to maternal insensitivity, hostility, and intrusiveness, which resulted in increases of externalizing behavior. Similarly, Bradley & Corwyn (2008) found that six-year-old boys and girls with difficult temperaments were more susceptible to maternal sensitivity, showing the highest levels of externalizing behavior with insensitive parenting but the lowest levels of externalizing behavior with sensitive parenting.

These examples point to the importance of considering the interaction between child temperament and parenting factors. Most research to date is built on the premise that the effects of parenting are equal for children, which neglects that the effectiveness of parenting may depend on the bidirectional relationship between individual child traits (such as temperament) and parenting style (Belsky, Bakermans-Kranenburg, & van Ijzendoorn, 2007). These findings support the differential susceptibility hypothesis in that some children may be more susceptible to the effects of parenting, such as maternal sensitivity, than others.

Since males are responsible for a significantly disproportionate amount of violent behavior, it is critical to consider that males may be more sensitive to these developmental experiences earlier in life. There may be sex differences in infant temperament, as found by Weinberg and Tronick (1999). At six months of age boys were more likely to express anger than girls and depended on feedback from mothers to regulate emotions, whereas girls showed more self-regulation abilities. Boys were also found to be more emotionally expressive than girls, showing higher rates of both positive and negative emotions. In addition, more negative interactions are observed between mother-son dyads (as a pose to mother-daughter dyads) when mothers experience high levels of depressive symptoms (Weinberg, Olson, Beeghley, & Tronick, 2006). These findings suggest that males may be particularly susceptible to the influence of maternal mood.

Exploring for whom prenatal substance exposure may be more likely to lead to violent behavior is necessary to account for the gaps in current literature. One important gap in the prenatal substance exposure literature concerns the need to account for the effects of early life stress—such as those from poverty, maternal depression, parental absence, marital conflict and so on--in addition to the effects of the drug on violent behavior. It is difficult to tease apart the effects of early life stress and prenatal substance exposure without the use of large sample sizes which are powered to detect the differential effects of the substance above and beyond early life stress exposure. Second, many studies examine prenatal substance exposure retrospectively, limiting the reliability of the reporting. Third, examining indices of susceptibility in samples of children with prenatal substance exposure may further our understanding of why only a subset of at-risk individuals exposed to similar circumstances go on to commit violent offenses.

Present Study

This study will examine sex differences in violent behavior outcomes for individuals with prenatal substance exposure. First, we will document changes in violent behavior across development, separately for males and females. Second, we will examine early life predictors of violence for both males and females. Deviating from a deficit model, where adverse experiences lead to accumulated risk for poor developmental outcomes, and shifting towards understanding how individuals differ in their susceptibilities to early life experiences, may advance the understanding of violent behavior. Utilizing a sample of 1,388 children with prenatal substance exposure from the Maternal Lifestyle Study, we will attempt to advance understanding on how males may be more susceptible to developmental experiences that place them at risk for violent outcomes.

Method

Study participants and sites

The Maternal Lifestyle Study is a multi-site, longitudinal study designed to examine the effects of prenatal cocaine exposure and associated stressors on child development. Participating study locations were Brown University (Providence, RI), University of Miami (Miami, FL), University of Tennessee (Memphis, TN), and Wayne State University (Detroit, MI). Mother-infant dyads were recruited for participation in the hospital following delivery. For details regarding inclusionary and exclusionary criteria, see Lester et al., 2002. Mothers were screened for illicit drug use (tobacco, marijuana, alcohol, cocaine, opiates) during pregnancy through meconium assays and self-report. Of 19,079 participants screened, 16,988 were found to be eligible for the study with 11,811 consenting for participation. Mothers were approached in the hospital following delivery, informed consent was obtained, and mothers were interviewed for history of smoking, alcohol use, and drug use during pregnancy and in the last year. Meconium was collected from the newborn in order to conduct toxicology screens for cocaine or opiate metabolites. Families were selected for the exposed group (i.e., maternal report of cocaine or opiate use during pregnancy or gas chromatography–mass spectrometry confirmation of presumptive positive meconium screens for cocaine or opiate metabolites) or the comparison group (i.e., maternal denial of cocaine or opiate use during the pregnancy and a negative enzyme multiplied immunoassay meconium screen for cocaine and opiate metabolites). If a woman denied use of cocaine during pregnancy but mass spectrometry results were positive for cocaine exposure then the child was considered cocaine-exposed. Exposed and comparison newborns were group matched on race, sex, and gestational age within each study site. Background substances associated with cocaine use (alcohol, tobacco, and marijuana) were present in both groups; thus, most participants were polysubstance exposed. Enrollment in the longitudinal portion of the study began at one month (N=1,388) and continued through 16 years of age. Children participating in the study were identified by their parents as African American (77%), Caucasian (16%), Hispanic (6%), and other racial background (1%). A NIDA Certificate of Confidentiality was acquired by each study site to ensure confidentiality to participants regarding maternal drug use. All study sites were approved through the Institutional Review Board and participants were consented prior to participation.

Measures

Prenatal substance exposure was measured through self-report at the hospital post-delivery, where mothers were interviewed regarding their smoking, drinking, and substance use over the past year, including pregnancy. Toxicology screens were also completed on newborn meconium samples in order to screen for the presence of cocaine and opiate metabolites. The Caretaker Inventory of Substance Use (CISU; Shankaran et al., 1996), was administered when the child was four months of age. The CISU measures frequency and duration of use for tobacco, marijuana, alcohol, cocaine, and opiates. Since the study was designed to study the effects of cocaine exposure, 658 cocaine-exposed infants and 730 non-cocaine exposed infants were included. There is a high likelihood that women who report substance use during pregnancy can be characterized as polysubstance users (Birnbach et al., 2001); as such, many infants in both groups (cocaine exposed and non-cocaine exposed) were exposed to multiple substances such as tobacco, alcohol, marijuana, and opiates. Only a smaller group of infants (21.3%) was characterized as not having any prenatal substance exposure. Through utilization of these self-reports and meconium samples, substance use was quantified dichotomously (0=no, 1=yes) by assigning a point per affirmative response, which is a frequently used method to examine prenatal substance exposure outcomes (Fisher et al., 2011). A summative index was subsequently created with a 0–5 scale to indicate whether mothers had partaken in the following substances: tobacco, alcohol, marijuana, opiates, and cocaine.

Infant temperament was operationalized through a modified version of Rothbart’s Infant Behavior Questionnaire (IBQ; Kunkel & Brown, 1993; Rothbart, 1981), which was administered to the mother when child was four months of age. A variety of behavioral events was identified by a yes/no categorization; if yes, participants provided follow-up information on a five-point response scale. Summary scores were obtained for activity level, smiling and laughter, distress and latency to approach sudden or novel stimuli, distress to limitations, soothability, and duration of orienting. Factor analysis was utilized to generate an “easy” temperament profile (smiling and laughter, soothability, and duration of orienting) and a “difficult” temperament profile (distress and latency to sudden approach and novel stimuli, distress to limitations).

Maternal sensitivity at 1 month

Mother-infant dyads were recorded while completing a bottle-feeding task (LaGasse et al., 2003) at one month (42 to 46 weeks) of age; 46 mothers who exclusively breastfed were not invited to participate in this specific task. Mothers were asked to withhold feeding, when possible, until arrival at the clinic site. Videos were coded (coders were blinded to prenatal substance exposure status) for behaviors of maternal engagement and flexibility (the mother’s response to feeding cues from infant and quality of maternal attentiveness) as well as maternal stimulation (frequency of behaviors such as maternal vocalizations, caressing/stroking, kissing).

Early adversity was a summative risk index from birth to the age 3 assessments and included nine risk factors. Cumulative risk models assume that combinations of risk factors are more powerful predictors of developmental outcomes than the measurement of a single risk factor, which is less ecologically valid in substance-exposed populations (Lester et al., 2005). Each risk factor was either a continuous scale or a count score that was dichotomized to create an overall risk index (0 = no/none , or 1 = yes/one or more ). Cut-offs were based on prior research findings indicating that extreme values on these risk indices represent valid indicators of risk for developing problem behavior (Fisher et al., 2011; Sheinkopf et al., 2007): (1) any maternal report of postnatal substance use of cocaine, opiates, tobacco, alcohol, or marijuana up to the Year 3 assessment; (2) chronic poverty status calculated as income below $10K for at least 75% of the visits; (3) low social status scored from the Hollingshead Index of Social Position (Hollingshead, 1975) using education and occupation averaged over annual visits and scored as 1 SD or more below the mean; (4) any primary caretaker changes assessed annually; (5) any report of sexual or physical abuse reported by caregivers; (6) annual assessments of caregiver depression of 1 SD or greater above the mean for averaged depressive symptoms on the caregiver reported Beck Depression Inventory (Beck & Steer 1993); (7) annual assessments of caregiver psychological distress of 1 SD or greater above the mean for total psychological symptoms on the Brief Symptom Inventory (Derogatis 1993); (8) poor quality home environment of 1 SD or more below the mean on the Home Observation Measurement of the Environment (Caldwell & Bradley, 1984; Caldwell & Bradley, 2003) as assessed by a home visitor when the child was 10 months old; and (9) any history of Child Protective Services Involvement for the target child assessed annually by caregiver report until the Year 3 assessment.

Violent behavior at 12, 13, and 14 years

The outcome variable of violence was operationalized through use of Things That You Have Done (Elliot, Ageton, & Huizinga, 1985), which is a 27-item, child-report questionnaire measuring the child’s involvement in delinquent activities over the past year. We used the “crimes against others” subscale, which included hitting, slapping or shoving other kids or getting into a physical fight; threatening to hit someone to get something; attacking someone; throwing objects such as rocks and bottles at people; and having been involved in a gang fight. We examined violent behavior outcomes at ages 12, 13, and 14.

Statistical analyses

We first ran unconditional models predicting change in violent behavior from ages 12–14, separately for males and females. Model fit was evaluated based on the comparative fit index (CFI) and RMSEA. Covariates were selected based on published literature and statistical evidence. We included prenatal substance exposure (prenatal cocaine, alcohol, tobacco, opioid, and marijuana exposure) indices of early adversity, and study site in all analyses. Missing data (n = 177 boys and n = 135 girls) were accounted for using multiple imputation. All statistical analyses were conducted using Mplus. We tested for significant differences at p <.05.

Results

Preliminary analyses

Data were examined for outliers and violations of normality. Outliers above or below 3 standard deviations from the mean were winsorized by replacing the value with the value at 3 standard deviations. Means and standard deviations for demographic characteristics and all covariates and predictors, separately for males and females, are included in Table 1. We first examined whether there were differences in amount of violent acts, separately between males and females, at each age. Males committed significantly more violent acts at 12 (Mmales = .41, SDmales = .71; Mfemales = .27, SDfemales = .64; F (1, 994) = 10.48, p <.001), age 13 (Mmales = .69, SDmales = .98; Mfemales = .54, SDfemales = .94; F (1, 990) = 6.36, p = .01) but there were only marginally significant differences in violent behavior between males and females at age 14 (Mmales = .77, SDmales = .99; Mfemales = .67, SDfemales = .90; F (1, 992) = 3.02, p = .08).

Table 1.

Demographic characteristics of males vs females

Males (n = 550) Females (n = 526)
Prenatal exposure to cocaine (1 = exposed) .43 (.50) .45 (.50), ns
Prenatal exposure to opioids (1 = exposed) .08 (.27) .09 (.29), ns
Prenatal exposure to alcohol (1 = exposed) .59 (.49) .60 (.49), ns
Prenatal exposure to tobacco (1 = exposed) .51 (.50) .57 (.50), p = .02
Prenatal exposure to marijuana (1 = exposed) .21 (.41) .26 (.44), ns
Race: Black (%) 73.3% 80.2%, ns
Maternal education 1.83 (.76) 1.80 (.75), ns
Marital status 1.83 (.46) 1.85 (.44), ns
Maternal age (years) 28.40 (5.82) 28.28 (5.84), ns
Early adversity 2.19 (1.42) 2.29 (1.40), ns
Flexibility and engagement −.03 (1.01) .04 (.98), ns
Stimulation and activity level −.02 (1.06) .03 (.93), ns
Infant negative temperament −.09 (.95) .10 (1.05)
Infant easy temperament .008 (.99) −.009 (1.02)
Open in a new tab

Unconditional growth models

Male model

We first tested for growth in violent behavior in males from ages 12–14 sans covariates or predictors. We found that violent behavior in males significantly increased from ages 12–14 was stable (Mlinear slope = .18, p < .001). However, there was significant variability in the linear slope (σ2= .12, p <.001), indicating that some children’s violent behavior increased, some decreased, and some remained the same.

Female model

We also tested for unconditional growth in violent behavior in females from ages 12–14. We found that violent behavior in females from ages 12–14 significantly increased (Mlinear slope = .21, p <.001). However, there was significant variability in the linear slope (σ2= .17, p < .001), indicating that some female children’s violent behavior increased, some decreased, and some remained the same.

Early life predictors of violent behavior

Male model

We examined the following early-life predictors of violent behavior in males: prenatal exposure to cocaine, opiates, tobacco, alcohol, and marijuana, as well as early adversity, maternal flexibility and engagement, maternal stimulation and activity level, “difficult” temperament, and “easy” temperament. Given our interest in testing for differential susceptibility vs deficit model we also tested for significant interactions between a “difficult” temperament and maternal flexibility and engagement, as well as “difficult” temperament x maternal stimulation and activity level. The only significant predictor on the intercept, violent behavior at age 12, was a “difficult” temperament. High levels of “difficulty” were related to higher levels of violent behavior at age 12, b = .12, p = .05. None of the other main effects or interactions were significant.

These same variables were examined as predictors of growth in violent behavior in males from ages 12–14. Higher exposure to tobacco prenatally was related to decreases in violent behavior from ages 12–14 (b = −.16, p = .04). Higher levels of early adversity were related to increases in violent behavior, b = .19, p = .02. Again, there was a main effect of infant “difficult” temperament on increases in violent behavior, with higher difficult temperament predicting decreases in violent behavior, b = −.24, p = .004. However, this main effect was qualified by a significant difficult temperament x maternal flexibility and engagement interaction, b = −.05, p = .056.

We probed this interaction at ± 1SD from the mean for difficult temperament and maternal flexibility and engagement using the online computational tools provided by Preacher and colleagues (Preacher, Curran, & Bauer, 2006). For mothers with low levels of flexibility and engagement, there were no significant differences in change in violent behavior, regardless of infant difficult temperament (Figure 1a). Among mothers with high levels of flexibility and engagement, infants who were rated as less difficult exhibited increases in violent behavior over time (b = .33, p = .002), while infants rated high on negativity exhibited no changes in violent behavior over time (Figure 1b).

Figure 1.

Figure 1

Open in a new tab

The interaction between infant negativity, maternal flexibility and engagement, and time. Among mothers low on flexibility and engagement, there are no significant differences in violent behavior in males among infants, regardless of their level of temperamental negativity (1a). Among mothers high on flexibility and engagement, there are no significant differences in violent behavior in males over time among infants rated high on negativity. Among infants rated low on negativity, violent behavior increases over time (1b).

Female model

We examined the same early-life predictors of violent behavior in females. The only significant predictor on the intercept, violent behavior at age 12, was maternal flexibility and engagement. High levels of maternal flexibility and engagement were related to lower levels of violent behavior at age 12, b = −.13, p = .01. None of the other main effects or interactions were significant.

These same variables were examined as predictors of growth in violent behavior in females from ages 12–14. Higher levels of prenatal opioid exposure were related to increases in violent behavior from ages 12–14, b = .14, p = .01. Difficult temperament was also significant, with higher levels of difficult temperament predicting increases in violent behavior in females from ages 12–14, b = .15, p = .04. None of the interactions were significant.

Discussion

Utilizing a high-risk population with prenatal substance exposure, we examined infant temperament, caregiving interactions, and early life stress exposure in an attempt to identify the early childhood experiences that contribute to violent outcomes from age 12 to 14. Leveraging a sample of over 1,300 children with prenatal substance exposure assessed from 1 month to 16 years, we had the power to detect whether these early childhood experiences and exposures were related to violent behavior almost one decade later. While there were some exceptions (e.g., male prenatal exposure to tobacco), the findings tended to lend support to the deficit model as exposure to challenges early in life resulted in greater risk for violent behavior outcomes. The results also indicate that some boys, based on their temperamental traits, were more sensitive to the effects of maternal caregiving than others; however, our findings did not meet evidentiary criteria for establishing a case of differential susceptibility (Belsky, Bakermans-Kranenburg, & van Ijzendoorn, 2007).

Overall, males committed more violent acts than females at all ages, and violent behavior increased from ages 12 to 14 for both boys and girls. These findings are not surprising given what is known about developmental trends in violent behavior. Nagin and Tremblay (1999) examined trajectories of externalizing behavior in males and found that physical aggression typically decreases from school-entry until adolescence, where there is a surge in aggression. Research to date has distinguished between an adolescent limited aggression and life-course persistent violent behavior (Moffitt 1993), so these adolescent outcomes may not necessarily be predictive of adult offending.

Higher levels of tobacco exposure were related to decreased levels of violent behavior from 12–14 in males. This finding was surprising as it is inconsistent with literature that finds prenatal tobacco exposure to increase risk of externalizing behavior, including conduct disorder and violent behavior outcomes. Higher levels of prenatal opioid exposure were related to increases in violent behavior from ages 12–14 in females. This contributes to a growing body of literature on the effects of prenatal opioid exposure at a time when such information is critical given the current opioid epidemic. This is the first study to find that prenatal opioid exposure occurring in the 1990s, that included both heroin and methadone exposure, is predictive of violent behavior 12–14 years later, but only in females.

One mechanism linking prenatal substance exposure and violent outcomes may be neurobehavioral disinhibition. Originally developed by Tarter (2003) in an effort to characterize a group of children at risk for delinquency later in life, neurobehavioral disinhibition is an intricate disinhibitory psychopathology (Iacono, Malone, & McGue, 2008) that cannot be distinguished by a single diagnosis. There are characteristics of neurobehavioral disinhibition that have been related to risk for violent behavior, including disruptive behavior disorders, executive functioning difficulties, and poor self-regulation (Fisher et al., 2011). As one example, Lambert and colleagues (2013) found that executive dysfunction partially mediated the relationship between prenatal substance exposure and being arrested at age 15.

In males, but not females, higher levels of infant negativity/difficulty were related to higher levels of violence at age 12. As expected from the review of previous literature on temperament, difficult temperament could be related to a heightened risk for poor behavioral outcomes (Patterson, 1976). High levels of infant negativity were also predictive of growth in violent behavior from 12–14 in both males and females. For females, the direction of effect was as hypothesized: greater infant negativity at 4 months of age was related to increases in violent behavior from ages 12–14. Males showed the opposite pattern of results. Greater infant negativity was associated with lower levels of violent behavior from ages 12–14. However, for males only, this main effect was qualified by a significant interaction with maternal caregiving behavior at 1 month.

For males, there appeared to be no difference in growth in violent behavior if their mothers were less flexible and engaged when interacting with them at 1 month. Both temperament groups showed increases in violent behavior from ages 12–14. On the other hand, males with less difficult temperaments displayed increases in violent behavior over time, but only in combination with high maternal flexibility and engagement. Males with more difficult temperament showed no increases in violent behavior over time if their mothers were more flexible and engaged when interacting with them at 1 month. For females, there were no significant interactions, but high levels of flexibility and engagement were related to lower levels of violent behavior at age 12.

These findings indicate that male infants who exhibit easy temperaments at 4 months may be more sensitive to maternal parenting practices characterized by flexibility and engagement. It may be that males are more vulnerable to the amount of stimulation in their early rearing environment. This finding contrasts with previous research that tends to show that individuals (male or female) with easy temperaments are less susceptible to environmental experiences (Belsky & Pluess, 2009; Bradley & Corwyn, 2008; Dopkins-Stright, Gallagher, & Kelley, 2008; Pluess & Belsky, 2009; van Aken et al., 2007). However, for a time period, socio-cultural concerns led many developmental researchers to neglect examining sex differences in datasets, therefore failing to account for possible individual differences.

One explanation of these findings is that males with less difficult temperaments may be exhibiting lower levels of arousal which may be overstimulated by high levels of maternal attention (flexibility and engagement). If this overstimulation persists it may manifest as externalizing behavior problems, which could, in turn, give rise to aggressive behavior. An alternative explanation is in accordance with the sensation seeking theory literature. This theory suggests that individuals low on arousal may prefer a stronger level of stimulation to be aroused by a stimulus, thus increasing their chances of engaging in risky behavior (Zuckerman, 2009).

Another strong predictor of increases in violent behavior in males was exposure to early adversity. Early life stress from birth to age 3 was related to increases in violent behavior from ages 12–14 in males but not females. The most comprehensive theory to date on the origins of sex differences in exposure to early life adversity comes from Sandman, Glynn, and Davis who propose that there could be a viability-vulnerability trade-off that contributes to sex differences in developmental outcomes. In other words, Sandman and colleagues (2013) argue that exposure to early adversity in males threatens their viability, or risk for early mortality. Females are more likely to survive following exposure to early adversity but that may lead to increased vulnerability for affective and anxiety disorders later in life (Sandman, Glynn, & Davis, 2013). We did not find support for this theory in this study, given that it appeared as though boys were more susceptible to the effects of both early adversity and maternal caregiving behaviors.

Instead the vulnerability for males may be vulnerability for committing violent behavior in early adolescence. In this sample males were significantly more likely to commit violent acts than females, which is consistent with overall sex differences in violent crime rates. Our results suggest that one important pathway to violent behavior included both infant temperament and maternal caregiving behavior. These findings require replication before firm conclusions can be drawn but these findings suggest that both infant temperament and caregiving behavior should be assessed by early interventionists motivated to prevent externalizing behavior.

There are limitations of this research. First, we cannot determine direction of effect with these data. We can in no way conclude, for example, that temperamental difficulty at 4 months causes increases or decreases in violent behavior in adolescence. Another limitation includes the assessment of maternal engagement and flexibility during a feeding task. Mothers who breastfed were not invited to participate. Since mothers who elect to breastfeed their infants tend to show higher rates of maternal sensitivity (e.g., Britton, Britton, & Gronwaldt, 2006; Tharner et al, 2012), the task may be limited in the range of maternal behaviors observed.

It is important to consider that processes outside the scope of the study could contribute to the findings presented here. For example, gene by environment interactions were not examined. In addition, women in this sample were not assessed prenatally. Developmental programming, including during the prenatal period, may have explanatory power for the developmental origins of violent behavior in children with prenatal substance exposure. In 2009 Lester and Padbury proposed three pathways through which prenatal substance exposure may impact fetal development: neurochemical mechanisms, vasoconstrictive mechanisms, and prenatal programming. Prenatal programming theory proposes that maternal environment and experiences can alter the uterine environment, triggering epigenetic changes that prime the developing infant for what the external world will be like. Developmental programming continues throughout the lifespan, with the early years of life likely playing an important role. When considering children with prenatal substance exposure and high levels of early life stress, programming effects may explain variance in how children respond to early life experiences and how this relates to their subsequent developmental outcomes.

Our findings underscore the importance of examining individual differences in combination with early developmental experiences to understand the mechanisms that lead to behavioral problems in adolescence. Since biological risk factors in combination with environmental conditions often provide a more complete picture than examining main effects alone, it is imperative to consider these interactions when identifying novel treatment and intervention methods (Beauchaine et al. 2008).

Acknowledgments

This work was supported by NIH grants: National Institute of Child Health and Human Development (NICHD) Neonatal Research Network and an inter-institute agreement with the National Institute on Drug Abuse (NIDA) through cooperative agreements: U10-DA-024119-01 and U10-HD-27904 (to B.M.L.); NICHD contract N01- HD-2-3159 (to B.M.L); 1RO1DA014918 (to L.L.); and a Career Development Award from the National Institute on Drug Abuse 7K08DA038959-02 (to E.C.). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Mental Health, National Institute on Drug Abuse, or the National Institutes of Health.

References

  1. Achenbach TM, Edelbrock CHCT. Empirically based assessment of the behavioral/emotional problems of 2- and 3- year old children. Journal of Abnormal Child Psychology. 1987;15(4):629–650. doi: 10.1007/BF00917246. [DOI] [PubMed] [Google Scholar]
  2. Bada HS, Bann CM, Bauer CR, Shankaran S, Lester B, LaGasse L, … Higgins R. Preadolescent behavior problems after prenatal cocaine exposure: Relationship between teacher and caretaker ratings (Maternal Lifestyle Study) Neurotoxicology and Teratology. 2011;33(1):78–87. doi: 10.1016/j.ntt.2010.06.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bada HS, Das A, Bauer CR, et al. Impact of prenatal cocaine exposure on child behavior problems through school age. Pediatrics. 2007;199(2) doi: 10.1542/peds.2006-1404. Available at: www.pediatrics.org/cgi/content/full/119/2/e348. [DOI] [PubMed] [Google Scholar]
  4. Beauchaine TP, Neuhaus E, Brenner SL, Gatzke-Kopp L. Ten good reasons to consider biological processes in prevention and intervention research. Development and Psychopathology. 2008;20(3):745–774. doi: 10.1017/S0954579408000369. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Beck AT, Steer RA. Beck Depression Inventory. San Antonio, TX: Psychological Corporation; 1993. [Google Scholar]
  6. Behnke M, Smith VC. Prenatal substance abuse: Short- and long-term effects on the exposed fetus. Pediatrics. 2013;131(3):e1009–e1024. doi: 10.1542/peds.2012-3931. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Belsky J, Bakermans-Kranenburg MJ, van Ijzendoorn MH. For better and for worse: Differential susceptibility to environmental influences. Current Directions in Psychological Science. 2007;16(6):300–304. [Google Scholar]
  8. Belsky J, Pluess M. Beyond diathesis stress: Differential susceptibility to environmental influences. Psychological Bulletin. 2009;135(6):885–908. doi: 10.1037/a0017376. [DOI] [PubMed] [Google Scholar]
  9. Birnbach DJ, Browne IM, Kim A, Stein DJ, Thys DM. Identification of polysubstance abuse in the parturient. BJA: British Journal of Anaesthesia. 2001;3(87):488–490. doi: 10.1093/bja/87.3.488. [DOI] [PubMed] [Google Scholar]
  10. Bradley RH, Corwyn RF. Infant temperament, parenting, and externalizing behavior in first grade: A test of the differential susceptibility hypothesis. Journal of Child Psychology and Psychiatry. 2008;49:124–131. doi: 10.1111/j.1469-7610.2007.01829.x. [DOI] [PubMed] [Google Scholar]
  11. Brennan PA, Grekin ER, Mednick SA. Maternal smoking during pregnancy and adult male criminal outcomes. Archives of General Psychiatry. 1999;56(3):215–219. doi: 10.1001/archpsyc.56.3.215. [DOI] [PubMed] [Google Scholar]
  12. Britton JR, Britton HL, Gronwaldt V. Breastfeeding, sensitivity, and attachment. Pediatrics. 2006;118(5):e1436–e1443. doi: 10.1542/peds.2005-2916. [DOI] [PubMed] [Google Scholar]
  13. Caldwell BM, Bradley RH. Administration Manual (Revised Edition): Home observation for measures of the environment. Little Rock, AK: University of Arkansas; 1984. [Google Scholar]
  14. Caldwell BM, Bradley RH. HOME Inventory Administration Manual. 2. Little Rock, AK: University of Arkansas; 2003. [Google Scholar]
  15. Conradt E, Beauchaine T, Abar B, Lagasse L, Shankaran S, Bada H, … Lester B. Early caregiving stress exposure moderates the relation between respiratory sinus arrhythmia reactivity at 1 month and biobehavioral outcomes at age 3. Psychophysiology. 2016;53(1):83–96. doi: 10.1111/psyp.12569. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Conradt E, Measelle J, Ablow JC. Poverty, problem behavior, and promise: Differential susceptibility among infants reared in poverty. Psychological Science. 2013;24(3):235–242. doi: 10.1177/0956797612457381. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Derauf C, Lagasse L, Smith L, Newman E, Shah R, Arria A, … Lester B. Infant temperament and high-risk environment relate to behavior problems and language in toddlers. Journal of Developmental and Behavioral Pediatrics. 2011;32(2):125–135. doi: 10.1097/DBP.0b013e31820839d7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Derogatis LR. Brief Symptom Inventory (BSI): Administration, Scoring, and Procedures Manual. 3. Minneapolis, MN: National Computer Systems, Inc; 1993. [Google Scholar]
  19. Dopkins-Stright A, Gallagher KC, Kelley K. Infant temperament moderates relations between maternal parenting in early childhood and children’s adjustment in first grade. Child Development. 2008;79:186–200. doi: 10.1111/j.1467-8624.2007.01119.x. [DOI] [PubMed] [Google Scholar]
  20. Elliot DS, Ageton SS, Huizinga D. Explaining delinquency and drug use. Beverly Hills, CA: Siegel; 1985. [Google Scholar]
  21. Ellis BJ, Boyce WT, Belsky J, Bakermans-Kranenburg MJ, Van Ijzendoorn MH. Differential susceptibility to the environment: An evolutionary- neurodevelopmental theory. Development and Psychopathology. 2011;23(1):7–28. doi: 10.1017/S0954579410000611. [DOI] [PubMed] [Google Scholar]
  22. Evans GW, Li D, Whipple SS. Cumulative risk and child development. Psychological Bulletin. 2013;139(6):1342–1396. doi: 10.1037/a0031808. [DOI] [PubMed] [Google Scholar]
  23. Fisher PA, Lester BM, DeGarmo DS, Lagasse LL, Lin H, Shankaran S, … Higgins R. The combined effects of prenatal drug exposure and early adversity on neurobehavioral disinhibition in childhood and adolescence. Development and Psychopathology. 2011;23(3):777–788. doi: 10.1017/S0954579411000290. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Goldschmidt L, Day NL, Richardson GA. Effects of prenatal marijuana exposure on child behavior problems at age 10. Neurotoxicology and Teratology. 2000;22(3):325–336. doi: 10.1016/S0892-0362(00)00066-0. [DOI] [PubMed] [Google Scholar]
  25. Iacono WG, Malone SM, McGue M. Behavioral disinhibition and the development of early-onset addiction: Common and specific influences. Annual Review of Clinical Psychology. 2008;4(1):325–348. doi: 10.1146/annurev.clinpsy.4.022007.141157. [DOI] [PubMed] [Google Scholar]
  26. Kunkel D, Brown JV. Unpublished Masters Thesis. Georgia State University; 1993. Infant temperament: Is there agreement between caregivers and trained observers? [Google Scholar]
  27. LaGasse LL. Prenatal drug exposure and maternal and infant feeding behaviour. Archives of Disease in Childhood - Fetal and Neonatal Edition. 2003;88(5):391F–399. doi: 10.1136/fn.88.5.F391. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Lambert BL, Bann CM, Bauer CR, Shankaran S, Bada HS, Lester BM, … Higgins RD. Risk-taking behavior among adolescents with prenatal drug exposure and extrauterine environmental adversity. Journal of Developmental & Behavioral Pediatrics. 2013;34(9):669–679. doi: 10.1097/01.DBP.0000437726.16588.e2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Lester BM, LaGasse LL. Children of addicted women. Journal of Addictive Diseases. 2010;29(2):259–276. doi: 10.1080/10550881003684921. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Lester BM, Padbury JF. Third pathophysiology of prenatal cocaine exposure. Developmental Neuroscience. 2009;31(1–2):23–35. doi: 10.1159/000207491. [DOI] [PubMed] [Google Scholar]
  31. Lester BM, Tronick EZ, LaGasse L, Seifer R, Bauer CR, Shankaran S, … Maza PL. The maternal lifestyle study: Effects of substance exposure during pregnancy on neurodevelopmental outcome in 1-month-old infants. Pediatrics. 2002;110(6):1182–1192. doi: 10.1542/peds.110.6.1182. [DOI] [PubMed] [Google Scholar]
  32. Linares LO, Heeren T, Bronfman E, Zuckerman B, Augustyn M, Tronick E. A mediational model for the impact of exposure to community violence on early child behavior problems. Child Development. 2001;72(2):639–652. doi: 10.1111/1467-8624.00302. [DOI] [PubMed] [Google Scholar]
  33. Liu J, Bann C, Lester B, Tronick E, Das A, Lagasse L, … Bada H. Neonatal neurobehavior predicts medical and behavioral outcome. Pediatrics. 2010;125(1):e90–e98. doi: 10.1542/peds.2009-0204. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Lupien SJ, Ouellet-Morin I, Hupbach A, Tu MT, Buss C, Walker D, … McEwen BS. Beyond the stress concept: Allostatic load--a developmental biological and cognitive perspective. In: Cicchetti D, Cohen DJ, editors. Developmental psychopathology: Developmental neuroscience. Hoboken, NJ, US: John Wiley & Sons Inc; 2006. pp. 578–628. [Google Scholar]
  35. McEwen BS. Stress, adaptation, and disease: Allostasis and allostatic load. Annals of the New York Academy of Sciences. 1998;840(1):33–44. doi: 10.1111/j.1749-6632.1998.tb09546.x. [DOI] [PubMed] [Google Scholar]
  36. McEwen BS, Stellar E. Stress and the individual mechanisms leading to disease. Archives of Internal Medicine. 1993;153(18):2093–2101. doi: 10.1001/archinte.1993.00410180039004. [DOI] [PubMed] [Google Scholar]
  37. Nagin D, Tremblay RE. Trajectories of boys’ physical aggression, opposition, and hyperactivity on the path to physically violent and nonviolent juvenile delinquency. Child Development. 1999;70(5):1181–1196. doi: 10.1111/1467-8624.00086. [DOI] [PubMed] [Google Scholar]
  38. Nanson JL, Hiscock M. Attention deficits in children exposed to alcohol prenatally. Alcoholism: Clinical and Experimental Research. 1990;14:656–661. doi: 10.1111/j.1530-0277.1990.tb01223.x. [DOI] [PubMed] [Google Scholar]
  39. Olds D. Tobacco exposure and impaired development: A review of the evidence. Mental Retardation and Developmental Disabilities Research Reviews. 1997;3:257–269. doi: 10.1002/(SICI)1098-2779(1997)3:3&#x0003c;257::AID-MRDD6&#x0003e;3.0.CO;2-M. [DOI] [Google Scholar]
  40. Patterson GR. Behavior modification and families. In: Mash EJ, Hamerlynck LA, Handy LC, editors. The aggressive child: Victim and architect of a coercive system. New York, NY: Brunner/Mazel; 1976. pp. 267–316. [Google Scholar]
  41. Pluess M, Belsky J. Differential susceptibility to rearing experience: The case of childcare. Journal of Child Psychology and Psychiatry and Allied Disciplines. 2009;50(4):396–404. doi: 10.1111/j.1469-7610.2008.01992.x. [DOI] [PubMed] [Google Scholar]
  42. Preacher KJ, Curran PJ, Bauer DJ. Computational tools for probing interactions in multiple linear regression, multilevel modeling, and latent curve analysis. Journal of Educational and Behavioral Statistics. 2006;31(4):437–448. doi: 10.3102/10769986031004437. [DOI] [Google Scholar]
  43. Rivkin MJ, Davis PE, Lemaster JL, Cabral HJ, Warfield SK, Mulkern RV, … Frank DA. Volumetric MRI study of brain in children with intrauterine exposure to cocaine, alcohol, tobacco, and marijuana. Pediatrics. 2008;121(4):741–750. doi: 10.1542/peds.2007-1399. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Rose TS, Johnson HL. Long-term effects of prenatal methadone maintenance. NIDA Research Monograph. 1985;59:73–83. [PubMed] [Google Scholar]
  45. Rothbart MK. Measurement of temperament in infancy. Child Development. 1981;52:569–578. [Google Scholar]
  46. Sandman CA, Glynn LM, Davis EP. Is there a viability-vulnerability tradeoff? Sex differences in fetal programming. Journal of Psychosomatic Research. 2014;75(4):327–335. doi: 10.1016/j.jpsychores.2013.07.009.IS. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Shankaran S, Bauer CR, Bada HS, Lester B, Wright LL, Katsikiotis V. Maternal Lifestyle Study: Patterns of cocaine use in term pregnancy and effect on birth weight. Pediatric Research. 1996;39:279A. [Google Scholar]
  48. Sheinkopf SJ, LaGasse LL, Lester BM, Liu J, Seifer R, Bauer CR, … Das A. Vagal tone as a resilience factor in children with prenatal cocaine exposure. Development and Psychopathology. 2007;19:649–673. doi: 10.1017/S0954579407000338. [DOI] [PubMed] [Google Scholar]
  49. Shonkoff JP, Garner AS The Committee on Psychosocial Aspects of Child and Family Health, Committee on Early Childhood, Adoption, and Dependent Care and Section on Developmental and Behavioral Pediatrics. The lifelong effects of early childhood adversity and toxic stress. Pediatrics. 2012;129(1):232–246. doi: 10.1542/peds.2011-2663. PEDIATRICS. [DOI] [PubMed] [Google Scholar]
  50. Streissguth AP, Bookstein FL, Barr HM, Sampson PD, O’Malley K, Young JK. Risk factors for adverse life outcomes in fetal alcohol syndrome and fetal alcohol effects. Journal of Developmental & Behavioral Pediatrics. 2004;25(4):228–238. doi: 10.1097/00004703-200408000-00002. [DOI] [PubMed] [Google Scholar]
  51. Stright AD, Gallagher KC, Kelley K. Infant temperament moderates relations between maternal parenting in early childhood and children’s adjustment in first grade. Child Development. 2008;79:186–200. doi: 10.1111/j.1467-8624.2007.01119.x. [DOI] [PubMed] [Google Scholar]
  52. Sood B, Delaney-Black V, Covington C, Nordstrom-Klee B, Ager J, Templin T, … Sokol RJ. Prenatal alcohol exposure and childhood behavior at age 6 to 7 years: I. dose-response effect. Pediatrics. 2001;108(2):e34–e34. doi: 10.1542/peds.108.2.e34. [DOI] [PubMed] [Google Scholar]
  53. Tarter RE, Kirisci L, Mezzich A, Cornelius JR, Pajer K, Vanyukov M, … Clark D. Neurobehavioral disinhibition in childhood predicts early age at onset of substance use disorder. American Journal of Psychiatry. 2003;160(6):1078–1085. doi: 10.1176/appi.ajp.160.6.1078. [DOI] [PubMed] [Google Scholar]
  54. Thapar A, Fowler T, Rice F, Scourfield J, van den Bree M, Thomas H, Harold G, Hay D. Maternal smoking during pregnancy and attention deficit hyperactivity disorder symptoms in offspring. The American Journal of Psychiatry. 2003;160(11):1985–1989. doi: 10.1176/appi.ajp.160.11.1985. [DOI] [PubMed] [Google Scholar]
  55. Tharner A, Luijk MP, Raat H, Ijzendoorn MH, Bakermans-Kranenburg MJ, Moll HA, … Tiemeier H. Breastfeeding and its relation to maternal sensitivity and infant attachment. Journal of Developmental & Behavioral Pediatrics. 2012;33(5):396–404. doi: 10.1097/DBP.0b013e318257fac3. [DOI] [PubMed] [Google Scholar]
  56. Thompson BL, Levitt P, Standwood GD. Prenatal exposure to drugs: effects on brain development and implications for policy and education. Nature Reviews Neuroscience. 2009;10:303–312. doi: 10.1038/nrn2598. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. United States Department of Justice, Federal Bureau of Investigation. Crime in the United States, 2016. 2017 Sep; Retrieved from https://ucr.fbi.gov/crime-in-the-u.s/2016/crime-in-the-u.s.-2016/topic-pages/violent-crime.
  58. van Aken C, Junger M, Verhoeven M, van Aken MAG, Dekovic M. The interactive effects of temperament and maternal parenting on toddlers’ externalizing behaviours. Infant and Child Development. 2007;16:553–572. doi: 10.1002/icd. [DOI] [Google Scholar]
  59. Weinberg MK, Olson KL, Beeghley M, Tronick EZ. Making up is hard to do, especially for mothers with high levels of depressive symptoms and their infant sons. Journal of Child Psychology and Psychiatry. 2006;47(7):670–683. doi: 10.1111/j.1469-7610.2005.01545.x. [DOI] [PubMed] [Google Scholar]
  60. Weinberg MK, Tronick EZ, Cohn JF, Olson KL. Gender differences in emotional expressivity and self-regulation during early infancy. Developmental Psychology. 1999;35(1):175–188. doi: 10.1037/0012-1649.35.1.175. [DOI] [PubMed] [Google Scholar]
  61. Zuckerman M. Chapter 31. Sensation seeking. In: Leary MR, editor. Handbook of Individual Differences in Social Behavior. New York, NY/London: The Guildford Press; 2009. pp. 455–465. [Google Scholar]

RESOURCES