Abstract
The purpose of this study was to examine the feasibility, acceptability, and initial outcome of a home-based adaptation of Parent-Child Interaction Therapy for at-risk infants with externalizing behavior problems. Seven 12- to 15-month-old infants and their families were recruited at a large pediatric primary care clinic to participate in a home-based parenting intervention to prevent subsequent externalizing behavior problems. Home-based assessments were conducted at baseline, postintervention, and a 4- to 6-month follow-up. Six of the 7 (86%) families completed the intervention, and all completers reported high satisfaction with the intervention. All of the mothers demonstrated significant improvements and statistically reliable changes in their interactions with their infant, and most reported clinically significant and statistically reliable changes in infant behavior problems. The current study provides preliminary support for the use of this brief, home-based parenting intervention in addressing behavior problems as early as possible to improve access to an intervention for at-risk infants and their families. Successes and challenges with the development and implementation of this intervention are discussed along with directions for future research and clinical practice.
Keywords: infancy, externalizing behavior problems, prevention, parenting, risk
Externalizing behavior problems are exceedingly common in early childhood, affecting between 10% to 15% of 2- to 3-year-old children (Carter, Briggs-Gowan, & Davis, 2004). These problems represent a significant public health concern because they are the most common child mental health referral reason (Keenan & Wakschlag, 2000; Luby & Morgan, 1997; Thomas & Guskin, 2001) and place children at increased risk for developing more severe conduct problems (Moffitt, Caspi, Harrington, & Milne, 2002; Shaw, Gilliom, Ingoldsby, & Nagin, 2003). In addition to the high impairment and poor prognosis, public expenditures are estimated to be $10 thousand more each year for a child with conduct problems (Foster & Jones, 2005), with total lifetime costs exceeding $2 million per individual (Muntz, Hutchings, Edwards, Hounsome, & O'Ceilleachair, 2004). Children from economically disadvantaged families are especially vulnerable to developing early externalizing behavior problems (Qi & Kaiser, 2003).
Substantial research over the past 40 years supports the use of parent training in the treatment of externalizing behavior problems in young children (Brestan & Eyberg, 1998; Eyberg, Nelson, & Boggs, 2008). Despite the large evidence base, however, parent-training treatments are least effective for low-income families (Lundahl, Risser, & Lovejoy, 2006). For example, families from socioeconomically disadvantaged backgrounds have disproportionately lower attendance and dropout rates (Lavigne et al., 2010; Reyno & McGrath, 2006), and limited financial resources (e.g., lack of transportation) have been cited as a primary reason for treatment dropout (Boggs et al., 2004). Therefore, adaptation of current parent-training treatments is necessary to meet the specific needs of these high-risk families.
Identifying externalizing behavior problems as early as possible would likely require less intensive and shorter interventions, thereby decreasing the burden for low-income families. Recent research has demonstrated the appropriateness of diagnosing preschoolers as young as 2 years with externalizing disorders (Wakschlag, Briggs-Gowan, et al., 2008; Wakschlag, Hill, et al., 2008). Early starter pathways of antisocial behavior have been established as young as 18 months (Aguilar, Sroufe, Egeland, & Carlson, 2000; Shaw et al., 2003), and there is considerable evidence that early parent-infant interactions predict later child psychopathology (Forcada-Guex, Pierrehumbert, Borghini, Moessinger, & Muller-Nix, 2006; Lyons-Ruth, Easterbrooks, & Cibelli, 1997; Tarabulsy, Tessier, & Kappas, 1996). Additionally, Patterson’s coercion model suggests behavior problems are inadvertently established and maintained by early parent-infant interactions (Patterson, 1982). Therefore, targeting infants can reduce the occurrence of these negatively reinforcing interactions and consequently prevent behavior problems from becoming more entrenched and developing into a disorder.
Identifying externalizing behavior problems in infancy can be challenging due to the developmental appropriateness of some of these behaviors (e.g., temper tantrums to assert autonomy). Many parents and professionals believe that early behavior problems are transitory and will decrease over time (Briggs-Gowan, Carter, Bosson-Heenan, Guyer, & Horwitz, 2006). However, these behaviors do not remit for all infants, and accurate identification of those at risk for long-term behavior problems is an important step in providing preventive interventions (Mouton-Simien, McCain, & Kelley, 1997). Signs of atypical development for this young age include differences in the frequency, intensity, and duration of behaviors in comparison to a normative sample, and recent research has demonstrated that behavior problems can be reliably measured in infants as young as 12 months (Carter et al., 2004; Carter, Briggs-Gowan, Jones, & Little, 2003; van Zeijl, Mesman, Stolk, et al., 2006). Examples of problems observed in this young age are similar to the toddler and preschool ages and include behaviors such as hitting and biting others, temper tantrums, and restlessness. Furthermore, high levels of negative parenting, family stress, and other socioeconomic risk factors (e.g., poverty) have been shown to increase the stability of early externalizing behavior problems (Campbell, Shaw, & Gilliom, 2000), underlining the importance of implementing prevention programs with at-risk infants and their families.
A recent review of the literature demonstrated limited evidence for parent-infant prevention programs (Olds, Sadler, & Kitzman, 2007). One of the programs with the most empirical support is the Nurse-Family Partnership (NFP), in which nurses regularly visit the homes of low-income, primiparous mothers during pregnancy and the subsequent 2 years to target maternal and child health and development (e.g., planning future pregnancies, teaching competent caregiving; Olds, 2002). The results of three large-scale randomized trials have demonstrated that NFP leads to significant improvements in parental care of the child and maternal welfare (Olds, 2006). Despite its strong support, the broad focus and long-term nature of the NFP can be expensive and time consuming, with an average of 32 to 33 visits lasting approximately 75 to 90 minutes each (Olds, 2006). Additionally, the NFP does not specifically target infant behavior problems. In addition to the NFP, other home-based parent-infant programs that received some empirical support in a recent report (Paulsell, Avellar, Sama Martin, & Del Grosso, 2010) include Early Head Start–Home Visiting (Jones Harden, Chazan-Cohen, Raikes, & Vogel, 2012), Healthy Families America (DuMont et al., 2008), Healthy Steps (Minkovitz et al., 2003), and Parents as Teachers (Wagner, Spiker, & Linn, 2002). However, many of these interventions yielded relatively small effect sizes, produced limited significant findings on infant behavior, were associated with high dropout rates (e.g., 50%; Roggman, Cook, Peterson, & Raikes, 2008), and, similar to the NFP, involved numerous sessions over the course of several years. Therefore, an intervention focused on decreasing infant behavior problems in a brief time frame can be a practical and cost-effective approach in preventing later conduct problems.
A promising short-term, three-session intervention focused on parenting practices and child behavior is the early childhood version of the Family Check-Up (FCU; Dishion & Kavanagh, 2003; Dishion & Stormshak, 2007). In a randomized trial with 17- to 27-month-old low-income boys (n = 120), FCU was associated with decreases in infant destructive behavior (Shaw, Dishion, Supplee, Gardner, & Arnds, 2006) and increases in positive parenting (Gardner, Shaw, Dishion, Burton, & Supplee, 2007). Additionally, more recent research has demonstrated immediate and long-term effectiveness of the FCU with a large-scale sample of 731 families including 49% girls (Dishion et al., 2008; Lunkenheimer et al., 2008), including a finding with this sample that child behavioral control mediated the relation between proactive parenting and subsequent child behavior problems (Shelleby et al., 2012). However, the FCU has not been examined in children younger than 17 months. Another encouraging brief intervention focusing on infant behavior is a six-session, home-based intervention called Video-Feedback Intervention to Promote Positive Parenting and Sensitive Discipline (VIPP-SD). Although a large randomized trial (n = 237; Van Zeijl, Mesman, Van, et al., 2006) suggested that VIPP-SD led to more positive mother discipline practices, there was no effect on child behavior problems, no follow-up results, and a wide child age range occurring during a period of rapid developmental growth (i.e., ages 12 to 36 months). Therefore, further research on developing a preventive parenting intervention with diverse and at-risk infants from a younger and narrower age range is needed.
Rationale for Implementing Parent-Child Interaction Therapy (PCIT) With Infants
Parent-Child Interaction Therapy (PCIT) is an evidence-based parent-training intervention that may be well suited for infants. With theoretical foundations in attachment and social learning, PCIT was designed to strengthen the parent-child interaction to change child behavior (Brinkmeyer & Eyberg, 2003). PCIT typically progresses through two phases. During the Child Directed Interaction (CDI), the parents learn to follow their child’s lead in play and use differential attention to strengthen the parent-child relationship. During the Parent Directed Interaction (PDI), the parents learn to use effective commands and time-out for noncompliance, but this procedure would be developmentally inappropriate for infants.
There has been extensive research demonstrating the efficacy and long-term maintenance of PCIT in the treatment of disruptive behavior disorders in preschoolers (Eisenstadt, Eyberg, McNeil, Newcomb, & Funderburk, 1993; Eyberg et al., 2001; Hood & Eyberg, 2003; Schuhmann, Foote, Eyberg, Boggs, & Algina, 1998), and in young children with and at risk for developmental delay (Bagner & Eyberg, 2007; Bagner, Sheinkopf, Vohr, & Lester, 2010). Similar to other evidence-based interventions, however, children from low-income families and with other risk factors are less likely to complete and benefit from PCIT (Fernandez & Eyberg, 2009; Werba, Eyberg, Boggs, & Algina, 2006). Identification of behavior problems during infancy, before coercive parent-child interactions become more entrenched, is feasible (Bagner, Rodríguez, Blake, Linares, & Carter, 2012) and would likely require a less intensive and shorter intervention with a prevention framework. Targeting infants from high-risk families (i.e., low income) and who display symptoms predicting later onset of a disorder (but not currently diagnosed) is consistent with an indicated preventive intervention (Muñoz, Mrazek, & Haggerty, 1996), which would involve a shorter time commitment for families compared to later treatment of the disorder. Additionally, implementing the intervention in the home would make it easier for low-income families to attend sessions regularly given their limited access to transportation. Therefore, adapting PCIT as a home-based preventive intervention for infants may help reduce the burden and improve outcome for high-risk families by decreasing the likelihood of the child developing a later disruptive behavior disorder.
Infants between 12 and 15 months are just beginning to walk and talk, providing an ideal opportunity to promote more positive parent-infant interactions in an effort to reduce coercive and punitive interactions. The PDI phase of standard PCIT would be developmentally inappropriate for this young age due to limited infant receptive language (i.e., understanding parental commands) and the lower frequency of noncompliant behavior (van Zeijl, et al., 2006). However, there are several developmental and practical reasons a home-based adaptation of the CDI phase would be desirable for an at-risk infant population. First, parents have been shown to interact with their infant in a directive manner, especially when the infant is perceived to be difficult (Guzell & Vernon-Feagans, 2004). Therefore, teaching parents to be nondirective in play may be effective in helping their infant develop autonomy. Additionally, the CDI phase may provide an opportunity for parents to scaffold and guide the development of self-regulation, which has been shown to predict later behavior problems (Eisenberg et al., 2000). The transition from infancy to early childhood has been identified as an excellent opportunity for parents to improve their child’s self-regulation skills (Fox & Calkins, 2003), providing further support for targeting infants ages 12 to 15 months.
Second, the CDI phase is significantly shorter (e.g., average of 6 sessions) than the average of 12 to 14 sessions in standard PCIT and other parent-training treatments. The shorter duration may significantly decrease the chances for dropout and increase family engagement, which has been shown to positively influence treatment outcome (Nix, Bierman, & McMahon, 2009). Second, the CDI phase alone has already been shown to demonstrate significant reductions in externalizing behaviors in preschoolers (Eisenstadt et al., 1993; Harwood & Eyberg, 2006). Third, providing an intervention in the home can help reduce barriers (e.g., lack of transportation) and increase access to high-risk families (McKay & Bannon, 2004). Home-based PCIT has shown preliminary support with preschoolers (Ware, McNeil, Masse, & Stevens, 2008) but has not yet been examined with infants. Fourth, consistent with early intervention efforts, the active coaching component provides parents with the unique opportunity to learn new skills prior to their child establishing more ingrained behavior problems. In the proposed home-based, preventive intervention, we expected families would show high levels of attendance and satisfaction, improvement in targeted parenting skills, and decreases in infant externalizing and related behavior problems.
Specific Adaptations of the CDI Phase of PCIT for Infants
Consistent with recommendations by Eyberg (2005) to adapt PCIT for new populations, we maintained the core features of CDI while addressing the unique developmental needs of infants. During the first session, families participated in a CDI “teach” session for approximately 1 hour, which included an orientation to the intervention and a discussion of the specific rules the parents should use to follow their infant’s lead in play. Following the teach session, each family participated in “coach” sessions, also lasting approximately 1 hour each, in which the therapist actively coached the parent “live” in using the skills with the child. In standard PCIT, parents participate in CDI coach sessions until they meet mastery criteria (described below), and previous research has shown success with 5 coach sessions (Bagner & Eyberg, 2007). Therefore, the adapted intervention was proposed to last approximately 6 sessions, including the teach session and coach sessions.
During the CDI teach session in the adapted infant version, parents were taught to avoid giving commands, asking questions, and using negative statements (e.g., criticisms). Additionally, parents learn to use the nondirective PRIDE skills, which include: Praising the infant, Reflecting the infant’s speech, Imitating the infant’s play, Describing the infant's behavior, and expressing Enjoyment in the play. Finally, parents are taught to direct the PRIDE skills to the infant’s appropriate play and ignore undesirable behaviors. The skills are coded individually during a 5-minute child-directed play and considered “mastered” when the parent uses a total of 10 labeled praises, 10 descriptions, and 10 reflections (i.e., “do skills”), as well as less than 3 questions, commands, and criticisms (i.e., “don’t skills”). These skills and criteria were the same as in standard PCIT, except for a couple of adaptations to meet the needs of infants and their families. Given most infants ages 12 to 15 months have limited verbalizations and vocalizations, parents were not required to state 10 reflections to meet mastery criteria. Consistent with previous work with children with developmental delay (Bagner & Eyberg, 2007), parents were encouraged to repeat their infant’s vocalizations and attempts at verbalizations and were required to reflect at least 75% of all infant verbalizations and/or vocalizations. All other mastery criteria for the adapted version were the same as in standard PCIT. In addition to this change in skill criteria, we also made some minor adaptations to the handouts provided to families. For instance, we replaced developmentally inappropriate examples of specific praises (e.g., “You drew a pretty tree”) with more relevant examples for infants (e.g., “Good job holding the toy in your hand” to encourage gentle play).
During each coach session, and consistent with standard PCIT, the therapist spends approximately 10 to 15 minutes in the beginning of the session discussing the previous week. Specifically, the therapist assesses the frequency with which the parent(s) practiced the skills and changes in the infant’s behavior during the previous week. Additionally, the therapist provides support to the parents about other issues or stressors raised (e.g., job stress) to help build rapport and decrease dropout (Prinz & Miller, 1994). Following the discussion, parent skills are then assessed during a 5-minute parent-infant observation, and data are collected by the therapist to coach the parents in their use of the skills. The remainder of each coach session consists of the therapist directly coaching the parent in using the skills while playing with their infant. The observations and coaching are typically conducted through a one-way mirror using a wireless headset. However, to reduce the burden of low-income families for this adapted version, we implemented the current intervention in the home. Therefore, we coached parents in vivo in their use of the skills by sitting close to the parent and providing feedback quietly, a technique that has been successfully implemented in previous pilot research (Ware et al., 2008). Specific adaptations to PCIT were implemented to reflect issues inherent in providing a home-based intervention. For example, the therapist problem solved with the family ways to optimize in-home coaching by choosing an appropriate location for the session and developing ways to minimize distractions (e.g., turning off the television, scheduling when siblings are not home).
In addition to standard coaching practices in PCIT described above, therapists also incorporated strategies relevant for the infant population. For example, given the lower receptive language abilities in infants, parents were especially encouraged to use positive physical touch (e.g., patting the infant’s back) and nonverbal praise (e.g., clapping hands) along with verbal praise to enhance reinforcement for appropriate behaviors. During coaching, therapists also conveyed important developmental expectations for infant behaviors and described the specific effects of the parents’ behavior on the infant. For example, although it is not uncommon for infants to yell to express frustration, parents were coached to address this behavior to help teach their infant to respond more appropriately when frustrated. Specifically, the therapist would help the parent ignore the infant when yelling and praise the infant for being calm and quiet as soon as the infant stopped yelling. In addition to these types of coaching adaptations, all handouts were tailored for infants, including an additional two new handouts that address common infant problems with sleeping and feeding.
In addition to being implemented with infants, we pilot tested the intervention in an open trial with Hispanic families, many of which spoke only Spanish. Therefore, the revised English manual and handouts were translated into Spanish. Although we considered further adaptations for Hispanic families, findings that standard PCIT was shown to be as effective as a culturally modified version (McCabe & Yeh, 2009) suggest changes were not necessary at this early stage of intervention development. Nevertheless, the intervention was tailored to meet the needs of each individual family, including cultural issues, as is done in standard PCIT (Eyberg, 2005). For example, efforts were made to ensure cultural sensitivity in the presentation and coaching of skills, particularly with less acculturated families.
Method
Participants and Recruitment
Participants were 7 mothers and their 12- to 15-month-old infants, who displayed elevated problems on a brief screener of social-emotional functioning. Families were actively recruited over the course of 3 months during infant well and sick visits at a large pediatric primary care clinic (housed in a children’s hospital) serving a majority of families (85%) without private insurance in a predominately Hispanic community. Given the large percentage of Hispanic families seen at this clinic and living in the surrounding areas, mothers were required to speak and understand either English or Spanish. For bilingual families, the mother was given the option whether she preferred to answer questions in English or Spanish.
For study inclusion, the mother had to rate their infant above the clinically significant range (i.e., > 75th percentile) on the Brief Infant-Toddler Social Emotional Assessment (BITSEA; Briggs-Gowan, Carter, Irwin, Wachtel, & Cicchetti, 2004), which is available in English and Spanish. Consistent with previous PCIT research and to ensure the ability of the parent to learn the skills, the mother had to receive an estimated IQ score ≥ 70 on the two-subtest (vocabulary and matrix reasoning) version of the Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, 1999) for mothers speaking English or an average standard score ≥ 4 on the vocabulary and matrix reasoning subtests of the Escala de Inteligencia Wechsler Para Adultos– Third Edition (EIWA-III; Pons, Flores-Pabón, et al., 2008) for mothers speaking Spanish. The mother also completed the Ages and Stages Questionnaire–Third Edition (ASQ-3; Squires, Twombly, Bricker, & Potter, 2009) for an estimate of infant developmental functioning. All 7 infants were reported to be in the normal range for the communication scale (includes both receptive and expressive items) of the ASQ-3 based on each infant’s age in months.
During the recruitment period, 27 families were approached at the primary care clinic and most (82%) agreed to participate in the study. One family contacted our staff by telephone after seeing a brochure about the study and was not recruited at the clinic. Of the 22 infants participating in the screening, 14 (64%) were screened out due to scores below the 75th percentile on the BITSEA, and no mothers were screened out due to scores on the WASI or EIWA-III. One infant was inadvertently consented but did not meet the age criteria and was provided referral information for alternative services. Exclusion criteria included major sensory impairments (e.g., deafness) or motor impairments significantly affecting mobility, but none of the 27 infants approached were excluded based on these criteria.
Procedure
After providing a description of the study, written informed consent was obtained from the infant’s mother before proceeding with the screening. Each family that met study criteria was contacted by one of two B.A.-level graduate students in clinical psychology (second and third authors), under the supervision of the first author and licensed clinical psychologist, to set up a baseline assessment in each family’s home. All families then received the intervention weekly for an average of 6 sessions, all of which were also conducted in each family’s home by one of the BA-level graduate students along with either the first author or the other graduate student. In accordance with PCIT international guidelines, both graduate students were trained in PCIT by the first author, who is also a PCIT Master Trainer. The graduate students participated in weekly group supervision to discuss each case along with the first author, and all sessions were audiotaped and later coded for therapist adherence to the intervention manual.
There were several practical and ethical considerations to implementing the intervention in the home that were carefully addressed. For example, safety of the clinicians and families was of utmost concern, so all therapists traveled to families for assessments and intervention sessions in pairs. The clinicians maintained confidentiality in the home setting by contacting families 1 hour prior to the session (in addition to the day before) to make sure it was a good time to visit their home. Additionally, families were informed about the limits to confidentiality at the initial baseline in the event a suspected child abuse incident arises, which may be more likely to observe in the home environment. Fortunately, this was not the case with any of the families described below.
Although data were collected only from the mother, fathers and other caregivers living in the home were invited to participate in the intervention. Sessions were conducted once a week for approximately 1 hour, and parents were expected to practice the skills daily in a 5-minute infant-led play in between sessions. Following the intervention and approximately 2 months after the baseline, families were seen for the postintervention assessment in their home, which included the same measures completed at baseline with an additional measure of program satisfaction. Families also participated in a home-based follow-up assessment that occurred between 4 and 6 months after the postintervention assessment. Families were reimbursed $50 for their participation in each assessment, totaling $150 for their involvement in the entire study. The protocol was approved by the Institutional Review Boards at the research site and the children’s hospital housing the pediatric clinic.
Screening Measures
Brief Infant-Toddler Social and Emotional Assessment (BITSEA; Carter & Briggs-Gowan, 2006)
The BITSEA is a 31-item, nationally standardized screener designed to assess behavioral problems and competencies in 12- to 36-month-olds. The problem scale has excellent test-retest reliability (r = .91 to .92) and very good interrater reliability (r = .70 to .78; Carter & Briggs-Gowan, 2006), as well as support for discriminative validity for those scoring above the clinical cutoff of the 75th percentile (Briggs-Gowan et al., 2004). Examples of items on the problem scale include “restless and can’t sit still,” “is destructive,” and “hits, bites or kicks” and are rated on a scale from 0 (not true/rarely), 1 (somewhat true/sometimes), or 2 (very true/often). Infants scoring above the 75th percentile on the BITSEA problem scale were included in the study.
Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, 1999) and Escala de Inteligencia Wechsler Para Adultos–Third Edition (EIWA-III; Wechsler, 2008)
The WASI is a brief measure of intelligence with high reliability and validity (Hays, Reas, & Shaw, 2002), and the EIWA-III is the Spanish version of the full Wechsler Scale of Intelligence (Wechsler, 1997) with demonstrated reliability (Pons, Flores-Pabón, et al., 2008) and validity (Pons, Matías-Carrelo, et al., 2008). The vocabulary and matrix reasoning subtests were administered, and mothers were required to receive an estimated IQ score ≥ 70 on the two-subtest version of the WASI or an average standard score ≥ 4 on the EIWA-III subtests, although no mothers were excluded based on this criterion.
Measure of Parent Satisfaction
Therapy Attitude Inventory (TAI; Eyberg, 1993)
The TAI is a 10-item parent-report measure that assesses parent satisfaction with treatment. Test-retest reliability over 4 months and correlations between the TAI and both parent-rating scales and observational measures of treatment change have been demonstrated (Brestan, Jacobs, Rayfield, & Eyberg, 1999). The TAI total score was administered at the postintervention assessment to assess parent satisfaction with the intervention.
Measures of Parenting Skills and Infant Behavior Outcomes
Dyadic Parent-Child Interaction Coding System–Third Edition (DPICS-III; Eyberg, Nelson, Duke, & Boggs, 2005)
The DPICS-III is a behavioral coding system of parent-child interactions with documented reliability and validity with Hispanic families (McCabe, Yeh, Lau, Argote, & Liang, 2010). Therapists were trained in DPICS skills to 80% reliability and coded the parent-infant interaction “live” at the beginning of each intervention session using a behavioral monitoring form to assess the frequency of each individual skill. For purposes of assessing mastery skills criteria (described above) and summarizing data, parent skills were combined into two composite categories of do (labeled or specific praises, behavior descriptions, and reflections) and don’t (questions, commands, and negative talk) skills reflecting verbalizations parents are taught during the intervention to use and not use during infant-led play. For the videotaped parent-infant interactions during the assessments, verbalizations in Spanish were first translated, and all verbalizations were transcribed before coding the interactions. Coders were trained to 80% agreement with a criterion tape and half of the observations at the baseline assessment in the current study were coded a second time for reliability. Inter-rater reliability was excellent and ranged from 91% to 96%.
Infant-Toddler Social and Emotional Assessment (ITSEA; Carter & Briggs-Gowan, 2006)
The ITSEA is a 166-item, nationally standardized questionnaire designed to assess behavioral problems and competencies in 12- to 36-month-olds and yields three problem scales: externalizing, internalizing, and dysregulation. The ITSEA problem scales have excellent test-retest reliability (r = .85 to .91) and very good interrater reliability (r = .70 to .76; Carter & Briggs-Gowan, 2006), as well as evidence for validity with other parent-report and observational measures (Carter et al., 2003; Carter, Little, Briggs-Gowan, & Kogan, 1999). The externalizing and dysregulation scales were used as measures of infant behavior problems at all assessments.
Child Behavior Checklist for 1½- to 5-Year-Olds (CBCL; Achenbach & Rescorla, 2000)
The CBCL is a 99-item parent-rating scale designed to measure the frequency of child behavioral and emotional problems. The CBCL is one of the most widely used questionnaires with excellent interrater and test-retest reliability (Achenbach & Rescorla, 2000). For this study, the externalizing and emotionally reactive scales were used as measures of infant behavior at the follow-up assessment when all infants were at least 18 months, provided an additional measure of behavior problems.
Results
Intervention Feasibility and Satisfaction
On average, the six families completed the intervention after meeting mastery criteria (described above) in an average of 6 sessions (range of 5 to 7, including the teach session). All sessions were audiotaped, and 94% were coded for integrity by an undergraduate research assistant uninvolved in providing the intervention. Accuracy, defined as the percent with which the therapist adhered to key elements of each session detailed in the treatment manual, was 99% (range = 97% to 100%). Parents were moderately compliant with practicing the parenting skills daily during an infant-led play (79% completion rate), and reported high satisfaction and acceptance with the intervention on the TAI (M = 46 out of a possible 50, range from 41 to 50).
Outcome Trends for Parenting Skills
Changes in observed parenting skills were measured during the infant-led play between the baseline and postintervention assessments and between the baseline and follow-up assessments. As shown in Table 1, changes in frequency counts of do and don’t skills between both pairs of assessments were statistically significant in the predicted direction with Cohen’s d ([baseline – post-intervention/follow-up]/pooled SD) ranging from 2.27 to 6.49. Specifically, mothers significantly increased their use of the do skills and significantly decreased their use of the don’t skills. The mean frequency scores of the do and don’t skills at the different assessments were comparable to previous research with preschoolers with developmental delay (Bagner & Eyberg, 2007).
Table 1.
Mother | Age (years) |
Baseline | Postintervention | Follow-up | |||
---|---|---|---|---|---|---|---|
Do Skills | Don’t Skills | Do Skills | Don’t Skills | Do Skills | Don’t Skills | ||
1 | 25 | 2 | 32 | 19R | 0R | 28R | 2R |
2 | 28 | 1 | 40 | 14R | 0R | 18R | 2R |
3 | 40 | 5 | 26 | 16R | 2R | 5 | 2R |
4 | 36 | 4 | 57 | 19R | 5R | 29R | 2R |
5 | 38 | 3 | 55 | 14R | 8R | 10R | 5R |
6 | 28 | 4 | 31 | 20R | 2R | 9R | 15R |
7 | 25 | 11 | 12 | -- | -- | -- | -- |
Mean | 31.43 | 4.14 | 35.86 | 16.92*** | 2.58*** | 16.25* | 4.50*** |
SD | 6.37 | 3.74 | 16.25 | 2.80 | 2.92 | 10.28 | 5.31 |
d | -- | -- | -- | -6.49 | 4.66 | −2.27 | 3.85 |
Note.
RCI < .05;
p < .05;
p < .01;
p < .001.
Outcome Trends for Infant Behavior
On the ITSEA externalizing scale, 4 of the 6 infants were rated above the clinical cutoff (i.e., T-score of 65) at baseline, 2 infants were rated above the clinical cutoff at the postintervention assessment, and only 1 infant was rated at the clinical cutoff at the follow-up. As shown in Table 2, changes between baseline and follow-up approached significance with an effect size of 1.34. On the ITSEA dsyregulation scale, 3 of the 6 infants were rated above the clinical cutoff at baseline, two infants were rated above the clinical cutoff at the postintervention assessment, and no infants were rated at or above the clinical cutoff at the follow-up. Changes between the baseline and follow-up also approached significance with an effect size of 1.37.
Table 2.
Child | Age (mo.) |
Baseline | Postintervention | Follow-up | |||||
---|---|---|---|---|---|---|---|---|---|
ITSEA-Ext | ITSEA-Dys | ITSEA-Ext | ITSEA-Dys | ITSEA-Ext | ITSEA-Dys | CBCL-Ext | CBCL-ER | ||
1 | 13 | 57 | 56 | 48R | 44R | 59 | 58 | 46R | 50 |
2 | 13 | 83 | 64 | 50R | 34R | 50R | 32R | 57R | 50R |
3 | 15 | 70 | 90 | 52R | 45R | 54R | 46R | 43R | 50R |
4 | 14 | 89 | 86 | 72R | 79 | 65R | 62R | 60R | 69R |
5 | 12 | 65 | 60 | 61 | 79 | 59 | 62 | 51R | 50R |
6 | 12 | 60 | 66 | 72 | 60 | 63 | 56R | 56 | 59 |
7 | 14 | 67 | 49 | -- | -- | -- | -- | -- | -- |
Mean | 13.33 | 70.67 | 70.33 | 59.17 | 56.83 | 58.33+ | 52.67+ | 52.17** | 54.67** |
SD | 1.15 | 12.82 | 14.18 | 10.89 | 19.07 | 5.57 | 11.71 | 6.68 | 7.89 |
d | -- | -- | -- | 0.97 | 0.81 | 1.34 | 1.37 | 1.78 | 1.42 |
Note. ITSEA-Ext/Dys = Infant-Toddler Social-Emotional Assessment Externalizing/Dysregulation T-Scores; CBCL-Ext/ER = Child Behavior Checklist Externalizing/Emotionally Reactive T-Scores.
RCI < .05;
p < .10;
p < .05;
p < .01
On the CBCL, which was administered only at the follow-up due to the age criterion (i.e., 18 months), all infants were below the clinical cutoff on the externalizing scale (i.e., T-scores of 63) and 5 of 6 were below the clinical cutoff on the emotionally reactive scale (i.e., T-score of 65). Given the similarity in scales, comparisons were made between the baseline ITSEA externalizing and dysregulation scales and the follow-up CBCL externalizing and emotionally reactive scales. As shown in Table 2, both comparisons were statistically significant with effect sizes of 1.78 and 1.42, respectively.
Case Examples and Individual Results
Overview
Given the small sample of the current trial, descriptions of each case are provided below in the order that the families were enrolled in the study. Additionally, we calculated the reliable change index (RCI; Jacobson & Truax, 1991) to examine individual clinically significant changes as has been done in previous pilot work with a small sample (e.g., Chu, Colognori, Weissman, & Bannon, 2009), and these scores are also presented in Tables 1 and 2. Specifically, the , where x1 is the baseline score, x2 = the postintervention score, and , the standard error (SE) of difference between the two scores. The , where s1 is the standard deviation of the normal population, and rxx = the reliability of the measure. An RCI ≥ 1.96 represents a reliable change at α = .05. Normative and reliability data for the DPICS-III were from a recent study examining this measure with Hispanic families (McCabe et al., 2010) and was as follows: s1 = .83 and rxx = .73 for do skills and s1 = 5.57 and rxx = .77 for don’t skills. For the ITSEA and CBCL, normative and reliability data were from the standardization samples (Achenbach & Rescorla, 2000; Carter & Briggs-Gowan, 2006) and was as follows: ITSEA externalizing scale: s1 = 8.81 and rxx = .92; ITSEA dysregulation scale: s1 = 10.90 and rxx = .94; CBCL externalizing scale: s1 = 9.60 and rxx = .87; CBCL emotion regulation scale: s1 = 6.30 and rxx = .87. To determine the RCI between the ITSEA at baseline and the CBCL at follow-up, average scores of normative and reliability data were calculated (externalizing: s1 = 9.21 and rxx = .90; dysregulation/emotion regulation: s1 = 8.60 and rxx = .91)
Child 1
The first infant was a Hispanic, 13-month-old girl with no significant medical history or developmental concerns, who lived with her biological mother and father, aged 25 and 32 years, respectively. Her parents were married and both immigrated to the United States from Columbia approximately 4 years prior to the screening. Both parents were college graduates and reported an annual income of $48,000. The mother, who was bilingual in English and Spanish, reported minimal concerns with her infant but acknowledged her “strong personality” and that she often engages in some difficult behaviors (e.g., easily upset, destructive, and restless). The mother reported scores below the clinical cutoff on the ITSEA externalizing and dysregulation scales, and used 2 do skills and 32 don’t skills during the infant-led play.
The mother regularly participated in 5 sessions occurring over 5 weeks and completed homework 77% of the time. The father was unable to attend any sessions. Although the mother did not initially express significant concerns with her daughter’s behavior, she enjoyed learning new skills to promote positive behaviors and noticed the effect of using the skills on her child. For example, when the therapists asked the mother about the previous week at the beginning of the fourth coach session, the mother reported that she noticed her child seemed calmer overall. This led to a discussion about how her use of the skills can also have an impact on the child’s behavior outside of playtime with the goal of having the mother generalize her use of these skills. At the postintervention and follow-up assessments, the mother displayed statistically reliable increases in her use of do skills and decreases in her use of don’t skills. The mother also reported statistically reliable change on the ITSEA externalizing and dysregulation scales with scores below the clinical cutoff at the postintervention. Although the scores were below the clinical cutoff at follow-up, they did not represent a statistically reliable change from baseline. Scores on the CBCL externalizing and emotion regulation scales at follow-up were also below the clinical cutoff, and scores between the baseline ITSEA externalizing scale and the follow-up CBCL externalizing scale represented statistically reliable change. Finally, the mother reported very high satisfaction on the TAI with a score of 50.
Child 2
The second infant was a Hispanic, 13-month-old girl who lived with her biological mother and father, aged 28 and 29 years, respectively, as well as her 8-year-old sister. Her medical history was not significant other than a 4-day hospital stay for a urinary tract infection and high fever at 7 months. Both parents reported some college/technical school and an annual income of $52,800. During the screening, the mother reported concerns about her infant’s behavior, including increased hitting and being “extremely strong-willed for her age,” and she reported scores above the clinical cutoff on the ITSEA externalizing and dysregulation scales. During the infant-led play, the mother used 1 do skill and 40 don’t skills.
The mother regularly participated in 6 sessions occurring over 8 weeks and had proactively cancelled three sessions due to illness and a family event. The father was invited but unable to participate in the intervention. The mother was very involved and excited to learn skills to reduce hitting and other aggressive behaviors in her infant, and she completed homework practice 88% of the time. Additionally, this mother initially reported feeling upset because her child “did not want to be close to her.” In response, the therapists validated her concerns and subsequently coached the mother to praise her child when demonstrating affection toward her (e.g., sitting on her lap). At the postintervention and follow-up assessments, the mother displayed statistically reliable increases in her use of do skills and decreases in her use of don’t skills, and she reported being pleased that her daughter wanted to be “close” with her. Additionally, the mother reported statistically reliable change on both the ITSEA externalizing and dysregulation scales at the postintervention and follow-up assessments, as well as statistically reliable change between the ITSEA baseline and CBCL follow-up scores with all scores well below the clinical cutoff. The mother also reported high satisfaction with the intervention with a 48 on the TAI.
Child 3
The third infant was a Hispanic, 15-month-old girl who lived with her biological mother, aged 40, and her 15-year-old sister. The family immigrated to the U.S. just 4 months prior to the first evaluation and was living with the mother’s 56-year-old aunt and her family. The infant’s father, aged 37 years, was married to the mother but had remained in Cuba. The mother attended some college and reported an annual income of $14,000. The infant was diagnosed with asthma shortly after arriving in the U.S., and was hospitalized for 7 days at 10 months due to an abdominal abscess. The mother reported feeding problems and some weight loss in her infant as primary concerns at the screening but also acknowledged that her infant was restless, often had temper tantrums, and was destructive. On the ITSEA, the mother reported scores in the clinical ranges on both the externalizing and dysregulation scales. During the infant-led play, she used 5 do skills and 26 don’t skills.
The mother participated in 5 sessions occurring over 7 weeks and completed homework practice 93% of the time. Despite the substantial amount of stress she was experiencing due to the recent move and unfavorable living situation, the mother actively participated in the program. Additionally, her culture played an important role in how the therapist coached the mother in using the skills. For example, the therapist noticed that the mother commonly said, “!Que lindo!,” which directly translates to “how pretty,” to praise toys or objects instead of the child’s behavior. After the therapist inquired more, the mother stated that it was common to use this phrase in her culture. The therapist validated the mother’s use of this phrase but also coached the mother in applying this phrase specifically to the child’s positive behaviors. At the postintervention assessment, the mother displayed statistically reliable increases in her use of do skills and decreases in her use of don’t skills. The mother maintained her gains with the don’t skills but decreased her use of do skills at the follow-up. With regard to report of the infant’s behavior, the mother reported statistically reliable change on both the ITSEA externalizing and dysregulation scales at the postintervention and follow-up, as well as statistically reliable change between the ITSEA baseline and CBCL follow-up scores, with all scores well below the clinical cutoff. The mother also reported very high satisfaction with the intervention with a 50 on the TAI.
Child 4
The fourth infant was a Hispanic, 14-month-old girl, with no significant medical history or developmental concerns, who lived her biological mother and father, aged 36 and 40 years, respectively, and her two brothers, ages 11 and 16. Her parents were married and immigrated to the U.S. from Venezuela 8 years prior to the screening. Both parents were of Middle Eastern descent and spoke Arabic as their primary language, although they self-identified as Hispanic and also spoke Spanish. The mother completed some college and was a homemaker, whereas the father had a college degree, and they reported an annual income of $24,000. The mother reported several concerns with her infant, including feeding and sleeping problems, as well as several disruptive behaviors (e.g., destructive). The mother’s report on the ITSEA externalizing and dysregulation scales was above the clinical cutoff. During the infant-led play, she used 4 do skills and 57 don’t skills.
During the intervention, the mother regularly participated in 7 sessions occurring over 7 weeks and completed homework practice 96% of the time. The father participated in the teach session, but was not present for any of the coaching sessions. Despite the mother’s high rate of homework completion, she had a difficult time following instructions during coaching, possibly due to Spanish being her second language. For example, the therapist suggested the mother move a small table to another room during play to prevent the child from climbing on it (which had been a problem during previous sessions). The mother did not respond until the third request when the therapist pointed at the table and subsequently to the adjacent room. Therefore, the therapist took advantage of using gestures to help facilitate communication during coaching, which would not have been possible from behind the one-way mirror in clinic-based treatment. At the postintervention and follow-up assessments, the mother displayed statistically reliable increases in her use of do skills and decreases in her use of don’t skills. Although the mother reported statistically reliable change on the ITSEA externalizing scale at the postintervention assessment, her scores on both ISTEA scales were still above the clinical cutoff. However, at the follow-up, the mother’s report on both ITSEA scales, as well as on the CBCL scales, was statistically reliable and below the clinical cutoff on the ITSEA dysregulation scale and the CBCL externalizing sale. The mother reported moderate satisfaction with a 43 on the TAI.
Child 5
The fifth infant was a Hispanic, 12-month-old girl who lived with her biological mother and father, aged 38 and 50 years, respectively, and had no significant medical history or developmental concerns. Her parents were married and both immigrated to the U.S. from Nicaragua several years prior to the screening. The mother, a high school graduate, and father, who attended some college, spoke only Spanish and reported a combined annual income of $36,000. The mother reported minimal concerns with her infant but mentioned her infant “throws herself backwards when she doesn’t get what she wants” and that she often engaged in some difficult behaviors (e.g., restless, problems sleeping, destructive). On the ITSEA, the mother’s report was above the clinical cutoff on the externalizing but not the dysregulation scale. During the infant-led play, she used 3 do skills and 55 don’t skills.
The mother participated in 7 sessions occurring over 11 weeks and completed homework practice only 21% of the time. The father participated only in the teach session and the first coach session. Both parents expressed a willingness to “do whatever it takes” for their infant. However, after the initial problem of “throwing herself backward” was no longer an issue, the mother seemed to lose interest in the program, as demonstrated by her low rate of homework completion. When the therapist asked the mother about other issues unrelated to the infant’s behavior at the beginning of the fourth coach session (as outlined in the intervention manual), the mother discussed her high level of stress due to her husband’s recent back surgery. In response, the therapist provided support and helped problem solve ways for this mother to cope with her stress (e.g., ask friends and family to occasionally help take care of the infant). At the postintervention and follow-up assessments, the mother displayed statistically reliable increases in her use of do skills and decreases in her use of don’t skills. Although the mother’s report on the ITSEA did not reliably change over time, both scales were below the cutoff by the follow-up. Alternatively, her report on the CBCL scales represented a statistically reliable change from the ITSEA scales at baseline with both scales below the clinical cutoff. The mother reported moderate satisfaction with the intervention with a 41 on the TAI. It is possible that the mother’s motivation for participating in the intervention decreased after the initial concern (i.e., infant throwing herself) was resolved, which may explain some of the lack of statistically reliable changes over time.
Child 6
The sixth infant, a 12-month-old Hispanic female who lived with her mother and 5-year-old sister, as well as her grandmother and two uncles, had no significant medical history or developmental concerns. The mother was single and never married the child’s father, who had minimal contact with the infant. The mother, aged 28 years, had a high school diploma and reported an annual income of only $1,080, although she lived in her mother’s house and likely received financial assistance from her mother. Of note, the mother also received a score of 75 on the WASI, suggesting borderline cognitive functioning. She reported minimal concerns about her infant’s behavior when asked during the screening but acknowledged on the BITSEA that her infant was often restless, had frequent tantrums, and became easily upset. On the ITSEA, the mother’s report was below the clinical cutoff on the externalizing scale but above the clinical cutoff on the dysregulation scale. During the infant-led play, she used 4 do skills and 31 don’t skills.
The mother completed 6 sessions occurring over 9 weeks and cancelled 4 sessions when the therapist called to confirm the appointments. Additionally, there were often several other family members in the home during sessions, which was distracting to the mother and infant. The mother reported practicing regularly (96%) but had some difficulty mastering the skills, such as understanding the difference between labeled praises and behavior descriptions. In response, the therapist conducted drills with the mother during coaching to help her increase the use of each skill. For example, the therapist and mother generated a list of labeled praises to use with the infant, and then the mother was instructed to only use those labeled praises for a 5-minute coaching period. This mother eventually was able master her use of the skills and reported the skills “were starting to feel more natural” by the end of the intervention. The mother displayed statistically reliable increases in her use of do skills and decreases in her use of don’t skills. The mother’s report was only statistically reliable for the ITSEA dysregulation scale and the CBCL externalizing scale at follow-up, and all scales at the follow-up were below the clinical cutoff, except the ITSEA externalizing scale, which was at the clinical cutoff. Given the mother’s lower cognitive functioning, it is possible that she misunderstood some of the questions on the ITSEA because her scores were not entirely consistent with her report of changes in the infant’s behavior. Nevertheless, her cognitive functioning did not limit her acquisition of the parenting skills, and she also reported somewhat high satisfaction with a 44 on the TAI.
Child 7
The seventh infant was a Hispanic, 14-month-old girl; she lived with her biological mother and father, aged 25 and 27, respectively, and had no significant medical history or developmental concerns. Her parents were married, grew up in the U.S., were bilingual in English and Spanish, were high school graduates, and reported a total annual income of $15,900. The mother was not approached at the pediatric clinic but contacted our staff after seeing a program brochure and reported “hitting and tantrums” as her main concerns, as well as some other difficult behaviors (e.g., easily upset, restless). On the ITSEA, the mother’s report was above the cutoff on the externalizing scale but below the cutoff on the dysregulation scale.
This family only completed the teach and first coach session and would not return calls to schedule subsequent sessions. The mother was about 7 months pregnant and moved 4 hours away from the research site after the second session, so it is likely these barriers prevented her from completing the program. Of note, the mother had the most do and fewest don’t skills during the mother-infant interaction at the baseline, so it is also possible that she was less motivated to complete the intervention given her strong skills at the outset. It is possible that she felt confident in using these skills to promote positive infant behavior after only 2 sessions, but we, unfortunately, were unable to contact her to administer the questionnaires over the phone to assess the infant’s behavior following those 2 sessions.
Discussion
The current study was an open trial of a novel, home-based adaptation of the CDI phase of PCIT for 12- to 15-month-old at-risk infants and their families. The intervention was feasible and acceptable to the families. Implementation of a preventive intervention can be challenging because families do not actively seek services (i.e., call a clinic to pursue services) and may not recognize behavior problems in their infant, which may reduce their perceived need for services. Our strategy to actively recruit families at a large pediatric primary care clinic was largely successful, with 78% of approached families agreeing to participate in the intervention, if eligible. It is possible that this recruitment approach can be generalized to community pediatric practices. However, nurses, physicians, or other health care professionals would need to be amenable to administering and scoring a brief questionnaire and referring the family to an agency to provide the intervention when appropriate. Research on the feasibility of using this recruitment strategy in nonresearch settings would be important. Additionally, it would be helpful to consider access to mental health services, particularly among high-risk families (McKay & Bannon, 2004), in order to understand how best to implement this intervention in community settings by state-funded providers, such as early intervention.
Only one family dropped out of the intervention (14%), which is considerably lower than standard PCIT (42%; Boggs et al., 2004) and other clinic-based parent-training interventions (approaching 50%; Reyno & McGrath, 2006). The retention rates in the current study are especially impressive because the sample included at-risk families that are more likely in the first place to drop out from traditional parent-training treatments (Lavigne et al., 2010; Reyno & McGrath, 2006). The therapists were consistently in contact with the families by calling them the day before and an hour before the session appointment time. We found that maintaining this consistent contact with the families helped to decrease rates of no shows and provided an opportunity for parents to cancel the session, if necessary. However, there were some instances in which families were not home when we arrived to conduct a session. Although we rescheduled these appointments, flexibility may not always be possible in practice. Therefore, future work should examine predictors of families not showing up for home visits and how to improve attendance to reduce the research-practice gap. Implementing the home-based intervention was, for the most part, similar to providing clinic-based treatments for the clinicians, except that each session took more time when including travel time to the home. However, the average number of total sessions was half the number of sessions in standard PCIT. Overall, parents had moderately high rates of homework completion and reported high satisfaction with the intervention. Finally, the therapists adhered strictly to the intervention model, highlighting the value of the structured manual developed for this open trial.
In addition to the feasibility and acceptability, mothers demonstrated statistically significant improvements in their interactions with their infant both immediately following the intervention and at the follow-up. Specifically, they were more positive and better able to follow their infant’s lead in play. Improvements in the mother-infant interactions were also demonstrated by effect sizes (Cohen’s d) larger than 2.27 and yielded an 8-fold increase in do skills and a 10-fold decrease in don’t skills from baseline to follow-up. Additionally, all families demonstrated statistically reliable change in skill acquisition from the baseline to postintervention assessments, and five of the six completers displayed reliable change in skill acquisition from the baseline to follow-up assessments. These findings suggest mothers completing the intervention successfully learned the skills taught during the intervention and maintained the use of these skills 4 to 6 months later.
Overall, maternal report of infant externalizing and dysregulation problems on the ITSEA decreased immediately following the intervention. Although we recognize the preliminary nature of this study and limitations to interpreting effect sizes with a small sample, the changes in scores from baseline to follow-up approached statistical significance with effect sizes that are consistent with other PCIT studies with at-risk toddlers and preschoolers (e.g., Bagner & Eyberg, 2007; Bagner et al., 2010). Additionally, we examined behavioral outcomes on the CBCL at follow-up when all infants were at least 18 months. Changes between the associated ITSEA scales at baseline and the CBCL scales at follow-up were statistically significant. Only 1 of the 6 infants that completed the intervention was at or above the clinical cutoff on the CBCL scales. Moreover, between 50% and 83% of the mothers reported statistically reliable change between the baseline and postintervention assessments and between the baseline and follow-up assessments. Taken together, the improvements in these behavior problems several months after completing the intervention suggest infants continue to make progress over time and highlight the potential long-term preventive impact of the intervention.
Study Limitations
The findings of the current open trial should be interpreted as preliminary due to some limitations. First, the sample size was small, and the lack of statistically significant findings in immediate changes in infant behavior problems may have been due to reduced power. In addition, the small sample limits generalizability of our findings. For example, all of the infants were female and Hispanic. We did not anticipate gender differences based on the BITSEA normative sample at this young age range (Carter & Briggs-Gowan, 2006), and the lack of males may have been due to a sampling error (e.g., only 27% of screened infants were male). The inclusion of all Hispanic families and 50% monolingual Spanish speakers characterizes a diverse and underrepresented sample in previous research. Nevertheless, the homogeneity of the sample narrows the generalizability to other ethnic groups.
Second, the lack of a control group limits the ability to rule out internal threats to validity, including regression to the mean and patient expectancy effects. A randomized controlled trial would increase confidence in interpreting the effects of the intervention and is currently in progress under the direction of the first author. Third, the follow-up period was modest given the preventive focus of the intervention, and future research should examine the longer-term outcomes of the infants and parents receiving this intervention. Fourth, data on the questionnaires and observations were collected only from the mothers. Although fathers and other caregivers were invited to participate in the intervention sessions, findings would be enhanced if data were collected from a second caregiver. Finally, intervention fidelity was measured by the extent to which the therapists adhered to the manual consistent with previous PCIT research. However, therapist competence is another important element of fidelity (Perepletchikova & Kazdin, 2005) that should be assessed in future studies to ensure the intervention was delivered in a skillful manner. Despite these limitations, however, the current open trial represents an important first step, and future research should include a randomized controlled trial to fully evaluate the efficacy of this intervention.
Summary and Implications
The current open trial provided promising preliminary effects of a home-based, preventive parenting intervention that can meet the challenging clinical needs of at-risk families. Toddlers and preschoolers from at-risk families are disproportionately affected by externalizing disorders and less likely to complete or benefit from traditional clinic-based parent-training treatments. Addressing these problems early in infancy before making a diagnosis can help reduce the economic burden of longer and more intensive treatments for families and enhance the long-term outcomes for these at-risk infants and their families. Additionally, providing the intervention in the home is feasible and a common practice for many clinicians working with infants and toddlers in the community (Sweet & Appelbaum, 2004). Existing evidence-based behavioral parent-training interventions do not typically address problems prior to 2 years, and the field of infant mental health is relatively young, with limited evidence for infant interventions. Therefore, it is our hope that further research is conducted on developing and testing behavioral parent-training interventions for infants and their families such as the one examined in this open trial to help promote the use of preventive services in clinical practice and ultimately to impact policy decisions.
Highlights.
A parent-training program was adapted as a preventive intervention for infants.
The intervention was brief and home-based to reduce burden for low-income families.
Initial outcome demonstrated promise of the intervention for infants and families.
Future research should examine this preventive intervention with a larger sample.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Contributor Information
Daniel M. Bagner, Florida International University
Gabriela M. Rodríguez, Florida International University
Clair A. Blake, Florida International University
Jose Rosa-Olivares, Miami Children’s Hospital.
References
- Achenbach TM, Rescorla LA. Manual for the ASEBA Preschool Forms & Profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families; 2000. [Google Scholar]
- Aguilar B, Sroufe LA, Egeland B, Carlson E. Distinguishing the early-onset/persistent and adolescence-onset antisocial behavior types: From birth to 16 years. Development and Psychopathology. 2000;12:109–132. doi: 10.1017/s0954579400002017. [DOI] [PubMed] [Google Scholar]
- Bagner DM, Eyberg SM. Parent-child interaction therapy for disruptive behavior in children with mental retardation: A randomized controlled trial. Journal of Clinical Child and Adolescent Psychology. 2007;36:418–429. doi: 10.1080/15374410701448448. [DOI] [PubMed] [Google Scholar]
- Bagner DM, Rodríguez GM, Blake CA, Linares D, Carter AS. Assessment of behavioral and emotional problems in infancy: A systematic review. Clinical Child and Family Psychology Review. 2012 doi: 10.1007/s10567-012-0110-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bagner DM, Sheinkopf SJ, Vohr BR, Lester BM. Parenting intervention for externalizing behavior problems in children born premature: An initial examination. Journal of Developmental and Behavioral Pediatrics. 2010;31:209–216. doi: 10.1097/DBP.0b013e3181d5a294. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Boggs SR, Eyberg SM, Edwards DL, Rayfield A, Jacobs J, Bagner D, Hood K. Outcomes of parent-child interaction therapy: A comparison of treatment completers and study dropouts one to three years later. Child and Family Behehavior Therapy. 2004;26:1–22. [Google Scholar]
- Brestan EV, Eyberg SM. Effective psychosocial treatments of conduct-disordered children and adolescents: 29 years, 82 studies, and 5,272 kids. Journal of Clinical Child and Adolescent Psychology. 1998;27:180–189. doi: 10.1207/s15374424jccp2702_5. [DOI] [PubMed] [Google Scholar]
- Brestan EV, Jacobs JR, Rayfield AD, Eyberg SM. A consumer satisfaction measure for parent-child treatments and its relationship to measures of child behavior change. Behavior Therapy. 1999;30:17–30. [Google Scholar]
- Briggs-Gowan MJ, Carter AS, Bosson-Heenan J, Guyer AE, Horwitz SM. Are infant-toddler social-emotional and behavioral problems transient? Journal of the American Academy of Child and Adolescent Psychiatry. 2006;45:849–858. doi: 10.1097/01.chi.0000220849.48650.59. [DOI] [PubMed] [Google Scholar]
- Briggs-Gowan MJ, Carter AS, Irwin JR, Wachtel K, Cicchetti DV. The Brief Infant-Toddler Social and Emotional Assessment: Screening for social-emotional problems and delays in competence. Journal of Pediatric Psychology. 2004;29:143–155. doi: 10.1093/jpepsy/jsh017. [DOI] [PubMed] [Google Scholar]
- Brinkmeyer M, Eyberg SM. Parent-child interaction therapy for oppositional children. In: Kazdin AE, Weisz JR, editors. Evidence-based psychotherapies for children and adolescents. New York, NY: Guilford; 2003. pp. 204–223. [Google Scholar]
- Campbell SB, Shaw DS, Gilliom M. Early externalizing behavior problems: toddlers and preschoolers at risk for later maladjustment. Development and Psychopathology. 2000;12:467–488. doi: 10.1017/s0954579400003114. [DOI] [PubMed] [Google Scholar]
- Carter AS, Briggs-Gowan MJ. ITSEA/BITSEA: Infant Toddler and Brief Infant Toddler Social and Emotional Assessment Examiner's Manual. San Antonio, TX: Harcourt Assessment; 2006. [Google Scholar]
- Carter AS, Briggs-Gowan MJ, Davis NO. Assessment of young children's social-emotional development and psychopathology: Recent advances and recommendations for practice. Journal of Child Psychology and Psychiatry. 2004;45:109–134. doi: 10.1046/j.0021-9630.2003.00316.x. [DOI] [PubMed] [Google Scholar]
- Carter AS, Briggs-Gowan MJ, Jones SM, Little TD. The Infant-Toddler Social and Emotional Assessment (ITSEA): Factor structure, reliability, and validity. Journal of Abnormal Child Psychology. 2003;31:495–514. doi: 10.1023/a:1025449031360. [DOI] [PubMed] [Google Scholar]
- Carter AS, Little C, Briggs-Gowan M, Kogan N. The Infant-Toddler Social-Emotional Assessment (ITSEA): Comparing parent ratings to laboratory observations of task mastery, emotion regulation, coping behaviors, and attachment status. Infant Mental Health Journal. 1999;20:375–392. [Google Scholar]
- Chu BC, Colognori D, Weissman AS, Bannon K. An initial description and pilot of Group Behavioral Activation Therapy for Anxious and Depressed Youth. Cognitive and Behavioral Practice. 2009;16:408–419. [Google Scholar]
- Dishion TJ, Kavanagh K. Intervening in adolescent problem behavior: A family-centered approach. New York, NY: Guilford Press; 2003. [Google Scholar]
- Dishion TJ, Shaw D, Connell A, Gardner F, Weaver C, Wilson M. The Family Check-Up with high-risk indigent families: Preventing problem behavior by increasing parents’ positive behavior support in early childhood. Child Development. 2008;79:1395–1414. doi: 10.1111/j.1467-8624.2008.01195.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dishion TJ, Stormshak EA. Intervening in children's lives: An ecological, family-centered approach to mental health care. Washington, DC: American Psychological Association; 2007. [Google Scholar]
- DuMont K, Mitchell-Herzfeld S, Greene R, Lee E, Lowenfels A, Rodriguez M, Dorabawila V. Healthy Families New York (HFNY) randomized trial: Effects on early child abuse and neglect. Child Abuse and Neglect. 2008;32:295–315. doi: 10.1016/j.chiabu.2007.07.007. [DOI] [PubMed] [Google Scholar]
- Eisenberg N, Guthrie IK, Fabes RA, Shepard S, Losoya S, Murphy BC, Reiser M. Prediction of elementary school children's externalizing problem behaviors from attentional and behavioral regulation and negative emotionality. Child Development. 2000;71:1367–1382. doi: 10.1111/1467-8624.00233. [DOI] [PubMed] [Google Scholar]
- Eisenstadt TH, Eyberg SM, McNeil CB, Newcomb K, Funderburk BW. Parent-child interaction therapy: Interim report of a randomized trial with short-term maintenance. Journal of Clinical Child Psychology. 1993;27:34–45. doi: 10.1207/s15374424jccp2701_4. [DOI] [PubMed] [Google Scholar]
- Eyberg SM. Consumer satisfaction measures for assessing parent training programs. In: VandeCreek L, Knapp S, Jackson TL, editors. Innovations in clinical practice: A sourcebook. Vol. 12. Sarasota, FL: Professional Resource Press; 1993. [Google Scholar]
- Eyberg SM. Tailoring and adapting parent-child interaction therapy for new populations. Education and Treatment of Children. 2005;28:197–201. Retrieved from http://www.educationandtreatmentofchildren.net/index.html. [Google Scholar]
- Eyberg SM, Funderburk BW, Hembree-Kigin TL, McNeil CB, Querido JG, Hood K. Parent-child interaction therapy with behavior problem children: One and two year maintenance of treatment effects in the family. Child and Family Behavior Therapy. 2001;23:1–20. [Google Scholar]
- Eyberg SM, Nelson MM, Boggs SR. Evidence-based psychosocial treatments for children and adolescents with disruptive behavior. Journal of Clinical Child and Adolescent Psychology. 2008;37:1–23. doi: 10.1080/15374410701820117. [DOI] [PubMed] [Google Scholar]
- Eyberg SM, Nelson MM, Duke M, Boggs SR. Manual for the dyadic parent-child interaction coding system. (3rd ed.) 2005 Available at www.pcit.org.
- Fernandez MA, Eyberg SM. Predicting treatment and follow-up attrition in parent-child interaction therapy. Journal of Abnormal Child Psychology. 2009;37:431–441. doi: 10.1007/s10802-008-9281-1. [DOI] [PubMed] [Google Scholar]
- Forcada-Guex M, Pierrehumbert B, Borghini A, Moessinger A, Muller-Nix C. Early dyadic patterns of mother-infant interactions and outcomes of prematurity at 18 months. Pediatrics. 2006;118:e107–e114. doi: 10.1542/peds.2005-1145. [DOI] [PubMed] [Google Scholar]
- Foster EM, Jones DE. The high costs of aggression: public expenditures resulting from conduct disorder. American Journal of Public Health. 2005;95:1767–1772. doi: 10.2105/AJPH.2004.061424. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fox NA, Calkins SD. The development of self-control of emotion: Intrinsic and extrinsic influences. Motivation and Emotion. 2003;27:7–26. [Google Scholar]
- Gardner F, Shaw DS, Dishion TJ, Burton J, Supplee L. Randomized prevention trial for early conduct problems: Effects on proactive parenting and links to toddler disruptive behavior. Journal of Family Psychology. 2007;21:398–406. doi: 10.1037/0893-3200.21.3.398. [DOI] [PubMed] [Google Scholar]
- Guzell JR, Vernon-Feagans L. Parental perceived control over caregiving and its relationship to parent-infant interaction. Child Development. 2004;75:134–146. doi: 10.1111/j.1467-8624.2004.00659.x. [DOI] [PubMed] [Google Scholar]
- Harwood MD, Eyberg SM. Child-directed interaction: Prediction of change in impaired mother-child functioning. Journal of Abnormal Child Psychology. 2006;34:335–347. doi: 10.1007/s10802-006-9025-z. [DOI] [PubMed] [Google Scholar]
- Hays JR, Reas DL, Shaw JB. Concurrent validity of the Wechsler abbreviated scale of intelligence and the Kaufman brief intelligence test among psychiatric inpatients. Psychological Reports. 2002;90:355–359. doi: 10.2466/pr0.2002.90.2.355. [DOI] [PubMed] [Google Scholar]
- Hood KK, Eyberg SM. Outcomes of parent-child interaction therapy: mothers' reports of maintenance three to six years after treatment. Journal of Clinical Child and Adolescent Psychology. 2003;32:419–429. doi: 10.1207/S15374424JCCP3203_10. [DOI] [PubMed] [Google Scholar]
- Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology. 1991;59:12–19. doi: 10.1037//0022-006x.59.1.12. [DOI] [PubMed] [Google Scholar]
- Jones Harden B, Chazan-Cohen R, Raikes H, Vogel C. Early head start home visitation: The role of implementation in bolstering program benefits. Journal of Community Psychology. 2012;40:438–455. [Google Scholar]
- Keenan K, Wakschlag LS. More than the terrible twos: The nature and severity of behavior problems in clinic-referred preschool children. Journal of Abnormal Child Psychology. 2000;28:33–46. doi: 10.1023/a:1005118000977. [DOI] [PubMed] [Google Scholar]
- Lavigne JV, Lebailly SA, Gouze KR, Binns HJ, Keller J, Pate L. Predictors and correlates of completing behavioral parent training for the treatment of oppositional defiant disorder in pediatric primary care. Behavior Therapy. 2010;41:198–211. doi: 10.1016/j.beth.2009.02.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Luby J, Morgan K. Characteristics of an infant/preschool psychiatric clinic sample: Implications for clinical assessment and nosology. Infant Mental Health Journal. 1997;18:209–220. [Google Scholar]
- Lundahl B, Risser HJ, Lovejoy MC. A meta-analysis of parent training: Moderators and follow-up effects. Clinical Psychology Review. 2006;26:86–104. doi: 10.1016/j.cpr.2005.07.004. [DOI] [PubMed] [Google Scholar]
- Lunkenheimer ES, Dishion TJ, Shaw DS, Connell AM, Gardner F, Wilson MN, et al. Collateral benefits of the family check-up on early childhood school readiness: Indirect effects of parents' positive behavior support. Developmental Psychology. 2008;44:1737–1752. doi: 10.1037/a0013858. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lyons-Ruth K, Easterbrooks MA, Cibelli CD. Infant attachment strategies, infant mental lag, and maternal depressive symptoms: Predictors of internalizing and externalizing problems at age 7. Developmental Psychology. 1997;33:681–692. doi: 10.1037//0012-1649.33.4.681. [DOI] [PubMed] [Google Scholar]
- McCabe K, Yeh M. Parent-child interaction therapy for Mexican Americans: A randomized clinical trial. Journal of Clinical Child and Adolescent Psychology. 2009;38:753–759. doi: 10.1080/15374410903103544. [DOI] [PubMed] [Google Scholar]
- McCabe K, Yeh M, Lau A, Argote CB, Liang J. Parent-child interactions among low-income Mexican American parents and preschoolers: Do clinic-referred families differ from nonreferred families? Behavior Therapy. 2010;41:82–92. doi: 10.1016/j.beth.2009.01.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- McKay MM, Bannon WM., Jr Engaging families in child mental health services. Child and Adolescent Psychiatric Clinics of North America. 2004;13:905–921. doi: 10.1016/j.chc.2004.04.001. [DOI] [PubMed] [Google Scholar]
- Minkovitz CS, Hughart N, Strobino D, Scharfstein D, Grason H, Hou W, Guyer B. A practice-based intervention to enhance quality of care in the first 3 years of life: The healthy steps for young children program. The Journal of the American Medical Association. 2003;290:3081–3091. doi: 10.1001/jama.290.23.3081. [DOI] [PubMed] [Google Scholar]
- Moffitt TE, Caspi A, Harrington H, Milne BJ. Males on the life-course-persistent and adolescence-limited antisocial pathways: Follow-up at age 26 years. Development and Psychopathology. 2002;14:179–207. doi: 10.1017/s0954579402001104. [DOI] [PubMed] [Google Scholar]
- Mouton-Simien P, McCain AP, Kelley ML. The development of the Toddler Behavior Screening Inventory. Journal of Abnormal Child Psychology. 1997;25:59–64. doi: 10.1023/a:1025759408417. [DOI] [PubMed] [Google Scholar]
- Muñoz RF, Mrazek PJ, Haggerty RJ. Institute of Medicine report on prevention of mental disorders: Summary and commentary. American Psychologist. 1996;51:1116–1122. doi: 10.1037//0003-066x.51.11.1116. [DOI] [PubMed] [Google Scholar]
- Muntz R, Hutchings J, Edwards RT, Hounsome B, O'Ceilleachair A. Economic evaluation of treatments for children with severe behavioural problems. The Journal of Mental Health Policy and Economics. 2004;7:177–189. Retrieved from http://search.proquest.com/docview/620653819?accountid=10901. [PubMed] [Google Scholar]
- Nix RL, Bierman KL, McMahon RJ. How attendance and quality of participation affect treatment response to parent management training. Journal of Consulting and Clinical Psychology. 2009;77:429–438. doi: 10.1037/a0015028. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Olds DL. Prenatal and infancy home visiting by nurses: from randomized trials to community replication. Prevention Science. 2002;3:153–172. doi: 10.1023/a:1019990432161. [DOI] [PubMed] [Google Scholar]
- Olds DL. The Nurse-Family Partnership: An evidence-based preventive intervention. Infant Mental Health Journal. 2006;27:5–25. doi: 10.1002/imhj.20077. [DOI] [PubMed] [Google Scholar]
- Olds DL, Sadler L, Kitzman H. Programs for parents of infants and toddlers: Recent evidence from randomized trials. Journal of Child Psychology and Psychiatry. 2007;48:355–391. doi: 10.1111/j.1469-7610.2006.01702.x. [DOI] [PubMed] [Google Scholar]
- Patterson GR. Coercive family process. Eugene, OR: Castilia; 1982. [Google Scholar]
- Paulsell D, Avellar S, Sama Martin E, Del Grosso P. Home visiting evidence of effectiveness review: Executive summary. Washington DC: Office of Planning, Research, and Evaluations, Administration for Children and Families, U.S. Department of Health and Human Services; 2010. [Google Scholar]
- Perepletchikova F, Kazdin AE. Treatment integrity and therapeutic change: Issues and research recommendations. Clinical Psychology: Science and Practice. 2005;12:365–383. [Google Scholar]
- Pons JI, Flores-Pabón L, Matías-Carrelo L, Rodríguez M, Rosario-Hernández E, Rodríguez JM, et al. Confiabilidad de la Escala de Inteligencia Wechsler para Adultos Versión III, Puerto Rico (EIWA-III) Revista Puertorriqueña de Psicología. 2008;19:112–132. Retrieved from http://web.me.com/nvaras/RePS/P%C3%A1gina_Inicial.html. [Google Scholar]
- Pons JI, Matías-Carrelo L, Rodríguez M, Rodríguez JM, Herrans LL, Jiménez ME, et al. Estudios de validez de la Escala de Inteligencia Wechsler para Adultos Versión III, Puerto Rico (EIWA-III) Revista Puertorriqueña de Psicología. 2008;19:75–111. Retrieved from http://web.me.com/nvaras/RePS/P%C3%A1gina_Inicial.html. [Google Scholar]
- Prinz RJ, Miller GE. Family-based treatment for childhood antisocial behavior: Experimental influences on dropout and engagement. Journal of Consulting and Clinical Psychology. 1994;62:645–650. doi: 10.1037//0022-006x.62.3.645. [DOI] [PubMed] [Google Scholar]
- Qi CH, Kaiser AP. Behavior problems of preschool children from low-income families: Review of the literature. Topics in Early Childhood Special Eduaction. 2003;23:188–216. [Google Scholar]
- Reyno SM, McGrath PJ. Predictors of parent training efficacy for child externalizing behavior problems: A meta-analytic review. Journal of Child Psycholology and Psychiatry. 2006;47:99–111. doi: 10.1111/j.1469-7610.2005.01544.x. [DOI] [PubMed] [Google Scholar]
- Roggman LA, Cook GA, Peterson CA, Raikes HH. Who drops out of early Head Start home visiting programs? Early Education & Development. 2008;19:574–599. [Google Scholar]
- Schuhmann EM, Foote RC, Eyberg SM, Boggs SR, Algina J. Efficacy of parent-child interaction therapy: Interim report of a randomized trial with short-term maintenance. Journal of Clinical Child Psychology. 1998;27:34–45. doi: 10.1207/s15374424jccp2701_4. [DOI] [PubMed] [Google Scholar]
- Shaw DS, Dishion TJ, Supplee L, Gardner F, Arnds K. Randomized trial of a family-centered approach to the prevention of early conduct problems: 2-year effects of the family check-up in early childhood. Journal of Consulting and Clinical Psychology. 2006;74:1–9. doi: 10.1037/0022-006X.74.1.1. [DOI] [PubMed] [Google Scholar]
- Shaw DS, Gilliom M, Ingoldsby EM, Nagin DS. Trajectories leading to school-age conduct problems. Develpmental Psychology. 2003;39:189–200. doi: 10.1037//0012-1649.39.2.189. [DOI] [PubMed] [Google Scholar]
- Squires J, Twombly E, Bricker D, Potter L. The Ages & Stages Questionnaires— Third edition. Baltimore, MD: Brookes; 2009. [Google Scholar]
- Shelleby EC, Shaw DS, Cheong J, Chang H, Gardner F, Dishion TJ, Wilson MN. Behavioral control in at-risk toddlers: The influence of the Family Check-up. Journal of Clinical Child & Adolescent Psychology. 2012;41:288–301. doi: 10.1080/15374416.2012.664814. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sweet MA, Appelbaum MI. Is home visiting an effective strategy? A meta-analytic review of home visiting programs for families with young children. Child Development. 2004;75:1435–1456. doi: 10.1111/j.1467-8624.2004.00750.x. [DOI] [PubMed] [Google Scholar]
- Tarabulsy GM, Tessier R, Kappas A. Contingency detection and the contingent organization of behavior in interactions: Implications for socioemotional development in infancy. Psychological Bulletin. 1996;120:25–41. doi: 10.1037/0033-2909.120.1.25. [DOI] [PubMed] [Google Scholar]
- Thomas JM, Guskin KA. Disruptive behavior in young children: What does it mean? Journal of the American Academy of Child and Adolescent Psychiatry. 2001;40:44–51. doi: 10.1097/00004583-200101000-00015. [DOI] [PubMed] [Google Scholar]
- van Zeijl J, Mesman J, Stolk MN, Alink LR, van Ijzendoorn MH, Bakermans-Kranenburg MJ, Koot HM. Terrible ones? Assessment of externalizing behaviors in infancy with the Child Behavior Checklist. Journal of Child Psychology and Psychiatry. 2006;47:801–810. doi: 10.1111/j.1469-7610.2006.01616.x. [DOI] [PubMed] [Google Scholar]
- van Zeijl J, Mesman J, Van IMH, Bakermans-Kranenburg MJ, Juffer F, Stolk MN, Alink LR. Attachment-based intervention for enhancing sensitive discipline in mothers of 1- to 3-year-old children at risk for externalizing behavior problems: A randomized controlled trial. Journal of Consulting and Clinical Psychology. 2006;74:994–1005. doi: 10.1037/0022-006X.74.6.994. [DOI] [PubMed] [Google Scholar]
- Wagner M, Spiker D, Linn MI. The effectiveness of the Parents as Teachers Program with low-income parents and children. Topics in Early Childhood Special Education. 2002;22:67–81. [Google Scholar]
- Wakschlag LS, Briggs-Gowan MJ, Hill C, Danis B, Leventhal BL, Keenan K, Carter AS. Observational Assessment of Preschool Disruptive Behavior, Part II: Validity of the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS) Journal of the American Academy of Child and Adolescent Psychiatry. 2008;47:632–641. doi: 10.1097/CHI.0b013e31816c5c10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wakschlag LS, Hill C, Carter AS, Danis B, Egger HL, Keenan K, et al. Observational Assessment of Preschool Disruptive Behavior, Part I: reliability of the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS) Journal of the American Academy of Child and Adolescent Psychiatry. 2008;47:622–631. doi: 10.1097/CHI.0b013e31816c5bdb. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ware LM, McNeil CB, Masse J, Stevens S. Efficacy of in-home parent-child interaction therapy. Child and Family Behavior Therapy. 2008;30:99–126. [Google Scholar]
- Wechsler D. Wechsler Adult Scale of Intelligence—Third edition. San Antonio, TX: Pychological Corportation; 1997. [Google Scholar]
- Wechsler D. Wechsler Abbreviated Scale of Intelligence. San Antonio, TX: Pychological Corportation; 1999. [Google Scholar]
- Wechsler D. Ecala de Inteligencia Wechsler Para Adultos. San Antonio, TX: Pearson; 2008. [Google Scholar]
- Werba BE, Eyberg SM, Boggs SR, Algina J. Predicting outcome in parent-child interaction therapy: Success and attrition. Behavior Modification. 2006;30:618–646. doi: 10.1177/0145445504272977. [DOI] [PubMed] [Google Scholar]