Validation of a Clinically Sensitive, Observational Coding System for Parenting Behaviors: The Parenting Clinical Observation Schedule

SYNOPSIS

Objective.

The psychometric properties of a newly developed, clinically-sensitive, observational measure of parenting behavior, the Parenting Clinical Observational System (P-COS), are presented. The P-COS expands on existing observational coding systems by integrating qualitative and quantitative aspects of parenting behavior to directly inform clinical decision-making with parents of young children with disruptive behaviors.

Design.

Reliability and validity data were examined in a sample of mother-preschooler dyads (N = 335), and test–re-test reliability was assessed on an independent sample (N = 29).

Results.

Evidence was found for the reliability and validity of the P-COS, including evidence of inter-rater reliability, internal consistency, and distinctness of the three theorized P-COS domains: Responsive Involvement, Constructive Discipline, and Problematic Discipline. Convergent and divergent validity data are presented as well. The P-COS demonstrated incremental validity in predicting child disruptive behavior over and above self-reported parenting and was associated with change in child clinical status over time.

Conclusions.

Clinically informative observation methods that assess multiple dimensions of parenting within the context of a child disruptive behavior assessment provide useful information for directing treatment efforts.

INTRODUCTION

The centrality of parenting as a proximal influence on children’s development is well established (Baumrind, 1991; Bornstein, 2002; Campbell, 1997; Darling & Steinberg, 1993). Most clinical interventions with young children focus on the parenting context. For example, the vast majority of empirically validated treatments for disruptive behavior place central emphasis on alteration of parenting strategies as a mechanism of change, particularly increasing consistency of, and reducing harshness of, discipline practices (e.g., Kazdin, 1985; Webster-Stratton, 1981). Similarly, Parent-Child Interaction Therapy (PCIT; Hembree-Kigin & McNeil, 1995) as well as interventions in the field of infant mental health designed to treat early problematic parent-child relationship problems (such as those common in children with disruptive behavior problems) also target parental contingent responsiveness (i.e., the capacity to read child cues and shift behavior appropriately) as a central focus of clinical intervention (Lieberman, 1992; Sameroff, McDonough, & Rosenblum, 2004).

Dimensions of Parenting

Two central dimensions of parenting – discipline practices and responsiveness – have consistently been identified as playing a critical role in the development and maintenance of child disruptive behavior (Locke & Prinz, 2001). Discipline practices include the ways parents issue commands or directions (e.g., harshness, firmness) as well as how parents track and respond to the child’s level of compliance/misbehavior (e.g., consistency, punishment). Discipline is comprised of both positive and problematic strategies. Positive discipline reflects the constructive strategies that parents utilize to help their children develop age-appropriate behavioral regulation skills and to internalize rules and norms, often through instruction and contingent responding. These include parenting behaviors such as monitoring, flexibility, and proactive parenting. Positive discipline has been associated with higher levels of child competence and lower levels of disruptive behavior (Gardner, Shaw, Dishion, Burton, & Supplee, 2007; Jackson, Brooks-Gunn, Huang, & Glassman, 2000; Koblinsky, Kuvalanka, & Randolph, 2006; Masten, Hubbard, Gest, Tellegen, Garmezy, & Ramirez, 1999; Pettit, Keiley, Laird, Bates, & Dodge, 2007). In contrast, problematic discipline, marked by coercive and inconsistent parenting behaviors, has been identified as a core contributor to the development and maintenance of child disruptive behavior (Frick, Lahey, Loeber, Stouthamer-Loeber, Christ, & Hanson, 1992; Haapasalo & Tremblay, 1994; Ipsa, Fine, Halgunseth, Harper, Robinson, & Boyce 2004; Patterson, Reid, & Dishion, 1992; Patterson & Stouthamer-Loeber, 1984; Stormshak, Bierman, McMahon, & Lengua, 2000).

Parenting encompasses more than management of behavior. It also includes the provision of a warm, supportive environment (O’Leary, 1995). Responsiveness reflects warm, sensitive, and contingent responding to child behavior, emotional availability, and the match between parent response and child behavior (Zeanah, Larrieu, Heller, & Valliere, 2000) and has been identified as a protective factor (Harrist, Pettit, Dodge, & Bates, 1994; Wakschlag & Hans, 1999). For example, responsiveness, including positive involvement, scaffolding, and emotional support, has been associated with greater child compliance and competence and lower risk of child disruptive behavior (Ipsa et al., 2004; Landry, Smith, & Swank, 2006; Martinez & Forgatch, 2001; Wakschlag & Hans, 1999). Conversely, lack of responsiveness is associated with the development of problem behaviors, including noncompliance, emotion regulation difficulties, and tantrums (Johnston, Murray, Hinshaw, Pelham, & Hoza, 2002; Kuczynski & Kochanska, 1990; McCarty, Zimmerman, DiGiuseppe, & Christakis, 2005; Pettit, Bates, & Dodge, 1997; Stormshak et al., 2000). Additional research highlights the multidimensional nature of responsiveness (Bornstein, Tamis-LeMonda, Hahn, & Haynes, in press; Landry, Smith, & Swank, 2006). For example, behavioral and emotional responsiveness are differentially related to child functioning (Wakschlag & Keenan, 2001), and different types of responsiveness demonstrate different developmental trajectories (Bornstein, Tamis-LeMonda, Hahn, & Haynes, in press).

Studies examining both discipline practices and responsiveness indicate that these dimensions are related but distinct constructs that provide incremental prediction of problematic child behavior as well as competence (Mize & Pettit, 1997; Pettit, Bates, & Dodge, 1997). Additional research suggests that the two domains are associated with child outcomes in a differential manner. For example, Stormshak, Bierman, McMahon, and Lengua (2000) reported that low levels of warmth were particularly associated with child oppositionality, whereas physical discipline was linked with child aggression. Finally, research highlights the importance of considering both positive and negative aspects of parenting behavior. Negative aspects of parenting constitute a limited minority of parent-child interactions (Gardner, Ward, Burton, & Wilson, 2003), and in addition, Martinez and Forgatch (2001) reported that changes in responsive parenting had twice the predictive validity for decreases in child noncompliance compared to problematic discipline practices. Positive parenting, such as warmth, also appears to moderate the association between physical punishment and children’s disruptive behavior (Deater-Deckard, Ivy, & Petrill, 2006).

Assessment of Key Parenting Dimensions

Despite the significance placed on parenting for the development of disruptive behavior in young children and as the target for intervention, nuanced and clinically informative assessment of multiple dimensions of parenting is lacking. Self-report measures have a long-standing tradition, both empirically and clinically, of providing important information on parenting behaviors. However, such measures reflect an aggregate of behaviors across time, contexts, and child behavior as well as parental subjective experience of their parenting behavior (Gonzalez & Mason, 1996). This is informative, but it may not fully capture the moment-to-moment strategies parents utilize in response to child behaviors. In addition, more nuanced parenting behaviors involving moment-to-moment shifting of behavior and attunement with the child may be more difficult to assess via self-report, compared to more concrete aspects of parenting behavior (e.g., physical discipline, praise).

Observation provides a method for assessing parenting behaviors within the dynamic context of interchange with the child. Furthermore, observational measurement of parenting has demonstrated stronger and more consistent prediction of child outcomes than self-reported parenting (Collins, Maccoby, Steinberg, Hetherington, & Bornstein, 2000; Rothbaum & Weisz, 1994; Zaslow, Weinfield, Gallagher, Hair, Ogawa, & Egeland, 2006). Consistent with this conclusion, the American Academy of Child and Adolescent Psychiatry (1997) recommends that direct assessment of parenting be a core component of clinical evaluation of young children.

Observational measurement of parenting.

Observational assessment of parenting within the parent-child context has been a hallmark of research in infant mental health and developmental psychology (Bornstein, 1989; Steelman, Assel, Swank, Smith, & Landry, 2002; Zeanah, Boris, Heller, Hinshaw-Fuselier, Larrier, & Lewis, 1997). However, perhaps because of the focus on typically developing children in this work, assessment of responsiveness has tended to yield continuous measures that do not specify cut-points for distinguishing competent from clinically concerning parenting. Furthermore, research from infant mental health and developmental psychology has focused less on problematic aspects of parenting. Based on research suggesting independent contributions of responsive and problematic parenting to negative child outcomes (Pettit, Bates, & Dodge, 1997; Deater-Deckard, Ivy, Petrill, 2006), the omission of the more negative dimension of parenting is a limitation of observational methodologies developed in infant mental health and developmental psychology, particularly from a clinical standpoint.

Patterson’s (1982) work on coercive family processes, which established an association between harsh parenting and escalating child behavior problems via naturalistic observation of interactional sequences, has been highly influential in identifying key aspects of parenting that contribute to emergent disruptive behavior. More clinically focused observational measures, such as the Dyadic Parent Child Interaction Coding System (DPICS; Robinson & Eyberg, 1981) and the Behavior Coding Scheme (BCS; Forehand & McMahon, 1981) have emphasized the importance of problematic discipline as well as more responsive dimensions of parenting (Locke & Prinz, 2001). In addition to assessing the use of harsh physical discipline, these coding systems have measured multiple components of discipline, including type of commands (e.g., direct, indirect), intrusiveness, and consistency. However, most observational measures have been developed and evaluated within the context of research assessments (Mash & Foster, 2001), and these measures have not typically characterized individual differences in parenting in a manner that is clinically informative. For example, frequency counts of discrete behaviors (e.g. number of times praise occurs) typically assessed by these coding systems (Locke & Prinz, 2001) permit identification of salient pieces of information (“the parts”), but do not directly capture sequences of behavior that are clinically important. As a result, frequency counts are not easily interpretable as a broad profile of strengths and weaknesses (“the whole”) that clinicians can extrapolate into intervention terms and convey to parents in clinical feedback.

Clinically informative parenting assessment.

Because there exists a wide range of “good enough” parenting behavior (Winnicot, 1965), the clinician organizes his/her questions around the issue of “when to worry.” The mere presence of some behaviors (e.g., harsh physical discipline, engagement in power struggles) is a clinical “red flag,” whereas other aspects of parenting must be assessed more qualitatively (e.g., parent laughs during child misbehavior) and/or within the context of a behavioral sequence (e.g., parent does not shift strategies in face of persistent noncompliance), to be clinically informative. The integration of these approaches is likely to more comprehensively capture parenting in a manner that will be clinically informative (Landry, Smith, & Swank, 2006).

Ideally, clinical observation should go beyond assessment of discrete behaviors to an integrated examination of qualitative and quantitative facets of behavior that weights their salience in determination of clinical concern (Wakschlag & Danis, in press). Thus, parenting may vary substantially, depending on its intent and context. For example, parents may differ in responses they initially display in an effort to convey a direction (e.g., firmness of direction): those they use in response to initial opposition (e.g., give in or remain firm in the face of noncompliance), and those employed in the face of persistent defiance from the child (e.g., flexibility in changing discipline strategies) (Garstein & Fagot, 2003; Vigilante & Wahler, 2005). These differences, contingent on the child’s behavior, are essential to flexible and effective parenting. The use of frequency ratios and sequential analyses of time-ordered behavior codes are better suited than simple frequency counts for capturing the interactive and bi-directional quality of family relationships (e.g., Gottman, Coan, Carrere, & Swanson, 1998; Greene & Anderson, 1999; Martinez & Forgatch, 2001), but the labor-intensive nature of microanalytic coding systems make them less suitable for application in clinical settings. Specifically, the results of these micro-level coding systems may be clinically informative, but clinicians are unlikely to use these methodologies because they are not clinically feasible. Consistent with this, there is growing recognition that simpler codes capturing more global properties of behavior have greater clinical utility than multicategory, microanalytic coding systems (Haynes, 2001; Mash & Foster, 2001).

The Parenting Clinical Observation Schedule (P-COS)

To address these gaps, we developed the Parenting Clinical Observation Schedule (P-COS; Wakschlag, Hill, Danis, Grace & Keenan, 2004). The P-COS is part of a larger observational schedule, the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS; Wakschlag, Hill, Carter, Danis, Egger, Keenan, Leventhal, Cicchetti, Maskowitz, & Briggs-Gowan, in press-c) and focuses on capturing key dimensions of competent and problematic parenting in a clinically informative manner during the Parent context of the DB-DOS. The Parent context includes “presses” (Lord, Rutter, Goode, Heemsbergen, Jordan, Mawhood, & Schopler, 1989) consisting of interactional tasks (e.g., compliance, withdrawal of parent’s attention), which increase the likelihood of observing key parent and child behaviors of clinical importance that might be unlikely to occur spontaneously during the course of more naturalistic observations (Mash & Foster, 2001). At the same time, presses are designed to reflect typical, naturally occurring situations between parent and child (e.g., putting crayons away, waiting while parent is busy).

The P-COS assesses parenting in three domains: Responsive Involvement, Constructive Discipline, and Problematic Discipline (see Appendix for list of items by P-COS domain). The P-COS codes are global judgments, integrating qualitative and quantitative features of parenting across the entire interactional context rather than counts of discrete behaviors. The P-COS uses multiple codes to capture behavior within a domain. In addition, codes take into account the child’s behavior and, therefore, the appropriateness and effectiveness of parental response. The system yields both distinct dimensions and an overall profile of parental competent and problematic behaviors.

To assess distinct aspects of competent parenting reflecting both responsiveness to positive behaviors and discipline in response to misbehaviors, the P-COS yields two parental competence domains: Responsive Involvement and Constructive Discipline. Responsive Involvement assesses parental engagement with the child, the provision of positive verbal and behavioral feedback (e.g., Responsivity to Positive Behaviors) and structure (e.g., Scaffolding). Constructive Discipline codes behaviors reflecting the parents’ ability to communicate directions in a clear, direct, and confident manner (i.e., Firmness), effectively manage the child’s behavior using positive behavior strategies (i.e., Positive Behavior Strategies), and the flexibility to shift strategies in the face of persisting noncompliance (i.e., Flexibility). The Problematic Discipline domain assesses multiple aspects of problematic parenting behavior, such as the use of inappropriate, harsh discipline (e.g., Physical Discipline, Verbally Aggressive Discipline) and immature behaviors (e.g., Hostile Behavior, Power Struggles) that contribute to harsh and otherwise ineffective parenting. In addition, the P-COS includes an emotional misattunement code drawn from research indicating that the emotional mismatch between mother and child is associated with stability of child behavior problems (Cole, Teti, & Zahn-Waxler, 2003). In the present study, we examine psychometric reliability and validity of the P-COS in a low-income sample of predominantly African American mothers of 3- to 5-year-old children, 70% of whom were recruited due to caregiver concerns about behavior.

METHODS

Participants

Two samples were recruited for the present study (i.e., Primary validation and Test – re-test; see below) to examine psychometric properties of the P-COS. Samples were recruited from clinics affiliated with two Midwestern universities serving urban, disadvantaged populations. Recruitment into the study was based on child behavior. Children were sampled along a behavioral continuum to ensure a high level of behavioral variability. Thus, we recruited mothers of children referred to an outpatient clinic due to disruptive behavior (including aggression, defiance, or problems controlling temper), mothers of nonreferred children with behavioral concerns, and mothers of nonreferred children without behavioral concerns. Nonreferred children were recruited from pediatric and family practice clinics. Inclusion criteria consisted of (1) child age between 3 and 5 years, (2) residence with biological mother, and (3) attendance in out-of-home day care or school. Children were excluded if they had a serious developmental disability.

Primary validation sample.

Three hundred thirty-six mother-child dyads were recruited for a longitudinal study of preschool disruptive behavior for children living in low-income environments (see Wakschlag, Briggs-Gowan, Carter, Hill, Danis, Keenan, McCarthy, & Leventhal, 2007, for additional details regarding participation rates by referral status). Forty percent (n = 134) of the participants were mothers of children referred to an outpatient clinic due to disruptive behavior, 30% (n = 102) were mothers of nonreferred children with behavioral concerns, and 30% (n = 100) were mothers of nonreferred children without behavioral concerns. One dyad was excluded due to missing data on the observation measure; thus, the analytic sample for the present study is 335. Eighty-four percent of the mothers were African American, with a mean age of 28.9 years (SD = 5.8; range = 18–46 years). Sixty-one percent of the mothers reported they were single parents, with an average annual income of $21,743 (SD = $16,544). The majority of mothers (87%) had completed at least a high school education. With regard to child demographics, 55% were male and the mean age was 53.6 months (SD = 10.1).

Test–re-test sample.

Fifty-nine percent (n = 17) were mothers of children referred for behavior problems, 10% (n = 3) were mothers of nonreferred children with behavioral concerns, and 31% (n = 9) were mothers of nonreferred children without behavioral concerns. The test–re-test sample was demographically comparable to the Primary validation sample.

Sociodemographic Variables

Mothers provided sociodemographic information on child and maternal age, sex, and ethnicity, maternal relationship status, family income, and maternal education.

Observed Parenting

The P-COS (Wakschlag et al., 2004) assesses competent and problematic parenting behavior during the Parent context of the DB-DOS (Wakschlag et al., in press-c). The Parent context is a 20-minute parent-child interaction that includes compliance “do” and “don’t,” frustration, and social play tasks. Parenting behavior is not scripted, and tasks are designed to simulate typical parent-child interactions. For example, three tasks are designed for active parent engagement (e.g., compliance, interactive play), and the remaining is a “withdrawal of attention” task in which the parent instructs the child to work independently while the parent completes a questionnaire. Procedures were explained to the mother prior to the start of the interaction, and simply worded task instructions were provided on flip cards, with transitions between tasks marked by the ringing of a bell to minimize intrusion by the examiner and to increase the likelihood that children viewed directions as being parent-initiated. Mothers were encouraged to act as they “typically would.”

Codes are global (e.g., capture parenting behavior across the context) and integrate quantitative (e.g., frequency) and qualitative (e.g., ability to shift strategies in the face of child noncompliance) dimensions of parenting behavior. As such, the continuum of parenting behaviors captured on the P-COS ranges from competent to clinically concerning, but codes are ordinal rather than continuous. That is, distinctions are made by defining qualitative “breakpoints,” marking the shift from competent to clinically concerning parenting behavior. Ratings are made on a four-point scale ranging from 0 (none) to 3 (high). Because the Problematic Discipline domain was specifically designed to identify clinical “red flags,” the presence of any behavior captured by this domain is considered indicative of clinically concerning behavior. In contrast, in the Responsive Involvement and Constructive Discipline domains, distinctions between “good enough” parenting and those behaviors which are clinically concerning are more qualitative in nature; thus, clinically, we are more concerned with behaviors that fall on the low end of the continuum (i.e., none and low), whereas distinctions at the higher end (moderate and high) are viewed as capturing behaviors within the range of “good enough” parenting.

Self-Reports of Parenting

Parenting Dimensions Inventory-Short Version (PDI)

(Power, 2002). Mother’s self-report of parenting styles and specific discipline practices were assessed using the Parenting Dimensions Inventory-Short Version. Mothers indicated the extent to which statements of parenting behaviors were descriptive of them, ranging from 1 (“not at all descriptive of me”) to 6 (“highly descriptive of me”). Items were summed to yield three subscales: Nurturance, Follow-Through, and Inconsistent Parenting. Scales demonstrate good reliability and differentiate parenting styles in a range of diverse populations, including low-income African American mothers (Kelly, Power, & Wimbush, 1992). Cronbach’s alpha for the subscales ranged from .69 for Follow-Through to .81 for Nurturance in the present sample.

Coping with Children’s Negative Emotions

(CCNES; Fabes, Poulin, Eisenberg, & Madden-Derdich, 2002) assessed mothers’ likelihood of responding to child negative emotions with various childrearing practices, using a seven-point Likert scale ranging from 1 (very unlikely) to 7 (very likely). Expressive Encouragement reflected supportive and encouraging parental responses. An aggregate of Harsh Parenting was calculated by taking the mean of the two harsh parenting subscales (Punitive and Minimization) (Fabes, Leonard, Kupanoff, & Martin, 2001). Subscales have demonstrated good internal consistency and test–re-test reliability, and scores have been related to children’s emotional competence (Fabes et al., 2002) and parental distress (Fabes, Leonard, Kupanoff, & Martin, 2001). Internal consistency for the present sample was acceptable (Cicchetti, 1994; Cronbach’s alpha equal .78 for Expressive Encouragement and .76 for Harsh Parenting).

Child Functioning

Maternal report.

Mothers were administered the Kiddie Disruptive Behavior Disorders Schedule (K-DBDS; Keenan, Wakschlag, Danis, Hill, Humphries, Duax, & Donald, 2007), a semi-structured clinical interview assessing DSM-IV disruptive behavior symptoms (i.e., Oppositional Defiant and Conduct Disorder) in preschool children. A developmentally enhanced scoring approach was used that incorporated multiple components of core symptoms (e.g., severity, frequency) to enhance developmental sensitivity was used (see Wakschlag, Briggs-Gowan, Hill, Danis, Leventhal, Keenan, & Carter, in press-a, for additional details). One-week test–re-test reliability for the total number of symptoms was good (ICC = .82), and referred children had significantly more mother-reported disruptive behavior symptoms (M = 4.34) compared to nonreferred children (M = 1.26) F (1, 333) = 75.57, p < .001.

Teacher report.

Teachers completed the Early Child Inventory (ECI; Gadow & Sprafkin, 1996), a DSM-based checklist developed to assess symptoms in preschool children. The ECI generates categorical and continuous symptom scores. Scores have demonstrated good test–re-test reliability, and they differentiate clinic-referred from nonreferred children (Gadow & Sprafkin, 1996). Internal consistency in the present sample was excellent (Cronbach’s alpha = .94), and referred children displayed significantly higher rates of teacher-reported disruptive behaviors symptoms (M = 3.01) compared to nonreferred children (M = .87), F (1, 294) = 33.06, p < .001.

Impairment.

Mothers and teachers independently completed the nonclinician version of the Children’s Global Assessment Scale (C-GAS; Shaffer, Gould, Brasic, Ambrosini, Fisher, & Bird, 1993), an impairment measure for children 4–18 years of age, used in multiple studies of preschool disruptive behavior with children as young as 2 (Lavigne, Arend, Rosenbaum, Binns, Christoffel, & Gibbons, 1998; Wakschlag & Keenan, 2001). Scores on the C-GAS range from 1–100; Scores of 60 or below indicate impairment. Parent and teacher C-GAS scores were moderately correlated (r = .34).

Observed child behavior.

Observational ratings of the child’s behavior were derived from interactions with an examiner during the DB-DOS (Wakschlag et al., in press-c). In addition to the Parent context described previously, child behavior was assessed during the Examiner engaged context of the DB-DOS, which consists of tasks parallel to those in the Parent context (e.g., compliance “do,” frustration, free play). Child problem behavior during the DB-DOS was coded along domains of problems in behavior regulation and anger modulation (Wakschlag et al., in press-c). As with the P-COS, coding was done globally and completed separately for each context. Ordinal ratings were made along a clinical continuum: 0 (no evidence of behavior), 1 (low levels of behavior), 2 (moderate levels of behavior), and 3 (high levels of behavior) (see Wakschlag et al., in press-c) for additional details regarding DB-DOS coding). Domain scores were generated separately by context by summing items. The DB-DOS demonstrated acceptable levels of inter-rater reliability (mean weighted kappa = .66 for Behavioral Regulation and .62 for Anger Modulation) as well as good internal consistency at the domain level (mean Cronbach’s Alpha = .85 for Behavioral Regulation and .92 for Anger Modulation) in the present sample and distinguished normative misbehavior from disruptive behavior problems (Fs range from 9.40 to 29.06, all ps < .01) (Wakschlag et al., in press-a).

Procedures

Prior to the laboratory visit, mothers were mailed a packet of questionnaires regarding the child’s functioning and their parenting practices, which they completed and brought to the laboratory. On the day of the visit, mothers completed the K-DBDS. Mother and child then participated in the DB-DOS. The mother completed additional questionnaires on her own behavior and family context while the child was administered the Examiner-engaged context of the DB-DOS. Questionnaires regarding the child’s behavior at school were mailed to teachers along with a self-addressed stamped envelope in which teachers could return school packets; teacher data were successfully obtained on 88% of the children. Ninety-one percent (n = 306) of the analytic sample returned approximately one year later (M = 392 days, SD = 53, range = 294–677 days) and repeated all assessment procedures; teacher data were successfully obtained on 87% of the children who were seen at follow-up. For participants in the test–re-test sample, mother and child returned to complete a second DB-DOS approximately four weeks after the baseline assessment (M = 28.83 days, SD = 2.05, range = 25–35 days).

Child clinical status was determined using parent and teacher report of child disruptive behaviors symptoms and impairment (see Wakschlag et al., 2007, for details). For children for whom there were no teacher data, clinical status was based on maternal report only, as the K-DBDS also probes for symptoms within the school setting. The resulting classification of clinical status at baseline was 24% Disruptive (n = 79), 26% Clinically at risk (i.e., with subclinical levels of symptoms; n = 87), and 50% Non-disruptive (n = 169). Groups did not vary on child age.

Coding procedures.

Coding was completed by teams of nonclinician, bachelors- and masters-level coders trained by two criterion coders: the first or second author. Training involved review of the coding system with emphasis on the continuum of behaviors comprising each code. Practice tapes were reviewed and discussed to illustrate behaviors along the continuum for each code. To preserve the quantitative and qualitative dimensions of the coding systems, coders were instructed to form an overall impression of the parent for each code based on the parent’s behavior, which could be supported by specific instances of observed behavior to validate the rating given for a particular code (e.g., mother appeared highly engaged, supported by frequent displays of positive engagement with the child and conveying instances of pleasure in the experience when mother was coded as “high” for positive engagement). Coders made two passes prior to making final coding determinations. Initial reliability was established via at least 80% exact agreement at the item level with the criterion coder. Similar procedures were followed for training an independent team to code the child’s behavior during the DB-DOS. For child behavior, coding was done separately for each context.

Coding was completed via videotape review. Contexts were randomly assigned to coders, who were blind to referral status. Approximately 20% of the contexts were randomly selected for double coding to monitor inter-rater reliability. Ongoing reliability was maintained via weekly coding meetings, and disagreements were resolved consensually.

Data Analysis

Traditional psychometric reliability properties for inter-rater agreement, internal consistency, and test–re-test stability were assessed. For item level inter-rater reliability in the Problematic Discipline domain, although coding was done using a four-point scale, because clinically we were interested in the presence of any problematic behaviors (i.e., “red flags”), items were re-coded into dichotomous categories of presence or absence. Thus, item-level inter-rater reliability for codes within the Responsive Involvement and Constructive Discipline domains was estimated using weighted kappas to take into account differences in degree of agreement, and simple kappas were used to examine inter-rater reliability of codes in the Problematic Discipline domain. Items were considered acceptable if they had kappas of .40 (Cicchetti, Bronen, Spencer, Haut, Berg, & Oliver, 2006; Fleiss, Levin, & Paik, 2003). Items with kappas greater than .60 were considered good or substantial (Cicchetti et al., 2006). No items had kappa below .40. Empirical support for the theoretically derived domains assessed by the P-COS was examined using tests for the internal consistency of domain scores (i.e., Cronbach’s alpha). Test–re-test reliability and inter-rater agreement at the domain score level were assessed using intraclass correlation coefficients (ICCs; Bartko, 1976; Shrout & Fleiss, 1979). We also examined mean level differences in domain scores at test and re-test.

Validity analyses were conducted to establish construct, convergent, and divergent validity. To assess construct validity, we assessed the correlation of P-COS scores with maternal self-report of parenting behavior. To reduce problems related to shared method variance, convergent validity was assessed by examining the extent to which P-COS domain scores were related to child functioning, including teacher as well as parent report and observation of the child’s behavior. We tested relations between observed parenting and child disruptive behavior, including testing for group differences in observed parenting as a function of child clinical status as well as the incremental utility of observed parenting to predicting concurrent child functioning over and above maternal self-report of parenting. Finally, we examined the clinical informativeness of the P-COS via associations between global classifications of parenting style and child clinical status as well as the extent to which parenting style was associated with change in child clinical status.

RESULTS

Item Level Inter-rater Reliability

Previous research suggests that observational assessment of clinically salient items with very low base rates may render estimation of reliability problematic because estimates of agreement can be unduly swayed by differences between a single pair of raters (Cicchetti, Sparrow, Volkmar, Cohen, & Rourke, 1991; Shrout, Spitzer, & Fleiss, 1987). Thus, prior to estimating inter-rater reliability (Lord, Rutter, Goode, Heemsbergen, Jordan, Mawhood, & Schopler, 1989), we identified those items with low-base rates (i.e., those where a rating other than “0” was given less than six times) whose reliability could not be calculated. One item assessing mothers’ use of verbally aggressive discipline toward the child demonstrated fewer than six positives; this item was excluded from reliability analyses.

In general, inter-rater reliability was good for the P-COS items (overall mean kappa = .65; mean kappas for domain scores ranged from .61–.68). Item-level kappas ranged from .49–.82, with 70% in the good range (Cicchetti, 1994; see Table 1). Inter-rater agreement was good in all three P-COS domains, although somewhat lower in the Constructive Discipline domain. Consistent with this, the only item, Firmness, with a weighted kappa below .50 as well as a percent agreement below 70%, was in the Constructive Discipline domain. Examination of this item suggested that coders had little difficulty agreeing on the distinction between concerning (i.e., none and low) and non-concerning (i.e., moderate and high) parenting behaviors; rather, most of the disagreements on this item were between a moderate and a high.

TABLE 1.

Item-Level Inter-Rater Agreement

	Kappa		Percent Exact Agreement
Domain	Mean	Range	Mean	Range
Responsive Involvement	.66	.55-.82	74%	61%–83%
Constructive Discipline	.61	.49-.72	76%	68%–88%
Problematic Discipline	.68	.53-.82	84%	73%–89%

Reliability of Domain Scores

Internal consistency of domain scores.

Table 2 presents domain score means and standard deviations, and summarizes reliability of the three domain scores. Labeling, from the Responsive Involvement domain, was excluded from domain score analyses due to poor loading on internal consistency analyses. In addition, two items in the Problematic Discipline domain were highly correlated (i.e., Highest Level of Angry/Irritable Affect and Predominance of Angry/Irritable Affect; r = .76, p < .01); thus, these items were combined into one item assessing maternal angry/irritable affectivity. Overall, the three domains exhibited adequate internal consistency (range = .66–.76; see Table 2). Inter-rater reliability for the three domain scores was excellent, ranging from .82 for Constructive Discipline to .86 for Problematic Discipline.

TABLE 2.

Domain Score Reliability Estimates

Domain	M (SD)	Cronbach’s Alpha	Item Loadings	Inter-Rater ICC	Test – Re-Test ICC	Test M (SD)	Re-Test M (SD)
Responsive Involvement	10.77 (2.99)	.76	.38-.65	.85	.63	11.90 (2.72)	11.30 (2.82)
Constructive Discipline	5.96 (1.53)	.70	.51-.55	.82	.48	5.77 (1.41)	6.07 (1.36)
Problematic Discipline	1.27 (1.41)	.66	.25-.52	.86	.72	1.07 (1.66)	1.03 (1.50)

Note: Differences between test and re-test Domain scores were nonsignificant (ts ranged from .08 to .84; all ps > .05).

Test–re-test reliability of domain scores.

Intraclass correlations (ICCs) were computed to examine test–re-test consistency in maternal behavior within the three domains. ICCs among test–re-test domain scores indicate somewhat higher stability in mothers’ Responsive Involvement and Problematic Discipline (ICCs = .63 and .72, respectively) relative to their Constructive Discipline (ICC = .48). Domain scores for Responsive Involvement and Problematic Discipline decreased slightly in the second testing, whereas the domain score for Constructive Discipline increased slightly; none of the changes from test to re-test was significant (ts range from −.84 to .84, ps > .05).

Domain Specificity

Pearson correlations were computed to examine the extent to which the three domains of the P-COS represented related, but distinct, dimensions of parenting behavior. Associations among the P-COS domains were low to moderate and in the expected directions. Responsive Involvement and Constructive Discipline were positively associated with one another (r = .37) and were negatively associated with Problematic Discipline (rs = −.24 and −.44, respectively).

Validity

Construct validity of the P-COS domains was examined using Pearson correlations to assess associations with maternal self-reports of their responsive and problematic discipline (see Table 3). Consistent with expectations, P-COS domains generally demonstrated modest associations in the expected directions with maternal self-report of their parenting. Responsive Involvement demonstrated the most consistent associations to self-report measures and was negatively associated with maternal self-report of Problematic Discipline (rs range from −.09 to −.31, ps < .05) and positively associated with warm, responsive parenting (rs = .16 and .18, ps < .05). In contrast, Constructive Discipline was only associated with maternal self-report of inconsistency (r = −.10, p < .05). Problematic Discipline was positively associated with mothers’ self-reports of inconsistency (r = .10, p < .05) and harsh parenting (r = .19, p < .05) and negatively associated with supportive parenting (r = −.12 with Expressive Encouragement).

TABLE 3.

Validity Indices: Associations Between P-COS Domains and Self-Reported Parenting and Child Functioning

	Observed Parenting
	Responsive Involvement	Constructive Discipline	Problematic Discipline
Self-reported parenting
Nurturance	.18**	−.04	−.07
Follow-Through	−.09*	.01	−.04
Inconsistent Parenting	−.10*	−.10*	.10**
Expressive Encouragement	.16**	−.03	−.12*
Harsh Parenting	−.31**	−.06	.19**
Maternal report of child behavior
Disruptive Behavior	−.20**	−.23**	.14**
Impairment	.20**	.17**	−.12*
Teacher report of child behavior
Disruptive Behavior	−.12*	.06	.05
Impairment	.13*	.04	−.11*
Observed child behavior
Problems in Behavioral Regulation
Examiner-Engaged context	.01	−.10*	.14**
Parent context	−.03	−.44**	.40**
Problems in Anger Modulation
Examiner-engaged context	.09*	.03	.03
Parent context	−.06	−.35**	.40**

^*p < .05.

^**p < .01.

Convergent validity was assessed via a series of analyses to examine whether deficits in observed parenting were associated with problematic child behavior. Table 3 summarizes concurrent associations between P-COS domains and child functioning. In general, P-COS domains were associated with child functioning assessed via multiple informants. Although all three P-COS domains were significantly associated with mother reports of child functioning in the expected directions, patterns of associations with teacher-reported and observed child behavior differed slightly by P-COS domain. For example, mothers’ Responsive Involvement but not their Constructive Discipline was associated with teacher ratings of child disruptive behavior and impairment. In contrast, associations were stronger for Constructive Discipline and Problematic Discipline and observed child problems with Behavior Regulation and Anger Modulation.

Discriminant validity was examined via a series of one-way ANOVAs to assess the extent to which the P-COS domains differed as a function of child clinical status (i.e., nondisruptive, at-risk, disruptive). Mothers of disruptive children displayed less Responsive Involvement (M = 10.16, SD =3.10) than mothers of nondisruptive children (M = 11.25, SD = 2.96) and mothers of at-risk children (M = 10.38, SD = 2.85), F (2,332) = 4.66, p < .05; mothers of at-risk children and nondisruptive children did not differ from one another. Mothers did not differ in their use of Constructive Discipline across the three clinical groups. However, there were significant differences in mothers’ Problematic Discipline across clinical groups, F (2,332) = 5.39, p < .01: Mothers of at-risk children displayed significantly higher rates of Problematic Discipline (M = 1.62, SD = 1.56) compared to mothers of nondisruptive children (M = 1.04, SD =1.29). Mothers of disruptive children did not differ from either of the other two groups (M = 1.37, SD =1.37).

Incremental Utility of P-COS Domains

We next performed a series of analyses to examine the extent to which observed parenting explained unique variance beyond that explained by self-reported parenting in prediction of reported and observed child disruptive behavior problems and reported child impairment. For observed disruptive behavior problems, a problem behavior count score was generated by first dichotomizing items from the DB-DOS into absence (score of 0 or 1) or presence (score of 2 or 3) and then summing the number of behaviors classified as present. Observed problem behavior counts were calculated separately for the two DB-DOS contexts (i.e., Examiner-Engaged and Parent).

We estimated zero-inflated Poisson regression models for reported and observed disruptive behavior problems using Mplus (Muthén & Muthén, 2006) because this model is appropriate when the dependent variable is a count variable with a high proportion of zero values. Mplus estimates two components in this type of model: (1) a zero-inflated component that estimates the odds of having at least one disruptive behavior symptom and (2) a Poisson regression coefficient that reflects the association between parenting variables and the frequency of disruptive behavior symptoms. For models predicting parent and teacher report of child impairment, we conducted general linear regression. For all models, we performed analyses separately for each of the P-COS domains and included only self-report scales assessing theoretically parallel dimensions of parenting.

Table 4 summarizes results of the zero-inflated Poisson regression models. In general, all three P-COS domains provided incremental prediction of maternal but not teacher’s reports of child disruptive behavior symptoms beyond that provided by more traditional self-report assessment of maternal parenting practices. Specifically, Responsive involvement was associated with a lower probability of having at least one maternal reported disruptive behavior symptom as well as decreased frequency of disruptive behaviors. In contrast, Constructive Discipline and Problematic Discipline were associated only with the frequency of maternal reported disruptive behavior symptoms in the expected directions. Results were similar for observed child disruptive behavior: P-COS domains were uniquely associated with children’s observed disruptive behavior during the Parent context of the DB-DOS but not during the Examiner-Engaged context. All three P-COS domains were uniquely associated in theoretically expectable ways with the frequency of disruptive behaviors the child displayed when interacting with the parent, and both Constructive Discipline and Problematic Discipline were associated with the probability of the child displaying at least one problem behavior during the Parent context.

TABLE 4.

Zero-Order Poisson Regression Analyses with Observed and Self-Reported Parenting Predictors of Child Disruptive Behavior Symptoms

	Reported Symptoms				Observed Behavior
	Maternal		Teachera		With Parent		With Examiner
Parenting Variable	OR	PR	OR	PR	OR	PR	OR	PR
Model 1a:
Self-reported Nurturance	.89	.88*	.99	.91	.81	1.00	.86	1.09
Self-reported Expressive Encouragement	1.02	1.24**	.85	.98	1.55**	1.00	.98	.96
Observed Responsive Involvement	.77*	.86*	.96	.86	1.22	.90*	1.12	1.06
Model 1b:
Self-reported Follow-Through	1.21	1.09	1.19	.98	1.06	1.02	1.21	.96
Self-reported Inconsistent Parenting	2.03**	1.13*	1.25	1.02	1.32	1.01	1.16	1.08
Observed Constructive Discipline	1.01	.82**	1.15	1.08	.64**	.77**	1.14	.87
Model 1c:
Self-reported Harsh Parenting	1.37*	1.09	1.03	1.04	1.02	1.00	1.05	.90
Observed Problematic Discipline	1.07	1.12*	1.13	1.04	1.86**	1.28**	1.19	1.12

Note. OR = odds ratio for probability of having at least one disruptive behavior symptom.

PR = predicted rate of increase in frequency of disruptive behavior symptoms.

^aTeacher data obtained for 88% of the sample.

^*p < .05.

^**p < .01.

The two dimensions of competent parenting, Responsive Involvement and Constructive Discipline, explained unique variance in mother-rated child impairment such that more competent behaviors were associated with higher child functioning (see Table 5). Again, models predicting teacher ratings of child impairment were not significant.

TABLE 5.

Linear Regression Analyses with Observed and Self-Reported Parenting Predictors of Child Impairment

	Maternal Report			Teacher Reporta
Parenting Variable	F	R 2	β	F	R 2	β
Model 2a:	8.82**	.08		1.88	.02
Self-reported Nurturance			.21**			−.01
Self-reported Expressive Encouragement			−.15*			.08
Observed Responsive Involvement			.18**			.11
Model 2b:	15.80**	.13		2.61	.03
Self-reported Follow-Through			−.32**			−.15*
Self-reported Inconsistent Parenting			−.13*			−.11
Observed Constructive Discipline			.14**			.03
Model 2c:	7.64**	.04		3.10*	.02
Self-reported Harsh Parenting			−.17**			−.10
Observed Problematic Discipline			−.09			−.09

^aTeacher data obtained for 88% of the sample.

^*p < .05.

^**p < .01.

Clinical Informativeness of P-COS Domains

Finally, we examined how global patterns of observed parenting style (i.e., problematic, mixed, and competent) were associated with child clinical status. First, we dichotomized mothers’ responsive and problematic behaviors. For the problematic discipline domain, mothers were classified as problematic if they displayed any problematic behaviors during the DB-DOS. A competence domain was then created by summing scores from Responsive Involvement and Constructive Discipline (alpha for combined domain = .78). A median split (i.e., competent parenting score = 17) was used to define low (at or below the median) and high (above the median) competence. This led to classifying 26% (n = 87) of the mothers as “problematic” (Low competence + problems); 42% (n = 140) of the mothers as “mixed” (high competence + problems or low competence + no problems); and 32% (n = 108) of mothers as “competent” (high competence + no problems).

Chi-square analyses indicated a significant association between maternal parenting style and child clinical status, χ² (4) = 11.15, p < .05; see Table 6. Consistent with expectations, competent mothers were more likely to have a nondisruptive child relative to problematic mothers (62% versus 39%). However, only one-quarter of the problematic mothers had a disruptive child, a percentage somewhat lower than what might be expected. Furthermore, nearly 20% of the competent mothers had a disruptive child.

TABLE 6.

Child Baseline Clinical Status by Maternal Parenting Style

	Child Baseline Clinical Status
Parenting Style	Nondisruptive	At-risk	Disruptive
Competent	62% (67)	19% (21)	19% (20)
Mixed	48% (68)	26% (36)	25% (36)
Problematic	39% (34)	35% (30)	26% (23)

We then examined how observed parenting style was associated with change in child clinical status over time for the 91% (n = 306) of the sample who participated in a longitudinal follow-up assessment one year later. Using a series of chi-square analyses, we examined whether parenting status was associated with change in child clinical status over time. Children were classified as demonstrating improved clinical status (i.e., moved from disruptive to at-risk, or moved from at-risk to nondisruptive), no change, or worsening clinical status (i.e., moved from nondisruptive to at-risk or to disruptive) from baseline to follow-up; analyses were conducted separately by baseline clinical status. For children classified as disruptive at baseline, children of competent mothers (80%) were more likely to demonstrate improvement (i.e., decreasing behavior problems) at follow-up compared to children of mothers in the problematic (35%) or mixed groups (48%), χ² (2) = 7.80, p < .05. Change in clinical status was not associated with parenting style for children identified as at-risk and nondisruptive at baseline.

DISCUSSION

Many observational measures exist to competently assess parenting behaviors of relevance to child disruptive behavior (e.g., DPICS) (Robinson & Eyberg, 1981). However, their clinical utility has been limited, owing to their focus on measuring the frequency of discrete behaviors, which (1) does not fully capture qualitative aspects important to understanding the gestalt of behaviors that encompass parenting in a manner that is clinically informative or (2) requires intensive coding that makes the system infeasible for use in clinical settings. The present study reports on the reliability and validity of a new observational coding system, the Parenting Clinical Observation Schedule (P-COS). The goal of the P-COS is to provide a clinically feasible tool for assessing parenting behavior within the context of parent-child interactions in a manner germane to evaluation and intervention with families of children with disruptive behaviors. Results demonstrate that the P-COS offers a brief standardized, laboratory-based assessment of parenting with good inter-rater reliability, internal consistency, and test–re-test reliability.

Our multidimensional assessment of parenting competencies and problems resulted in three internally consistent, clinically derived, and distinct dimensions of parenting that were differentially related to self-reported parenting and child outcome. Consistent with previous research demonstrating modest associations between observed and self-reported behaviors (e.g., Kochanska, Kuczynski, & Radke-Yarrow, 1989), P-COS domains were moderately associated with a number of self-reported parenting dimensions in theoretically expectable ways. It is possible that this pattern of modest associations may be a measurement artifact, resulting from the fact that our self-report parenting measures did not map cleanly onto our observed parenting dimensions. Furthermore, other variables, such as maternal depression (Kochanska, 1990) and personality characteristics (Metsäpelto & Pulkkinen, 2005) previously shown to moderate the association between observed and self-reported parenting were not examined in the present study.

However, if the differential associations reported here are replicated in future studies with the P-COS, these findings suggest that parents may be better able to report on some aspects of their behavior than others. For example, of the P-COS domains, Constructive Discipline demonstrated the fewest associations with maternal self-report, perhaps because this parenting dimension assesses more nuanced behaviors that reflect interactional sequences contingent on judgments about the child’s moment-to-moment behavior (e.g., flexible shifting of parenting behavior), which is difficult for parents to report on about themselves. It may be that the aspects of parenting that involve an awareness of the dynamic nature of parenting are more difficult to report on because parents see their parenting behavior as a stable quality rather than a shifting set of behaviors contingent on the child’s behavior.

Relatedly, these findings on the P-COS suggest that observational measures of parenting within a clinical context can contribute to understanding the complex interrelation between parenting and young children’s disruptive behavior problems. Specifically, at the bivariate level, P-COS dimensions were differentially related to the presence of preschoolers’ disruptive behavior across informants and contexts. As would be expected, observed parenting was most strongly and consistently related to child behavior problems within the parent-child context (both by maternal report and during the DB-DOS observation), but observed parenting was also associated with behavior problems based on teacher report as well as those observed in interaction with the examiner on the DB-DOS. The stronger associations between parenting and child behavior within the parent-child context point to the transactional nature of problematic parenting and child behavior problems (e.g., children with disruptive behavior are more difficult to parent and more likely to elicit less responsive and more problematic discipline, which in turn may lead to increased disruptive behavior (Stormshak et al., 2000).

The findings also suggest that neither parenting behavior nor child disruptive behavior should be viewed as separate or unidimensional constructs. Whereas Responsive Involvement was associated with observed child problems in Anger Modulation, the two observed dimensions of discipline practices were associated with observed child problems in Behavioral Regulation. Because Responsive Involvement reflects parental responsiveness, positive affectivity, and anticipatory scaffolding, it is likely to reduce child negative affectivity by providing sensitive support in the face of frustration and via modeling. In contrast, the two disciplinary dimensions capturing parenting behaviors in response to child misbehavior are more likely to influence child noncompliant and aggressive behavior.

Findings from the present study suggest that the “added value” associated with observational assessment of parenting in predicting child behavior may vary depending on the context. Consistent with previous research, P-COS dimensions provided overlapping but unique information in the prediction of child behavior beyond that offered by maternal report of parenting (Sessa, Avenevoli, Steinberg, & Morris, 2001; Zaslow et al., 2006). However, the present study extends this work by demonstrating differential utility of observed parenting as a function of the context in which the child disruptive behavior is assessed. Specifically, observed parenting added incremental utility only when predicting the child’s behavior within the parent-child context (e.g., whether by maternal report or “objective” observation). These findings suggest that observational assessment of parenting, while more time-consuming than self-report measures, may be particularly informative because it captures the transactional aspects of parent-child interactions.

In contrast, the present study suggests the possibility that observations of parenting add little information beyond that captured by self-reported parenting when identifying concurrent child disruptive behavior outside the parent-child context (e.g., at school). Consistent with previous research (Achenbach, McConaughy, & Howell, 1987; Renk & Phares, 2004), parent and teacher reports of child functioning were only modestly associated in the present study. This in part reflects the true variation in children’s behavior across contexts and relationships (Achenbach, McConaughy, & Howell, 1987); however, it may also reflect individual attributes of reporters that influence perceptions of the child (Kroes, Veerman, & De Bruyn, 2003) and/or method variance when both informant and source vary (Renk & Phares, 2004). To more fully explicate the incremental utility of observed parenting for predicting variations in child disruptive behavior, future studies might more comprehensively examine the relation of observed parenting with self-report measures that more closely parallel observed dimensions and that use a multi-informant, multisource assessment of child disruptive behavior, in order to disentangle reporter biases from contextual effects. Such work could elucidate when the added “cost” of observation assessment is warranted and when more “economical” self-report measures suffice.

Although individual dimensions of parenting behavior are important to assess, fundamentally the “gestalt” of parenting style (i.e., how these dimensions fit together) is likely to be central to children’s overall experience of being parented. Two findings from our analysis of the relation of the pattern of parenting to child disruptive behavior are of note. First, despite assertions that preschool disruptive behavior is primarily a “parent-child relationship problem” (Emde, 2003), a significant minority of competent mothers in this study had children with clinically significant disruptive behavior, and, at the same time, nearly three-quarters of problematic mothers had children without clinically significant disruptive behavior. Because observed parenting and child disruptive behavior were concurrently assessed, these findings cannot be interpreted in terms of causal relations. For example, observed problematic parenting may reflect an antecedent process that contributes to the development of disruptive behavior in some children, but concurrently it may reflect a response to difficult-to-manage child behavior that maintains these maladaptive processes for others. Longitudinal studies of parenting dimensions and infant temperamental vulnerability to behavioral problems will help to clarify causal relations and indicate to clinicians what aspects of parenting are most important for intervention based on concurrent transactions.

Second, these parenting patterns predicted reduction in child disruptive behavior over time, but only for children exhibiting clinically significant disruptive behavior at baseline. Identification of clinically concerning parenting (i.e., parenting profiles) can inform clinical intervention with families presenting with a young child with disruptive behaviors because these intervention strategies would likely differ based on specific patterns of observed parenting.

To highlight how specific patterns of observed competent and problematic parenting might influence intervention, consider the following clinical examples: One parent ranks high on Responsive Involvement, is warm and positive with her/his young child when the child is compliant and behaving well, but is low on Constructive Discipline, becoming passive and helpless in the face of his/her child’s aggressive and oppositional behaviors. At these times, the parent laughs, withdraws, cajoles, or pleads with him/her to stop. In contrast is the parent who shares little warmth and is unresponsive or withdrawn when the child is happily engaged but who scores high on Problematic Discipline and is quick to intervene and set limits – in fact, too quick – as the parent is intolerant of any assertions of independence. The focus of the interventions for these parents would likely differ, as the first parent may benefit from help in being assertive and in clarifying his/her expectations for the child’s behavior, and the second may need help attending to positive behavior and psychoeducation about the developmental needs of preschool children for asserting independence. The P-COS demonstrates promise as a clinically informative tool, although its sensitivity to treatment effects remains an unknown but important question. We are currently collecting data to assess whether the P-COS is sensitive to treatment effects for parents participating in a brief, parenting intervention.

A strength of the present study is its focus on low-income, predominantly minority families of referred and nonreferred children. First, we underscore that the association between responsive parenting and child functioning was obtained among a sample of low-income, predominantly minority women rearing their children in the context of urban poverty. Much of the existing research on low-income mothers has focused on the ways in which living in poverty encourages elevated levels of harsh, inconsistent parenting (McLoyd & Wilson, 1994). Although understanding how problematic parenting is associated with the development and maintenance of child behavior problems for children living in low-income contexts provides inroads for prevention of disruptive behavior problems, it is equally important to understand the role of more positive aspects of parenting (i.e., responsiveness) in the context of sociodemographic risk.

Second, recent research with ethnically and socioeconomically diverse samples has suggested that harsh discipline is more characteristic of African American families and is associated with less detrimental outcomes for this group (Deater-Deckard, Dodge, Bates, & Pettit, 1996; Ipsa et al., 2004). In such samples, ethnicity and low-income status are often confounded, as families of color tend to cluster within the poverty group. In the present, more homogeneous sample, in which all families were living in low-income environments, and the majority of families were African American, substantial variability in observed parenting was evident. Within this context, as in those with predominantly majority and higher income samples, higher levels of harsh, problematic discipline, and lower levels of responsive, constructive parenting were associated with increased rates of preschool disruptive behavior problems. Results highlight the need to move beyond global group differences that may oversimplify findings (e.g., physical discipline is a protective factor in African American families) to a focus on individual differences within groups, including examination of how the combination of various parenting practices is associated with child outcomes in studies that systematically distinguish effects of poverty from ethnic variation.

Finally, African American families, particularly those from low-income environments have typically received less attention in the observational literature. Although recent research has bolstered the use of structured observation in the assessment of parenting in low-income, African American samples (e.g., Zaslow et al., 2006), its clinical utility has largely been unexplored. The present study provides good evidence that the P-COS is a valid tool that discriminates between clinically concerning and competent parenting in this sample of families, many of whom had children with behavioral concerns. This is important, given that this population is at “increased risk” for disruptive behavior problems but is often not the target of clinical studies identifying “when to worry.”

Limitations

Observational methodology has much to contribute to the identification and understanding of dimensions of parenting related to child functioning, but it is not without limitations (Gardner, 2000). Observations necessarily are a “snapshot” of parenting and do not assess history of the behavior. As such, the P-COS is intended as a complement to parent interviews and questionnaires. Furthermore, as is evident from the low base rates of verbally and physically aggressive discipline behaviors in the P-COS, the brief nature of observation makes it less optimal for assessing clinically concerning, but infrequently occurring behaviors.

In addition, for the present study, P-COS coding was conducted via videotape review by nonclinician coders, which included review of multiple “passes” of the videotape and took approximately one hour. Although this level of review was important for initial psychometric validation, it is not feasible for clinicians. Demonstration of the reliability and validity of the P-COS is the first step toward its feasible use as a clinical instrument. An essential next step for the P-COS is creation of live clinical coding procedures to enable clinicians to code parenting behaviors observed immediately after the administration of the DB-DOS, including establishing the psychometric properties of this clinical administration.

Although this study indicates the utility of observational methods for understanding parenting and predicting child outcomes, the extent to which findings generalize beyond low-income, primarily African American mothers remains unclear. Future research exploring the nature of associations among the dimensions assessed by the P-COS and child functioning using a more representative sample are necessary. In addition, future examinations of the P-COS might move beyond the focus on associations between maternal and child behavior, to include parenting within the family, including how fathers’ competent and problematic discipline are related to child concurrent and longitudinal functioning.

Conclusions

The development of a reliable and valid clinical, observational tool assessing problematic and competent parenting offers much promise for advancing our understanding of the role that parenting plays in the development and maintenance of early emerging disruptive behavior problems. This is particularly important, given that disruptive behavior problems are the most commonly occurring reason for referral to outpatient mental health clinics (Loeber, Green, Lahey, Frick, & McBurnett, 2000). The P-COS holds promise for elucidating parenting behavior in a nuanced manner that is directly designed to guide and inform family-based interventions to prevent and treat emergent disruptive behaviors.

APPENDIX

P-COS Items by Domain

Responsive Involvement Domain

1. Scaffolding

2. Responsivity to positive behaviors

3. Warmth

4. Positive engagement

5. Labeling

6. Intensity of positive affect

7. Predominance of positive affect

Constructive Discipline Domain

1. Firmness

2. Positive behavior strategies

3. Flexibility

Problematic Discipline Domain

1. Hostile behavior

2. Verbally aggressive discipline

3. Physical discipline

4. Power struggles

5. Emotional misattunement

6. Intensity of angry/irritable affect

7. Predominance of angry/irritable affect