Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Oct 1.
Published in final edited form as: Child Youth Care Forum. 2016 Apr 5;45(5):745–758. doi: 10.1007/s10566-016-9356-z

Using Observational Assessment to Help Identify Factors Associated with Parent Participation Engagement in Community-Based Child Mental Health Services

Nicole A Stadnick 1,2, Rachel Haine-Schlagel 2,3, Jonathan I Martinez 4
PMCID: PMC5003526  NIHMSID: NIHMS775006  PMID: 27587943

Abstract

Background

Parent engagement in child mental health (MH) services has received growing attention due to its significance in intervention outcomes and evidence-based care. In particular, parent participation engagement (PPE) reflects active and responsive contributions in and between sessions. Yet, limited research has examined factors associated with PPE, particularly within community-based MH services where PPE is low and highly diverse families are often served.

Objective

This study examined child, parent, and therapist factors associated with PPE in a sample of racially/ethnically diverse parent–child dyads receiving publicly-funded, community-based MH services.

Methods

This prospective study included 18 parent–child dyads receiving community-based MH services from 17 therapists in five outpatient clinics for child disruptive behaviors. PPE was measured using in-session observational assessment of therapy recordings. Child factors that were examined included age, first time child MH service use, and intensity of child behavior problems. Parent factors included ethnicity, education, depression symptoms, and parent motivation to participate in therapy. Therapist factors included therapist training in parent-mediation interventions, attitudes towards organizational functioning, and attitudes towards parent participation strategies.

Results

Results from linear regression analyses indicated that first time child MH service use, intensity of child behavior problems, parent ethnicity and motivation to participate in therapy, as well as therapists’ training and attitudes about their practice were each significantly associated with PPE.

Conclusions

Results highlight specific child, parent, and therapist characteristics that may impact observed PPE in child MH therapy. These findings underscore the importance of considering the influence of family and provider factors on PPE in community-based child MH services.

Keywords: Parent, Participation, Engagement, Community-based, Child mental health treatment, Disruptive behavior problems

Introduction

Professional organizations, research funders and state mental health (MH) systems have exhibited increasing efforts to improve the quality of child MH care through implementation of evidence-based interventions in community settings (American Psychological Association Presidential Task Force on Evidence-Based Practice 2006; Hoagwood et al. 2014; McHugh and Barlow 2010; Nakamura et al. 2011; Starin et al. 2014; U.S. Department of Health and Human Services 2015). An important therapy process within evidence-based youth interventions that is garnering increased empirical attention is child and family treatment engagement (Becker et al. 2015; Haine-Schlagel and Walsh 2015; Gopalan et al. 2010; Ingoldsby 2010; Kim et al. 2012; Lindsey et al. 2014). In particular, parent1 engagement is important because it can facilitate treatment attendance, improve child outcomes, and generalize therapeutic changes to non-therapy settings (Dowell and Ogles 2010; Karver et al. 2006). Parents are also a key target for therapeutic intervention as seen in evidence-based child MH interventions (David-Ferdon and Kaslow 2008; Eyberg et al. 2008; Evans et al. 2014; Keel and Haedt 2008; Silverman et al. 2008).

Parent participation engagement (PPE) is defined as active, independent, and responsive contributions from the parent during therapy sessions, including sharing opinions, asking questions, participating in session activities, and following through on parent- or child-directed between-session activities (e.g., between-session parent practice of behavior management strategies with their child; parent support of their child's practice of therapy skills) (Haine-Schlagel and Walsh 2015; Karver et al. 2006). PPE is considered an “evidence-based process” inherent in effective child MH interventions and complementary to specific evidence-based intervention strategies (Huang et al. 2005). PPE includes both components of treatment engagement, which are attitudinal and behavioral engagement (Staudt 2007). Attitudinal engagement encompasses the client's perception that treatment benefits outweigh costs (Becker et al. 2015; Staudt 2007). Behavioral engagement includes seeking or initiating treatment, attending treatment sessions, and meaningfully participating in therapy activities within and outside of sessions (Nock and Ferriter 2005; Staudt 2007). There is agreement that mere attendance at treatment is not sufficient to represent participation engagement (Haine-Schlagel and Walsh 2015; Nock and Ferriter 2005; Staudt 2007).

PPE is an important construct to examine because it is a core component of evidence-based MH interventions for a range of child mental health problems, particularly for child disruptive behavior problems (Eyberg et al. 2008) and Attention Deficit/Hyperactivity Disorder (Evans et al. 2014). This is particularly salient in the community mental health service system for children given that Attention Deficit/Hyperactivity Disorder and disruptive behaviors disorders have the highest prevalence rates for psychiatric disorders in publicly-funded youth service sectors (Garland et al. 2001). Standard community mental health care for children with disruptive behavior problems is eclectic with therapists using a breadth of intervention strategies, some consistent with evidence-based practices, but delivering these strategies with moderate to low intensity (Garland et al. 2010; Haine-Schlagel et al. 2012). In particular, while parents are typically present in at least part of sessions, parent-directed skill-building strategies are infrequently used and with similarly low intensity. Due, in part, to the infrequent and low intensity use of parent-directed skill-building strategies, there is limited understanding regarding the role of PPE in child MH care, specifically for children receiving care for disruptive behavior problems (Lindsey et al. 2014; Haine-Schlagel and Walsh 2015). Even though the potential benefits of strong PPE are multifold, including possible improvements in child symptoms and decreased functional impairment (Haine-Schlagel and Walsh 2015), findings from community child MH settings indicate that PPE is limited (Baker-Ericzén et al. 2013; Haine-Schlagel et al. 2012; Garland et al. 2010). The limited PPE seen in community-based settings is in part due to several barriers that parents report encountering, including perceptions of stigma from therapists or the service system and general feelings of dissatisfaction with their child's MH services (Baker-Ericzén et al. 2013).

Limited attention has been paid to factors associated with PPE in both community and controlled research settings (Haine-Schlagel and Walsh 2015). Rather, the extant literature in child MH treatment has focused on treatment attendance as the outcome and factors at the child- or parent/family-level as predictors (Gopalan et al. 2010; Kim et al. 2012). MH treatment occurs within distinct but interactional contextual levels including policy/funding, organizational, individual therapist, and child/family (Garland et al. 2012). The complex context in which child MH services occurs underscores the importance of examining factors at different contextual levels to better inform potential targets of change in the therapeutic process and outcomes of MH treatment.

In their recent review of PPE, Haine-Schlagel and Walsh (2015) surveyed the existing literature on PPE in child and family MH treatment broadly. They identified child, parent, and therapist attributes associated with PPE and highlighted limitations in the extant PPE literature. From this review, the following child-level factors were identified as significant correlates of PPE: child language and ethnicity, previous service use history, and cooccuring medical conditions (Dumas and Albin 1986; Fawley-King et al. 2012). Parent/family-level factors were the most frequently examined correlates of PPE. Several parent/family factors were found to be associated with PPE, including: parent age, socioeconomic status, parent treatment attendance, parent attitudes towards their child's mental health treatment, parent psychopathology, referral source to mental health treatment, and parenting behaviors (Chamberlain et al. 1984; Dumas and Albin 1986; Hansen and Warner 1994; Fawley-King et al. 2012; Nock and Kazdin 2005; Jensen and Grimes 2010; Podell and Kendall 2011; Stoolmiller et al. 1993). Not available for the Haine-Schlagel and Walsh (2015) PPE review, two recent studies uniquely identified that negative parent beliefs about their child's symptoms (Pereira et al. 2015) and specific beliefs about readiness to engage in treatment (Andrade et al. 2015) were associated with PPE. Finally, Haine-Schlagel and Walsh (2015) noted that limited attention has been paid to therapist-level characteristics associated with PPE. One study identified a significant associations between therapist behaviors in sessions with parents, specifically directive and confrontational communication, and poorer PPE (Patterson and Forgatch 1985).

In addition to highlighting the need for examining broader range of factors associated with PPE, specifically at the child and therapist-levels, the Haine-Schlagel and Walsh (2015) review also highlighted both the limited variability in measurement format of PPE and the limited contexts in which PPE has been examined. Specifically, approximately half of the available PPE measures were therapist or parent self-report and approximately one-third were ratings of homework completion. A minority (15 %) used observational coding of video-recorded treatment sessions to examine specific participation behaviors. The benefits and unique value added from use of observational assessment methods in intervention research have been documented (Synder et al. 2006), including the potential advantages of providing a more unbiased and realistic view of specific behaviors of interest within the treatment session. The Haine-Schlagel and Walsh review also highlighted that PPE as a construct and intervention strategies to improve PPE have been largely examined within structured treatment protocols as opposed to routine MH treatment.

Overall, the Haine-Schlagel and Walsh (2015) review draws attention to several gaps in the extant literature regarding the correlates associated with and role of PPE within child community MH services, which the current study aims to address. First, a broader array of factors across different levels of treatment influence, including parent and therapist attitudes associated with PPE, merit examination. Second, outcome measurement of PPE has largely been limited to treatment attendance or homework adherence, neither which directly capture core PPE behaviors. An observational measure of PPE could provide this type of direct, specific measurement and appears needed. Third, there is a need to examine PPE within MH care provided to families served in standard, community settings, who often have greater sociodemographic diversity and available resources compared to families who are included in intervention efficacy trials.

Given the emphasis on implementing evidence-based practices in community settings (American Psychological Association Presidential Task Force on Evidence-Based Practice, 2006; Hoagwood et al. 2014; McHugh and Barlow 2010; Nakamura et al. 2011; Starin et al. 2014; U.S. Department of Health and Human Services National Institutes of Health, National Institute of Mental Health, 2015) and the multiple gaps in the PPE literature, identifying the conditions under which the evidence-based process of PPE can be enhanced is warranted. To address these gaps, this study offers a preliminary examination of PPE assessed through observational assessment of community child MH services. Specifically, the purpose of this study was to utilize available pilot data to extend the review of Haine-Schlagel and Walsh (2015) by identifying potential child, parent, and therapist characteristics that are associated with PPE in a small sample of ethnically diverse child-parent dyads receiving community-based child MH services for child disruptive behavior problems, the most common presenting problem in this service context. Based on the literature, we hypothesized that: (1) children with more parent-reported behavior problems and no previous MH service use would have greater observed PPE and (2) parents from racial/ethnic minority backgrounds, parents with greater self-reported psychopathology, and parents with lower motivation to engage in their child's treatment would have lower observed PPE. Given the paucity of research that has examined therapist-level correlates of PPE, the following is our third, exploratory hypothesis: (3) therapist training and attitudes towards their practice and the value of utilizing parent engagement strategies would be associated with observed PPE. Continuing to expand understanding of child, parent, and therapist characteristics associated with parent engagement may provide direction for clinical modifications needed to improve implementation of evidence-based practice efforts in community settings.

Methods

Design

The current prospective study drew data from a randomized pilot study that examined the preliminary effectiveness of a toolkit (the Parent And Caregiver Active Participation Toolkit or PACT; Haine-Schlagel and Bustos 2013) to promote PPE in community-based child MH services (Haine-Schlagel et al. 2015).

Procedures

Therapists were recruited and enrolled prior to parent–child dyads. Therapists were recruited through staff meetings at their agency or individually by program directors and research staff. Therapist eligibility criteria were: (1) employed at their agency for at least the next 5 months, (2) provides clinic-based psychotherapy to children and their families, and (3) able to start a new episode of care with an eligible parent–child dyad during the recruitment window. Therapists who consented were randomized to standard care or PACT plus standard care. Therapists with an eligible parent–child dyad asked the dyad to consent to be contacted by research staff to receive details about study participation. Eligibility criteria for parent–child dyads were: (1) four or fewer sessions had occurred with the therapist, (2) English-speaking, (3) parent was at least 18 years old, (4) child was between 4 and 13 years old, and (5) parent had identified disruptive behavior problems (e.g., aggression, noncompliance, delinquency) as a presenting problem. A total of 20 parent–child dyads that were receiving MH services from 19 therapists provided study consent (child assent obtained for children 7 years and older). Both therapists and parents agreed to participate in the study for 4 months.

After informed consent and child assent, as appropriate, were obtained, therapists and parents completed a set of baseline study questionnaires. Therapists and parents were asked to complete a monthly survey (up to four) and therapists submitted recordings of their therapy sessions with their enrolled parent–child dyad to the research team. At the end of 4 months of treatment from the date of study consent, therapists and parents were asked to complete a final set of study questionnaires. Note that a subset (n = 6) of parent–child dyads ended treatment before the 4 months of study duration. For their study participation, therapists in the PACT condition received $45 and therapists in the standard care condition received $30 upon data submission. Parents in the PACT condition received up to $50 and parents in the standard care condition received up to $35 upon data submission. Therapists and parents in the PACT condition received higher incentive amounts due to completing a greater number of study activities. Both therapists and parents were entered into small opportunity drawings that were each worth $10 throughout the study. Study procedures were approved by the Institutional Review Boards at San Diego State University and Rady Children's Hospital-San Diego. The authors have no conflicts of interest to declare.

Participants

Seventeen therapists providing publicly-funded MH treatment for 18 child-parent dyads in five outpatient community MH clinics had complete outcome assessment data and were included in the current study. Therapists were 88 % female, an average of 34.89 years old (SD 9.26; range 24.82–61.51), and 59 % self-identified as Hispanic/Latino for ethnicity. Primary discipline reported was 47 % Marriage and Family Therapy, 24 % Psychology, and 29 % Social Work. Approximately 41 % reported Cognitive-Behavioral or Behavioral as their primary theoretical orientation. The majority were unlicensed (71 %) and held Masters-level degrees (74 %). On average, therapists had 7.17 years of professional experience (SD 9.18; range 0.25–39.00). The majority (94 %) reported that they had received training in evidence-based practices.

Parents were predominantly biological parents (94 %), female (89 %), single (67 %), and, on average, 35.80 years old (SD 11.00; range 24.93–67.30). Regarding parent race/ ethnicity (not mutually exclusive categories), 44 % self-identified as Hispanic/Latino, 50 % as White/Caucasian, 17 % as Black/African American, 6 % as Asian, 6 % as Multiracial, and 33 % as Other. Parent-reported education was: 11 % less than high school, 39 % high school graduate/GED, 22 % Associate's Degree, 22 % Bachelor's Degree, and 6 % Other. The majority (89 %) reported their annual income at $35,000 or less. Children were 83 % boys, and, on average, 8.70 years old (SD 2.28; range 5.19–12.39). Children's primary DSM-IV Axis I diagnoses per therapist-report were: ADHD (47 %), an Anxiety Disorder (18 %), a Disruptive Behavior Disorder (12 %), a Mood Disorder (6 %), and Other Disorder (18 %). Per parent report, all children had disruptive behaviors as a primary presenting problem.

Measures

A subset of targeted measures from the larger parent study were selected and informed by the extant literature. Selection of measures was guided by the need to balance inclusion of the most salient factors associated with PPE to maximize statistical power given the pilot study sample size.

Child-Level Measures

Sociodemographics and Service Use

At the baseline study assessment, parents were asked to complete questionnaires regarding child/parent sociodemographic characteristics and history of MH service use. For the current study, child age and first time utilization of MH services were examined as predictors.

Eyberg Child Behavior Inventory (ECBI) (Eyberg and Pincus 1999)

The ECBI is a 36-item parent-reported standardized measure with strong psychometric properties that assesses the frequency and intensity of child disruptive behaviors (Eyberg and Pincus 1999). The ECBI yields two scores: an Intensity score that represents the frequency of occurrence of the 36 behaviors listed, and a Problem score that represents the total number of behaviors that parents endorsed as being a problem for them. For the current study the baseline Intensity raw score was used. Internal consistency of this measure in the current sample was strong (α = 0.93).

Parent-Level Measures

Sociodemographics

At baseline, parents were asked to report on sociodemographics variables. For the current study parent self-reported ethnicity and education (high school vs. greater than high school) were examined as predictors.

Patient Health Questionnaire (PHQ-8) (Kroenke and Spitzer 2002)

This is a self-report MH screening tool with strong psychometric properties used to assess symptoms of depression, anxiety, substance use, somatoform, and eating disorders (Kroenke and Spitzer 2002). For this study only the eight-item depression subscale (PHQ-8), which parents completed at baseline, was used. Higher scores indicate greater frequency of depression symptoms. Internal consistency in this sample was good (α = 0.88).

Parent Motivation Inventory (PMI) (Nock and Photos 2006)

This is a 25-item self-report measure of parents’ motivation to participate in their child's therapy that has yielded strong psychometric properties in a previous study of children with disruptive behavior problems (Nock and Photos 2006). The PMI has three subscales and all were used in this study: Readiness (to change parenting behaviors), Desire (for change in the child and family), and Perceived Ability (to change parenting behaviors). Internal consistency for the Readiness subscale was good, α = 0.86, acceptable for the Desire subscale, α = 0.70, and adequate for the Ability subscale, α = 0.57. Item-to-total correlations were examined for the Ability subscale given that the alpha fell below the conventional standards for adequate internal consistency, per recommended practice (Clark and Watson 2003). The item-to-total correlations were large (M of r = 0.69; range 0.56–0.79).

Therapist-Level Measures

Therapist Training

At study initiation, therapists were asked about their professional training and background. For this study previous training in parent-mediated interventions was selected because therapists with this type of prior training may have different attitudes towards PACT, a parent-mediated engagement intervention.

Texas Christian University Survey of Organizational Functioning (TCU-SOF) (Lehman et al. 2002)

The TCU-SOF is a therapist-report measure regarding attitudes about therapy skills and training and the functioning of the program in which treatment services are provided. There are 31 subscales in the original measure. For the current study, only the Efficacy (five items that measure staff confidence in professional skills and performance) and Influence (six items that inquire about staff interactions, sharing, and mutual support) subscales were used, as the adult MH treatment literature has indicated an association between those subscales and client engagement (Broome et al. 2007; Landrum et al. 2012). Internal consistency was acceptable for the Influence subscale, α = 0.67, and good for the Efficacy subscale, α = 0.82. Given that the internal consistency for the Influence subscale was below conventional standards for adequate reliability, the item-to-total correlations were examined, per recommended practice (Clark and Watson 2003). These correlations were all large, (M of r = 0.67, range 0.57–0.74).

PACT Therapist Attitudes, Confidence, and Knowledge in Promoting Parent Participation Survey

(TACK) The TACK is a therapist-report scale assessing attitudes, knowledge, and confidence about strategies to promote parent participation created for the parent study. This measure has three subscales that assess Values of, Knowledge of, and Confidence with strategy use. Based on previous literature (Brookman-Frazee et al. 2009) reporting that community MH therapists highly value many psychotherapy strategies common in evidence-based interventions for child disruptive behavior problems, only the nine-item Values subscale from the therapist baseline report was used in the current study. Internal reliability was strong in this sample, α = 0.88.

Outcome Measure

Parent Participation Engagement in Child Psychotherapy Observational Coding System (Haine-Schlagel and Martinez 2014)

This is a 5-item observational measure developed for the study that assesses the extensiveness of the following specific in-session parent behaviors that reflect PPE: (1) To what extent did the parent share his/her perspective in general? (2) To what extent did the parent share his/her perspective about parent and/or child home actions? (3) To what extent did the parent agree with/appear enthusiastic about home actions? (4) To what extent did the parent ask the therapist questions? and (5) To what extent did the parent demonstrate commitment to therapy in the session? Each item is rated on a five-point likert scale (1–5) with higher numbers indicating greater extensiveness of parent participation. Of the 126 full length (approximately 50 min) session video recordings received, 93 session recordings were selected for coding by five undergraduate coders who had achieved greater than 80 % inter-rater reliability on training recordings. Session recordings selected for coding included: (a) the initial four sessions after parent/child dyad enrollment, (b) three middle treatment sessions that were approximately 1 month apart, and (c) the final treatment session. Approximately 30 % of sessions were double-coded by a second coder to assess inter-rater reliability. Inter-rater agreement for these double coded videos was fair to good according to accepted standards (ICCs ranged from 0.51 to 0.72) (Cicchetti 1994). A Total Mean Score of PPE was computed across coded sessions for each parent–child dyad and used in the current study. Internal reliability was good, α = 0.84.

Data Analytic Plan

Since data were drawn from a parent pilot intervention study, study condition (PACT plus standard care vs. standard care alone) was included as a covariate in all analyses. Descriptive statistics were performed for each correlate and the outcome variable to assess its distribution. A separate linear regression model was conducted for each child, parent, and therapist characteristic controlling for study condition. This approach of including one predictor variable and the covariate in each regression model is in line with recent research reporting the requirement of only two subjects per variable in accurate estimation of linear regression coefficients and standard errors (Austin and Steyerberg 2015). Results from the linear regression models are reported. Cohen's f2 (Cohen 1988), which is appropriate for estimating the effect size within regression analyses, was calculated for each regression model.

Results

Descriptive characteristics of the sample are displayed in Table 1. Results from the regression analyses are displayed in Table 2 and described below. Given the small sample size and exploratory, novel focus on observational measurement of PPE, effect size estimation based on commonly used standards (Cohen 1988) was examined in addition to the more traditional p value examination. Specifically, regression models with f2 ≥ 0.15, representing a medium effect size, were interpreted.

Table 1.

Participant Characteristics

n M or % SD Observed range
Child characteristics
    Age 18 8.70 2.28 5.19–12.39
    First time MH service use 6 33 % – –
    ECBI intensity 18 145.56 37.76 73.00–203.00
Parent characteristics
    Ethnicity (Hispanic) 8 44 % – –
    Education (high school/GED vs. more than high school/GED) 18 50 % – –
    PHQ 8 18 7.67 6.41 0.00–22.00
    PMI desire for change 18 31.28 3.25 26.00–35.00
    PMI readiness to change 18 65.83 4.53 58.00–70.00
    PMI ability to change 18 17.56 2.25 14.00–20.00
    ppe observational coding scale 18 2.97 0.50 2.13–3.78
Therapist characteristics
    Training in parent-mediated interventions 9 56 % – –
    TACK values 17 4.70 0.37 3.83–5.00
    TCU efficacy 17 40.71 30.00–46.00 4.47
    TCU influence 17 37.16 4.89 28.33–48.33
Open in a new tab

Table 2.

Child, Parent, and Therapist Characteristics Associated with Observed PPE Controlling for Study Condition

B ΔR 2 R 2 p value f 2
Child-level
    Age 0.03 0.01 0.04 0.76 0.04
    First time MH service use –0.41 0.14 0.17 0.25 0.20*
    ECBI intensity 0.004 0.11 0.14 0.33 0.16*
Parent-level
    Ethnicity (Hispanic) –0.58 0.34 0.37 0.03 0.59*
    Education (high school/GED vs. more than high school/GED) 0.16 0.03 0.05 0.52 0.05
    PHQ 8 –0.001 0.00 0.03 0.81 0.03
    PMI desire for change <0.001 0.00 0.00 0.82 <0.001
    PMI readiness to change 0.02 0.03 0.06 0.63 0.06
    PMI ability to change 0.10 0.19 0.22 0.16 0.28*
Therapist-level
    Training in parent-mediated interventions 0.42 0.16 0.19 0.23 0.23*
    TACK values –0.43 0.10 0.12 0.38 0.14
    TCU efficacy 0.05 0.17 0.20 0.19 0.25*
    TCU influence 0.18 0.03 0.06 0.64 0.06
Open in a new tab

A separate model was performed for each child, parent and therapist characteristic with study condition included as the sole covariate; B coefficients reported are unstandardized; f2 ≥ 0.02, f2 ≥ 0.15, and f2 ≥ 0.35 represent small, medium, and large effect sizes, respectively (Cohen 1988)

*

An effect size of medium to large magnitude; bolded and underlined p values are less than 0.05

At the child-level, the regression models for child age, first time receipt of MH services, and ECBI-Intensity, controlling for treatment condition, were not statistically significant. However, first time MH services use and the total score on the EBCI Intensity scale had an effect size in the moderate range, suggesting that children with no previous MH service utilization had lower observed in-session PPE while those with greater intensity of behavior problems at baseline had greater observed in-session PPE.

At the parent level, the regression model that examined parent-reported ethnicity (Hispanic vs. Non-Hispanic), controlling for treatment condition, was statistically significant, F(2, 15) = 4.34, p < 0.05 and explained 37 % of the variance in observed PPE. Observed PPE extensiveness scores were on average 0.58 less for Hispanic parents relative to non-Hispanic parents. The effect size for this model was large. The regression models for parent education, the PHQ-8, and the PMI subscales were not statistically significant. However, the effect size for the PMI Ability to Change subscale was in the moderate range, suggesting that greater belief in ability to change parenting behaviors was associated with greater observed PPE in sessions.

At the therapist-level, the regression models for training in parent-mediated interventions, TACK Values, TCU Efficacy, and TCU Influence were not statistically significant. However, the effect size for therapist training in parent-mediated interventions and TCU Efficacy were in the moderate range, suggesting that therapists who received former training in parent-mediated interventions and those with higher perceived therapeutic self-efficacy had greater observed PPE in sessions.

Discussion

This study examined child, parent, and therapist characteristics that may be associated with observational assessment of in-session PPE across 4 months of treatment in a small sample of parent/child dyads receiving community-based MH care for disruptive behavior problems. Study findings, while preliminary due to the pilot nature of the study, provide partial support for hypotheses and suggest that unique client and therapist attributes may affect PPE behaviors observed in session. Specifically, consistent with study hypotheses, effect size estimates indicate that children with greater behavior problems at the start of treatment had higher observed PPE. Contrary to study hypotheses, effect size estimates indicate that children who were first-time MH service users had lower observed PPE. At the parent level, in line with study hypotheses, Hispanic parents had significantly lower observed PPE than non-Hispanic parents, and effect size estimates indicate that those with stronger beliefs about their ability to change their parenting behaviors at the start of treatment had higher observed PPE. Level of parent psychopathology was not associated with observed PPE in this sample. Finally, effect size estimates indicate that therapists who had prior training in parent-mediated interventions and those with greater confidence in their professional skills and performance as a therapist had greater observed PPE across 4 months of treatment, which is consistent with therapist-level study hypotheses.

In addition to partial support of study hypotheses, findings from the current study also provide partial support for the extant, albeit limited, literature with regard to child- and parent-level characteristics associated with PPE. Specifically, this study's findings regarding the child/family-level correlates are consistent with past literature identifying family ethnicity, child service use, and clinical characteristics as significant correlates with PPE in MH services (Dumas and Albin 1986; Fawley-King et al. 2012). In addition, aspects of parent motivation to participate in treatment have been associated with greater PPE (Andrade et al. 2015; Nock and Kazdin 2005), which is consistent with this study's finding that parent perceptions regarding their ability to change their parenting skills may be linked to greater observed PPE. Therefore, family sociodemographics, child clinical attributes, and parent attitudes represent potentially important targets to impact PPE in children's MH services.

Study results also offer new insights with regard to therapist-level factors associated with PPE within the context of treatment for disruptive behavior problems given the scant literature in this area (Haine-Schlagel and Walsh 2015). Findings suggest that previous MH training and therapist attitudes about their practice may be important variables for further study in PPE research. These findings underscore the importance of including assessment of therapist self-efficacy in future research to both understand and enhance PPE in community-based care.

While the depth of data collected and observational measurement of PPE are significant study strengths, the small sample size mitigated statistical power, the ability to include a broader range of family and therapist characteristics and the option to utilize multivariate or multilevel analytic techniques. It should be emphasized that the effect sizes for many regression models were medium to large suggesting that insufficient power strongly contributed to statistical results. In addition, a small subset of parent–child dyads did not complete 4 months of treatment, the length of study duration, thus limiting the consistency of data collection across participants. Further, this study only included children with a disruptive behavior presenting problem, thus limiting the examination of PPE within this clinical context. However, prevalence rates are highest for ADHD and disruptive behavior disorders in community mental health service settings (Garland et al. 2001); thus, results may still be generalizable to a large portion of youth receiving community mental health care. Related, because the context of this study was set in standard care community mental health settings, it is not known what specific structured interventions (if any) therapists delivered. A valuable next step in this research would be to examine the relation between therapist intervention delivery and PPE.

This study adds to the literature by examining a broad range of factors associated with observational assessment of PPE to better understand PPE in a diverse sample of parent/child dyads receiving community-based MH treatment for child disruptive behavior problems. Although preliminary, study findings underscore the need to assess factors at multiple levels to optimally understand contributors to PPE in community-based MH settings where PPE is typically low. Greater understanding of the contextual and individual characteristics that optimize PPE, particularly within community-based MH settings where PPE is typically low, is an important step towards identifying targets of change to improve the quality of care provided in child MH treatment through implementation of evidence-based practices in community settings.

Acknowledgments

This study was supported by the National Institute of Mental Health of the National Institutes of Health under Award Number K23MH080149 (PI: Haine-Schlagel). The authors would like to acknowledge Cristina Bustos, Ph.D., Amy Drahota, Ph.D., Scott Roesch, Ph.D., Ann Garland, Ph.D., Cortney Janicki, and Emily Ewing for their contributions to the project as well as the participating clinics, therapists, and families.

Footnotes

1

The term “parent” throughout this article represents any primary caregiver.

Compliance with Ethical Standards

Conflict of interest All authors have no conflict of interest to declare.

Ethical Approval All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent Informed consent was obtained from all individual participants included in the study.

References

  1. American Psychological Association Presidential Task Force on Evidence-Based Practice Evidence-based practice in psychology. American Psychologist. 2006;61:271–285. doi: 10.1037/0003-066X.61.4.271. doi:10.1037/0003-066X.61.4.271. [DOI] [PubMed] [Google Scholar]
  2. Andrade BF, Browne DT, Naber AR. Parenting skills and parent readiness for treatment are associated with child disruptive behavior and parent participation in treatment. Behavior Therapy. 2015;46:365–378. doi: 10.1016/j.beth.2015.01.004. doi:10.1016/j.beth.2015.01.004. [DOI] [PubMed] [Google Scholar]
  3. Austin PC, Steyerberg EW. The number of subjects per variable required in linear regression analyses. Journal of Clinical Epidemiology. 2015;68:627–636. doi: 10.1016/j.jclinepi.2014.12.014. doi:10.1016/j.jclinepi.2014.12.014. [DOI] [PubMed] [Google Scholar]
  4. Baker-Ericzén MJ, Jenkins MM, Haine-Schlagel R. Therapist, parent, and youth perspectives of treatment barriers to family-focused community outpatient mental health services. Journal of Child and Family Studies. 2013;22:854–868. doi: 10.1007/s10826-012-9644-7. doi:10.1007/s10826-012-9644-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Becker KD, Lee BR, Daleiden EL, Lindsey M, Brandt NE, Chorpita BF. The common elements of engagement in children's mental health services: Which elements for which outcomes? Journal of Clinical Child and Adolescent Psychology. 2015;44:30–43. doi: 10.1080/15374416.2013.814543. doi:10.1080/15374416.2013.814543. [DOI] [PubMed] [Google Scholar]
  6. Brookman-Frazee L, Garland AF, Taylor R, Zoffness R. Therapists' attitudes towards psychotherapeutic strategies in community-based psychotherapy with children with disruptive behavior problems. Administration and Policy in Mental Health and Mental Health Services Research. 2009;36:1–12. doi: 10.1007/s10488-008-0195-6. doi:10.1007/s10488-008-0195-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Broome KM, Flynn PM, Knight DK, Simpson DD. Program structure, staff perceptions, and client engagement in treatment. Journal of Substance Abuse Treatment. 2007;33:149–158. doi: 10.1016/j.jsat.2006.12.030. doi:10.1016/j.jsat.2006.12.030. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Chamberlain P, Patterson G, Reid J, Kavanagh K, Forgatch M. Observation of client resistance. Behavior Therapy. 1984;15:144–155. doi:10.1016/S0005-7894(84)80016-7. [Google Scholar]
  9. Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment. 1994;6(4):284. doi:10.1037/1040-3590.6.4.284. [Google Scholar]
  10. Clark LA, Watson D. Constructing validity: Basic issues in objective scale development. In: Kazdin AE, editor. Methodological issues & strategies in clinical research. 3rd ed. American Psychological Association; Washington, DC: 2003. pp. 207–231. [Google Scholar]
  11. Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Lawrence Erlbaum; Hillsdale, NJ: 1988. [Google Scholar]
  12. David-Ferdon C, Kaslow NJ. Evidence-based psychosocial treatments for child and adolescent depression. Journal of Clinical Child and Adolescent Psychology. 2008;37:62–104. doi: 10.1080/15374410701817865. doi:10.1080/15374410701817865. [DOI] [PubMed] [Google Scholar]
  13. Dowell KA, Ogles BM. The effects of parent participation on child psychotherapy outcome: A meta-analytic review. Journal of Clinical Child and Adolescent Psychology. 2010;39:151–162. doi: 10.1080/15374410903532585. doi:10.1080/15374410903532585. [DOI] [PubMed] [Google Scholar]
  14. Dumas JE, Albin JB. Parent training outcome: Does active parental involvement matter? Behaviour Research and Therapy. 1986;24:227–230. doi: 10.1016/0005-7967(86)90097-5. doi:10.1016/0005-7967(86)90097-5. [DOI] [PubMed] [Google Scholar]
  15. Evans SW, Owens JS, Bunford N. Evidence-based psychosocial treatments for children and adolescents with attention-deficit/hyperactivity disorder. Journal of Clinical Child and Adolescent Psychology. 2014;43:527–551. doi: 10.1080/15374416.2013.850700. doi:10.1080/15374416.2013.850700. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Eyberg SM, Nelson MM, Boggs SR. Evidence-based psychosocial treatments for children and adolescents with disruptive behavior. Journal of Clinical Child and Adolescent Psychology. 2008;37:215–237. doi: 10.1080/15374410701820117. doi:10.1080/15374410701820117. [DOI] [PubMed] [Google Scholar]
  17. Eyberg SM, Pincus D. Eyberg child behavior inventory and sutter-eyberg student behavior inventory: professional manual. Psychological Assessment Resources; Odessa, FL: 1999. [Google Scholar]
  18. Fawley-King K, Haine-Schlagel R, Trask EV, Zhang J, Garland AF. Caregiver participation in community-based mental health services for children receiving outpatient care. Journal of Behavioral Health Services and Research. 2012;40:180–190. doi: 10.1007/s11414-012-9311-1. doi:10.1007/s11414-012-9311-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Garland AF, Brookman-Frazee L, Hurlburt M, Arnold E, Zoffness R, Haine-Schlagel R, Ganger W. Mental health care for children with disruptive behavior problems: A view inside therapists' offices. Psychiatric Services. 2010;61:788–795. doi: 10.1176/appi.ps.61.8.788. doi:10.1176/appi.ps.61.8.788. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Garland AF, Haine-Schlagel R, Accurso EC, Baker-Ericzén MJ, Brookman-Frazee L. Exploring the effect of therapists' treatment practices on client attendance in community-based care for children. Psychological Services. 2012;9:74–88. doi: 10.1037/a0027098. doi:10.1037/a0027098. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Garland AF, Hough RL, McCabe KM, Yeh M, Wood PA, Aarons GA. Prevalence of psychiatric disorders in youths across five sectors of care. Journal of the American Academy of Child and Adolescent Psychiatry. 2001;40:409–418. doi: 10.1097/00004583-200104000-00009. doi:10.1097/00004583-200104000-00009. [DOI] [PubMed] [Google Scholar]
  22. Gopalan G, Goldstein L, Klingenstein K, Sicher C, Blake C, McKay MM. Engaging families into child mental health treatment: Updates and special considerations. Journal of the Canadian Academy of Child and Adolescent Psychiatry. 2010;19:182–196. http://www.academia.edu/15451997/Engaging_families_into_child_mental_health_treatment_Updates_and_special_considerations. [PMC free article] [PubMed] [Google Scholar]
  23. Haine-Schlagel R, Brookman-Frazee L, Fettes DL, Baker-Ericzén M, Garland AF. Therapist focus on parent involvement in community-based youth psychotherapy. Journal of Child and Family Studies. 2012;21:646–656. doi: 10.1007/s10826-011-9517-5. doi:10.1007/s10826-011-9517-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Haine-Schlagel R, Bustos C. The parent and caregiver active participation toolkit (PACT): Therapist training manual. San Diego State University; San Diego: 2013. [Google Scholar]
  25. Haine-Schlagel R, Martinez J. Parent participation engagement in child psychotherapy observational coding system. San Diego State University; San Diego: 2014. [Google Scholar]
  26. Haine-Schlagel R, Martinez J, Roesch S, Bustos C, Janicki C. Randomized trial of the Parent and Caregiver Active Participation Toolkit for child mental health treatment. 2015 doi: 10.1080/15374416.2016.1183497. Manuscript submitted for publication. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Haine-Schlagel R, Walsh NE. A review of parent participation engagement in child and family mental health treatment. Clinical Child and Family Psychology Review. 2015;18:133–150. doi: 10.1007/s10567-015-0182-x. doi:10.1007/s10567-015-0182-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Hansen DJ, Warner JE. Treatment adherence of maltreating families: A survey of professionals regarding prevalence and enhancement strategies. Journal of Family Violence. 1994;9:1–19. doi:10. 1007/BF01531965. [Google Scholar]
  29. Hoagwood KE, Olin SS, Horwitz S, McKay M, Cleek A, Gleacher A, Lewandowski E. Scaling up evidence-based practices for children and families in New York state: Towards evidence-based polices on implementation for state mental health systems. Journal of Child and Adolescent Psychology. 2014;43:145–157. doi: 10.1080/15374416.2013.869749. doi:10.1080/15374416.2013.869749. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Huang L, Stroul B, Friedman R, Mrazek P, Friesen B, Pires S, Mayberg S. Transforming mental health care for children and their families. American Psychologist. 2005;60:615–627. doi: 10.1037/0003-066X.60.6.615. doi:10.1037/0003-066X.60.6.615. [DOI] [PubMed] [Google Scholar]
  31. Ingoldsby EM. Review of interventions to improve family engagement and retention in parent and child mental health programs. Journal of Child and Family Studies. 2010;19:629–645. doi: 10.1007/s10826-009-9350-2. doi:10.1007/s10826-009-9350-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Jensen SA, Grimes LK. Increases in parent attendance to behavioral parent training due to concurrent child treatment groups. Child and Youth Care Forum. 2010;39:239–251. doi:10.1007/s10566-010-9101-y. [Google Scholar]
  33. Karver MS, Handelsman JB, Fields S, Bickman L. Meta-analysis of therapeutic relationship variables in youth and family therapy: The evidence for different relationship variables in the child and adolescent treatment outcome literature. Clinical Psychology Review. 2006;26:50–65. doi: 10.1016/j.cpr.2005.09.001. doi:10. 1016/j.cpr.2005.09.001. [DOI] [PubMed] [Google Scholar]
  34. Keel PK, Haedt A. Evidence-based psychosocial treatments for eating problems and eating disorders. Journal of Clinical Child and Adolescent Psychology. 2008;37:39–61. doi: 10.1080/15374410701817832. doi:10.1080/15374410701817832. [DOI] [PubMed] [Google Scholar]
  35. Kim H, Munson MR, McKay MM. Engagement in mental health treatment among adolescents and young adults: A systematic review. Child and Adolescent Social Work Journal. 2012;29:241–266. doi: 10.1007/s10560-022-00893-x. doi:10.1007/s10560-012-0256-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Kroenke K, Spitzer RL. The PHQ-9: A new depression and diagnostic severity measure. Psychiatric Annals. 2002;32:509–521. doi:10.3928/0048-5713-20020901-06. [Google Scholar]
  37. Landrum B, Knight DK, Flynn PM. The impact of organizational stress and burnout on client engagement. Journal of Substance Abuse Treatment. 2012;42:222–230. doi: 10.1016/j.jsat.2011.10.011. doi:10.1016/j.jsat.2011.10.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Lehman WEK, Greener JM, Simpson DD. Assessing organizational readiness for change. Journal of Substance Abuse Treatment. 2002;22:197–209. doi: 10.1016/s0740-5472(02)00233-7. doi:10.1016/j.jsat.2011.10.011. [DOI] [PubMed] [Google Scholar]
  39. Lindsey MA, Brandt NE, Becker KD, Lee BR, Barth RP, Daleiden EL, Chorpita BF. Identifying the common elements of treatment engagement interventions in children's mental health services. Clinical Child and Family Psychology Review. 2014;17:283–298. doi: 10.1007/s10567-013-0163-x. doi:10.1007/s10567-013-0163-x. [DOI] [PubMed] [Google Scholar]
  40. McHugh RK, Barlow DH. The dissemination and implementation of evidence-based psychological treatments. A review of current efforts. American Psychologist. 2010;65:73–84. doi: 10.1037/a0018121. doi:10.1037/a0018121. [DOI] [PubMed] [Google Scholar]
  41. Nakamura BJ, Chorpita BF, Hirsch M, Daleiden E, Slavin L, Amundson MJ, Rocco S. Large-scale implementation of evidence-based treatments for children 10 years later: Hawaii's evidence-based services initiative in children's mental health. Clinical Psychology Science and Practice. 2011;18:24–35. doi:10.1111/j.1468-2850.2010.01231.x. [Google Scholar]
  42. Nock MK, Ferriter C. Parent management of attendance and adherence in child and adolescent therapy: A conceptual and empirical review. Clinical Child and Family Psychology Review. 2005;8:149–166. doi: 10.1007/s10567-005-4753-0. doi:10.1007/s10567-005-4753-0. [DOI] [PubMed] [Google Scholar]
  43. Nock MK, Kazdin AE. Randomized controlled trial of a brief intervention for increasing participation in parent management training. Journal of Consulting and Clinical Psychology. 2005;73:872–879. doi: 10.1037/0022-006X.73.5.872. doi:10.1007/s10826-006-9022-4. [DOI] [PubMed] [Google Scholar]
  44. Nock MK, Photos V. Parent motivation to participate in treatment: Assessment and prediction of subsequent participation. Journal of Child and Family Studies. 2006;15:345–358. doi:10.1007/s10826-006-9022-4. [Google Scholar]
  45. Patterson GR, Forgatch MS. Therapist behavior as a determinant for client noncompliance: A paradox for the behavior modifier. Journal of Consulting and Clinical Psychology. 1985;53:846–851. doi: 10.1037//0022-006x.53.6.846. doi:10.1037//0022-006X.53.6.846. [DOI] [PubMed] [Google Scholar]
  46. Pereira AI, Muris P, Mendonça D, Barros L, Goes AR, Marques T. Parental involvement in cognitive-behavioral intervention for anxious children: Parents' in-session and out-session activities and their relationship with treatment outcome. Child Psychiatry and Human Development. 2015 doi: 10.1007/s10578-015-0549-8. doi:10. 1007/s10578-015-0549-8. [DOI] [PubMed] [Google Scholar]
  47. Podell JL, Kendall PC. Mothers and fathers in family cognitive-behavioral therapy for anxious youth. Journal of Child and Family Studies. 2011;20:182–195. doi:10.1007/s10826-010-9420-5. [Google Scholar]
  48. Silverman WK, Pina AA, Viswesvaran C. Evidence-based psychosocial treatments for phobic and anxiety disorders in children and adolescents. Journal of Clinical Child and Adolescent Psychology. 2008;37:105–130. doi: 10.1080/15374410701817907. doi:10.1080/15374410701817907. [DOI] [PubMed] [Google Scholar]
  49. Starin AC, Atkins MS, Wehrmann KC, Mehta T, Hesson-McInnis MS, Marinez Lora A, Mehlinger R. Moving science into state child and adolescent mental health systems: Illinois' evidence-informed practice initiative. Journal of Child and Adolescent Psychology. 2014;43:169–178. doi: 10.1080/15374416.2013.848772. doi:10.1080/15374416.2013.848772. [DOI] [PubMed] [Google Scholar]
  50. Staudt M. Treatment engagement with caregivers of at-risk children: Gaps in research and conceptualization. Journal of Child and Family Studies. 2007;16:183–196. doi:10.1007/s10826-006-9077-2. [Google Scholar]
  51. Stoolmiller M, Duncan T, Bank L, Patterson GR. Some problems and solutions in the study of change: Significant patterns in client resistance. Journal of Consulting and Clinical Psychology. 1993;61:920–928. doi: 10.1037//0022-006x.61.6.920. doi:10.1037/0022-006X.61.6.920. [DOI] [PubMed] [Google Scholar]
  52. Synder J, Reid J, Stoomiller M, Howe G, Brown H, Dagne G, Cross W. The role of behavior observation in measurement systems for randomized prevention trials. Prevention Science. 2006;7:43–56. doi: 10.1007/s11121-005-0020-3. doi:10.1007/s11121-005-0020-3. [DOI] [PubMed] [Google Scholar]
  53. U. S. Department of Health and Human Services, National Institutes of Health, National Institute of Mental Health NIMH Strategic Plan for Research (NIH Publication No. 02-2650) 2015 http://www.nimh.nih.gov/about/strategic-planning-reports/index.shtml.

RESOURCES