Exploring the Effect of Therapists’ Treatment Practices on Client Attendance in Community-Based Care for Children

Abstract

Sustained treatment attendance has been reported to be poor in publicly-funded community-based clinic settings serving children and families. Several child and family characteristics have been shown to predict attendance in community-based care, but virtually no research has been conducted to examine how experiences in care, including psychotherapists’ within-session practices, influence client attendance. The goal of this exploratory study was to examine how observed practice within sessions, in particular the extent to which therapists delivered elements consistent with evidence-based practices, impacts total number of sessions attended, while accounting for an array of other potential predictors. Participants include 181 children ages 4–13 and their parents entering a new episode of care for disruptive behavior problems in publicly-funded clinics. Data sources include administrative billing records on treatment attendance, coded videotaped treatment sessions, and self-reports from children, parents, and therapists. Results indicate that parent education, service funding source, parent alliance with therapist, and therapist experience predicted number of sessions attended; intensity of evidence-based treatment techniques delivered to children was marginally associated with attendance (p=.059). Implications for improving engagement in community-based care are discussed.

Routine community-based psychosocial care for children with a variety of mental health problems has demonstrated limited effectiveness, especially when compared to impressive effects demonstrated in research trials of evidence-based (EB) interventions (Kazak, Hoagwood, & Weisz, et al., 2010; Weisz, Jensen-Doss, & Hawley, 2006). Current efforts to improve routine care effectiveness focus largely on implementation and dissemination of EB practices (National Advisory Mental Health Council, 2000). However, given that these treatment practices can only be effective if families are engaged (e.g., attending and participating consistently), complementary efforts to identify factors associated with treatment engagement are needed (Miller, Southam-Gerow, & Allin, 2008; Nock & Kazdin, 2001).

Treatment engagement is a broadly defined construct that can include a number of components such as treatment visit attendance (e.g., total duration or number of visits, “dose,” consistency of attendance), adherence/compliance with treatment recommendations, active participation within treatment sessions, and/or premature versus consensual treatment termination (see Miller et al., 2008 for a review). Treatment visit attendance is the most commonly studied basic indicator of engagement and is the focus of this study. Achieving sustained, consistent treatment attendance with families in community-based routine care settings is particularly challenging (Armbruster & Kazdin, 1994; Harpaz-Rotem, Leslie, Rosenhack, 2004; Kazdin, Mazurick, & Siegel, 1994; McKay & Bannon, 2004; Miller et al., 2008). For example, McKay and colleagues (Gopalan, Goldstein, Klingenstein, Sicher, Blake, & McKay, 2010; McKay, Harrison, Gonzales, Kim, & Quintana, 2002) have reported that average attendance in routine care settings is approximately 3 or 4 sessions, compared to an average of 11 sessions in a research context (Weisz, Doss, & Hawley, 2005). As many as 40 to 60% of families stop attending treatment sessions before the provider deems treatment to be complete (Kazdin, 1996; Wierzbicki & Pekarik, 1993). This is particularly concerning because poor treatment attendance has been associated with poorer treatment outcomes in child and adult services (Angold, Costello, Burns, Erkanli, & Farmer, 2000; Kopta, Howard, Lowry, & Beutler, 1994).

Poor treatment attendance, specifically inconsistent or brief client attendance, also has significant pragmatic and fiscal implications for community-based clinics (Pellarin, Costa, Weems, & Dalton, 2010). Providers express frustration with high “no show” rates, and anecdotal reports suggest that many clients in the publicly funded service system come in and out of multiple episodes of treatment. These inconsistent attendance patterns often have significant negative fiscal implications as missed sessions may reduce revenue and limit access for new patients. Understanding the factors affecting attendance is important to service providers for planning and policy decisions, and is vital in the context of increasing pressure to implement EB treatments, which rely upon sustained attendance.

Studies in both research and community contexts have identified a variety of client characteristics associated with treatment attendance (e.g., Fernandez & Eyberg, 2009; Kendall & Sugarman, 1997; Miller et al., 2008; Nock & Ferriter, 2005). For example, socio-demographics such as single parent status, higher parent education, White race/ethnicity, and public insurance coverage have been found to be associated with greater treatment attendance (Gaskin, Kouzis, & Richard, 2008; Miller et al., 2008). Clinical characteristics such as greater child symptom severity, more severe diagnoses and diagnostic comorbidity, and Axis IV stressors at service entry have also predicted greater attendance (Brookman-Frazee, Haine, Gabayan, & Garland, 2008; Evans, Radunovich, Cornette, Wienx, & Roy, 2008; Miller et al., 2008), but there are some inconsistencies across studies (e.g., Burns, Cortell, Wagner, 2008). Parent-youth agreement on treatment goals and parents’ expectations about treatment at service entry have also been associated with attendance (Brookman-Frazee et al., 2008; Nock & Kazdin, 2001; Shuman & Shapiro, 2002).

Thus, despite some inconsistencies in findings, several factors present (i.e., predetermined) at the start of treatment have been shown to predict treatment attendance in child/family services. However, there is very little research examining how experience in treatment sessions may be associated with attendance. The limited available research supports the logical assumption that clients’ experiences in treatment sessions influence subsequent attendance. Specifically, Noser and Bickman (2000) report that more positive youth-reported therapeutic alliance and parent-reported satisfaction with services are associated with better attendance. Qualitative data from a study of parents of children served in the public mental health sector also suggest that within-session therapeutic practices may be strong predictors of client attendance (Baker-Ericzén, Jenkins, & Haine-Schlagel, under review). Specifically, parents and youth reported that the receipt of ineffective therapy lacking directive skill-building strategies and limited involvement in shared decision making regarding their child’s needs contributed to their poor participation in treatment (Baker-Ericzén et al., under review). In another study, family-focused treatment was associated with greater client attendance than child-focused treatment (Evans et al., 2008). This finding highlights one of the many complexities of child mental health care, namely that multiple people can be involved in treatment engagement.

We have identified no studies to date that have utilized more objective observational data to examine how variation in within-session practices delivered by community-based therapists may be associated with client attendance, nor any studies that have examined the extent to which practice consistent with EB treatment is associated with attendance in community-based care. Examining the relationship between EB practice elements and attendance is particularly timely and important given current pressures to deliver EB practices in community-based care in the context of previous research suggesting that these elements are used with relatively low intensity (Garland, Brookman-Frazee et al., 2010). If client attendance is better when treatment sessions are more consistent with EB practice, this information could be useful in dissemination and implementation efforts to advocate for increased delivery of EB practice since providers are motivated to improve client attendance. Learning more about how specific within-session practices may influence client attendance could elucidate the therapist’s role in client attendance and complement existing data on predetermined (at service entry) client characteristics that predict attendance.

The primary aim of this study is to examine how therapists’ within-session treatment practices (specifically the extent to which treatment includes strategies consistent with elements of EB treatments) may be associated with total number of sessions attended by children with disruptive behavior problems and their families, while accounting for a wide array of other potential predictive factors that have been previously identified in the literature.

Method

Data from this study were collected as part of a larger observational study comprehensively examining routine community-based care for children with disruptive behavior problems and more details regarding the practice settings and recruitment procedures are provided in Garland, Brookman-Frazee, et al. (2010).

Participants

Participating Clinics

The six participating clinics were selected because they represent the largest contractors for publicly-funded, clinic-based out-patient care for children in one of the largest counties in the U.S., and they serve ethnically and diagnostically diverse children and their families. There was no intervention to influence service delivery during the study period.

Child and Parent Participants

A total of 181 children who had complete data were included in the current study, which is a subsample of the full 218 who participated in the larger study. Inclusion criteria for child participants were (a) presenting problems included a disruptive behavior problem (aggression, defiance, delinquency, oppositional behavior by parent report), (b) age 4–13 years, (c) primary language for child and parent was English or Spanish, and (d) child entered a new episode of psychotherapy (defined as no therapy for previous three months) with a participating therapist. The intent was to have a sample generally representative of children entering treatment for disruptive behavior problems in this large Southwestern county.

Characteristics of the 181 participating children and their caregivers are provided in Table 1. As indicated, the average child age was 9 years, the majority were male (n=121; 67%) and less than half of the sample was Caucasian. Although all children had disruptive behavior problems, their clinician-assigned primary diagnoses varied, with ADHD being the most common (n=73; 40%). Caregivers (herein referred to as parents) of these children were primarily female (n=170, 93.9%) mothers (n=147, 81.2%) of the children. Average annual household income was $36,026 (SD=30,173).

Table 1.

Sample Descriptives on All Study Variables

	n	Mean (SD) or %	Actual Range
Outcome of Interest
Attendance (number of sessions attended)a	181	22.4 (15.5)	[1,70]
Treatment processes (Entry through 4 months)
Average intensity of treatment processes consistent with evidence-based practices
Child-directed content	181	1.3 (0.8)	[0,4]
Child-directed techniques	181	1.3 (0.6)	[0.1,3.8]
Parent-directed content	165	0.7 (0.6)	[0,2.4]
Parent-directed techniques	165	1.4 (0.7)	[0,4]
Average intensity of other treatment processes
Child-directed content	181	0.7 (0.5)	[0,2.4]
Child-directed techniques	181	1.2 (0.4)	[0.1,2.6]
Parent-directed content	165	1.0 (0.6)	[0,3]
Parent-directed techniques	165	1.0 (0.5)	[0,2.7]
Demographic Characteristics
Child age	181	9.0 (2.7)	[4,14]
Child gender; Male	121	66.9%
Child race/ethnicity
Caucasian	89	49.2%
Latino/Hispanic	54	29.8%
African American	16	8.8%
Other/Mixed	22	12.2%
Parent highest level of education; Some college or more	108	61.4%
Single-parent status	101	56.1%
Child Clinical Factors at Entry
Primary diagnosis
Disruptive Behavior Disorder	36	19.9%
Attention Deficit Hyperactivity Disorder	73	40.3%
Mood Disorder	42	23.2%
Anxiety Disorder	17	9.4%
Autism Spectrum Disorder/Other	13	7.2%
Diagnostic comorbidity; Two or more diagnoses	86	47.5%
Eyberg Child Behavior Inventory: Problem Intensity Score	181	146.6 (36.0)	[59,237]
Parent/Family Clinical Factors at Entry
Brief Symptom Inventory	179	57.0 (11.5)	[33,80]
Caregiver Strain Questionnaire	181	2.7 (0.8)	[1.4,6]
Family Relationship Inventory	181	9.1 (4.6)	[−5,17]
Parent Expectations at Entry and Parent Alliance at 4 Months
Parent Expectations about Counselingb	180	4.4 (3.3)	[1,22]
Therapeutic Alliance Scales for Children – Parent Report	151	42.2 (6.2)	[15,48]
Service System
Funding source
Medi-Cal	125	69.1%
School-based	47	27.3%
Primary referral source
Parent	95	54.6%
School staff	39	22.4%
Other	40	23.0%
Therapist Characteristics
Therapist months practiced	78	31.2 (41.3)	[0,300]
Therapist professional discipline
Marriage & Family Therapy	45	57.7%
Psychology	16	20.5%
Social Work	17	21.8%
Therapist primary theoretical orientation
Cognitive-Behavioral Therapy	21	26.9%
Family Systems	30	38.5%
Nondirective	7	9.0%
Eclectic/Other	20	25.6%

^aSkewness for attendance was .51 (SD =.18), and kurtosis was −.45 (SD =.36). Both values indicate that skewness and kurtosis are not problematic for this variable.

^bBecause previous studies found a curvilinear relation between parent expectations and session attendance, a quadratic effect was tested in the covariate analyses. In addition, because the variable was heavily skewed, the square root transformation was used for analyses; the transformed variable had a mean of 2.0 (SD = 0.7) and a range of 1.0 to 4.7.

^cBecause months in practice was heavily skewed, the square root transformation was used for analyses; the transformed variable had a mean of 4.7 (SD = 3.0) and a range of 0.0 to 17.3.

Therapist Participants

Of the 100 participating therapists in the larger study, 78 were identified as the “primary therapist” (defined by having the most recorded sessions) for a child study participant with data available on the outcome of interest (number of sessions attended). Table 1b lists the characteristics of the 78 therapists included in the current analyses. These therapists were primarily female (n=67; 86%) and Caucasian (n=51; 65%) with an average of 2.6 years of therapy experience. Consistent with national samples of therapists in community-based mental health care (e.g., Glisson, Landsverk, Schoenwald, et al., 2008), therapists were primarily master’s level clinicians (n=46, 59.0%).

Procedures

Data were collected from multiple sources including (1) baseline in-person interviews with children (age 9 and over), parents, and therapists to assess demographic, clinical and family data; (2) telephone follow-up interviews with family participants four months after baseline to assess for satisfaction and perceptions of therapeutic alliance; (3) videotapes of psychotherapy sessions within first four months of treatment to assess for practice elements delivered (first four months were used because all participants attended in that time-frame and then subsequently gradually left treatment); and (4) abstraction from administrative data (billing records) for most reliable information about service attendance for the entire 16-month study period and funding source. Refer to Garland, Brookman-Frazee et al. (2010), and Brookman-Frazee, Haine, Baker-Ericzén, Zoffness, & Garland (2010) for more details on procedures.

Measures

Outcome of Interest: Attendance

Total number of treatment sessions attended during the 16-month study period served as the outcome of interest and was collected from billing data.

Predictor of Primary Interest: Treatment Processes Consistent with EB Practice

Treatment processes were assessed using the PRAC Therapeutic Process Observational Coding System for Child Psychotherapy – Strategies scale (PRAC TPOCS-S) (Garland, Brookman-Frazee, & McLeod, 2008), which is an adapted version of the original TPOCS-S (McLeod, 2001; McLeod & Weisz, 2010). The PRAC-TPOCS-S assesses for a wide array of practice elements through coding of treatment session videotapes. Detailed description of the development and coder training for the PRAC TPOCS-S are provided in Garland, Brookman-Frazee et al. (2010) and Garland, Hurlburt et al. (2010).

The PRAC TPOCS-S assesses delivery of 27 different practice elements, including a subset of elements previously identified as common to EB treatments for children with disruptive behavior problems (Garland, Hawley, Brookman-Frazee, & Hurlburt, 2008). This subset of practice elements includes therapeutic techniques such as using positive reinforcement, limit-setting, and assigning/reviewing homework. It also includes therapeutic content common to EB treatments for these children including affect education, problem-solving skills, and affect management. Coders rate the intensity of delivery of each practice element targeting children, parents, or both in a treatment session. Intensity reflects both the time spent on the practice element and the thoroughness with which it was pursued and is rated from 0–6 on a Likert scale with 0 representing no observed delivery of the practice element, 1–2 = low intensity, 3–4 = medium, and 5–6 = high intensity. Thirty-one percent (n = 379) of all (1215) coded sessions were double coded to assess inter-rater reliability. Adequate inter-rater reliability was demonstrated across all PRAC TPOCS-S codes with a mean ICC of.78 (Garland, Brookman-Frazee et al., 2010).

This study utilized average intensity scores on the following PRAC TPOCS-S composites to reflect within-session practice consistent with elements of evidence-based practice: (1) EB practice element content directed to children, (2) EB practice element techniques directed to children, (3) EB practice element content directed to parents, (4) EB practice element techniques directed to parents. In addition, to control for a potential general intensity effect, the following mean intensity composites were examined as potential covariates: (1) practice elements other than EB content directed to children, (2) practice elements other than EB techniques directed to children, (3) practice elements other than EB content directed to parents, (4) practice elements other than EB techniques directed to parents. The “other” practice elements include strategies such as “interpreting meaning of behavior,” and “addressing therapist-client relationship.”

Additional Potential Covariate Predictors

Additional potential covariate predictors of attendance from several domains were examined based on the extant literature to provide the most rigorous test of the associations between practice consistent with EB techniques and attendance.

Demographic characteristics

Demographic variables were reported by the parent and included child age, gender, and race/ethnicity, as well as parent level of education and single parent status.

Child clinical factors at entry

Child clinical factors include therapist-assigned primary child psychiatric diagnosis, diagnostic comorbidity, and severity of child behavior problems as assessed by the problem intensity score on the Eyberg Child Behavior Inventory (ECBI; Eyberg & Ross, 1978). The ECBI is a well established 36-item parent-report measure designed to assess behavior problems in children ages 2 to 16. The inventory has strong psychometric characteristics including strong reliability, convergent validity, internal consistency and discriminative power (Boggs, Eyberg, & Reynolds, 1990; Eyberg & Ross, 1978; Robinson, Eyberg, & Ross, 1980; Rich & Eyberg, 2001). The Cronbach’s alpha for the ECBI is.92 in this sample.

Parent/family clinical factors at entry

Parent psychopathology was assessed using the Brief Symptom Inventory (BSI; Derogatis & Spencer, 1982). The BSI is a 53-item instrument that is one of the most widely accepted screening tools of general psychopathology and it has demonstrated excellent reliability and good convergent validity (Derogatis & Spencer, 1982). The global severity index of the BSI was used to measure parent psychopathology. The Cronbach’s alpha is.96 in this sample.

Parent strain was assessed using the Caregiver Strain Questionnaire (CGSQ; Brannan, Heflinger, & Bickman, 1997). The CGSQ is a 21-item scale that measures the extent to which parents experience disruption, strain, or other negative effects resulting from caring for a child with emotional and/or behavioral problems; the mean item score is used in this study. The CGSQ is a reliable and valid measure of parental strain (Brannan et al., 1997). Cronbach’s alpha for the global strain score in this sample is.93, with similarly high internal consistency reported in other studies of parents whose children were receiving public services (Brannan et al., 1997; Garland, Haine, & Boxmeyer, 2007).

Family functioning was assessed using the Family Relationship Index (FRI; Holahan & Moos, 1983). The FRI is a 27-item true-false parent-report measure that assesses the quality of family relationships. This measure has demonstrated good internal consistency and construct validity (Hoge, Andrews, Faulkner, & Robinson, 1989; Holahan & Moos, 1983). In this sample, the Cronbach’s alpha for the three subscales that comprise the FRI were.68,.46, and.71.

Parent expectations at entry

The Parent Expectations about Counseling (PEC) measure was created for the current study and includes six items administered at baseline that examine parents’ expectations about the usefulness of treatment, how much they will like treatment, and expected involvement in treatment. Cronbach’s alpha for the PEC is.84 in the current sample. Because previous literature has indicated a curvilinear association between expectations and attendance, a quadratic term was calculated and also examined as a covariate (Nock & Kazdin, 2001).

Parent-reported alliance at 4 months

The Therapeutic Alliance Scales for Children (TASC; Shirk & Saiz, 1992) was used to measure alliance. The TASC is a 12-item scale designed to measure child therapeutic alliance, from both the child and therapist perspective. The TASC has demonstrated good reliability and validity in previous studies (Creed & Kendall, 2005; Hawley & Weisz, 2005). A parallel parent-report form has been developed that demonstrated good internal consistency in this study’s sample (Cronbach’s alpha =.87) as well as good internal consistency and one week test-retest reliability in another study (Hawley & Weisz, 2005). Four months into treatment, the parent-report TASC was administered to parents to assess the strength of their alliance with their child’s therapist.

Service system and therapist characteristics

Service system variables included treatment funding source (i.e., school-based funding versus Medi-Cal/Medicaid abstracted through administrative records) and referral source, which was ascertained by parent report. Treatment factors included therapist self-reported professional discipline, theoretical orientation, and length of experience.

Table 1 lists descriptive data on all study variables.

Analysis Plan

SPSS (now called PASW; v. 18; Release April, 2010) was used to calculate basic sample descriptives and to calculate unadjusted correlations between key study variables (Appendix A). Given the nested data structure (children/parents within therapist), multi-level random intercept models were used to test for predictors of session attendance using SuperMix Version 1.1 (Hedecker, Gibbons, du Toit, & Patterson, 2008). Prior to conducting the random intercept model analyses, the intraclass correlation (ICC) was calculated for number of sessions to assess the percent of variability that is attributable to the therapist level (to determine whether it is necessary to account for the therapist level in subsequent analyses).

The analytic strategy required initial screening of all potential covariates in bivariate analyses to identify variables with a significant association with attendance. Covariates that were associated with the attendance outcome at p<.05 were then placed into a multivariate regression model to identify a parsimonious set of covariates. As suggested by Raudenbusch and Bryk (2002) and Snijders and Bosker (1999), any nonsignificant covariates from this multivariate model were removed for statistical efficiency within the multilevel modeling context. Final parsimonious multivariate models were then run for each of the four EB practice element composite predictors of interest to test the primary aim of the study.

Results

In the 16 month study period, child and family participants attended an average of 22.4 sessions (SD = 15.5; Range: [1,70]). Unadjusted significant correlations were found between the child-targeted EB practice content (Pearson coefficient =.18; p<.01) and EB technique (Pearson coefficient =.19; p<.01) composite scores and the attendance outcome. Neither of the parent-targeted EB practice composite scores, nor the Other composite scores, were significantly correlated with attendance. Thus, the final multivariate models were conducted including just the child-directed EB technique and content composites. Significant variability in total number of sessions was accounted for at the therapist level of the multi-level data structure (ICC=.29), confirming the need to account for the nested data structure in subsequent analyses.

Identifying potential covariate predictors of attendance

Multi-level random intercept models were used to identify significant potential covariate predictors of attendance from among all the child, family, therapist and service characteristics listed in the methods section. Significant variables in these bivariate analyses positively associated with attendance included child female gender (B = −4.45, SE = 2.27, p =.049), parent education of some college or more (B = 7.08, SE = 2.26, p =.002), school-based funding source (B = 9.95, SE = 2.62, p <.001), fewer therapist months in practice (B = −0.95, SE = 0.42, p =.025), child diagnostic comorbidity (B = 5.92, SE = 2.21, p =.007), and greater parent-reported therapeutic alliance (B = 0.61, SE = 0.19, p =.001). To identify a parsimonious set of covariates to include in our final multivariate tests of EB therapy process as a predictor of attendance, all six of the significant covariates were placed into a multivariate model. Four remained significant (parent education level, funding source, therapist months in practice, and parent-reported therapeutic alliance) and thus were included in the final multivariate models of interest.

Final multivariate tests

Results of the final multivariate tests of the associations between EB therapy process predictors that had an unadjusted relationship with attendance (child-directed techniques and content) and attendance, controlling for significant covariates and accounting for the nested data structure are presented in Tables 2a and 2b. Results indicate that the covariates (specifically, parent education level, funding source, parent-reported alliance, and therapist experience) retained significant associations with attendance in the multi-level multivariate models. The statistical significance of the child-directed EB practice content intensity score was not retained in the multivariate models (B = 2.11, SE = 1.44, p =.14). The statistical significance of the child-directed EB practice technique intensity score reflected a trend (B = 3.66, SE = 1.94, p =.059)..

Table 2a. Multivariate models predicting number of sessions.

Child-directed Content Consistent with EBP as a Predictor of Number of Sessions (n=140)

	B	SE	p-value
Covariates
Parent level of educationa	6.86	2.47	.005
Funding sourceb	7.53	2.60	.004
Parent-reported alliance	0.53	0.21	.014
Therapist months practicedc	−0.86	0.42	.040
Predictor of Interest
Child-directed EBP content intensity	2.11	1.44	.14

Table 2b.

Child-directed Techniques Consistent with EBP as a Predictor of Number of Sessions (n=140)

	B	SE	p-value
Covariates
Parent level of educationa	6.76	2.46	.006
Funding sourceb	7.43	2.58	.004
Parent-reported alliance	0.50	0.22	.021
Therapist months practicedc	−0.84	0.42	.046
Predictor of Interest
Child-directed EBP technique intensity	3.66	1.94	.059

^a0 = some high school or high school diploma/GED; 1 = some college.

^b0 = Medi-Cal; 1 = school-based funding.

^cSquare root transformation.

Discussion

Consistent with the existing literature, we found great variability in treatment session attendance among children and families receiving care in community-based settings. This study extends the literature on factors associated with attendance by testing the effects of observed within-session practices on attendance. Significant unadjusted bivariate relationships between attendance and intensity of within-session delivery of evidence-based practice elements (content and techniques) directed to children were found, with greater average intensity in EB practice elements associated with more sessions attended. However, the statistical significance of these effects attenuated when a more conservative multi-level, multivariate test was applied that included other significant predictors of attendance (i.e., parents’ education level, funding source, therapist experience, and parent reported therapeutic alliance) and the nested data structure. In the final multivariate model, there was only a trend (p=.059) for greater intensity in child-directed EB practice techniques to be associated with greater attendance.

Most of the existing research on factors associated with treatment attendance has identified client characteristics (e.g., demographic or clinical characteristics or parental expectations) predetermined at the start of treatment. The findings from this exploratory study suggest that experiences in therapy sessions, including perceptions of the therapeutic alliance and the extent to which therapists’ intervention strategies are consistent with EB practices, are also associated with attendance. The trend for more intensive delivery of strategies consistent with EB practice to be associated with a greater number of sessions attended is relevant to the current push for stronger integration of EB practices in routine care community-based settings. It is important to note there was no intervention to strengthen the delivery of EB practices in these clinics and the observed intensity of EB practice elements was low overall (Garland et al., 2010), which may have resulted in a “floor effect” whereby it was difficult to find an association between these elements and outcomes. It is also important to note that intensity of “other” practice elements not identified as evidence-based had no significant relationship with attendance, which provides preliminary support for the potential specific impact of EB practices as opposed to a more generic effect of therapeutic strategy intensity. It is possible that interventions designed to strengthen delivery of EB practices may be associated with stronger effects on client attendance. Our results indicate that a one point increase in mean intensity of child EB practice techniques delivered (rated on a 0–6 scale) was associated with 3.7 more treatment visits attended. While a one point increase represents a considerable shift in the mean for this observational study without any intervention to change provider behavior, a one point or greater shift in mean intensity may not be difficult to achieve with targeted training interventions.

In addition to the observed intensity of EB techniques directed toward children, the other unique predictors of total number of sessions attended included demographics (higher parent education level), service system (school-based funding), therapist experience (inverse direction), and parent reported therapeutic alliance. Some of these factors are consistent with the existing literature. For example, higher level of parent education has been previously associated with greater attendance (Gaskin et al., 2008). Parent education level may be a proxy for potential treatment engagement barriers such as practical resources to attend treatment regularly (e.g., access to transportation, child care) and organizational and problem-solving skills that can assist the parent and family in engaging actively in treatment. It is also possible that parental education is associated with attitudes about mental health treatment, such as reduced stigma, that may be associated with enhanced attendance.

Other significant covariates reflect both somewhat surprising and expected effects. Specifically, the association between parents’ positive reports of the therapeutic alliance and attendance is consistent with existing research and intuitively logical (Burns et al., 2008). Clients will likely be more inclined to attend sessions when they experience a strong positive alliance with the therapist. Given that parents have more influence on session attendance than do children, their perceived alliance with the therapist is likely to be a strong predictor of attendance. In this study, therapist experience (number of months practiced) was inversely correlated with parents’ perceived alliance with therapist and, likewise, therapist experience inversely predicted session attendance. Duration of experience was the only therapist characteristic associated with attendance (e.g., academic degree, discipline, and theoretical orientation were not associated). The experience effect may be associated with relatively more recent training in EB techniques, or potentially greater emphasis on strategies to engage clients. Finally, the funding source predictor was highly significant, indicating that clients receiving school-based funding for their out-patient mental health care attended more sessions than clients with Medicaid funding. Clients with school-based funding have mental health care included in their Individualized Education Program, which often provides more structural support to facilitate attendance (e.g., transportation) and some accountability to the IEP team for attendance, which may explain greater attendance among this group.

Some of the negative findings in our study are noteworthy. For example, race/ethnicity has been associated with session attendance in some studies with race/ethnic minority youths attending fewer sessions (Gaskin et al., 2008; Miller et al., 2008; Nix, Bierman, McMahon, & The Conduct Problems Prevention Research Group, 2009), whereas other studies have found no differences in attendance by race/ethnicity (Stevens, Kelleher, Ward-Estes, & Hayes, 2006). Likewise, some studies have found an effect for severity of child symptoms at treatment entry (Brookman-Frazee et al., 2008; Evans et al., 2008), whereas others have reported an inverse relationship (Burns, Cortell, & Wagner, 2008), and we found no significant effect in either direction. Diagnostic comorbidity did have a significant positive relationship with attendance in the bivariate analysis, but this effect was not retained in the multivariate model. Finally, previous research had identified a curvilinear relation between parent expectations and session attendance such that low and high expectations predicted greater session attendance (Nock & Kazdin, 2001). This study revealed neither a linear nor curvilinear relation between parent expectations and session attendance; however, the measure used in this study was much broader than that used in Nock and Kazdin’s study and may not have been specific enough to capture the association.

This study has numerous methodological strengths, most notably the rich data on within-session practice that extends the research on attendance predictors. Many different types of variables potentially associated with treatment attendance were assessed prospectively, and valid data sources were used to assess the variables of interest. Specifically, administrative billing record data was used to assess treatment attendance; observational coding (as opposed to self-report) was used to assess within-session practice, and established self-report measures were used to assess key clinical and parent/family characteristics. The analytic approach, using random effects multi-level modeling, accounts for the clustered nature of the data and provides a rigorous test of the relationship between the wide array of predictor variables and the attendance outcome. The sample includes a relatively large and diverse group of patients and providers who are generally representative of other samples from community-based clinical settings (Garland, Brookman-Frazee et al., 2010). However, the extent to which the findings generalize to other types of service systems in other geographic areas, and/or other clinical populations (e.g., children with different presenting problems or in privately funded service systems), is unknown.

The sample does not represent all children/families seeking care in that families participating in the study had progressed to scheduling an appointment in a clinic and participating in an initial intake interview for the research project. Research suggests that only about half of the families who contact a clinic for services actually begin treatment (Pellerin et al., 2010). Our sample is biased in favor of families who demonstrated initial commitment to attend treatment and consented to participate in a research baseline interview, which may explain the relatively high average number of sessions attended overall. Although the study sample was moderately large, it was not large enough to support more detailed analyses of potential interaction effects, and/or effects specific to different clinics. In addition, given that this was an observational study of treatment attendance in usual care, there was great variation in duration of treatment attendance, which provides sufficient variability in the outcome of interest but also presents some methodological challenges such as natural attrition over time. Given this challenge, we used within-session practice data for the first four months of treatment only because all participants were in treatment at that time and there was gradual attrition from treatment over the 16 month study period. Finally, it is important to note that the set of evidence-based practice elements were based on treatment of disruptive behavior problems, not other diagnoses. All child participants had disruptive behavior problems, but as reflected in clinician assigned diagnoses (Table 1), they had diverse primary diagnoses. In a related study (Brookman-Frazee et al., 2010) we report that the child’s primary diagnosis is not associated with observed intensity of EB elements for disruptive behavior disorders. However, it is possible that the current findings would differ if the set of EB elements were defined for other presenting problems.

Implications and Future Directions

Most of the existing literature on factors associated with client attendance for community mental health care visits focuses on client demographic and/or clinical characteristics predetermined at service-entry. Clinicians and clinical administrators may be somewhat encouraged by our findings indicating that experiences within treatment sessions are associated with attendance, even while accounting for some of the significant demographic factors at service-entry. Specifically, parents’ experience in treatment, as reflected in their report of the quality of the therapeutic alliance with the clinician, predicted attendance in an expected direction. Parents who endorsed a more positive therapeutic alliance with the clinician within the first four months of treatment brought their children in for more treatment visits over 16 months. In addition, when treatment visits within the first four months included more intensive delivery of practice techniques consistent with EB treatment to children there was a trend for greater attendance within 16 months, even while accounting for the effects of other significant predictors such as alliance, parental education and funding source. These findings may help to allay some of the concerns that community-based therapists’ have expressed about EB practices, namely that they may interfere with alliance building and contribute to poorer retention of families in community-based treatment (Nelson, Steele, & Mize, 2006). Also, parents’ therapeutic alliance ratings were positively associated with intensity of EB element delivery to children in the initial bivariate analyses. Taken together, these results may be helpful in addressing therapists’ ambivalent attitudes about EB practice implementation among some stakeholders in community care settings (Essock et al., 2003), suggesting that delivery of practice elements most consistent with EB practices could potentially enhance treatment engagement, which is highly valued by community clinicians.

Several different composite scores representing EB practice elements and other practice elements delivered to children and parents were assessed, but only the intensity of EB techniques directed to children resulted in a trend finding. These elements include active skill-building strategies such as modeling of skills, behavioral rehearsal of skills, assignment and review of homework tasks, and positive reinforcement of skill building. On average, community-based therapists utilize these types of strategies with low intensity (Garland et al., 2010). Interventions to raise the intensity of these elements could potentially improve engagement (i.e., attendance) and also potentially improve the clinical effectiveness of treatment (Garland, Bickman, Chorpita, 2010).

The fact that the intensity of “other” elements was not significantly associated with attendance provides a control for the observed EB practice trend. It is also noteworthy that the intensity of EB elements directed to parents was not associated with attendance, but parents’ perceptions of the quality of the therapeutic alliance were. This reflects some of the complexity in child/family treatment where the “client” may interchangeably be identified as the focal child or children, the parent(s)/caregiver(s), or the family as a whole. Clinicians are often challenged with engaging and intervening with all these variations in the course of treatment.

Our findings regarding the other predictors of treatment attendance highlight important risk factors for poor attendance. Specifically, lower levels of parent education and Medicaid funding for services were strong predictors of fewer sessions attended. These results suggest that individual providers and agencies may need to exert more effort to engage these families in treatment. Successful engagement interventions rely upon addressing both objective and subjective barriers to engagement with high risk families (Gopalan, et al., 2010). For example, Mary McKay’s engagement intervention focuses on clarifying the need for services, increasing caregiver investment and efficacy, and identifying negative attitudes about previous experiences with mental health services in addition to encouraging problem solving around concrete barriers such as child care and transportation (McKay, Stoewe, McCadam, & Gonzales, 1998). Devoting resources to reinforcing engagement up front may be cost-effective for families that are at highest risk for terminating treatment early.

Our study examined only one specific indicator of treatment attendance, namely total number of sessions attended. Future research is needed to assess whether EB practice elements in general, or some specific subset of EB elements specifically, are associated with other types of treatment engagement, such as consistency in attendance over time, adherence to treatment recommendations, parent participation in child treatment, and premature drop-out. Future research is also needed to address the most important questions about how variability in treatment engagement is associated with treatment outcomes. Previous research testing a dose-effect relationship for number of treatment sessions attended associated with differential clinical outcomes in children’s routine mental health care has yielded mixed findings (Andrade, et al., 2000; Angold, et al., 2000; Shapiro et al., 1997; Warren et al, 2010). More detailed investigations are warranted to assess how both the quality and quantity (and possibly interaction thereof) of practice elements delivered are associated with differential outcomes. We hope that our initial exploration of practice elements associated with attendance moves the field a small step in that direction.

Table 3.

Correlation Matrix (Appendix)

Key Variables	n	Correlation Matrix
		# of Visits	2	3	4	5	6	7	8	9	10	11	12	13	14
1. # of Visits	181
2. Child EBP Content	181	.18*
3. Child EBP Technique	181	.19*	.61***
4. Parent EBP Content	165	.06	.25***	.25***
5. Parent EBP Technique	165	−.01	.20**	.44***	.67***
6. Child Other Content	181	.15	.49***	.42***	.31***	.21**
7. Child Other Technique	181	.11	.57***	.46***	.17*	.07	.50***
8. Parent Other Content	165	−.02	.05	.13	.48***	.48***	.28***	.12
9. Parent Other Technique	165	.00	.09	.17*	.48***	.41***	.24**	.28***	.67***
10. Parent Educ.: Some college +	176	.25***	.05	.06	−.13	−.04	−.09	.14	−.12	−.04
11. Funding source: School-based	172	.33***	.19*	.11	.06	.05	.13	.14	−.03	−.13	.19*
12. Parent Alliance (TASC)	151	.28***	.19*	.25**	.05	.10	.11	.03	.03	.02	.04	−.07
13. Therapist Months Practiced	78	−.15*	−.06	−.12**	−.05	−.10	.01	−.04	−.10	−.11	.02	.08	−.23**
14. Gender: Female	181	−.13	.05	.03	.07	−.03	.07	.03	−.13	−.03	−.11	−.13	.03	.02
15. Comorbidity	181	.16*	.23*	.18*	−.02	.05	.15*	.17*	−.05	.04	.09	.21**	.02	.00	−.06

^*= p <.05;

^**=p ≤ .01;

^***= p ≤ .001; 2-tailed significance.

Note: All correlations are Pearson correlations, with the exception of variables associated with parent education, funding source, gender, and comorbidity (Spearman’s Rho was calculated for those correlations given the categorical nature of those two variables).