Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2013 May 8.
Published in final edited form as: Psychol Assess. 2012 Oct 22;25(1):244–252. doi: 10.1037/a0030551

Psychometric Properties of the Therapeutic Alliance Scale for Caregivers and Parents

Erin C Accurso a,b, Kristin M Hawley c, Ann F Garland d,e
PMCID: PMC3647370  NIHMSID: NIHMS458924  PMID: 23088205

Abstract

This study examines the psychometric properties of the Therapeutic Alliance Scale for Caregivers and Parents (TASCP) in a sample of 209 caregivers whose children (ages 4–13) presented with disruptive behavior problems to publicly funded outpatient mental health clinics in San Diego County. Information about therapeutic alliance was collected from children, caregivers, and their therapists across the course of therapy (up to 16 months). Results supported the reliability, temporal stability, convergent validity, and discriminant validity of scores on the TASCP. The multilevel factor structure of this new measure was consistent with the parallel child-report version, with two within-level factors and one between-level factor. Furthermore, predictive validity was strong, with stronger caregiver-reported alliance associated with more sessions attended, greater satisfaction with perceived improvement, and less drop-out.

Keywords: alliance, usual care, child psychotherapy, psychometrics

Introduction

A positive working relationship, or therapeutic alliance, is considered an important part of the therapeutic process across multiple theoretical orientations, including humanistic, psychodynamic, interpersonal, and cognitive-behavioral models. Interventions researchers have also posited that a strong therapeutic alliance may be crucial for client motivation to attend sessions and engage in the work of therapy, and for positive client outcomes (e.g., Binder & Strupp, 1997; Brent & Kolko, 1998; Follette, Naugle, & Callaghan, 1996; Horvath & Luborsky, 1993; Raue & Goldfried, 1994; Rogers, 1957; Webster-Stratton & Herbert, 1993). A majority of practicing clinicians also report that the alliance is one of the most important variables influencing therapeutic outcomes (Bickman et al., 2000; Kazdin, Siegal, & Bass, 1990).

Widespread interest in the therapist-client relationship across theoretical orientations led to the development of a pantheoretical definition of the client-therapist alliance. Building upon and refining earlier, orientation-specific conceptualizations of alliance, Bordin (1979) and Luborsky (1976) similarly defined the alliance as consisting of both affective bond between client and therapist (i.e., the personal relationship; Hougaard, 1994) and client-therapist agreement and collaboration on therapeutic tasks, goals, methods, and intensity (i.e., the collaborative relationship; Hougaard, 1994). Factor analysis of the most common measures of adult alliance has confirmed the presence of these components (i.e., affective bond and agreement on tasks and goals; Andusyna, Tang, DeRubeis, & Luborsky, 2001; Hatcher & Barends, 1996; Hatcher & Gillaspy, 2006; Munder, Wilmers, Leonhart, Linster, & Barth, 2010). With this integrative conceptualization, therapeutic alliance has been increasingly researched, particularly in the adult area. Two independent meta-analyses found the alliance to be a fairly consistent predictor of treatment outcome for various diagnoses (e.g., depression, anxiety) and treatment orientations (e.g., psychodynamic, cognitive-behavioral), with average effect sizes from r=.22–.26 (Horvath & Symonds, 1991; Martin, Garske & Davis, 2000).

The alliance in child therapy has also received empirical attention, particularly in the past decade, though child alliance research still lags well behind that of adult alliance. In a meta-analysis examining associations between child-therapist relationship variables and treatment outcomes, Shirk and Karver (2003) identified 23 studies, showing a mean effect size of r=.20, just below that found with adults. In an updated analysis that focused specifically on child-therapist alliance, 10 studies were identified with an average effect size of r=.21 for the association between child-therapist alliance and treatment outcomes (Karver, Handelsman, Fields, & Bickman, 2006). Finally, the latest meta-analysis of 16 studies found a weighted mean alliance-outcome correlation for child- and caregiver-therapist alliance of r=.22 (Shirk, Karver, & Brown, 2011). However, given that this meta-analysis averaged findings across child- and caregiver-therapist alliance, it was not possible to examine them separately as distinct constructs.

A handful of studies have examined caregiver-therapist alliance, distinct from child-therapist alliance (e.g., Hawley & Garland, 2008; Kazdin & Whitley, 2006; Diamond, Diamond & Liddle, 2000). As noted by Shirk and Karver (2003), the examination of alliance in child and family therapy may be more complex than in adult therapy, in part because it involves both child and caregiver relationships with the therapist. Even in the most child-focused interventions, caregivers are involved at some level throughout treatment; at the least, caregivers are responsible for getting the child to therapy and for structuring the family environment in ways conducive to the therapy recommendations (Hawley & Weisz, 2005). Furthermore, child-therapist and caregiver-therapist alliances may be driven by different factors and associated with different aspects of therapy process and outcome. Indeed, some data bears this out. For example, Hawley and Weisz (2005) found that caregiver-therapist alliance was associated with fewer cancellations, no-shows, and drop-outs, whereas child-therapist alliance was associated with greater symptom improvement. Thus, it seems that both child-therapist and caregiver-therapist alliance are deserving of clinical and empirical attention in child and family therapy.

In order to examine the alliance in child and family therapy, and have some confidence in the resultant findings, it is imperative to have reliable and valid measures of both child-therapist and caregiver-therapist alliance. Much of the research on child-therapist alliance has employed the Therapeutic Alliance Scale for Children (TASC; Shirk & Saiz, 1992). The TASC is unique among alliance measures in that it was designed specifically for use with children and adolescents, employs short, easy to understand items, and covers both positive and negative aspects of the alliance. The original TASC scores demonstrated good reliability (DeVet, Kim, Charlot-Swilley, & Ireys, 2003; Shirk & Saiz, 1992), as did a parallel caregiver-therapist version (see Hawley & Weisz, 2005), although its 7 items may have overemphasized the affective bond aspect of the alliance. A revised 12 item version was developed to more fully address both the affective bond and the mutual collaboration aspects of alliance (TASC-r; Shirk, Gudmundsen, Kaplinski, & McMakin, 2008; Shirk & Saiz, 1992). The TASC-r scores have shown adequate reliability and validity in a study by Creed and Kendall (2005). Similar to what was done for the original TASC, a caregiver report version for the 12 item TASC-r was developed to permit examination of the caregiver-therapist alliance. The goals of this study are to examine the psychometric properties of this newly developed Therapeutic Alliance Scale for Caregivers and Parents (TASCP), including examination of reliability, temporal stability, factor structure, and predictive validity as assessed by its relationship with treatment attendance and client satisfaction.

Methods

This study utilized data from a larger examination of child psychotherapy processes and outcomes in a representative sample of children with disruptive behavior problems receiving outpatient treatment-as-usual at one of six community mental health clinics in San Diego County (Garland, Brookman-Frazee, et al., 2010; Garland, Hurlburt, Brookman-Frazee, Taylor, & Accurso, 2009; Garland, Hurlburt, & Hawley, 2006). Written informed consent/assent was obtained from therapists, caregivers, and children over age 8, and verbal assent was obtained from younger children. All participants were financially compensated for their participation in research interviews. All procedures were approved by affiliated Institutional Review Boards.

Participants

Clinic administrative staff screened all eligible new patients during the initial call to the clinic for services, 90 percent of whom agreed to be contacted by research staff for recruitment. Of the 550 who agreed to be contacted and met the inclusion criteria, 55% (n = 292) did not engage in treatment at the clinics, leaving 258 potential participant families who were actively recruited into this study. Eighty- ve percent (n = 218) of these families agreed to participate in the study. Of these, 209 engaged in at least one psychotherapy session. Due to HIPAA restrictions, data cannot be collected on non-participants, preventing analysis of how non-participants may have differed from participants.

Children

The sample included 209 children (ages 4–13) referred to publicly funded outpatient mental health services in San Diego County for disruptive, oppositional, defiant, or conduct related problems. Inclusion criteria for child participants were (a) presenting problems included a disruptive behavior problem (including aggression, de ance, delinquency, oppositional behavior), (b) age between 4 and 13 years at the time of recruitment, (c) primary language for child and caregiver was English or Spanish, and (d) child was entering a new episode of psychotherapy (de ned as no therapy for previous 3 months) with a participating therapist. As assessed through caregiver report, children with mental retardation (IQ < 70), those with significant organic brain damage, and those with major medical problems were excluded from the study because these factors may have been associated with unique treatment characteristics. Although children needed to present with a disruptive behavior problem to meet inclusion criteria, children were included regardless of primary or comorbid diagnoses such that the sample represented the diverse range of children presenting with disruptive behavior problems to community-based outpatient care. The mean age of participants was 9 years (SD = 2.7); 68% were boys. Race/ethnicity was fairly diverse, with 45% Caucasians, 28% Latinos, 9% African Americans, and 18% Mixed/Other.

Caregivers

Primary caregivers (n = 209) were predominantly women (94%), and included biological mothers (79%), grandmothers (9%), biological fathers (4%), foster mothers (3%), aunts (2%), and others (3%). The mean age of caregivers was 40 years (SD = 10.3) with a median household income of $25,000 and mean income of $36,452 (SD = 30,525). Caregivers were married or living with a partner (44%), divorced (33%), never married and single (20%), or widowed (3%). There was diverse race/ethnic representation, with 53% Caucasians, 29% Latinos, 10% African Americans, and 8% Mixed/Other. Spanish-speakers comprise 16% of the caregiver sample for this study.

Therapists

This sample included 92 therapists practicing in six community-based clinics in San Diego County. Initially, therapists were randomly selected for recruitment into the study from clinic lists of active therapists. Recruitment continued until cells were filled to reflect the distribution of therapists by mental health discipline. Subsequently, all new staff and trainees who worked at least half time in the clinics were recruited into the study. Of the 163 therapists recruited, 131 (80%) agreed to participate, but only 92 initiated psychotherapy with a child participant in the study. At study entry, 59% of participating therapists were trainees and 41% were staff; therapists had a mean of three years of psychotherapy experience (range 0–25 years). Most therapists were Caucasian (68%) and female (84%). Other race/ethnicities included Latino (9%), African American (3%), and Mixed/Other (20%). Regarding mental health discipline, 60% endorsed Marital & Family Therapy, 23% Psychology, and 17% Social Work. With respect to primary theoretical orientation, 34% identified with Family Systems, 26% Eclectic, 25% Cognitive Behavioral, 4% Psychodynamic, 4% Humanistic, 3% Behavioral, and 3% Other. Therapists were limited to a maximum of eight participating families; most had one or two participating families.

Attrition

Given the naturalistic nature of this study, child and caregiver sample sizes decreased across time points due to therapy termination. All 209 families were active in treatment within the first 4 months, 141 (67.5%) were still active within 5–8 months, 100 (47.8%) within 9–12 months, and 61 (29.2%) within 13–16 months. Within each time point, families attended a mean of 9.2 (sd = 4.8, n=209) sessions from baseline to 4 months, 8.8 (sd = 4.3, n=141) sessions from 4 months to 8 months, 7.7 (sd = 4.2, n=100) sessions from 8 months to 12 months, and 8.2 (sd = 4.1, n=61) sessions from 12 months to 16 months.

Due to age restrictions in child data collection (i.e., only children 9 years or older completed interviews), 58.4% (122) of children who were active in treatment were eligible to participate in 4-month follow-up interviews, 58.2% (82) at 8 months, 58.0% (58) at 12 months, and 59.0% (36) at 16 months. The research team was not always successful in reaching families and/or therapists eligible to participate in follow-up interviews, accounting for additional missing data. Of those children eligible to participate in interviews and active in treatment, information about therapeutic alliance was collected from 73.8% (90) at 4 months, 69.5% (57) at 8 months, 69.0% (40) at 12 months, and 69.4% (25) at 16 months. Information about caregiver alliance was collected from 80.9% (169) of caregivers active in treatment at 4 months, 80.9% (114) at 8 months, 79.0% (79) at 12 months, and 77.1% (47) at 16 months. Caregiver alliance data were collected from 68.4% (143) of therapists with active families at 4 months, 62.4% (88) at 8 months, 58.0% (58) at 12 months, and 60.7% (37) at 16 months. Information about child alliance was collected from 67.9% (142) of therapists with active families at 4 months, 63.8% (90) at 8 months, 58.0% (58) at 12 months, and 60.7% (37) at 16 months.

Procedures

Data were collected from multiple sources including 1) telephone follow-up interviews with children (age 9 and over) and caregivers, 2) facsimile communication with therapists, and 3) abstraction from administrative data (billing records) for information about service attendance for the entire 16 month study period. Treatment session attendance intensity was calculated by dividing the total number of sessions attended by the number of weeks in which families were considered to be “active” in the treatment episode, providing an estimate of service visit intensity. Follow-up phone interviews with families were conducted in their preferred language (English or Spanish) at 4, 8, 12, and 16 months. Certified translators used established forward translation and back-translation methods in order to create Spanish versions of the measures. Information about therapeutic alliance and satisfaction was collected at each interview, provided the family had been active in therapy during the 4 months preceding the interview time point. Therapists also reported on alliance with both the child and the caregiver. If the family was no longer in therapy, therapists reported on whether the family terminated treatment prematurely (i.e., dropped-out) and their agreement with terminating therapy.

Measures

Therapeutic Alliance Scale for Children, Revised (TASC-r)

The TASC-r (Shirk, Karver, & Brown, 2011; Shirk & Saiz, 1992) was designed specifically to measure child-therapist alliance, with parallel forms for child report and therapist report. In accordance with Bordin’s (1979) conceptualization of alliance, the TASC-r distinguishes between (a) the affective bond (e.g., the extent to which the therapist is an ally) and (b) client-therapist collaboration on therapeutic tasks and goals (e.g., extent to which it is difficult to work with therapist on solving problems). Twelve items are rated on a Likert scale from 1 (not true) to 4 (very much true). The TASC-r scores have demonstrated good reliability and validity in previous studies (Creed & Kendall, 2005; DeVet, Kim, Charlot-Swilley, & Ireys, 2003).

Therapeutic Alliance Scale for Caregivers and Parents (TASCP)

Two parallel caregiver-therapist alliance forms of the 12-item TASC-r were created for the present investigation—one for caregiver report and one for therapist report of the caregiver-therapist alliance. Scores from a previous caregiver report version, based on the 7-item TASC, demonstrated good internal consistency (Cronbach’s alpha = .81) and one week test-retest reliability (correlation coefficient = .82; Hawley & Weisz, 2005). The 7-item versions of the child-therapist and caregiver-therapist alliance measure scores also showed good convergent validity, with moderate and expected positive associations with treatment satisfaction (Hawley & Weisz, 2005). In addition, caregiver-therapist alliance was positively associated with treatment attendance (γ = .02) and symptom improvement (γ = .54) in an outpatient setting (Hawley & Weisz, 2005). For the present study, we developed 12 item forms for caregiver report of caregiver-therapist alliance to parallel the 12-item TASC (e.g., changing ‘my therapist’ to ‘my child’s therapist’). Caregivers with a Spanish-speaking preference (n=12) were administered a translated version of the measure. As described above, forward translation and back-translation procedures were used to create a version of the TASCP in Spanish. A parallel therapist report form was also included.

Multidimensional Adolescent Satisfaction Scale (MASS)

The MASS (Garland, Aarons, Saltzman, & Kruse, 2000; Garland, Saltzman, & Aarons, 2000) is a 21-item self-report instrument that measures consumer satisfaction with psychotherapy. This measure has been adapted for caregivers to report on satisfaction with youth services. Although the measure was originally developed to assess satisfaction with services for youths ages 11 and over, it has demonstrated adequate psychometrics with caregivers of children as young as age five (Stadnick, Drahota, & Brookman-Frazee, 2012). Caregivers completed the Perceived Effectiveness subscale, which measures satisfaction with the effectiveness of services, was administered to caregivers at four months post service entry. These scores have good internal consistency, strong test-retest reliability, as well as convergent, divergent, and predictive validity with publicly-funded outpatient treatment youth samples (Garland, Aarons et al., 2000; Garland, Saltzman et al., 2000). The internal reliability on this subscale for the caregiver sample used here was strong (Cronbach’s alpha = .90).

Parent Service Use Questionnaire for PRAC (PSUQ)

The PSUQ (Garland, 2003) was developed for the study and administered following therapy termination in order to obtain additional information about reasons for termination and the context of termination. Several items were used from this questionnaire, including the caregiver’s desire for therapy to end (no, somewhat, yes) and the main reasons for termination (e.g., financial reasons, practical reasons [transportation, work conflict], moved out of the area, disliking the therapist, services not relevant or helpful, services no longer needed due to improvement, decision to try other types of services).

Data analyses

Internal consistency

Internal consistency was examined across all time points using Cronbach’s alphas.

Temporal Stability

Autoregressive regression models were tested to explore the temporal stability of caregiver-reported therapeutic alliance from 4 to 16 months, with good model fit indicative of good temporal stability. The relations in these models were tested using the statistical modeling program Mplus (Muthén & Muthén, 1998–2007) to account for the nested data structure (i.e., sessions nested within caregiver, caregivers nested within therapist, therapists nested within clinics). Traditionally, the likelihood ratio chi-square test has been used to determine overall model fit. However, this test statistic has been deemed unsatisfactory for numerous reasons, including the heavy influence of sample size on this statistic (Hoyle, 2000; Tanaka, 1993). Therefore, likelihood ratio chi-square tests are reported for statistical completeness, but model fit will be determined by examining three descriptive fit indices recommended by Bentler (2007): (1) the Comparative Fit Index (CFI; Bentler, 1990), with values greater than .95 indicative of a well-fitting model and values greater than .90 indicative of a plausible model; (2) the root mean square error of approximation (RMSEA; Steiger, 1990), with values less than .05 indicative of a well-fitting model and values less than .08 indicative of a plausible model; and (3) the standardized root mean square residual (SRMR; Hu & Bentler, 1999), with values less than .05 indicative of a well-fitting model and values less than .08 indicative of a plausible model.

Convergent/divergent validity

Convergent and divergent validity were assessed through intraclass correlations (ICCs) between child, caregiver, and therapist reports of alliance at 4 months.

Predictive validity

Multilevel random intercept models were used to examine outcomes (e.g., visit attendance, treatment drop-out, client satisfaction, etc.) predicted by caregiver-reported alliance using SuperMix Version 1.1 (Hedeker, Gibbons, du Toit, & Patterson, 2008). These models were chosen because they account for the nested data structure (caregivers within therapists). The proportional reduction in error variance (PRE) was calculated and provides a measure of the proportion of variance from 0 to 1 explained by caregiver-reported alliance in the outcome, or the extent to which alliance reduced the error associated with predicting the value of the outcome (Raudenbush & Bryk, 2002). Given that some therapists saw multiple families who may be more similar to each other than to families of other therapists, ICCs were first calculated for each outcome to assess the percent of variability in each outcome that is attributable to the therapist level and determine whether the therapist level needed to be accounted for in subsequent analyses.

Factor analysis

Single-level factor analysis has traditionally been used to examine cross-sectional data with the individual as the unit of analysis. However, when longitudinal data are available, single-level factor analysis is not recommended. Limitations of this approach include treating individual observations as independent (disaggregation approach) by factor analyzing the total variance/covariance matrix, therefore ignoring between-individual (co)variation across time. On the other hand, factor analyzing a variance/covariance matrix in which variables have been summed or averaged across time (aggregation approach) ignores within-individual variability across time. Ignoring variability at both levels of a hierarchical data structure (e.g., individuals nested within time) can result in biased parameter estimates, including factor loadings (Kaplan, Kim, & Kim, 2009). In addition, single-level factor analysis does not allow for the possibility that factor structures can differ at levels of the nested data structure (Kaplan et al., 2009; Zimprich & Martin, 2009).

Multilevel factor analysis overcomes these limitations and enables the simultaneous modeling of between-person and within-person variation. By using variables from both levels of a nested data structure, multilevel factor analysis controls for the confound between within-person and between-person variation (Heck & Thomas, 2009) and allows for the development of factor scores at each level of the nested data structure. Furthermore, the aggregation of within-person assessments across time reduces error relative to single assessments and provides a more statistically reliable and powerful measure of the construct of interest.

As outlined by Roesch et al. (2010) and Mertz and Roesch (2011), multilevel exploratory factor analysis examining within level factors (i.e., items within individual caregivers) and between level factors (i.e., total item scores between caregivers) was conducted in Mplus (Muthén & Muthén, 1998–2007). Geomin rotation was used for all models, with up to two factors specified at each level of the nested data structure. At the between-person level, variation represents an individual’s reported alliance relative to others’ reported alliance. At the within-person level, covariance represents an individual’s current reported alliance relative to their average alliance. The likelihood ratio Chi-square (χ2) and degrees of freedom is reported for statistical completeness. However because χ2 tests may be unsatisfactory to determine model fit, descriptive indices (i.e., CFI, RMSEA, and SRMR) were also utilized, as discussed above. If two of the three descriptive indices indicated good fit, the model was determined to be well-fitting. To test for differences between nested models, Chi-square difference tests (Δχ2) were performed between the less and more constrained models. In addition to the variance accounted for by the solution (i.e., Δχ2), variance accounted for by each individual factor and the interpretability of the factors were evaluated to determine the initial plausibility of the factor structure.

Results

Reliability

Internal consistency

Internal consistency of caregiver-reported caregiver-therapist alliance was high across all four time points, with Cronbach’s alphas ranging from .85 to .88. Within the Latino subsample (n=65), twelve caregivers indicated a preference for completing measures in Spanish. Cronbach’s alphas were similar across language versions for Latino caregivers at four months (.89, .90) and eight months (.83, .84). Cronbach’s alphas were also similar across language versions for all caregivers (i.e., Latinos and non-Latinos) at four months (.87, .87) and eight months (.86, .80). Due to naturalistic attrition, Cronbach’s alphas for were not calculated beyond the eight month time point.

Temporal stability

The caregiver and child alliance models fit well statistically (χ2 [1, N = 116] = 0.241, p = .624; χ2 [1, N = 60] = 0.206, p = .650; respectively) and descriptively (CFI = 1.000, RMSEA < .001, SRMR = .017; CFI = 1.000, RMSEA < .001, SRMR = .013; respectively). Caregiver-reported caregiver alliance autoregressive models revealed medium to large statistically significant standardized regression coefficients for first-order paths (β [.374, .720], p < .0001) and explained 22.6% of the variance in alliance at 8 months, 46.5% of the variance in alliance at 12 months, and 68.5% of the variance in alliance at 16 months. Specifically, the regressions of alliance at 8 months (β = .479, z = 5.592, p < .0001) and 12 months (β = .416, z = 4.155, p < .0001) on alliance at 4 months were moderate and statistically significant. The regression of alliance at 12 months on alliance at 8 months was moderate and statistically significant (β = .374, z = 2.803, p = .005), but the regression of alliance at 16 months on alliance at 8 months was not (β = .057, z = 0.317, p = .751). Finally, the regression of alliance at 16 months on alliance at 12 months was large and statistically significant (β = .717, z = 4.209, p < .0001). These results indicate that temporal stability of caregiver-therapist alliance as reported by caregivers is moderate between months 4 through 12, and high between months 12 and 16.

For comparison, temporal stability of child-reported child alliance was also tested. Like caregiver-reported caregiver alliance, standardized regression coefficients for first-order paths were moderate to large and statistically significant (β = [.552, .881], p < .0001). These results indicate that temporal stability of child-therapist alliance as reported by children is high across months 4 through 16.

Validity

Convergent/divergent validity

Caregiver-reported caregiver alliance was strongly and significantly associated with therapist-reported caregiver alliance (r = .67, p < .0001) at four months. Caregiver report of alliance with the therapist was also more strongly associated with therapist report of caregiver-therapist alliance than it was with child report of child-therapist alliance (ICC = .54, p < .0001) or therapist report of child-therapist alliance (ICC = .40, p < .0001) at 4 months. Please see Table 1 for ICCs between other reporters.

Table 1.

Intraclass correlations across reports of caregiver and child alliance.

Caregiver Alliance Child Alliance
Reporter Caregiver Therapist Child Therapist
Caregiver Alliance Caregiver ----
Therapist .674*** ----
Child Alliance Child .535*** .410* ----
Therapist .400** .524*** .497*** ----
Open in a new tab
*

p < .05,

**

p < .001,

***

p < .0001

Predictive validity

Early caregiver-reported alliance (4 months) was positively and significantly associated with total number of sessions attended (B = .62, SE = .18, p < .001; PRE = .23) but not service intensity (i.e., number of sessions attended over number of weeks; B < .01, SE < .01, p = .40). Early caregiver-reported alliance also predicted greater satisfaction with perceived improvement at 8 months (B = .03, SE = .01, p < .005; PRE = .74) and 12 months (B = .04, SE = .01, p < .005; PRE = .74). Poorer early caregiver alliance predicted subsequent caregiver reports of wanting to end therapy (B = .07, SE = .03, p < .05; PRE = .12) and endorsing disliking the therapist as one of the main reasons for termination (B = .05, SE = .02, p < .05; PRE = .11). Furthermore, poorer early caregiver-reported alliance predicted drop-out from therapy (B = .07, SE = .03, p < .01; PRE = .11) and was negatively associated with therapist agreement with termination (B = −.04, SE = .02, p = .09; PRE = .13).

Factor structure

Factor structures with up to two factors at the within level aggregate level (i.e., within level, examining items within individual caregivers) and up to two factors at the between level (i.e., between level, examining total item scores between caregivers) were examined. Descriptive fit for the factor structure with one within-level factor and one between-level factor fit poorly statistically (χ2 [108, N = 473] = 293.80, p < .00001). Descriptive fit for the factor structure with one within-level factor and two between-level factors also fit poorly statistically (χ2 [97, N = 473] = 275.90, p < .00001). Descriptive fit for the factor structure with two within-level factors and one between-level factor fit poorly statistically as well (χ2 [97, N = 473] = 210.70, p < .00001). Descriptive fit for the factor structure with two within-level factors and two between-level factors also fit poorly statistically (χ2 [86, N = 473] = 191.22, p < .00001). A comparison between the models revealed that the model with two within-level factors and one between-level factor was the most parsimonious (Δχ2 [11, N = 473] = 19.48, p = .053). Although the factor structure did not fit well statistically (χ2 [97, N = 473] = 210.7, p < .001), it had acceptable descriptive fit (CFI = .914, RMSEA = .050, SRMR within factors = .059, SRMR between factors = .122).

Most items at the between-level loaded on factor one, which will be referred to as Positive Alliance; factor two will be referred to as Negative Alliance (see Table 2 for the item loadings and Appendix A for a listing of items). Items loading on the Positive Alliance factor (items 1, 3, 4, 5, 9, 10, and 12) largely refer to liking the therapist and using time with the therapist to make changes. Items loading on the Negative Alliance factor (items 5, 7, and 8) include wanting the sessions to end quickly, thinking the therapist spends too much time working on problems, and preferring to do other things than meet with the therapist. Items that cross-load conceptually fit better (items 2 and 11) with the Negative Alliance factor (i.e., finding it hard to work with the therapist and preferring to not work on problems with the therapist).

Table 2.

Item loadings within and between factors for caregiver-reported caregiver alliance and child-reported child alliance.

Item Caregiver Alliance
Child Alliance
Within Between Within Between
1w 2w 1b 1w 2w 1b
1 .65 .26 .83 .64 .13 .91
2 .54 .41 .47 .05 .39 .79
3 .77 .28 .88 .62 .22 .98
4 .66 .15 .97 .71 .16 .95
5 .21 .74 .90 .22 .45 .91
6 .51 .19 .92 .65 .04 .96
7 .01 .36 .01 .06 .48 .75
8 .37 .49 .56 .27 .50 .99
9 .35 .03 .80 .67 .02 .86
10 .77 .22 .69 .74 .26 .90
11 .56 .59 .55 .36 .45 .91
12 .83 .24 .92 .73 .15 .98
Open in a new tab

Note: (w) denotes within-level factors while (b) denotes between level factors. Bolded text denotes an item loading on the corresponding factor.

Factor structures with up to two factors at the within level and up to two factors at the between level were also examined for child-reported child-therapist alliance as a comparison to caregiver-reported caregiver-therapist alliance. Factor structures with two within-level factors fit better than those with one at this level. The model with two within-level factors and one between-level factor was retained due to being the most parsimonious and interpretable, in addition to demonstrating adequate descriptive fit (CFI = .936, RMSEA = .056, SRMR within factors = .061, SRMR between factors = .139). The factor loadings for the child alliance model were fairly comparable to those in the caregiver alliance model (see Table 2). This factor structure indicates that within informant, items on the TASCP load best onto two factors, whereas between informants, items on the TASCP are best represented by a single factor.

Discussion

The Therapeutic Alliance Scale for Caregivers and Parents is one of few measures of caregiver-therapist alliance. These analyses support the psychometric characteristics of this measure and identify interesting potential relations between caregiver alliance and treatment engagement. The TASCP scores demonstrated excellent reliability, with good internal consistency. Temporal stability was moderate, as might be expected for a relational variable that would naturally fluctuate over time. Convergent and divergent validity were also established in that caregiver-reported caregiver-therapist alliance was correlated more strongly with therapist-report of caregiver alliance than child- or therapist-reported child alliance.

Furthermore, the factor structure of the TASCP was examined, with the majority of fit indices within or close to the cutoff values indicating generally adequate model fit for two within-level factors and one between-level factor. This factor structure was well-aligned with that of the TASC-r measure of child-therapist alliance in this sample. However, model fit was not ideal and was negatively affected by two items on the scale that cross-loaded on both within-level factors. Despite a couple of items that weaken the factor structure of this measure, TASCP total scores were associated with important caregiver outcomes, including session attendance and caregiver satisfaction with child’s perceived improvement. As in past research with an earlier version of the caregiver report TASC (Hawley & Weisz, 2005), poor alliance on the TASCP was also associated with poor attendance and with drop-out from therapy. Indeed, it accounted for approximately 20% of the variance in total number of sessions attended, approximately 10% of the variance around termination, and approximately 75% of the variance in caregiver satisfaction with therapy.

Given the significant role that caregivers play in child therapy, reliable and valid measurement of caregiver alliance is important. Research has primarily focused on child-therapist alliance, with much less attention to caregiver-therapist alliance (Shirk et al., 2011). Although moderately correlated, our findings are consistent with prior evidence which suggests that children and caregivers form distinct relationships with the therapist, and that these relationships are differentially related to outcomes (e.g., Hawley & Weisz, 2005; Hawley & Garland, 2008). While child and caregiver alliance share some commonalities, the two are not interchangeable. Meta-analysis indicates that child alliance may account for more variance in outcomes than caregiver alliance (r=.21 versus r=.11; Karver et al., 2006). However, in individual studies, the findings are mixed.

For example, among families receiving treatment-as-usual mental health services, Hawley and Weisz (2005) found that caregiver-therapist alliance was significantly associated with fewer cancellations and no-shows and greater therapist concurrence with termination decision whereas child-therapist alliance was not. In contrast, child-therapist alliance was significantly associated with symptom improvement while caregiver-therapist alliance was not. For children receiving evidence-based treatment for disruptive behavior, symptom change was also more associated with child-therapist than caregiver-therapist alliance (Kazdin, Marciano, & Whitley, 2005). However, the opposite was true for internalizing children in a similar context (McLeod & Weisz, 2005). In substance-abusing adolescents receiving family therapy, adolescent-therapist alliance was more associated with decreased drug use than was caregiver-therapist alliance in one study, while the opposite was true in another study (Shelef, Diamond, Diamond, & Liddle, 2005; Hogue et al., 2006; respectively). Given the distinct (albeit inconsistent) correlates of child and caregiver alliance, both deserve further empirical attention to disentangle their associations with various outcomes.

To our knowledge, this is the first study to date of therapeutic alliance in usual care child psychotherapy examining caregiver alliance across time. Its examination of therapeutic alliance from multiple perspectives (i.e., caregiver, child, and therapist) is fairly unique in this literature. Furthermore, assessing alliance at multiple time points allowed for more thorough examination of the psychometric properties of caregiver alliance, although there was some missing data across time-points. Finally, this sample of children with disruptive behavior problems is representative of children seen in community-based mental health settings (Foster, Kelsch, Kamradt, Sosna & Yang, 2001; Rosenblatt & Rosenblatt, 2000). Despite its strengths, this study has several limitations. First, long-term follow-up with four-month intervals did not allow for examination of changes in alliance early in treatment. Also, caregivers’ perceptions of the alliance may differ by child diagnosis and associated treatment modality/orientation, which could not be examined in this study. Future research should examine client, therapist, and treatment characteristics associated with stronger alliance that may help to guide intervention. In addition, efforts should be made to examine child and caregiver alliances from multiple perspectives and across multiple types of service settings to better understand the complexities of these relationships. Finally, future studies including children with a wider range of problems are essential in order to better understand whether diagnosis might moderate the relation between alliance and outcome.

A necessary precursor for such work is a reliable and valid way to measure caregiver-therapist alliance. The TASCP is a measure with reliable test scores whose interpretation indicated good convergent validity, and discriminant validity, with a factor structure matching that of its parallel child-report version. Furthermore, results were supportive of the predictive validity of scores on this measure, with stronger caregiver-reported alliance associated with more sessions attended, greater satisfaction with perceived improvement, and less drop-out. The caregiver-therapist relationship is particularly important in caregiver and family-directed therapies but also play an important role in child-oriented therapy. At the most basic level, caregiver-therapist alliance has been associated with therapy retention (Hawley & Weisz, 2005). Therapy retention is a necessary condition for therapeutic intervention to be delivered and is of great importance to providers and administrators dealing with the high costs of therapy dropouts. Given the importance of the caregiver’s role, caregiver alliance needs to be examined more often and in the context of child and family psychotherapy—assessing caregiver alliance using a measure intended to assess adult alliance in individual therapy may not accurately reflect this construct.

Appendix A

Therapeutic Alliance Scales for Children

  1. I like spending time with my therapist.

  2. I find it hard to work with my therapist on solving problems in my life.

  3. I feel like my therapist is on my side and tries to help me.

  4. I work with my therapist on solving my problems.

  5. When I’m with my therapist, I want the sessions to end quickly.

  6. I look forward to meeting with my therapist.

  7. I feel like my therapist spends too much time working on my problems.

  8. I’d rather do other things than meet with my therapist.

  9. I use my time with my therapist to make changes in my life.

  10. I like my therapist.

  11. I would rather not work on my problems with my therapist.

  12. I think my therapist and I work well together on dealing with my problems.

Therapeutic Alliance Scales for Caregivers and Parents

  1. I like spending time with my child’s therapist.

  2. I find it hard to work with my child’s therapist on solving problems in our lives.

  3. I feel like my child’s therapist is on my side and tries to help me.

  4. I work with my child’s therapist on solving our problems.

  5. When I’m with my child’s therapist, I want the sessions to end quickly.

  6. I look forward to meeting with my child’s therapist.

  7. I feel like my child’s therapist spends too much time working on our problems.

  8. I’d rather do other things than meet with my child’s therapist.

  9. I use my time with my child’s therapist to make changes in our lives.

  10. I like my child’s therapist.

  11. I would rather not work on our problems with my child’s therapist.

  12. I think my child’s therapist and I work well together on dealing with our problems.

References

  1. Bentler PM. On tests and indices for evaluating structural models. Personality and Individual Differences. 2007;45:825–829. [Google Scholar]
  2. Bentler PM. Comparative fit indexes in structural models. Psychological Bulletin. 1990;107:238–246. doi: 10.1037/0033-2909.107.2.238. [DOI] [PubMed] [Google Scholar]
  3. Bickman L, Rosof-Williams J, Salzer MS, Summerfelt T, Noser K, Wilson SJ, Karver MS. What information do clinicians value for monitoring client progress and outcomes? Professional Psychology: Research and Practice. 2000;31:70–74. [Google Scholar]
  4. Binder JL, Strupp HH. Supervision of psychodynamic therapies. In: Watkins CE Jr, editor. Handbook of psychotherapy supervision. New York: Wiley; 1997. pp. 44–62. [Google Scholar]
  5. Bordin E. The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy: Theory, Research, and Practice. 1979;16:252–260. [Google Scholar]
  6. Brent DA, Kolko DJ. Psychotherapy: Definitions, mechanisms of action, and relationship to etiological models. Journal of Abnormal Child Psychology. 1998;26:17–25. doi: 10.1023/a:1022678622119. [DOI] [PubMed] [Google Scholar]
  7. Creed TA, Kendall PC. Therapist alliance-building behavior within a cognitive-behavioral treatment for anxiety in youth. Journal of Consulting and Clinical Psychology. 2005;73:498–505. doi: 10.1037/0022-006X.73.3.498. [DOI] [PubMed] [Google Scholar]
  8. DeVet KA, Kim YJ, Charlot-Swilley D, Ireys HT. The therapeutic relationship in child therapy: perspectives of children and mothers. Journal of Clinical Child and Adolescent Psychology. 2003;32:277–283. doi: 10.1207/S15374424JCCP3202_13. [DOI] [PubMed] [Google Scholar]
  9. Diamond GM, Diamond GS, Liddle HA. The therapist-parent alliance in family-based therapy for adolescents. Journal of Clinical Psychology. 2000;56:1037–1050. doi: 10.1002/1097-4679(200008)56:8<1037::AID-JCLP4>3.0.CO;2-4. [DOI] [PubMed] [Google Scholar]
  10. Follette WC, Naugle AE, Callaghan GM. A radical behavioral understanding of the therapeutic relationship in effecting change. Behavior Therapy. 1996;27:623–641. [Google Scholar]
  11. Garland AF. Parent Service Use Questionnaire for PRAC. San Diego, CA: University of California, San Diego; 2003. [Google Scholar]
  12. Garland A, Aarons G, Saltzman M, Kruse M. Correlates of adolescents’ satisfaction with mental health Services. Mental Health Services Research. 2000;2:127–39. doi: 10.1023/a:1010137725958. [DOI] [PubMed] [Google Scholar]
  13. Garland AF, Brookman-Frazee L, Hurlburt MS, Accurso EC, Zoffness R, Haine RA, Ganger W. Characterizing community-based psychotherapy for children with disruptive behavior problems. Psychiatric Services. 2010;61:788–795. doi: 10.1176/appi.ps.61.8.788. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Garland AF, Hurlburt MS, Brookman-Frazee L, Taylor RM, Accurso EC. Methodological challenges of characterizing usual care psychotherapeutic practice. Administration and Policy in Mental Health and Mental Health Services Research. 2010;37:208–220. doi: 10.1007/s10488-009-0237-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Garland AF, Hurlburt MS, Hawley KM. Examining psychotherapy processes in a services research context. Clinical Psychology: Science and Practice. 2006;13:30–46. [Google Scholar]
  16. Garland A, Saltzman M, Aarons G. Adolescent satisfaction with mental health services: Development of a multidimensional scale. Evaluation and Program Planning. 2000;23:165–175. [Google Scholar]
  17. Hatcher RL, Barends AW. Patients’ view of the alliance in psychotherapy: Exploratory factor analysis of three alliance measures. Journal of Consulting and Clinical Psychology. 1996;64:1326–1336. doi: 10.1037//0022-006x.64.6.1326. [DOI] [PubMed] [Google Scholar]
  18. Hatcher RL, Gillapsy JA. Development and validation of a revised short version of the Working Alliance Inventory. Psychotherapy Research. 2006;16:12–25. [Google Scholar]
  19. Hawley KM, Garland AF. Working alliance in adolescent outpatient therapy: Youth, parent and therapist reports and associations with therapy outcomes. Child and Youth Care Forum. 2008;37:59–74. [Google Scholar]
  20. Hawley KM, Weisz JR. Youth versus parent working alliance in usual clinical care: Distinctive associations with retention, satisfaction and treatment outcome. Journal of Clinical Child and Adolescent Psychology. 2005;34:117–128. doi: 10.1207/s15374424jccp3401_11. [DOI] [PubMed] [Google Scholar]
  21. Heck RH, Thomas SL. An introduction to multilevel modeling techniques. 2. New York, NY: Routledge; 2009. [Google Scholar]
  22. Hedeker D, Gibbons RD, Du Toit SHC, Patterson D. SuperMix: a program for mixed-effects regression models. Chicago, IL: Scientific Software International; 2008. [Google Scholar]
  23. Hogue A, Dauber S, Stambaugh LF, Cecero JJ, Liddle HA. Early therapeutic alliance and treatment outcome in individual and family therapy for adolescent behavior problems. Journal of Consulting and Clinical Psychology. 2006;74:121–129. doi: 10.1037/0022-006X.74.1.121. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Horvath AO, Luborsky L. The role of the therapeutic alliance in psychotherapy. Journal of Consulting and Clinical Psychology. 1993;61:561–573. doi: 10.1037//0022-006x.61.4.561. [DOI] [PubMed] [Google Scholar]
  25. Horvath AO, Symonds BD. Relation between working alliance and outcome in psychotherapy: A meta-analysis. Journal of Counseling Psychology. 1991;38:139–149. [Google Scholar]
  26. Hougaard E. The therapeutic alliance—A conceptual analysis. Scandanavian Journal of Psychology. 1994;35:67–85. doi: 10.1111/j.1467-9450.1994.tb00934.x. [DOI] [PubMed] [Google Scholar]
  27. Hoyle RH. Confirmatory factor analysis. In: Tinsley HEA, Brown SD, editors. Handbook of Applied Multivariate Statistics and Mathematical Modeling. San Diego, CA: Academic; 2000. pp. 465–497. [Google Scholar]
  28. Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling. 1999;6:1–55. [Google Scholar]
  29. Kaplan D, Kim J-S, Kim S-Y. Multilevel latent variable modeling: Current research and recent developments. In: Millsap RE, Maydeu-Olivares A, editors. The SAGE handbook of quantitative methods in psychology. Thousand Oaks, CA: SAGE; 2009. pp. 592–612. [Google Scholar]
  30. Karver MS, Handelsman JB, Fields S, Bickman L. Meta-analysis of therapeutic relationship variables in youth and family therapy: The evidence for different relationship variables in the child and adolescent treatment outcome literature. Clinical Psychology Review. 2006;26:50–65. doi: 10.1016/j.cpr.2005.09.001. [DOI] [PubMed] [Google Scholar]
  31. Kazdin AE, Siegel TC, Bass D. Drawing on clinical practice to inform research on child and adolescent psychotherapy: Survey of practitioners. Professional Psychology: Research and Practice. 1990;21:189–198. [Google Scholar]
  32. Kazdin AE, Marciano PL, Whitley M. The therapeutic alliance in cognitive-behavioral treatment of children referred for oppositional, aggressive, and antisocial behavior. Journal of Consulting and Clinical Psychology. 2005;73:726–730. doi: 10.1037/0022-006X.73.4.726. [DOI] [PubMed] [Google Scholar]
  33. Kazdin AE, Whitley M. Child–therapist and parent–therapist alliance and therapeutic change in the treatment of children referred for oppositional, aggressive, and antisocial behavior. Journal of Consulting and Clinical Psychology. 2006;74:346–355. [Google Scholar]
  34. Luborsky . Helping alliances in psychotherapy. In: Cleghhorn JL, editor. Successful psychotherapy. New York: Brunner/Mazel; 1976. pp. 92–116. [Google Scholar]
  35. Martin DJ, Garske JP, Davis MK. Relation of the therapeutic alliance with outcome and other variables: A meta-analytic review. Journal of Consulting and Clinical Psychology. 2000;68:438–450. [PubMed] [Google Scholar]
  36. McLeod BD, Weisz JR. The therapy process observational coding system alliance scale: Measure characteristics and prediction of outcome in usual clinical practice. Journal of Consulting and Clinical Psychology. 2005;73:323–333. doi: 10.1037/0022-006X.73.2.323. [DOI] [PubMed] [Google Scholar]
  37. Mertz EL, Roesch SC. Modeling trait and state variation using multilevel factor analysis with PANAS daily diary data. Journal of Research in Personality. 2011;45:2–9. doi: 10.1016/j.jrp.2010.11.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Munder T, Wilmers F, Leonhart R, Linster HW, Barth J. Working Alliance Inventory-Short Revised (WAI-SR): Psychometric properties in outpatients and inpatients. Clinical Psychology and Psychotherapy. 2010;17:231–239. doi: 10.1002/cpp.658. [DOI] [PubMed] [Google Scholar]
  39. Muthén LK, Muthén BO. Mplus User’s Guide. Fourth Edition. Los Angeles, CA: Muthén & Muthén; 1998–2007. [Google Scholar]
  40. Raudenbush SW, Bryk AS. Hierarchical linear models: Applications and data analysis methods. 2. Thousand Oaks, CA: Sage Publications; 2002. [Google Scholar]
  41. Raue PJ, Goldfried MR. The therapeutic alliance in cognitive-behavior therapy. In: Horvath AO, Greenberg LS, editors. The working alliance: Theory, research, and practice. New York: Wiley; 1994. pp. 131–152. [Google Scholar]
  42. Roesch SC, Aldridge AA, Stocking SN, Villodas F, Leung Q, Bartley CE, et al. Multilevel factor analysis and structural equation modeling of daily diary coping data: modeling trait and state variation. Multivariate Behavioral Research. 2010;45:767–789. doi: 10.1080/00273171.2010.519276. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Rogers C. The necessary and sufficient conditions of therapeutic personality change. Journal of Consulting Psychology. 1957;21:95–103. doi: 10.1037/h0045357. [DOI] [PubMed] [Google Scholar]
  44. Rosenblatt A, Rosenblatt J. Demographic, clinical, and functional characteristics of youth enrolled in six California systems of care. Journal of Child and Family Studies. 2000;9:51–66. [Google Scholar]
  45. Shelef K, Diamond GM, Diamond GS, Liddle HA. Adolescent and parent alliance and treatment outcome in multidimensional family therapy. Journal of Consulting and Clinical Psychology. 2005;73:689–698. doi: 10.1037/0022-006X.73.4.689. [DOI] [PubMed] [Google Scholar]
  46. Shirk SR, Gudmundsen G, Kaplinski HC, McMakin DL. Alliance and outcome in cognitive-behavioral therapy for adolescent depression. Journal of Clinical Child and Adolescent Psychology. 2008;37:631–639. doi: 10.1080/15374410802148061. [DOI] [PubMed] [Google Scholar]
  47. Shirk SR, Karver M. Prediction of treatment outcome from relationship variables in child and adolescent therapy: A meta-analytic review. Journal of Consulting and Clinical Psychology. 2003;71:452–464. doi: 10.1037/0022-006x.71.3.452. [DOI] [PubMed] [Google Scholar]
  48. Shirk SR, Karver MS, Brown R. The alliance in child and adolescent psychotherapy. Psychotherapy. 2011;48:17–24. doi: 10.1037/a0022181. [DOI] [PubMed] [Google Scholar]
  49. Shirk SR, Saiz CC. Clinical, empirical, and developmental perspectives on the therapeutic relationship in child psychotherapy. Developmental Psychopathology. 1992;4:713–728. [Google Scholar]
  50. Stadnick NA, Drahota A, Brookman-Frazee L. Parent perspectives of an evidence-based intervention for children with autism served in community mental health clinics. Journal of Child and Family Studies. 2012 doi: 10.1007/s10826-012-9594-0. Advance online publication. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Steiger JH. Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research. 1990;25:173–180. doi: 10.1207/s15327906mbr2502_4. [DOI] [PubMed] [Google Scholar]
  52. Tanaka J. Multifaceted conceptions of fit in structural equation models. In: Bollen JKA, editor. Testing structural equation models. Newbury Park, CA: Sage; 1993. pp. 10–39. [Google Scholar]
  53. Webster-Stratton C, Herbert M. What really happens in parent training? Behavior Modification. 1993;17:407–456. doi: 10.1177/01454455930174002. [DOI] [PubMed] [Google Scholar]
  54. Zimprich D, Martin M. A multilevel factor analysis perspective on intellectual development in old age. In: Bosworth HB, Hertzog C, editors. Aging and cognition: Research methodologies and empirical advances. Washington, DC: American Psychological Association; 2009. pp. 53–76. [Google Scholar]

RESOURCES