Child, caregiver, and therapist perspectives on therapeutic alliance in usual care child psychotherapy

Abstract

This study examined the temporal stability and cross-informant agreement on multiple perspectives of child and caregiver alliance with therapists in usual care psychotherapy. Baseline predictors of alliance were also examined. Children with disruptive behavior problems (n=209) and their caregivers were followed for up to 16 months after initiating psychotherapy at a community-based clinic. Alliance was rated by children, caregivers, and therapists every four months for as long as families participated in treatment. Repeated-measures analyses using linear mixed models with random intercepts were conducted to determine whether child and caregiver alliance differed across time, as well to examine factors associated with each perspective on alliance. Intraclass correlations (ICCs) between child, caregiver, and therapist reports of alliance were also examined. Alliance was rated relatively high overall across perspectives. Clients (children and caregivers) tended to report the strongest and most stable alliance, while therapists reported the weakest alliance and perceived deteriorations in child alliance over time. Inter-informant agreement was variable for child and caregiver alliance; agreement was moderate between clients and therapists. Several predictors of alliance emerged, including child gender, anxiety diagnosis, caregiver race/ethnicity, and therapist experience. This study provides methodological information about reports of therapeutic alliance across time and informants that can inform current efforts to understand the alliance-outcome association.

Introduction

Historically, therapeutic alliance has received considerable attention in adult treatment, with over 7000 studies of therapeutic alliance in adult psychotherapy (Horvath, Del Re, Flückiger, & Symonds, 2011). However, as of 2011, only 38 studies have examined the therapeutic relationship and outcomes in child psychotherapy (McLeod, 2011). The most common definition of therapeutic alliance makes reference to the development of an affective bond, agreement on tasks, and agreement on goals (Bordin, 1979). In child psychotherapy, therapeutic alliance has been strongly associated with improved parenting (Kazdin & Whitley, 2006), reductions in the child’s symptomatology (Kazdin & Durbin, 2012; Marker, Comer, Abramova, & Kendall, 2013; Shirk, Gudmundsen, Kaplinski, & McMakin, 2008), and improved family functioning (Hawley & Garland, 2008). However, the relation between alliance and outcome appears to be less clear when patients are treated with non-specific treatments more characteristic of usual care, compared to empirically supported treatments (Ormhaug, Jensen, Wentzel-Larsen, & Shirk, 2014). Thus, the importance of therapeutic alliance is supported by research, but its role in usual care is less clear.

Unfortunately, current research on alliance in child psychotherapy is further limited in scope and methodological strength. Therapeutic alliance in child psychotherapy is complicated by multiple relationships—the child-therapist relationship (herein referred to as child alliance) and the caregiver-therapist relationship (herein referred to as caregiver alliance)—and multiple perspectives on those relationships (e.g., therapist, child, caregiver, observer). These methodological differences complicate the interpretation of discrepant findings in the alliance literature. For example, alliance-outcome findings have different interpretations depending on timing (e.g., early v. late alliance: Kazdin & Whitley, 2006 v. Hukkelberg, & Ogden, 2013) and analytic techniques (e.g., multilevel models that account for change over time v. one-time assessment: Marker et al., 2013 v. Shirk et al., 2008). Comparison across informants is complicated by the fact that research has often used different measures to assess child and caregiver alliance, which are based on differing conceptualizations of therapeutic alliance and therefore measure slightly different constructs. In addition, few studies have utilized repeated assessment, so very little is known about the stability of therapeutic alliance across treatment, despite its importance in understanding the relation between alliance and outcomes (McLeod, 2011). Nevertheless, several studies have found patterns of increased alliance over time in child outpatient treatment (Kazdin & Whitley, 2006; Kendall et al., 2009), which may be important given that patterns of change in adult alliance are more strongly predictive of outcome than single-point analysis (Kivlighan & Shaughnessy, 1995).

The primary aim of this exploratory study was to examine the temporal stability and cross-informant agreement of child and caregiver alliance across 16 months of psychotherapy, using multiple informants on alliance. A secondary aim was to examine baseline demographic and clinical predictors of greater alliance and different alliance trajectories over time. This study addresses some of the methodological limitations in research on child therapeutic alliance by collecting data from multiple perspectives, across time, and within the context of usual care psychotherapy (i.e., routine psychotherapy provided in community-based, non-research settings).

Methods

This study utilized data from the larger Practice and Research: Advancing Collaboration (PRAC) study, which examined usual care psychotherapy processes and outcomes in a representative sample of children with disruptive behavior problems. Therapists and their patients were recruited from six clinics, which represented the largest contractors for community-based publicly funded outpatient clinical care for children in one of the largest counties in the United States. All therapists from participating clinics were recruited sequentially, and 80% of those recruited agreed to participate. Additional details about study participants and methods are provided elsewhere (Garland et al., 2010; Garland, Accurso, Haine-Schlagel, Brookman-Frazee, Roesch, & Zhang, 2014).

Participants

Written informed consent was obtained from caregivers and children over age 8, and verbal assent was obtained from younger children. Informed consent was also obtained from therapist participants and other family members who attended psychotherapy sessions. Participants were financially compensated for their participation in research interviews. All protocols were approved by affiliated human subjects review committees.

Children and Caregivers

The sample included 209 children with disruptive behavior problems and their primary caregiver who contacted one of the six community-based publicly funded clinics in San Diego County to initiate outpatient mental health services. Of the potential 258 participants identified as meeting eligibility criteria, 40 parents declined to participate, leaving 218 (84%) child/caregiver dyad participants. Of these, nine were excluded from this study due to non-engagement in psychotherapy. English- and Spanish-speaking participants between the ages of 4 and 13 who were entering a new episode of outpatient treatment for a disruptive behavior problem with a participating therapist were eligible. Participants with mental retardation, brain damage, and major medical problems were excluded from the study. The mean age of participants was 9 years (SD=2.7), and 68% (n=141) were male. Race/ethnicity was diverse, with 48% Caucasian (n=100), 29% Latino (n=61), 10% African American (n=20), 14% Multiracial/Other (n=28). Clinician-assigned primary diagnoses varied, but 75% met criteria for a disruptive behavior or ADHD diagnosis and near half had psychiatric comorbidity (i.e., two or more diagnoses). Average annual household income was $36,205 (SD=30,205). Nineteen percent (n=40) of caregivers were Spanish-speakers.

Therapist Participants

This sample included 85 therapists practicing in six community-based clinics. Therapists had a mean of three years of psychotherapy experience (range: 0–25 years), and 58% (n = 49) were trainees. Most therapists were Caucasian (67%, n = 57) females (85%, n = 72). Therapists came from different mental health disciplines, including marriage and family therapy (59%, n = 50), psychology (21%, n = 18), and social work (20%, n = 17).

Procedures

Descriptive data on children, parents, and therapists were collected during in-person baseline interviews. Follow-up interviews were conducted by phone at months 4, 8, 12, and 16, during which child and caregiver therapeutic alliance were assessed. Therapeutic alliance from the therapist perspective was also assessed at the same time points by written questionnaire. Children, caregivers, and therapists were asked to report on current therapeutic alliance, or if they recently terminated psychotherapy, to retrospectively report on therapeutic alliance while they were active in treatment.

Measures

Therapeutic Alliance Scales for Children, Revised (TASC-r; Shirk & Saiz, 1992)

Child alliance was measured using the TASC-r, which includes 12 items rated from 1 (not true) to 4 (very much true) by children and therapists, with scores ranging from 12 to 48. The TASC-r includes items that distinguish between the affective bond (e.g., “I like my therapist”) and client-therapist collaboration on therapeutic tasks and goals (e.g., “I think my therapist and I work well together on dealing with my problems”), in line with Bordin’s (1979) conceptualization of therapeutic alliance. The TASC-r has demonstrated good reliability and validity in previous studies (Creed & Kendall, 2005; Hawley & Weisz, 2005; Shirk & Saiz, 1992) and had a Cronbach’s alpha of .91 in this study.

Therapeutic Alliance Scales for Caregivers and Parents (TASCP; Accurso, Hawley, & Garland, 2013)

Caregiver alliance was measured using the TASCP, a parallel version of the TASC-r developed for use with both caregivers and therapists. It has demonstrated good reliability (Hawley & Weisz, 2005) and validity (Accurso et al., 2013) in previous studies, with a Cronbach’s alpha of .85 in this study.

Analysis

All analyses were conducted using SPSS Version 19.0. In order to examine the temporal stability of child and caregiver alliance, repeated-measures analyses using linear mixed models with random intercepts were conducted. Second, a series of paired-sample t-tests were used to compare patient (child and caregiver) and therapist means on alliance at each time point. Third, Shrout and Fleiss’ (1979) intraclass correlations [ICC(3,1)] were used to assess cross-informant agreement between child, caregiver, and therapist reports of alliance at 4 months. Finally, follow-up analyses using the above repeated-measures analyses were conducted to examine baseline predictors of alliance, including child age, gender, race/ethnicity, diagnosis (disruptive behavior disorder, anxiety disorder, mood disorder), therapist months of experience, and therapist theoretical orientation. Caregiver age, gender, race/ethnicity, and number of children living in the home were additionally examined as potential predictors of caregiver alliance. These predictors were first examined in separate linear mixed models, including main effects and interactions with time. Caregiver age was included as a covariate in the model examining number of children to ensure that its effect was not confounded by age. Given the exploratory nature of these analyses, variables that predicted alliance at the trend level (p < .07) were entered into a final multivariable model, in order to identify any potential effect of baseline characteristics on alliance.

Results

Missing Data

All 209 families were active in treatment at 4 months, 141 (67.5%) were active at 8 months, 100 (47.8%) at 12 months, and 61 (29.2%) at 16 months. Data were considered “missing” when a child and/or caregiver was lost to follow-up despite being actively engaged in treatment. Across reporters, alliance scores at 4 months did not differ between families who terminated treatment by 4 months and those remained in treatment for 16 months (ps > .05). There were no missing data patterns for child-reported therapeutic alliance at 4 months (child gender, age, race/ethnicity, household income, and child disruptive behavior severity at baseline: ps > .05). There were no missing data patterns for child alliance at any other time point, except for age at 12 months (t[58] = −2.054, p = .045), such that children with data were older (M = 11.1, SD = 1.4) than those with missing data (M = 10.2, SD = 1.5). There were no missing data patterns for caregiver-reported therapeutic alliance (caregiver gender, age, race/ethnicity, and child diagnosis: ps > .05), or for therapist-reported therapeutic alliance (therapist gender, age, theoretical orientation, and child diagnosis: ps > .05).

Temporal stability of therapeutic alliance

Child (children: M = 38.07, SD = 8.54; therapists: M = 35.08, SD = 7.22) and caregiver alliance (caregivers: M = 42.80, SD = 5.52; therapists: M = 39.47, SD = 6.78) were relatively high across time and informants. Initial inspection of within-individual change over time indicated minimal variability across child-reported (SD = 3.96) and therapist-reported child alliance (SD = 3.43), as well as caregiver-reported (SD = 3.41) and therapist-reported caregiver alliance (SD = 3.32). Both children (B = −0.432, SE = 0.345, t[df = 171.7] = −1.252, p = .21) and caregivers (B = 0.057, SE = 0.212, t[df = 331.7] = 0.271, p = .79) perceived their alliance as stable over time. In contrast, therapists perceived small but significant deteriorations in child alliance over time (B = −0.741, SE = 0.259, t[df = 238.7] = −2.859, p = .005) but a stable pattern of alliance with caregivers (B = 0.097, SE = 0.259, t[df = 232.5] = 0.374, p = .71). Figure 1 shows child and caregiver alliance means over time.

Figure 1.

Child and caregiver alliance patterns across 16 months (plotted means).

Mean levels of alliance across informants

Clients tended to rate alliance higher than therapists. Within child alliance, children rated alliance higher than therapists at 4 months (t₆₈ = 2.73, p = .008) and 8 months (t₅₀ = 5.71, p < .001), but this discrepancy was attenuated at 12 months (t₃₃ = 2.00, p = .054) and 16 months (t₂₀ = 1.78, p = .090). Within caregiver alliance, caregivers rated alliance higher than therapists across time: 4 months (t₁₃₀ = 6.76, p < .001), 8 months (t₉₀ = 4.56, p < .001), 12 months (t₆₀ = 3.34, p = .001), and 16 months (t₄₄ = 3.79, p < .001).

Cross-informant agreement

Within reports of child alliance and caregiver alliance

ICCs between child, caregiver, and therapist reports of alliance were examined at 4 months. Within child alliance, there was moderate reliability between child and therapist reports (ICC = .50, p = .003). Within caregiver alliance, the results were similar to those within child alliance, with substantial reliability between therapist reports and caregiver reports (ICC = .67, p < .001).

Between reports of child and caregiver alliance

There was moderate consistency between child and caregiver reports of their respective alliance with the therapist (ICC = .54, p < .001) as well as between child-reported child alliance and therapist-reported caregiver alliance (ICC = .41, p = .016). There was also moderate consistency between therapist-reported child alliance and caregiver-reported caregiver alliance (ICC = .40, p = .002) and between therapist-reported child alliance and therapist-reported caregiver alliance (ICC = .52, p < .001).

Baseline predictors of therapeutic alliance

Child alliance

One multivariable model was run for therapist report child alliance that included child gender and anxiety disorder diagnosis. Results indicated that therapist-reported child alliance improved over time for girls but remained stable in boys (B = −1.561, SE = 0.748, t[df = 124.8] = −2.086, p = .039), and that alliance improved for children with anxiety disorders but remained stable in those without (B = −1.538, SE = 0.773, t[df = 120.0] = −1.988, p = .049). There were no significant predictors of child-reported child alliance.

Caregiver alliance

Two separate multivariable models were run for caregiver report caregiver alliance (including caregiver race/ethnicity and therapist experience) and therapist report caregiver alliance (including caregiver sex, caregiver race/ethnicity, and therapist experience). For caregiver-reported alliance, results indicated that non-Hispanic White caregivers perceived alliance to start high and improve less, while caregivers of other races/ethnicity perceived alliance to start lower but improve more across time (B = −5.317, SE = 1.405, t[df = 297.9] = −3.783, p = .001). Caregivers also perceived a stronger alliance overall with therapists who had less experience (B = −0.044, SE = 0.016, t[df = 423.8] = −2.774, p = .006).

For therapist-reported caregiver alliance, there was a trend-level main effect for caregiver sex, such that therapists perceived marginally stronger alliances with mothers than fathers (B = −5.209, SE = 2.719, t[df = 148.7] = −1.916, p = .057). There was also a trend-level interaction of therapist experience with time, such that therapists with more experience perceived caregiver alliance to increase over time, while those with lesser experience perceived alliance as stable over time (B = 0.019, SE = 0.010, t[df = 229.9] = 1.855, p = .065). Finally, the initial main effect of race/ethnicity (p = .049) (whereby therapists perceived stronger alliances with non-Hispanic White caregivers compared to caregivers of other races/ethnicity) was no longer significant in the multivariate model (p = .65).

Discussion

Temporal stability and cross-informant agreement of child and caregiver alliance in usual care was examined in this study, using multiple perspectives (child, caregiver, and therapist) across 16 months. Overall, analyses showed that the perceived quality of the child and caregiver alliance was high across perspectives and time, with minimal within-individual variability in alliance slopes over time and moderate agreement between informant pairs. Clients (children and caregivers) tended to report the most stable alliance. Therapists, on the other hand, consistently rated the alliance lower than did clients, and even reported deteriorations over time for child alliance. This study was unable to identify factors that predicted child-reported alliance, but several factors emerged as predictors of therapist-reported child alliance, including child gender and anxiety disorder diagnosis. In contrast, caregiver-reported caregiver alliance was predicted by caregiver race/ethnicity and therapist years of experience, but there were only trend-level predictors of therapist-reported caregiver alliance.

These findings revealed that clients generally perceive strong and stable alliances with their therapist, but that therapists may underestimate the extent to which families feel allied with them. Anecdotally, interactions with therapist participants suggest that the alliance is highly valued and sometimes prioritized more highly than therapeutic tasks. Therefore, therapists may be highly sensitive to perceived ruptures in the alliance or highly perceptive of actual caregiver or child dissatisfaction that is expressed in a subtle manner, about which clients may less aware or fail to report. Therapists may also have exceptionally high expectations for the quality of the alliance and therefore be less susceptible to ceiling effects. Alternatively, therapists may be less aware of alliance quality given that they received relatively little explicit feedback on alliance. Indeed, although the client therapeutic relationship was addressed in almost one fifth of psychotherapy sessions, but most instances of this were fleeting, with only 1% considered to be “high-intensity” (Garland et al., 2010).

In addition, several factors influenced caregivers’ and therapists’ perceptions of the alliance. For child alliance, therapists perceived significant improvements over time in child alliance with girls and children with anxiety disorders, while alliance with boys and those without anxiety disorders remained stable. This is in keeping with prior research in finding improvements in alliance for children with anxiety (Creed & Kendall, 2005) or greater overall alliance in children with greater anxiety (Chu, Skriner, & Zandberg, 2013). For caregiver alliance, non-Hispanic Whites caregivers perceived alliance to start high and improve less, while those of other races/ethnicity perceived alliance to start lower but improve more across time. It is possible that this pattern could be the result of initially perceived mistrust or misunderstanding that improves over time. This would be in keeping with research on race/ethnicity in therapy. For example, racial/ethnic minorities believe that White therapists do not understand their cultural experiences, but that this perception is mitigated by therapist compassion, acceptance, and comfort in discussing cultural issues (Chang & Yoon, 2011). It is therefore encouraging that therapists were effective in improving initially poorer alliances with caregivers of non-White race or Hispanic ethnicity. Caregivers also reported an overall stronger alliance with therapists who had less experience. This finding might be due in part to therapist burn-out, such that less experienced therapists might have been more energetic and eager to build new therapeutic alliances than therapists who had seen many clients/families over the years and who were potentially experiencing burn-out. In contrast, there was a trend-level interaction such that therapists with more experience perceived caregiver alliance as increasing over time, while those with lesser experience perceived alliance as stable. While not statistically significant, this trend suggests that more experienced therapists may overestimate their ability to develop a stronger caregiver alliance over time.

This study is unique in its examination of therapeutic alliance in usual care settings with relatively large and representative samples of children, caregivers, and therapists, thereby increasing its ecological validity. Assessment with parallel child and caregiver measures, administered to multiple informants across 16 months of treatment allowed for a better understanding of perspectives on alliance over time. In addition, the use of multilevel statistical techniques allowed for appropriate handling of the nested data structure and missing data. Despite strengths related to the usual care setting, the challenges of conducting research in such settings were also associated with several limitations. For example, temporal stability of alliance was increasingly based on the subset of participants who had not terminated treatment (i.e., approximately two thirds at 8 months, one half at 12 months, and one third at 16 months). While this allowed for a longer follow-up for the significant proportion of families who were in treatment that long, it also failed to account for the temporal stability of alliance for clients who received relatively brief courses of treatment. Although participants who remained in treatment longer are unique, their alliance scores at four months were not significantly different from scores for those who terminated therapy earlier. Nevertheless, the collection of alliance data every four months rather than every session may have diminished our ability to detect more nuanced changes in alliance trajectories.

Implications for research and practice

These data provide evidence that children with disruptive behavior problems and their caregivers receiving usual care services perceive positive and temporally stable therapeutic alliances across time. While more frequent assessment of therapeutic alliance might have captured nuanced peaks or ruptures in the alliance, this study found that early alliance strength did not differ between families who terminated treatment early and those received long-term treatment. This is in direct contrast to previous efficacy studies in which poorer child and caregiver alliance predict treatment dropout (de Haan, Boon, de Jong, Hoeve, & Vermeiren, 2013), suggesting that alliance in usual care may function differently with respect to treatment engagement. For example, there may be a variety of situational factors (e.g., employment status or schedule, relocation, family stressors) that have a greater impact on treatment engagement for clinically-referred patients in usual care. Alternatively, earlier and more frequent assessment of alliance may have better captured potential ruptures associated with treatment drop-out.

Agreement between children, caregivers, and therapists on alliance was remarkably high in this sample, particularly in light of previous research finding poor child-caregiver-therapist agreement on therapeutic goals (Hawley & Weisz, 2003). Nevertheless, therapist reports on alliance were more variable over time and differed from client reports. Interestingly, therapists tended to rate alliance lower than children and caregivers, and it is possible that specifically requesting feedback on the therapeutic relationship from patients may be helpful to therapists. It was surprising that, by caregiver report, less experienced therapists were more effective in building stronger caregiver alliances than more experienced therapists, since more experienced therapists have had more time to refine their therapeutic skills. However, less experienced therapists have had less opportunity for burn-out, and therefore may approach therapy sessions with greater enthusiasm and energy. Alternatively, less experienced therapists may be more likely to implement evidence-based treatments and explicitly set therapeutic goals and tasks, both of which factor into alliance ratings. Additionally, there was a trend such that less experienced therapists perceived alliance more accurately (in comparison to caregiver reports) than more experienced therapists. This would suggest that more experienced therapists might benefit more from explicitly requesting caregiver feedback on the therapeutic process. However, this finding did not reach statistical significance and therefore is quite tenuous, so future research would be needed to clarify this relation.

This is one of the first studies to examine cross-informant agreement and temporal stability of therapeutic alliance in child mental health care, and its results inform our interpretation of alliance assessment. Although this study does not have a controlled design, it captured therapeutic alliance in a real world context, and provides some promising data on the strength of alliance in usual care, as well as the implications for interpreting reports from different stakeholders (e.g., consistent findings that therapists reported relatively lower quality of alliance than clients). However, future research is needed to better understand changes in alliance early in therapy (i.e., within the first few months) and how child and caregiver alliance relate to outcome. More research is also needed to understand differences in how stakeholders assess alliance and/or practice aspects of alliance.