Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Nov 1.
Published in final edited form as: J Couns Psychol. 2020 Mar 26;67(6):706–711. doi: 10.1037/cou0000424

Assessing the alliance-outcome association adjusted for patient characteristics and treatment processes: A meta-analytic summary of direct comparisons

Christoph Flückiger 1, AC Del Re 1, Daniel Wlodasch 1, Adam O Horvath 2, Nili Solomonov 3, Bruce E Wampold 4
PMCID: PMC7529648  NIHMSID: NIHMS1554404  PMID: 32212755

Abstract

Objective:

The alliance is widely recognized as a robust predictor of posttreatment outcomes. However, there is a debate regarding whether the alliance is an epiphenomenon of intake characteristics and/or treatment processes occurring over the course of treatment. This meta-analysis aimed to synthesize the evidence on this issue.

Methods:

We identified 125 effect sizes in sixty independent samples (6061 participants) of studies that reported alliance-outcome correlations as well as parallel intake or process characteristics. We examined the impact of these potential confounds on the alliance-outcome correlations. We meta-analyzed the studies estimates by computing omnibus effects models as well as in multivariate models.

Results:

We identified three variable types that were used to adjust the alliance-outcome correlations: (a) intake characteristics (k = 35); (b) simultaneous processes, such as adherence or competence (k = 13); and (c) both intake and simultaneous processes (k = 24). We found moderate alliance-outcome correlations with or without adjustments for intake and simultaneous processes (range from r = .23 to .31).

Conclusion:

Our results provide robust empirical evidence for the assertion that the alliance-outcome association is an independent process-based factor. Findings suggest that alliance is positively related to outcome above and beyond the studied patient intake characteristics and treatment processes.

Keywords: working alliance, adherence, process-based therapy, between-patient effects, meta-analysis

The therapeutic alliance is one of the most widely investigated process-based factors associated with psychotherapy outcome (Norcross & Lambert, 2019). The alliance is conceptualized as a pan-theoretical construct (Bordin, 1979; Horvath, 2018) that includes the collaboration between therapists and patients in the context of a supporting (working) relationship (e.g., Norcross & Goldfried, 2019; Hatcher, 2010; Luborsky, 1976; Norcross & Lambert, 2019). The alliance can be characterized as the subjective experience of a collaborative, trusting and goal-oriented environment; a dyadic quality that can be evaluated in face-to-face therapy (Horvath, 2018) as well as internet-based treatments (Flückiger, Del Re, Wampold, & Horvath, 2018; Probst, Berger & Flückiger, 2019). The majority of meta-analytic research on the alliance–outcome association has investigated the impact of alliance on outcome from a between-patient (BP) perspective – patients who report strong alliance during treatment also report better outcomes at treatment end (e.g., Del Re, Flückiger, Horvath, Symonds, & Wampold, 2012; Flückiger, Del Re, Horvath, Symonds, Ackert & Wampold, 2013; Flückiger, Del Re, Wampold, Symonds & Horvath, 2012; Horvath, Del Re, Flückiger & Symonds, 2011).

The most recent meta-analytic synthesis of the overall BP alliance-outcome association indicated that the alliance predicted on average r = .278 (95% CI [.256, .299], k = 295; Flückiger, et al., 2018). However, some argue that the alliance may be a byproduct of other treatment, patient, and therapist intake factors that are more directly related to outcome (DeRubeis, Brotman, & Gibbons, 2005). Conversely, findings of a recent individual participant data meta-analysis suggested that the BP alliance-outcome association is only marginally impacted by patients’ symptomatic severity at intake (k = 17; Flückiger, Rubel, Del Re et al., 2019). Nonetheless, the degree to which the alliance-outcome correlation is affected by a range of patient intake characteristics as well as other treatment processes (e.g., therapist competence, homework compliance) remains controversial (e.g., DeRubeis et al., 2005; Huppert, Fabbro, & Barlow, 2006; Siev, Huppert & Chambless, 2009).

The zero-order alliance-outcome correlation is the most straight-forward statistical estimate of the effect of the alliance (assessed during therapy) on treatment outcome (e.g., Horvath, Del Re, Flückiger & Symonds, 2011). However, it does not consider other factors that may mitigate this association. To test whether the BP alliance-outcome correlation is robust with respect to potentially confounding factors, researchers often report coefficients that are adjusted for the effect of these factors. For example, partial-correlations describe the linear association of alliance and outcome after controlling for the effect of one or more potentially confounding factors. Most alliance-outcome studies adjust for intake characteristics such as demographics and/or pre-treatment symptomatic severity (e.g. Barber, Zilcha-Mano, Gallop, Barret, McCarthy & Dinger, 2014; DeRubeis et al., 2005). Researchers have also focused on other patient characteristics, such as personally traits, interpersonal distress, and treatment expectancy (see Online Supplemental Material).

Some studies also adjust for treatment processes that may affect the alliance-outcome association. Within these simultaneously assessed processes, therapist adherence and competence (observer-ratings of delivery of manualized interventions) were investigated most often (k = 8). This line of inquiry is based on the premise that the alliance-outcome associations may be explained by the variability of how therapists follow the prescriptive treatment manuals (viz. adherence and competence). That is, adherent and/or competent therapists may develop stronger alliances and provide more effective interventions. Thus, a strong alliance may be a “byproduct” of adherence rather than a therapeutic factor (e.g., Barber, Gallop, Crits-Christoph, Frank, Thase, Weiss & Connolly Gibbons, 2006; Kazantzis, Dattilio & Dobson, 2017; Weck, Richtberg, Jakob, Neng, & Höfling, 2015; Siev et al., 2009). Conversely, the direct impact of adherence and competence observer-ratings on outcome may be overestimated (e.g., Webb, DeRubeis & Barber, 2010). Moreover, a prior meta-analysis found that manual use was not a moderator of the alliance-outcome association (Flückiger et al., 2012). However, no meta-analytic investigation directly examined adherence and competence ratings as a confound of the alliance-outcome association.

The primary aim of the present meta-analysis was to determine the robustness of the alliance-outcome correlation by synthesizing available prior research which included adjustments for intake characteristics and treatment processes. More specifically, we examined direct within-study comparisons of zero-order alliance-outcome correlations and partial alliance-outcome correlations when accounting for potential confounding factors. We evaluated whether the alliance-outcome association is reduced when adjusting for these factors. We anticipated that the adjusted associations would be substantially larger than zero. We also examined the unique effect of adherence and competence on the alliance-outcome association. We expected that the adjusted correlations will not be attenuated.

Methods

The initial corpus for this study was based on data collected as part of a large-scale published meta-analysis (k = 295; see Flückiger et al. 2018 for details; search in PsycINFO and PSYNDEX from 1978 to April, 2017 via EBSCO; keywords: helping alliance, working alliance, or therapeutic alliance; report of an alliance-posttreatment outcome association). To qualify for the present meta-analysis, at least two correlations had to be reported for within-study comparisons (e.g., report of a zero-order correlation and an adjusted correlation for intake in the same sample). From the Flückiger et al. 2018 data, we identified 60 independent samples (6061 participants) which reported at least one of the following alliance-outcome correlations adjusted for: (a) intake characteristics (k = 35); (b) simultaneous process characteristics such as adherence, competence, or homework compliance assessed during therapy (k = 13); (c) intake characteristics as well as simultaneous treatment processes assessed at both intake and during therapy (k = 24). In addition, we identified k = 53 zero-order correlations within these samples. In most studies, alliance was assessed early in therapy, before session 6; in the cases where alliance was assessed later, the alliance-outcome association was not substantially different from the early alliance assessment (rdiff = .026), Qm(1) = .53; p = .47. We considered all adjusted alliance-outcome correlations selected by the researchers of the primary studies to be relevant as potential confounding variables within the primary study conditions. We report descriptive characteristics as well as the references of the included primary studies in the Online Supplemental Material. For each reported alliance–outcome association, one correlational effect size was computed1. In some cases multiple alliance–outcome correlations were reported in the original study, e.g., due to the report of multiple outcome measures. In order to maintain the assumption of independent samples and avoid greater weight attributed to studies that reported multiple estimates, we aggregated within-study estimates (Del Re & Hoyt, 2010), such that each study contributed only one estimate for each adjustment type. Next, we computed the meta-analytic overall effects across study-level estimates by calculating overall omnibus tests for the zero-order and each of the three adjusted correlations separately. Finally, we used multivariate models to compare the effects of the different adjustment types. For the analyses, the correlations were transformed to Fisher’s z and then transformed back to r for interpretative purposes. We assumed an unstructured variance-covariance matrix and a correlation of .70 across the adjustment types (Pustejovsky & Tipton, 2017; Viechtbauer, 2019). We used a random-effects model estimator (REML), assuming that the included studies were representative samples of a population of studies.

Heterogeneity was assessed using the Q and I2 statistics (Higgins & Thompson, 2002). I2 represents the degree of heterogeneity across studies, computed as a percentage of the observed variability among studies due to true differences. Furthermore, we calculated credibility intervals as a further indicator of heterogeneity (Wiernik, Kostal, Wilmot & Dichert, 2017). Using funnel plots and regression tests for asymmetry (Egger, Smith, Schneider, & Minder, 1997), we examined the possibility that our search may have been biased because we did not capture unpublished studies with potentially low or nonsignificant results. Analyses were conducted using the R statistical software packages for meta-analysis “MAc” for data aggregation (Del Re & Hoyt, 2010), “clubSandwich” for variance-covariance matrix imputation (Pustejovsky & Tipton, 2017) and “metafor” for the omnibus tests and the multivariate analyses (Viechtbauer, 2019).

Results

Separate omnibus effects with and without adjusting for intake and simultaneous processes

First, we calculated omnibus effects for each of the reported adjustment types (Table 1). The omnibus effect of the correlation between alliance and posttreatment symptoms zero-order correlations was r = .304 (95% CI [.253, .354], p < .001, k = 53). When the authors of the primary studies used intake characteristics as control variables, the adjusted alliance-outcome correlations was r = .286 (95% CI [.226, .344], p < .001, k = 35). When the adjusted correlations were controlled for treatment processes the omnibus effect was r = .242 (95% CI [.179, .306], p < .001, k = 13). Finally, when we calculated the correlation adjusting for both intake characteristics and treatment processes (k = 24), the omnibus effect was r = .244 (95% CI [.193, .296], p < .001). Overall, these correlations are in line with the prior meta-analysis that reported a zero-order correlation of r = .278 (k = 295; Flückiger, et al., 2018). We observed some heterogeneity for the zero-order correlations between alliance and outcome and for the omnibus effect of the adjusted for intake characteristics (Q > 91, I2> 62%, p < .001). We found significant asymmetry in the plots for both of these analyses; studies with small samples reported higher estimates compared to larger studies (asymmetry: z > 2.9, p < .01). For the last two types of adjustments (that both adjust for simultaneous process characteristics), the results did not indicate substantial heterogeneity (Q > 30.6, I2 < 33%, p > .10) and there was no indication for publication bias (asymmetry: z < 1.5, p > .10).

Table 1.

Separate Models of the Alliance-Outcome Correlation With and Without Adjusting for Intake and Simultaneous Processes

Alliance-outcome correlation k r [95% CI]a) tau2 80%CrIn Q (df)b) I2 z asymmetry c)
Zero-order correlation 53 .304 [.253, .354]*** .024 .106, .502 164.5 (52)*** 69.3% 4.1***
Adjusted correlation for intake 35 .286 [.226, .344]*** .020 .103, .469 91.1 (34)*** 64.2% 2.94**
Adjusted correlation for process 13 .242 [.179, .306]*** .0 - 8.9 (12) 0% .5
Adjusted correlation for intake + process 24 .244 [.193, .296]*** .005 .152, .334 30.6 (23) 32.4% 1.5
Open in a new tab

Note. CI = confidence interval, 80%CrIn = 80% credibility interval

***

p < .001

**

p < .01

a)

p-value indicate difference from r = 0

b)

p-value indicates significance of heterogeneity

c)

p-value indicate significance of asymmetry as a traditional indicator for publication bias

Multivariate model with and without adjusting for intake and simultaneous processes

In the second part of our analysis, we used a multivariate model to directly compare the four adjustment types: zero-order correlations, and the three correlations that adjusted for intake and/or process characteristics (Table 2). Results were in line with the separate omnibus tests showing that all adjusted correlations were significantly greater than zero (r > .23, p < .0001). Moreover, we observed significant differences between the adjustment types (QM[df 3] = 12.6, p = .006), whereas zero-order correlations r = .311 (95% CI [.264, .357]) and adjusted correlations controlled for intake characteristics r = .289 (95% CI [.249, .333]) were not significantly different (z = − 1.4, p = .16). Adjusted correlations that controlled for treatment processes (r = .232, 95% CI [.179, .285]) as well as for intake characteristics and treatment processes (r = .247, 95% CI [.202, .290]) had slightly lower correlations (direct comparison with zero-order correlations, z = − 2.6, p = .022 and z = −3.1, p = .025 respectively). Moreover, there was considerable residual heterogeneity unexplained by the adjustment type (QE(df = 121) = 269.4, p < .001).

Table 2.

Multivariate Model of the Alliance-Outcome Correlation With and Without Adjusting for Intake and Simultaneous Processes

Alliance-outcome correlation k r [95% CI]a) QM (df)b) rdiff [95% CI] c) tau2 80%CrIn
Zero-order correlation 53 .311 [.264, .357]*** 12.6 (3) ** .024 .113, .509
Adjusted correlation for intake 35 .289 [.249, .333]*** −.024 [.010, −.060] .016 .127, .451
Adjusted correlation for process 13 .232 [.179, .285]*** −.085 [−.022, −.146]** .001 .192, .272
Adjusted correlation for intake + process 24 .247 [.202, .290]*** −.070 [−.025, −.114]** .008 .132, .312
Open in a new tab

Note. 125 effect sizes nested in 60 independent samples, CI = confidence interval, 80%CrIn = 80% credibility interval

***

p < .001

**

p < .01

a)

p-value indicate difference from r = 0

b)

p-value indicate significance of the overall moderator test

c)

p-value indicate difference from zero-order correlation (i.e., r = .311)

Multivariate model with and without adjusting for adherence and competence

We explored a multivariate model, where zero-order correlations were contrasted with the adjusted correlations that controlled for adherence and/or competence ratings (k = 8, Table 3). In this model, both correlational types significantly differed from zero (for both r > .25, p < .0001), but not from each other, (QM[df 1] = .4, p = .50). Moreover, within this contrast we found no evidence of substantial heterogeneity (QE[df 14] = 9.7, p = .78).

Table 3.

Multivariate Model of the Alliance-Outcome Correlation With and Without Adjusting for Adherence and Competence

Alliance-outcome correlation k r [95% CI]a) QM (df)b) rdiff [95% CI] tau2 80%CrIn
Zero-order correlation 8 .256 [.172, .336]**** .4 (1) p = .50 .001 .216, .296
Adjusted correlation 8 .276 [.188, .364]**** .023 [.092, −.045] .003 .236, .316
Open in a new tab

Note. 16 effect sizes nested in 8 independent samples, CI = confidence interval, 80%CrIn = 80% credibility interval

****

p < .0001

a)

p-value indicate difference from r = 0

b)

p-value indicate difference from zero-order correlation (i.e., r = .256)

Discussion

The primary purpose of this research was to examine using meta-analytic methods whether the alliance-outcome association contribute to treatment outcome, above and beyond potential confounds, or whether it is a by-product of patient factors at intake or early therapeutic processes (i.e., the epiphenomenon hypothesis). The authors of the primary studies considered a broad range of variables, instruments and rater-perspectives to adjust for intake and simultaneous processes. The authors of the primary studies justified the choice of these factors and accordingly we summarized the factors examined in the primary studies. Our principal finding is that the investigated intake characteristics, process variables, and adherence/competence ratings did not have a substantial impact on either the magnitude of the alliance-outcome association or the variability in alliance effects across studies.

The results confirmed our hypothesis that the adjusted alliance-outcome correlation are substantially larger than zero. Similarly, the omnibus tests indicated moderate associations of alliance with outcome when adjusting for confounding factors (with .23 < r < .29). These results are in line with previous meta-analytic studies reporting robust omnibus effects (e.g., Horvath et al., 2011; Flückiger et al., 2018). They provide further evidence that the alliance-outcome association is a unique therapeutic factor that affects treatment efficacy beyond intake characteristics and process variables (e.g., Hatcher, 2010; Horvath, 2018).

In contrast to prior meta-analyses, we found asymmetry in the data (i.e., smaller studies reported higher effect sizes), especially in zero-order correlations and adjusted correlation that controlled for intake characteristics. Traditionally, such asymmetry may suggest a publication bias. However, we did not observe asymmetry in the other two types of adjusted correlations, suggesting that this issue requires further investigation. It is possible that smaller studies are more likely to investigate zero-order correlations, rather than adjust for confounds given limited statistical power (e.g., Algina & Olejnik, 2003).

Using multivariate models, we were able to compare the different adjustment types under identical study conditions (i.e., direct within-study comparisons). The multivariate analyses showed that the adjusted correlations for treatment processes were slightly lower than the zero-order correlations, suggesting that some of possible confounds explained a portion of the alliance-outcome correlation. However, our data indicated a high variability across studies in the intake and process characteristics investigated, suggesting a diversity of views of what the relevant confounding factors might be. As a result, there was only a limited number of particular variables available for meta-analysis across studies (see Online Supplemental Material).

This meta-analysis was the first to investigate the effects of adherence and competence on the alliance-outcome association. It is important to note that the majority of competence ratings focused on cognitive behavioral therapy competence. We did not find reduced adjusted alliance-outcome correlations in these samples. This result contradicts the assumption that low levels of particular treatment-competence ratings may moderate the alliance-outcome association (e.g., Siev et al., 2009; Kazantzis, et al., 2017).

This study supports the notion that a collaborative engagement within the therapist– patient relationship is a relatively independent indicator of successful outcome (e.g., APA, 2006; Ribeiro, Ribeiro, Gonçalves, Horvath, & Stiles, 2013). Traditionally, research in psychiatry and clinical psychology places great emphasis on investigating patients’ levels of distress and symptomatology as a predictor of treatment success (e.g. Cujipers et al., 2010; Weitz et al., 2015). Our findings confirm the positive relation of the alliance on treatment outcomes, above and beyond the effect of patients’ initial distress levels. Moreover, the results are in line with session-by-session analyses at the within-patient level, where higher alliances predicted lower symptoms in the subsequent sessions early in therapy (Flückiger, Rubel, et al., 2020; Zilcha-Mano, 2017). Overall, our findings highlight the relevance of evaluating relationships factors and employing two-person perspective when investigating treatment efficacy.

The present investigation has some limitations. The size of the available primary studies that explicitly adjusted for particular processes was limited (for adherence and competence: k = 8). Moreover, the primary studies included in the meta-analysis reported a variety of outcome measures across different disorders and samples. This lack of uniformity likely contributed to the observed heterogeneity in some of the analyses. Additionally, several studies investigated interpersonal intake characteristics (such as social support or patients’ interpersonal problems), but none of these characteristics were systematically investigated separately across multiple studies (k < 6). This variation may have contributed to the observed heterogeneity of effects. Nonetheless, the observed range of effects (indicated by the confidence intervals) was far from uninterpretable and suggested some heterogeneity with moderate positive correlational effects. Moreover, most studies measured patient characteristics only at intake, which limits the ability to investigate the effects of longitudinal change in these characteristics on the alliance-outcome association. Lastly, as noted, relatively few of all the potential mediators of alliance-outcome relation were investigated in sufficient numbers to synthesize in a meta-analysis. Thus, it is still possible that there is a significant overlap between some intake or process variables yet to be discovered, e.g., when investigating multiple perspectives on the alliance (e.g., Kivlighan, 2007; Marmarosh & Kivlighan, 2012; O’Connor, Kivlighan, Hill & Gelso, 2019) or when assessing alliance feedback in routine outcome monitoring (e.g., Brattland et al., 2019). Nonetheless, it is important to note that the alliance-outcome association was relatively robust in those studies, that investigated a very broad range of intake characteristics (i.e., Fakhoury et al., 2007; Hersoug et al., 2013; Marzial et al., 1999; Meyer et al, 2002; see Online Supplemental Material).

In conclusion, the present investigation provides further empirical support for the robustness of the alliance-outcome correlation indicating that the alliance is a reliable process-based factor of therapy success independent of patients’ characteristics and initial levels of distress, as well as therapists’ adherence and competence ratings (e.g., Crits-Christoph, Gallop, Gaines, Rieger, & Connolly Gibbons, 2018; Norcross & Goldfried, 2019; Hofmann & Hayes, 2019; Muran & Barber, 2010; Norcross & Lambert, 2019; Wampold & Imel, 2015).

Supplementary Material

Supplemental Material 1
Supplemental Material 2

Public Health Significance Statements.

The alliance is a robust predictor of outcome at the between-patient level. Patients who report a stronger alliance during treatment are also likely to report better treatment outcome. This association remains significant when controlling for patients’ intake characteristics and therapists’ adherence and competence. These results demonstrate that the alliance is an independent process-based factor with unique contribution to outcome across many psychotherapeutic contexts.

Footnotes

1

Studies were included if they provided sufficient data to extract standardized estimates. Alliance-outcome correlations were considered when alliance was assessed during therapy (usually at post-session, but not at post-treatment) and outcome was assessed at post-treatment (and not during therapy). In the few cases, where inconsistencies emerged, the first and third author conducted a consensus meeting (for study inclusion: 94% agreement prior to consensus; for effect size and study extraction: 96%, i.,e., Andrews et al., 2016 and Barnicot et al., 2016).

Contributor Information

Adam O. Horvath, Faculty of Education, Simon Fraser University, Vancouver, Canada

Nili Solomonov, Weill Cornell Institute of Geriatric Psychiatry, Weill Cornell Medical College, New York, USA.

Bruce E. Wampold, Modum Bad Psychiatric Center, Norway and University of Wisconsin – Madison

References

  1. Algina J & Olejnik S. (2003). Sample Size Tables for Correlation Analysis with Applications in Partial Correlation and Multiple Regression Analysis. Multivariate Behavioral Research, 38, 309–323, doi: 10.1207/S15327906MBR3803_02 [DOI] [PubMed] [Google Scholar]
  2. Andrews M, Baker AL, Halpin SA, Lewin TJ, Richmond R, Kay-Lambkin FJ, Filia SL, Castle D, Williams JM, Clark V, & Callister R. (2016). Early therapeutic alliance, treatment retention, and 12-month outcomes in a healthy lifestyles intervention for people with psychotic disorders. Journal of Nervous and Mental Disease, 204(12), 894–902. 10.1097/NMD.0000000000000585 [DOI] [PubMed] [Google Scholar]
  3. American Psychological Association, Presidential Task Force on Evidence-Based Practice (2006). Evidence-based practice in psychology. American Psychologist, 61, 271–285. doi: 10.1037/0003-066X.61.4.271 [DOI] [PubMed] [Google Scholar]
  4. Barber JP. Gallop R, Crits-Christoph P, Frank A , Thase ME, Weiss RD & Connolly Gibbons MB. (2006). The role of therapist adherence, therapist competence, and alliance in predicting outcome of individual drug counseling: Results from the National Institute Drug Abuse Collaborative Cocaine Treatment Study. Psychotherapy Research, 16, 229–240, doi: 10.1080/10503300500288951 [DOI] [Google Scholar]
  5. Barber JP, Zilcha-Mano S, Gallop R, Barret M, McCarthy KS, & Dinger U. (2014). The associations among improvement and alliance expectations, alliance during treatment, and treatment outcome for major depressive disorder. Psychotherapy Research, 24(3), 257–268. 10.1080/10503307.2013.871080 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Barnicot K, Gonzalez R, McCabe R, & Priebe S. (2016). Skills use and common treatment processes in dialectical behaviour therapy for borderline personality disorder. Journal of Behavior Therapy and Experimental Psychiatry, 52, 147–156. doi: 10.1016/j.jbtep.2016.04.006 [DOI] [PubMed] [Google Scholar]
  7. Bordin ES. (1979). The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy: Theory, Research & Practice, 16(3), 252–260. 10.1037/h0085885 [DOI] [Google Scholar]
  8. Brattland H, Koksvik JM, Burkeland O, Klöckner CA, Lara-Cabrera ML, Miller SD, Wampold B, Ryum T, & Iversen VC. (2019). Does the working alliance mediate the effect of routine outcome monitoring (ROM) and alliance feedback on psychotherapy outcomes? A secondary analysis from a randomized clinical trial. Journal of Counseling Psychology, 66(2), 234–246. 10.1037/cou0000320 [DOI] [PubMed] [Google Scholar]
  9. Crits-Christoph P, Gallop R, Gaines A, Rieger A, & Connolly Gibbons MB. (2020). Instrumental variable analyses for causal inference: Application to multilevel analyses of the alliance–outcome relation. Psychotherapy Research. 30, 53–67. 10.1080/10503307.2018.1544724 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Cuijpers P, Li J, Hofmann SG, & Andersson G. (2010). Self-reported versus clinician-rated symptoms of depression as outcome measures in psychotherapy research on depression: a meta-analysis. Clinical Psychology Review, 30(6), 768–778. [DOI] [PubMed] [Google Scholar]
  11. Del Re AC, & Hoyt WT. (2010). MAc: Meta-analysis with correlations. R package version 1.1.1.
  12. Del Re AC, Flückiger C, Horvath AO, Symonds D & Wampold BE. (2012). Therapist effects in the therapeutic alliance–outcome relationship: A restricted-maximum likelihood meta-analysis. Clinical Psychology Review, 32(7), 642–649, 10.1016/j.cpr.2012.07.002 [DOI] [PubMed] [Google Scholar]
  13. DeRubeis RJ, Brotman MA & Gibbons CJ. (2005). A conceptual and methodological analysis of the nonspecifics argument. Journal of Clinical Psychology, Science and Practice. 12(2), 174–183, 10.1093/clipsy/bpi022 [DOI] [Google Scholar]
  14. Egger M, Smith GD, Schneider M, & Minder C. (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315(7109), 629–634. 10.1136/bmj.315.7109.629 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Fakhoury WK, White I, Priebe S, & PLAO Study Group. (2007). Be good to your patient: how the therapeutic relationship in the treatment of patients admitted to assertive outreach affects rehospitalization. The Journal of Nervous and Mental Disease, 195(9), 789–791. 10.1097/NMD.0b013e318142cf5e [DOI] [PubMed] [Google Scholar]
  16. Flückiger C, Del Re AC, Horvath AO, Symonds D, Ackert M & Wampold BE. (2013). Substance use disorders and racial/ethnic minorities matter: A meta-analytic examination of the relation between alliance and outcome. Journal of Counseling Psychology, 60, 610–616. doi: 10.1037/a0033161 [DOI] [PubMed] [Google Scholar]
  17. Flückiger C, Del Re AC, Wampold BE, & Horvath AO. (2018). The alliance in adult psychotherapy: A meta-analytic synthesis. Psychotherapy, 55(4), 316–340. 10.1037/pst0000172 [DOI] [PubMed] [Google Scholar]
  18. Flückiger C, Del Re AC, Wampold BE, Symonds D & Horvath AO. (2012). How central is the alliance in psychotherapy? - A multilevel longitudinal meta-analysis. Journal of Counseling Psychology. 59(1), 1zero-17. doi: 10.1037/a0025749 [DOI] [PubMed] [Google Scholar]
  19. Flückiger C, Hilpert P, Goldberg S, Caspar F, Wolfer C, Held J & Vîslă A. (2019). Investigating the impact of early alliance scores on predicting subjective change at post treatment: An evidence-based souvenir of overlooked clinical perspectives. Journal of Counseling Psychology, 66, 613–625. 10.1037/cou0000336 [DOI] [PubMed] [Google Scholar]
  20. Flückiger C, Rubel J, Del Re AC, Horvath AO, Wampold BE, …, Barber J. (2020). The reciprocal relation between alliance and early treatment symptoms: An individual participant data meta-analysis. Manuscript submitted for publication. [DOI] [PubMed]
  21. Hatcher RL. (2010). Alliance theory and measurement In Muran JC & Barber JP. (Eds.), The therapeutic alliance: An evidence-based guide to practice (pp. 7–28). New York, NY, US: The Guilford Press. [Google Scholar]
  22. Hersoug AG, Høglend P, Gabbard GO, & Lorentzen S. (2013). The combined predictive effect of patient characteristics and alliance on long-term dynamic and interpersonal functioning after dynamic psychotherapy. Clinical Psychology & Psychotherapy, 20(4), 297–307. 10.1002/cpp.1770 [DOI] [PubMed] [Google Scholar]
  23. Higgins JPT, & Thompson SG. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21(11), 1539–1558. 10.1002/sim.1186 [DOI] [PubMed] [Google Scholar]
  24. Hofmann SG & Hayes SC. (2019). The future of intervention science: Process-based therapy. Clinical Psychological Science, 7(1), 37–50. 10.1177/2167702618772296 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Horvath AO. (2018). Research on the alliance: Knowledge in search of a theory. Psychotherapy Research. 28(4), 499–516. 10.1080/10503307.2017.1373204 [DOI] [PubMed] [Google Scholar]
  26. Horvath AO, Del Re AC, Flückiger C, & Symonds D. (2011). Alliance in individual psychotherapy. Psychotherapy, 48(1), 9–16. 10.1037/a0022186 [DOI] [PubMed] [Google Scholar]
  27. Huppert JD, Fabbro A, & Barlow DH. (2006). Evidence-based practice and psychological treatments In Goodhart CD, Kazdin AE, & Sternberg RJ. (Eds.), Evidence-based psychotherapy: Where practice and research meet (pp. 131–152). Washington, DC: American Psychological Association. [Google Scholar]
  28. Kazantzis N, Dattilio FM, & Dobson KS. (2017). The therapeutic relationship in cognitive behavior therapy: A clinician’s guide. New York: Guilford. [Google Scholar]
  29. Kivlighan DM Jr. (2007). Where is the relationship in research on the alliance? Two methods for analyzing dyadic data. Journal of Counseling Psychology, 54(4), 423–433. 10.1037/0022-0167.54.4.423 [DOI] [Google Scholar]
  30. Luborsky L. (1976). Helping alliances in psychotherapy In Cleghhorn JL. (Ed.), Successful psychotherapy (pp. 92–116). New York, NY: Brunner/Mazel. [Google Scholar]
  31. Marmarosh C, & Kivlighan D. (2012). Relationships among client and counselor agreement about the working alliance, session evaluations, and change in client symptoms using response surface analysis. Journal of Counseling Psychology, 59, 352–367. doi: 10.1037/a0028907 [DOI] [PubMed] [Google Scholar]
  32. Marziali E, Munroe-Blum H, & McCleary L. (1999). The effects of the therapeutic alliance on the outcomes of individual and group psychotherapy with borderline personality disorder. Psychotherapy Research, 9(4), 424–436. 10.1080/10503309912331332841 [DOI] [Google Scholar]
  33. Meyer B, Pilkonis PA, Krupnick JL, Egan MK, Simmens SJ, & Sotsky SM. (2002). Treatment expectancies, patient alliance and outcome: Further analyses from the National Institute of Mental Health Treatment of Depression Collaborative Research Program. Journal of Consulting and Clinical Psychology, 70(4), 1051–1055. 10.1037/0022-006X.70.4.1051 [DOI] [PubMed] [Google Scholar]
  34. Muran JC & Barber JP. (2010). The therapeutic alliance: An evidence-based guide to practice. New York, NY: Guilfors Press. [Google Scholar]
  35. Norcross JC & Goldfried MR. (Eds.). (2019). Handbook of Psychotherapy Integration (3rd ed.). 10.1093/med-psych/9780190690465.001.0001 [DOI] [Google Scholar]
  36. Norcross JC & Lambert MJ. (2019). Psychotherapy relationsships that work (Third Edition, Vol. I). New York, NY: Oxford University Press. [Google Scholar]
  37. O’Connor S, Kivlighan DM Jr., Hill CE, & Gelso CJ. (2019). Therapist–client agreement about their working alliance: Associations with attachment styles. Journal of Counseling Psychology, 66(1), 83–93. 10.1037/cou0000303 [DOI] [PubMed] [Google Scholar]
  38. Probst GH, Berger T & Flückiger C. (2019). The alliance-outcome relation in internet-based interventions for psychological disorders: A correlational meta-analysis. Verhaltenstherapie, 29, 182–195, 10.1159/000503432 [DOI] [Google Scholar]
  39. Pustejovsky JE & Tipton E. (2017). clubSandwich: Cluster-robust (Sandwich) Variance Estimators with small-sample corrections. Statistical software package version 0.3.5. https://cran.r-project.org/web/packages/clubSandwich/index.html
  40. Ribeiro E, Ribeiro AP, Gonçales MM, Horvath AO, & Stiles WB. (2013). Howe collaboration in therapy becomes therapeutic: The therapeutic collaboration coding system. Psychology and Psychotherapy: Theory, Research and Practice, 86, 294–314. 10.1111/j.2044-8341.2012.02066.x [DOI] [PubMed] [Google Scholar]
  41. Siev J, Huppert J, & Chambless DL. (2009). The dodo bird, treatment technique, and disseminating empirically supported treatments. The Behavior Therapist, 32, 69–75. [Google Scholar]
  42. Viechtbauer W & Cheung MWL. (2010). Outlier and influence diagnostics for meta-analysis. Research Synthesis Methods 1(2), 112–25. [DOI] [PubMed] [Google Scholar]
  43. Wampold BE, & Imel ZE. (2015). The great psychotherapy debate: The evidence for what makes psychotherapy work (2nd ed.). New York, NY, US: Routledge. [Google Scholar]
  44. Webb CA, DeRubeis RJ & Barber JP. (2010). Therapists adherence / competence and treatment outcome: A meta-analytic review. Journal of Consulting and Clinical Psychology, 78, 200–211. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Weck F, Richtberg S, Jakob M, Neng JM, & Höfling V. (2015). Therapist competence and therapeutic alliance are important in the treatment of health anxiety (hypochondriasis). Psychiatry Research, 228(1), 53–58. 10.1016/j.psychres.2015.03.042. [DOI] [PubMed] [Google Scholar]
  46. Weitz ES, Hollon SD, Twisk J, Van Straten A, Huibers MJ, David D, ... & Faramarzi M. (2015). Baseline depression severity as moderator of depression outcomes between cognitive behavioral therapy vs pharmacotherapy: an individual patient data meta-analysis. JAMA Psychiatry, 72(11), 1102–1109. [DOI] [PubMed] [Google Scholar]
  47. Wiernik BM, Kostal JW, Wilmot MP, Dilchert S, & Ones DS. (2017). Empirical benchmarks for interpreting effect size variability in meta-analysis. Industrial and Organizational Psychology: Perspectives on Science and Practice, 10, 472–479, 10.1017/iop.2017.44 [DOI] [Google Scholar]
  48. Zilcha-Mano S. (2017). Is the alliance really therapeutic? Revisiting this question in light of recent methodological advances. The American Psychologist, 72(4), 311–325. 10.1037/a0040435 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental Material 1
NIHMS1554404-supplement-Supplemental_Material_1.pdf (941.8KB, pdf)
Supplemental Material 2
NIHMS1554404-supplement-Supplemental_Material_2.docx (14.4KB, docx)

RESOURCES