Abstract
The course of diverse dimensions of the therapeutic alliance as seen by the therapy participants was examined in two studies. In Study 1, use of the Penn Helping Alliance Method, Therapeutic Alliance Rating System, and Vanderbilt Psychotherapy Process Scale at the 3rd, 10th, and next-to-last therapy sessions revealed significant increases over time for therapists' mean scores on 5 of the 12 subscales. Consistency of participants' scores over time varied depending on rater perspective (therapist or client) and time interval. In Study 2, the Helping Alliance Questionnaire, Working Alliance Inventory, and California Psychotherapy Alliance Scales showed no significant change in participants' average alliance scores on the 9 subscales from the 5th to 10th sessions, and their same-scale ratings were in general consistent. Few differences were observed between therapists' and clients' average alliance ratings, but the relationship was generally perceived differently within dyads.
Keywords: Therapeutic Alliance, Therapists' Perceptions of Therapy, Patients' Perceptions of Therapy
The therapeutic alliance of client and therapist has been the focus of considerable research attention over the past 15 to 20 years. Bordin's1 writings on the generalizability across therapy systems of this originally analytic concept, together with increasing interest in the alliance as an integrative therapeutic factor,2–4 stimulated intensive research into the role of the alliance in therapy (see Horvath and Symonds5 for a review of the alliance-outcome relationship). More recently, attention has turned to the examination of other relevant issues, including the developmental course of the alliance over therapy. The stability or variability of the alliance has important clinical as well as empirical implications, including whether increased focus on the relationship is warranted at specific time points in therapy and whether commonly used one-time assessments can be assumed to represent the quality of the alliance throughout therapy.
From a conceptual standpoint, Luborsky6 viewed the helping alliance as a dynamic, rather than static, phenomenon that is responsive to the changing demands of different phases of therapy. The experience of the therapist as helpful and supportive, one aspect of the alliance, is held to be more evident at the beginning of therapy, whereas the sense of shared responsibility regarding treatment goals, a second aspect, is more typical of later phases of treatment. According to Bordin,7 the working alliance—defined in terms of mutuality of goals and tasks supported by the client-therapist bond—grows and deepens with time as inevitable strains or disruptions are worked through and repaired.
Also of the view that the alliance varies over time, Gelso and Carter8 proposed that its course during therapy—particularly in time-limited interventions—involves a weakening after an initial development, followed in successful therapy by an increase to earlier, high levels. Along similar lines, Horvath et al.9 posited an initial phase of development for the alliance, held to occur within the first five therapy sessions (and probably peaking during the third session), followed by a second, more critical phase, during which the therapist challenges maladaptive patterns, the effect of which is a weakening or rupturing of the alliance that must be repaired if therapy is to continue successfully. On the other hand, Greenberg10 challenged the high-low-high pattern suggested by these authors, maintaining that in successful therapies the alliance either rises or holds a steady value over time. In sum, although consensus has not been reached on the specific pattern of development, several theorists view the alliance, or individual aspects of the alliance, as shifting over the course of therapy.
Relatively few studies have specifically investigated the time course of the alliance. Three recent studies tested the rupture/repair, or U-shaped pattern, proposed by Bordin7 and Gelso and Carter8 to characterize the course of the alliance in therapy. Horvath and Marx,11 examining the working alliance (as assessed by the Working Alliance Inventory; WAI) at the session-by-session level in 4 clients and their 2 therapists, found that therapists' combined ratings increased over the first 5 of 10 therapy sessions and then declined slightly, to recover marginally after the 7th session. Although these results suggested a cyclical developmental model, clients' alliance ratings showed, to the contrary, a linear increase over the therapy. Golden and Robbins,12 in a similar study, reported the reverse pattern: the therapist's working alliance scores gradually increased over the 12 therapy sessions, whereas his two clients' scores were lowest during the middle phase (5th–8th sessions) of therapy—and in one client, scores remained at this level on the task and goal alliance subscales. Kivlighan and Shaughnessy,13 in contrast, demonstrated that a direct linear pattern, as opposed to a quadratic (cyclical) pattern, best accounted for both client and therapist working alliance scores over therapy. Sexton et al.,14 although not specifically testing the cyclical model of development, reported, in a sequential study that focused on the client's perspective, that the working alliance was largely determined by the first therapy session.
A high-low-high pattern of alliance development does not appear, then, to be strongly supported by the data. However, there is some evidence of different developmental patterns over time among therapists' and clients' alliances.
Some studies, though not focused specifically on the course of the alliance, have employed multiple assessments of the client-therapist relationship or its components and have tested for time differences. Although findings remain inconclusive, several results suggest differential temporal patterns for specific components of the therapeutic relationship, with positive-toned features showing greater variation than negative facets. For example, using the Therapeutic Alliance Rating System, Marziali15 found early session mean ratings for positive patient and therapist contributions to the alliance to be significantly lower than final session scores; negative contributions, however, showed no change over the course of therapy. Similarly, Klee et al.,16 examining positive patient alliance contributions, reported an increase in these from early to late sessions. Using a factor-analytic version of the same measure (the California Therapeutic Alliance Rating System), Marmar et al.,17 however, found both positive and negative patient contributions to the alliance, such as commitment and hostility, to increase over therapy, whereas therapist negative behaviors did not vary. An increase from early to late sessions in clients' positive scores with no change in negative scores was also reported by Luborsky et al.18 in more improved patients, using the Penn Helping Alliance (counting signs) Method. In contrast, in Hartley and Strupp,19 last-session, compared with first-session, alliance ratings of both positive and negative patient components (e.g., motivation, resistance), as assessed by the Vanderbilt Therapeutic Alliance Scale, showed a significant decrease regardless of outcome status. Still other studies, using these and other measures, found no differences in levels of the alliance or its components attributable to phase of therapy.20–23
The consistency of the alliance has been examined in a few studies. Moderate to high correlations between early and late sessions have been reported for the Penn scale (counting signs and rating methods) and the WAI scale,18,22,24 suggesting relative stability of these variables. Other results,25 however, indicated substantial changes in the Penn characteristics between these time points.
Clearly, more work is needed on how the components of the therapeutic relationship evolve over time. Furthermore, the above results are, in general, based on independent observer assessments of the therapeutic alliance, which represent but one perspective on the client-therapist relationship. Indeed, different evaluative sources (observer, therapist, client) are viewed as providing unique, nonsubstitutable information on the relationship.3,26,27 Several studies have in fact substantiated the low convergence of therapists' and clients' perceptions of alliance variables,11,12,23,28,29 supporting separate study of the therapy partners' alliances.
Given that the therapy participants are privy to a wealth of information at the emotional and empathic levels30 and may provide the more authentic versions of the quality of the treatment relationship,31 participants' evaluation of the alliance warrants more attention. Empirical data on the time course of the alliance as seen by the therapy participants have primarily involved the WAI,2 which focuses on therapist-client agreement.32 Little is known about the course of other alliance ratings and characteristics. Despite some preliminary work,23,32–34 there is insufficient evidence to conclude the comparability of different alliance measures and components. The use of multiple measures, in capturing different facets of a construct of interest, may also contribute to a broader understanding of the underlying construct.35
The goal of the current study, then, was to track the course of different facets of the therapeutic alliance across therapy, from the perspective of the therapy participants. Two studies are presented that examined participants' perceptions by using different alliance measures and assessment times. The first study examined the alliance at the 3rd, 10th, and next-to-last sessions. The second, later study, although not addressing the developmental pattern of the alliance, allowed for study of the briefer time segment from the 5th to 10th therapy sessions. To provide a more thorough account of the time course of relationship components, perceptions were assessed both between and within participants.
STUDY 1
Participants and Procedure
Participants were 27 Caucasian therapist-client dyads at a university consultation service serving a large francophone community and used as a training facility for graduate students in clinical psychology. This sample was drawn from the 47 participant dyads of a previous study, the report of which includes additional details on the constitution of the sample.33 However, none of the analyses effected in the present study have been previously reported. The length of therapy ranged from 15 to 37 sessions, with a mean of 26±6.44 sessions. Therapy length was limited to the duration of the academic year, ending with the summer semester (although clients could continue therapy the following year with another therapist, if desired). Clients were seen on a once-a-week basis. These clients were 20 women and 7 men, with a mean age of 30.15±7.61 years. Means and standard deviations are reported.
Sixty-seven percent of clients were single, 15% were married, and 18% were separated or divorced. Sixty-five percent had completed junior college or had some university education. Fifty-two percent of clients were unemployed, 18% were students, and 30% held skilled or semiskilled jobs. Clients' overall psychological functioning, as rated by trainees' supervisors on the Global Assessment Scale (GAS),36 fell within the range of “moderate symptoms or generally functioning with some difficulty” (mean=59.19±11.36) and their diagnoses (n=26) included anxiety-related (38%), personality-related (31%), and psychosexual-related (8%) disorders, as well as interpersonal problems (23%). Self-ratings on the Psychiatric Symptom Index (PSI), a measure of self-reported psychiatric status,37 indicated the presence of perceived high symptomatology (mean=39.37±19.21). Fourteen clients had had prior therapy with other therapists, mostly at this same consultation service. Clients were seen by 17 master's-level trainees in a first-year clinical psychology practicum (12 women and 5 men; mean age 31.35±6.14 years). Nine therapists saw 1 client each, six therapists saw 2 clients, and two saw 3 clients. Therapists' therapeutic orientation, based on their supervisors' self-described approach, was predominantly humanistic or humanistic-existential (70% of therapists); other orientations included bioenergetic (18%) and psychodynamic (12%).
Consenting clients, who had been informed at phone registration of an ongoing research project, were met individually prior to their initial appointment by a research assistant to complete demographic and consent forms and the PSI as well as other outcome measures. Alliance ratings were subsequently collected on an individual basis by a research assistant following clients' 3rd, 10th, and next-to-last sessions. Therapists completed alliance measures at assessment phases corresponding to those of clients. Because of administrative slips, cancellations, or other delays, in a few cases client or therapist ratings of adjacent sessions were collected. Trainees' supervisors, all clinical psychology faculty members, provided GAS ratings and diagnoses shortly after trainees' initial sessions with clients (on the basis of verbal and audiotaped material presented at weekly supervision). Clients, therapists, and supervisors completed various outcome measures, including the GAS and PSI, generally at 2 weeks following termination. Significant improvement from intake to termination was observed on all measures in the original sample.33
Alliance Measures
The Penn Alliance Rating Method (Penn)22 comprises ten 10-point Likert-type items, six of which measure the patient's experience of receiving help or a helpful attitude from the therapist (HA 1) and four of which measure the patient's experience of being involved in a joint or team effort with the therapist (HA 2). Evidence of the scale's internal consistency and predictive validity is reported by the authors.22
The Vanderbilt Psychotherapy Process Scale (VPPS)21 comprises 44 5-point Likert-type items assessing seven dimensions of therapist and patient attitudes and behaviors: Patient Exploration (PEXP), Therapist Exploration (TEXP), Patient Participation (PPAR), Patient Hostility (PHOS), Therapist Warmth and Friendliness (TWFR), Negative Therapist Attitude, and Therapist Directiveness (TDIR). Adequate internal consistencies of individual subscales, and the predictive validity of a broad dimension of “patient involvement” (comprising two patient subscales) have been demonstrated.21,38 In the current study, the Penn and VPPS items, originally developed for external observer evaluation, were reformulated, following Marziali,15 by altering pronouns to suit the perspectives of client and therapist. Unfortunately, the self-report versions (client and therapist) of the Penn items39 were not available in a French version at the time Study 1 was conducted.
The Therapeutic Alliance Rating System (TARS),15 in patient and therapist versions, comprises 42 items, of which 21 evaluate the therapist's positive and negative contributions to the alliance (TPOS, TNEG, respectively) and 21 evaluate the patient's positive and negative contributions (PPOS, PNEG, respectively). Marziali15 reported satisfactory internal consistencies for the subscales, as well as good discriminant and predictive validity. In the present study, a 5-point Likert-type scale was substituted for the original visual analog format, using the original end points (“not at all” and “a lot”).
This study employed the French versions of the three alliance scales used in Bachelor,33 for which adequate reliabilities (coefficient alpha) were obtained (ranging from 0.68 to 0.87) for the individual subscales excepting the VPPS scales for Negative Therapist Attitude, which was deleted from further analysis, and Patient Hostility (α=0.54).
Results
Given the variation in length of treatment across clients, preliminary analyses were conducted to ascertain the effect of number of sessions on participants' alliance perceptions. No significant Pearson (Bonferroni-adjusted, P<0.00l) correlations were found between individual client-rated or therapist-rated alliance characteristics and number of sessions at any of the three assessment points. Further, when we divided the client sample into two groups, with 21±3.23 (n=14) and 31.92±3.40 (n=13) mean therapy sessions, respectively, average levels of the individual alliance characteristics, either client- or therapist-rated, were not found to differ significantly between the two groups, regardless of assessment time.
Taken together, these results indicate that treatment length did not influence participants' alliance perceptions. Thus, the two client (and corresponding therapist) subsamples were pooled in further analyses.
Preliminary analyses further showed no significant differences between male and female subjects either on the demographic variables or on the variable of prior consultation. Clients' gender and age were also unrelated to their alliance ratings at all assessment points. In addition, pretherapy GAS and PSI scores, as well as client diagnoses (categories involving 23% or more of clients), were unrelated to either therapists' or clients' 3rd-session alliance assessments (all r- and F-values nonsignificant at the Bonferroni-adjusted level, P<0.001).
To determine whether alliance ratings differed over time and between rater perspectives, two-factor analyses of variance, with time (3rd vs. 10th and next-to-last sessions—a repeated measure) and group (client vs. therapist) as the independent variables, were performed on each of the 12 alliance variables. Observed means, standard deviations, and F-ratios are presented in Table 1.
TABLE 1. Means, standard deviations, and comparison of alliance subscales at three assessment times (N=27).
As can be seen from Table 1, 8 of the 12 alliance dimensions demonstrated significant time effects (using a Bonferroni-adjusted significance level of 0.004 to counter familywise error) in the combined ratings of clients and therapists: the Penn HA 1 (therapist-offered helpfulness and support) and HA 2 (joint collaboration in the work of therapy); the VPPS TEXP (Therapist Exploration) and TDIR (Therapist Directiveness); PPAR (Patient Participation) and PHOS (Patient Hostility); and the TARS PPOS (patient positive contributions) and TPOS (therapist positive contributions). Significant effects for group were found on the VPPS PEXP (Patient Exploration) and on the TARS TNEG (therapist negative contributions) scales. No group-by-time interactions were observed.
Repeated-measures analyses of variance performed separately on clients' and therapists' ratings on the eight scales displaying time effects indicated that the observed overall time effects were mainly attributable to changes in therapists' perceptions. As indicated by post hoc contrasts, therapists' mean scores on the five scales HA 1, HA 2, TEXP, PPOS, and TPOS showed significant differences between the 3rd and next-to-last sessions, and their mean scores on the latter three scales (TEXP, PPOS, TPOS) also differed significantly between the 3rd and 10th sessions. All differences were in the direction of increased scores. No significant effects for time were observed for clients' ratings of these eight alliance dimensions. Despite observed significant overall time effects for the VPPS TDIR, PPAR, and PHOS scales, no significant effects were found once clients' and therapists' scores were examined separately. With regard to observed main effects for group, Newman-Keuls comparisons indicated that clients gave higher ratings than therapists on the VPPS PEXP and lower ratings on the TARS TNEG at each of the 3 assessment points.
As shown in Table 2, Pearson correlations computed between therapists' 3rd- and 10th-session same-scale ratings on each of the 12 alliance dimensions revealed significant coefficients for five scales (Bonferroni-adjusted level of 0.002): the VPPS PPAR and TWFR and the TARS PPOS, PNEG, and TPOS; the correlation for the Penn HA 1 was near-significant. Between the 10th and next-to-last sessions, seven same-scale correlations were significant: the Penn HA 1, the VPPS PEXP, TEXP, and TWFR, and the TARS PPOS, TPOS, and TNEG; the same-scale correlations for HA 2 and PNEG were near-significant. Thus, across the two time periods, consistency was observed for ratings of TPOS, PPOS, and TWFR, while ratings of HA 1 and PNEG were fairly consistent. Between the 3rd and next-to-last sessions, only two same-scale correlations (HA 1 and TWFR) were significant.
TABLE 2. Pearson same-scale correlations between assessment points (N=27).
As also shown in Table 2, correlational analyses of clients' same-scale ratings on each of the 12 alliance dimensions indicated six significant Pearson correlations between ratings at the 3rd and 10th sessions: HA 2, PEXP, TEXP; PHOS, PPOS, and TPOS; the correlation for TWFR was near-significant. Five significant same-scale correlations were found between 10th and next-to-last session ratings: HA 1, HA 2, PEXP, TWFR, TPOS; the correlation for PPOS was near-significant. Across both time periods, consistency of ratings was observed for HA 2, PEXP, and TPOS; ratings of TWFR and of PPOS were fairly consistent. Between the 3rd and the next-to-last sessions, only three correlations (PEXP, TEXP, TPOS) computed on same-scale ratings proved significant.
STUDY 2
Participants and Procedure
Participants were 30 Caucasian French-speaking therapist-client dyads drawn from three different sites (the same university consultation service as in Study 1, site A, n=8; a community services center, site B, n=10; and private practice, site C, n=12). Clients were 18 women and 12 men (mean age=35.10±9.47 years), of whom 40% were single, 30% were married or lived with a partner, and 30% were separated, divorced, or widowed. Forty-four percent of clients held skilled or semiskilled jobs, 23% held professional or minor professional positions, and the remainder were students or unemployed. Fifty-seven percent held junior college or university diplomas. Diagnoses of the clients (n=29) included anxiety-related (35%), dysthymic-related (10%), and personality-related (17%) disorders, as well as interpersonal problems (38%). Clients' level of psychological functioning (GAS),36 as provided by their therapists, fell in the range of “moderate symptoms or generally functioning with some difficulty” (mean=59.46±8.37). Clients' self-reported symptomatic status (PSI)37 suggested the presence of perceived high symptomatology (mean=42.34±16.90). Clients were seen, in general, on a weekly basis, and were in therapy for an average of 18±5.63 sessions (range 9–29). Half of these clients had previous counseling experience.
Therapists, who were solicited either individually or at staff meetings to collaborate in a study on the client-therapist relationship, were 12 women and 8 men (mean age 35.9±9.77 years), representing 54%, 68%, and 86% of solicited therapists in sites A, B, and C, respectively. The majority (65%) described their theoretical orientation as humanistic or humanistic-existential; other approaches included Gestalt, eclectic, bioenergetic, and transpersonal. Fifteen therapists saw 1 client, two therapists saw 2 clients, one therapist saw 3 clients, and two therapists saw 4 clients each. Three therapists were licensed psychologists (2.5–17 years post-M.A. experience), one was a licensed social worker, one a registered nurse (11 and 15 years of counseling experience, respectively), 13 were clinical or counseling psychology practicum students (11 at the master's and 2 at the bachelor's level), and two were screened volunteer helpers at the community services center, with undergraduate university diplomas but no formal counseling training. Prior results indicated no significant relationship between therapist experience level and the alliance dimensions currently examined in a subset of this sample (A. Bachelor, unpublished report).
Consenting therapists, who were asked to recruit from one to three incoming clients (two therapists provided more) to participate, gave clients at their initial therapy session the research packet (containing a consent form assuring anonymity and confidentiality of responses, a demographic information sheet, and the PSI), to be returned in a sealed envelope at the following session. Virtually all solicited clients in sites B and C, and 58% in site A, agreed to participate. At the 5th and 10th therapy sessions, therapists gave participating clients the alliance measures in a sealed packet that contained instructions to return the completed questionnaires, sealed in the accompanying envelope, at their next session. Thirty-five clients provided data for the two assessment times. Of these 35 clients, 5 outliers, who had an average of 55 therapy sessions, were excluded, thus yielding the current sample of 30 clients. Therapists completed the GAS and provided client diagnoses following the 2nd therapy session, and they completed the alliance measures after the 5th and 10th sessions. Pre–post differences in clients' self-rated symptomatic status (PSI) and in therapists' ratings of clients' level of functioning (GAS)—both measures completed, in general, 2 weeks following termination—indicated significant client change (t=3.27, df=27, P<0.01, and t=8.37, df=24, P<0.001, respectively). Both clients and therapists rated therapy as “quite helpful” on a 9-point post-therapy index of overall change40 (ranging from “extremely harmful” to “extremely helpful”; means: 7.28±1.22 and 7.11±0.87, respectively).
Alliance Measures
The Helping Alliance Questionnaire (HAQ)39 assesses two types of helping alliance in 11 items, 8 of which reflect perceived helpfulness and support of the therapist (HA 1) and 3 of which reflect the patient's collaboration with the therapist toward the goals of treatment (HA 2). Its therapist counterpart, the Therapist Facilitating Behavior Questionnaire, is an 11-item scale assessing the same two types of alliance. Both scales were shown to predict therapy outcome as assessed by various indices.41 For purposes of statistical calculations, the original scoring scheme (+3 to –3) was transformed into a positive (1 to 6) scale.
The Working Alliance Inventory (WAI), short form,42 is a 12-item, 7-point, Likert-type measure of the three alliance dimensions of agreement on tasks (TASK), agreement on goals (GOAL), and development of bonds (BOND), with parallel forms for client and therapist that consist of each of the four highest-loading items from the original3 12-item subscales. Tracey and Kokotovic42 report results that support the construct validity of the WAI-short form, as well as high internal consistency estimates for its three subscales.
The California Psychotherapy Alliance Scales (CALPAS)43 (C. Marmar and L. Gaston, unpublished), patient and therapist versions, comprise 24 7-point items assessing four scales: Patient Working Capacity (PWC), Patient Commitment (PC), Working Strategy Consensus (WSC), and Therapist Understanding and Involvement (TUI). Gaston43 reported satisfactory reliability (α=0.83) for the total patient scale, as well as evidence of criterion-related validity.
This study used the French translation of the CALPAS patient scale (C. Marmar and L. Gaston, unpublished) and French translations of the other alliance scales made by two experienced bilingual clinicians. Estimates of internal consistencies of the scales, computed on 5th-session ratings of the current sample, were adequate, ranging, for therapist ratings from 0.66 to 0.91, and for client ratings from 0.69 to 0.93 with the exception of the HAQ HA 2 and the CALPAS PWC scales (means=0.57 and 0.46, respectively). The latter subscale, as well as the other three CALPAS subscales, showed comparatively higher reliabilities in the current study than the original estimates reported.43 The low estimate for HA 2 could be a result of the limited number of items (three). Results obtained with regard to these two subscales should thus be viewed with caution.
Results
As in Study 1, because of the range in number of sessions across clients, preliminary analyses were conducted to assess the effect of number of sessions on participants' alliance ratings. No significant (Bonferroni-adjusted, P<0.003) correlations were observed between clients' or therapists' alliance ratings and number of sessions at either assessment point. Further, when we divided the client sample into two groups, with a mean of 21.62±4.77 sessions (n=16) and 13.29±2.13 sessions (n=14), respectively, mean client and therapist ratings of the two groups on the nine alliance characteristics were not found to differ significantly, either at the 5th or the 10th therapy session. The two client subsamples (and the corresponding therapist subsamples) were thus combined in further analyses.
Preliminary analyses also indicated no significant differences among the three research sites with regard to clients' age, sex, and marital status, as well as initial psychological status (PSI and GAS scores). Also, male and female clients did not differ with regard to age, marital status, GAS, or PSI scores. Further, clients' gender, age, and diagnoses (categories involving 17% or more of clients) were unrelated to either therapist- or client-rated alliance dimensions (all r- and F-values nonsignificant at the Bonferroni-adjusted level of 0.005).
Two-factor analyses of variance, with one repeated measure (time) and one between-subjects factor (client vs. therapist), conducted on each of the nine alliance variables and using a Bonferroni-corrected alpha of 0.005, yielded a near-significant (P=0.006) time effect (combined client and therapist ratings) for the CALPAS PWC scale, and significant group effects for the CALPAS TUI and WSC scales. No group-by-time interactions were observed. Scale means, standard deviations, and F-ratios are reported in Table 3.
TABLE 3. Means, standard deviations, and comparison of alliance subscales at two assessment times.
Tests of the difference between means, conducted separately on clients' and therapists' PWC scores, indicated that the overall near-significant effect was due to a difference in therapists' ratings between the 5th and 10th sessions (t=2.90, df=29, P<0.02). With regard to observed group effects, inspection of mean scores indicated that clients gave higher ratings than therapists on the two variables of TUI and WSC both at the 5th and 10th therapy sessions.
Pearson correlations computed between participants' 5th- and 10th-session same-scale ratings of the nine alliance characteristics revealed that among therapists, six of the nine same-scale coefficients proved significant: the Penn HA 1, the WAI Goal and Bond, and the CALPAS TUI, PC, and WSC (range 0.51–0.72, all P<0.005, Bonferroni-adjusted). Same-scale coefficients for the Penn HA 2, the WAI Task, and the CALPAS PWC were nonsignificant. Among clients, all but one (the CALPAS WSC) of the nine same-scale associations were significant (range 0.52–0.76).
DISCUSSION
The current findings support, in general, the view that individual characteristics of the therapeutic relationship evolve differentially over time. The variability or stability of relationship characteristics was specifically related to rater perspective (therapist or client) and time phase, as well as data-analytic procedure (between- or within-subjects analyses).
Therapists' Alliance Perceptions Across Therapy
In Study 1, therapists' average levels of five positive-toned relationship characteristics—their helpfulness and support (HA 1), joint work efforts with clients (HA 2), and exploration of underlying dynamics (TEXP), as well as their own and their clients' positive attitudes and affect (TPOS, PPOS)—increased significantly from the near-beginning to the near-conclusion of therapy. The latter three characteristics (TEXP, TPOS, PPOS) also showed significant increases from the 3rd to the 10th therapy session. Contrary to theory,6 the alliance component of helpfulness and support (HA 1), and not only the sense of shared responsibility in achieving treatment goals (HA 2), was stronger later in therapy, as judged by therapists. Also, contrary to data drawn from observer ratings,21 exploratory, uncovering interventions (TEXP) also improved over time in the perspective of therapists. Consistent with Marziali,15 positive but not negative therapist and patient attitudes and affect (TNEG, PNEG) increased over time.
Between the 10th and near-final sessions, however, no significant change in therapists' average perceptions was observed. As indicated by the findings of Study 2, which examined the more specific time segment from the 5th to 10th therapy sessions, group changes were not observed in therapists' perceptions after the 5th session (with one exception, clients' working capacity [PWC], e.g., self-disclosure and self-exploration, which tended to improve).
Results of intra-individual analyses, however, indicated differences among therapists in their perceptions over time of most of the relationship characteristics, including those that showed improvement in therapists as a group, particularly between the 3rd and 10th and the 3rd and next-to-last therapy sessions. (In these two time intervals, 50% and 83%, respectively, of the variables of Study 1 were not rated consistently.) When these correlational results were considered together with the findings based on mean ratings, over the 3rd to the 10th therapy sessions, perceptions of their uncovering interventions (TEXP) proved variable among therapists: some perceived such efforts to increase, and others judged these interventions to diminish or remain stable. Similarly, 5 of the other 11 therapist-rated alliance characteristics of Study 1 displayed variation at the individual level over this time interval. Thus, although some therapists retained their initial judgments of clients' collaborative efforts (HA 2), self-exploration behaviors (PEXP), and hostile attitudes (PHOS), as well as their own directive (TDIR) and negative behaviors (TNEG), other therapists viewed these characteristics as decreasing and still others saw them as increasing. On the other hand, the observed mean increases from the 3rd to 10th session in therapists' appraisal of their own as well as their clients' positive attitudes and affects (TPOS, PPOS), proved consistent across therapists. The other four relationship characteristics—perceived client participation (PPAR) and negative attitudes (PNEG) and their own warmth and friendliness (TWFR) as well as helpfulness and support (HA 1)—were stable (or fairly stable as regards HA 1) across therapists over this time interval.
From the 10th session to the near-conclusion of therapy, all therapist-rated characteristics of Study 1 evidenced stability, except for perceived directiveness (TDIR) and client hostility (PHOS), which continued to fluctuate, and perceived client involvement (PPAR), now variable in individual therapists. Stability was also evidenced for most alliance characteristics over the 5th to 10th therapy sessions (Study 2). Three exceptions were perceived collaboration with clients (HA 2), agreement on therapeutic tasks (Task), and client working capacity (PWC).
Between the 3rd and near-final therapy sessions, again taking into account both mean comparisons and intra-individual associations, therapists' perceptions for the most part showed variability, including the dimensions that evidenced improvement in therapists as a group (HA 2, TEXP, TPOS, PPOS; the exception was HA 1 [offered helpfulness and support], which increased across therapists). Similarly, the initial assessments of the other alliance characteristics of Study 1 (clients' self-exploration [PEXP], participation [PPAR], hostility [PHOS], and negative attitudes [PNEG], as well as their own negativeness [TNEG] and directiveness [TDIR]) increased in some therapists and decreased or did not change in others; the exception was perceived warmth and friendliness (TWFR), which was the only alliance component that proved stable across therapists over this interval.
Clients' Alliance Perceptions Across Therapy
With regard to the perceptions that clients entertained of the various facets of the therapeutic relationship assessed in both studies, the finding of no significant change in average levels, regardless of time period, is in line with prior results14,20,22–24 and adds support to the view that the quality of the therapeutic relationship is established early in therapy.19,22,44 The results of intra-individual analyses suggest, however, that this conclusion applies mainly to clients viewed as a group. Similar to the therapists, individual clients revised their perceptions of several facets of the therapeutic relationship over time.
Over the interval encompassing the 3rd to 10th therapy sessions, when mean ratings as well as intra-individual associations were considered, clients showed variability in their perceptions of 5 of the 12 characteristics of Study 1: their therapists' helpfulness, support (HA 1), and directiveness (TDIR); their own active engagement in therapy (PPAR); and their own as well as therapists' negative attitudes (PNEG, TNEG). On the other hand, the following proved stable across clients: initial impressions of their therapists' uncovering interventions (TEXP), positive attitudes and affect (TPOS), their own collaboration in the work of therapy (HA 2), self-exploration efforts (PEXP), and both positive and hostile attitudes (PPOS, PHOS). Perceptions of their therapists' warmth and friendliness (TWFR) also proved fairly stable over this time interval.
From the 10th session to the near-conclusion of therapy, individual clients continued to revise their impressions of their therapists' directiveness (TDIR), their own active participation in therapy (PPAR) and their own as well as their therapists' negative attitudes (PNEG, TNEG). Their own hostility (PHOS), as well as their therapists' uncovering work (TEXP), which were stable prior to the 10th session, now also proved variable in individual clients. The helpfulness and support of their therapist (HA 1), variable prior to the 10th session, stabilized among clients. The five characteristics of perceived therapist warmth and friendliness (TWFR) and other positive contributions (TPOS), as well as their own positive attitudes (PPOS), collaboration (HA 2), and self-observation efforts (PEXP), remained stable. Between the 5th and 10th sessions, as assessed in Study 2, perceptions of 8 of the 9 alliance facets examined also proved stable across clients; the exception was consensus on work strategies (WSC).
As was observed with therapists, clients' assessment of most alliance characteristics at the near-conclusion, as compared with the early relationship, showed variability. Only 3 of the 12 characteristics of Study 1 proved stable across clients between the 3rd and near-final sessions: perceived efforts to explore underlying dynamics both on their part (PEXP) and on the part of therapists (TEXP), and therapists' positive attitudes and affect (TPOS).
Remarks on Participants' Alliance Perceptions Over Time
In sum, although the pattern of variation differed in therapists and clients, participants' perceptions of most of the 12 relationship characteristics in Study 1 (when both comparisons of means and intra-individual associations are considered) showed variation over one time period or another and, for select variables, over all time phases assessed. Therapists' alliance perceptions were found to shift more prior to the 10th therapy session (i.e., there were shifts in two-thirds of the relationship characteristics during what proved to be the first third to first half of therapy). After the 10th session, therapists' perceptions generally stabilized. In contrast, shifts in clients' alliances occurred both before and after the 10th therapy session, although fewer facets varied for clients than for therapists. In general, the characteristics that showed variation among individual clients prior to the 10th session were the ones that continued to vary subsequently.
Given that alliance perceptions for the most part proved stable from the 5th to the 10th therapy session both in therapists and clients, as observed in Study 2, it may be that many facets of the therapeutic alliance, particularly in the perceptions of therapists, have stabilized by the 5th session. This conclusion remains tentative, however, because the comparability of alliance measures across the two studies (with the possible exception of the conceptually similar Penn scales39) remains to be established. More work is needed to delineate the distinctive cross-theoretical or cross-method features of the therapeutic alliance.3,45
Although not an objective of the current report, interrelationships among the alliance measures within samples were explored. (The comparability of dimensions across studies could not be evaluated.) In Study 1, of the 44 interscale coefficients from clients' alliance ratings at the 3rd, 10th, and next-to-last sessions, 0, 8, and 4, respectively, exceeded 0.70. Of the coefficients from therapists' ratings at these three assessment points, 4, 8, and 6, respectively, were greater than 0.70. In Study 2, of the 26 interscale coefficients at the 5th and 10th sessions, 15 and 7, respectively, of those based on clients' ratings exceeded 0.70, and 18 and 16, respectively, of those based on therapists' ratings were larger than 0.70. None of these correlations attained a magnitude of 0.90, the level at which redundancy of constructs can be concluded.46
In both therapy partners, variation in the relationship was most pronounced between the early and near-final therapy sessions. This result could reflect the variability observed in the relationship components prior to the 10th session and the cumulative effect of small, nonsignificant changes in other variables over the 3rd to the 10th and the 10th to near-final sessions. Negative components of the alliance, such as therapist-perceived client hostility (PHOS) and client-perceived negative client and therapist attitudes (PNEG, TNEG), together with client-perceived active participation (PPAR) and both participants' perceptions of therapist directiveness (e.g., offering advice, modeling behavior; TDIR), appeared to be particularly susceptible to individual perceptual differences, and these components may be more sensitive to contextual factors, such as session content or specific therapeutic work. On the other hand, components such as therapist-perceived warmth and friendliness (TWFR) and other positive (e.g., understanding, esteem-enhancing) attitudes and affect (TPOS), as well as client-perceived efforts to examine and explore feelings and experiences (PEXP), proved stable throughout the therapeutic process and may reflect trait-like characteristics of the participants.
These results, though requiring replication, suggest that single assessments of many facets of the participants' perceptions of the relationship cannot be assumed to be representative of their perceptions throughout the course of therapy, even though specific perceptions may persist over longer periods of time. Participants' ratings early in therapy, in particular, were poor predictors of their appraisal of the relationship qualities at the near-conclusion of the therapeutic process. Another implication of these findings, if replicated, is that significant predictor alliance relationships established at a specific point in therapy may not generalize to other therapy sessions. Depending on the alliance characteristic investigated and the measurement point in therapy, more than one assessment of such relationships may be necessary.
The current results also offered little support for the cyclical model of alliance development. Although neither study reported was specifically devised to test this model, the observed absence (on average ratings) in Study 1 of significantly lower scores in positive-toned and of higher scores in negative-toned variables at the 10th compared with the 3rd or near-final sessions seems inconsistent with the proposition7,8 that the alliance displays a pattern of strengthening, decay, and repair over the course of therapy. Similarly, the lack in Study 2 of significant decreases between the 5th and 10th therapy sessions (as indicated by a posteriori t-tests, all P>0.005, Bonferroni-adjusted) in the mean ratings of participants for whom the 10th session was near the midpoint of therapy (n=16) seems at variance with a purported critical phase9 for the alliance during the middle therapy sessions. Although these results may have been influenced by the predominantly humanistic therapy modality, which may involve less active use of challenging interventions, they are in line with prior data.10,13,14 However, such a cyclical pattern, or other patterns including steady improvement or stability,10 may characterize individual clients' and therapists' alliances. Indeed, the current results suggest the existence, in successful therapy, of different developmental patterns of the relationship components and measures among therapists and clients.
Although there has been some acknowledgment of such patterns,13,47 individual trajectories of development have not been the focus of empirical attention in the study of the time course of the alliance. A clinically relevant avenue of research suggested by the current results could involve the identification of participant subgroups who differentially appraise the alliance over time (i.e., participants whose perceptions either improve, worsen, remain stable, or wax and wane). Such an investigation could then seek pre-therapy or in-therapy factors (e.g., attachment style, specific therapist interventions, alliance rupture markers) that influence characteristic developmental patterns. A recent study along these lines has been conducted by Piper et al.,29 who reported different patterns of change in the alliance in clients with high, as opposed to low, quality of object relations.
Comparison of Participants' Alliance Perceptions
Results of group comparisons indicated that the therapeutic relationship was perceived similarly by therapists and clients in terms of average levels of the relationship components. Exceptions were clients' self-reflection and self-exploration efforts (PEXP), which were consistently rated higher by clients than by therapists across therapy, and, congruent with Marziali,15 therapists' negative attitudes (TNEG), which were consistently rated lower by clients than by therapists (Study 1). Both at the 5th and 10th therapy sessions (Study 2), clients also perceived their therapists' understanding and involvement (TUI), as well as the agreement with their therapists regarding treatment goals and how to proceed in therapy (WSC), to be significantly higher than did therapists. Clients appreciated, it appears, the commitment of their therapists to understand and help them, and they held equally favorable judgments of their own contributions and joint work efforts with therapists. Possibly clients entertain a different notion than therapists of what is entailed in collaborating in the work of therapy. Therapist trainees' comparatively more negative self-evaluations (Study 1) regarding counterproductive attitudes and affect could reflect increased sensitivity to these attitudes as a result of the supervision process.
To further explore participants' agreement regarding the quality of the alliance, post hoc correlations between therapists' and clients' alliance ratings were calculated. Results indicated, in general, poor convergence of perceptions of the relationship characteristics, regardless of assessment time.
The few exceptions generally reflected aspects of the client's involvement and positive contributions at select assessment points. At the 3rd session, client-therapist dyads agreed on the strength of the client's active involvement (PPAR); at the 5th session, they agreed on the client's commitment to therapy (PC); and at the 10th session, they agreed on the therapist's directive behaviors (TDIR); all P<0.001. They were in near agreement (P=0.002) at session 10 regarding the client's positive attitudes and affects (PPOS).
Thus, although at the group level therapists and clients held generally similar views of the alliance, within particular dyads the therapy partners did not have the same opinions regarding most features of the therapy relationship. Consistent with several other studies,11,12,23,29 these results suggest that clients and therapists employ different reference bases in appraising the quality of the therapeutic relationship. Although there has been some speculation on the source of participants' differential evaluations (e.g., therapists may evaluate alliance quality with reference to clients' progress, while clients may evaluate it with reference to extratherapy relationships13,48), this issue awaits further empirical investigation. Because clients' alliance perceptions are significant determinants of therapy outcome—superior to those based on other evaluative sources5—the observed low consensus underlines that it is important for therapists to attend more closely to their clients' viewpoints and regularly check with clients their perceptions of the therapy relationship.
Limitations
Several limitations to the generalizability of the current findings should be noted. First, sample sizes were small, and male clients and therapists, particularly in the first study, were underrepresented. A high proportion of therapists were inexperienced, although therapist level of experience, including lack of experience but presence of an “inherently helpful” disposition, generally appear to be unrelated to the quality of the therapeutic relationship.43,49–51 Even highly experienced therapists have low-alliance clients in the therapies they conduct.14 Second, therapists in the two current studies were predominantly humanistic in orientation, and the results may not generalize to other approaches. It should be noted, however, that participants' ratings of the alliance, as tapped by the CALPAS and WAI scales, have not been found to differ among various therapeutic modalities.24,43,52 Third, findings regarding the course of the relationship over therapy were limited to broad time intervals; the WAI and CALPAS variables, in particular, were examined at only two time points. More frequent assessment would have allowed for a more sensitive analysis of the developmental course of the therapy relationship and its components. The assessments of the alliance near the conclusion of therapy may be confounded with outcome, although this concern has not received strong empirical support.9,22,53 Finally, the low observed reliabilities, in the second sample, of the HAQ HA 2 and the CALPAS PWC scales warrant caution in interpreting the results obtained on these dimensions, and additional research is needed to establish their content validity.
CONCLUSION
The present findings, despite their limitations, add to the understanding of the time course of a broad range of relationship characteristics as seen by the therapy participants. Research in this area has predominantly focused on the three components (or global score) of the Working Alliance Inventory. The current results point to the relevance, in studying the time course of the alliance, of examining the perceptions of each therapy partner as well as the individual components of the therapeutic relationship.
An attempt was made to integrate results from mean comparisons and correlational analyses in discussing the time course of the alliance. In line with prior observations,9 variability of perceptions over the time course appears greater when individual clients' or therapists' alliance perceptions are monitored, as compared with a focus on pooled ratings. As noted by others,29,54 findings based on aggregate data may mask important individual differences in perceptions of the alliance. To improve our understanding of the actual therapist-client relationship, it appears worthwhile to pursue investigation of individual differences in perceptions of relationship qualities across the course of therapy, along with factors that enhance or strain the relationship in particular clients or therapists, or in participant subgroups that share similar perceptions.
Acknowledgments
The authors gratefully acknowledge the assistance of Hans Ivers in data analysis. A portion of this paper was presented at the annual meeting of the Society for Psychotherapy Research, Berkeley, CA, 1992.
References
- 1.Bordin E: The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy: Theory, Research and Practice 1979; 16:252–260 [Google Scholar]
- 2.Horvath AO, Greenberg L: Development and validation of the Working Alliance Inventory. Journal of Counseling Psychology 1989; 36:223–233 [Google Scholar]
- 3.Horvath AO, Greenberg LS: Introduction, in The Working Alliance: Theory, Research and Practice, edited by Horvath AO, Greenberg LS. New York, Wiley, 1994, pp 1–9
- 4.Stiles W, Shapiro D, Elliott R: Are all psychotherapies equivalent? Am Psychol 1986; 41:165–180 [DOI] [PubMed] [Google Scholar]
- 5.Horvath AO, Symonds BD: Relation between working alliance and outcome in psychotherapy: a meta-analysis. Journal of Counseling Psychology 1991; 38:139–149 [Google Scholar]
- 6.Luborsky L: Helping alliance in psychotherapy, in Successful Psychotherapy, edited by Cleghhorn JL. New York; Brunner/Mazel, 1976, pp 92–116
- 7.Bordin E: A psychodynamic view of counseling psychology. The Counseling Psychologist 1980; 9:62–66 [Google Scholar]
- 8.Gelso CJ, Carter JA: Components of the psychotherapy relationship: their interaction and unfolding during treatment. Journal of Counseling Psychology 1994; 41:296–306 [Google Scholar]
- 9.Horvath AO, Gaston L, Luborsky L: The therapeutic alliance and its measures, in Psychodynamic Treatment and Research, edited by Miller L, Luborsky L, Barber J et al. New York, Basic Books, 1993, pp 247–273
- 10.Greenberg LS: What is “real” in the relationship? Comment on Gelso and Carter (1994). Journal of Counseling Psychology 1994; 41:307–310 [Google Scholar]
- 11.Horvath AO, Marx RW: The development and decay of the working alliance during time-limited counseling. Canadian Journal of Counseling 1990; 24:240–259 [Google Scholar]
- 12.Golden BR, Robbins SB: The working alliance within time-limited therapy: a case analysis. Professional Psychology: Research and Practice 1990; 21:476–481 [Google Scholar]
- 13.Kivlighan D, Shaughnessy P: Analysis of the development of the working alliance using hierarchical linear modeling. Journal of Counseling Psychology 1995; 42:338–349 [Google Scholar]
- 14.Sexton HC, Hembre K, Kvarme G: The interaction of the alliance and therapy microprocess: a sequential analysis. J Consult Clin Psychol 1996; 64:471–480 [DOI] [PubMed] [Google Scholar]
- 15.Marziali E: Three viewpoints on the therapeutic alliance. J Nerv Ment Dis 1984; 172:417–423 [DOI] [PubMed] [Google Scholar]
- 16.Klee MR, Abeles N, Muller RT: Therapeutic alliance: early indicators, course, and outcome. Psychotherapy 1990; 27:166–174 [Google Scholar]
- 17.Marmar C, Weiss D, Gaston L: Towards the validation of the California Therapeutic Alliance Rating System. Psychological Assessment 1989; 1:46–52 [Google Scholar]
- 18.Luborsky L, Crits-Christoph P, Alexander L, et al: Two helping alliance methods for predicting outcomes of psychotherapy: a counting signs vs global ratings method. J Nerv Ment Dis 1983; 171:480–491 [DOI] [PubMed] [Google Scholar]
- 19.Hartley D, Strupp H: The therapeutic alliance: its relationship to outcome in brief psychotherapy, in Empirical Studies of Psychoanalytic Theories, vol 1, edited by Masling J. Hillsdale, NJ, Lawrence Erlbaum, 1983, pp 1–37
- 20.Eaton T, Abeles N, Gutfreund M: Therapeutic alliance and outcome: impact of treatment length and pretreatment symptomatology. Psychotherapy: Theory, Research and Practice 1988; 25:536–542 [Google Scholar]
- 21.Gomes-Schwartz B: Effective ingredients in psychotherapy: prediction of outcome from process variables. J Consult Clin Psychol 1978; 46:1023–1035 [DOI] [PubMed] [Google Scholar]
- 22.Morgan R, Luborsky L, Crits-Christoph P, et al: Predicting the outcomes of psychotherapy by the Penn Helping Alliance Rating method. Arch Gen Psychiatry 1982; 39:397–402 [DOI] [PubMed] [Google Scholar]
- 23.Tichenor V, Hill C: A comparison of six measures of working alliance. Psychotherapy: Theory, Research and Practice 1989; 26:195–199 [Google Scholar]
- 24.Salvio M, Beutler L, Wood J, et al: The strength of the therapeutic alliance in three treatments for depression. Psychotherapy Research 1992; 1:31–36 [Google Scholar]
- 25.Crits-Christoph P, Barber J, Kurcias JS: The accuracy of therapists' interpretations and the development of alliance. Psychotherapy: Theory, Research and Practice 1993; 3:25–35 [Google Scholar]
- 26.Gurman A: The patient's perception of the therapeutic relationship, in Effective Psychotherapy, edited by Gurman A, Razin A. New York, Pergamon, 1977, pp 503–543
- 27.Marmar CR, Horowitz MJ, Weiss DS: The development of the therapeutic alliance rating system, in The Psychotherapeutic Process: A Research Handbook, edited by Greenberg LS, Pinsof WM. New York, Guilford, 1986, pp 367–390
- 28.Mallinckrodt B, Nelson M: Counselor training level and the formation of the psychotherapeutic working alliance. Journal of Counseling Psychology 1991; 38:133–138 [Google Scholar]
- 29.Piper W, Boroto D, Joyce A, et al: Pattern of alliance and outcome in short-term individual psychotherapy. Psychotherapy: Theory, Research and Practice 1995; 32:639–647 [Google Scholar]
- 30.Carter JA: Refining notions through converging paradigms. The Counseling Psychologist 1994; 22:79–81 [Google Scholar]
- 31.Marziali E, Marmar C, Krupnick J: Therapeutic alliance scales: development and relationship to psychotherapy outcome. Am J Psychiatry 1981; 138:361–364 [DOI] [PubMed] [Google Scholar]
- 32.Hatcher RL, Barends AW: Patients' view of the alliance in psychotherapy: exploratory factor analysis of three alliance measures. J Consult Clin Psychol 1996; 64:1326–1336 [DOI] [PubMed] [Google Scholar]
- 33.Bachelor A: Comparison and relationship to outcome of diverse dimensions of the helping alliance as seen by client and therapist. Psychotherapy: Theory, Research and Practice 1991; 28:534–549 [Google Scholar]
- 34.Safran J, Wallner K: The relative predictive validity of two therapeutic alliance measures in cognitive therapy. Psychological Assessment 1991; 3:188–195 [Google Scholar]
- 35.Kazdin A: Preparing and evaluating research reports. Psychological Assessment 1995; 7:228–237 [Google Scholar]
- 36.Endicott J, Spitzer R, Fleiss J, et al: The Global Assessment Scale. Arch Gen Psychiatry 1976; 33:766–771 [DOI] [PubMed] [Google Scholar]
- 37.Ilfeld F: Further validation of a psychiatric symptom index in a normal population. Psychological Reports 1976; 39:1215–1228 [Google Scholar]
- 38.Suh C, Strupp H, O'Malley S: The Vanderbilt process measures: the Psychotherapy Process Scale (VPPS) and the Negative Indicators Scale (VNIS), in The Psychotherapeutic Process: A Research Handbook, edited by Greenberg L, Pinsof W. New York, Guilford, 1986, pp 285–323
- 39.Alexander L, Luborsky L: The Penn Helping Alliance Scales, in The Psychotherapeutic Process: A Research Handbook, edited by Greenberg L, Pinsof W. New York, Guilford, 1986, pp 325–366
- 40.Green B, Gleser G, Stone W, et al: Relationships among diverse measures of psychotherapy outcome. J Consult Clin Psychol 1975; 43:689–699 [DOI] [PubMed] [Google Scholar]
- 41.Luborsky L, McLellan T, Woody G, et al: Therapist success and its determinants. Arch Gen Psychiatry 1985; 42:602–611 [DOI] [PubMed] [Google Scholar]
- 42.Tracey T, Kokotovic A: Factor structure of the Working Alliance Inventory. Psychological Assessment 1989; 1:207–210 [Google Scholar]
- 43.Gaston L: Reliability and criterion-related validity of the California Psychotherapy Alliance Scales. Psychological Assessment 1991; 3:68–73 [Google Scholar]
- 44.Henry WP, Strupp HH: The therapeutic alliance as interpersonal process, in The Working Alliance: Theory, Research and Practice, edited by Horvath AO, Greenberg LS. New York, Wiley, 1994, pp 51–84
- 45.Rice LN, Greenberg LS (eds): Patterns of Change: Intensive Analysis of Psychotherapy Process. New York, Guilford, 1984
- 46.Tabachnick BG, Fidell LS: Using Multivariate Statistics, 2nd edition. New York, Harper and Row, 1989
- 47.Gaston L, Marmar C: The California Psychotherapy Alliance Scales, in The Working Alliance: Theory, Research and Practice, edited by Horvath AO, Greenberg LS. New York, Wiley, 1994, pp 85–108
- 48.Tyron GS, Kane AS: Relationship of working alliance to mutual and unilateral termination. Journal of Counseling Psychology 1993; 40:33–36 [Google Scholar]
- 49.Dunkle J, Friedlander M: Contribution of therapist experience and personal characteristics to the working alliance. Journal of Counseling Psychology 1996; 43:456–460 [Google Scholar]
- 50.Gomes-Schwartz B, Schwartz J: Psychotherapy process variables distinguishing the “inherently helpful” person from the professional psychotherapist. J Consult Clin Psychol 1978; 46:196–197 [DOI] [PubMed] [Google Scholar]
- 51.Strupp H, Hadley S: Specific vs nonspecific factors in psychotherapy: a controlled study of outcome. Arch Gen Psychiatry 1979; 36:1125–1129 [DOI] [PubMed] [Google Scholar]
- 52.Marmar CR, Gaston L, Gallagher D, et al: Therapeutic alliance and outcome in behavioral, cognitive, and brief dynamic psychotherapy in late-life depression. J Nerv Ment Dis 1989; 17:464–472 [DOI] [PubMed] [Google Scholar]
- 53.Gaston L, Marmar CR, Thompson LW, et al: Alliance prediction of outcome beyond in-treatment symptomatic change as psychotherapy progresses. Psychotherapy Research 1991; 1:104–112 [Google Scholar]
- 54.Henry WP, Strupp HH, Schacht TE, et al: Psychodynamic approaches, in Handbook of Psychotherapy and Behavioral Change, edited by Bergin AE, Garfield SL. New York, Wiley, 1994, pp 467–508