Identifying who benefits most from supportive vs. expressive techniques in psychotherapy for depression: Moderators of within- vs. between-individual effects

Abstract

Objective:

A recent randomized controlled trial (RCT) indicated that individuals with higher levels of attachment anxiety exhibited better treatment outcomes in supportive-expressive therapy (SET) relative to supportive therapy (ST). But to gain insight into within-patient therapeutic changes a within-individual design is required. The present study contrasts previous findings based on theory-driven between-patient moderators with data-driven moderators of within-patient processes to investigate whether findings converge or diverge across these two approaches.

Methods:

We used data of 118 patients from the pilot and active phases of a recent RCT for patients with major depressive disorder, comparing ST with SET, a time-limited psychodynamic therapy. The predefined primary outcome measure was the Hamilton Rating Scale for Depression. Supportive vs. expressive techniques were rated based on patients’ end-of-session perspective. We compared previous findings based on moderators of between-patient effects with a data-driven approach for identifying moderators of within-patient effects of techniques on subsequent outcome.

Results:

After false discovery rate corrections, of 10 pre-selected moderators, patients’ attachment anxiety and domineering style remained significant. Of these, bootstrap resampling revealed significant differences between ST and SET techniques for the attachment anxiety moderator: those with higher attachment anxiety benefited more from greater use of ST than SET techniques in a particular session, as evidenced by lower levels of symptoms at the subsequent session.

Conclusions:

Our within-individual findings diverge from previously published between-individual analyses. This proof-of-concept study demonstrates the importance of complementing between-individuals with within-individual analyses to achieve better understanding of who benefits most from specific treatment techniques.

The question “what works for whom?” lies at the core of the field of psychotherapy research. This question is relevant for all mental and physical health interventions, but it is especially relevant to major depressive disorder (MDD) given that it is the leading cause of disability worldwide, a main contributor to the overall global burden of disease (Friedrich, 2017), and a highly heterogeneous disorder (Goldberg, 2011). Hundreds of active treatments for MDD are available, such as cognitive behavioral, interpersonal, behavioral activation, emotion-focused, and supportive-expressive therapies (DeRubeis & Strunk, 2017). These treatments differ in their underlying mechanisms theorized to drive therapeutic change (Crits-Christoph & Gibbons, 2021), but do not seem to differ in their overall efficacy, and all show an average treatment response rate of about 50% (Cuijpers, 2017). It has been repeatedly argued that although for the average patient there may be little difference between the various treatments, some subpopulation of patients may be particularly well-suited to a particular intervention and derive relatively greater benefit (DeRubeis et al., 2014a,b). This assumption underlies personalized treatments approaches that seek to optimize treatment outcomes by identifying pre-treatment characteristics by which to match individuals to the intervention which is expected to provide them the most therapeutic benefit (Cohen et al., 2021).

Moderation analyses are commonly used to test whether a particular patient characteristic (e.g., baseline clinical or demographic variable) is related to treatment group differences in outcome (Kazdin, 2007; Kraemer et al., 2002). Moderators are referred to as “prescriptive” if they predict different outcomes depending on the type of treatment (e.g., as reflected by a significant predictor × treatment group interaction), as opposed to “prognostic variables” that predict treatment outcome irrespective of treatment type (Hollon & Beck, 1986). Personalized treatment approaches assume that variability in treatment outcomes between individuals can in part be accounted for by baseline patient characteristics, and that these baseline moderators can be translated into actionable, prescriptive information about which interventions are best suited for which patients (Cohen et al., 2021). Two main approaches have been used in the literature to identify moderators: theory-driven and data-driven (Zilcha-Mano, 2018).

Theory-driven (or “top-down”) approaches are based on theoretical conceptualizations of which subpopulation can benefit most from each treatment or group of treatments. In contrast, data-driven (or “bottom-up”) approaches often use machine learning methods to identify moderators from a larger pool of candidate variables, some or all of which could be selected based on relevant prior theory (Cohen & DeRubeis, 2018). Although the two approaches differ in their conceptual models and methodology, they converge in their focus on moderators of between-group effects. That is, they focus on identifying moderators of treatment group difference in outcome (e.g., which baseline patient characteristics are associated with better outcome to Treatment A vs. Treatment B).

Identifying moderators of the between-group association between treatment packages and treatment outcome is clinically relevant as it can inform recommendations regarding which treatment package is expected to be most therapeutically beneficial for a given individual. It does not, however, answer the common question in clinical practice: What is the most effective technique to use with a particular patient at a particular session to achieve the best outcome for this individual? That is, the most common approaches to identifying moderators focus on between-individual effects (of whole treatment packages on outcome) but ignores within-individual psychotherapeutic processes (of the use of specific techniques in a given session on subsequent outcome). In recent years, there has been a growing understanding of the importance of focusing on within-individual processes in psychopathology (e.g., Wright & Woods, 2020) and psychotherapy (e.g., Fisher & Boswell, 2016; Fisher & Bosley, 2015; Zilcha-Mano & Webb, 2021; Zilcha-Mano, 2021). The methodological literature highlights the profound differences that can emerge between inferences drawn from conventional between-person analyses and within-person ones (Hamaker, 2023), and the two may even be in opposite directions. For example, the association between patients’ levels of insight and outcome when implementing a treatment package focused on raising insight may have inconsistent (even opposite) associations at the between- vs. within-individual levels. At the between-patient level, when implementing a treatment package aimed at increasing insight, the association may be negative, meaning that those with poorer insight (low trait-like insight) may benefit most from a treatment package aimed at improving insight (e.g., supportive-expressive treatment). By contrast, at the within-patient level, the association may be positive so that gaining insight at a particular session (higher state-like insight) may predict subsequent symptom improvement (Zilcha-Mano, 2021). Another example is the association between patients’ levels of behavioral activation (BA) skills and outcome. Individuals with pre-treatment deficits in BA skills (low trait-like BA) may be more likely than those with relatively high levels of skills to benefit from a BA treatment protocol which specifically targets that skill set (i.e., lower trait-like skills predict relatively enhanced response to BA therapy). By contrast, greater state-like within-individual increases in BA skills may predict better outcomes over the course of treatment (i.e., the opposite relation for the state-like effect). Given findings from between- and within-individual analyses may not converge, it is also possible that results from moderation analyses conducted at the between- vs. within-individual level run in opposite directions.

To the best of our knowledge, only one study to date has focused on identifying moderators of the within-individual relation between therapeutic skills and outcome. Specifically, Webb et al. (2021) were interested in identifying which therapeutic skill domain was most emotionally beneficial for a given patient. The authors used a data-driven approach to identify pre-treatment characteristics that moderate the effect of therapeutic skills (behavioral therapy [BT] skills vs. dialectical behavior therapy [DBT] skills), implemented by patients upon discharge from a behavioral health partial hospital program, on positive affect. Findings revealed that higher levels of non-suicidal self-injury and sleep disturbance were associated with a stronger within-person relationship between use of DBT skills and improved affect, whereas predictors of the effect of BT skills on better affect included higher emotional lability and anxiety disorder comorbidity, and lower psychomotor retardation/agitation and feelings of worthlessness/guilt. These findings demonstrate the potential clinical utility of identifying moderators of within-individual effects of therapeutic skills on affect to inform personalized prescriptions of which skills a therapist may wish to focus on for a given patient to optimize outcomes.

Building on Webb et al. (2021), the present proof of concept study contrasts moderators of the between-patients effect with those of the within-patient effect. Nearly all the knowledge gained so far about moderators is based on between-person effects (identifying moderators of the association between treatment condition and outcome). But unlike moderators of between-patient effects, only moderators of the within-patient effects can answer questions that are most relevant for clinical practice: Which techniques should be implemented during the session to improve subsequent treatment outcome with this particular patient? The present study aimed to explore whether findings of moderators of the within- vs. between-patients effects converge. To this end, we sought to contrast findings from a recent randomized controlled trial (RCT) designed to test pre-registered theory-driven moderators of the between-patient effect of treatment packages on treatment outcome (Zilcha-Mano et al., 2021), with an approach for identifying moderators of within-individual effects of techniques on subsequent symptom improvement. In the original RCT, as hypothesized, individuals with higher levels of attachment anxiety exhibited better treatment outcomes in supportive-expressive therapy (SET) than in supportive therapy (ST). We aimed to test whether, when focusing on within-patient effects, the same or different moderators are identified, and if the same moderator (attachment anxiety) is identified, whether the effects are in the same or opposite direction. Thus, we used attachment orientation as a potential moderator, together with a set of moderators previously identified in the literature (Cohen et al., 2021; Lutz et al., 2021) that were available in the abovementioned RCT, including: (a) interpersonal characteristics of the patients – interpersonal tendencies and expectations from the alliance with the therapist; (b) clinical characteristics – depressive symptom severity at the previous session (as a time-variant predictor) and personality disorders; and (c) demographic characteristics (age, gender, and education).

Methods

Study design

Participants were individuals participating in the pilot and active phases of an RCT conducted in Israel (Zilcha-Mano et al., 2021). Random assignment was conducted by an outside institution, not involved in the study, which is specialized in mechanisms of assignment in clinical trials. Minimization algorithm (Pocock & Simon, 1975) was used, with the factors for balancing being age (30>= vs. 30<), gender (male vs. female), family status (married/cohabiting vs. not married/cohabiting), baseline 17 item Hamilton Rating Scale for Depression (HRSD; Hamilton, 1967) (20>= vs. 20<), baseline attachment avoidance (3.5>= vs. 3.5< on the avoidance subscale in the Experience in Close Relationships, ECR; Brennan et al., 1998), baseline attachment anxiety (3.5>= vs. 3.5< on the anxiety subscale in the ECR), and personality disorders (present vs. absent). Comparing the two conditions on patients’ pre-treatment characteristics showed no significant differences between the two conditions (Zilcha-Mano et al., 2021). Only the therapists and their supervisors knew the patients’ treatment assignment. Treatments were face-to-face until the start of the pandemic, after which 13 patients out of 118 were treated remotely. The trial protocol (Zilcha-Mano et al., 2018) and main outcome (Zilcha-Mano et al., 2021) provide further details about the trial. All procedures were approved by the institutional review board (#186/15) and participants signed an informed consent.

Treatments manuals and adherence

Patients received 16 50-min sessions of supportive-expressive treatment (SET; Luborsky et al., 1995), a time-limited psychodynamic therapy adapted for depression. They were randomized to either an SET-focused condition (including the use of expressive techniques, such as interpretation, confrontation, clarification) or an ST-focused condition (including the use of supportive techniques, such as affirmation and empathic validation). For SET, the Luborsky et al. (1995) manualized treatment was used. The ST condition included all supportive techniques detailed in the manual used by Luborsky et al. (1995), but forbade the use of expressive techniques (Leibovich et al., 2018).

For both treatments, we assessed adherence using the Penn Adherence-Competence Scale (PACS) (Barber & Crits-Christoph, 1996). The PACS includes 3 subscales: general therapeutic behaviors (e.g., “The therapist encourages (directly or by a facilitating atmosphere) the patient’s expression; that is, to say what he or she thinks or feels;” 9 items; ICC=.71 for amount and ICC=.76 for quality), a supportive component (e.g., “The therapist conveys a sense of respect, understanding, and acceptance to the patient;” 9 items; ICC=.86 for amount and ICC=.83 for quality), and an expressive component (e.g., “The therapist helps the patient to realize the various manifestations of the patient’s central relationship difficulty or conflict across situations.”; 21 items; ICC=.91 for amount and ICC=.83 for quality). Coders were four trained PhDs or PhD/MA students in clinical psychology. The number of coders per session varied from two to four. For each patient, we randomly selected between one to three sessions from sessions 4, 6, or 8. These sessions were chosen based on the treatment protocol (Book, 1998; Luborsky et al., 1995). A total of 161 sessions were coded, 80 SET and 81 ST. Inter-judge reliability was calculated as two-way mixed with absolute agreement (Shrout & Fleiss, 1979). The research team was supervised by an international expert on the use of PACS, with vast experience in using PACS in RCTs involving SET. As reported in the main outcome article, adherence for both treatments was high (Zilcha-Mano et al., 2021). Specifically, using a permutation t-test with 10,000 Monte-Carlo permutations, we found no differences between treatment conditions for the amount (p=0.37) or quality (p=.90) of general therapeutic adherence, and the amount (p=.88) or quality (p=0.31) of ST-focused adherence. As expected, SET showed higher levels of adherence than ST in SET-focused adherence (p<.0001) and in SET-focused amount (p<.0001).

Participants

A total of 118 patients participated in the pilot and active phase of the RCT. The patients’ demographic and clinical characteristics are presented in Table 1. Inclusion criteria: (a) current MDD diagnosis using structured clinical interviews for DSM-V, with scores above 14 on the 17-item HRSD (Hamilton, 1967) at two evaluations, one week apart, and current MDD based on the Mini-International Neuropsychiatric Interview (MINI; Sheehan et al., 1998); (b) if on medication, patients’ dosage had to be stable for at least three months before the start of the study, and patients were asked to maintain stable dosage for the duration of treatment; (c) age between 18 and 60 years; (d) Hebrew language fluency; (e) written informed consent. Exclusion criteria: (a) current high risk of suicide or self-harm (HRSD suicide item >2); (b) current substance abuse disorder; (c) current or past schizophrenia, psychosis, bipolar disorder, or severe eating disorder, requiring medical monitoring; (d) history of organic mental disease; (e) currently in psychotherapy.

Table 1.

Demographic and clinical characteristics as a function of treatment condition

Variables	ST (n=59)	SET (n=59)	Total (N=118)	Statistical Test	p Value
Demographics
Age, y, mean (SD)	31.00 (6.7)	31.13 (9.4)	31.06 (8.14)	t(116)= −.09	0.93
Education, y, mean (SD)	14.26(2.23)	14.36 (2.23)	14.31 (2.22)	t(116)=−0.24	0.80
Female	62.7(37)	55.9(33)	59.3(70)	χ2(1)=.31	0.57
Married/cohabitating	16.9(10)	20.3(12)	18.6(22)	χ2(3)=1.4	.69
Religion, Jewish	79.6(43)	79.2(42)	79.4(85)	χ2(4)= 3.11	.54
Clinical Features
Current medication, yes	12.1 (7)	12.3 (7)	12.2(14)	χ2(1)=0	1
Previous medication, yes	23.7(14)	25.4(15)	24.6(29)	χ2(1)=0	1
Previous psychotherapy, yes	50(29)	45.8(27)	47.9(56)	χ2(1)=0.07	0.78
Comorbidities
Any disorder	74.6(44)	76.3(45)	75.4(89)	χ2(1)=.00	1
Any anxiety disorder	72.9(43)	64.4(38)	68.6(81)	χ2(1)=.63	.43
Any personality disorder	72.9(43)	71.2(42)	72(85)	χ2(1)=.00	1
Drop-outs	8.5(5)	5.1(3)	6.8(8)	χ2(1)=0.13	.72
Remote treatment, yes	11.86(7)	10.16(6)	11.01(13)	χ2(1)=.00	1

Note. Values shown as % (n).

Abbreviations: ST = Supportive Treatment, SET = Supportive-Expressive Treatment

The original research was powered to study the theory-driven moderating effect of attachment orientation using a between-individuals approach (ClinicalTrials.gov Identifier: NCT02728557; Study protocol: Zilcha-Mano et al., 2018). To calculate the required sample size for the moderation model, Monte Carlo simulations were used, as estimated using R code generated by the MLPowSim software package. A total of 10,000 Monte Carlo simulations were applied, which produce more accurate results for power estimates than other methods for relatively small sample sizes. The power calculation was based on effect sizes reported by Newman et al., (2015) and on previous studies by the authors. Assuming alpha=0.05 and 16 repeated measurements of the outcome variable (including missing data), the simulations indicated a required sample size of 99 participants to ensure a power of at least 0.80. The data-driven moderators of the within-individual effects of techniques on outcome were not pre-registered and the parent clinical trial was not initially designed based on a power analysis of these specific data-driven analyses. Yet, it should be noted that generally there are more degrees of freedom and power for the within-individual effect, which is based on 15 observations per patient (16 sessions minus 1 because of the lagged effect), whereas the between-patients analyses are based on a single observation per patient.

Therapists

Therapists acted as their own controls, providing treatment in both conditions. Eight therapists with at least five years of expertise in psychodynamic treatment attended a 20-hour training workshop in supportive and expressive techniques. Therapists completed treatment of two pilot cases, one of each treatment condition, and demonstrated acceptable treatment adherence before the trial phase. During the pilot phase and the trial, each therapist received weekly group supervision from two supervisors, as well as individual supervision. In all supervisions, extensive use was made of videotaped sessions for feedback. The supervisors were licensed clinical psychologists, with extensive supervision experience. They received supervision concerning the supervision process from an international expert in SE, with more than 20 years of experience in psychodynamic treatment for depression and more than 15 years of experience in SE treatments in RCTs. Two of the therapists did not continue after the training phase (one being offered a full-time position elsewhere, the other demonstrating low levels of adherence). Therapists mean age was 39.89 (SD=6.15), and five were female. All were married or cohabitating. Their mean years of experience was 11.89 (SD=5.73). All therapists had psychodynamic training, two also had CBT training, and one also had biofeedback training. The mean number of patients each therapist treated was 13.1. (SD=10.7), range 2–33.

Measures

Psychiatric disorders.

The Mini-International Neuropsychiatric Interview (MINI; Sheehan et al., 1998) was administered to assess the presence and severity of depression and comorbid conditions.

Patients’ perception of the techniques used in the session.

We used the Multitheoretical List of Therapeutic Interventions (MULTI; McCarthy & Barber, 2009; Solomonov et al., 2019) to assess patients’ perception of the techniques used in the session. MULTI is a 30-item self-reported measure assessing intervention use from eight therapy orientations from the patient’s perspective. In the present study, we focused on the psychodynamic subscale to assess SET-based techniques and on the common factors to assess ST-based techniques. Five items were used for the SET (e.g., “The therapist encouraged clients to talk about feelings they had previously avoided or never expressed”) and 4 items for the ST (e.g., “The therapist was warm, sympathetic, and accepting”). Items were rated on a 5-point Likert scale, rating each item based on how representative it was of the session they have just completed, on a scale ranging from 1 (Not at all typical of the session) to 5 (Very typical of the session). Internal consistency for the psychodynamic and common factors subscales from the patients’ perspective were .82 and .80, respectively.

Treatment outcome.

The outcome measure was the HRSD (Hamilton, 1967), a semi-structured interview containing 17 items assessing the patient’s symptoms in the preceding week. For the present study, the inter-rater reliability of the HRSD was ICC=.98.

Potential moderators

Attachment orientation.

We used the Experience in Close Relationships scale (ECR; Brennan et al., 1998) to assess attachment orientation. The ECR is a 36-item self-reported measure assessing the construct of adult attachment. Participants rated the extent to which each item was descriptive of their experiences in close relationships on a 7-point Likert scale ranging from 1 (not at all) to 7 (very much). Eighteen items assessed attachment anxiety and 18 assessed attachment avoidance. In the present study, Cronbach’s α was .90 for the anxiety items and .89 for the avoidance items.

Interpersonal problems.

We used the Inventory of Interpersonal Problems-Circumplex (IIP-C) (Alden, Wiggins, & Pincus, 1990; Horowitz, Rosenberg, Baer, Ureño, & Villaseñor, 1988), a 32-item self-reported inventory, to assess behaviors related to interpersonal problems. Items were rated on a 5-point Likert scale ranging from 0 (Not at all) to 4 (Extremely). Based on the 32 items, two dimensions were calculated and used in the present study: dominance and affiliation (Wiggins, 1996). Dominance reflects a tendency toward interpersonal assertion and a demanding approach (as opposed to passivity); affiliation reflects a tendency toward friendly interaction with others (as opposed to interpersonal distance). Internal reliability for each of the subscales compositing the two dimensions ranged between .70-.78.

Expected Alliance.

We used the Expected Working Alliance Short Form pre-treatment (EWAI; Barber et al., 2014; Horvath & Greenberg, 1989; Tracey & Kokotovic, 1989) to assess expected alliance before meeting with the therapist. Following Barber at el. (2014), the EWAI was developed by tailoring the instructions provided in the original 12-item WAI measure to assess the expected alliance. The following sentence was added to the instructions of the WAI: “Because you have not yet experienced treatment as part of this study, answer the following questions by thinking about how you expect treatment to be.” Items were rated on a 7-point Likert scale ranging from 1 (Never) to 7 (Always). Internal reliability for the EWAI in the present study was .90.

Personality disorders.

We used the Structured Interview for the Diagnosis of Personality Disorders (SIDP-IV; Pfohl et al., 1997) to assess the patient’s personality disorders. The SIDP-IV is a comprehensive semi-structured clinical interview to determine the existence of personality disorders. The SIDP-IV includes non-pejorative questions organized into topical sections to produce a natural flow in the interview. The number of criteria for each PD diagnosis varies from 7 to 9. Criteria were rated as follows: 0=Absent, 1=Sub-threshold, 2=Present, 3=Strongly present. A score of 2 or more on at least 3–5 criteria (depending on the PD in question) is required for a diagnosis of PD. For the present study, we used a binary score representing the absence (0) or presence of at least one personality disorder (1). SIDP-IV interviewers were masters or doctoral level clinical psychologists, who received extensive training and supervision in the administration of the SIDP-IV. Inter-judge reliability for the 79 items of the SIDPIV, assessed by intraclass correlation (ICC; Shrout & Fleiss, 1979), was .93.

Procedure

ECR, MINI, and SIDP-IV were administered at baseline and HRSD was administered at baseline and then weekly. For HRSD and MINI, evaluators were advanced undergraduate, graduate, and PhD students in clinical psychology; for the SIDP, they were graduate and PhD students in clinical psychology. All evaluators were extensively trained and found to be reliable in the use of the HRSD, MINI, and SIDP. Patients reported their perception of the techniques used in the session immediately after the end of each session, for a total of 16 sessions.

Statistical analyses

As a preliminary analysis, we tested the within-patient effect of ST and SET on subsequent outcome. To eliminate between-patients effects, we included patients as a fixed effect. The formula was as follows:

$$\text{HRS}{\text{D}}_{\text{i}\left(\text{t+1}\right)}\hspace{0.17em}\text{=}\hspace{0.17em}{\text{b}}_{\text{0}}\hspace{0.17em}\text{+}\hspace{0.17em}{\text{b}}_{\text{1}}\text{*HRS}{\text{D}}_{\text{it}}\hspace{0.17em}\text{+}\hspace{0.17em}{\text{b}}_{\text{2}}\text{*techniqu}{\text{e}}_{\text{it}}\hspace{0.17em}\text{+}\hspace{0.17em}{\text{u}}_{\text{i}}\hspace{0.17em}\text{+}\hspace{0.17em}{\text{e}}_{\text{it}}$$ Formula 1.

where $\text{HRS}{\text{D}}_{\text{i}\left(\text{t+1}\right)}$ is the outcome of patient $\text{i}$ for session $\text{t+1}$, $\text{techniqu}{\text{e}}_{\text{it}}$ is the score of the use of each technique (SET or ST) with this patient in that session, ${\text{u}}_{\text{i}}$ is the patient effect (fixed), and ${\text{e}}_{\text{it}}$ is the error. All ${\text{e}}_{\text{it}}$ were normally distributed and independent of each other.

To test the differences between the effect (i.e., ${\text{b}}_{\text{2}}$ in the formula) of SET and ST on subsequent outcome, we tested the 95% confidence interval (CI) using bootstrap resampling (2,000 samples within each type of technique). If the CI of the effect of the difference did not contain 0, it suggested that the difference is significant with 95% confidence.

To identify patient characteristics that moderate the effect of the patient-reported use of techniques (SET and ST) on subsequent treatment outcome, we conducted a series of linear regressions, with separate models for each moderator. This approach estimates the moderation effect directly as part of the linear model by including the interaction of the moderator with techniques in the model. In each model, we used the interaction between the type of techniques (SET or ST) and the potential moderators, as well as their main effects and the level of outcome at the current session ($\text{time t}$) to predict the outcome at the next session ($\text{time}\hspace{0.17em}\text{t+1}$). Given the focus on within-patient effects, to eliminate any between-patients effects, we included patient as a fixed effect. Before entering them into the model, we standardized all continuous variables, except HRSD. The rationale for using a separate model for each moderator was that using all moderators in the model would answer a different question from the one we were interested in, namely, what the contribution of each moderator was while all others were kept constant. This was not our interest, therefore we conducted separate models for each. Given the multiple comparisons, we also calculated an adjusted p value based on the false discovery rate (FDR) correction method.

We used the following formula:

$$\text{HRS}{\text{D}}_{\text{i}\left(\text{t+1}\right)}\hspace{0.17em}\text{=}\hspace{0.17em}{\text{b}}_{\text{0}}\hspace{0.17em}\text{+}\hspace{0.17em}{\text{b}}_{\text{1}}\text{*HRS}{\text{D}}_{\text{it}}\hspace{0.17em}\text{+}\hspace{0.17em}{\text{b}}_{\text{2}}\text{*moderato}{\text{r}}_{\text{i}}\hspace{0.17em}\text{+}\hspace{0.17em}{\text{b}}_{\text{3}}\text{*techniqu}{\text{e}}_{\text{it}}\text{+}{\text{b}}_{\text{4}}\text{*techniqu}{\text{e}}_{\text{it}}\text{*moderato}{\text{r}}_{\text{i}}\hspace{0.17em}\text{+}\hspace{0.17em}{\text{u}}_{\text{i}}\hspace{0.17em}\text{+}\hspace{0.17em}{\text{e}}_{\text{it}}$$ Formula 2.

where $\text{HRS}{\text{D}}_{\text{it}}$ is the outcome of patient $\text{i}$ for session $\text{t}$, $\text{techniqu}{\text{e}}_{\text{it}}$ is the score of the use of each technique (SET or ST) with this patient in that session; $\text{moderato}{\text{r}}_{\text{i}}$ is the value of the moderator of patient $\text{i}$, ${\text{u}}_{\text{i}}$ is the subject effect (fixed), and ${\text{e}}_{\text{it}}$ is the error. All ${\text{e}}_{\text{it}}$ were normally distributed and independent of each other.

After identifying potential moderators, we used the same bootstrap procedure to test the significance of the differences between the moderation effects (i.e., ${\text{b}}_{\text{4}}$ in the formula above) for the SET vs. ST techniques. No observations for the outcome variable (except for seven patients who dropped out and were lost to follow-up) or moderators were missing. For patient perception of the techniques, 5.4% of the observations were missing.

The analyses code appears in the online supplements.

Results

Preliminary analyses

The effect of techniques on subsequent outcome was significant for both ST (B=−0.73, S.E.=0.17, t=−4.26, p<.0001) and SET (B=−0.85, S.E.=0.18, t=−4.64, p<.0001). The bootstrap procedure yielded no significant differences between SET and ST in their effect outcome (a difference of 0.12, 95% CI [−0.29,0.55]).

Identifying significant moderators of the technique-outcome relationship

Table 2 reports the estimated interaction (b₄ in the Formula 2), separately for each potential moderator, including both the uncorrected and the FDR-corrected p-values. As can be seen in Table 2, for SET techniques, two potential moderators were significant, gender and years of education, but they became non-significant after FDR-correction. For ST techniques, both attachment anxiety and domineering interpersonal pattern were found to be significant moderators, and they remained significant following FDR-correction.

Table 2.

Potential moderators of the effect of SET and ST techniques on subsequent outcome

Technique	Potential moderator	Beta	SE	p value	Adjusted p
SET	HRSD	−0.051	0.193	0.793	0.793
	Attachment avoidance	0.070	0.178	0.694	0.771
	Attachment anxiety	−0.083	0.177	0.639	0.771
	Dominance	−0.239	0.177	0.178	0.297
	Affiliation	0.161	0.172	0.348	0.497
	E-WAI	−0.300	0.183	0.101	0.201
	Age	−0.396	0.210	0.060	0.199
	Gender	0.979	0.367	0.008	0.076
	Education	−0.179	0.084	0.033	0.167
	Personality disorders	−0.699	0.420	0.096	0.201
ST	HRSD	−0.114	0.175	0.517	0.603
	Attachment avoidance	−0.137	0.173	0.428	0.603
	Attachment anxiety	−0.522	0.185	0.005	0.024
	Dominance	−0.459	0.159	0.004	0.024
	Affiliation	0.088	0.170	0.606	0.606
	E-WAI	−0.097	0.157	0.538	0.603
	Age	0.115	0.189	0.542	0.603
	Gender	0.555	0.341	0.103	0.345
	Education	−0.092	0.083	0.269	0.603
	Personality disorders	−0.372	0.361	0.304	0.603

These findings can be understood as the effect of each potential moderator on SET-outcome and ST-outcome associations. For attachment anxiety, an increase of 1 standard deviation reduces the coefficient of the effect of ST on subsequent outcome by 0.52 (controlling for the outcome level in the current session) but it reduces the effect of SET on the outcome coefficient only by 0.08. That is, attachment anxiety has a 6.5 (0.54/0.08) times greater effect on moderating the within-individual effect of the use of ST techniques in a particular session on subsequent outcome than on moderating the within-individual SET-outcome association. Similarly, for domineering interpersonal pattern, an increase of 1 standard deviation reduces the coefficient of the effect of ST on subsequent outcome by 0.46 (controlling for the outcome level in current session) but it reduces the effect of SET on the outcome coefficient only by 0.23. That is, it has a 1.95 (0.46/0.23) times greater effect on ST-outcome association than on the SET-outcome association.

Testing the significance of the differences between SET and ST for each moderator

The differences between the estimated interactions (b₄ in the Formula 2) of ST and SET, and their 95% CI based on bootstrap resampling (2,000 samples) are presented in Table 3. As can be seen in the table, the differences were significant only for attachment anxiety and age. Specifically, for attachment anxiety, the difference between SET and ST was significant, such that those with higher attachment anxiety benefiting more from greater use of ST than SET techniques, as exhibited by greater subsequent symptom reduction (see Figure 1). By contrast, in the case of age, older patients benefited more from SET than ST, as manifested by greater subsequent symptom reduction.

Table 3.

Significance of the differential effects for each moderator for SET vs. ST

Potential moderator	Differences	LL	UL
HRSD	−0.06	−0.45	0.34
Attachment avoidance	−0.21	−0.57	0.17
Attachment anxiety	−0.44	−0.86	−0.02
Dominance	−0.22	−0.57	0.15
Affiliation	−0.07	−0.44	0.28
E-WAI	0.20	−0.17	0.55
Age	0.51	0.10	0.95
Gender	−0.42	−1.21	0.40
Education	0.09	−0.08	0.24
Personality disorders	0.33	−0.55	1.23

Figure 1. The effects of technique (SET vs. ST) on subsequent outcomes.

Note. Effect = The within-patient effects of technique (SET vs. ST) on subsequent outcome; ST = Supportive treatment; SET = Supportive-expressive treatment.

Sensitivity analyses.

Raising the number of bootstrap resampling from 2,000 to 5,000 samples within each type of technique, yielded similar findings (see Table S1 in the online supplements).

Discussion

Identifying which therapeutic techniques are most effective for a given patient in a particular session is of critical importance for optimizing clinical practice. Yet, to date, no study tested this question directly, and empirical knowledge has mainly been based on moderators of the between-group differences between whole treatment packages and treatment outcome. To the best of our knowledge, this study is the first to explore whether the two converge. The findings are unambiguous: the moderation effect identified at the between-patient level not only cannot be simply applied to within-patient processes, but the effects may be in opposite directions. Such distinct patterns of findings between moderators of between-individuals and within-individual processes raise critical concerns about the ability to infer from one to the other and suggest that psychotherapy research should investigate moderators where the processes of interest occur: between-individual vs. within-individuals.

The present study is based on an RCT designed to test pre-registered moderators at the between-group level. Specifically, the RCT was designed to test whether patients’ attachment orientation may moderate their ability to benefit from ST vs. SET treatment packages (Zilcha-Mano et al., 2018). The findings supported the pre-registered hypothesis that individuals with higher levels of attachment anxiety exhibit better treatment outcomes in SET than in ST (Zilcha-Mano et al., 2021). The findings from the present study’s within-patient analysis converge in supporting the importance of attachment anxiety as a moderator, but suggest a meaningfully different effect: patients with higher levels of attachment anxiety benefited from greater use of ST than SET techniques. Therefore, the two analyses suggest conflicting clinical recommendations for which techniques should be implemented with patients with higher levels of attachment anxiety.

Complementing analyses identifying moderators of between-patient effects (i.e., the effects of treatment packages on outcome) with analyses identifying moderators of within-patient processes (i.e., the effects of techniques used in a given session on subsequent outcomes) is timely given the growing interest in within-patient therapeutic processes (Webb et al., 2021). The recent increased focus on within-patient over between-patients processes in psychotherapy research has been the product of two main factors. First, it became increasingly feasible to conduct within-patient analyses due to psychotherapy research designs including repeated measurements of process and outcome variables, replacing the traditional, yet suboptimal, pre-post treatment study design (Lutz et al., 2021). Second, it has become increasingly essential to focus on within-patient processes because methodological studies in and outside of the psychological sciences have demonstrated convincingly that within-individual processes cannot be inferred from results of between-individual analyses (Fisher et al., 2018). Informed by these findings, researchers have started to disentangle the two effects (Curran & Bauer, 2011; Wang & Maxwell, 2015). As a result, studies showing distinct associations at the between- vs. within-individual levels have started to accumulate (e.g., Webb et al., 2019; Zilcha-Mano, 2021; Zilcha-Mano & Fisher, 2022). The present study contributes to this growing body of research by showing that moderators of between- vs. within-individual effects may also be strikingly inconsistent. Thus, the abundant literature on moderators of between-group difference in outcome does not seem to provide a reliable guide to answering one of the most critical questions in psychotherapy practice: Which techniques should be used at a particular session to achieve most effective subsequent outcome with a particular patient? Such between-group moderation findings are at a level too coarse to inform personalized, patient-specific therapeutic prescriptions.

Moderation of the between-patient vs. within-patient effects may have implications for both selecting the most effective treatment for an individual patient and for deciding which techniques to prioritize in a given session. Regarding treatment selection, those with higher levels of attachment anxiety may benefit most from assignment to SET treatment. Regarding personalized decisions of which technique to use in a given session, those with higher levels of attachment anxiety may benefit most from increasing use of ST techniques. But what might account for the fact that the between- vs within-patient moderation effects were in the opposite direction? At the between-person level, according to attachment theory, individuals with high levels of attachment anxiety tend to show hyperactivation of the attachment system, as manifested in exaggerated proximity-seeking tendencies (Mikulincer & Shaver, 2017). An assignment to a treatment that challenges their characteristic level of activation of the attachment system may be most effective (Mallinckrodt, 2010). Thus, patients with higher levels of attachment anxiety are expected to benefit most from treatments where the main mechanism of change challenges their maladaptive interpersonal behavior of exaggerated proximity seeking (Daly & Mallinckrodt, 2009), such as SET (Luborsky et al., 1995). But at the within-person, session-to-session level, such therapeutic challenges may be best implemented while using ST techniques to support the patient’s struggles and efforts, to ensure engagement in treatment and reduce the risk of dropout. Pending replication, patients characterized by high levels of attachment anxiety may benefit from being assigned to supportive-expressive treatment but may experience relatively greater symptom reduction if their therapists provide them with direct guidance on how to change (the expressive part of the treatment; Luborsky et al., 1995) in a way that validates their perspective and accepts them (Linehan, 2014; Wachtel, 2013). Adopting a validating approach while encouraging patients to change may contribute to updating maladaptive interpersonal expectations that characterize individuals with high levels of attachment anxiety (Mikulincer & Shaver, 2017). That is, although therapists seek to facilitate change in their patients’ perceptions and reactions to interpersonal events, they do so in an empathic manner, conveying that they can see the world through the patients’ eyes, understanding and appreciating their perspective. This post hoc explanation is consistent with the literature suggesting the advantages not only of strategies contradicting but also complementing the individuals’ interpersonal tendencies (Daly & Mallinckrodt, 2009). That is, whereas at the treatment package level, it is effective to contradict the patients’ interpersonal demanding tendencies using expressive techniques, at the session-to-session level, providing support may be important to keep the patient engaged and motivated.

While the present study focused on SET vs. ST as a proof of concept, its implications are expected to go beyond the particular techniques used here. The study stresses that it is important not to take for granted that moderators of between- vs. within-patient effects will converge. Using an example from a different treatment package, if findings suggest that a particular subgroup of patients do particularly well in CBT compared to another intervention, it does not necessarily mean that this is due to cognitive restructuring or BA techniques. There may be other elements of CBT that are therapeutic for that subgroup (e.g., factors related to the concrete problem-solving structure of CBT, the collaborative nature of CBT, Socratic dialogue, etc.). None of this would necessarily be captured by a “CBT techniques” measure. In other words, it is one type of decision to identify the treatment package best suited for an individual and another to identify which techniques to prioritize during the sessions and which are most therapeutic with a particular patient.

Among other notable findings from this study, a domineering interpersonal pattern and age moderated the within-patient effects of technique on outcome. These findings should be interpreted with caution because they either did not survive the alpha correction within the type of technique or did not show significant differences between the two types of techniques in the bootstrap resampling analysis. For age, the findings suggest that older patients benefited more from SET than ST with regards to symptom improvement. Because this finding did not emerge in the initial stage of testing the potential moderators for each technique, replication is needed. For the moderating effect of domineering, the findings suggest similar patterns of moderating effects as for attachment anxiety: Both higher levels of attachment anxiety and higher levels of domineering tendencies benefited more from ST than from SET. The two effects, for attachment anxiety and domineering interpersonal pattern can be interpreted as a greater need to receive support from the therapists for patients with greater interpersonal maladaptive tendencies characterized by a more demanding interpersonal style, as manifested in hyperactivation of the attachment system in attachment anxiety and in a more domineering attitude toward others. Furthermore, the distinct patterns of findings of the prescriptive vs. prognostic effects suggest the need to have a clear conceptual model of the effect of interest before conducting any analyses because different analyses answer different questions.

Several important limitations of the present study should be noted. Differences in methods and analytic approach between the between- vs within-patient moderation analyses may have accounted for at least some of the differences between the two: in the within-patient analyses, technique used in session were reported by the patient, the analytic models were based on different assumptions and modeling strategies (e.g., predicting next session outcome while accounting for current session outcome vs. group differences in the slope of symptom development in the between-person analysis), etc. The fact that the techniques were rated by the patients (e.g., as opposed to trained raters coding videotaped sessions for the use of various techniques) is a potential limitation of the present study, and future research should use observer ratings or automatic coding of sessions (e.g., Flemotomos et al., 2022), to mitigate the large number of coded sessions required to accurately estimate within-sessions effects. Patients’ perceptions of the use of techniques may differ from those of the therapists and external observers. Although therapists and external observers, who were not exposed to treatment conditions, rated the supportive sessions as not being characterized by the use of expressive techniques (Zilcha-Mano et al., 2021), patients did show variance in their ratings. This interesting observation is consistent with qualitative analyses demonstrating that patients develop a holistic perception of therapeutic sessions (Levitt et al., 2016, 2022). That is, even if therapists are not using expressive techniques, patients may find meaning to their experiences merely by sharing them with an attentive and supportive therapist, and, thus, may rate the session as including such elements. Moreover, whereas the between-patients findings may be cautiously interpreted as implying causality due to the use of an experimental manipulation (an RCT design), the within-individual findings (which are based on observational effects of patient’s perceptions of technique use on outcome) may not. Future studies implementing within-individual designs are needed to strengthen causal inferences. Additionally, in the present RCT, a pre-registered theory-driven approach for between-subjects analysis was chosen, whereas a data-driven approach was chosen for the within-subject analysis. However, this does not have to be the case, and a theory-driven approach can be used for within-subject analysis and a data-driven approach for between-subjects analysis. Other limitations of the present study are the relatively small sample size; notwithstanding this limitation, it should be noted that we had about 1,800 time points of data collection, supporting the validity of the present findings that were revealed by analyses correcting for multiple comparisons. Moreover, in the present RCT, the same therapists provided the two treatment conditions to control for the effect of therapist characteristics (years of experience, interpersonal abilities, skill in treating complex cases, etc.) on treatment outcome. Such a design may enable disentangling the effect of therapist characteristics from the effect of treatment. But the approach has its limitations, in particular it has been suggested that therapists may show a preference for one treatment over the other, which may in turn affect treatment outcome (therapist allegiance; Falkenström et al., 2012). To mitigate such risks, therapists were supervised in each treatment condition by supervisors practicing and publishing on each of the protocols. Additionally, before seeing the first patient in the trial, therapists were asked to report which of the conditions they expected to be more effective: the supportive, the supportive-expressive, or both. All responded that both were expected to be equally effective. Finally, the terms between- vs. within-patient effects refer here to treatment group differences in outcome and session-to-session technique-outcome associations, respectively. It should be noted that in both cases (the between- and within-patients effects), the moderators were between-patient characteristics (i.e., scores on baseline patient variables). Future studies can explore the potential effects of time-varying within-individual moderators (e.g., attachment anxiety ratings as they develop from one session to the next).

The present study demonstrates the critical importance of investigating moderators of the within-patient effect of therapeutic techniques on outcome. The study highlights the problem of trying to translate findings from studies on moderators of the effect of treatment packages on outcome to the clinically-relevant question of which techniques to use in a given session with a particular patient. The findings demonstrate the potential of data-driven approaches to identify moderators of within-patient processes to help inform which techniques are most effective for a given patient based on their pre-treatment characteristics. If replicated in future studies with larger samples, these findings may inform algorithm-based personalized prescriptions of which techniques are expected to have the greatest therapeutic benefit for a given patient.

Public health significance.

The study highlights the problem of trying to translate findings from studies on moderators of the effect of treatment packages on outcome to the clinically-relevant question of which techniques to use in a given session with a particular patient. The findings suggest that whereas at the treatment selection level, patients with higher levels of attachment anxiety may benefit most from assignment to supportive-expressive treatment, at a given session greater use of supportive techniques is most effective.

Data Transparency Statement:

The data reported in this manuscript were collected as part of a randomized controlled trial (RCT). This RCT has yielded several manuscripts with separate foci. However, only one study to date has used the outcome data of the trial and the treatment condition assignment to test moderators, that is the main outcome paper. Whereas the main outcome paper reported moderators of between-patients effect of treatment packages on outcome, the current work report on moderators of the within-patients effect of therapeutic techniques on outcome. The focus of the current work is the great contrast between the two.