Pre‐Post Differences in Facilitative Interpersonal Skills and Self‐Assessment Following Deliberate‐Practice Training

ABSTRACT

Problem

Although deliberate practice (DP) has shown results in other areas of research, its potential benefits in the field of training psychotheraists are still unclear.

Objectives

To analyze whether DP training improves facilitative interpersonal skills (FIS) and to understand whether the trainees' self‐assessment is congruent with their performance.

Method

A pre‐experimental study with a quantitative analysis was carried out in order to analyze the data from the Facilitative Interpersonal Skills Performance Task and Facilitative Interpersonal Skills Self‐Report instruments. The study utilized a convenience sample of 23 psychologists in the first year of an advanced specialization program in psychotherapy. In order to compare a hetero with a self‐assessment of therapists' performance, the data were collected before and after a DP training.

Results

Participants' observer‐rated FIS were significantly higher after the DP training than before it. Their self‐rated FIS exceeded observer ratings both before and after the training; nonetheless, self‐ratings increased in parallel with observed performance, suggesting that participants were aware of their own pre–post change.

1. Introduction

Despite the various contemporary challenges inherent in psychotherapy, the latter is effective (Abu Sabra et al. 2023; Barkham and Lambert 2021; Barkowski et al. 2020; Cuijpers 2017; Eielsen et al. 2022; Hilbert et al. 2019; Lambert 2013; Munder et al. 2019; Rahmani et al. 2020; Sousa 2017). Different psychotherapies do not differ significantly in terms of effectiveness, as it is the common factors, rather than the specific techniques, that predict most of the therapeutic results (Poster 2019). Specifically, the effects of the therapist, that is, the variance of therapeutic results derived exclusively by the therapist themselves, is around 5%–8% (Barkham et al. 2017; Chow et al. 2015; Johns et al. 2019). This variance, together with the study of expertise, shows us that there are therapists who consistently obtain better results when compared to their peers, as well as their opposite (Barkham et al. 2017; Heinonen and Nissen‐Lie 2020; Johns et al. 2019; Pereira et al. 2023; Vaz et al. 2023; Wampold 2015).

The characteristics of therapists that emphasize these differences have increased interest in research into clinical expertise, that is, therapists with a high level of proficiency (Barkham et al. 2017; Heinonen and Nissen‐Lie 2020). In general, the most effective therapists are those who take professional advantage of their interpersonal skills, showing a responsiveness that makes the therapeutic results independent of the patient's characteristics, such as the type of disorder or personality (Heinonen and Nissen‐Lie 2020). On the other hand, the characteristics of therapists that do not predict the effectiveness of psychotherapy relate to the age and gender of the therapist, the theoretical model and even their experience (Goldberg et al. 2016b; Pereira et al. 2023; Wampold 2015).

The construct of “facilitative interpersonal skills” (FIS), in the context of psychotherapy, relates to patterns of interpersonal and emotional behavior used by the therapist, based on the theory of common factors, which help to alleviate the suffering experienced by the patient (Allen et al. 2023; Anderson et al. 2019; Anderson and Patterson 2013; Anderson and Perlman 2022). The use of FIS has been shown in research to be a significant predictor of psychotherapeutic outcomes (Allen et al. 2023; Anderson et al. 2009, 2016). In other words, patients whose therapists have higher levels of FIS have higher rates of improvement compared to patients whose therapists have lower levels of FIS (Anderson et al. 2009, 2016).

The results of these studies are measured using the FIS Performance Task (Anderson and Patterson 2013). The aim of this instrument is to measure the effectiveness of a psychotherapist in particularly challenging and difficult interpersonal therapeutic situations (Allen et al. 2023; Anderson et al. 2009; Anderson and Patterson 2013). This instrument assesses the following skills: (1) Verbal Fluency; (2) Hope and Positive Expectations; (3) Persuasiveness; (4) Emotional Expression; (5) Warmth, Acceptance and Understanding; (6) Empathy; (7) Alliance Bond Capacity; (8) Alliance Rupture‐Repair Responsiveness (Anderson et al. 2019; Anderson and Patterson 2013). Regarding the self‐assessment of FIS, there is a tendency for these skills to be overestimated (Longley et al. 2023). In a study by Longley et al. (2023), the participants' self‐assessed FIS scores were significantly higher when compared to the FIS rating made by assessors (Longley et al. 2023). However, there seems to be a congruence between self‐assessment and evaluation, in the sense that the higher the self‐assessment, the higher the evaluation (Longley et al. 2023).

The study of expertise, with its increased interest in research, has had a major impact on the way professionals are understood and trained, not only in psychology, but in the community in general (Vaz et al. 2023). Experts, as a rule, master a specific area, where they process information more quickly, as well as performing their duties with lower margins of error than their peers, having superior self‐monitoring skills (Ericsson and Pool 2016; Vaz et al. 2023). Deliberate practice (DP) is a fundamental component in acquiring these high standards of competence (Vaz et al. 2023).

The renewal of psychotherapy training methods, which move trainees towards a more experiential and systematic method, seems to be the most appropriate way to improve clinical results and, consequently, achieve the expected expertise (Vaz et al. 2023). One of the main factors that differentiates DP from other methods is the focus on procedural learning, as opposed to conceptual learning (Mahon 2022; Vaz and Rousmaniere 2022; Vaz et al. 2023). The more traditional approach, focusing on conceptual and intellectual work remain important elements in DP, however they are best practiced in conjunction with procedural learning methods that involve repeated rehearsal of specific skills with immediate feedback (Mahon 2022; Vaz and Rousmaniere 2022; Vaz et al. 2023).

In the field of psychotherapy, the first steps are being taken to examine DP as a training method. Rather than focusing on isolated aspects of training, DP is better understood as a continuous, structured cycle aimed at fostering professional growth: (1) observing clinical work to identify specific areas for improvement; (2) receiving expert feedback to guide learning; (3) setting small, achievable learning goals that are challenging but attainable; (4) engaging in behavioral rehearsal through structured repetition of specific tasks; and (5) assessing performance critically to refine skills, closing the loop and restarting the cycle, fostering continuous professional growth (Rousmaniere et al. 2017). In other words, expertise is acquired through repetitive cycles of competence development activities, selected on the basis of the subject's level of performance, with the corresponding feedback and the pursuit of new objectives depending on the student's progress (Ericsson 2006; Ericsson and Harwell 2019).

Mahon (2022) presents a literature review of 14 articles related to the acquisition of therapeutic skills through DP, demonstrating that this type of training not only increases psychotherapeutic effectiveness, but is also more favorable in the acquisition of therapeutic techniques, communication skills and evidence‐based relational and interpersonal factors, when compared to didactic workshops.

Chow et al. (2015) were the first to study the effect of DP on psychotherapy. To do this, they compared the clinical results of psychotherapists with the total time spent on training and supervision activities (Chow et al. 2015). The results showed that the more time the participants invested in improving the proposed skills, outside of session time, the better the clinical results (Chow et al. 2015). These more effective therapists invested 2.8 times more in DP‐related activities (Chow et al. 2015). Three similar studies (Di Bartolomeo et al. 2021; Shukla et al. 2021; Westra et al. 2021) tried to study the difference between training through deliberate practice versus training through more traditional, that is, more didactic, methods through workshops. In all three studies, DP training proved to be more effective than more traditional training (Di Bartolomeo et al. 2021; Shukla et al. 2021; Westra et al. 2021).

In an empirical study, Anderson et al. (2019) came up with results that demonstrate the effectiveness of training in the acquisition of FIS. Specifically, the training consisted of observing model responses to videos, as well as repeated practice of different responses (Anderson et al. 2019). It has also been shown that this training helps the therapist to focus on the experiential process and emotions of their patients (Anderson et al. 2022). Similarly, Perlman et al. (2020, 2023) positively correlated training focused on the therapeutic alliance and FIS with a more effective acquisition of skills when compared to more didactic teaching methods.

Although DP in psychotherapy, particularly in the training of psychotherapists, has been showing its benefits, some caution is needed when making generalizations. As research in this area is in its infancy, there are still different criteria used to define deliberate practice training (Chow et al. 2015; Ericsson and Harwell 2019; Mahon 2022). In any case, the identification of the 8 facilitating interpersonal skills considered in the FIS Performance Task as predictors of results shows the relevance of using them in therapist training (Allen et al. 2023; Anderson and Patterson 2013; Anderson et al. 2009). If research makes it possible to identify the skills of master therapists, as well as showing their effectiveness in training and evaluating therapists, it is assumed that this training will promote a general tendency for the average therapeutic results to increase (Allen et al. 2023; Anderson et al. 2009). This research has two objectives, the first of which focuses on examining whether trainees' interpersonal facilitation skills differ following deliberate‐practice training and the second on comparing the hetero‐evaluation of FIS with the respective self‐assessment. In this sense, two hypotheses are put forward: (H1) trainees' interpersonal facilitating skills will be significantly higher following the deliberate‐practice training; (H2) self‐assessment of interpersonal facilitating skills will be higher than hetero‐assessment of these same skills.

2. Method

2.1. Overview

This study uses a quantitative research methodology in order to meet the hypotheses mentioned above. This type of methodology uses statistics as a method of analyzing quantitative data, based on numbers and indicators that allow, among other things, the visualization and analysis of graphs (Paranhos et al. 2016). This is a pre‐experimental study, since in order to observe the evolution of the participants, data were collected at two different points in time—before and after the DP training. This data collection took place on the Skillsetter platform between February and May, as the trainees provided their answers. At the same time, they emailed their self‐assessment. Three evaluators rated their answers and then calculated the intraclass correlation coefficient (ICC) in order to test the reliability, agreement and correlation between them.

2.2. Participants

The study sample was a convenience sample of 23 psychologists in the first year of an advanced specialization program in psychotherapy. Specifically, the data were collected as part of the “deliberate practice in psychotherapy I” course. From the first to the second moment there was one dropout, so the analyses that include the post‐training of DP are made up of only 22 participants.

Sociodemographic data were collected online and anonymously using the Google Forms platform. Participants were aged between 25 and 60 (M = 38.09; SD = 9.93), 18 were female (78.26%) and 5 were male (21.74%) (Table 1). The number of years of clinical practice varied between 1 year and 29 years (M = 5.85; SD = 7.29) and the number of hours per week of practice varied between 1 h and 38 h (M = 18.5; SD = 10.79) (Table 1). As for the participants' theoretical model, 19 use the integrative model (82.61%), 2 the humanistic‐existential model (8.7%), 1 the psychodynamic model (4.35%) and 1 the cognitive‐behavioral model (4.35%) (Table 1).

Table 1.

Participants' sociodemographic data (N = 23).

Features
	Minimum‐Maximum	M (SD)
Age	25–60	38,09 (9,93)
Years of clinical practice	1–29	5,85 (7,29)
Weekly clinical practice hours	1–38	18,5 (10,79)

	Frequencies (n)	Percentages (%)
Gender
Male	5	21,74
Female	18	78,26
Theoretical model
Humanist‐existential	2	8,70
Integrative	19	82,61
Psychodynamic	1	4,35
Cognitive‐behavioral	1	4,35

2.3. Materials

2.3.1. Sociodemographic Questionnaire and Informed Consent

As mentioned above, the sociodemographic questionnaire was completed using Google Forms. The informed consent form was sent, duly completed and returned by e‐mail.

2.3.2. “Stimulus‐Videos”

The participants had access to the “stimulus videos” on the Skillsetter online platform. This platform is intended to be used for psychotherapy courses based on deliberate practice. Totaling a set of 9 videos (before and after DP training), where each one represents a part of a real psychotherapy session (staged by actors), particularly challenging (Allen et al. 2023; Anderson et al. 2019; Anderson and Patterson 2013). At critical moments, the video‐stimulus is interrupted and the participant is asked to respond to the patient, as if they were the therapist (Allen et al. 2023; Anderson et al. 2019; Anderson and Patterson 2013).

Video 1 (48 s) portrays a patient who wants to draw up a plan and set goals in order to see rapid changes. Video 2 (70 s) depicts a patient confused by the therapeutic process, demanding greater professionalism and faster results from the therapist. Video 3 (49 s) shows a patient who is unmotivated and frustrated with the therapeutic process because she feels that everything remains the same. Video 4 (58 s) shows a patient who says she is frustrated because she feels the therapist can't help her. Video 5 (57 s) depicts a patient who, after following his therapist's advice, claims that it didn't work, demanding an explanation. Video 6 (53 s) shows a patient who feels judged by her therapist. Video 7 (32 s) shows a patient questioning the therapist about what the latter thinks of him. Video 8 (115 s) shows a patient who has doubts about whether the therapist will be able to support and understand her. Video 9 (144 s) shows a patient who feels that he is not making as much progress as he would like.

2.3.3. Facilitative Interpersonal Skills Performance Task

In order to measure the trainees' progress throughout the program, the Facilitative Interpersonal Skills Performance Task (Anderson and Patterson 2013) was used. The FIS performance task is an instrument that aims to measure the effectiveness of a psychotherapist in particularly challenging interpersonal therapeutic situations (Allen et al. 2023; Anderson et al. 2009; Anderson and Patterson 2013). The “FIS method” consists of two parts, the performance‐based task and the rating of the respective responses (Anderson et al. 2019). The first part corresponds to the viewing and recording of the responses to the videos mentioned, and the second to the respective rating by the evaluators of these responses, with the aid of a manual (Appendix A) (Allen et al. 2023; Anderson et al. 2019; Anderson and Patterson 2013).

In part, the FIS Performance Task is made up of a set of competencies recognized as early as 1957 by Carl Rogers as being necessary and sufficient conditions for therapeutic change (Rogers 1957). The 8 items included in this assessment measure are (1) Verbal Fluency; (2) Hope and Positive Expectations; (3) Persuasiveness; (4) Emotional Expression; (5) Warmth, Acceptance and Understanding; (6) Empathy; (7) Alliance Bond Capacity; (8) Alliance Rupture‐Repair Responsiveness (Anderson et al. 2019; Anderson and Patterson 2013). The evaluators rate each answer following a 5‐point likert scale for the 8 items, ranging from (1) “Not Characteristic” to (5) “Extremely Characteristic,” with the help of a more elaborate description of each of the 5 points (Anderson et al. 2019; Anderson and Patterson 2013). Scores of “1” and “2” represent deficits in this skill, “3” is the neutral level and “4” and “5” indicate mastery (Anderson et al. 2009, 2020; Anderson and Patterson 2013).

Psychometrically, the FIS Performance Task has strong validity and reliability (Allen et al. 2023; Anderson and Patterson 2013). The way the videos were created means that the FIS inherently has very high ecological and face validity (Allen et al. 2023; Anderson and Patterson 2013). The 8 items are shown to represent a single, internally consistent construct (Anderson and Patterson 2013). The internal consistency between the items is moderate to high, with values ranging from α = 0.43 to 0.89 (Anderson and Patterson 2013).

2.3.4. FIS Self‐Report

To analyze the participants' self‐assessment of their FIS, we used the FIS Self‐Report, a questionnaire with eight questions representing the 8 dimensions of the FIS, in order to facilitate the comparison between self‐ and hetero‐assessment (Anderson et al. 2019). To do this, the participants answered the questionnaire for each of the nine videos, before and after the DP training. In this case, the order of the items differs from the order previously presented for the FIS, as follows: (1) Empathy; (2) Warmth, Acceptance and Understanding; (3) Persuasiveness; (4) Alliance Bond Capacity; (5) Hope and Positive Expectations; (6) Alliance Rupture‐Repair Responsiveness; (7) Verbal Fluency; (8) Emotional Expression. The rating is inverted in relation to the FIS rating, i.e. (1) “Strongly Agree” to (5) “Strongly Disagree”. This instrument has a high internal consistency with α = 0.91 (Anderson et al. 2019). The questionnaire was sent out, duly completed and returned by email.

2.4. Procedures

The data for this study were collected from a group of 23 psychologists who were attending the first year of an advanced specialization program in psychotherapy. The program is based on a contextual and transtheoretical logic which, in other words, implies training based on factors common to different theoretical models. This training has a total of three academic years and a syllabus that divides its workload between theoretical/methodological classes, clinical supervision and clinical practice. In this research, the data were collected within the framework of the module “deliberate practice in psychotherapy I,” a first‐year module with a workload of 22 h.

Prior to the start date of the module, participants were sent access to the Skillsetter platform and FIS‐SR via email, so that they could respond to the stimulus videos before starting their training, as well as self‐assess their performance for each of the nine videos. These responses consisted of responding to the “patients” as if they were their therapists, with only one possible response, so as to resemble a real‐life context as much as possible. The answers were recorded via webcam and made accessible on the platform. Along with this, informed consent and a sociodemographic questionnaire were also sent, which were duly completed. The module took place between February 19, 2024 and May 6, 2024, and shortly after it ended, the participants repeated this procedure, that is, they answered the same stimulus videos again on the Skillsetter platform, as well as filling in the FIS‐SR.

During the module, the training followed the deliberate‐practice cycle outlined above: trainees engaged in repeated behavioral rehearsal of responses to the challenging clinical stimuli, received individualized corrective feedback, set specific learning goals slightly beyond their current level of skill, and revised their responses across successive iterations under the guidance of the instructor. For example, a trainee would respond to a given stimulus, receive structured feedback on the targeted skills, set a concrete goal for the next attempt, and then re‐attempt the response. What primarily distinguished this training from more didactic, DP‐informed formats was this emphasis on repeated, feedback‐driven rehearsal of the eight FIS rather than on conceptual instruction. Formal fidelity or adherence monitoring of the deliberate‐practice protocol was not conducted; this should be borne in mind when interpreting the findings, as it limits the extent to which the delivered training can be verified against the intended protocol.

2.5. Data Analysis

Three raters were assigned to rate the answers, two second‐year students on the master's degree in Clinical Psychology and one first‐year student on the master's degree in Clinical Psychology. Before grading, the three graders studied the FIS manual extensively. To ensure alignment and support throughout the process, the raters participated in supervision meetings led by their professor, who provided guidance and clarification. The raters were graduate students in Clinical Psychology who were not involved in teaching, supervising, or evaluating the trainees within the specialization program. Because the recordings were obtained at two timepoints, the raters were not masked as to whether a given response was recorded before or after the training; to mitigate potential bias, all responses were rated independently before agreement was examined. The answers were rated individually in order to eliminate possible biases. Initially, to align the raters with each other, the answers corresponding to the first video were rated individually, followed by a meeting to discuss these answers. After confirming that all the raters were guided by the same criteria, the next three videos (videos 2, 3, and 4) were rated individually. Once the four videos had been rated, the intraclass correlation coefficient (ICC) was calculated, a measure that aims to test the reliability, agreement and correlation between raters who rated the same group of subjects (Koo and Li 2016). The ICC was calculated with the aid of IBM SPSS Statistics 29.0 using the two‐way mixed‐effects model for absolute agreement between three raters (k = 3).

In video 1 the ICC intraclass correlation value had an average of 0.945 with 95% CI [0.906, 0.973] (F(22, 506 = 22.42, p < 0.001); in video 2 there was an average of 0.957 with 95% CI [0.926, 0.979] (F(22, 506) = 31.74, p < 0.001); in video 3 there was an average of 0.931 with 95% CI [0.883, 0.966] (F(22, 506) = 17.31, p < 0.001); finally, in video 4 the ICC was 0.953 with 95% CI [0.920, 0.977] (F(22, 506) = 24.37, p < 0.001 (Appendix B), thus demonstrating excellent reliability between the 3 evaluators (Koo and Li 2016).

Once inter‐rater reliability had been confirmed, the remaining videos were rated individually. The data for the FIS and FIS‐SR were entered into IBM SPSS Statistics 29.0. In the case of the FIS, the average of the three raters for each dimension of each video‐stimulus was used for statistical analysis. This analysis used t‐student tests for paired samples in order to make comparisons between the different variables.

When compared to the FIS Performance Task, the FIS‐SR scores are inverted and the items are arranged in a different order. For data analysis purposes, the FIS‐SR score was again inverted by entering it into IBM SPSS Statistics 29.0 to resemble the FIS Performance Task score, and the items were rearranged to match the FIS Performance Task items.

3. Results

3.1. Comparison of FIS (Hetero‐Evaluation) Between Before and After DP Training

By averaging the FIS dimensions and applying a t‐student for paired samples, we can see in Figure 1 that the difference between the FIS before DP training (M = 2.71, SD = 0.29) and the FIS after DP training (M = 2.98, SD = 0.19) is statistically significant (t (21) = 8.01, p < 0.001, Cohen's d = 1.71).

Figure 1.

Comparison of FIS (hetero‐evaluation) between before and after DP training.

Each bar in Figure 2 represents the average score of the three evaluators, at the respective moment, for each FIS dimension, with the respective error bar with a 95% confidence interval. We can see that after the DP training, on average, all the participants improved each of the FIS skills.

Figure 2.

Comparison of FIS dimensions (hetero‐evaluation) between before and after DP training.

After applying a t‐student test for paired samples (Appendix C), it was found that the difference between before and after training was statistically significant in all dimensions. Item 1—Verbal Fluency (t (21) = 8.249, p < 0.001), item 2—Hope and Positive Expectations (t (21) = 4.473, p < 0.001), item 3—Persuasiveness (t (21) = 11.063, p < 0.001), item 4—Emotional Expression (t (21) = 7.763, p < 0.001), item 5—Warmth, Acceptance and Understanding (t (21) = 2.9, p = 0.009), item 6—Empathy (t (21) = 3.969, p <0.001), item 7—Alliance Bond Capacity (t (21) = 3.304, p = 0.003), item 8—Alliance Rupture‐Repair Responsiveness (t (21) = 5.835, p < 0.001) (Appendix C).

3.2. Comparison of the FIS (Hetero‐Assessment) With the FIS‐SR (Self‐Assessment)

The analyses indicate that the difference between the FIS and the FIS‐SR is statistically significant both before the DP training (FIS M = 2.73, SD = 0.29; FIS‐SR M = 2.97, SD = 0.49; t (22) = 2.59, p = 0.017, Cohen's d = 0.54) and after the DP training (FIS M = 2.98, SD = 0.19; FIS‐SR M = 3.47, SD = 0.63; t (21) = 3.63, p = 0.002, Cohen's d = 0.77). In other words, the participants overestimated their FIS skills. Figures 3 and 4 illustrate these differences, before and after DP training, respectively.

Figure 3.

Comparison of FIS (hetero‐evaluation) with FIS‐SR (self‐evaluation) before DP training.

Figure 4.

Comparison of FIS (hetero‐evaluation) with FIS‐SR (self‐evaluation) after DP training.

In Appendix D and Appendix E, respectively, we can find the dimension‐by‐dimension pairings of the comparison between the FIS and the FIS‐SR before the DP training and the comparison of the FIS with the FIS‐SR after the DP training. All the dimensions of the FIS‐SR scored higher than the FIS at both times. The dimensions that showed a statistically significant difference between the FIS and the FIS‐SR before DP training were item 4—Emotional Expression (t (22) = 3.163, p = 0.005), item 6—Empathy (t (22) = 2.293, p = 0.032), item 7—Alliance Bond Capacity (t (22) = 2.172, p = 0.041) and item 8—Alliance Rupture‐Repair Responsiveness (t (22) = 5.055, p < 0.001). The items whose difference was not statistically significant were item 1—Verbal Fluency (t (22) = 0.464, p = 0.647), item 2—Hope and Positive Expectations (t (22) = 0.034, p = 0.973), item 3—Persuasiveness (t (22) = 0.535, p = 0.598) and item 5—Warmth, Acceptance and Understanding (t (22) = 1.857, p = 0.077) (Appendix D).

Regarding the differences between the FIS and the FIS‐SR after DP training, the dimensions corresponding to item 4—Emotional Expression (t (21) = 4.237, p < 0.001), item 5—Warmth, Acceptance and Understanding (t (21) = 3.424, p < 0.001), item 6—Empathy (t (21) = 4.923, p < 0.001), item 7—Alliance Bond Capacity (t (21) = 4.146, p < 0.001), item 8—Alliance Rupture‐Repair Responsiveness (t (21) = 3.178, p = 0.005). The dimensions whose differences were not statistically significant concern item 1—Verbal Fluency (t (21) = 1.548, p = 0.137), item 2—Hope and Positive Expectations (t (21) = 1.713, p = 0.101), item 3—Persuasiveness (t (21) = 0.57, p = 0.575) (Appendix E).

3.3. Comparison of FIS‐SR (Self‐Assessment) Between Before and After DP Training

As illustrated in Figure 5, participants rated their own FIS as significantly higher after the DP training (FIS‐SR before M = 2.97, SD = 0.50; after M = 3.47, SD = 0.63) than before it (t(21) = 3.45, p = 0.002, Cohen's d = 0.74).

Figure 5.

Comparison of FIS‐SR (self‐assessment) before and after DP training.

Appendix F shows the dimension‐by‐dimension pairings for the purpose of comparing the FIS‐SR before and after DP training. In all dimensions, the score after DP training was higher than the score before DP training. With the exception of item 8—Alliance Rupture‐Repair Responsiveness (t (21) = 1.81, p = 0.085), the remaining dimensions showed a statistically significant difference: item 1—Verbal Fluency (t (21) = 2.144, p = 0.044), item 2—Hope and Positive Expectations (t (21) = 2.681, p = 0.014), item 3—Persuasiveness (t (21) = 2.381, p = 0.027), item 4—Emotional Expression (t (21) = 3.194, p = 0.004), item 5—Warmth, Acceptance and Understanding (t (21) = 3.567, p = 0.002), item 6—Empathy (t (21) = 3.916, p < 0.001), item 7—Alliance Bond Capacity (t (21) = 3.523, p = 0.002) (Appendix F).

4. Discussion

The main aim of this study was to examine pre–post differences in FIS following DP training, as well as to assess participants' perceptions of these skills. To this end, a convenience sample of 23 psychologists training in advanced specialization in psychotherapy was used. During a 22‐h “deliberate practice in psychotherapy I” module, the participants had the opportunity to train the FIS following the DP training model. There was thus the opportunity to quantitatively measure the responses to the video‐stimuli before and after this training.

The results were in line with both H1 and H2. Regarding H1, the participants' responses after the DP training were significantly better, on average, when compared to their responses before the training (Figure 1). More specifically, there was a significant improvement in all 8 skills (Appendix C). These results indicate that participants' observer‐rated FIS were significantly higher following the DP training. In general, knowledge of outcome‐predicting skills not only expands our understanding of how therapy works, but also has practical implications for therapist training (Barkham et al. 2017; Caçador et al. 2023; Edmondstone et al. 2023; Johns et al. 2019; Nissen‐Lie et al. 2017; Heinonen and Nissen‐Lie 2020). Overall, therapist training should reduce the magnitude of the discrepancy between both ends of a continuum of inter‐therapist effectiveness (Edmondstone et al. 2023). In other studies, even though it is a movement in its infancy and the principles of DP may sometimes not be perfectly mirrored in psychotherapist training, this body of work suggests that DP may be a promising method for improving therapeutic skills (Axelsson et al. 2023; Barrett‐Naylor et al. 2020; Chow et al. 2015; Clements‐Hickman and Reese 2020; Di Bartolomeo et al. 2021; Goldberg et al. 2016a; Hill et al. 2020; Larsson et al. 2023; Mahon 2022; McLeod 2021; Miller et al. 2020; Newman et al. 2022; Ogles et al. 2023; Shukla et al. 2021; Westra et al. 2021; Yamin et al. 2023; Young and Maack 2021; Young et al. 2023).

Although the data analysis showed that there was not only a statistically significant overall increase in FIS skills (Figure 1), but also in each skill individually (Appendix C), it should be noted that this increase was from an average FIS of 2.711–2.976 (Figure 1). Although participants' FIS were significantly higher following the training, the post‐training scores did not reach values that represent a higher level of proficiency. As a reminder, scores of “1” and “2” represent deficits in the skill in question, “3” is the neutral level and “4” and “5” indicate mastery (Anderson et al. 2009; Anderson and Patterson 2013). The literature states that, although there is no “magic number” of hours, the more time invested in DP, the greater the possibility of improving performance in a given domain (Ericsson 2021; Ericsson and Harwell 2019; Ericsson et al. 1993; Miller et al. 2020). Thus, given that the process of developing skills is, in a sense, never‐ending, these results would be expected, given that these are therapists in training, whose training was spread over 2 h a week, for a total of 22 h. It is therefore hypothesized that more time would be needed to reach higher levels of proficiency, since the process of improvement tends to take longer. Nevertheless, participants showed clear pre–post differences in FIS following the training.

Overall, DP, focused on individualized learning in a challenging skill zone, with immediate feedback and successive improvements and evaluations, has been identified as a method that takes people to higher levels of expertise and outcomes (Mahon 2022). In general, once the DP criteria have been met, it can be used to improve any therapeutic practice (Mahon 2022). The FIS of the therapists in this study were significantly higher following the DP training, which involved consistent training and supervision with individualized feedback. Supervision by a specialist is an essential factor during DP (Mahon 2022; Young and Maack 2021). They help to provide feedback and appropriate exercises, to identify cases where progress is not being made, to direct the students' attention to more specific areas and to adapt and identify the limits of each individual (Mahon 2022; Young and Maack 2021). DP requires a high level of concentration, a full focus on the task, as well as a greater personal effort than usual, so those who practice it, in addition to assuming it as essential training, also see DP as an activity that is not inherently enjoyable, nor with an immediate reward associated with it (Ericsson and Pool 2016; Hill et al. 2020; McLeod 2021). The participants' own sustained effort and engagement should likewise be acknowledged.

With regard to H2, it was observed that the participants' perception of their performance in the stimulus videos was, on average, significantly overestimated, both before and after the DP training (Figures 3 and 4). However, when we look at each dimension, with the help of Appendix D and Appendix E, although the self‐assessment was always higher than the hetero‐assessment, we notice that this difference was only significant in the “Emotional Expression,” “Empathy,” “Alliance Bond Capacity” and “Alliance Rupture‐Repair Responsiveness” dimensions, adding the “Warmth, Acceptance, & Understanding” dimension after the DP training.

It is interesting to note that, despite the incongruence with the hetero‐evaluation of the FIS, there was a congruence in its evolutionary sense. In other words, the participants were aware of their evolution, since the FIS‐SR after DP training had significantly higher results compared to the FIS‐SR before DP training (Figure 5), as was the case with the FIS (Figure 1). Similar results were found in a study by Longley et al. (2023), in which this same over‐evaluation of the FIS itself was found, and it was also found that the higher the hetero‐evaluation, the higher the self‐evaluation. According to these same authors, becoming aware that this overestimation is a common phenomenon could help improve the acceptance of feedback and self‐questioning of their own skills (Longley et al. 2023). This tendency can be explained by a cognitive bias called the “Dunning‐Kruger effect,” a bias where people with poorer performances are unaware of the level of their expertise and overestimate their skills (Dunning 2011). Although this phenomenon may be associated with a greater production of errors, as well as a lack of self‐awareness, Dunning (2006) argues that this overestimation may be correlated with levels of optimism which may be beneficial in dealing with challenging situations.

Although the training of psychotherapists through the use of DP is in its infancy, it has shown itself to be an emerging topic in the literature, particularly with a view to the effectiveness of improving expertise and clinical outcomes (Mahon 2022; Vaz and Rousmaniere 2022). In order for the field of psychotherapy to progress, it is necessary to go beyond traditional teaching methods, and begin to take a closer look at more experiential training practices such as DP (Mahon 2022). The systematic training of psychotherapists has been shown to be a promising method for developing professional expertise, especially through the acquisition of evidence‐based psychotherapeutic skills (Anderson et al. 2019; Mahon 2022; McLeod 2021; Perlman et al. 2020; Vaz et al. 2023; Young et al. 2023). For a therapist seeking to improve their clinical outcomes through DP, what makes sense to train, in general, are the evidence‐based relational variables, unless a critical deficit is identified in another modality (Mahon 2022).

4.1. Limitations and Future Studies

This study has some limitations to take into account which, consequently, contribute to future studies. The first, and perhaps the main one, concerns the sample. Although the study benefits from the fact that the sample is represented only by therapists, its size represents a limitation—23 therapists before DP training and 22 after DP training. Its small size reduces the confidence with which these results can be generalized. By comparison, a similar study by Anderson et al. (2019) had a sample of 101 participants, but the latter were not therapists but psychology students.

As a second limitation, we can hypothesize that the fact that the therapists, when responding after the DP training, did not have the “surprise factor” of the therapeutic situation in question, may have influenced their responses in a positive direction, making it difficult to isolate the “DP” variable. To combat this bias, a possible solution for a future study would be for the participants to be presented with nine different videos, but equally representative of the previous ones. An alternative to this last solution would be, with a larger sample size, to separate it into two groups where at first “group A” would respond to the first set of nine videos and “group B” to the second set of 9 videos, and at the second moment, after the DP training, they would alternate the set of videos to respond to. In this way, it would be possible to identify whether there had been an increase in performance in relation to the dimensions of the FIS, coming as close as possible to a real context.

In a future study, it might be interesting to follow these same participants again, to see if, as the course went on and consequently as they practiced, their performance would be closer to expertise.

Finally, the single‐group pre–post design, without a control or comparison condition, means that the observed differences cannot be unambiguously attributed to the deliberate‐practice training; alternative explanations such as repeated exposure to the task, concurrent training activities, or maturation cannot be ruled out. As noted above, the raters were also not masked to the timepoint of each performance, which may have introduced expectancy bias despite independent rating and high inter‐rater reliability, and fidelity to the deliberate‐practice protocol was not formally monitored. Taken together, these features support interpreting the present findings as preliminary and observational rather than as evidence of a causal effect of the intervention.

5. Conclusion

Research into DP training in psychotherapy is in its infancy, so this study complements this area of research. The aim of this study was to examine pre–post differences in FIS following DP training, as well as the participants' self‐perception of their performance. Within the limitations presented, participants showed statistically significant pre–post improvements in their interpersonal facilitation skills following the training, in a single‐group pre–post study in which raters were not masked to timepoint.

In addition to the importance of DP in the training of therapists, it is also important to consider the training of FIS skills, knowing that these are evidence‐based therapeutic skills. By combining these two variables, we assume that therapeutic results will tend to increase in a positive direction. With regard to the overestimation of skills made by therapists themselves, the importance of adjusting expectations in relation to one's own abilities is highlighted.

Although it is hoped that this study will contribute to an advance in deliberate practice, particularly in the field of psychotherapy, more studies will be needed to give continuity to this growing theme.

Funding

The authors have nothing to report.

Consent

All data protection, participant anonymity and informed consent procedures were ensured.

Appendix A. Facilitative Interpersonal Skills Rating Scale (FIS), Anderson & Patterson, 2013

Appendix B. Intracall Correlation Coefficient (ICC), for an Absolute Agreement Between Three Evaluators (K = 3)

Appendix C. T‐Student Test for Paired Samples to Compare the FIS (Hetero‐Evaluation) Dimensions Between Before and After DP Training

Appendix D. T‐Student Test for Paired Samples to Compare the FIS (Hetero‐Evaluation) Dimensions With the FIS‐SR (Self‐Assessment) Dimensions, Before DP Training

Appendix E. T‐Student Test for Paired Samples to Compare the FIS (Hetero‐Evaluation) Dimensions With the FIS‐SR (Self‐Assessment) Dimensions, After DP Training

Appendix F. T‐Student Test for Paired Samples to Compare the FIS‐SR (Self‐Assessment) Dimensions Between Before and After DP Training

Data Availability Statement

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions. The data used in this study are currently not publicly available but will be made accessible upon future publication.