Mentalizing in the Presence of Another: Measuring Reflective Functioning and Attachment in the Therapy Process

Abstract

In this paper, we test the reliability and validity of two novel ways of assessing mentalizing in the therapy context: the Reflective Functioning scale (RF) applied to code psychotherapy transcripts (In-session RF), and the Exploring scale of the Patient Attachment Coding System (PACS), which measures in-session autonomy and is linked with secure attachment in psychotherapy. Before treatment, one hundred and sixty patients in different types of psychotherapy and from three different countries were administered the Adult Attachment Interview (AAI), which was rated with the RF scale. One early psychotherapy session for each patient was independently rated with the In-session RF scale and with the PACS Exploring scale. Both scales were found to be reliable and to have concurrent validity with the RF scale rated on the AAI, with the PACS Exploring scale found to be a better predictor of RF on the AAI. These results suggest that the PACS Exploring scale might be a practical method for assessing RF in psychotherapy research and a way for researchers and clinicians to track patients’ RF on an ongoing basis. These results also provide information regarding the ways in which differences in RF manifest during psychotherapy sessions.

It is a common observation among therapists of different orientations that psychotherapy patients vary in their capacity to understand themselves and others. Some patients are more able to reflect on their own and others’ mental states; they can contemplate intentions that may be implicit in someone’s behavior and understand how mental states change and develop over time. Other patients may struggle to move beyond a concrete understanding of behavior in relationships, or may even fail to distinguish their own mental states from other people’s. Patients’ ability to reflect on mental states is integral to the tasks of many therapeutic approaches as well as to psychological health. Therefore, such an ability is likely to facilitate therapeutic work and make change easier to achieve. For this reason, researchers have proposed a host of constructs to describe and measure the capacity to reflect on mental states and have linked such constructs to therapy outcomes, e.g., psychological mindedness (Bohart, & Wade, 2013), alexithymia (Ogrodniczuk, Piper, & Joyce, 2011), experiencing (Yeryomenko, 2012), and metacognition (Dimaggio, & Lysaker, 2015). More recently, the concept of mentalizing has been proposed as an umbrella term for all of these concepts (Fonagy, & Bateman, 2016). In parallel, treatments have been explicitly developed to help patients recover or strengthen their reflective capacities: mentalizing-based therapy (MBT, Bateman, & Fonagy, 2016), metacognitive therapy (MCT, Wells, 1997), meta-cognitive interpersonal therapy (Dimaggio, Montano, Popolo, & Salvatore, 2015), and mindfulness-based cognitive therapy (Segal, Williams, & Teasdale, 2012), just to name a few.

This paper focuses on the concept of mentalizing (Fonagy, Steele, Steele, Moran, & Higgitt, 1991) and how to measure it in psychotherapy. Mentalizing is defined as the capacity to consider the behavior of oneself and others as a product of underlying mental states (Fonagy, Target, Steele, & Steele, 1998). In the past twenty-five years, a successful research program led by Fonagy and his colleagues has used the concept of mentalizing in developmental, personality, and clinical research. However, despite the growing number of clinically relevant studies in the area of mentalizing and therapy processes (see Katznelson, 2014 for a review), more efficient ways of measuring mentalizing in clinical settings are sorely needed. Today, psychotherapy researchers assess mentalizing primarily by coding the Reflective Functioning scale (RF, Fonagy, et al., 1998) on the Adult Attachment Interview, an interview about childhood attachment experiences (AAI, George, Kaplan, & Main, 1996), or other semi-structured interviews (e.g., Parent Development Interview, Slade, Aber, Bresgi, Berger, & Kaplan, 2004). Reliance on such interviews limits the application of the RF scale to the few settings where administering, transcribing, and coding a long interview is feasible and affordable. Furthermore, using interviews may be a problem when attempting to track changes in mentalizing over time, as interviewees may grow too accustomed to protocols that are administered more than once or twice (Katznelson, 2015).

In order to advance the study of mentalizing in psychotherapy, in this paper we test two new assessment methods: an in-session RF coding system based on the RF scale developed for the AAI (In-session RF, Talia, Steele, & Taubner, 2015), and the Exploring scale from the Patient Attachment Coding System (PACS, Talia & Miller-Bottome, 2014). The PACS is a measure of attachment security based on session transcripts that has recently been validated with the AAI (Slade, 2016; Talia, et al., 2014; Talia, Miller-Bottome, & Daniel, 2017). The PACS Exploring scale measures a particular component of in-session security: the capacity to communicate about mental states while remaining open to the therapist’s feedback. In the current study, we evaluate both the In-session RF scale and the Exploring scale of the PACS by testing whether and to what extent each predicts patients’ pre-treatment RF score on the AAI.

Beyond testing two new ways of assessing RF, our work in this paper can help researchers and clinicians understand the ways in which RF influences the therapy process. Although RF has been shown to predict both the therapeutic alliance and treatment outcome, the ways in which patients’ RF affects the moment-to-moment interaction with the therapist are much less well known. Even the evidence showing that patients’ level of RF during the AAI is related to their readiness to mentalize during a psychotherapy session is still sparse. By investigating which observable in-session markers are associated with RF, we can help move the field closer to considering RF as an activity that influences the process of psychotherapy, rather than just as a patient factor.

In the following paragraphs, we first discuss previous attempts by researchers to measure RF in psychotherapy. Next, we provide a rationale for predicting patients’ RF score on the AAI by assessing in-session attachment with the PACS Exploring scale. We then present the methodology and results of our study and discuss its theoretical and clinical implications.

Assessing RF in the therapy context

Today there exists a body of literature demonstrating the clinical relevance of mentalizing as a construct and its operationalization as RF (see e.g., Katznelson, 2014; Taubner, Hörz, Fischer-Kern, Doering, Buchheim, & Zimmermann, 2013). Studies have linked low RF to a number of pathological outcomes, including eating disorders, potential for violence, and personality disorders (see, e.g., Fonagy, et al. 1996; Fischer-Kern et al., 2010; Gullestad, Johansen, Høgland, Karterud, and Wilberg, 2013; Maxwell, et al., 2017; Taubner, Zimmerman, Ramberg, & Schröder, 2016). Researchers have also investigated the impact of patients’ and therapists’ RF on the process and outcome of psychotherapy. Some studies have shown that RF may be a moderator of therapy outcome (Antonsen, Johansen, Rø, Kvarstein, & Wilberg, 2015; Gullestad, et al., 2013). Other studies have found evidence that RF is a predictor of both alliance and outcome (Ekeblad, Falkenstrom, & Holmqvist, 2016; Müller, Overbeck, & Grabhorn, 2006; Taubner, Kessler, Buchheim, Kächele, & Staun, 2011). Finally, many researchers maintain that an improvement in patient’s capacity to mentalize may indicate an improvement in general psychological functioning, and that facilitating mentalizing might be in itself a mechanism of therapeutic change (Fischer-Kern, et al., 2015; Levy, et al., 2006; Rudden, Milrod, Target, Ackerman, & Graf, 2006).

In order to expand the promising but still limited research in this field, several investigators have proposed alternative methodologies for rating RF. Some have attempted to use interviews that are shorter than the AAI (Rudden, Milrod, & Target, 2005; Rutimann & Meehan, 2012), or have coded RF through a computerized system that can be applied to interview transcripts (Fertuck, Mergenthaler, Target, Levy, Clarkin, 2012). Although these methods are more time efficient, they still rely on administering an interview – the limitations of which we discussed earlierⁱ. More recently, Fonagy and his colleagues have begun to assess RF with self-report questionnaires (RFQ, Fonagy, et al. 2016). The RFQ offers a more practical alternative to interviews. However, its concurrent validity with the RF scale rated on the AAI has not yet been studied, and its association with other observer-based measures of attachment such as the Strange Situation appears to be trivial or non-significant.

Other investigators have attempted to track mentalizing as it occurs during therapy. Meehan, Levy, Reynoso, Hill, & Clarkin (2009) found a moderate association between patients’ RF rated on the AAI and a questionnaire about patients’ mentalizing completed by clinicians who had treated the patients for an entire year (N = 32, r = .54, p < .01). More recently, Möller and her colleagues (Möller, Karlgren, Sandell, Falkenström, & Philips, 2016) have found a medium-sized correlation (N = 15; r = .64, p < .01) between patients’ RF rated statement by statement during a single audio-recorded session and RF independently scored on an abbreviated version of the AAI. While these studies introduced promising new assessment methodologies, their small and uniform samples (all patients had been diagnosed with BPD) and the treatment modality (MBT, which actively elicits mentalizing, in Möller et al’s study) limit the generalizability of their findings.

At least three studies have used the RF scale to rate verbatim therapy transcripts, a method that may offer a more reliable way of assessing RF in psychotherapy. In Karlsson’s and Kermott’s study (2006), the raters used the RF manual to rate every patient’s narrative about interpersonal interactions in a session and then assigned a global RF rating. A limitation of this particular approach is that the number and length of interpersonal narratives told by patients may vary greatly, so that the final RF scores assigned may be dependent on several variables beyond patients’ individual RF level (e.g., therapeutic modality, phase of therapy, etc.). A single-case study by Josephs and colleagues (Josephs, Anderson, Bernard, Fatzer, & Streich, 2004) introduced a more standardized way of assessing RF from session transcripts. In this method, the rater scores blocks of 150 words with the RF scale, and then gives a global score to the whole session by taking into account the ratings given to the different passages. Hörz-Sagstetter, Mertens, Isphording, Buchheim, and Taubner (2015) used Josephs et al.’s method to code the sessions of two patients in psychoanalysis who had been interviewed with the AAI. The authors found a good correspondence between session-based RF ratings obtained across multiple sessions and independent RF ratings based on the AAI. In the current study, we assessed reliability and concurrent validity against the RF scale of a modified version of Josephs et al.’s rating method, the In-session RF scale (Talia, et al., 2015). The In-session RF scale will be further described in the Methods section.

The Exploring scale of the PACS as a predictor of RF

In this study, we also tested whether patients’ ratings on the Exploring scale of the PACS predict patients’ RF score on the AAI. The PACS (Talia, & Miller-Bottome, 2014) is used to assess patient’s attachment by tracking characteristic communication patterns (secure, avoidant, and resistant) employed by patients to express their own mental states (e.g., emotions, wants, needs), or to reflect on other people’s mental states. In a recent large scale validation study involving patients in five different types of treatment and in three different countries, the PACS coding of one single transcribed therapy session has demonstrated good to excellent reliability (all of the five main scales had ICC > .75) and high concurrent validity with patients’ pre-treatment AAI classifications (N = 156; .87, k = .82; for more details on PACS psychometric qualities, see Talia, et al. 2017).

The PACS considers the activity of articulating a mental state as a distinctively interpersonal process. Differently from the RF scale, which assesses qualities of narratives about mental states, the PACS examines patients’ communication about mental states in a more interpersonal perspective – i.e. how such communications influence and are influenced by the interaction with the listener. As soon as we share our thinking about a mental state or express our current internal experience, our listener is implicitly invited (whether or not this invitation is accepted) to validate, correct, or elaborate on our disclosures (e.g., when we share sad feelings, we expect a supportive response; when we describe a positive experience, we elicit an expression of rejoice). Thus, each speech act that conveys or alludes to a mental state may be seen as doing two things: articulating that mental state, and opening (or not) a space for the listener’s re-elaboration. With their language, patients with different attachment classifications either carry out both actions (secure), or they may restrict their role (avoidant) or the listener’s (resistant) in this process (Talia, et a., 2017).

The PACS tracks the frequency and intensity of speech acts that characterize patients with different attachment classifications in psychotherapy. The PACS has five scales, each of which rates its own distinct set of speech acts. The PACS Proximity seeking and the PACS Contact Maintaining scales rate, respectively, speech acts in which the patient conveys present distressful mental states (e.g., sharing distressful feelings in the here and now) or reveals the positive impact that the therapy or the therapist has been having on him or her (e.g. thanking the therapist, affirming a therapist intervention). Both scales reflect a patient’s ability to articulate mental states and to invite a supportive or mirroring response from the therapist. The PACS Avoidance scale rates patients’ reluctance to openly discuss mental states and their tendency to leave too much space to the therapist to guess and probe. For example, the patient may dismiss the importance of a feeling previously articulated, or may fail to provide a narrative of a distressful experience. The PACS Resistance scale rates communications in which patients articulate their views on their own or others’ mental states, but leave little space for the therapist to participate. For example, patients may enlist the therapist’s approval of their interpersonal judgments, or they may express their views at great length and in a vague manner, so that little room is left to the therapist to add anything.

The PACS Exploring scale, the focus of the current study, rates speech acts in which patients openly share their views on their own and others’ mental states in a way that leaves the therapists free to respond in various ways. For example, the PACS Exploring scale rates communications in which patients express their independent intentions, share their positive experiences, or make tentative conjectures about others’ mental states, all of which leave room for a variety of re-elaborations or comments while eliciting the therapist’s acknowledgement. This is best understood by comparing and contrasting this scale with the other PACS scales. The Avoidance and the Resistance scales restrict the role of either the patient or the therapist in articulating mental states, respectively; and the Proximity seeking and Contact maintaining tend to prompt for a narrow set of supportive or mirroring responses from the therapist. On the other hand, communications rated on the PACS Exploring scale allow for patients and therapists to engage in a fully mutual dialogue. (The PACS Exploring scale will be further described in the methods section and in Table 3).

Table 3.

Subscales and markers of the PACS Exploring Scale (with examples)

Subscales and markers	Examples
Self-asserting
(1) Expresses independent will (2) Describes the action he or she will take to cope with a problem (3) Proposes tasks/goals for therapy (4) Expresses misgivings or concerns regarding therapeutic tasks	I don’t want to be involved in this situation (1). I have too much on my plate. I want to voice that to her (1). You know what, I will call her tomorrow – yes, that’s what I’m gonna do (2). You know, we actually haven’t been talking about this issue in here as much as I’d like to (4). Do you think we could talk through together how I might bring this up to her? (3)
Affective Sharing
(5) Discloses a vivid narrative of a self- or other-defining experience (6) Describes a past instance of being cared for by another person and the emotional effect of the experience (7) Praises a significant other’s positive characteristics or loving actions including their emotional effects (8) Describes positive characteristics of a specific relationship	He’s just, I dunno I just feel really listened to when I’m with him, like he really cares about what I have to say (7). Like, our relationship is just – something I’m really proud of. We care about each other but we are not on top of each other, like we have our own separate lives, but we both prioritize each other, and that makes it stronger (8). One time, I was really nervous about this job interview I had, and I was talking to him about it and he just offered to take off work and come with me, without me even asking. I remember that moment was like – wow. That just really meant a lot to me, that he would take time like that, just for me (5, 6).
Autonomous Reflection:
(9) Reflects in the moment on others’ or the patient’s own internal states, sharing a new or alternative perspective beyond what is apparent and in such a way that invites collaborative reflection from the therapist	I still feel anger about my father leaving us. He left his kids - us - when we were so young and, you know, without explaining or telling us what was going on (sighs). And yet when I think about it at some level I think that he must have felt a lot of grief about it, after the fact. I don’t think he let it show but I dunno (9)… I think that - maybe that’s part of why I’ve still kept in touch with him (9).

We hypothesize that the PACS Exploring scale will predict patients’ pre-treatment RF rating on the AAI; we based this hypothesis on two considerations. First, research on mentalizing following in the footsteps of Fonagy and colleagues’ seminal work from 1991 (Fonagy, et al., 1991) has emphasized that the development and maintenance of mentalizing abilities are closely tied to secure attachment (Fonagy, & Target, 2005), with empirical evidence demonstrating that AAI attachment security is associated with RF (e.g., Jessee, et al., 2016). It may thus be possible to predict patient’s RF rating on the AAI with the PACS Exploring scale, which is the only PACS scale to be exclusively associated with attachment security (according to the findings of Talia, et al., 2017).

Secondly, we hypothesize that the capacity that is rated by the PACS Exploring scale is also involved when speakers demonstrate RF during the AAI. At first glance, the RF scale and the Exploring scale seem to tap two different constructs. However, most markers of RF (for example the act of emphasizing the opaqueness of mental states, e.g, in saying “I think that she’s angry, but I’m not sure”) implicitly show that the speaker is not afraid of challenge and invites re-elaboration from the listener (e.g., “Yeah, it sounds like she’s angry, I wonder what’s underneath that?”). In so doing, a speaker seems to create a space for the listener to confirm, add to, or even challenge the speaker’s understanding (even though in the context of an interview the listener usually refrains from doing so), thereby conveying a sense of independence and agency. These interpersonal aspects, which are rarely mentioned in the theoretical literature on the RF scale, are precisely what the PACS Exploring scale rates. They also happen to be in essence features of secure attachmentⁱⁱ.

Methods

Participants

This study included a combined sample of 160 outpatients treated in five different treatment modalities and from three different countriesⁱⁱⁱ. Sixty-eight patients came from a Danish randomized controlled trial study with patients with Bulimia Nervosa that took place in Denmark (Poulsen et al., 2014), where patients received either two years of psychoanalytic psychotherapy (PPT) or twenty sessions of cognitive-behavioral therapy-enhanced (CBT-E); 72 patients came from a treatment facility in New York, where they received up to 30 sessions of Brief Relational Therapy (BRT; Safran & Muran, 2000) or CBT (Beck, 2011); 20 patients came from a counseling facility in Italy, where they received Supportive Psychotherapy (SPT) with varying treatment lengths, but up to four years. Each therapy was conducted in the language native to the country in which it took place, and all patients received individual psychotherapy on average once a week. 90.4% of the patients were Caucasian, while 7.6% were African–American and 2% were of other origin. Patients’ ages ranged from 19 to 65 years. More demographic information on the patients is presented for each subsample in Table 1.

Table 1.

Sample characteristics

	PPT	BRT	CBT-E	CBT	SPT	Total
Country	Denmark	USA	Denmark	USA	Italy
Patients, n (%)	33 (20.6)	40 (25.0)	35 (21.9)	32 (20.0)	20 (12.5)	160 (100)
Age, M (SD)	26.2 (4.4)	38.6 (13.7)	26.4 (5.2)	39.1 (11.3)	23.8 (3.7)	32.4 (20)
Women (%)	33 (100)	16 (40.0)	35 (97.2)	17 (53.1)	16 (80.0)	116 (72.5)
Mental disorder, n (%)	33 (100)	25 (73.5)	35 (100)	21 (72.4)	“	114 (71.3)
PD, n (%)	9 (27.3)	16 (51.6)	13 (37.1)	18 (78.3)	“	57 (35.6)
RF-AAI, M (SD)	3.9 (1.4)	3.4 (1.7)	4.1 (1.8)	2.6 (1.5)	2.6 (1.3)	3.4 (1.7)
In-session RF, M (SD)	3.7 (1.1)	3.6 (1.3)	3.8 (1.1)	2.8 (1.2)	3.3 (1.0)	3.5 (1.3)
Exploring, M (SD)	2.8 (1.1)	3.0 (1.7)	3.0 (1.3)	2.3 (1.2)	2.5 (1.5)	2.7 (1.4)

Mental disorder: mental disorder other than personality disorder (diagnosis according to DMS-IV-TR). PD: personality disorder (diagnosis according to DMS-IV-TR).

The treatment facilities were chosen based on the availability of recorded therapy sessions and AAIs administered before the beginning of treatment. All materials had been previously collected for other studies. The cases were chosen at each treatment facility by selecting consecutively admitted cases with available transcribed data, from the most recent ones to the older ones.

One hundred therapists were involved in this study. The eighty-eight therapists from the New York and Padua subsamples were trainees in their second to fourth year of graduate clinical training. In these subsamples, each therapist was paired up with a different patient. The twelve therapists from the Copenhagen subsample had more years of clinical experience (M= 14.0 years, SD= 5.35) and saw multiple patients in this study (M = 5.7; SD = 3.2). In the Padua subsample, all therapists were female; in the New York and in the Copenhagen subsamples, 80.5% and 75% of the therapists were female, respectively.

Measures

The RF scale

In our study, patients’ overall capacity to mentalize was measured with the RF scale coded on the AAI (Fonagy et al., 1998; George, et al., 1996). The AAI is an interview that asks individuals to describe their childhood relationship with their parents, along with a set of standardized probes. The interview also requires that participants reflect on their parents’ caregiving and consider how childhood experiences with their parents may have influenced their personality. In this context, the RF scale assesses whether participants understand attachment-related experiences in terms of mental states.

The RF scale rates the degree to which the speaker is able to understand their own and other people’s behavior as a function of underlying mental states. The coding of RF is based on scoring the following dimensions: (a) the awareness of the speaker about the nature of mental states; (b) the explicit effort made by the speaker to tease out mental states underlying behavior; (c) the speaker’s recognition of the developmental nature of mental states; and (d) the recognition of the probable mental states of the interviewer. The answers to the AAI questions are coded on an eleven-point scale, from −1 (anti-reflective), to 9 (exceptionally reflective).

The questions in the AAI are divided in two types: those that explicitly probe for RF (demand questions e.g., “Why do you think your parents behaved as they did?”), and those that allow mentalizing, but do not require it (permit questions, e.g., “Can you think of five adjectives that describe your relationship with your mother when you were little?”). When coding the RF scale on the AAI, the rater can assign a low rating (i.e. < 4) only to answers that do not demonstrate RF after a demand question, and not to answers that follow permit questions. After having assigned an individual score to each question in the interview, the rater assigns a global RF score by weighing and aggregating the ratings of the individual questions (Fonagy, et al. 1998).

The In-session RF scale

During a session, the discourse flows freely, and the rater who intends to score RF in-session cannot rely on rating responses to predetermined questions. The In-session RF scale (Talia, et al., 2015; adapted from Fonagy, et al. 1998) is a version of the RF scale with minimal adaptations to assess mentalizing in the therapy context. Similarly to Josephs et al.’s method (2004), with the In-session RF scale the rater divides the entire psychotherapy transcript in 150-words segments (instead of giving a rating to each answer as in the AAI), and then codes each segment from −1 to 9. In contrast to Josephs et al.’s method, with the In-session RF scale the rater codes mentalizing whenever it occurs, not only in attachment-related contexts. Howard Steele and Svenja Taubner (the two main instructors worldwide in scoring the RF scale), together with Talia, adapted the RF manual to code therapy transcripts. Steele and Taubner then rated in consensus sixty 150-words segments of sessions of low, medium, and high RF. The segments were included in the In-session RF manual as rating examples for the coders of this study.

When rating with the In-session RF scale, no difference is made between demand and permit ratings - any segment that does not contain any mention to mental states, regardless of whether there has been a previous prompt to mentalize by the therapist, is given a rating of 1. Similarly to rating RF on the AAI, the global score is obtained by individually weighing and aggregating the ratings of the individual 150-words blocks according to an algorithm provided in the manual. Table 2 presents example of session segments rated with the In-session RF scale.

Table 2.

Examples of segments rated with In-session RF and the PACS Exploring scale

Description of the rating	Example segments that will receive a corresponding rating
1: Mental states are not mentioned at all throughout the segment	P: (chuckles) - - um - - well so, if I’m not wrong you asked me last week about um my relationship with women, and that it was like I was in a rut? I guess - something happened over the summer that made me go out more and be more proactive in general. That’s when I was in California, at my parents’ summerhouse, in July, late July. So what happened is that my high school girlfriend, Ella, she said…she said to me that I might be the one, which was odd because she and I don’t have the best history with each other you know (chuckles)? I mean at least if we lived in the same city something could happen, maybe it could! But she lives down there and I live here in the East.
3; Mental states are mentioned but none of the ‘qualitative markers’ of RF are present	P: The weekend was good, we battened down the hatches and got a lot done for the wedding party which was very relieving, mhm, we have been quite happy overall. We got the invitations all settled on. Eventually what I kinda realized is that sometimes I just need to make a decision and just make it happen. I can’t leave it open-ended like, “well what do you think?’ cuz she’ll just go, “well I don’t now Billy, let me think about this let me look at that” and then nothing ever happens. And sometimes I think I just need to take the lead on some of these things. At least when it seems to me that she’s kind of fine about it, like she won’t get mad because it’s something quite small or trivial.
5: One ‘qualitative marker’ of RF in a segment in which P reflects on mental states	P: I dunno, my father’s birthday is coming up and I just don’t want to like give him a call or anything of that sort [self-asserting, 1]. With my mom I’m able to tune it out, with my dad he’s got this grip on me, this hold on me. It really does bother me when I am forced to have some form of communication with him…. And I think like these thoughts, this negativity, I think it primarily stems from what I feel are his expectations of me and where I ought to be in life [autonomous reflection 9] (B) Stuff he was always ranting about at the table when we were little and he still likes to moan about.
7: Three ‘qualitative markers’ of RF in a segment in which P reflects on mental states	P: My parents, of course, criticized me for some things, but it was more like maybe I’m lazy, or that I’m careless, or whatever. But they saw me as very successful with girls, very popular, and they praised me for that - which in many ways gave me a true sense of security and of being appreciated and they built this kind of image of me that, as a child you want to keep as long as you can [affective sharing 7] (C). And when I came out - I realized it was so hard for my mom, something in her image of me broke (B). It totally collapsed. And for my dad it wasn’t, it was bad, but not as bad because he didn’t have that image of me. (B)
9: Highly reflective segment, surprising and elaborate in a way that is not fully captured by a score of ‘7’ or ‘8’	P: I mean, I remember…I never told anybody that I had been bullied. I never shared it with my parents and that’s where I think I put my parent in a position of, they couldn’t help me they couldn’t support me [autonomous reflection, 9] (B), because I didn’t tell them anything. Looking back on it I think I was too proud…but I think I was mad at them – for not knowing...[autonomous reflection 9] (C) Not… not that I was really aware of it (A). I was too shy but…I hope this makes sense to you? (D) T: Yeah, almost like, wouldn’t it have been great if they had known anyway? P: It just seemed like the worst thing to let them know, because then I wouldn’t be sure which side they’re going to take. I think I didn’t want them to see me how the kids saw me [autonomous reflection 9] (B)

P: Patient; T: Therapist. The passages where markers of RF are present are underlined, and the related qualitative marker are presented in brackets thereafter: A awareness of nature of mental states B effort to tease out underlying mental states C recognizing developmental aspects of mental states D mental states in relation to therapist. PACS Exploring markers are added in bold and explained in brackets for comparison (see Table 3 for information about the PACS Exploring markers).

The PACS Exploring scale

The PACS Exploring scale is the only PACS scale to be exclusively associated with secure attachment (Talia, et al. 2017). It was developed through identifying in-session characteristics or markers associated with patients’ pre-treatment attachment security rated with the AAI. The PACS Exploring scale was then refined by coding a set of sessions of eight patients and referring to the patients’ RF scores obtained independently on the AAI^iv.

The PACS Exploring scale is rated based on the frequency and intensity with which nine discourse markers appear in a transcript. The markers are grouped under three subscales: a subscale that groups markers related to conveying intentions that emerge in the present (‘Self-asserting’, which may also be viewed as a measure of patients’ expressed agency, see Bohart, & Wade, 2013), a subscale that groups markers related to conveying presently-felt positive experience (‘Affective sharing’), and a subscale that groups markers related to assuming in the present alternate perspectives on the internal experience of oneself or other people, beyond what readily apparent or observable (‘Autonomous reflection’). Table 3 presents examples of the PACS Exploring scale, its three subscales, and its markers.

When rating with the Exploring scale, the transcript is rated as a whole, without segmenting the text in advance. By referring to the verbatim therapy transcript, the rater identifies the presence and intensity (low, average, or high) of nine markers described in the coding manual. Markers are coded as they occur; they can be assigned to a single utterance, or to a whole speech turn. The rater gives a score from 1 to 7 in .5 increments to ‘Self-asserting’, ‘Affective sharing’, and ‘Autonomous reflection’ based on the frequency and intensity with which the markers belonging to each subscale appear in the transcript, in a continuum where “1” indicates the absence of the related markers, and “7” represents a pervasive presence. The final rating of the Exploring scale for each session is then established based on (i.e. is equal with) the rating of the highest rated subscale of the Exploring scale (Self-asserting, Affective sharing, or Autonomous reflection), and can be incremented up to 1.5 points to take into account the ratings of the other scales, according to a simple algorithm.

It is important to underscore that, although the ‘Autonomous reflection’ subscale and the RF scale may appear to be as conceptually similar, they differ in at least two significant ways. First, while RF codes the degree to which mental states are referred to, the ‘Autonomous reflection’ subscale captures the patient’s attempt to open a reciprocal conversation with the therapist about mental states. In order to code a passage as an occurrence of Autonomous reflection, the speaker must convey that the reflection is occurring in the here and now (and thus can be amended or corrected by the listener), through the use of what the authors of the scale term ‘reflective tags’ (e.g., I think; but maybe; it’s almost as if, and so on). Any indication by the speaker that the reflection has being made in the past (e.g. yesterday I thought) immediately disqualifies the passage from being coded. Similarly, evidence that the speaker is too certain of the mental states discussed (e.g., I know for sure that she is feeling humiliated) disqualifies the passage from being rated.

Further, a passage is scored on the Autonomous reflection subscale only if the patient offers his or her personal reflection as an additional perspective to otherwise established facts (for example, the patient proposes a subjective interpretation of the mental states underlying the behavior of a significant other). In doing this, the patient presents her perspective as ‘just one’ perspective and implicitly offers to the therapist the context with which to understand and to potentially challenge the patient’s guesses.

Procedure

All participants were interviewed with the AAI before treatment by trained research assistants. In New York and Copenhagen, trained research assistants also administered SCID-II and collected demographic data; in New York, SCID-I was administered as well. The AAI interviews were scored for RF by six reliable coders^v, with each of them coding a different set of interviews (each coder rated between 20 and 30 interviews). 20% of the interviews (N = 32) were coded by two raters chosen at random to calculate intra-class correlation coefficient, which was good (ICC =.84). Four patients were excluded from analyses involving their AAIs because their interviews could not be transcribed due to insufficient audio quality.

The third psychotherapy session for each patient was transcribed following similar guidelines to those indicated by Main for the AAI (Main et al. 2003), with the inclusion of laughter and crying. For CBT-E, we chose session six, because in that treatment modality (Poulsen, et al. 2014), this session occurs in the third week of treatment. When the targeted session was not available because of missing, inaudible, or incomplete recordings, we selected the nearest available session. Sessions included ranged from session 1 to 8, with the mean being session number 4.4 (SD = 1.6).

All raters using the In-session RF scale had previously received (with one exception) formal training in scoring RF on the AAI either from Howard Steele, or from Svenja Taubner and Tobias Nolte, and they were certified reliable RF coders^vi. They were given the In-session RF scale manual (Talia, et al. 2015) and received an additional three-hours training in coding with the In-session RF scale. To determine if they were reliable in the use of the In-session RF scale and to assess formal inter-rater reliability of the scale, all raters were asked to rate a set of twelve sessions randomly selected from our sample; a rater was considered reliable if he or she attained an ICC of .70 or more against Svenja Taubner; one rater did not achieve this mark and was then excluded from the following rating procedure. The two way absolute agreement single measure ICC was calculated on the ratings of these sessions made by the six remaining reliable raters and is reported in Table 4. The six raters rated all therapy sessions with the In-session RF scale (each rater scored between twenty and thirty sessions). All raters were blind to patients’ pre-treatment RF score, as well as to any other patients’ information.

Table 4.

Descriptives and correlation matrix

	Descriptives			ICC	Correlations
	M (SD)	Range	Skew (Kurt.)		1	2	3	4	5
RF-AAI	3.4 (1.7)	0–8	0.31 (−0.62)	.78*	.54*	.72*	.31*	.48*	.66*
1: In-ses RF	3.5 (1.3)	1–7	0.16 (−0.25)	.71*		.50*	.29*	.38*	.47*
2: Exploring	2.7 (1.4)	1–7	0.47 (−0.59)	.84*			.64*	.71*	.82*
3: SA	1.9 (1.1)	1–6	1.33 (1.56)	.73*				.43*	.38*
4: AS	1.8 (1.2)	1–6	1.50 (1.49)	.73*					.41*
5: AR	2.2 (1.2)	1–6	0.59 (−0.73)	.85*

Skew SE = 0.19; Kurtosis SE = 0.39. ICC: two ways, single measure intraclass correlation coefficient. SA: Self-asserting. AS: Affective sharing. AR: Autonomous reflection. The table displays observed ranges only (theoretical range always 1–7 for the PACS scales, and −1 to 9 for the RF-AAI scale and the In-session RF scale).

^*p < .001

Six raters used the PACS Exploring scale to code all the session transcripts of this sample, with each individual rater assessing between twenty-four and twenty-six sessions. Coders had been trained in the use of the Exploring scale for a minimum of twelve hours; only one of the six coders with the PACS Exploring scale had also received training in scoring with the RF scale. The coders were blind to any information on the patient (including patients’ RF score and patients’ In-session RF score). Since the Exploring scale was coded in the context of rating with the PACS, the raters were not blind to patients’ in-session attachment status. To obtain inter-rater reliability data, 50% of the sessions (N = 80) were double coded by a second independent rater chosen at random.

Data analytic plan

Since the treatment facilities involved in this study were treating different populations of patients, it is likely that there would be systematic differences in their patients’ mean RF levels (our dependent variable). Further, it was likely that the different treatment modalities involved in our study (i.e., psychodynamic, CBT, etc.) may have an effect on In-session RF and Exploring, because different forms of therapy may in theory encourage some in-session processes over others (e.g., psychodynamic therapy tends to encourage patients to reflect about their relationship with parents more than CBT-E). Since both effects may have an influence over the associations between In-session RF and Exploring with pre-treatment RF on the AAI, the ratings of all the variables in this study are expected to be correlated and non-independent within the same treatment facility, while the associations between predictors and our dependent variable might vary between treatment modalities. To account for this, in all of our analyses we used multilevel models, where the patients (level 1) were nested within treatment facilities (level 2), and we modeled an interaction parameter to account for the possibility that the dummy coded-treatment modalities had varying effects on the association between In-session RF, Exploring, and RF coded on the AAI. Because most of the therapists in this study (N= 88) only treated one patient, it was not possible to include therapists as an additional level in our model (i.e. the Hessian matrix of that model would be numerically singular and would make it difficult to differentiate such parameters).

We estimated random intercepts for treatment facilities and fixed effects for In-Session RF and Exploring. We then modeled the relationship of In-Session RF and the Exploring scale with RF rated on the AAI. Finally, we analyzed the subscale components of the Exploring scale as fixed effects in order to find out their individual contribution in predicting RF assessed on the AAI.

Restricted maximum likelihood was used as estimator in all models. Confidence intervals and significance values for fixed effects were computed using Kenward-Roger approximation (Kenward, Roger 1997). Assuming normality for the errors, parametric bootstrap simulating the response under the null model, then comparing the likelihood ratio of the alternate model with the null model, both predicting the simulated response over 1000 iterations was used to compute significance values for random effects (Faraway, 2006). We then used Kenward-Roger approximation to compare the models.

No multivariate outliers were found, as the largest within-cell Mahalanobis’ distance (9.32) was smaller than the χ² critical value of 16.27 (3, 156, p < .001). Multivariate collinearity was tested through several multiple regressions where we tested each variable as predictor for the other ones, and no signs of multivariate collinearity were found (R² < .75). Plotting the fitted values against the residual ones did not indicate nonconstant error variance for any of the models. In the same vein, visual inspections of the QQ plots did not show meaningful divergence from normality for any of the models. Due to the nature of the study (which is based on observer-based measurements) there were no cases with missing data.

Results

Table 4 shows mean values, standard deviation, observed range, skew and kurtosis of all the scales used in this study, as well as intra-class correlation coefficients for each scale; the table also displays correlations between the scales. Gender and age were not significantly related to any one of the study variables (with ps from .09 to .70), so these variables were excluded from the following analyses. Of note, RF scored on the AAI had a moderate correlation with the AAI Coherence of mind scale, the golden standard measure of adult attachment security (N = 156, r = .46, p < .001). AAI Coherence of mind was coded independently by reliable raters (see Daniel, et al., 2017), and this finding is consistent with recently published empirical evidence (Jessee, et al. 2016).

As shown in Table 4, patients’ RF scores obtained on the AAI were strongly associated with patients’ scores on the Exploring scale, and significant (albeit weaker) associations were found between patients’ RF assessed on the AAI and all of the Exploring subscales. Patients’ RF scores obtained on the AAI were also moderately associated with the RF scores independently obtained from patient’s therapy sessions.

In order to test whether patients’ in-session RF would add to the explained variance in a model where the Exploring scale predicts patients’ RF on the AAI, we ran a two-step multilevel model where we added In-session RF as explanatory variable in the second step. The model with In-session RF added fit the data significantly better than the first model, although it only explained an additional 3.4% of the total variance. Table 5 reports parameter estimates for these analyses. None of the interaction effects with treatment modality attained significance, indicating that there were no significant differences between treatment modalities in regard to the relationship between In-session RF and Exploring with regards to RF-AAI; thus, we did not include interaction effects in subsequent analyses.

Table 5.

Multilevel Analyses of Exploring and In-session RF as Predictors of RF (AAI)

	Step 1:			Step 2:
	B	CI	p	B	CI	p
Fixed Parts
(Intercept)	1.01	0.26 – 1.76	.097	0.34	-0.47 – 1.15	.464
Exploring	0.82	0.71 – 0.94	.002	0.70	0.57 – 0.83	<.001
In-session RF				0.28	0.13 – 0.43	.027
Random Parts
σ2	1.137			1.051
τ00 (treatment facility)	0.331		<.001	0.311		<.001
ICC (treatment acility)	0.225			0.228
R2	.601/.601			.634/.634
Model Comparison
F				13.592
p				<.001

B: Regression coefficient; CI: Confidence interval; p: p value; σ²: Residual variance; τ₀₀: Random part variance; ICC: Intra-class-correlation coefficient; R²: R squared; F: F-statistic

We then compared both the regression coefficients of Exploring and Autonomous Reflection independently predicting the RF scale on the AAI, with In-session RF predicting the same. This analysis yielded significant results for both comparisons (F = 11.32(1), p <.001; F = 5.86 (1), p = .017), which indicates that the association between Exploring and RF and between Autonomous reflection and RF was significantly greater than the association between In-session RF and RF on the AAI in their respective models.

Finally, we tested the three subscales of Exploring in the same stepwise fashion. The first model (Step 1) encompassed RF scale on the AAI as dependent variable, predicted by the Autonomous Reflection subscale score and treatment facilities as random intercept. The second model (Step 2), adding the Affective sharing subscale as explanatory variable, fit the data significantly better than the first model. The third model (Step 3), adding the Self-asserting subscale, did not fit the data significantly better than the more parsimonious second model. Table 6 reports parameter estimates for these analyses.

Table 6.

Multilevel analyses of the PACS Exploring subscales as Predictors of RF (AAI)

	Step 1:			Step 2:			Step 3:
	B	CI	p	B	CI	p	B	CI	p
Fixed Parts
(Intercept)	1.36	0.61–2.10	.050	0.96	0.15–1.77	.123	0.87	0.01–1.72	.151
Autonomous Reflection	0.87	0.72–1.02	.003	0.71	0.56–0.87	.006	0.69	0.53–0.85	.005
Affective Sharing				0.39	0.23–0.56	.027	0.36	0.19–0.53	.032
Self-asserting							0.11	−0.08–0.30	.350
Random Parts
σ2	1.376			1.203			1.199
τ00, treatment facility	0.312 <.001			0.393 <.001			0.425 <.001
ICC treatment facility	0.185			0.246			0.262
R2	.517/.517			.581/.581			.585/.585
Model Comparison
F				22.540			1.261
p				<.001			.263

B: Regression coefficient; CI: Confidence interval; p: p value; σ²: Residual variance; τ₀₀: Random part variance; ICC: Intra-class-correlation coefficient; R²: R squared; F: F-statistic

Discussion

This is the first study to test rigorously the construct validity of a session-based method for assessing RF in a large sample of patients. Our results present the PACS Exploring scale and - to a lesser extent - the In-session RF scale as practical observer-based methods for reliably assessing patients’ RF. These findings are particularly robust as they were obtained from a large sample combining participants from three countries, three different treatment facilities, and in different therapeutic modalities. Our results also provide preliminary evidence that RF can be coded regardless of there being a discussion of attachment-related topics.

The findings of this study provide valuable information regarding how higher and lower levels of RF may impact patients’ interpersonal behavior and engagement in the process of psychotherapy. As we predicted, patients with higher RF on the AAI showed higher reflective functioning in-session and a greater capacity to reflect autonomously (as coded on the Autonomous Reflection subscale). We also discovered that patients with higher RF on the AAI are more likely to disclose and elaborate upon their positive experiences and relationships in-session (as coded on the Affective sharing subscale) and to convey their intentions and goals in the here and now (as coded on the Self-asserting subscale). All three of these features may help explain why patients with higher RF may engage more meaningfully in the process of psychotherapy, and why they seem to build better alliances with their therapists (Taubner, et al. 2011). Since such differences could be discerned from the beginning of treatment and predicted from pre-treatment AAI, we also know that they are relatively independent of the therapist and the therapist’s activity.

Our study has three main limitations. First, no test-retest reliability data for any one of the scales was obtained. However, the agreement between the Exploring scale and the RF scale scored on the AAI, which was administered on average one month prior, suggests that their association may be stable enough so that one can generalize the ratings of one session to other contiguous sessions. Second, all sessions coded are from the early phase treatment (session 2 to 8); this is a limitation because the three main measures involved in this study (RF, In-session RF, and PACS Exploring) may change at different rates. Finally, it is possible that the lower-than-expected association between In-session RF and RF measured on the AAI was due to the way in which In-session RF was measured. Clearly, the traditional method for measuring RF on the AAI (which makes a distinction between demand and permit questions, and only considers attachment-related narratives) cannot be easily applied to sessions in a standardized fashion. Sessions vary in the extent to which attachment-related topics are addressed, and therapists vary in the extent to which they probe mentalizing with demand questions. Thus, a possible (albeit labor-intensive) solution for increasing measurement validity may be to assess several sessions from the same patient, as done for example by Hörz-Sagstetter and her colleagues (2015).

Future studies should test the associations between the PACS Exploring scale, its subscales, and other key psychotherapy process variables. The role of these scales as mediators and moderators of therapy outcome, both at the beginning and at later stages of treatment, would also be a fruitful area of investigation. Future research should also elucidate why RF on the AAI was found to be less closely related to the In-session RF scale than to the PACS Exploring Scale. In our view, it may be that the RF scale captures a slightly different construct when it is applied to the AAI than when it is applied to therapy sessions. Fonagy and his colleagues (1998) devised the RF scale to assess the capacity to consider mental states underlying behavior; and yet the demand questions in the AAI specifically prompt the speaker to discuss mental states from a current perspective (e.g., the question “Why do you think your parents behaved as they did” implicitly asks the speaker to reassess in the here and now the experiences she has been sharing). That is, when applied to the AAI, the RF scale may capture the capacity of the speaker to reflect in the here and now about mental states and openly talk about them with the interviewer, rather than mentalizing in general. These aspects are much less explicit when the RF scale is applied to psychotherapy sessions, where demand questions may be absent. For the same reason, these aspects may be better captured by the PACS Exploring scale, which focuses on patients’ here-and-now interaction with the therapist. Such view of the RF construct seems supported by our finding that, beyond the Autonomous reflection subscale, the Affective sharing subscale too was found to independently predict patients’ pre-treatment RF, and that the interaction of the two subscales seemed to be a better predictor of pre-treatment RF than any one of them considered individually.

It is important to emphasize that, in this vein, both the RF scale applied to the AAI and the PACS Exploring scale may reflect an underlying capacity to remain emotionally connected to another person without becoming dependent, i.e. the capacity to be autonomous. Patients with high RF do not hide their reflections, but neither do they exact the listener’s approval or support, as if they rest relatively assured that they will be trusted and listened to (Miller-Bottome, Talia, Safran, Muran, 2017). Difficulties in mentalizing, on the other hand, may be understood as difficulties in engaging in a collaborative meaning making process. Some speakers are reluctant to make guesses about mental states (e.g., “I don’t know how she feels”). Other speakers sound too self-assured or entitled (e.g., “She is in an Oedipal relationship, I know her better than she knows herself”), or vague and difficult to understand (e.g., “She’s, I guess, totally, yeah, I mean I think unconsciously she kinda wants to be sort of like that”) and thus restrict the possibility of their interlocutors to contribute to their reflections. In this context, being autonomous rests paradoxically on being spontaneously recognized by another, or on what Donald Winnicott called “the capacity to be alone in the presence of another” (1958). This capacity is the foundation of secure attachment, and it may also be an essential component of therapeutic relationships that truly foster change.

Clinical or Methodological Significance of this Article.

• Researchers and clinicians can assess patients’ mentalizing based on any single psychotherapy transcript, in many therapeutic modalities

• The Exploring scale of the Patient Attachment Coding System can yield a reliable measure of reflective functioning based on any single psychotherapy transcript, in many therapeutic modalities

• Client differences in mentalizing manifest in part independently of the therapist’s contributions

Clinical and research significance of the results.

• Researchers and clinicians can assess patients’ mentalizing based on any single psychotherapy transcript, in many therapeutic modalities

• The Exploring scale of the Patient Attachment Coding System can yield a reliable measure of patients’ overall reflective functioning

• Client differences in reflective functioning manifest in part independently of the therapist’s contributions