Multimethod Investigation of Interpersonal Functioning in Borderline Personality Disorder

Abstract

Even though interpersonal functioning is of great clinical importance for patients with borderline personality disorder (BPD), the comparative validity of different assessment methods for interpersonal dysfunction has not yet been tested. This study examined multiple methods of assessing interpersonal functioning, including self- and other-reports, clinical ratings, electronic diaries, and social cognitions in three groups of psychiatric patients (N=138): patients with (1) BPD, (2) another personality disorder, and (3) Axis I psychopathology only. Using dominance analysis, we examined the predictive validity of each method in detecting changes in symptom distress and social functioning six months later. Across multiple methods, the BPD group often reported higher interpersonal dysfunction scores compared to other groups. Predictive validity results demonstrated that self-report and electronic diary ratings were the most important predictors of distress and social functioning. Our findings suggest that self-report scores and electronic diary ratings have high clinical utility, as these methods appear most sensitive to change.

A core feature of Borderline Personality Disorder (BPD) is chronic and severe interpersonal dysfunction, which is defined by intense and stormy relationships, fears of abandonment, and vacillations between idealization and devaluation within relationships (American Psychiatric Association, 2000; Clarkin, Widiger, Frances, Hurt, & Gilmore, 1983; Gunderson, 2007; Kehrer & Linehan, 1996). These interpersonal problems are often a central focus of treatment and endure even after other symptoms have remitted (Bateman & Fonagy, 1999; Benjamin, 1996; Clarkin, Kernberg, & Yeomans, 1999; Munroe-Blum & Marziali, 1995; Gunderson, 2001).

Despite the clinical importance of the interpersonal characteristics of BPD, research on this topic has suffered from two significant problems. First, previous research has not examined the comparative validity of different methods (e.g., self-report questionnaires versus clinical interviews) for measuring interpersonal dysfunction in patients with BPD. Second, assessment instruments and research designs have not been sufficiently specific to BPD and have often lacked theoretical relevance to the interpersonal dysfunction characteristic of the disorder. Researchers and clinicians have often relied on global summary measures of interpersonal functioning, such as disruptions in romantic relationships (e.g., marital separations, divorce) or the presence of abuse by a partner (e.g., Daley, Burge, & Hammen, 2000; Labonte & Paris, 1993; Linehan, Tutek, Heard, & Armstrong, 1994). Such measures are informative but are not well suited to capturing maladaptive relationship patterns assumed to be specific to BPD. Moreover, comparison groups usually consist of healthy participants, limiting our understanding of the validity of these interpersonal indicators for measuring dysfunction particular to BPD relative to other forms of psychopathology.

The most common interpersonal functioning measures are self-report scales (most often from the perspective of the patient and less frequently from the perspective of other informants; e.g., Social Adjustment Scale—Self-Report; Weissman & Bothwell, 1976; Inventory of Interpersonal Problems; Horowitz, Rosenberg, Baer, Ureno, & Villasenor, 1988), structured interviews that are usually focused on broad social roles or areas of interpersonal life (e.g., Social Functioning Schedule; Remington & Tyrer, 1979), or assessments of life events and developmental milestones reflective of interpersonal functioning (e.g., marital status). Other methodologies that are more indicative of day-to-day experience and interpersonal processes and transactions are also available, however. These techniques include experience sampling and day reconstruction methods (Russell, Moskowitz, Zuroff, Sookman, & Paris, 2007; Kahneman, Krueger, Schkade, Schwartz, & Stone, 2004), assessments of social network structure (Clifton, Pilkonis, & McCarty, 2007), and paradigms from research in experimental psychopathology aimed at documenting “real-time” processing of social information and mechanisms of social cognition (Fertuck et al., 2009; Lynch, Rosenthal, Kosson, Cheavens, Lejuez, & Blair, 2006). In this report, we compare the predictive validity of self- and other-reports from the Inventory of Interpersonal Problems (IIP); clinical ratings derived from a semi-structured interview, the Revised Adult Personality Functioning Assessment (RAPFA), designed to elicit information about interpersonal processes theoretically relevant to BPD; data from a 7-day diary of social interactions modeled on the Rochester Interaction Record (Wheeler & Nezleck, 1977; Reis & Wheeler, 1991); and data from an emotion recognition and attribution task using human faces.

The daily diary assessment method is designed to fill the “gap in understanding how social relationships and interactions are enacted and experienced in the moment” (Tidwell, Reis, & Shaver, 1996, p. 729). When interpersonal behavior in BPD is tracked using daily diaries, findings indicate that individuals with BPD have more difficulties in everyday interpersonal interactions. Using this methodology, Russell and colleagues (2007) found that individuals with BPD were more frequently quarrelsome and submissive and less dominant than healthy controls. Moreover, they found that intraindividual variability in quarrelsomeness, dominance, and agreeableness was greater in BPD, suggesting a pattern of alternation between anger/antagonism and friendliness. Similarly, Stepp and colleagues (2009) used an electronic diary to assess daily interpersonal interactions and found that patients with BPD reported more disagreements and more confusion during their social interactions when compared to patients with other personality disorders and psychiatric patients without personality disorders. Patients with BPD also reported more hostility, emptiness, and ambivalence during social interactions compared to patients in the other psychiatric groups. Taken together, results using daily diary methods have corroborated clinical impressions that the interpersonal worlds of patients with BPD are characterized by hostile control, ambivalence, and oscillations between hostility and friendliness.

According to cognitive theories of BPD, information-processing biases may be linked to underlying beliefs or assumptions, which, in turn, affect interpersonal behavior. For example, Beck and Freeman (1990) argued that three beliefs are central to BPD: (a) the social environment is generally malevolent, (b) one is personally powerless and vulnerable, and (c) one is inherently unacceptable. These beliefs are thought to give rise to biased appraisals, such that persons with BPD perceive hostility and rejection, even in ambiguous or innocuous situations. Butler and colleagues (2002) found that several dysfunctional beliefs distinguished BPD patients from patients with other personality disorders, including beliefs about dependency, helplessness, mistrust, and fears of abandonment, rejection, and loss of control. Moreover, studies have documented that patients with BPD systematically misinterpret interpersonal cues, revealing a negative bias to neutral faces (Wagner & Linehan, 1999); tend to selectively remember negative information (Korfine & Hooley, 2000); and evaluate interpersonal situations in extreme terms (Veen & Arntz, 2000). These biased perceptions and memories may explain, in part, the interpersonal and emotional disturbances that characterize BPD. Lynch and colleagues (2006) used a facial morphing task in which faces slowly changed from neutral to intense emotional expressions to assess sensitivity to facial emotion in BPD. Individuals with BPD were able to identify accurately facial emotion earlier than healthy controls across six emotions (anger, fear, sadness, surprise, happiness, and disgust), suggesting that BPD is characterized by enhanced awareness of others’ emotions. Using a related paradigm (‘Reading the Mind in the Eyes’; Baron-Cohen, Wheelwright, Hill, Raste, & Plumb, 2001), Fertuck and colleagues (2009) demonstrated that individuals with BPD discriminated others’ positive and neutral emotions more accurately than healthy controls when viewing only a picture of the person’s eyes.

In summary, studies to date have most often used a single, global method to assess interpersonal functioning in BPD and have not examined which methods best discriminate between patients with BPD and those with other psychiatric disorders. Additionally, the predictive validity of different assessment methods remains unknown. This study was designed, in part, to address these limitations. Thus, the overall aim of this study was to examine multiple methods of interpersonal functioning, including self- and other-reports of interpersonal problems, clinical ratings of social functioning, electronic social interaction diaries, and indices of social information processing in patients with BPD compared to psychiatric controls. We expected that participants with BPD would report higher interpersonal dysfunction when compared to other groups of patients. The methods differed in the timeframe used to capture interpersonal functioning, from lifetime summary indices to daily recordings to moment-to-moment judgments. The methods examined varied in temporal context from summary measures of interpersonal functioning, to day-to-day accounts of interpersonal interactions, to moment-to-moment interpersonal appraisals. We also included measures reflecting the behavioral patterns presumed to be specific to interpersonal dysfunction in BPD (e.g., the RAPFA, descriptors of interpersonal events in the daily diary relevant to BPD) and predicted that these methods would be most likely to discriminate BPD patients from those with other psychiatric disorders. Lastly, we expected that interpersonal dysfunction would demonstrate validity in predicting clinically relevant outcomes, namely symptom distress and social functioning, six months after the initial assessment battery. We examined which method(s) were most important in predicting these outcomes.

Method

Sample

Patients (N=138) from 21 to 60 years old were solicited from the general adult outpatient clinic at Western Psychiatric Institute and Clinic and were active in treatment at the time of participation in this study. Patients with psychotic disorder, organic mental disorders, and mental retardation were excluded, as were patients with major medical illnesses that influence the central nervous system and might be associated with organic personality change (e.g., Parkinson’s disease, cerebrovascular disease, seizure disorders). All study procedures were approved by the University of Pittsburgh Institutional Review Board.

To examine the specificity of interpersonal dysfunction in BPD, we recruited patients from three groups: 1) those with BPD; 2) those with another personality disorder (OthPD); and 3) those without a personality disorder (NoPD). Announcements describing the study were posted in the clinic. Patients interested in participating contacted research staff and were pre-screened by phone for the presence or absence of personality disorder symptoms. An intake appointment was then scheduled. After completing intake procedures, participants completed a six-month follow-up assessment session.

The mean age of the sample was 38 years (SD=10.6) and 104 participants (75.4%) were female. One hundred and three participants (74.6%) identified as Caucasian, 33 (23.9%) as African American, 1 (.7%) as Native American, and 1 (.7%) as Asian. Two participants (1.4%) identified ethnicity as Hispanic. In terms of marital status, 71 participants (51.4%) were single and never married, 36 (26.1%) were separated or divorced, 29 (21%) were married or in a long-term committed relationship, and 2 (1.4%) were widowed. A large majority of the sample obtained education beyond high school (n=111; 80.4% with at least some vocational or college training), but the financial deprivation of the sample was high: 45.0% of the participants reported annual household incomes of less than $10,000, and 66.7% less than $20,000.

All participants were interviewed according to a standardized assessment schedule that including the completion of several intake appointments as well as a 6-month follow-up visit (see Table 1). During the first assessment session, all patients were administered the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-I; First et al., 1997b). The most prevalent current diagnoses were affective and anxiety disorders (n=54; 39.1%), followed by more complex presentations (‘other disorders’) that included eating, somatoform, dissociative, and sexual disorders comorbid with more common affective, anxiety, and substance use disorders (n=29; 21.0%).

Table 1.

Assessment Schedule

Intake Assessment Schedule	Time
SCID-I	Week 1
Self-report forms (IIP, BSI, SAS-SR)	Between Weeks 1 and 2
Other-report forms (IIP)	Sent out between Weeks 1 and 2
Interpersonal Relations Assessment (interview used during diagnostic case conference)	Week 2
Electronic social interaction diary	Between Weeks 2 and 3
SCID-II	Week 3
RAPFA	Week 4
Faces Task	Week 5
Six-Month Follow-Up Assessment Schedule	Time
Self-report forms (BSI, SAS-SR)	6 months from initial visit

Diagnostic Procedures

Diagnostic assessments required three sessions, and each session lasted approximately 2 hours. Session 1 included administration of the SCID-I and other measures of current symptomatology. In session 2, a detailed social and developmental history was taken, using a semi-structured interview, the Interpersonal Relations Assessment (IRA; Heape et al., 1989), developed for this purpose. During session 3, the Structured Clinical Interview for DSM-IV Axis II Personality Disorders (SCID-II; First et al., 1997a) was administered.

Following the intake evaluation, the primary interviewer presented the case at a 3-hour diagnostic conference with other colleagues from the research team (all members had a minimum of a master’s degree in social work or clinical psychology). A minimum of three judges participated. All available data (historical and concurrent) were reviewed and discussed at the conference. Judges had access to all that had been collected: current and lifetime Axis I information, symptomatic status, social and developmental history, and personality features endorsed from the Axis II interview. For the present purpose, the key measures that emerged from the best-estimate consensus were the specific DSM-IV criteria and resulting diagnoses assigned.

During the diagnostic conference, a checklist of Axis II criteria for all PDs was completed by consensus, with each item rated absent (0), present (1), or strongly present (2). Based on the best-estimate diagnostic case conference, the total sample included 54 BPD participants, 55 participants with another personality disorder, and 29 participants with no significant personality dysfunction. The majority of the other personality disorder group was composed of patients with a DSM-IV cluster C personality disorder (n=54; 98.2%). Participants in this group also met criteria for Cluster A (n=5; 9.1%) and non-BPD Cluster B (n=15; 27.3%) personality disorders.

Self-report

Between diagnostic interview sessions, participants completed an adapted version of the Inventory of Interpersonal Problems (IIP; Horowitz, et al., 1988), which contained 88 items assessing difficulties in interpersonal relatedness (assertiveness, sociability, intimacy, submissiveness, responsibility, and control). We selected 64 items to score the eight circumplex scales (Alden, Wiggins, & Pincus, 1990) and 24 items to allow scoring of the IIP-Personality Disorder scales (Pilkonis, Kim, Proietti, & Barkham, 1996).The internal consistency of the IIP items in this study was .96. The mean score across all 88 IIP items (i.e., total IIP score) was used to represent the overall mean level of interpersonal stress and has been shown to be a reliable and valid measure of interpersonal distress (Kaye & Shea, 2000).

Other-report

During the patient diagnostic assessment sessions, interviewers solicited collateral information from significant others (SOs) who knew the participant well (usually spouses, parents, or close friends). Participants could nominate up to three SOs who completed a third-person version of the 88-item version of the IIP. Of the 138 participants in the original sample, 108 (78.3%) also had IIP data from at least one SO. The mean total IIP score across all available SO reports was used to represent the overall other-reported mean level of interpersonal distress for the current study. The internal consistency of the other-reported IIP items in this study was .96. SO participation did not differ by PD group status (BPD vs. all others; χ²=.10, p=.755).

Clinical Interview

A second clinical interviewer who was blind to the results from the diagnostic assessments measured interpersonal functioning using the Revised Adult Personality Functioning Assessment (RAPFA; Hill, Harrington, Fudge, Rutter, & Pickles, 1989; Hill & Stein, 2000a; Hill & Stein, 2000b). The RAPFA enquires about interpersonal functioning over the previous 5 years in the domains of work, romantic relationships, friendships, and non-specific social relationships. Through structured questions and probes that explore the range of functioning in each domain, a trained interviewer rates each domain on a nine-point scale according to severity and pervasiveness of dysfunction. For example, a ‘1’ rating on romantic relationships requires temporal stability, positive trust, marked confiding, and enjoyment, whereas a ‘9’ rating requires the absence of sustained committed relationships or the maintenance of relationships in the face of sustained discord or violence. This interview provides a standard structure and operationalization for coding dysfunction in each domain. All interviewers received on-going training and consultation from the developer of the measure, Jonathan Hill, throughout the course of the study. The interviews were audio-recorded, and detailed reports were prepared from the tapes.

RAPFA ratings of the severity of impairment in each domain were made in a second consensus conference. Each conference included a minimum of three judges (the RAPFA interviewer and two other members of the research team) who were different from the judges in the diagnostic case conference and were also blind to the results of the Axis I and II diagnostic assessment and conference. The mean score across all four RAPFA domains (i.e., total RAPFA score), was used to represent the overall level of interpersonal functioning. The total RAPFA score ranged from 2–9 (M=6.08, SD=1.46). Throughout the course of the study, 10 cases were randomly selected as reliability cases and were rated by an average of seven judges to measure inter-rater reliability. The intraclass correlation coefficient (ICC) for the total RAPFA score in the current study was .84.

Electronic Diary

Following the diagnostic and social functioning assessment sessions, each participant was scheduled a 30–45-minute social interaction electronic diary (ED) orientation session. Of 138 total participants, 110 (80.0%) completed the ED. Participation in the social interaction ED protocol did not differ by PD group status (BPD vs. all others; χ²=.79, p=.38). A complete description of the ED protocol has been provided elsewhere (see Stepp et al., 2009). Participants were asked to complete the ED for seven consecutive days, twice a day (at midday and before going to bed) regarding one social interaction that lasted at least 10 minutes. If participants did not have any social interaction for the time period of interest, they completed a record reflecting the lack of social interaction.

Participants rated their behavior and emotional responses during or immediately after the interaction. The behavior ratings consisted of 7 items reflecting the degree of personal disclosure, other disclosure, intimacy, control, disagreement, confusion, and closeness (1=very little to 10=a great deal); and one additional item measuring satisfaction with the interaction (1=I got less than I expected to 10=I got more than I expected). The emotion ratings consisted of 33 unipolar items reflecting both positive and negative emotions. Each of these items was rated on a 10-point scale (1=not at all to 10=a great deal). The emotional response items chosen for the ED included items to tap four main aspects of emotional experiences: (a) depression, (b) anxiety, (c) anger, and (d) positive emotions. We also chose emotional experiences particularly relevant for BPD patients, such as “emptiness,” and “shame.”

Factor analysis was used to reduce the number of response ratings to a more manageable yet conceptually meaningful set of variables. Principal axis factoring with varimax rotation was employed on the 41 behavior and emotion ratings. Examination of the scree plot and eigenvalues suggested that a seven-factor solution was most appropriate. One item was deleted because it failed to significantly load on any one factor (< .35: “I influenced the interaction”), while four items were deleted because of significant loadings on more than one factor (“happy”, “comfortable,” “sad,” and “mad”). To ensure that the factor structure remained the same once these items were removed, the EFA was re-run. The seven factors were used to define seven variables: Affiliation ( “I disclosed” and “Degree of closeness”), Anxiety (“nervous” and “shaky”), Rejection ( “feeling left out” and “feeling ignored”), Confusion (“feeling ambivalent” and “feeling unsure”), Shyness (“feeling shy” and “feeling embarrassed”), Emptiness (“feeling numb” and “feeling empty”), and Shame (“feeling that I treated others badly” and “feeling bad”). Six factors that captured negative interpersonal experiences were summed (i.e., Anxiety, Rejection, Confusion, Shyness, Emptiness, and Shame) to create negative interpersonal experiences and the Affiliation factor was used to capture positive interpersonal experiences. An overall mean across all social interactions of these two variables was also created for use in the correlational analyses that reflected positive and negative interpersonal experiences over the course of the week rating period.

Social Cognition

Facial stimuli

To assess participants’ appraisal of social stimuli, a series of 24 photographs of male and female faces was selected from Matsumoto and Ekman’s (1988) Japanese and Caucasian Facial Expressions of Emotion and Neutral Faces. Slides consisted of 4 happy faces, 4 sad faces, 4 fearful faces, 4 angry faces, and 8 neutral faces, with equal numbers of male and female faces. All faces were Caucasian.

Dependent measures

For each stimulus slide, participants made three sets of judgments. All ratings were on a six-point scale (1=Not at all; 6=A great deal). First, they rated the extent to which each face looked sad, angry, fearful, or happy. Second, participants rated the extent to which the face looked rejecting and friendly. Finally, they rated the extent to which (1) they liked the person in the photograph and (2) the person in the photograph would like the participant. To ensure a balance in the number and content of items used to measure positive and negative social attributions, the last item was scored so that higher scores indicated less likelihood that the person would like them. Groups did not significantly differ on ratings of facial expressions as sad, angry, fearful, or happy. To reduce the remaining four variables into a smaller set of conceptually meaningful variables, we created two summary variables. The overall means of these items were moderately correlated (r=.39, p<.001). To capture negative social attributions, we created a sum score consisting of “This person is rejecting of others” and “This person would like me” (item is reverse-scored). To capture positive social attributions, we created a sum score consisting of two items: 1) “This person is friendly and engaging” and 2) “I like this person.” The overall means of these items were also moderately correlated (r=.55, p<.001). An overall mean across all facial stimuli of these two variables was also created for use in correlational analyses to measure the average of positive and negative social attributions across facial stimuli.

Procedure

All 24 photographs were imported into Microsoft PowerPoint, and images are presented as a “slide show,” with each face appearing on the screen for 90 seconds. Participants were instructed that the purpose of the experimental task was to obtain ratings for facial slides about the emotions, personality, and attitudes expressed. After instructions were presented, a different face was displayed on the screen for 90 seconds each, with gender alternated between slides.

Six-month Follow-up

Symptom distress

The Brief Symptom Inventory (BSI; Derogatis, 1993) consists of 53 items covering nine symptom dimensions: Somatization, Obsession-Compulsion, Interpersonal Sensitivity, Depression, Anxiety, Hostility, Phobic anxiety, Paranoid ideation and Psychoticism. To measure current levels of distress, we computed a composite of these symptoms, the Global Severity Index. This instrument was administered at baseline and at 6-month follow-up. The internal consistency of this measure was .96 and .97 at baseline and 6-months, respectively. The test-retest reliability from baseline to 6-month follow-up was high (r = .82, p <.001).

Social functioning

The Social Adjustment Scale – Self Report (SAS-SR; Weissman & Bothwell, 1976) contains 42 items that assess role performance in the past two weeks across six domains: work/school role, social/leisure time, family outside the home, primary relationship, parental role, and family unit. Each item is rated on a five-point scale, with higher scores indicating greater impairment in functioning. An overall adjustment score was obtained by summing the scores of all the items and dividing by the number of items actually answered. Since not all sections apply to all participants (e.g., those that do not have children do not complete questions regarding their parental role functioning), internal consistency estimates cannot be calculated. This measure was also administered at baseline and 6-month follow-up and demonstrated high test-retest (r = .77, p < .001).

Data Analytic Approach

A major aim of our study was to determine the method of interpersonal assessment that is most sensitive to personality disorders, specifically BPD. Our data provided a rich set of potential indicators of interpersonal functioning stemming from several sources, including self-reports, semi-structured clinical interview, experimental psychopathology, and significant others’ reports about participants. We first computed bivariate correlations among all interpersonal functioning variables to examine method convergence. Then, we examined mean level differences in interpersonal functioning variables across personality disorder groups (NoPD, OthPD, and BPD). In most clinical and research settings, it would be too burdensome to characterize interpersonal dysfunction in such depth. Thus, we used dominance analysis (DA; Azen & Budescu, 2003) to identify the most salient and potent interpersonal functioning predictors of social adjustment and symptom distress at 6-month follow-up.

DA is a robust method for comparing the predictive strength of each predictor by comparing model R² values for all possible subset models. The dominance of one predictor over another is established by its contributing more variance to the prediction of the outcome for all possible subset models relative to another predictor. Azen and Budescu (2003) outlined three forms of predictor dominance representing varying strengths: complete, conditional, and general (from strongest to weakest). A predictor is said to “completely dominate” another predictor when it contributes more variance to the prediction of the outcome than the comparison predictor for each subset model of any size. Conditional dominance is established when a predictor contributes more variance, on average, to models at each possible model size, relative to another predictor. For example, for 7 predictors, subset models may range from 0–6 predictors (excluding the full model). Conditional dominance requires that the addition of one predictor explains more variance, on average, than another predictor for each model size. Lastly, general dominance is established when one predictor explains more variance than another, on average, across models of all sizes (i.e., the average R² contribution for the predictor across all model sizes).

DA has many attractive features when examining predictor importance compared to traditional methods. First, DA provides measures of predictive strength for all possible combinations of predictors, rather than deciding importance on the sole basis of the full model (e.g., standardized beta coefficients) or pairwise correlations (Azen & Budescu, 2003; Budescu, 1993). In addition, DA provides a measure of importance that represents information about the predictor’s total effect (i.e., in the presence of all other predictors), partial effect (i.e., in the presence of some other predictors), and direct effect (i.e., the predictive strength when considered alone).

It is difficult to know whether the pattern of predictor dominance observed in a given sample would be replicated in a separate sample because basic DA provides an order of predictor importance, but no confidence estimates on this ordering. Thus, Azen and Budescu (2003) developed a bootstrap resampling procedure that yields replicability estimates of the observed dominance pattern. In the present study, dominance results were bootstrapped by drawing 1000 bootstrap samples (of size n = 138) with replacement from the original dataset and predictor dominance was computed in each bootstrap sample.

Given that approximately 40% of the sample was missing at least one of the seven interpersonal functioning variables, multiple imputation (MI) was used to handle missingness (Schafer & Graham, 2002). More specifically, 50 multiple imputation datasets were generated using Markov-Chain Monte Carlo simulation (as implemented by PROC MI in SAS 9.2), where each MI dataset contained complete observations for all variables, with plausible values imputed for data missing in the original dataset (see Rubin, 1989, for an introduction to MI). R² values for DA subset models were averaged across all imputed datasets, thereby providing the best estimate of predictor importance. Bootstrapped dominance values were combined with the MI procedure by computing dominance values across 1000 bootstrap samples per imputed dataset. Thus, for each MI dataset, bootstrapping was carried out separately, and bootstrapped dominance values were combined across MI datasets by arithmetic averaging.

Results

Correlations among Measures of Interpersonal Functioning and Outcomes

Correlations among self- and other-reported IIP scores, clinician RAPFA ratings, electronic diary ratings for positive and negative interpersonal experiences (averaged across the week rating period), and positive and negative face attribution ratings (averaged across face condition) are presented in Table 2. Generally, correlations among measures of conceptually similar constructs were low, with many nonsignificant. However, the self-reported IIP scores were moderately correlated with other-reported IIP scores, clinician RAPFA rating, electronic diary ratings for negative interpersonal experiences, and negative face attribution ratings (rs ranged from .29 to .55, ps < .05). Additionally, the correlations between negative face attribution ratings and other-reported IIP scores, clinician RAPFA ratings, and electronic diary ratings for negative interpersonal experiences were generally low in magnitude (rs ranged from .23 to .32, ps < .05). The other-reported IIP scores were modestly correlated with clinician RAPFA ratings and negative face attribution ratings (r = .29, p < .05; r = .20, p < .05, respectively). Finally, the clinician RAPFA ratings were modestly correlated with electronic diary ratings for negative interpersonal experience and negative face attributions (r = .21, p < .05; r = .23, p < .05, respectively).

Table 2.

Correlations among Measures of Interpersonal Functioning, Distress, and Social Functioning

	1.	2.	3.	4.	5.	6.	7.	8.	9.	10.	11.
Self-Report
1. IIP score	--
Other-Report
2. IIP score	.29**	--
Clinical Rating
3. RAPFA	.37***	.29**	--
Electronic Diarya
4. Positive interpersonal experiences	.01	.10	−.15	--
5. Negative interpersonal experiences	.55***	.19	.21*	.06	--
Social Cognition: Faces Task
6. Positive attributions	.01	−.01	−.11	.09	.02	--
7. Negative attributions	.31***	.20*	.23**	−.10	.32***	−.54***	--
Distress
8. BSI (Baseline)	.64***	.31***	.40***	.00	.46***	.04	.14	--
9. BSI (6-month follow-up)	.58***	.34***	.38***	−.08	.55***	−03	.17	.82***	--
Social Functioning
10. SAS-SR (Baseline)	.64***	.21*	.46***	−.05	.45***	−.08	.29***	.71***	.66***	--
11. SAS-SR (6-month follow-up)	.60***	.23*	.46***	−.22*	.57***	−.09	.28**	.58***	.69***	.77***	--

Notes.

^aElectronic Diary ratings were represented by factor scores of positive and negative interpersonal experiences.

^*p < .05

^**p < .01

^***p < .001

In most cases, the interpersonal variables were moderately correlated with distress and social functioning at intake and 6-month follow-up. Overall, self-reported IIP scores, clinician RAPFA ratings, and negative electronic diary ratings had the strongest associations with distress (rs ranged from .38 to .64, ps < .001) and social functioning (rs ranged from .45 to .64, ps < .001) at intake and six-month follow-up.

Mean Differences in Interpersonal Functioning across Personality Disorder Group

We examined differences between patients in the control conditions (NoPD or OthPD) and BPD patients on measures of interpersonal functioning. Means and standard deviations for all interpersonal functioning variables are presented in Table 3. Data were inspected to ensure that the assumptions of homogeneity of variance and normality were met. The BPD group was compared to the two control patient groups (NoPD and OthPD) on self-report, other-report, and clinical ratings interpersonal functioning variables via analysis of variance (ANOVA). The BPD group was compared to the two control patient groups on electronic diary interpersonal functioning variables via random-effects ANOVA. Additionally, the BPD group was compared to the two control patient groups on social cognition indices using linear mixed models with face condition (Angry, Sad, Scared, Happy, and Neutral) as the within-group variable and patient group (BPD, OthPD, and NoPD) as the between-group variable.

Table 3.

Means Differences in Interpersonal Functioning across Personality Disorder Group

	NoPD	OthPD	BPD	F	df(1,2)	NoPD-OthPDa	NoPD-BPDa	OthPD-BPDa
Self-report
1. IIP score	1.35 (.58)	1.66 (.63)	1.97 (.50)	10.95***	2, 129	−0.32	−0.62***	−0.30*
Other-report
2. IIP score	1.15 (.39)	1.63 (.65)	1.66 (.56)	7.12***	2, 105	−0.48**	−0.51**	−0.03
Clinical Ratings
3. RAPFA	4.44 (1.70)	6.23 (1.38)	6.77 (.94)	35.02***	2, 130	−1.79***	−2.33***	−0.54
Electronic Diaryb
4. Positive interpersonal experiences	0.10 (.11)	−0.13 (.08)	0.00 (.08)	1.61**	2, 1117	0.23	0.10	−0.13
5. Negative interpersonal experiences	−0.19 (.17)	−0.12 (.10)	0.21 (.10)	5.63**	2, 1117	−0.06	−0.40*	−0.34*
Social Cognition: Faces Taskc
6. Positive attribution	3.11 (.10)	3.10 (.07)	3.03 (.07)	.25	2, 135	.01	0.07	0.06
7. Negative attribution	3.11 (.11)	3.33 (.07)	3.50 (.08)	4.33*	2, 135	−.22	−0.39*	−0.17

Notes.

^aTukey test of multiple comparisons of group differences.

^bElectronic Diary ratings were represented by factor scores of positive and negative interpersonal experiences.

^cThe ANOVA results presented control for face condition (Angry, Sad, Scared, Happy, and Neutral) because face condition did not significantly interact with patient group.

^*p < .05

^**p < .01

^***p < .001

Results indicated a main effect for patient group for all variables with the exception of positive face attribution ratings (see Table 3). Tukey tests of multiple comparisons of group differences showed that BPD participants had significantly higher (1) self-reported IIP scores and (2) electronic diary ratings of negative interpersonal experiences compared to NoPD and OthPD participants. BPD participants had significantly higher (1) other-reported IIP scores, (2) clinician RAPFA ratings, and (3) negative face attribution ratings compared to NoPD participants but not compared to OthPD participants.

Interpersonal Functioning Associations with Six-Month Follow-up

Dominance analysis (DA) was used to determine the relative importance of each measure of interpersonal functioning in the prediction of symptom distress and overall social functioning six months following the initial assessment. Separate dominance analyses were computed for BSI symptom distress and SAS-SR overall social functioning indices. Predictor variables in the DA included all seven interpersonal functioning scores (electronic diary ratings were averaged across weeks and social cognition ratings were averaged across face emotional valences). In addition, BSI and SAS scores at intake were entered into the DA, thereby controlling for differences in baseline status and providing an analysis of interpersonal functioning variables that predict change in symptoms and social functioning, respectively.

DA of BSI symptom distress revealed that, as expected, baseline BSI scores completely dominated all other predictors, indicating that initial BSI status is closely linked with follow-up status. Complete dominance could not be readily established among other predictors of either BSI or SAS-SR scores, largely because complete dominance is a very stringent test (i.e., one predictor must best another in all possible submodels that include the pair) that often results in indeterminate dominance patterns (Azen & Budescu, 2003). Thus, comparisons among interpersonal functioning predictors centered on conditional and general dominance, which provide more information about the overall strength of each predictor relative to the larger set. Note that conditional dominance is more stringent than general dominance and that conditional dominance implies that general dominance holds as well.

As can be seen in Table 4, in terms of general dominance (the average contribution across model sizes), the ordering of importance (from most to least) was: Self-report IIP scores > Negative electronic diary ratings > RAPFA clinical ratings > Other-report IIP scores > Positive attributions > Negative attributions > Positive electronic diary ratings. Conditional dominance among the predictors was less clear, but largely followed the same pattern (see Table 4 for details). One notable finding was that negative electronic diary ratings tended to have greater predictive strength in larger models (> 4 predictors) relative to self-report IIP scores and clinical RAPFA ratings, suggesting that negative diary ratings data represent a relatively unique source of variance in predicting BSI scores.

Table 4.

Additional R² contributions for each predictor to models of each size predicting follow-up BSI

Predictor	Model Size (number of predictors without the predictor in question)								Average (across sizes)
	0	1	2	3	4	5	6	7
BSI (Baseline)	.66	.54	.46	.40	.35	.32	.29	.27	.41
Self-report, IIP Score	.33	.22	.15	.09	.06	.03	.01	.001	.11
ED, Negative Interpersonal Experiences	.25	.17	.11	.08	.06	.04	.03	.03	.10
Clinical Rating, RAPFA	.14	.08	.05	.03	.01	.01	.003	.001	.04
Other-report, IIP Score	.11	.07	.04	.02	.02	.01	.007	.006	.04
Social Cognition, Positive Attribution	.008	.007	.008	.009	.01	.01	.02	.02	.01
Social Cognition, Negative Attribution	.03	.01	.004	.003	.005	.007	.008	.007	.09
ED, Positive Interpersonal Experiences	.005	.006	.005	.005	.006	.006	.006	.007	.006

Notes. ED = Electronic Diary; Social Cognition was measured with the Faces Task

Estimates of BSI predictor dominance in our sample were subjected to bootstrap resampling in order to ascertain the reliability of the established dominance pattern. One thousand bootstrap samples were drawn for each MI dataset and the proportion of bootstrap samples that were consonant with the sample dominance results was used as an estimate of reproducibility (Azen & Budescu, 2003). As described above, the general dominance and conditional dominance orderings of predictor importance were largely identical for model sizes 0–4 and the general dominance results provided the clearest ordering of predictor importance. Thus, for clarity and brevity, the focus of the bootstrapped results (which provide information about the reproducibility of the observed dominance pattern) is on general dominance. For each bootstrapped sample a single D_ij value was computed for each pair of predictors, where predictor i is the dominant predictor and predictor j is the subdominant predictor in the observed data. If i dominated j in a particular bootstrapped sample, then D_ij = 1; if j dominated i, D_ij = 0; if neither dominated, D_ij = 0.5. D_ij values were averaged across the bootstrapped samples to derive a best estimate for the reproducibility of the dominance pattern in the observed data.

Results from the bootstrapping procedure for BSI are presented in Table 5. Several conclusions can be drawn from the bootstrap results. Self-report IIP scores and Negative electronic diaries generally dominate all of the other predictors with a high degree of confidence (reproducibilities > .9), but we have less confidence that self-report IIP is superior to Negative electronic diary ratings (reproducibility = .64). Clinical RAPFA ratings and other-report IIP scores are roughly of comparable importance (reproducibility = .55), albeit with less explanatory power than self-report IIP scores and negative electronic diary ratings. The general dominance pattern of the positive attributions, negative attributions, and positive electronic diary indices was more difficult to establish (reproducibilities <= .7), but these predictors are of minor importance in the prediction of symptom distress (Average R² contributions ranging from .006-.01).

Table 5.

BSI general dominance bootstrapped D_ij values (reproducibility) and Standard Errors

Dominant Predictor	Subdominant Predictor	Avg Dij (Reproducibility)	SE(Dij)
Self-report, IIP score	ED, Negative	0.64	0.43
Self-report, IIP score	Clinical rating, RAPFA	0.99	0.06
Self-report, IIP score	Other-report, IIP score	0.98	0.10
Self-report, IIP score	Social cognition, Positive	1.00	0.03
Self-report, IIP score	Social cognition, Negative	1.00	0.001
Self-report, IIP score	ED, Positive	1.00	0.01
ED, Negative	Clinical rating, RAPFA	0.92	0.20
ED, Negative	Other-report, IIP score	0.92	0.22
ED, Negative	Social cognition, Positive	0.99	0.07
ED, Negative	Social cognition, Negative	1.00	0.02
ED, Negative	ED, Positive	0.99	0.05
Clinical rating, RAPFA	Other-report, IIP score	0.55	0.44
Clinical rating, RAPFA	Social cognition, Positive	0.89	0.29
Clinical rating, RAPFA	Social cognition, Negative	0.97	0.17
Clinical rating, RAPFA	ED, Positive	0.95	0.19
Other-report, IIP score	Social cognition, Positive	0.84	0.33
Other-report, IIP score	Social cognition, Negative	0.91	0.26
Other-report, IIP score	ED, Positive	0.92	0.23
Social cognition, Positive	Social cognition, Negative	0.52	0.46
Social cognition, Positive	ED, Positive	0.67	0.43
Social cognition, Negative	ED, Positive	0.70	0.42

Notes. Predictors are arranged in order of most generally dominant to least dominant. Intake BSI is omitted from the table because of its status as a baseline covariate (rather than a substantive predictor) and because it completely dominated all other predictors.

ED, Negative = Electronic Diary, Negative Interpersonal Experiences; ED, Positive = Electronic Diary, Positive Interpersonal Experiences; Social Cognition, Positive = Social Cognition, Positive Attributions; Social Cognition, Negative = Social Cognition, Negative Attributions. Social Cognition was measured with the Faces Task.

A second DA was conducted to understand the relative importance of interpersonal functioning indices to general social functioning measured by the SAS-SR. As expected, baseline SAS-SR scores completely dominated all other predictors, indicating that initial SAS-SR status is closely linked with follow-up status. In terms of general dominance in the prediction of the SAS-SR, the ordering of relative importance (from most to least) was: self-report IIP scores > negative electronic diary ratings > RAPFA clinical ratings > positive electronic diary ratings > other-report IIP scores > Negative attributions > Positive attributions. Of note, the three most important predictors were identical between SAS-SR and BSI (i.e., self-report IIP scores, negative electronic diary ratings, and RAPFA clinical ratings), as was the ordering of their relative importance. In terms of conditional dominance, the pattern was largely similar to general dominance and followed closely with the BSI results: self-report IIP scores showed greater predictive strength in smaller models (0–4 predictors), but negative electronic diary ratings dominated self-report IIP scores and RAPFA clinical ratings for larger model sizes.

General dominance results for the SAS-SR were subjected to the bootstrapping procedure described to determine the reliability of the observed dominance pattern¹. Self-report IIP scores generally dominated all other predictors with a high degree of confidence (reproducibilities >= .9) with the exception of negative electronic diary ratings (reproducibility = .71). Negative electronic diary ratings generally dominated RAPFA clinical ratings, Positive attribution ratings, positive electronic diary ratings, other-report IIP scores, negative attribution ratings, and positive attribution ratings with moderate to high confidence (reproducibilities between .76 and 1.0). RAPFA clinical ratings generally dominated positive electronic diary ratings, other-report IIP scores, and negative attribution ratings with moderate to high confidence (reproducibilities between .74 and .97). General dominance among positive electronic diaries, other-report IIP scores, negative attribution ratings, and positive attribution ratings was more difficult to establish (reproducibilities < .9), but the pattern of dominance in the bootstrapped results was generally consistent with sample dominance results, and these predictors were of lesser importance (Average R² contributions ranging from .005–.04).

Of note, two-way interactions between group status (BPD, OthPD, and NoPD) and each predictor variable were computed for both BSI and SAS-SR follow-up outcomes. We tested each interaction term in a separate regression model containing the main effects of group status and the predictor, as well as the predictor x group status interaction. In no case was any interaction term significant (all ps > .20), suggesting that methods for measuring interpersonal dysfunction are equally predictive of symptom distress and social functioning, despite group differences in the mean level of interpersonal dysfunction (BPD was often highest).

Discussion

This study measured several assessment methods of interpersonal functioning, including self- and other-reports of interpersonal problems, clinical interview ratings of social functioning, electronic social interaction diaries, and indices of social information processing in order to examine which method(s): (1) discriminated BPD patients from patients with other psychiatric disorders and (2) predicted change in symptom distress and social functioning at six-month follow-up. The methods differed in the timeframe used to capture interpersonal functioning, from lifetime summary indices to daily recordings to moment-to-moment judgments.

Overall, we found low to modest associations among these assessment methods of interpersonal functioning. Other studies have also found low convergence among measures of conceptually similar constructs, including affective instability in patients with BPD (Solhan, Trull, Jahng, & Wood, 2009) and behavioral activation in patients with bipolar disorder (Hayden et al., 2008). These findings highlight the importance of examining the comparative validity across assessment modalities because without this type of information, it is difficult to evaluate and integrate results from different studies that do not use the same assessment method.

As expected, patients with BPD reported higher levels of interpersonal dysfunction across assessment methods when compared to other psychiatric patients. These findings are consistent with previous work demonstrating higher levels of interpersonal impairment in BPD as measured with self-report instruments (e.g., Hilsenroth, Menaker, Peters, & Pincus, 2007), electronic diary methods (Russell et al., 2007; Stepp et al., 2009), and biases in social cognitive processes (Fertuck et al., 2009; Korfine & Hooley, 2000; Lynch et al., 2006; Veen & Arntz, 2000; Wagner & Linehan, 1999). Additionally, Oltamnns and colleagues (2004) demonstrated the utility of other-reports of personality pathology to predict work performance in the military. Lastly, clinical ratings of BPD symptoms have been found to predict less change in depressive symptoms six months after therapy (Meyer, Pilkonis, Proietti, Heape, & Egan, 2001). The majority of previous studies used healthy participants as controls, which limits our understanding of the specificity of these methods to interpersonal dysfunction in BPD.

By employing stringent comparator groups, the current study was able to provide an important extension of previous work. Although BPD participants consistently demonstrated higher levels of interpersonal dysfunction relative to the NoPD group, the level of interpersonal dysfunction in patients with BPD did not always trump that reported by patients in the OthPD group. Specifically, patients with BPD had higher levels of interpersonal dysfunction as measured with self-reported IIP scores and electronic diary ratings of negative interpersonal experiences when compared to patients in the NoPD and OthPD groups. Patients in the BPD group demonstrated higher levels of interpersonal dysfunction as measured with other-report IIP scores, clinical interview RAPFA ratings, and negative face attribution ratings when compared to participants in the NoPD group but did not differ from the level of interpersonal dysfunction reported in the OthPD group. Findings from this work indicate that self-report methods and negative electronic diary ratings were able to discriminate patients with BPD from psychiatric patients with another personality disorder. On the other hand, other-report, clinical interviewer, positive electronic diary, and face attribution ratings were not able to discriminate patients with BPD from patients with other types of personality pathology.

The discriminating ability of the negative electronic diary ratings may be due, at least in part, to the inclusion of indices that are presumed to uniquely characterize BPD, such as feelings of emptiness during social interactions. However, the RAPFA was also designed to specifically measure interpersonal behavioral patterns in BPD but, as an aggregate index of social functioning, did not discriminate BPD from other personality disorders. Future work will evaluate whether specific subscales of the RAPFA (e.g., rapid turnover in romantic relationships) are better suited to discriminating BPD from more general personality pathology. Finally, even though the self-report IIP scores were not designed with BPD in mind, this score was able to discriminate BPD from other personality pathology. Previous research has also found that BPD patients have higher self-report IIP scores compared to patients with other disorders (Hilsenroth, et al., 2007) and that subscales of the self-report IIP are related to BPD symptoms (Lejuez, Daughters, Nowak, Lynch, Rosenthal, & Kosson, 2003).

These findings have implications for the association between BPD and attachment styles. Results regarding the severity of interpersonal dysfunction among patients with BPD are consistent with the finding that insecure attachment styles, specifically unresolved, preoccupied, and fearful, are strongly associated with BPD (for a review see Agrawal, Gunderson, Holmes, & Lyons-Ruth, 2004). Choi-Kan and colleagues (2009) found that BPD participants were much more likely to have both preoccupied and fearful attachment styles compared to participants with depression and community controls. In turn, these insecure attachment styles were linked to interpersonal dysfunction, including avoidance behaviors as well as dependency in close relationships. It is plausible that insecure attachment styles account for, or mediate, the relation between BPD symptoms and interpersonal dysfunction. Future work should explore attachment style as a possible mechanism for these interpersonal behaviors in patients with BPD.

Results from the dominance analyses revealed the relative importance of these different interpersonal assessment methods in predicting change in symptom distress and social functioning six months after the initial assessment. The three methods that consistently provided the most information (in order from most to least) for both change in symptom distress and social functioning were: (1) self-reported IIP scores, (2) negative electronic diary ratings, and (3) RAPFA clinical interview ratings. Results from the bootstrapping procedures confirmed that self-report IIP scores and negative electronic diary ratings would be likely to generally dominate all other predictors in independent samples. However, it was more difficult to establish the dominance of self-report IIP scores relative to the negative electronic diary ratings. Additionally, when examining the strength of predictors across all model sizes (i.e., models with 0–7 interpersonal predictors), negative electronic diary ratings tended to outperform self-report IIP scores and RAPFA clinical ratings in larger models. These findings suggest that negative electronic diary ratings provide relatively unique information about changes in symptom distress and social functioning. Finally, results from tests of two-way interactions between group status (BPD, OthPD, and NoPD) and each predictor variable suggested that methods for measuring interpersonal dysfunction are equally predictive of symptom distress and social functioning, despite the finding that the BPD group often had the highest mean level of interpersonal dysfunction.

Strengths of the study include the mutimethod, intensive, longitudinal assessment process; the use of stringent comparator groups; and the sophisticated use of dominance analyses to determine the predictive utility of each assessment method. The use of multiple methods to assess interpersonal functioning allows us to make direct comparisons regarding the validity of each assessment modality, which informs assessment recommendations for research as well as applied settings. Including psychiatric control groups increases our confidence that self-report IIP scores and negative electronic diary ratings measure dysfunction unique to BPD.

Our study is not without limitations. In order to limit our analyses to a reasonable set of indicators, we had to summarize information contained in each assessment. Thus, subscales of these methods might be superior to the indices measured here. Additionally, the positive electronic diary rating consisted of relatively few items when compared to the negative diary ratings, which might have limited the ability of this index to distinguish BPD patients from those with other disorders and hindered its predictive utility.

Also, it is possible that the self-report IIP scores were superior to the other assessment methods because this predictor shared a common method with the outcomes (i.e., all were self-report). This study cannot disentangle the shared method variance between the IIP self-report scores and the six-month outcome variables. Future studies should replicate these findings using other types of methods as outcome variables, such as behavioral observations or chart review. This work represents preliminary steps necessary for better understanding the measurement of interpersonal functioning using several complex and novel methods, such as social interaction diaries and facial stimuli. Future work is necessary to continue the process of measurement refinement and validation using different samples.

Since interpersonal functioning is often a treatment target, especially for patients with BPD (e.g., Linehan, 1993; Clarkin et al., 1999), these findings have important implications for assessment in treatment settings. Self-report IIP scores and diary ratings of negative interpersonal experiences might be useful as screening instruments for BPD, as these two methods were able to distinguish patients with BPD from those with other types of personality pathology. Self-report IIP scores and diary ratings of negative interpersonal experiences provide unique information regarding changes in symptoms of distress and social functioning over a period of six months, indicating that these methods are probably the best predictors of short-term clinical outcomes. However, the failure to detect interactions suggests that these measures are sensitive to short-term clinical outcomes across Axis I and Axis II psychopathology, but are not specific to BPD. We chose to focus on symptom distress and interpersonal functioning as outcomes for this work because of their inherent clinical utility. However, we do not intend to suggest that interpersonal functioning might not also be predictive of other important clinical outcomes, such as suicide behaviors and emergency room visits. Longer follow-up intervals may be necessary to demonstrate changes in such low base-rate behaviors, even among psychiatric patients. Finally, when choosing an assessment instrument, costs associated with method must be weighed against its predictive value. Self-report measures of interpersonal functioning are relatively cheap to administer and score and involve low patient/participant burden. However, electronic diary methods reduce recall bias of events and provide greater flexibility compared to traditional assessment methods (c.f. Solhan et al., 2009). Piasecki and colleagues (2007) outline practical considerations, such as protocol design, for clinicians who want to use electronic diary assessments in their practice.