Development of the Self-report Version of the Zanarini Rating Scale for Borderline Personality Disorder

Abstract

The purpose of this study was to assess the psychometric properties of the self-report version of the Zanarini Rating Scale for Borderline Personality Disorder (ZAN-BPD). The measure covers a one-week time frame and each of the nine criteria for BPD is rated on a five-point anchored rating scale of 0–4. Seventy-five subjects meeting DSM-IV criteria for BPD were recruited from the community. The convergent validity of the interview and self-report versions of the ZAN-BPD was found to be high (with a median value of 0.70). In terms of reliability, the internal consistency of the nine criteria scores of the ZAN-BPD was found to be good (Cronbach’s alpha=0.84). In addition, 13 of 14 intraclass correlations for same day test-retest reliability were in the excellent range (>0.75). Finally, the sensitivity of both versions of the ZAN-BPD to change was assessed 7–10 days after they were first administered and found to be adequate (e.g., r=0.66 for total score of ZAN-BPD). Taken together, the results of this study suggest that the self-report ZAN-BPD is a promising self-report scale for the assessment of change in the severity of borderline psychopathology over time.

Borderline personality disorder (BPD) is a common psychiatric disorder (Lenzenweger, Lane, Loranger, & Kessler, 2007; Swartz, Blazer, George, & Winfield, 1990; Trull, Jahng, Tomko, Wood, & Sher, 2010). It is also a serious psychiatric disorder (Bender et al., 2001; Skodol et al., 2002), often involving impaired psychosocial functioning and high rates of mental health utilization. Five semistructured interviews based on the DSM criteria for personality disorders, which were first developed over 20 years ago, have proven to be reliable in assessing the presence of BPD and other personality disorders (Zimmerman, 1994). However, no assessment instrument specifically designed as a change measure of the severity of borderline psychopathology was available until 10 years ago. Prior to that time, most treatment studies following borderline subjects over time used a complex battery of instruments that included measures that were either designed to assess the presence of an axis I disorder, most commonly major depression, or that focused on one or more general symptom areas characteristic of BPD, such as hostility or impulsivity.

We developed the interview-based Zanarini Rating Scale for Borderline Personality Disorder (ZAN-BPD) to fill that gap—a gap that was particularly acute in treatment studies (Zanarini, Vujanovic, Parachini, Boulanger, & Frankenburg, 2003). We found that this interview had good convergent and discriminant validity in a sample of 200 non-psychotic men and women with a history of psychiatric treatment—139 (69.5%) who met criteria for BPD and 61 subjects (30.5%) who did not (but did meet lifetime criteria for an axis I disorder). We also found that it had good inter-rater and test-retest reliability. In addition, we found that it was sensitive to change (over a one-week period).

This ZAN-BPD quickly became the accepted standard for use in psychotherapy and medication outcome trials involving patients with BPD (Bateman & Fonagy, 2008; Black et al., 2014; Blum et al., 2008; McMain et al., 2009). Despite its wide use in treatment studies, the ZAN-BPD’s use in clinical practice was limited by the training required to use it competently and the clinician time needed to administer it.

For these reasons, we more recently developed a self-report version of the ZAN-BPD (Zanarini & Frankenburg, 2008). This paper details the development of this measure and its psychometric properties.

Development of the Self-report ZAN-BPD

This version of the ZAN-BPD is organized and scored much as the interview-based version. It has a five-level set of anchored rating points for each of the nine criteria for BPD found in the DSM-IV. In this system, 0=no symptoms, 1=mild symptoms, 2=moderate symptoms, 3=serious symptoms, and 4=severe symptoms. Table 1, which details the anchored-rating points for intense, unstable relationships, provides an example of this.

Table 1.

Self-Report ZAN-BPD Section Pertaining to Intense, Unstable Relationships

DURING THE PAST WEEK:
0	I have not had any conflict in my close relationships.
1	I have occasionally argued with someone I am close to or threatened to end our relationship.
2	I have often argued with someone I am close to or repeatedly threatened to end our relationship.
3	I have often had intense arguments with someone I am close to or impulsively broken up with someone important to me.
4	I have had intense arguments every day with someone I am close to or impulsively broken up a number of times with someone important to me.

In addition to the criterion-based scales, the interview has four sector scores reflecting the four core areas of borderline psychopathology (Zanarini, Gunderson, Frankenburg, & Chauncey, 1990): affective, cognitive, impulsive, and interpersonal symptoms. There are three affective symptoms in the ZAN-BPD (with a sector score ranging from 0–12): inappropriate anger/frequent angry acts, mood instability, and chronic feelings of emptiness. There are two cognitive symptoms (with a sector score ranging from 0–8): stress-related paranoia/dissociation and severe identity disturbance. (Identity disturbance was placed in the cognitive realm because it is based on a series of false beliefs or overvalued ideas, such as that one is a bad person.) There are also two impulsive symptoms (with a sector score ranging from 0–8): self-mutilative/suicidal efforts and at least two other forms of impulsivity. Finally, there are two symptoms in the interpersonal realm of BPD (with a sector score ranging from 0–8): frantic efforts of avoid abandonment and intense, unstable relationships. The four sector scores sum to provide a total score of borderline psychopathology, which ranges from 0–36.

Method

Subjects were recruited by an internet ad on craigslist. The ad, which specified that we were looking for men and women between the ages of 18–60, asked: Are you extremely moody? Do you often feel distrustful of others? Do you frequently feel out of control? Are your relationships painful and difficult?

The inclusion criteria for the study were meeting the DSM-IV criteria for BPD and having a history of psychiatric treatment. The exclusion criteria for the study were: 1) ever met the DSM-IV criteria for schizophrenia, schizoaffective disorder, or bipolar I disorder, 2) met criteria for a substance use disorder in the preceding month, and 3) had an estimated IQ of 70 or lower. These exclusion criteria follow standard practice in borderline research of excluding subjects whose axis I state (i.e., psychosis, mania, intoxication/withdrawal) or cognitive impairment is likely to interfere with an assessment of their more enduring personality traits or symptoms. Potential subjects were prescreened by telephone to determine if they met presumptive criteria for DSM-IV BPD as assessed by the McLean Screening Instrument for Borderline Personality Disorder (MSI-BPD) (Zanarini, Vujanovic, Parachini, Boulanger, Frankenburg, & Hennen, 2003) or any of our exclusion criteria. The MSI-BPD is a 10-item self-report measure with a cutoff of seven or higher indicating good sensitivity (.81) and specificity (.85) for the borderline diagnosis.

Those who were not excluded were invited to participate in a face-to-face-interview. After written informed consent was obtained, four semistructured interviews of demonstrated reliability were administered to each subject: 1) the Background Information Schedule (BIS), which assesses psychosocial functioning and treatment history (Zanarini, Frankenburg, Khera, & Bleichmar, 2001), 2) the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID I) (Williams et al., 1992), 3) the BPD module of the Diagnostic Interview for DSM-IV Personality Disorders (DIPD-IV) (Zanarini et al., 2000), and 4) the interview version of the Zanarini Rating Scale for Borderline Personality Disorder or ZAN-BPD (Zanarini, Vujanovic, Parachini, Boulanger, & Frankenburg, 2003), inquiring about symptom severity during the past week. The self-report version of the ZAN-BPD (Zanarini & Frankenburg, 2008) was also administered at this time.

Three separate sub-studies were then undertaken to assess the psychometric properties of the self-report ZAN-BPD. The first assessed its convergent validity, the second assessed the internal consistency/same-day test-retest reliability of this measure, and third assessed its one-week sensitivity to change.

Convergent validity was assessed using Spearman’s rho to determine the correlations between the continuous scores of the two versions of the ZAN-BPD. The internal consistency of the ZAN-BPD was assessed using Cronbach’s alpha. Test-retest reliability was assessed using intraclass correlation coefficients or ICCs. Spearman’s rho was also used to assess sensitivity to change, which determined the relationship between difference scores of the two versions of the ZAN-BPD (one week post baseline value minus baseline value).

Results

In all, 120 subjects were screened by telephone. Fifteen were excluded from further participation because they did not meet screening criteria for BPD. Nine were excluded after reporting a current substance use disorder, while eight were excluded after reporting that they had a long-standing clinical diagnosis of a psychotic or bipolar I disorder. Seven were excluded for not having a history of psychiatric treatment, and four of the screened subjects reported that they were unable and/or unwilling to schedule the interview for the study. The 77 remaining subjects were interviewed in person. Two were excluded because they did not meet study criteria for BPD.

Table 2 details the demographic characteristics, treatment histories, and lifetime axis I disorders of the remaining 75 subjects. All 75 subjects participated in the three assessment sessions. The majority were female and white. On average, they were in their late 20s, came from a middle class background, had two years of post-high school education, and were functioning in the fair range of the GAF. Almost 90% had been in individual therapy at some point in their life, two-thirds had taken psychotropic medications, and about a third had been hospitalized for psychiatric reasons. Almost 90% met lifetime criteria for a mood disorder and/or an anxiety disorder, and about half met lifetime criteria for a substance use disorder and/or an eating disorder.

Table 2.

Demographic Characteristics, Lifetime Treatment History, and Lifetime Axis I Psychopathology of Sample (N=75)

Demographic Characteristics
% Female	80.0
% White	85.0
Mean Age	28.3(SD 10.8)
Mean SES	2.7(SD 1.3)
Mean Years of Education	14.6(SD 1.9)
Mean GAF	53.3(SD 7.8)
Lifetime Treatment
% Individual Therapy	89.0
% Psychotropic Medication	65.0
% Psychiatric Hospitalization	35.0
Lifetime Axis I Disorders
% Mood Disorder	89.0
% Substance Use Disorder	48.0
% Anxiety Disorder	89.0
% Eating Disorder	55.0

Table 3 details the convergent validity of the self-report and interview versions of the ZAN-BPD. As can be seen, the convergent validity of these two measures, both of which assess symptoms in the past week, was highly significant. It was also high for the total ZAN-BPD score, the four sector scores, and for the majority of the criterion-based scales, with only one of the criterion-based scales (stress-related paranoia/dissociation) assessed being in the poor range (r<0.40).

Table 3.

Convergent Validity of Interview and Self-report Versions of ZAN-BPD (N=75)

	Spearman’s Rho	P-level
Affect Sector Scores	.78	<0.001
Chronic Anger/Frequent Angry Acts	.59	<0.001
Affective Instability	.57	<0.001
Chronic Emptiness	.82	<0.001
Cognitive Sector Scores	.55	<0.001
Stress-Related Paranoia/Dissociation	.35	0.002
Serious Identity Disturbance	.66	<0.001
Impulsivity Sector Scores	.69	<0.001
Self-destructive Efforts	.80	<0.001
Other Impulsivity	.70	<0.001
Interpersonal Sector Scores	.80	<0.001
Frantic Efforts to Avoid Abandonment	.71	<0.001
Stormy Relationships	.70	<0.001
Total ZAN-BPD Scores	.78	<0.001

The internal consistency of the nine criteria scores of the ZAN-BPD was found to be good (Cronbach’s alpha=0.84). This means that scores on the nine scales were related to one another. This also suggests that these scores appear to be assessing the same construct—DSM-IV BPD.

Table 4 details the test-retest reliability of the self-report version of the ZAN-BPD. Using the criteria of Fleiss (1981), correlations below .40 are considered to be poor, between .40–.75 to be fair-good, and higher than .75 to be excellent. Using these standards, all test-retest reliabilities were in the excellent range except for stress-related dissociation/paranoia—which was closer to the good range.

Table 4.

Test-retest Reliability of Self-report ZAN-BPD

	Test-retest Reliability(N=75)
	ICC	P-level
Affect Sector Score	0.89	<0.001
Chronic Anger/Frequent Angry Acts	0.83	<0.001
Affective Instability	0.87	<0.001
Chronic Emptiness	0.75	<0.001
Cognitive Sector Score	0.83	<0.001
Stress-Related Paranoia/Dissociation	0.65	<0.001
Serious Identity Disturbance	0.90	<0.001
Impulsivity Sector Score	0.82	<0.001
Self-destructive Efforts	0.76	<0.001
Other Impulsivity	0.83	<0.001
Interpersonal Sector Score	0.88	<0.001
Frantic Efforts to Avoid Abandonment	0.87	<0.001
Stormy Relationships	0.82	<0.001
Total ZAN-BPD Score	0.91	<0.001

The ability to detect change reliably is one of the most important properties of a continuous measure of change of severity of psychopathology. Of the 75 subjects who were reinterviewed 7–10 days after their initial interview, only 7 or 9.3% had exactly the same score on the self-report version of the ZAN-BPD at both administrations. In contrast, 52% (n=39) of these subjects were less symptomatic at the time of the second administration and 38.7% (n=29) were more symptomatic. Of the 90.7% (n=68) of subjects who reported a change on the ZAN-BPD at the second administration, 42.6% (n=29) were judged to have experienced a 1–2 point difference, 32.4% (n=22) a 3–4 point difference, and 25% (n=17) a 5–15 point difference.

Table 5 shows data related to another way of assessing change. It examines the correlations between the difference scores of the interview-based and self-report versions of the ZAN-BPD. As can be seen, all of the correlations between the difference scores (one week post baseline value minus baseline value) of these measures were statistically significant. This shows that the ratings on these two measures of the severity of borderline psychopathology were changing in similar directions over time.

Table 5.

Sensitivity to Change Between Interview and Self-Report Versions of ZAN-BPD: One Week Interval (N=75)

	Spearman’s Rho	P-level
ZAN-BPD Affect Sector Scores	0.60	<0.001
ZAN-BPD Cognition Sector Scores	0.30	0.01
ZAN-BPD Impulsivity Sector Scores	0.50	<0.001
ZAN-BPD Interpersonal Sector Scores	0.35	0.002
ZAN-BPD Total Scores	0.66	<0.001

Discussion

Three important issues need to be addressed in the development of a new continuous measure of psychopathology that will be administered multiple times during the course of a treatment study or during the course of a psychosocial treatment and/or pharmacotherapy. These issues are validity, reliability, and sensitivity to change. In terms of convergent validity, most of the scores on the various scales of the self-report ZAN-BPD were highly correlated with those obtained from the interview version of the ZAN-BPD. As noted above, one symptom had poor convergent validity (stress-related paranoia/dissociation). This outcome is probably due, at least in part, to the slightly different ways that these symptoms were described in the two versions of the ZAN-BPD. For example, the interview version had examples that were related to BPD, while the self-report version did not due to space constraints. It may also be that the other eight symptoms of BPD resonate more with those with BPD and so, they are rated more similarly by mental health professionals and subjects themselves. However, we have revised the wording on this criterion’s anchor points on the self-report version of the ZAN-BPD and will test its convergent validity in the months to come.

Reliability is also an important concept in the development of any measure of psychopathology. The self-report ZAN-BPD was found to have high internal consistency, an important component of reliability. In addition, high levels of test-retest reliability were obtained in the current study for the self-report ZAN-BPD.

The interview-based ZAN-BPD was developed for use in treatment studies involving borderline patients. The self-report ZAN-BPD was developed with two uses in mind. The first would be an inexpensive way to assess change in the severity of borderline psychopathology in treatment trials—whether of a psychosocial or pharmacologic intervention. The second would be for clinicians to determine the initial level of severity of a patient and/or to track the severity of borderline psychopathology over time. With minimal training, primary care providers as well as mental health professionals could use this measure in their work with borderline patients. However, its usefulness will depend on whether it can accurately assess symptomatic change over time. The results of this study suggest that it has this capability. More specifically, difference scores on the self-report ZAN-BPD were significantly correlated with difference scores on the interview-based ZAN-BPD. And the levels of sensitivity to change were slightly higher than those found in the study of the psychometric properties of the interview version of the ZAN-BPD (Zanarini, Vujanovic, Parachini, Boulanger, & Frankenburg, 2003).

After we developed the self-report version of the ZAN-BPD, a paper was published describing the psychometric properties of another self-report measure that assesses the severity of DSM-IV borderline psychopathology over time (Pfohl et al., 2009). This measure—the Borderline Evaluation of Severity over Time or BEST—was found to have high internal consistency, moderate test-retest reliability, and high discriminant reliability. It was also found to be sensitive to change in a treatment study, revealing declining scores as the study progressed. However, the BEST does not have anchored rating points as the self-report ZAN-BPD does. Rather, subjects rate the degree to which each symptom has interfered with their life in the past week. More specifically, this scoring system is: 1=none/slight, 2=mild, 3=moderate, 4=severe, and 5=extreme. In addition, it assesses anger with two items, abandonment concerns with two items, and the other seven criteria for BPD with one item. Suicidal ideation is also assessed with one item despite not being a DSM-IV criterion for BPD. And the greatest difference between the two self-report measures in describing a criterion is that found for intense, unstable relationships. The self-report ZAN-BPD, as can be seen in Table 1, emphasizes the stormy quality of these relationships, while the BEST focuses on the shifts between idealization and devaluation, with no mention of turbulence.

It is a sign of the growth of the field that we now have two self-report measures of the severity of DSM-IV borderline psychopathology that can be reliably used in clinical settings and treatment studies. It may be useful in these treatment studies to use both measures to assess the same symptom cluster in somewhat different but equally useful ways.

A limitation of the current study is that we did not assess the full array of axis II disorders. This was also true of our study of the psychometric properties of the interview-based ZAN-BPD (Zanarini, Vujanovic, Parachini, Boulanger, & Frankenburg, 2003). We adopted this design in both studies to lessen subject burden. Another limitation is that we did not assess discriminant validity of the self-report version of the ZAN-BPD as we had for the interview version of this measure—which was found to be highly significant. This design feature of this study was primarily driven by the knowledge that this measure would be used in studies or treatment settings where all of the subjects or patients were thought to meet criteria for BPD. Another limitation is that all of the subjects in this study had a treatment history. Whether the self-report ZAN-BPD would perform as well in a sample without such a history is an open question. Finally, our subjects tended to have about two years of post-high school education and, on average, to come from a middle middle class background. Whether the self-report ZAN-BPD would perform as well in a sample with less education and from a lower socioeconomic background is also an open question.

Further research is needed to assess the psychometric properties of the self-report ZAN-BPD, particularly its sensitivity to change. For now, this version of the ZAN-BPD seems well positioned to join the BEST in filling the void in BPD instrumentation created by the absence of a self-report measure of DSM-IV/5 borderline psychopathology and its stability or change over time.