Evidence for the criterion validity and clinical utility of the Pathological Narcissism Inventory

Abstract

In this study we evaluated aspects of criterion validity and clinical utility of the grandiosity and vulnerability components of the Pathological Narcissism Inventory (PNI) using two undergraduate samples (Ns = 299, 500). Criterion validity was assessed by evaluating the correlations of narcissistic grandiosity and narcissistic vulnerability with established indices of normal personality traits, psychopathology and clinical concerns, and pathological personality traits. Overall, the pattern of correlations supported the convergent and discriminate validity of grandiose and vulnerable conceptualizations of pathological narcissism as measured by the PNI. Clinical utility was assessed by evaluating the extent to which clinicians without specific training in pathological narcissism as well as clinicians with expertise in pathological narcissism could accurately predict the correlates of PNI grandiosity and vulnerability with normal and pathological personality traits and psychopathology. The r_contrast-cv coefficient (Westen & Rosenthal, 2003) provided a global index of accuracy in clinicians’ predictions that was more fully elaborated by examining systematic discrepancies across groups. Overall, novice and expert clinicians were generally able to predict criterion correlations, with some exceptions (e.g., counter to predictions, pathological narcissism was negatively associated with treatment resistance). These results provide further evidence regarding the validity and utility of the narcissistic grandiosity and narcissistic vulnerability constructs as measured by the PNI.

The assessment of narcissism has been complicated by inconsistent definitions across areas of clinical psychology, social/personality psychology, and psychiatry (Cain, Pincus, & Ansell, 2008). For instance, some elements of the most commonly employed measure of narcissism in social/personality psychology, the Narcissistic Personality Inventory (NPI; Raskin & Terry, 1988), reliably show weak or even negative correlations with indicators of dysfunction (e.g., Rhodewalt & Morf, 1995; Watson, 2005), suggesting potential limitations of the NPI for clinical conceptualizations and assessment of narcissistic pathology. Further, both the NPI and the current Diagnostic and Statistic Manual of Mental Disorders (DSM-IV-TR; American Psychiatric Association, 2000), tend to focus primarily on grandiose aspects of personality pathology, although vulnerability has clinically been considered a prominent feature of narcissism since the construct’s inception (see Cain et al., 2008). The Personality and Personality Disorder workgroup for the upcoming DSM-5 has maintained the relatively exclusive focus on grandiosity in representing Narcissist Personality Disorder (NPD). Specifically, the diagnosis of NPD will be made when individuals meet general criteria for personality disorder and have specific elevations on two maladaptive traits: grandiosity and attention seeking.

The Pathological Narcissism Inventory

To more fully account for the clinically observed phenotypic variation in pathological narcissism, Pincus and colleagues (2009) developed the Pathological Narcissism Inventory (PNI), a 52-item self-report measure focusing on maladaptive expressions of narcissism. The PNI consists of seven subscales that serve as markers for two higher order factors reflecting grandiose and vulnerable aspects of pathological narcissism (Wright, Lukowitsky, Pincus, & Conroy, 2010). Narcissistic grandiosity is conceptualized as a strong motivation to self-enhance, a tendency to repress negative aspects of the self and to view oneself as superior, and to behave in an exploitative, exhibitionistic, and unempathic style interpersonally. Narcissistic vulnerability is conceptualized as a fragile self-concept with limited capacities for self- and affect- regulation, associated with a tendency to experience shame and/rage when personal needs are not met, sensitivity to perceived threats to self-esteem, and an avoidance of relationships for fear of not being admired (Pincus et al., 2009).

Accumulating evidence supports the reliability and validity of the PNI (Pincus, in press). For instance, Pincus and colleagues (2009) showed in the initial validation study that that the theoretical structure of the PNI fit data from a large sample of undergraduates and that the PNI scales had acceptable internal consistency coefficients (see also Wright et al., 2010) and that the PNI total score had expected correlation patterns with measures of self-esteem (r = −.37), empathy (r = −.14), shame (r = .55), primitive defenses (r = .60), identity diffusion (r = .62), impaired reality testing (r = .47), aggression (r = .36), and low moral values (r = .45) in student samples. Within a clinical sample (Pincus et al., 2009), the PNI was predictive of treatment-relevant variables, such as medication use (r = −.35) and suicidal behavior (r = .43).

Several recent studies have extended the nomological net of the PNI. Tritt et al. (2010) replicated the structural validity of the PNI and showed that its scales relate systematically and predictably to depressive temperament. Specifically, in regression models PNI vulnerability related strongly to depressive (β = .61), anxious (β = .53), angry (β = .38), aesthenic (β = .38), and cyclothymic (β = .55) temperament, whereas PNI grandiosity was not related to these variables in analyses controlling for narcissistic vulnerability. Conversely, grandiosity related positively (β = .56) and vulnerability related negatively (β = −.35) to hyperthymic (e.g., energetic/adaptive) temperament. Overall, these results were interpreted as relating to the core affective content of the PNI scales: “Vulnerability items tend to reflect negative affects when a narcissistic need is not met, whereas grandiosity items tend to reflect positive affects associated with narcissistic self-enhancement strategies” (Tritt et al., p. 282). Miller et al. (2010) showed that the vulnerable PNI scales relate to the Hypersensitive Narcissism Scale (Hendin & Cheek, 1997), another measure of vulnerable narcissism (range r = .34 - .55) as well as indicators of borderline personality disorder (range r = .28 - .48), which overlaps theoretically with vulnerable aspects of pathological narcissism (McWilliams, 2011). Maxwell et al. (2011) showed that the PNI correlated significantly with a measure of DSM-IV narcissistic PD (r = .35) and self-esteem (r = −.34). Ackerman et al. (2010) reported a relation between the PNI total score and psychological entitlement (r = .28). Ziegler-Hill et al. (2010) reported systematic relations between PNI scales and early maladaptive schemas. In particular, in regression models the PNI grandiose composite was predicted by mistrust (β = .31), insufficient self-control (β = −.14), entitlement (β = .12), self-sacrifice (β = .18), and unrelenting standards (β = .13) whereas the PNI vulnerable composite was predicted by mistrust (β = .20), subjugation (β = .19), dependence (β = −.16), abandonment (β = .23), and entitlement (β = .13).

Taken together, this initial research attests to the validity of the PNI as a measure of pathological narcissism, but this evidence is also preliminary in that a wider net of criterion variables would be desirable. Researchers have tended to focus on variables that are theoretically linked to narcissism, resulting in limited information concerning how narcissistic grandiosity and vulnerability relate to more general measures of normal and abnormal personality and psychopathology. Correlating the grandiose and vulnerable composites of the PNI with such variables would permit tests of the measure’s discriminant validity, expand the nomological net of pathological narcissism and provide further evidence for its construct validity.

Investigating the Clinical Utility of the PNI

Traditionally, researchers have placed greater emphasis on evaluating the reliability and validity of newly developed measures than on ascertaining the clinical utility of these measures. Clinical utility is a broad element of psychological assessment that encompasses clinicians’ ability to use data in a way that facilitates their understanding of their clients, predict clients’ future behaviors, and engage with the scientific community. We define clinical utility more narrowly throughout this paper as clinicians’ ability to articulate expectations regarding patterns of convergent and discriminant relations of the PNI with various criterion measures. Ultimately, a psychological instrument is made more or less useful based on clinicians’ clear understanding of the underlying constructs it assesses and the wider array of constructs and functioning to which it relates. Thus, it is important to evaluate how well clinicians’ expectations match empirical patterns of convergent and discriminant associations. This point is increasingly salient with the impending publication of DSM-5, particularly as some have suggested more could be done to enhance the clinical utility of the DSM-5 personality disorders. First (2010) specifically describes four types of clinical utility that should be considered in revising the DSM, stating that the manual should ultimately: (1) facilitate communication, (2) aid in selecting effective interventions, (3) assist clinicians in predicting course, prognosis and future needs, and (4) differentiate disorder from non-disorder. Our study is somewhat broader in that it focuses on personality pathology in general rather than the DSM-5 per se. Within this framework, however, it focuses on how well clinician’s can use a measure of personality pathology to make clinically relevant predictions, and in this way overlaps with First’s third type of clinical utility. In examining clinical utility in this way, it is informative to quantify the clinical utility of measures using approaches that move beyond simple qualitative impressions. Westen and Rosenthal (2003) discuss a method that provides a single number reflecting how well the empirical pattern of correlations for a measure and a set of criterion-variables match with a priori expectations. Using this procedure, point-estimates are made about how a given scale will relate to an array of criterion variables. The consistency of these estimates with the observed correlations is then correlated to evaluate the accuracy of these a priori predictions. This procedure results in a single coefficient (r_contrast-cv)¹ that provides an overall and readily interpretable index of accuracy that can be compared across varying raters, measures, and subsequent studies. Quantitative indices like the r_contrast-cv are an improvement over approaches in which researchers largely “rely on subjective estimates and eyeballing of complex patterns” (Westen & Rosenthal, 2005, p. 409).

Despite the intuitive appeal and apparent utility of r_contrast-cv and related approaches like alerting correlations, concerns have been raised about the uses and interpretations of these coefficients (Westen & Rosenthal, 2003). Specifically, Smith (2005) notes that “because both indices [r_contrast-cv and r_alerting-cv.] quantify predictive accuracy with a single number, their method does not aid in the formal identification of which correlations fail to support…construct validity” (p. 404). Given this limitation, in the current study we further evaluate discrepancies between obtained and predicted correlations at the scale-level.

A second criticism of r_contrast-cv is the degree to which it is an accurate reflection of construct validity (Smith, 2005). To be clear, the r_contrast-cv metric provides a test of the accuracy of predictions more so than an evaluation of the actual validity of an instrument. However, we believe that quantifying the agreement between empirical patterns and a priori predictions can provide important insights about how constructs are understood by clinicians. An alternative methodology that has not been used in previous research is to evaluate clinicians’ predictions, rather than expert researchers’ predictions, of how a measure of interest will correlate with other constructs. Our approach can therefore provide a, global index of clinical utility (as we have defined it in this paper) and side-step the thorny issue of whether such a coefficient actually quantifies construct validity, per se. Accordingly, we computed r_contrast-cv using clinician predictions and consider the coefficients in our study to represent a quantitative estimate of one important aspect of the clinical utility of PNI grandiosity and vulnerability rather than construct validity per se or clinical utility in general. In estimating clinical utility as it pertains to the ability of clinicians to understand the criterion validity of assessment measures, it is important to discern how well both experienced and novice clinicians perform in this task. We therefore examined the accuracy of predictions made by clinicians without particular expertise in pathological narcissism as a proxy of the global utility of the PNI for clinical psychologists as well as the accuracy of predictions made by experts in pathological narcissism as a proxy for the ceiling of potential clinical utility of the PNI as it is currently understood.

Study Aims

We have two primary aims in this study. The first is to further evaluate the criterion validity of narcissistic grandiosity and vulnerability by establishing their patterns of convergence and divergence against a variety of personality and psychopathology measures. Accordingly, we correlated the PNI grandiosity and vulnerability composites with measures of normal personality, abnormal personality, and psychopathology in two samples. The second aim is to evaluate the clinical utility of the PNI by quantifying the agreement between clinicians’ predictions and actual empirical correlations for the suite of criterion variables we evaluated. We examined the consistency of predictions across clinician raters with and without expertise in pathological narcissism and quantified the accuracy of these predictions using the r_contrast-cv coefficient (Westen & Rosenthal, 2003). Given criticism regarding the possibility that a single coefficient might obscure critical discrepancies at the scale-level (Smith, 2005), we also examined differences between obtained and predicted correlations for each scale. This study is novel in that it evaluates clinical intuitions about the nomological network of narcissistic grandiosity and vulnerability, and thus evaluates both the empirical correlations of the PNI with criterion-measures as well as the predictability of the obtained patterns of convergence and divergence using clinician ratings.

Method

We utilized data from two samples of undergraduate students attending large, public universities to compute criterion correlations. Participants in both samples completed the PNI (all scale alphas > .70).

Data from the first sample were used in a previous study which did not report PNI criterion validity correlations of interest in the present study (Hopwood et al., 2011). This sample contained 299 students, most of whom (> 80%) were white and 194 (65%) of whom were women. In addition to the PNI, they completed the Big Five Inventory (BFI; John, Naumann, & Soto, 2008) and the Personality Assessment Inventory (PAI; Morey, 1991). The BFI is a 44-item self-report measure of the big five domains neuroticism (sample Cronbach’s α = .81), extraversion (α = .84), openness (α = .79), agreeableness (α = .82), and conscientiousness (α = .78). The PAI is a 344-item self-report measure of personality, psychopathology and other clinically relevant constructs. The PAI clinical and treatment consideration scales were used as criteria. All alphas across the 14 PAI scales used in this study were > .70.

The second sample included 500 students who were primarily white (> 80%) and balanced on gender (251 male, 50.2%). In addition to the PNI, participants completed the Schedule for Nonadaptive and Adaptive Personality, 2^nd Edition (SNAP-2; Clark, Simms, Wu, & Casillas, in press). The SNAP-2 has three normal temperament scales (negative temperament, positive temperament, and disinhibition) and 12 pathological trait scales (mistrust, manipulativeness, aggression, self-harm, eccentric perceptions, dependency, exhibitionism, entitlement, detachment, impulsivity, propriety, and workaholism). All alphas for the SNAP-2 scales were > .70.

Procedures and Analyses

We computed bivariate correlations between the PNI grandiosity and vulnerability scale scores and each of the criterion variables in both samples. Because of the large number of correlations, we provided benchmarks for statistical significance at a conservative p < .01 value.

We also obtained predictions for these correlations from two sets of clinician raters: “non-experts” and “experts” in pathological narcissism. Five advanced graduate student clinicians (in their fourth or fifth year of graduate training) without specific expertise in pathological narcissism or prior familiarity with the PNI served as our “non-expert” clinician raters. These clinicians were provided brief descriptions of narcissistic grandiosity and vulnerability as measured by the PNI as well as brief descriptions of each criterion scale. Clinicians were told that the high scores on the PNI grandiosity indicate that an individual would be “strongly motivated to self-enhance in most contexts, have a tendency to repress negative aspects of self, feel entitled, have an inflated self-image, engage in fantasies of superiority, and to be interpersonally exploitative, exhibitionistic, and to lack empathy.” They were told that high scorers on the PNI vulnerability scale would likely have “fragile self and affect regulatory capacities, a tendency to experience shame or rage when narcissistic needs are disappointed, to be sensitive to perceived threats to self-esteem, to avoid showing others faults or concerns, and to shy away from relationships for fear of not receiving desired admiration and recognition.” Similarly descriptions were provided for the scales of the criterion measures. Brief descriptions of the BFI scales were written by study authors and descriptions of the PAI and SNAP scales were each obtained from book chapters describing the measures (Morey & Hopwood, 2008; Simms & Clark, 2006, respectively).

Each clinician independently provided a point estimate, to one decimal place, of the correlation between the grandiosity and vulnerability composites and each criterion scale. Four clinical experts on pathological narcissism² who have previously conducted research using the PNI, though not in conjunction with the specific criterion measures assessed in the present study, independently provided “expert” ratings by also supplying a point estimate of the correlation between the grandiosity and vulnerability composites and each criterion scale. These raters were given the same brief descriptions of scales on the criterion measures as non-expert raters were given.

The global accuracy of these predictions across raters can only be meaningfully interpreted to the extent that clinicians tended to agree in their ratings. Therefore, we first computed inter-rater reliability for non-expert and expert ratings using two-way random effects, absolute agreement intraclass correlations (ICCs; Bombel, Mihura, & Meyer, 2009; McGraw & Wong, 1996). These values, presented in Table 1, were generally acceptable across both non-expert (median ICC = .72) and expert (median ICC = .87) clinician raters. Accordingly, we averaged ratings across each set of clinician raters and computed a global indicator of clinical utility, r_contrast-cv (Westen & Rosenthal, 2003), for PNI grandiosity and vulnerability in each sample. This value is based in contrast analysis as commonly employed in an ANOVA framework and therefore provides a single overall index of the extent to which the experts correctly predicted criterion-related validity correlations. This index also takes into account the intercorrelations of the criterion measures with the variable being considered and the absolute values of the observed correlations (see Westen & Rosenthal, 2003, for computational details for deriving this coefficient). This overall index can be interpreted as a correlation coefficient and tested for statistical significance against the null hypothesis that r = .00.

Table 1.

Intraclass correlation (ICC) and rcontrast-cv coefficients for predicted PNI grandiosity and vulnerability correlations with measures of normal traits (BFI), psychopathology and clinical concerns (PAI), and pathological traits (SNAP-2) for both non-expert and expert clinician raters.

	ICC	rcontrast-cv
BFI
Grandiosity
Non-expert	.69	.49
Expert	.98	.41
Vulnerability
Non-expert	.73	.64
Expert	.98	.57
PAI
Grandiosity
Non-expert	.75	.74
Expert	.51	.77
Vulnerability
Non-expert	.67	.61
Expert	.86	.66
SNAP-2
Grandiosity
Non-expert	.62	.78
Expert	.87	.70
Vulnerability
Non-expert	.58	.64
Expert	.87	.77

Note: all ICC and r_contrast-cv coefficients are statistically significant (p < .01).

BFI = Big Five Inventory; PAI = Personality Assessment Inventory; SNAP-2 = Schedule for Nonadaptive and Adaptive Personality, 2nd Edition.

Results

PNI criterion validity

Correlations between the criterion scales and the PNI grandiosity and vulnerability composites are presented in Tables 2 and 3, respectively. These composites correlated .51 (p < .05) in the first sample and .55 (p < .05) in sample 2, as would be expected given that they are presumed to arise from the core construct of pathological narcissism. Thus, it is not surprising that grandiosity and vulnerability showed similar patterns of convergence and discrimination with some criterion variables. Specifically, both composites positively related to neuroticism and negative temperament, although vulnerability correlated more strongly with these traits, indicating a general propensity for individuals high in ratings of pathological narcissism to experience negative emotions. This was further evidenced in their association with symptoms of anxiety related disorders. Grandiosity and vulnerability were also both positively associated with the tendency to behave without regard for consequences or social norms (SNAP-2 disinhibition), to manipulate others and derive pleasure from capitalizing on others’ weaknesses (SNAP-2 manipulative), to value reputation and appearance over personal comfort (SNAP-2 propriety), to invest significant time in goal-directed activities (SNAP-2 workaholism), and to possess a unique world-view relative to others (SNAP-2 eccentric perceptions). Neither grandiosity nor vulnerability was associated with BFI openness or with substance abuse.

Table 2.

Observed and predicted correlations between PNI grandiosity and criterion scales.

	Observed r	Non-expert predicted r	Non-expert difference	Expert predicted r	Expert difference
BFI
Neuroticism	.23*	.06	.17	−.03	.26
Extraversion	−.01	.50	−.51	.25	−.26
Openness	.05	−.14	.19	.18	−.13
Agreeableness	−.09	−.18	.09	−.28	.19
Conscientiousness	.16	−.10	.26	.00	.16
PAI
Somatic Complaints	.09	.06	.03	−.03	.12
Anxiety	.16	.00	.16	.01	.15
Anxiety Disorders	.25*	−.04	.29	.01	.24
Depression	.15	.16	−.01	−.12	.27
Mania	.32*	.38	−.06	.27	.05
Paranoia	.22*	.22	.00	.04	.18
Schizophrenia	.16	.20	−.04	−.16	.32
Borderline	.24*	.30	−.06	−.04	.28
Antisocial	.19*	.52	−.33	.30	−.11
Alcohol	.04	.32	−.28	.15	−.11
Drug	.03	.34	−.31	.15	−.12
Aggression	.10	.44	−.34	.25	−.15
Suicide	.06	.04	.02	.08	−.02
Treatment Rejection	−.24*	.56	−.80	.60	−.84
SNAP-2
Negative Temperament	.25*	.04	.21	.10	.15
Positive Temperament	.23*	.16	.07	.23	.00
Disinhibition	.23*	.50	−.27	.25	−.02
Mistrust	.28*	.48	−.20	.15	.13
Manipulative	.31*	.56	−.25	.40	−.09
Aggression	.19*	.40	−.21	.20	−.01
Self-harm	.04	.04	.00	−.08	.12
Eccentric perceptions	.34*	.30	.04	−.13	.47
Dependency	.02	−.10	.12	−.20	.22
Exhibitionism	.37*	.40	−.03	.33	.04
Entitlement	.36*	.64	−.28	.35	.01
Detachment	−.07	.20	−.27	−.20	.13
Impulsivity	.11	.40	−.29	.20	−.09
Propriety	.22*	.14	.08	.00	.22
Workaholism	.18*	.12	.06	.08	.10

^*Indicates p < .01.

BFI = Big Five Inventory; PAI = Personality Assessment Inventory; SNAP -2 = Schedule for Nonadaptive and Adaptive Personality, 2nd Edition.

Note: r_alerting-cv can be computed by correlating the z-transformed observed correlations with the z-transformed [expert or non-expert] predicted correlations (Westen & Rosenthal, 2003).

Table 3.

Observed and predicted correlations between PNI vulnerability and criterion scales.

	Observed r	Non-expert predicted r	Non-expert difference	Expert predicted r	Expert difference
BFI
Neuroticism	.29*	.46	−.17	.45	−.16
Extraversion	−.19*	−.10	−.09	−.20	.01
Openness	−.01	−.02	.01	−.03	.02
Agreeableness	−.15	.04	−.19	−.30	.15
Conscientiousness	−.03	.02	−.05	−.15	.12
PAI
Somatic Complaints	.14	.22	−.08	.11	.03
Anxiety	.26*	.30	−.04	.33	−.07
Anxiety Disorders	.26*	.34	−.08	.22	.04
Depression	.34*	.38	−.04	.36	−.02
Mania	.11	−.02	.13	−.02	.13
Paranoia	.33*	.30	.03	.24	.09
Schizophrenia	.25*	.08	.17	.17	.08
Borderline	.33*	.52	−.19	.43	−.10
Antisocial	.11	.04	.07	.14	−.03
Alcohol	.08	.32	−.24	.20	−.12
Drug	.09	.34	−.25	.20	−.11
Aggression	.04	.18	−.14	.27	−.23
Suicide	.25*	.36	−.11	.33	−.08
Treatment Rejection	−.42*	.24	−.66	−.23	−.19
SNAP-2
Negative Temperament	.55*	.38	.17	.50	.05
Positive Temperament	−.12*	−.10	−.02	−.33	.21
Disinhibition	.23*	.18	.05	.00	.23
Mistrust	.53*	.36	.17	.38	.15
Manipulative	.35*	.16	.19	.18	.17
Aggression	.37*	.22	.15	.30	.07
Self-harm	.34*	.36	−.02	.30	.04
Eccentric perceptions	.38*	.28	.10	.20	.18
Dependency	.28*	.32	−.04	.23	.05
Exhibitionism	.15*	−.22	.37	−.13	.28
Entitlement	.18*	.06	.12	.30	−.12
Detachment	.30*	.34	−.04	.30	.00
Impulsivity	.12*	.16	−.04	.10	.02
Propriety	.16*	.16	.00	−.05	.21
Workaholism	.18*	.34	−.16	−.08	.26

^*Indicates p < .01.

BFI = Big Five Inventory; PAI = Personality Assessment Inventory; SNAP-2 = Schedule for Nonadaptive and Adaptive Personality, 2^nd Edition.

Note: r_alerting-cv can be computed by correlating the z-transformed observed correlations with the z-transformed [expert or non-expert] predicted correlations (Westen & Rosenthal, 2003).

In addition, both grandiosity and vulnerability were positively related to paranoia, interpersonal mistrust, borderline personality functioning, and a desire to change (negative relation with PAI treatment rejection), although these associations were stronger with vulnerability. Both composites also displayed positive relations with exhibitionistic tendencies and feelings of entitlement, but these associations were higher for grandiosity. These patterns are largely consistent with theories of pathological narcissism (Pincus et al., 2009).

Curiously, grandiosity and vulnerability showed discrepant patterns across two different measures of aggression drawn from the PAI (Sample 1) and the SNAP-2 (Sample 2). One explanation for this discrepancy could be related to differences across the two samples. A second possibility is that pathological narcissism is more strongly associated with becoming easily angered and provoked by perceived insults (items represented in SNAP-2 aggression), but less so with overtly aggressive behaviors (items represented in PAI aggression). Further research exploring differences between these aggression scales would be useful to better understand this divergence with respect to the PNI. Consistent with theoretical expectations, this association with provoked aggression was particularly strong for vulnerability.

Despite some overlap, the grandiosity and vulnerability composites were distinguished by several correlations. Consistent with theoretical expectations for the construct, grandiosity was associated with high levels of manic behaviors, cognitions, and affect, including manic grandiosity (PAI mania), as well as features of antisocial personality. This composite also related to a tendency to view oneself as active and energetic and one’s life as exciting (SNAP-2 positive temperament). Though these correlates are among the highest observed between grandiosity and other constructs, the magnitude of these associations is still somewhat weak. In contrast, vulnerability was uniquely associated with internalizing problems (PAI anxiety and depression), social isolation (SNAP-2 detachment), low extraversion (BFI), relying on others for approval (SNAP-2 dependency), suicidal ideation (PAI suicide), and self-harming and impulsive behaviors (SNAP-2 self-harm and PAI borderline). Notably, the vulnerability composite was significantly associated with all pathological personality traits (as assessed by the SNAP-2) and generally showed stronger associations with other indicators of psychopathology than the grandiosity composite.

PNI clinical utility

The clinical utility of the PNI grandiosity and vulnerability composites, as represented by r_contrast-cv values, are presented in Table 1. Overall, these values were all significant, with a mean r_contrast-cv value of .65 for both grandiosity and vulnerability. Examination of the values indicates that predictions tended to be better for how vulnerability would relate to normative traits (BFI), whereas predictions for grandiosity were higher relative to psychopathology and clinical concerns (PAI). Comparing clinician raters yields no differences in the overall accuracy of predictions made from experts and non-experts (mean r_contrast-cv = .65 for both groups). Although these results suggest that clinicians can make fairly accurate predictions regarding the array of clinically-relevant personality and psychopathology constructs with which grandiosity and vulnerability are associated, it is important to examine discrepancies between predicted and obtained values given the limitations of r_contrast-cv for evaluating specific inferences (Smith, 2005). These data can be gleaned from the “difference” columns in Tables 2 (grandiosity) and 3 (vulnerability), which simply displays the difference between predicted and obtained correlations, with negative values indicating that raters estimated the correlation to be more positive than was observed.

Predictions were generally accurate for both non-expert and expert clinicians regarding the positive associations between grandiosity and mania, positive temperament, and entitlement. In contrast, both sets of raters underestimated the degree to which grandiosity would correlate with neuroticism/negative temperament and associated dysfunction (e.g., anxiety disorders). Non-expert clinician raters overestimated the degree to which grandiosity would correlate with externalizing pathology (i.e., antisocial, drug, alcohol, and aggression), whereas expert raters did not anticipate the association grandiosity displayed with eccentric perceptions. The largest discrepancy between predictions and observed correlations for both non-expert and expert raters was the expectation that grandiosity would be highly positively associated with treatment rejection, when in fact this correlation was found to be significantly negative. Notably, any differences between predicted and obtained values could be the result of either inaccurate clinician predictions or, alternatively, could indicate that the PNI is not measuring these constructs in a way that would be anticipated based on clinical theory. In either case, discrepancies suggest potential problems in the clinical interpretation of the PNI vis-a-vis treatment rejection.

With regard to vulnerability, both sets of clinicians accurately anticipated the association between vulnerability and internalizing psychopathology (i.e., anxiety, anxiety disorders, depression, and self-harm) as well as introversion, detachment, dependency, and impulsivity. Non-expert clinicians incorrectly anticipated a moderately positive correlation with alcohol problems, drug problems, and treatment rejection. Both non-experts and experts anticipated slightly negative correlations between vulnerability and exhibitionism, but the observed correlation between these scales was positive. This may relate to an important distinction between overt and covert expressions of narcissism (Pincus & Lukowitsky, 2010), in which individuals with narcissistic vulnerability experience feelings of entitlement, but may express these feelings covertly making them more difficult for others to detect or anticipate.

Discussion

This study evaluated both the convergent and discriminant validity and clinical utility of the Pathological Narcissism Inventory (PNI), a recently developed self-report measure designed to capture both grandiose and vulnerable aspects of narcissistic personality pathology. We evaluated bivariate correlations between PNI grandiosity and vulnerability composites and a host of criterion variables as an index of validity and compared clinician predicted correlations to observed criterion correlations using a method of contrast analysis as an index of clinical utility. Overall, our results extend evidence for the validity and utility of the PNI as a measure of pathological narcissism. The grandiosity composite related to exhibitionistic, entitled, and manic behaviors, whereas the vulnerability composite tended to relate more specifically to negative emotions and interpersonal dysfunction. In short, the current results support and extend previous research related to the validity of the PNI (e.g., Pincus et al., 2009) and suggest that this instrument is a clinically useful self-report measure of pathological narcissism.

Results from the present study indicate that narcissistic grandiosity and narcissistic vulnerability share features associated with behavioral disregard for consequences (disinhibition), cognitive disregard of normative world-views (eccentric perceptions), and interpersonal disregard for the rights of others (manipulativeness). Our results also provide further support distinguishing grandiose and vulnerable expressions of pathological narcissism. The overall pattern of results indicated that grandiosity was more strongly related to indicators of pathological positive emotionality (mania, exhibitionism and entitlement) whereas vulnerability was more strongly related to indicators of negative emotionality (depression, negative temperament and self-harm) and interpersonal dysfunction (paranoia, social detachment, mistrust, and aggression).

It is interesting to consider these results in light of the forthcoming DSM-5 and specifically the proposed representation of Narcissistic PD. As mentioned above, Narcissistic PD has been reformulated for the DSM-5, in that it will now be represented by symptoms involving dysregulation of self and interpersonal functioning and two traits, grandiosity and attention seeking. Initial evidence suggests that traits beyond these two, such as hostility, deceitfulness, and callousness, are strongly related to DSM-IV NPD (Hopwood, Thomas, Markon, Wright, & Kreuger, in press). Furthermore, as in the DSM-IV (Pincus & Lukowitsky, 2010) traits that represent vulnerable aspects of narcissism are missing from the proposed DSM-5 representation of the disorder although they may be represented in other traits in the DSM-5 system. Further work is needed to evaluate the ability of the proposed DSM-5 system to capture the breadth of phenomena considered relevant to narcissism in the theoretical model underlying the PNI.

In addition to providing further evidence for the criterion validity of the PNI, this study also evaluated clinicians’ ability to predict validity coefficients a priori. Clinicians were, on the whole, consistent and generally accurate in their predictions about the nomological network of narcissistic grandiosity and narcissistic vulnerability. Furthermore, clinicians who are not experts in pathological narcissism were as accurate in their predictions as clinicians with expertise in pathological narcissism when accuracy was assessed at a global level. In considering this finding it is important to note that each predicted correlation was based both on clinicians’ understanding of the psychometric properties of the PNI grandiosity and vulnerability composites (in which experts had an advantage) as well as the psychometric properties of the criterion scales (in which experts had no distinct advantage). The overall accuracy of clinician predictions, as reflected in the r_contrast-cv values, indicates that if a clinician is privy to an individual’s PNI grandiosity and vulnerability scores, s/he can infer more than just grandiose and vulnerable tendencies. For instance, someone with a high score on PNI vulnerability and a low score on grandiosity is likely to experience a host of negative emotions, an absence of positive emotions, interpersonal dysfunction, and strong treatment motivation. The finding that clinicians without prior experience using the PNI were as accurate as the clinicians with expertise in pathological narcissism also suggests that most clinicians would be able to make important clinical inferences based on scores from the PNI. Importantly, for any given client these inferences would need to be tested using additional clinical or empirical data, but they can nonetheless guide clinicians by pointing towards potentially relevant avenues of consideration.

It is important to acknowledge that there are no benchmarks for evaluating the magnitude of r_contrast-cv coefficients. Because few studies have been conducted using the Westen and Rosenthal (2003) method to test the validity of psychological measures, it is difficult to judge when r_contrast-cv values are unusually high or unusually low. Accordingly, we evaluated and presented specific discrepancies between obtained and predicted values to obtain more nuanced information regarding the accuracy of clinicians’ predictions. Overall the values reported here are similar to or higher than the levels of r_contrast-cv coefficients in a previous, analogous study (Bombel et al., 2009), suggesting that these data support the clinical utility of the PNI with regard to clinicians’ ability to anticipate a wide-range of personality and psychopathology associations between grandiosity and vulnerability.

There were, however, some notable discrepancies in predicted values that merit further consideration. Non-expert clinicians tended to inaccurately predict that grandiosity would associate with externalizing pathology, such as substance abuse and aggression, whereas in our study grandiosity was better accounted for by associations with pathological aspects of positive emotionality, such as mania, exhibitionism, and entitlement. This distinction will be important to continue testing and articulating in future research, particularly in order to clarify whether this is a (perhaps common) clinical misconception or whether the PNI is not accurately assessing associated features of externalizing pathology. Despite the association between grandiosity and pathological manifestations of positive emotionality, the PNI was uncorrelated with a normal expression of positive emotionality (BFI extraversion). This finding was inconsistent with clinician predictions and research with some narcissism instruments (e.g., NPI; c.f., Holtzman, Vazire, & Mehl, 2010), though meta-analytic research has identified only a modest relation between narcissism and extraversion (Samuel & Widiger, 2008; weighted effect size = .09).

Another prominent inaccurate prediction across both sets of raters was anticipating a positive correlation between grandiosity and treatment rejection. Grandiosity is marked by inflated feelings of self-worth; therefore, assuming this scale would relate to treatment rejection makes conceptual sense. Thus, this finding might reflect an inability of the PNI grandiosity composite to capture resistance to change. Pincus et al. (2009) found that grandiosity was positively associated with increased psychotherapy appointment cancellations and no shows and negatively associated with adherence to medication regiments and contact with hospital staff. Notably, however, these outcome variables are features of treatment utilization, whereas the PAI scale measures treatment rejection, or a lack of desire to understand and change oneself. Consistent with this discrepancy in findings across studies, research is mixed regarding whether or not the PAI treatment rejection scale is significantly associated with non-mutual therapy termination (e.g., Charnas, Hilsenroth, Zodan, & Blais (2010) found that it was; Hopwood, Ambwani, & Morey (2007) found that it was not). Another factor that could account for this distinction is that features of treatment utilization were measured based on actual behaviors where treatment rejection is a self-reported indicator. Taken together, these results suggest that individuals with high levels of grandiosity may indeed express a desire to change, but given their pathology, may have difficulty following through with treatment. This intriguing possibility indicates the need for additional research, but for now it suggests that clinicians should be aware of the distinction between a desire for change and treatment utilization and strive to find ways to engage clients with narcissistic grandiosity.

Another systematic discrepancy across both non-expert and expert predictions was the positive association between grandiosity and neuroticism/negative temperament. Recent research indicates that the zero-order association between grandiosity and negative emotionality may be accounted for by vulnerability, as this association dissipates when controlling for vulnerability (Tritt et al., 2010). Similarly, additional analyses in the present study indicated that the partial correlation between grandiosity and both (BFI) neuroticism and (SNAP-2) negative emotionality were null after removing the association between grandiosity and vulnerability. Clinician raters in the present study may have been making predictions based on the pure grandiosity construct and not considering the overlap between grandiosity and vulnerability. This is an empirically important distinction; however, it is also clinically important to know that pathological narcissism tends to correlate with indices of negative emotionality. Further, there is generally a negative manifold among most clinical constructs such that indicators of psychopathology tend to correlate with one another, even when the constructs are conceptually distinct. This negative manifold was often unanticipated by clinician raters, indicating that clinicians may overestimate the discriminant validity of psychological measures.

All in all, we believe that the contrast analysis procedure developed by Westen and Rosenthal (2003) represents a promising method for quantifying the clinical utility of a measure when predictions are made by clinicians and when supplemented with specific information regarding inaccurate predictions, as we have demonstrated in the present study. This method provides a much stricter test of understanding the network of associations for a given construct than the typical procedure of describing, post-hoc, patterns of correlations, or more problematically, focusing only on those correlations that support the validity of the measure being evaluated and discarding the rest. Indeed, this approach aids in establishing the clinical utility of measures. In practice, clinicians must make a priori predictions about how a measure will work and the other constructs to which it will relate, and those predictions must be accurate, in order for that measure to be useful. We recommend that future studies examining the criterion validity of psychological measures employ a range of criteria and utilize the r_contrast-cv procedure in conjunction with examining the accuracy of specific predictions in greater detail.

This study has several limitations. First, all measures were self-report and all participants were college students. It has often been suggested that the use of self-report assessment and non-clinical samples for studying personality pathology can be limited, particularly regarding response validity and generalizability. Accordingly, future research should investigate the validity of the PNI in other kinds of samples, particularly those that over-sample individuals with pathological narcissism. Given that individuals high in narcissism may be unlikely to seek treatment based on the very nature of their pathology, we used a sampling strategy whereby we were likely to identify individuals with high levels of narcissism who might not present these problems in a clinical setting. An important question for future research is to ascertain whether the patterns we obtained in this study are generalizable to typical treatment seeking individuals. Researchers should also investigate pathological narcissism using different methodologies such as interviews, informant reports, clinician reports, performance-based measures, and lab tasks as well as different analytic techniques (e.g., person-centered approaches).

A similar limitation is that seven of the nine clinician raters were graduate student clinicians, representing a limited range of practicing clinicians. This may have provided a more conservative test of the utility of the PNI. Additionally, non-expert clinician raters ranged in their area of expertise, treatment orientation, and preferred treatment populations, providing heterogeneity to this group of student clinician raters. A final limitation with regard to generalizing the clinical utility of the PNI observed in our study is that we provided raters with a description of the primary and criterion scales, whereas practicing clinicians may not always have this information readily available. This highlights the importance of clinicians familiarizing themselves with scale descriptions of measures they use in their practice.

In conclusion, these data provide evidence for both the validity and clinical utility of the Pathological Narcissism Inventory by showing that the grandiosity and vulnerability composites demonstrate a pattern of correlations with a wide range of criterion variables that was largely predictable by theory and by practicing clinicians. These results add to the growing evidence for the validity of this promising instrument, point to its research and clinical utility for measuring narcissistic personality pathology, and build confidence in the accuracy of predictions made using the PNI.

Appendix A

Pathological Narcissism
Grandiosity	Strongly motivated to self-enhance in most contexts. Tendency to repress negative aspects of self, feel entitled, have an inflated self- image, engage in fantasies of superiority, and to be interpersonally exploitative, exhibitionistic, and to lack empathy
Vulnerability	Fragile self and affect regulatory capacities. Tendency to experience shame or rage when narcissistic needs are disappointed, to be sensitive to perceived threats to self-esteem, to avoid showing others faults or concerns, and to shy away from relationships for fear of not receiving desired admiration and recognition
Personality
Neuroticism	Tendency to experience negative emotions such as sadness, anxiety,  and anger
Extraversion	Tendency to experience positive emotions such as happiness and to  seek relationships, be gregarious
Openness	Tendency to value new and different experiences and to be tolerant of  others’ values and world views
Agreeableness	Tendency to be interested in getting along with others, willing to defer  personal agendas for the sake of smoother relationships
Conscientiousness	Tendency to be organized, dutiful, careful and achievement-striving
Psychopathology/Functioning
Somatic Complaints	Focus on physical health related issues
Anxiety	Experience of generalized anxiety across different response modalities
Anxiety Related  Disorders	Symptoms and behaviors related to OCD, phobias, and PTSD
Depression	Experience of depression across different response modalities
Mania	Experience of behavioral, affective, and cognitive symptoms of mania  and hypomania
Paranoia	Experience of paranoid symptoms and traits
Schizophrenia	Symptoms relevant to the broad spectrum of schizophrenic disorders
Borderline Features	Attributes indicative of borderline levels of personality functioning
Antisocial Features	Focuses on behavioral and personological features of antisocial  personality
Alcohol Problems	Use of and problems with alcohol
Drug Problems	Use of and problems with drugs
Aggression	Characteristics and attitudes related to anger, assertiveness, and hostility
Suicidal Ideation	Frequency and intensity of thoughts of self-harm or fantasies about  suicide
Treatment Rejection	Attitudes that represent obstacles or indicate low motivation for  treatment
Personality Pathology
Negative Temperament	High scorers are prone to negative emotional experiences. They are  moody and feel chronically nervous and stressed. They are easily  annoyed or irritated, and with little provocation may feel even  stronger emotions, such as anxiety or anger. They worry a great deal  and have difficulty concentrating and sleeping as a result. Low scorers are not at all nervous or anxious. They take life’s  difficulties in stride and remain calm in most situations. They are no  easily upset and recover quickly from negative experiences.
Mistrust	High scorers are suspicious and mistrustful of others, and are  generally cynical about interpersonal relationships. They feel that  have often been betrayed or disappointed by those close to them, and  that other take unfair advantage of them. They are self-protective and  believe it is best to keep others from getting to know them too well. Low scorers are trustful, even naïve, about interpersonal relationships.  They feel stable and secure in their friendships, and do not feel the  need to hide their feelings from others. They feel they are treated  fairly and honestly by others.
Manipulativeness	High scorers admin not only to manipulating others, but also to  enjoying the many ways – both direct and indirect- that they take  advantage of others’ weaknesses. They have little regard for the  rights and feelings of others, and will bend the rules as they can get  away with it. They consider themselves quite clever and skillful in  pulling of their various con games. Low scorers do enjoy trying to beat the system. They respect the  propriety of others and would not hurt others to get what they want.  They do not try to avoid work or to bend the truth to their advantage.  They view those who do so as lazy.
Aggression	High scorers are easily angered have difficulty controlling their anger.  They frequently get into quarrels and admit that their violent temper  often gets them into trouble. They enjoy physical fights and are  readily provoked by perceived insults. They hold grudges and seek  revenge when they feel wronged. Low scorers do not readily become angry and easily control their  temper when they do. They do not enjoy fighting and would go out of  their way to avoid a fight. When insulted, they prefer to forgive and  forget.
Self-harm	High scorers have very low self-esteem – they feel they have made a  mess of their lives and do not like themselves. They deal with tension  or frustration by hurting themselves. They often feel that suicide is  the only way out of their troubles and may have tried to commit  suicide. Low scorers are satisfied with the way they are handling their lives.  They like themselves, are not self-abusers, and do not seriously  consider suicide as a solution to their problems.
Eccentric Perceptions	High scorers have depersonalization or derealization experiences.  They feel that they have ESP or other special abilities such as  clairvoyance. They claim to have unusual experiences, such as out-  of-body episodes, or synesthetic experiences (e.g., sensing odors as  colors). Low scorers do not admit to either depersonalization or derealization  experiences. They do not feel they have any “special” abilities or that  their way of viewing the world is at all unusual. They deny odd  perceptual experiences.
Dependency	High scorers depend on others for direction and approval. They do not  like to make decisions and prefer to have others choose for them,  even in important matters. They do not have confidence in their  decisions, have difficulty making up their minds, and base their  decisions on what others think. They are concerned with please  others and frequently check with others for approval. Low scorers are self-reliant. They enjoy handling their own problems  and making their own decisions. They are confident in their choices  and decide things easily.
Positive Temperament	High scorers enjoy their active and exciting lives. They have a good  deal of energy, work hard, and undertake projects with enthusiasm.  They are lively and cheerful, alert and interested in many things, and  optimistic about the future. Low scorers do not find their lives interesting or exciting. They are  easily fatigued, rarely feel enthusiastic or inspired, have few interests,  and are not excited by the thought of starting something new.
Exhibitionism	High scorers love the limelight. They like to perform and do so  whenever they have the chance. They like being the center of  attention, and the life of the party. They dress so that people will  notice them, and especially to attract sexual attention; they enjoy  flirting. Low scorers do not enjoy being the focus of attention. They prefer not  to be noticed and they dress and act accordingly. They avoid public  performance of any sort and do not attempt to attract sexual attention.
Entitlement	High scorers view themselves as very special and extraordinary  people. They feel they are knowledgeable and talented, and have  many admirable and enviable qualities. They believe they deserve  special recognition and privileges beyond what they are currently  getting. Low scorers are self-effacing and humble. They do not feel they are  special, unusually talented, or particularly admirable. They do not  believe they deserve special privileges or recognition.
Detachment	High scorers are loners. They are aloof and distant from others and  have few friends. They keep to themselves even when around others  and prefer to spend their time alone. They do not have warm feelings  toward others and claim not to experience many strong emotions at  all. Low scorers are “people people.” They go out of their way to meet  people and enjoy the feeling of being with friends. They view  themselves as warm and open with others.
Disinhibition	High scorers act spontaneously with little regard for the consequences  of their behavior for themselves or others. They pursue stimulating  experiences without regard to social or legal norms or to potential  risks. They are disorganized and often fail to honor their  commitments to others. Low scorers are not impulsive, nor do they manipulate others for their  own ends. They hold conventional, conservative views regarding  social, legal, ethical, and moral issues. They are serious people who  believe in doing things in proper order and in following rules of all  kinds.
Impulsivity	High scorers are reckless individuals who prefer to do the first thing  that comes to mind rather than stop and think things over. They live  moment-to-moment and day-to-day, without plans for the immediate  or long-term future. They like to take changes and to spend their  money and their time on impulse. Low scorers are cautious individuals who life a safe, quiet life. They  prefer a level-headed, “sensible” approach to things, in which  decisions are thought through and everything is planned out slowly  and carefully. They keep track of their money, rationally plan their  daily activities, and prepare for their future.
Propriety	High scorers are greatly concerned with proper standards of conduct,  Workaholismnot just for themselves, but for the community at large. They value  Workaholismtheir reputation highly and place appearances over personal comfort  or convenience. They emphasize the importance of following social  Workaholismconventions to the letter and are offended when others violate the  rules. Low scorers are not concerned with social standards. They are  comfortable with rude or boorish behavior and do not care about  protecting their reputation. They are not strict about “right” and  “wrong,” and place little importance on social rules or conventions.
Workaholism	High scorers enjoy work more than play. They place work above all  else, and neglect their friends and family to do so. They are  perfectionists who feel they must finish everything they start and do  not consider a job finished until it is perfect. They drive themselves  hard, continuing to work on a problem even when exhausted, long  after others have given up. Low scorers do not enjoy hard work. They never get so caught up in  their work that they neglect other aspects of their lives, and they find  time to have fun no matter how busy they are. They do not push  themselves and even enjoy being lazy. They do not feel the need to  finish everything they start and will give up on a project when they  are tired.