Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2010 Mar 19.
Published in final edited form as: Assessment. 2008 Jun 11;15(4):483–492. doi: 10.1177/1073191108319022

Assessment Procedures for Narcissistic Personality Disorder

A Comparison of the Personality Diagnostic Questionnaire-4 and Best-Estimate Clinical Judgments

Joshua D Miller 1, W Keith Campbell 1, Paul A Pilkonis 2, Jennifer Q Morse 2
PMCID: PMC2841972  NIHMSID: NIHMS171531  PMID: 18550845

Abstract

This study examined the degree of correspondence between two assessments for narcissistic personality disorder (NPD) in a mixed clinical and community sample—one using a self-report measure (Personality Diagnostic Questionnaire-4) and the other using clinical judgments derived from an assessment based on the longitudinal, expert, all data (LEAD) methodology. NPD scores demonstrated moderate convergence for the total scores but weak convergence for the individual criteria. The authors also examined the correlates created by each set of NPD scores using Cloninger's Temperament and Character Inventory (TCI). The NPD scores demonstrated areas of convergence (e.g., Cooperativeness, Self-directedness) and divergence (i.e., Harm Avoidance, Novelty Seeking) with these personality scores. These divergences may be due to the wording of certain items on the Personality Diagnostic Questionnaire-4 NPD scale, which may require rewriting if it is to provide an assessment that is more highly convergent with the Diagnostic and Statistical Manual of Mental Disorders NPD construct.

Keywords: narcissistic personality disorder, assessment, Personality Diagnostic Questionnaire-4 (PDQ-4), convergent validity

Reliable and valid assessment of Diagnostic and Statistical Manual of Mental Disorders (DSM-IV; 4th ed., American Psychiatric Association, 1994) personality disorders (PDs) has been a notoriously difficult task due to the multidimensional quality and complexity of the overarching PD constructs and definitions of the specific criteria. In addition, error variance is introduced into these ratings as a result of the source of data used to inform the diagnoses. Self-report measures, which are advantageous in terms of time and resources, may be adversely affected by individuals' limited self-awareness, idiosyncratic understanding of the questions, and/or purposeful deception (Lilienfeld & Fowler, 2006). Alternatively, interviews require judgments about traits that have been observed during a relatively short clinical interview.

The difficulty of assessing DSM-IV PDs using self-reports may be exacerbated for narcissistic personality disorder (NPD; see Hilsenroth, Handler, & Blais, 1996) as these individuals may have particularly limited insight into their own personality. For example, Miller, Pilkonis, and Clifton (2005) found that NPD was the only PD in which there were substantial divergences in the self- versus informant-reported personality correlates associated with the disorder. Klonsky, Oltmanns, and Turkheimer (2002) found in their meta-analysis that NPD demonstrated the smallest self–other correlation of all the PDs. In addition to poor self–informant agreement, Widiger and Coker (2001) found relatively poor agreement between the dimensional self-report ratings of NPD and the dimensional NPD ratings, as derived from semistructured interviews. In their review of these types of studies, the median correlation for NPD was .29, which was lower than all but two other PDs. It is clear that the self-reported NPD scores do not converge closely with ratings by peers, significant others, and expert raters. However, the nature or cause of this divergence is not entirely clear.

One way to examine this issue is to examine the nomological network of relations that surround the scores on different measures. That is, one can test whether the PD scores from a given measure demonstrate the same pattern of relations with other relevant criteria such as general personality traits, axis I constructs, or ratings of functioning/impairment. This technique is not only useful for testing why self-report PD ratings diverge from peer or expert ratings but also why measures of the same construct diverge even when the same assessment methodology is used. For example, Saulsman and Page's (2004) meta-analysis of the relations between the Five-Factor Model (FFM) personality and PDs demonstrated that the use of one of Millon's Clinical Multiaxial Inventories (e.g., MCMI-III; Millon, 1994) resulted in correlations between certain PDs (e.g., histrionic, narcissistic, obsessive–compulsive) and certain FFM domains (e.g., Neuroticism), which were in a different direction from the majority of the correlations derived from other measures, suggesting that the MCMI-III variant of certain PDs (e.g., NPD) measures a different, potentially more adaptive, variant. In general, Saulsman and Page reported that most measures of NPD other than the MCMI-III variants were primarily negatively correlated with Agreeableness and positively correlated with Extraversion; these findings are quite congruent with expert ratings of prototypical NPD using the FFM traits (e.g., Lynam & Widiger, 2001; Samuel & Widiger, 2004). This is not true, however, for all DSM-IV-based assessments of NPD. Miller and Campbell (2008) examined the FFM correlates of the Personality Diagnostic Questionnaire-4 (PDQ-4), which is a 99-item self-report measure of DSM-IV PDs. The PDQ-4 uses one question (e.g., “Some people think I take advantage of others.”) to assess each specific DSM-IV PD criterion (e.g., NPD: lacks empathy), and the PDQ-4 is the self-report measure that is most “directly coordinated with the DSM-IV personality disorder diagnostic criteria” (Widiger & Coker, 2001, p. 412). Across two samples and using both self-reported and informant-reported FFM traits, Miller and Campbell found that the PDQ-4 NPD scale was positively correlated with Neuroticism and negatively correlated with Agreeableness, Extraversion, and Conscientiousness. This suggests that the PDQ-4 NPD scale may be assessing a construct that is only partially overlapping (i.e., Disagreeableness) with the NPD construct assessed by other measures.

Further complicating the assessment of NPD is the problem that there may be different types or forms of narcissism, including “overt” versus “covert” or “grandiose” versus “vulnerable” (Cain, Pincus, & Ansell, in press; Wink, 1991). Overt or grandiose narcissism is thought to be related to traits such as “self-assuredness, aggressiveness, exhibitionism, self-indulgence, and disrespect for the needs for others” (Wink, 1991, p. 596). An examination of grandiose narcissism, using basic personality traits, suggests that it is primarily positively related to certain aspects of Extraversion such as dominance and excitement seeking as well as interpersonal antagonism and negatively related to the experience of negative affectivity (Miller & Campbell, 2008; Wink, 1991). Conversely, Dickinson and Pincus (2003, p. 189) suggested that covert or vulnerable narcissists put forth a false facade of modesty and maintaining “underlying grandiose expectations for oneself and others (Gabbard, 1989, 1998).” Findings from Wink (1991) suggested that vulnerable narcissism is positively correlated with negative affectivity and interpersonal antagonism and negatively related to sociability and dominance (i.e., Extraversion). Dickinson and Pincus suggested that DSM-IV NPD assesses overt/grandiose narcissism. At the DSM-IV symptom level, however, the results are unclear regarding the existence of a one- (i.e., grandiose) or two-factor (i.e., grandiose and vulnerable) solution. Fossati et al. (2005), using exploratory and confirmatory factor analyses, argued that the DSM-IV NPD criteria are underlain by two correlated factors, which they labeled overt and covert. However, Miller, Hoffman, Campbell, and Pilkonis (2008) found stronger support for a one-factor model. This single factor looked more like the grandiose form than the vulnerable form of narcissism.

The extent to which various PD measures assess either the grandiose or vulnerable form or a blend of the two will also influence the level of convergent validity found for measures of NPD. For example, given the personality traits associated with these two forms of narcissism, it is unclear how to interpret the Miller and Campbell (2008) results from the PDQ-4 NPD scale. On the basis of its correlates, Miller and Campbell argued that it appears to assess more of a vulnerable form of narcissism, despite the fact that the PDQ-4 items are supposed to be explicitly tied to the DSM-IV PD symptoms. There may be something about the wording of certain PDQ-4 NPD items that affects the personality correlates and inadvertently assesses more of the vulnerable form of narcissism when it was designed to be akin to the DSM-IV construct (e.g., grandiose narcissism).

Our goal in the current study was to compare two assessment methodologies for DSM-IV NPD. To this end, we examine the relations between NPD scores from a commonly used self-report measure of PD (PDQ-4; Hyler, 1994) and best estimate, clinical judgments of NPD, which involved a far more intensive, comprehensive assessment approach—the LEAD (longitudinal, expert, all data; Spitzer, 1983) model of assessment. The LEAD NPD ratings were determined after 6 to 10 hr of direct assessment of the participant and 2 to 4 hr of case conference in which three or more expert raters reviewed all of the available data to derive a consensus rating for each DSM-IV PD symptom. A limited number of studies have examined the correlation between PDQ-4 NPD scores and interview or consensus ratings of DSM-IV NPD symptoms and all have found medium-effect sizes of .27 (Wilberg, Dammen, & Friis, 2000), .31 (Yang et al., 2000), and .42 (Fossati et al., 1998). None of these studies, however, were focused solely on NPD and thus they did not explore these relations further to explicate this level of convergence. In the current study, we examine the following: (a) the convergent correlation between the total PDQ-4 NPD scores and the LEAD ratings of NPD symptoms, (b) the convergent correlations between the specific PDQ-4 NPD items and their convergent DSM-IV NPD criteria as well as the total LEAD NPD scores, and (c) the personality correlates generated by both the NPD scores and their specific items/symptoms using Cloninger's model of temperament and character (Cloninger, Svrakic, & Przybeck, 1993).

To accomplish our third objective, we use Cloninger's seven-factor Temperament and Character Inventory (TCI; Cloninger, Przybeck, Svrakic, & Wetzel, 1994) to study the personality correlates of PDQ-4 and LEAD NPD scores as this model has demonstrated substantial relations with PD symptoms (e.g., Bagby, Marshall, & Georgiades, 2005; De Fruyt, De Clercq, Van De Wiele, & Van Heeringen, 2006). We believe that it is informative to examine the general personality traits that underlie the two NPD scores and specific items as it can help clarify the ways in which the constructs converge and diverge. The TCI is particularly well suited to studying NPD as De Fruyt and colleagues (2006) found that it explained 39% of the variability in NPD scores and was one of the only two PDs in which the FFM domains did not account for any additional variance once the TCI dimensions were used. Cloninger and colleagues have argued that all PDs are associated with lower scores on the domains of Self-directedness (SD) and Cooperativeness (CO) but that cluster-B PDs such as narcissism are differentiated from other PDs by high scores on Novelty Seeking (NS: Cloninger et al., 1993; Svrakic, Whitehead, Przybeck, & Cloninger, 1993). Although most studies have found a negative relation between NPD and SD and CO (e.g., Ball, Tennen, Poling, Kranzler, & Rounsaville, 1997; De Fruyt et al., 2006; De La Rie, Duijsens, & Cloninger, 1998; Svrakic et al., 1993), the relations between NPD and NS as well as the other TCI domains (e.g., Harm Avoidance) are less consistent. For instance, some studies found a significant relation between NS and NPD (e.g., Bagby et al., 2005; Bayon, Hill, Svrakic, Przybeck, & Cloninger, 1996; De La Rie et al., 1998), but others did not (e.g., Ball et al., 1997; De Fruyt et al., 2006; Mulder, Joyce, Sullivan, Bulik, & Carter, 1999). Given these findings, we expect both measures of NPD to be significantly negatively correlated with SD and CO. Because of previous research on the basic traits associated with PDQ-4 NPD (e.g., Miller & Campbell, 2008), we expect to find a significant positive relation between PDQ-4 NPD and Harm Avoidance, and a negative correlation with NS. We also examine the correlations between the individual NPD symptoms (i.e., PDQ-4 and LEAD ratings) and the TCI to explore how the individual items relate to these personality dimensions.

Method

Participants and Procedures

This sample (n = 151) was composed of 70 psychiatric patients and 81 nonpsychiatric participants. Nonpsychiatric patients were either diabetic patients (n = 23) or university faculty or staff (n = 58). This sample was part of a larger sample (N = 624) that was first screened for PDs using brief, self-report scales (e.g., the PD scales from the Inventory of Interpersonal Problems). The larger sample was stratified on the basis of initial screening scores (above vs. below the threshold indicating the presence of any PD), and individuals were randomly selected within the strata to participate in the interview portion of the study. The goal was to create an interview sample with a 50% PD prevalence rate, which was optimal for examining the operating characteristics (i.e., sensitivity, specificity) of the screening measures (the primary goal of the earlier protocol).

Of the 151 individuals, 128 had data from both narcissism assessments and comprise the sample reported in the current results. Of the participants, 86 (67%) were women, 111 (87%) were Whites, 13 (10%) were Blacks, 4 (3%) were Asian Americans, and the mean age was 38.74 years (range = 21 to 60, SD = 11.36). In terms of axis I and axis II disorders, 81 (63%) individuals had an axis I diagnosis, and 75 (59%) received a PD diagnosis, using the LEAD ratings. The individuals with complete data (n = 128) did not differ from those who did not (n = 23) with regard to age, sex, race, or consensus ratings of NPD.

Measures

Consensus ratings of NPD criteria

Complete details of the assessment methodology are provided elsewhere (Pilkonis et al., 1995). At intake, participants were interviewed for 6 to 10 hr in a minimum of three assessment sessions. The assessments included structured symptom ratings, structured interviews for axis I (Structured Clinical Interview for DSM-IV; First, Gibbon, Spitzer, & Williams, 1997) and II disorders (Structured Interview for DSM-IV Personality; Pfohl, Blum, & Zimmerman, 1997), and a detailed social and developmental history. Patients completed self-report questionnaires between interviews. Following the evaluation, the interviewer presented the case at a diagnostic conference with colleagues from the research team. On the basis of the structured interview and other data, the interviewer put forth his or her rating for each PD symptom; these ratings were discussed, for each PD symptom, by the research team until a consensus rating could be made on a scale of 0 (absent), 1 (present), or 2 (marked). The LEAD NPD scores are the addition of these consensus scores across symptoms (M = 3.04, SD = 3.55, a = .81). Five individuals (4%) met or exceeded the diagnostic threshold to receive an NPD diagnosis. Of the five individuals who met LEAD NPD criteria, four also met the criteria for a PDQ-4 NPD diagnosis. PDQ-4 data were not scored or discussed prior to or during these case conferences.

PDQ-4+

The PDQ-4+ is 99-item self-report measure of DSM-IV PDs on which items are answered using a yes/no format. PD symptom counts are computed by summing the items for each PD (M NPD = 1.98, SD = 1.61, α = .55). Of the participants, 13 (10%) met the diagnostic threshold using the PDQ-4+ to receive an NPD diagnosis. Of the 13 participants who met NPD diagnostic criteria on the PDQ-4, 4 met LEAD criteria for a diagnosis of NPD.

TCI

The TCI is a 240-item questionnaire that measures dimensions of NS, Harm Avoidance, Reward Dependence, Persistence, SD, CO, and Self-transcendence. Alphas for the seven factors ranged from .46 (Persistence) to .83 (Harm Avoidance) with a median of .74.

Results

Correlations were calculated using polychoric correlations (using SAS [Version 9.1], except for the correlation between the total NPD scores in which a Pearson correlation was calculated), which estimate the underlying continuous distributions and provide a better fit to the data when response scales have limited options. In the present case, most of our variables were scored true–false (PDQ-4) or rated on a 3-point scale (0 = absent, 1 = present, 2 = marked). At the total score level, PDQ-4 and LEAD NPD scores were significantly correlated, r = .46, p ≤ .01. We examined the relations between the individual PDQ-4 and LEAD NPD items (as well as the PDQ-4 items with the total LEAD NPD scores and the LEAD NPD symptoms with the PDQ-4 NPD total scores). The convergent correlations for the specific NPD items across the assessment procedures ranged from −.21 (DSM-IV NPD Symptom 7: lacks empathy) to .43 (DSM-IV NPD Symptom 2: preoccupied with fantasies of unlimited success, power, brilliance, etc.) with a median of .33 (see Table 1). An examination of the 95% confidence intervals (CIs) surrounding these correlations suggested that only three PDQ-4 NPD items were significantly correlated with their respective items from the LEAD ratings (e.g., CIs that do not include zero). Only two of the nine PDQ-4 NPD items demonstrated convergent validity correlations with the congruent LEAD NPD criterion that were greater than all of the discriminant validity correlations (i.e., PDQ-4 Items 2 and 5; although the convergent validity correlation for Item 5 was larger than all of the discriminant rs, it was not statistically significant). The correlations between the individual PDQ-4 NPD items and the overall LEAD NPD ratings ranged from −.01 (DSM-IV NPD Symptom 7: lacks empathy) to .49 (DSM-IV NPD Symptom 1: grandiose sense of self-importance) with a median of .36. Of these, six of the nine PDQ-4 NPD items were significant. Although not included in Table 1, we also calculated the polychoric correlations between the individual LEAD NPD items and the PDQ-4 NPD total score. All of the nine LEAD NPD symptoms were significantly correlated with the PDQ-4 NPD total score, with correlations ranging from .27 (DSM-IV NPD Symptom 9: arrogant, haughty behavior) to .50 (DSM-IV NPD Symptom 5: sense of entitlement), with a median of .41.

Table 1.

Correlations Between PDQ Items and DSM-IV NPD Criteria and Total Scores

LEAD NPD
Ratings (specific
DSM-IV symptoms)a
 Discriminant
Validity
 Total LEAD
NPD Symptoms
PDQ-4 NPD r 95% CI Range Mdn r 95% CI
PDQ-4 NPD 1 (grandiosity) .35 .02-.68 .26- .47 .41 .49 .31-.68
PDQ-4 NPD 2 (fantasies of unlimited success, power, etc.) .43 .13-.74 −.05-.39 .18 .31 .05-.58
PDQ-4 NPD 3 (special and unique) .37 .10-.65 .07-.47 .24 .36 .15-.56
PDQ-4 NPD 4 (requires excessive admiration) .14 −.16-.44 .02-.57 .32 .39 .18-.60
PDQ-4 NPD 5 (sense of entitlement) .37 −.03-.76 −.10b−.34 .04 .15 −.22-.54
PDQ-4 NPD 6 (interpersonally exploitative) .33 .00-.66 −.05-.46 .26 .37 .08-.65
PDQ-4 NPD 7 (lacks empathy) −.21 −.56-.15 −.38-.10 −.06 −.01 −.31-.28
PDQ-4 NPD 8 (envious of others / others envious
of him or her)		 .27 −.02-.55 .22-.42 .28 .41 .20-.62
PDQ-4 NPD 9 (arrogant, haughty behavior) .17 −.18-.51 −.16-.40 .18 .22 −.05-.50
Open in a new tab

Note: CI = confidence interval; DSM-IV = Diagnostic and Statistical Manual of Mental Disorders (4th ed.; American Psychiatric Association, 1994).

a

All correlations are polychoric correlations between the PDQ (Personality Diagnostic Questionnaire-4) and LEAD NPD (LEAD = longitudinal, expert, all data; NPD = narcissistic personality disorder).

b

One discriminant validity correlation was left out as it could not be estimated.

Next, we examined the correlations between the NPD total scores and Cloninger's TCI (see Table 2). To test whether these correlations were significantly different across the two NPD measures, we used LISREL 8.54 to calculate an omnibus test of the two sets of correlations; results indicated a significant difference existed, χ2(df = 7) = 19.19, p ≤ .01 (see Cheung & Chan, 2004). We next tested each of the seven individual pairs of correlations (e.g., correlation between PDQ-4 NPD and Harm Avoidance vs. the correlation between LEAD NPD and Harm Avoidance). Of the seven tests, two were significant; the correlation between LEAD NPD total score and NS was significantly stronger, χ2(df = 2) = 6.50, p ≤ .05, than the correlation between the PDQ-4 NPD scores and NS, whereas the correlation between PDQ-4 NPD and Harm Avoidance was significantly stronger, χ2 (df = 2) = 7.27, p ≤ .01, than the correlation between LEAD NPD and Harm Avoidance. Overall, PDQ-4 NPD total scores were significantly positively correlated with NS, Harm Avoidance, and Self-transcendence, and negatively correlated with SD and CO. Alternatively, LEAD NPD total scores were significantly positively correlated with NS and negatively correlated with SD and CO. We also conducted profile analyses in which we examined the similarity of the overall sets of correlations generated by the narcissism scores with regard to the TCI dimensions. These analyses quantify the degree to which the personality correlates of the PDQ-4 NPD scores, as assessed with the TCI, are similar to personality correlates of the LEAD NPD scores, as assessed with the TCI. The similarities of the personality profiles for the two narcissism assessments are quantified in the final column of Table 2; the similarity scores represent double-entry correlations, which take into account the similarity in both shape and magnitude (see McCrae, in press, for a review). For the total scores, the two profiles were quite similar overall, r = .82, p ≤ .01.

Table 2.

Correlations Between NPD Items and Scores and Cloninger's TCI

Cloninger's TCI Dimensions
NS HA RD PERS SD CO ST ICCDE
PDQ-4 NPD Total .18a,b .22a,b −.04 .08 −.45a −.35a .23a .82**
LEAD NPD Total .35a −.05 −.05 .08 −.28a −.34a .11
PDQ-4 NPD 1 .11 .07 −.19 .25a −.23a −.19 .17 .43
LEAD NPD 1 .44a −.04 .08 −.05 −.24 −.12 .09
PDQ-4 NPD 2 .11 −.18 .09 .35a .03 −.13 .16 .72**
LEAD NPD 2 .32a −.49a −.02 .20 .08 −.12 .21
PDQ-4 NPD 3 .13 .30a −.02 .15 −.42a −.28a .15 .55a,*
LEAD NPD 3 .57a −.07 .28a .05 −.32a −.25a .12
PDQ-4b NPDb 4 .24a .28a .35a −.28a −.69a −.38a .08 .53
LEADb NPD 4 .36a .08 −.21 −.05 −.29a −.24a .14
PDQ-4 NPD 5 −.05 .04 −.14 −.22 −.49a −.54a −.14 .43
LEAD NPD 5 .24a .01 .13 .09 −.35a −.24a .07
PDQ-4 NPD 6 .37a .05 .00 −.24 −.48a −.25 .21 .58a,*
LEAD NPD 6 .21a .04 −.19 .18 −.22a −.44a .07
PDQ-4 NPD 7 −.03 .08 −.32a .27 −.24 −.14 .31a .29
LEAD NPD 7 .15 −.08 .05 .04 −.08 −.14 .13
PDQ-4 NPD 8 .17 .07 −.01 .04 −.13 −.27a .09 .88**
LEAD NPD 8 .28a −.05 −.05 .12 −.24a −.31a .11
PDQ-4 NPD 9 −.19 .32a −.18 .13 −.18 −.08 .28a −.23
LEAD NPD 9 .23a −.29a −.17 .00 −.02 −.21 .10
Open in a new tab

Note: All correlations are polychoric correlations. CO = Cooperativeness; ICCDE = intraclass correlation–double entry (for ICCDE statistics, corresponding p values provided below as * and **); HA = Harm Avoidance; NS = Novelty Seeking; PERS = Persistence; RD = Reward Dependence; SD = Self-directedness; ST = Self-transcendence; and LEAD = longitudinal, expert, all data; NPD = narcissistic personality disorder; PDQ-4 = Personality Diagnostic Questionnaire-4; Cloninger's TCI = Temperament and Character Inventory.

a

Indicate that the 95% confidence intervals do not include zero.

b

Correlations that are significantly different.

*

p < .05.

**

p < .01.

We also examined the TCI correlates and profile similarity scores of the individual PDQ-4 and LEAD NPD items as a way of elucidating the characteristics of the individual items. These correlations were not tested against one another to reduce the probability of making Type 1 errors. At the item level, both the PDQ-4 and LEAD NPD symptoms were most consistently significantly correlated (negatively) with the TCI dimensions SD and CO. The items from neither NPD measure were consistently related to Reward Dependence or Self-transcendence. The findings were most discrepant between the PDQ-4 and LEAD NPD measures for NS and Harm Avoidance. As with the personality correlates of the total NPD scores, we calculated similarity scores for the TCI correlates of each NPD symptom (see Table 2). These correlations ranged from −.23 (DSM-IV NPD Symptom 9: arrogant, haughty behavior) to .88 (DSM-IV NPD Symptom 8: envious of others/believes others are envious of him or her). Three of the four symptoms (i.e., DSM-IV NPD Symptoms 5, 7, 9) with the least similarity in terms of personality correlates were also the cases where the PDQ-4 NPD item had demonstrated the weakest convergence with the LEAD NPD total scores.

Discussion

Reliable and valid assessments of personality disorders have been difficult to accomplish with the various methodologies (e.g., self-report vs. informant reports vs. interviews) having both positive and negative aspects. This task may be particularly difficult for narcissistic PD because narcissistic individuals see themselves in a manner that is discrepant from their peers or significant others (e.g., Klonsky et al., 2002; Miller et al., 2005). In addition, there are fundamental disagreements about the nature of the construct (e.g., role of vulnerability and grandiosity), which may hinder the field's ability to develop a cohesive body of research on this disorder without applying increased attention to the latent constructs and the measurements used to assess these constructs.

In the current study, we focused on two assessments of NPD that are supposed to directly reflect the DSM-IV NPD criteria. Our findings can be recounted as follows: First, overall, the self-reported PDQ-4 NPD score demonstrated moderate convergent validity with consensus ratings of NPD (r = .46); in fact, the current convergence was greater than that found in the earlier examinations using the PDQ-4 (e.g., Wilberg et al., 2000; Yang et al., 2000), although it was quite similar to that found by Fossati et al. (1998), r = .42. This convergence is actually higher than one typically finds between a self-report measure of NPD and a semistructured interview of NPD; the median convergence between these two types of measures for NPD is .29 (Widiger & Coker, 2001). Second, the specific items in the PDQ-4 did not converge on the specific DSM-IV NPD criteria that they were meant to assess. Rather, the PDQ-4 NPD scale was more successful at capturing the overall level of NPD than the specific DSM-IV NPD criteria. Only two of the nine PDQ-4 NPD items demonstrated their strongest correlation with the matching DSM-IV symptom, as rated by the experts. Alternatively, many of the PDQ-4 NPD items demonstrated stronger correlations with other LEAD NPD scores than with the convergent symptom. For instance, the PDQ-4 item meant to assess the need for admiration (i.e., “I very much need other people to take notice of me or compliment me.”) had a convergent correlation of only .14; however, it demonstrated a much stronger correlation with the LEAD rating for the NPD symptom measuring sense of entitlement (r = .57). Interestingly, although this specific PDQ-4 NPD item demonstrated poor convergence with its specific LEAD rating, it still demonstrated a significant correlation with the total LEAD NPD score (i.e., r = .39). Alternatively, one item demonstrated poor convergent validity (lacks empathy: “People have often complained that I did not realize they were upset.”) with the specific LEAD item (i.e., r =-.21) and the total LEAD NPD score (i.e., r = −.01). This item appears to be a poor indicator of both the “lacks empathy” criterion specifically and the more global NPD construct.

Some of the poor convergence found between the PDQ NPD items and LEAD may reflect items that were not written in a way that captures the intended content (e.g., PDQ-4 NPD item: lacks empathy). Another possible reason for the lack of congruence is that individuals may be willing to endorse certain types of narcissistic traits but not others. For instance, an examination of the endorsement frequencies for the PDQ-4 and LEAD NPD symptoms suggested that individuals may be more likely to endorse grandiosity (Item 1 of PDQ-4 NPD; 43% endorsed this item) and the belief that one is “special and unique” (Item 3 of PDQ-4 NPD; 43% endorsed this item) on the self-report instrument, even though the experts rated more infrequently the presence of these traits (1 = present or 2 = marked; at 19% and 26%, respectively). This pattern of overendorsement is a problem that is endemic to self-report assessments of personality disorder more generally. Conversely, the two PDQ-4 NPD symptoms self-reported the least frequently (i.e., PDQ-4 NPD Item 5: entitlement, 5% endorsed; PDQ-4 NPD Item 6: exploitativeness, 10% endorsed) were the most frequently endorsed items by the experts at 34% and 34%, respectively. It is unclear whether this reflects differential attitudes about the nature of those traits (e.g., admitting to feeling special and unique may seem less pejorative than being entitled or exploitative) or differences in how these NPD symptoms were operationalized in the PDQ-4.

Third, an examination of the general personality traits related to each NPD score suggested substantial overlap in terms of the general personality correlates of each (i.e., similarity score = .82) but with some important exceptions. As hypothesized, both NPD assessments were significantly negatively related to the character scores of SD (e.g., “irresponsible, purposeless, helpless, poor self-acceptance, poor impulse control,” Cloninger, 2000, p. 102) and CO (e.g., “intolerant, narcissistic, hostile, revengeful, opportunistic,” Cloninger, 2000, p. 102); these traits are thought to be general to all PDs (Svrakic et al., 1993). However, the two NPD scores diverged in important ways with regard to two of the temperament dimensions: NS and Harm Avoidance. Although both NPD scores were significantly correlated with NS (e.g., “easily bored, impulsive, quick-tempered, extravagant,” Cloninger, 2000, p. 103), the correlation was larger for the LEAD NPD scores. Alternatively, PDQ-4 NPD demonstrated a positive correlation with Harm Avoidance (e.g., “pessimistic, fearful, shy, anxious, and fatigable,” Cloninger, 2000, p. 103), whereas LEAD NPD scores were nonsignificantly related to this dimension. Both scores also demonstrated a positive relation with Self-transcendence, although only the relation with the PDQ-4 NPD scores was significant. This finding is unexpected (e.g., Ball et al., 1997; Svrakic et al., 1993) although not unprecedented (e.g., Bayon et al., 1996; Gutierrez, Sangorrin, MartinSantos, Torres, & Torrens, 2002).

The differential pattern of findings for the two NPD scores with NS and Harm Avoidance is important because it is consistent with the aforementioned concepts of grandiose and vulnerable narcissism in terms of what is shared across these variants and what is unique to each. Miller and Campbell (2008) argued that from an FFM perspective, grandiose and vulnerable narcissism are divergent due to the influence of the basic domains of Neuroticism and Extraversion, whereas the two overlap because of the role of interpersonal antagonism (in fact, we prefer the terms grandiose-agentic and grandiose-vulnerable to represent these distinctions as they account for the overlap in grandiosity and antagonism that is often found in both forms). The current results appear to be consistent with this perspective as the two narcissism measures are equally related to low interpersonal CO (i.e., low Agreeableness). Both are also negatively correlated with SD, which suggests that NPD is linked to the difficulties in the ability to “control, regulate, and adapt behavior to fit the situation in accord with individually chosen goals and values” (Cloninger et al., 1993, p. 979). The PDQ-4 and LEAD NPD profiles diverge on the factors of NS and Harm Avoidance. Only the PDQ-4 NPD scores were significantly positively related to Harm Avoidance, which is a known correlate of Neuroticism and Introversion from the FFM traits (De Fruyt, De Wiele, & Van Heeringen, 2000).

Overall, the personality correlates of the PDQ-4 NPD suggest that this scale may be assessing some traits that are consistent with the existing notions of vulnerable narcissism (e.g., Cain et al., in press; Wink, 1991) in which these individuals are interpersonally cold, immodest, deceptive, and low in empathy for others and also being prone to negative affective states such as depression, anger, and self-consciousness. For example, Hendin and Cheek (1997) examined the correlations between the FFM and the hypersensitive narcissism scale (HNS) and found that HNS scores were primarily positively correlated with Neuroticism (r = .51) and negatively correlated with Agreeableness (r = −.44) and Extraversion (r = −.28). This pattern is consistent with those found for the PDQ-4 NPD scale here and in the past examinations (i.e., Miller & Campbell, 2008), albeit the pattern is more muted with the PDQ-4 than the HNS (i.e., less negative affectivity and introversion). Combining the results of the past research with the current findings, it appears that the PDQ-4 NPD creates a profile of general personality trait correlates, which falls between measures of grandiose narcissism (i.e., Narcissistic Personality Inventory; Raskin & Hall, 1979, 1981) and measures of vulnerable narcissism (i.e., HNS). Alternatively, LEAD NPD scores were more strongly positively related to NS, which is conceptually and empirically similar to FFM Extraversion (e.g., r = .43; De Fruyt et al., 2000). Again, this profile of low interpersonal cooperativeness paired with an extroverted, reward-seeking tendency is similar to empirical (Miller & Campbell, 2008; Wink, 1991) and expert profiles (Lynam & Widiger, 2001) of grandiose narcissism/NPD.

In sum, although the overall personality profiles generated by the PDQ-4 and LEAD NPD scores are relatively similar, they diverge in some important ways as well, which suggest that the PDQ-4 NPD scale includes more of the vulnerable, hypersensitive content than assessments such as the LEAD NPD ratings or the Narcissistic Personality Inventory. This melding of both grandiose and vulnerable aspects in the PDQ-4 NPD scale may be responsible for the poor-to-moderate convergence typically found between this scale and other NPD assessment instruments. We believe that this difference was not the result of an intentional construction of the NPD scale in PDQ-4 but rather an inadvertent difference due to the wording of certain items on the PDQ-4 NPD scale that injects more fragility and need-iness than is found in the DSM-IV NPD criteria. It is important to note than an alternative explanation may be that PDQ-4 NPD profile does not reflect differences in the degree of grandiosity or vulnerability assessed but rather the limitations of PDQ-4 more broadly. Other studies (e.g., Wilberg et al., 2000) have demonstrated that the PDQ-4 manifests only weak-to-moderate convergence with PD ratings including LEAD scores, which suggests that these problems are not specific to NPD. For example, the PDQ-4 PD scores in general may be oversaturated with certain content such as Harm Avoidance (e.g., in the current study the median r between Harm Avoidance and PDQ-4 PDs = .36 vs. a median r between Harm Avoidance and LEAD PDs = .06). As such, the problems identified here with NPD may be but a small part of a larger problem.

Finally, we also examined the personality correlates of the individual NPD symptoms using both the PDQ-4 and LEAD NPD ratings in an attempt to understand the convergence and divergence found for these two NPD assessments. Several conclusions can be drawn from these findings. First, very few of the PDQ-4 NPD items were significantly related to NS, which is an important component of DSM-IV NPD. In addition, two PDQ-4 NPD items were significantly related to Harm Avoidance (positively) and SD (negatively), which together suggest a vulnerable, fragile personality configuration. The wording of these two items (PDQ-4 NPD Item 3: “Only certain people can really appreciate and understand me” and PDQ-4 NPD Item 4: “I very much need other people to take notice of me or compliment me.”) appears to pull for endorsement by individuals who may have more of the vulnerable form of narcissism. Other PDQ-4 items work poorly (i.e., poor convergence with specific LEAD NPD symptom and total score) because the items do not appear to assess their intended target. For example, in attempting to assess for a lack of empathy, the PDQ-4 NPD item—“People have often complained that I did not realize that they were upset”—is primarily negatively related to Reward Dependence and positively related to Self-transcendence. One can speculate that individuals who endorse this item may not lack empathy in the manner typical of individuals with NPD but instead may become so absorbed by their own thoughts and experiences that the feelings/needs of others are not recognized.

Limitations and Conclusions

The primary limitation of the current study involves the issue of shared method variance as the relations between PDQ-4 NPD scores and the TCI dimensions were both based on self-reports, which may have inflated these relations. In fact, it is possible that the differential correlations found between the NPD scores and some of the TCI dimensions may be due to the different methodologies used for the NPD scores. Ultimately, it would have been preferable to have collected TCI ratings from an alternative, nonoverlapping perspective (e.g., informant report). In addition, it is possible that the limited degree of narcissistic pathology present in the current sample may have attenuated our correlations. This issue is likely to be a problem for most studies of NPD, as the prevalence rate for NPD in this sample is very similar to the prevalence rates (2.3%) found in other studies using outpatients (Zimmerman, Rothschild, & Chelminski, 2005) and community samples (median = .05%; see Torgersen, 2005, for a review). In fact, Pagan, Eaton, Turkheimer, and Oltmanns (2006) found that narcissistic individuals are less likely to volunteer for research studies and that more extensive sampling procedures may be needed if researchers want to ensure adequate power for analyses involving these types of personality pathology.

In general, the NPD scores generated similar personality correlates, as measured by the TCI, but diverged on the domains of NS and Harm Avoidance. These differences occurred in a manner that suggests that the PDQ-4 NPD may measure both vulnerable and grandiose characteristics of narcissism, whereas the LEAD NPD scores assess only the characteristics consistent with grandiose narcissism (Dickinson & Pincus, 2003; Miller & Campbell, 2008; Wink, 1991). We believe that this is an important and clinically useful distinction to make, as these different forms of narcissism may have different etiologies and outcomes (Cain et al., in press; Miller & Campbell, 2008; Pincus et al., 2008). Individuals interested in assessing vulnerable narcissism might be best served using existing measures aimed at capturing only vulnerable narcissism (i.e., HNS) or both vulnerable and grandiose narcissism but using distinct subscales (i.e., Pathological Narcissism Inventory; Pincus et al., 2008). Although the PDQ-4 NPD scale seems to assess some of these vulnerable aspects, using it as an indicator of vulnerable narcissism may be problematic for two reasons: (a) These findings are still preliminary, and (b) the PDQ-4 NPD scale appears to be a blend of the two forms of narcissism rather than a pure measure of vulnerable narcissism. With regard to assessing the DSM-IV-based conceptualization of NPD, certain PDQ-4 NPD items need to be reworked if the PDQ-4 NPD is to converge more closely with measures of DSM-IV NPD. We believe that the current analytic strategy is an interesting and fruitful way to examine the similarity of two measures of the same construct in that one can examine “narrow” or specific convergence (i.e., PDQ-4 NPD scores correlated with LEAD NPD scores) as well as “broad” or general convergence (i.e., the two NPD scores create a similar nomological network of relations and, in this case, with general personality dimensions).

Acknowledgments

We thank Chuck Lance for his statistical consultation on this project.

References

  1. American Psychiatric Association . Diagnostic and statistical manual of mental disorders. 4th ed. Author; Washington, DC: 1994. [Google Scholar]
  2. Bagby RM, Marshall MB, Georgiades S. Dimensional personality traits and the prediction of DSM-IV personality disorder symptom count in a nonclinical sample. Journal of Personality Disorders. 2005;19:53–67. doi: 10.1521/pedi.19.1.53.62180. [DOI] [PubMed] [Google Scholar]
  3. Ball SA, Tennen H, Poling JC, Kranzler HR, Rounsaville BJ. Personality, temperament, and character dimensions and the DSM-IV personality disorders in substance abusers. Journal of Abnormal Psychology. 1997;106:545–553. doi: 10.1037//0021-843x.106.4.545. [DOI] [PubMed] [Google Scholar]
  4. Bayon C, Hill K, Svrakic DM, Przybeck TR, Cloninger CR. Dimensional assessment of personality in an out-patient sample: Relations of the systems of Millon and Cloninger. Journal of Psychiatric Research. 1996;30:341–352. doi: 10.1016/0022-3956(96)00024-6. [DOI] [PubMed] [Google Scholar]
  5. Cain NM, Pincus AL, Ansell EB. Narcissism at the crossroads: Phenotypic description of pathological narcissism across clinical theory, social / personality psychology, and psychiatric diagnosis. Clinical Psychology Review. doi: 10.1016/j.cpr.2007.09.006. in press. [DOI] [PubMed] [Google Scholar]
  6. Cheung MWL, Chan W. Testing dependent correlation coefficients via structural equation modeling. Organizational Research Methods. 2004;7:206–223. [Google Scholar]
  7. Cloninger CR. A practical way to diagnosis personality disorder: A proposal. Journal of Personality Disorders. 2000;14:99–108. doi: 10.1521/pedi.2000.14.2.99. [DOI] [PubMed] [Google Scholar]
  8. Cloninger CR, Przybeck T, Svrakic DM, Wetzel R. The Temperament and Character Inventory (TCI): A guide to its development and use. 1994 Unpublished manuscript. [Google Scholar]
  9. Cloninger CR, Svrakic DM, Przybeck TR. A psychobiological model of temperament and character. Archives of General Psychiatry. 1993;50:975–990. doi: 10.1001/archpsyc.1993.01820240059008. [DOI] [PubMed] [Google Scholar]
  10. De Fruyt F, De Clercq B, Van De Wiele L, Van Heeringen K. The validity of Cloninger's psychobiological model versus the five-factor model to predict DSM-IV personality disorders in a heterogeneous psychiatric sample: Domain facet and residualized facet descriptions. Journal of Personality. 2006;74:479–510. doi: 10.1111/j.1467-6494.2006.00382.x. [DOI] [PubMed] [Google Scholar]
  11. De Fruyt F, De Wiele LV, Van Heeringen C. Cloninger's psychobiological model of temperament and character and the five-factor model of personality. Personality and Individual Differences. 2000;29:441–452. [Google Scholar]
  12. De La Rie SM, Duijsens IJ, Cloninger RC. Temperament, character, and personality disorders. Journal of Personality Disorders. 1998;12:362–372. doi: 10.1521/pedi.1998.12.4.362. [DOI] [PubMed] [Google Scholar]
  13. Dickinson KA, Pincus AL. Interpersonal analysis of grandiose and vulnerable narcissism. Journal of Personality Disorders. 2003;17:188–207. doi: 10.1521/pedi.17.3.188.22146. [DOI] [PubMed] [Google Scholar]
  14. First MB, Gibbon M, Spitzer RL, Williams JBW. Structured clinical interview for DSM-IV axis I disorders (SCID-I) American Psychiatric Press; Washington, DC: 1997. [Google Scholar]
  15. Fossati A, Beauchaine TP, Grazioli F, Carretta I, Cortinovis F, Maffei C. A latent structure analysis of Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, narcissistic personality disorder criteria. Comprehensive Psychiatry. 2005;46:361–367. doi: 10.1016/j.comppsych.2004.11.006. [DOI] [PubMed] [Google Scholar]
  16. Fossati A, Maffei C, Bagnato M, Donati D, Donini M, Fiorilli M, et al. Criterion validity of the Personality Diagnostic Questionnaire-4+ (PDQ-4+) in a mixed psychiatric sample. Journal of Personality Disorders. 1998;12:172–178. doi: 10.1521/pedi.1998.12.2.172. [DOI] [PubMed] [Google Scholar]
  17. Gutierrez F, Sangorrin J, Martin-Santos R, Torres X, Torrens M. Measuring the core features of personality disorders in substance abusers using the Temperament and Character Inventory. Journal of Personality Disorders. 2002;16:344–359. doi: 10.1521/pedi.16.4.344.24125. [DOI] [PubMed] [Google Scholar]
  18. Hendin HM, Cheek JM. Assessing hypersensitive narcissism: A reexamination of Murray's narcissism scale. Journal of Research in Personality. 1997;31:588–599. [Google Scholar]
  19. Hilsenroth MJ, Handler L, Blais MA. Assessment of narcissistic personality disorder: A multi-method review. Clinical Psychology Review. 1996;16:655–683. [Google Scholar]
  20. Hyler SE. Personality Diagnostic Questionnaire-4. New York State Psychiatric Institute (NYSPI); 1994. Unpublished test. [Google Scholar]
  21. Klonsky ED, Oltmanns TF, Turkheimer E. Clinical Psychology: Science and Practice. Vol. 9. 2002. Informant reports of personality disorder: Relation to self-reports and future directions; pp. 300–311. [Google Scholar]
  22. Lilienfeld SO, Fowler KA. The self-report assessment of psychopathy: Problems, pitfalls, and promises. In: Patrick CJ, editor. Handbook of psychopathy. Guilford; New York: 2006. pp. 107–132. [Google Scholar]
  23. Lynam DR, Widiger TA. Using the five-factor model to represent the DSM-IV personality disorders: An expert consensus approach. Journal of Abnormal Psychology. 2001;110:401–412. doi: 10.1037//0021-843x.110.3.401. [DOI] [PubMed] [Google Scholar]
  24. McCrae RR. A note on some measures of profile agreement. Journal of Personality Assessment. doi: 10.1080/00223890701845104. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Miller JD, Campbell WK. Comparing clinical and social-personality conceptualizations of narcissism. Journal of Personality. 2008 doi: 10.1111/j.1467-6494.2008.00492.x. Available under “OnlineEarly Articles” at Blackwell Synergy Web site: http://www.blackwell-synergy.com/doi/abs/10.1111/j.1467-6494.2008.00492.x. [DOI] [PubMed] [Google Scholar]
  26. Miller JD, Hoffman B, Campbell WK, Pilkonis PA. An examination of the factor structure of DSM-IV Narcissistic Personality Disorder Criteria: One or two factors? Comprehensive Psychiatry. 2008;49:141–145. doi: 10.1016/j.comppsych.2007.08.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Miller JD, Pilkonis PA, Clifton A. Self and other-reports of traits from the five-factor model: Relations to personality disorder. Journal of Personality Disorders. 2005;19:400–419. doi: 10.1521/pedi.2005.19.4.400. [DOI] [PubMed] [Google Scholar]
  28. Millon T. Manual for the MCMI-III. National Computer Systems; Minneapolis, MN: 1994. [Google Scholar]
  29. Mulder RT, Joyce PR, Sullivan PF, Bulik CM, Carter FA. The relationship among three models of personality psychopathology: DSM-III-R personality disorder, TCI scores and DSQ defences. Psychological Medicine. 1999;29:943–951. doi: 10.1017/s0033291799008533. [DOI] [PubMed] [Google Scholar]
  30. Pagan JL, Eaton NR, Turkheimer E, Oltmanns TF. Peer-reported personality problems of research non-participants: Are our samples biased? Personality and Individual Differences. 2006;41:1131–1142. [Google Scholar]
  31. Pfohl B, Blum N, Zimmerman M. Structured interview for DSM-IV personality (SIDP-IV) American Psychiatric Press; Washington, DC: 1997. [Google Scholar]
  32. Pilkonis PA, Heape CL, Proietti JM, Clark SW, McDavid JD, Pitts TE. The reliability and validity of two structured diagnostic interviews for personality disorders. Archives of General Psychiatry. 1995;52:1025–1033. doi: 10.1001/archpsyc.1995.03950240043009. [DOI] [PubMed] [Google Scholar]
  33. Pincus AL, Ansell EB, Pimental CA, Cain NM, Wright AGC, Levy KN. Initial construction and validation of the Pathological Narcissism Inventory. 2008 doi: 10.1037/a0016530. Manuscript submitted for publication. [DOI] [PubMed] [Google Scholar]
  34. Raskin R, Hall CS. A narcissistic personality inventory. Psychological Reports. 1979;45:159–162. doi: 10.2466/pr0.1979.45.2.590. [DOI] [PubMed] [Google Scholar]
  35. Raskin R, Hall CS. The Narcissistic Personality Inventory: Alternate-form reliability and further evidence of construct validity. Journal of Personality Assessment. 1981;45:159–162. doi: 10.1207/s15327752jpa4502_10. [DOI] [PubMed] [Google Scholar]
  36. Samuel DB, Widiger TA. Clinicians' personality descriptions of prototypic personality disorders. Journal of Personality Disorders. 2004;18:286–308. doi: 10.1521/pedi.18.3.286.35446. [DOI] [PubMed] [Google Scholar]
  37. Saulsman L, Page A. The five-factor model and personality disorder empirical literature: A meta-analytic review. Clinical Psychology Review. 2004;23:1055–1085. doi: 10.1016/j.cpr.2002.09.001. [DOI] [PubMed] [Google Scholar]
  38. Spitzer RL. Psychiatric diagnosis: Are clinicians still necessary? Comprehensive Psychiatry. 1983;24:399–411. doi: 10.1016/0010-440x(83)90032-9. [DOI] [PubMed] [Google Scholar]
  39. Svrakic DM, Whitehead C, Przybeck TR, Cloninger CR. Differential diagnosis of personality disorders by the seven-factor model of temperament and character. Archives of General Psychiatry. 1993;50:991–999. doi: 10.1001/archpsyc.1993.01820240075009. [DOI] [PubMed] [Google Scholar]
  40. Torgersen S. Epidemiology. In: Oldham JM, Skodol AE, Bender DS, editors. Textbook of personality disorders. American Psychiatric Publishing; Washington, DC: 2005. pp. 129–141. [Google Scholar]
  41. Widiger TA, Coker LA. Assessing personality disorders. In: Butcher JN, editor. Clinical personality assessment: Practical approaches. 2nd ed. Oxford University Press; New York: 2001. pp. 407–434. [Google Scholar]
  42. Wilberg T, Dammen T, Friis S. Comparing Personality Diagnostic Questionnaire-4+ with longitudinal, expert, all data (LEAD) standard diagnoses in a sample with a high prevalence of axis I and axis II disorders. Comprehensive Psychiatry. 2000;41:295–302. doi: 10.1053/comp.2000.0410295. [DOI] [PubMed] [Google Scholar]
  43. Wink P. Two faces of narcissism. Journal of Personality and Social Psychology. 1991;61:590–597. doi: 10.1037//0022-3514.61.4.590. [DOI] [PubMed] [Google Scholar]
  44. Yang J, McCrae RR, Costa PT, Yao S, Dai X, Cai T, et al. The cross-cultural generalizability of axis II constructs: An evaluation of two personality disorder assessment instruments in the People's Republic of China. Journal of Personality Disorders. 2000;14:249–263. doi: 10.1521/pedi.2000.14.3.249. [DOI] [PubMed] [Google Scholar]
  45. Zimmerman M, Rothschild L, Chelminski I. The prevalence of DSM-IV personality disorders in psychiatric outpatients. American Journal of Psychiatry. 2005;162:1911–1918. doi: 10.1176/appi.ajp.162.10.1911. [DOI] [PubMed] [Google Scholar]

RESOURCES