Unpleasant and Pleasant Referential Thinking: Relations with Self- Processing, Paranoia, and Other Schizotypal Traits

Abstract

Referential thinking is the tendency to view innocuous stimuli as having a specific meaning for the self and is associated with personality traits and disorders. In three studies, this research examined the relations among referential thinking, self-processing, and paranoia. In study 1, follow-up questions on the Referential Thinking Scale (Lenzenweger, Bennett, & Lilenfeld, 1997) revealed that referential thoughts are experienced as unpleasant and pleasant. In Study 2, unpleasant referential thinking was more strongly associated with paranoia and maladaptive self-processing and personality. CFAs in Study 1 and 2 found that unpleasant and pleasant referential thinking loaded on different factors. In Study 3, a group of participants with elevated schizotypal personality reported more unpleasant and pleasant referential thoughts than a control group.

Referential thinking is a common feature of schizophrenia-spectrum conditions (Lenzenweger, et al., 1997), which includes schizophrenia, Cluster A personality disorders, which represent odd or eccentric behavior and include schizotypal, schizoid, and paranoid personality disorders (The American Psychiatric Association, 2000), and schizophrenia-like symptoms thought to reflect liability for schizophrenia (i.e., schizotypy; Chapman, Chapman, Raulin, & Edell, 1978; Meehl, 1962; Raine, 2006). Research has found that over two-thirds of people with schizophrenia experience delusions of reference (Frith, 1992), and ideas of reference are frequently reported in schizotypal personality disorder (Raine, 1991). Researchers interested in the development of schizophrenia have also suggested that irregularities in self-concept are one of the most important features of the onset of the disorder (e.g., Moller & Husby, 2000; Raballo, Saebye, & Parnas, 2009). At the same time, referential thinking might be related to other personality traits such as reliance on intuition (King & Hicks, 2009). Despite the potential importance of referential thinking in basic personality and schizophrenia-spectrum personality disorders, relatively little research has focused on what psychological mechanisms might contribute to referential thinking (Lenzenweger, et al., 1997).

Given that referential thoughts reflect viewing information as self-relevant, researchers have suggested that referential thinking might be related to problems in self-relevant information processing (Lenzenweger, et al., 1997). One self-processing variable that may be associated with referential thinking is self-esteem. Self-esteem is broadly defined as how people feel about themselves (Kernis, 2003). It is possible that referential thoughts might reflect either low or high self-esteem. For instance, people might have unpleasant referential thoughts, such as thinking they are being blamed by others, because of a low sense of self-worth. In contrast, some referential thoughts might reflect increased self-esteem. For instance, people might have pleasant referential thoughts, such as people playing songs on the radio just for them, because of a high sense of self-worth. However, to our knowledge no previous research has examined the relation between referential thinking and self-esteem.

In addition, based on previous research it is unclear whether referential thinking can be discriminated from paranoia. As mentioned, the central feature of referential thinking is the over-interpretation of stimuli as having a special meaning for the self (Lenzenweger, et al., 1997). Similarly, paranoia is the tendency to be inappropriately suspicious of other people’s motives and behaviors directed towards oneself (Fenigstein & Vanable, 1992). Although the term, “paranoia” has been used to denote any type of delusional thought (see Lewis, 1970 for an historical review), the current research uses the term to reflect the more narrow definition related to suspiciousness, distrust, and persecutory ideation (Combs & Penn, 2004; Fenigstein & Vanable, 1992). Researchers have hypothesized that both referential thinking and paranoia are personality traits that are associated with cluster A personality disorders and are related to self-relevant information processing (Lenzenweger, et al., 1997). Previous research has found that referential thinking is strongly correlated with paranoia (e.g., Meyer & Lenzenweger, 2009; Stefanis et al., 2004). Furthermore, research on the factor structure of schizotypy has found at least three factors: paranoid, cognitive-perceptual, and negative (e.g., Compton, Goulding, Bakeman, & McClure-Tone, 2009; Stefanis et al., 2004). Among these three factors, referential thinking has been found to most frequently load on the paranoid factor. However, in part due to limited measurement of referential thinking, none of these studies actually examined whether referential thinking could form a factor separate from paranoia. A recent study that could examine this found that referential thinking formed a factor separate from paranoia (Cicero & Kerns, 2010). Hence, this suggests that referential thinking and paranoia might be distinct constructs.

One issue in examining the relation between referential thinking with both self-processing variables and paranoia is that, as suggested by a number of psychopathologists, referential thinking could be multidimensional (Startup & Startup, 2005; Wing, Cooper, & Sartorious, 1974). In particular, referential thoughts might vary in terms of their experienced emotional valence. For example, the most comprehensive measure of referential thinking, The Referential Thinking Scale, was designed to include both positively and negatively valenced referential thoughts (Lenzenweger et al., 1997, Study 1). In contrast, paranoia might involve exclusively negatively valenced thoughts. This is because paranoia involves a threat to self. Hence, paranoid thoughts always involve some unpleasant emotional content. On the other hand, referential thoughts do not necessarily involve a threat to the self and could be either unpleasant or pleasant (Lenzenweger et al., 1997). For example, referential thinking may include unpleasant thoughts, such as “when I see something broken, I often wonder if people blame me for it.” However, it may also include pleasant thoughts, such as “when I hear a favorite song, I often wonder if it was written with me in mind.” Thus, although paranoia seems to always involve negatively valenced thoughts, referential thinking can refer to negatively or positively valenced thoughts. The current research builds on the work of Lenzenweger and colleagues (1997) by empirically testing whether referential thoughts can be experienced as positively valenced, as opposed to exclusively unpleasant.

The first goal of the current research was to empirically test whether referential thoughts are experienced as both unpleasant and pleasant. The second goal was to examine whether unpleasant and pleasant referential thoughts could be discriminated from each other and whether they could be discriminated from paranoia. Finally, the third goal of the current research was to examine whether unpleasant and pleasant referential thinking had differential relations with facets of self-relevant information processing, paranoia, other schizotypal personality traits, and Big 5 personality traits.

In the current research, we hypothesized that unpleasant and pleasant referential thoughts, although correlated, could be discriminated from each other and could be discriminated from paranoia. In addition, we expected to find that unpleasant referential thinking would be more strongly correlated with paranoia than pleasant referential thinking would be. We hypothesized that unpleasant referential thoughts would be associated with unpleasant self-relevant information processing including lower explicit and implicit self-esteem, higher self-consciousness, and lower facets of narcissism. In contrast, we expected to find that pleasant referential thoughts would be associated with higher implicit and explicit self-esteem, lower self-consciousness, and higher facets of narcissism. Finally, we expected to find that unpleasant referential thinking would be associated with maladaptive Big 5 personality traits while pleasant referential thinking would be more associated with adaptive personality traits. In general, we expected unpleasant referential thinking and paranoia to display similar relations with self-processing and big 5 personality.

The current research examined the relations among referential thinking, self-processing, paranoia, and other schizotypal characteristics in three studies. In Study 1, follow-up questions were added to the Referential Thinking Scale (Lenzenweger, et al., 1997) to determine whether items were sometimes experienced as pleasant as well as unpleasant. In Study 2, ratings of the items from Study 1 were used to create unpleasant and pleasant subscales of the Referential Thinking Scale and these subscales were used to examine the relations between unpleasant and pleasant referential thinking with self-processing, paranoia, other schizotypal characteristics, and Big 5 personality. In addition, we also tested a series of confirmatory factor analyses that examined whether unpleasant referential thinking, pleasant referential thinking, and paranoia could be discriminated from each other. Finally, in Study 3, we tested whether participants with elevated schizotypal personality had a higher level of both unpleasant and pleasant referential thoughts than a control group.

Study 1

Method

Participants

Participants (n = 348) were native English-speaking undergraduate college students at the University of Missouri who completed the study as partial completion of a course requirement. Twenty-six participants were excluded for having Chapman Infrequency scores of 3 or higher (see below), which resulted in a final sample of 322 participants. Participants ranged from 18-37 years old, with an average age of 19.16 (SD = 1.55). Participants were 47% female, 87.9% White, 9.0% African-American, 0.6% Asian-American, and 2.7% other.

Measures

Referential Thinking

The Referential Thinking Scale (REF; Lenzenweger et al., 1997) is a 34-item true-false questionnaire that measures referential thinking. For Study 1, the administration of the REF was modified to further assess the experience of referential thoughts. None of the items in the REF were modified. Instead, participants were asked two follow-up questions for each “true” response. First, they were asked, “to what extent was this experience positive?” on a scale from 0 (not at all positive) to 6 (extremely positive). Second, they were asked “to what extent was this experience negative?” on a scale from 0 (not at all negative) to 6 (extremely negative). This allowed for the calculation of unpleasant and pleasant referential thinking scores, by summing the 0-6 scores for the follow-up unpleasant and pleasant questions. Additionally, this modification made it possible to empirically examine the valence associated with specific referential thoughts.

Paranoia

Paranoia was measured with the Paranoia and Suspiciousness Questionnaire (Rawlings & Freeman, 1996), a 47 item yes-no questionnaire designed to measure paranoia in a non-psychiatric sample (e.g., Would you have been more successful if others around you had not put difficulties in your way?). The scale contains five subscales including interpersonal suspiciousness/hostility, negative mood/withdrawal, anger/impulsiveness, mistrust/wariness, and perceived hardship/resentment. The PSQ was developed from several existing paranoia scales: the PEN Psychoticism scale (Eysenck & Eysenck, 1975), the Paranoia scale of the MMPI (Hathaway & McKinley, 1989), the Buss Hostility scale (Buss & Perry, 1992), the 16PF Suspiciousness scale (Cattell, Eber, & Tatsuoka, 1970), and the STA Paranoid Ideation subscale (Hewitt & Claridge, 1989).

Infrequency

Participants also completed the Chapman Infrequency scale which measures careless or invalid responding (e.g., I cannot remember a time when I talked to a person wearing eyeglasses). The Chapman Infrequency scale is composed of questions that should rarely truthfully be answer in the affirmative. Based on previous research, 26 participants endorsing three or more items were excluded from the analysis (M. Chmielewski, Fernandes, Yee, & Miller, 1995).

Data Analysis

To test whether unpleasant referential thinking, pleasant referential thinking, and paranoia are distinct from each other, we compared the statistical fit of five confirmatory factor measurement models, using the sum of the valence scores for unpleasant and pleasant referential thinking. First, we tested a three-factor model in which unpleasant referential thinking, pleasant referential thinking, and paranoia all loaded on separate factors (Model 1). Second, we tested three two-factor models including: unpleasant referential thinking/paranoia, pleasant referential thinking (Model 2), unpleasant referential thinking/pleasant referential thinking, paranoia (Model 3), and unpleasant referential thinking, pleasant referential thinking/paranoia (Model 4). Finally, we tested a one factor model in which unpleasant referential thinking, pleasant referential thinking, and paranoia all loaded on a single factor (Model 5). We examined whether models with more factors exhibited significantly better fit than models with fewer factors.

All models were fit using Mplus3 software (Muthen & Muthen, 2004). Models were fit using maximum likelihood parameter estimates and with standard errors and a mean adjusted chi-square statistic that is robust to non-normality (the Satorra-Bentler χ2; Satorra & Bentler, 1994). χ2 difference tests of model comparisons were done using a scaled-difference test statistic (Satorra & Bentler, 2001). Models were also compared with the Incremental Fit Index (Tucker & Lewis, 1973; which is also referred to as the Tucker-Lewis Index), which compares the fit of models while adjusting for degrees of freedom. IFI values greater than 0.90 indicate substantial increases in model fit. In all models, the latent factors were allowed to correlate freely with each other. The latent factors were allowed to correlate freely, as opposed to being constrained to equal zero, because we expected to find that unpleasant referential thinking, pleasant referential thinking, and paranoia would be moderately to strongly correlated with each other. In most studies attempting to examine distinct schizotypy factors, the, factors are specified to correlate freely (e.g., P. M. Chmielewski & Watson, 2008; Kwapil, Barrantes-Vidal, & Silvia, 2008; Raine et al., 1994; Stefanis et al., 2004). Thus, latent factors may be correlated and still considered to be distinct constructs. Four test statistics were used to assess whether models provide a good fit to the data (Hu & Bentler, 1998): (a) χ2/df ratio < 2.5, (b) CFI (comparative fit index) > .95, (c) RMSEA (root mean squared error of approximations) < .08, and (d) SRMR (standardized root mean square residual) < .05.

In order to more accurately measure unpleasant referential thinking, pleasant referential thinking, and paranoia, each of the scales were randomly divided into three facets. For example, items 1, 4, 7 etc. were summed to create the first unpleasant referential thinking facet, items 2, 5, 8, etc. comprised the second facet, and items 3, 6, 9, etc. comprised the third facet. Previous research has used similar techniques to examine the factor structure of similar constructs including schizotypy (Kwapil, et al., 2008) and self-consciousness (Lischetzke & Eid, 2003). Monte Carlo studies have found that this method for measuring constructs is more valid than using manifest variables (Alhija & Wisenbaker, 2006). Additionally, in model testing, the errors of the manifest variables for the pleasant and unpleasant ratings of the referential thoughts were specified to be freely correlated with each other. This was done because unpleasant and pleasant referential thinking items shared important method variance (e.g., scores for unpleasant referential thinking item 1 and for pleasant referential thinking item 1 were based on initially endorsing having experienced the same referential thinking item).

Results

Unpleasant and Pleasant Referential Thoughts

As can be seen in Table 1, 14 items were experienced as more unpleasant than pleasant and 20 items were experienced as more pleasant than unpleasant. The most unpleasant experience was “traffic lights usually turn red because I am driving in a hurry.” Other relatively unpleasant experiences included participants feeling like they were being blamed for things, feeling like people say unpleasant things about the participant while in private conversations (e.g., laughing as the participant walks by, two people criticizing the participant), and noticing things about the participant that the participant tried to hide. The most pleasant referential experiences included strangers waving at the participant, radio DJs playing songs specifically for the participant, favorite songs written with the participant in mind, and others imitating the participant’s style of dressing.

Table 1.

Mean difference scores of pleasant ratings minus unpleasant ratings

	Mean Difference Score
I have noticed that people I do not know often wave at me.	3.46
I often wonder if radio DJs play songs just for me.	3.42
When I hear a favorite song, I think that it was probably written with me in mind.	3.18
It seems to me that other people often imitate my style of dressing	3.16
People I do not know often notice how I dress.	2.71
I have read books that seem to have been written about me.	2.65
I am not sure why, but people often seem to pay a lot of attention to me.	2.39
Professors (or speakers) often seem to direct their lectures to me.	2.38
Small animals seem to take special notice of me as I walk by.	2.36
I think others often imitate my manner of speaking	2.34
Films often seem to be very similar to my life story	2.05
I sometimes think that newspaper articles contain messages for me.	2.02
Strangers often smirk at me.	1.99
I do not think that people on the street pay special attention to me.	1.79
I often think other comment to each other about my clothing.	1.53
I often wonder if people are in a class because I am there.	1.50
I often feel that people are looking at me.	0.98
Even if they do not say it, it seems to me that other people are always wondering how smart I am.	0.73
I often think that people talk about me when I walk down the street.	0.23
Dogs seem to bark a lot when I am near.	0.02
People often fidget in their seats when I enter a room.	−0.34
When I am on a train or bus, it seems that people often watch me closely	−0.54
I often wonder why so many people leave the highway using the same exit that I use	−0.63
When I overhear a conversation, I often wonder if people are saying bad things about me.	−0.93
If I see someone laughing, I often wonder if they are laughing at me.	−1.07
When I hear two people speaking a foreign language, I often think they are commenting on my behavior.	−1.25
Quite often I wonder if people are laughing as I walk by.	−1.83
When I feel ashamed, I think others often know why I feel that way.	−1.92
I often think that people are making accusations about my behavior	−1.95
People almost always notice the pars of my personality or character that I try to hide.	−2.05
When I see two people talking at work, I usually think they are criticizing me.	−2.16
Although I know deep down inside it is not true, I often feel that others blame me for things.	−2.84
When I see something broken, I often wonder if others blame me for it.	−2.97
Traffic lights usually turn red because I am driving in a hurry	−3.36

Discriminability of Unpleasant Referential Thinking, Pleasant Referential Thinking, and Paranoia

As can be seen in Table 2, the three-factor model (Model 1) with separate factors fit the data well and fit the data significantly better than all of the other models according to the chi-square difference test and the Incremental Fit Index.¹ None of the other models fit the data even moderately well. Thus, it appears that unpleasant referential thinking, pleasant referential thinking, and paranoia may be distinct constructs.

Table 2.

Fit Statistics for Confirmatory Factor Analysis Measurement Models of Referential Thinking and Paranoia in Study 1.

Model	χ2	df	CFI	RMSEA	SRMR	χ2diff (vs. Model 1)	IFI (vs. Model 1)
Model 1	26.47	21	0.99	0.03	0.03	-
Model 2	121.14	23	0.82	0.12	0.14	14.94**	0.94
Model 3	176.70	23	0.73	0.15	0.11	48.32**	0.96
Model 4	206.14	23	0.68	0.17	0.13	183.22**	0.97
Model 5	334.70	24	0.45	0.21	0.15	81.04**	0.98

Model 1: 3-factor model (unpleasant referential thinking, pleasant referential thinking, paranoia); Model 2: 2-factor model (unpleasant referential thinking/paranoia, pleasant referential thinking);

Model 3: 2-factor model (unpleasant referential thinking/pleasant referential thinking, paranoia);

Model 4: 2-factor model (unpleasant referential thinking, pleasant referential thinking/paranoia);

Model 5: 1-factor model (unpleasant referential thinking/pleasant referential thinking/paranoia); χ2diff = Satorra-Bentler chi-square difference test. Significant difference represents worse model fit. IFI = Incremental Fit Index.

In model 1, unpleasant and pleasant referential thinking were positively correlated ( r = .71), unpleasant referential thinking and paranoia were positively correlated ( r = .57), and pleasant referential thinking and paranoia were positively correlated ( r = .36). In Model 2, pleasant referential thinking was positively correlated with the unpleasant referential thinking/paranoia factor ( r = .53). In Model 3, the unpleasant/pleasant referential thinking factor was positively correlated with the paranoia factor ( r = .55). In Model 4, the pleasant referential thinking/paranoia factor was positively correlated with the unpleasant referential thinking factor ( r = .71).

Discussion

The first goal of Study 1 was to examine whether referential thoughts were experienced as pleasant in addition to unpleasant. Indeed, Study 1 found that there was a great deal of variability in the valence associated with the referential thoughts. This is consistent with the original conceptualization of the REF, which was designed to include both positively valenced and negatively valenced items (Lenzenweger et al., 1997). Study 1 also provided some evidence that unpleasant referential thinking, pleasant referential thinking, and paranoia are all correlated but distinct from one another. The best fitting CFA model included separate unpleasant referential thinking, pleasant referential thinking, and paranoia factors. In addition to the results of confirmatory factor analysis, if unpleasant referential thinking and pleasant referential thinking are distinct constructs, they should display differential associations with other theoretically meaningful variables.

Study 2

In Study 1, we found that referential thoughts could be experienced as both unpleasant and pleasant and that these thoughts could be discriminated from each other. The goal of Study 2 was to use the information about the valence of referential thoughts from Study 1 to examine the relations between unpleasant and pleasant referential thoughts with paranoia, self-processing, schizotypal personality, and Big 5 personality in a separate sample. Based on the valence ratings from Study 1, pleasant and unpleasant subscales of the Referential Thinking Scale were created and the correlations between scores on these subscales and other variables were examined.

Method

Participants

Participants (n = 347) were native English-speaking undergraduate college students at the University of Missouri who completed the study as partial completion of a course requirement. Following previous research, participants (n = 35) were excluded due to Chapman infrequency scores of 3 or greater (Chapman & Chapman, 1983). In addition, 17 participants were excluded due to failing to complete all the questionnaires, resulting in 295 useable participants. Participants ranged from 18-42 years old, with an average age of 18.87 (SD = 1.85). Participants were 59% female, 90.1% White, 5.8% African-American, 2.0% Asian-American, and 1.7% other. One participant declined to specify ethnicity.

Measures. Referential Thinking

Participants completed the Referential Thinking Scale (Lenzenweger et al., 1997), and participants were not asked about the valence of their experiences in this study.

Paranoia

Four measures of paranoia were administered in Study 2. One measure was the Paranoia and Suspiciousness Questionnaire (Rawlings & Freeman, 1996) as in Study 1. A second paranoia measure was the 8-item Suspiciousness subscale from the Schizotypal Personality Questionnaire (SPQ-S; Raine, 1991; e.g., Do you sometimes get concerned that friends or coworkers are not really loyal or trustworthy?). Overall, the full Schizotypal Personality Questionnaire (SPQ; Raine, 1991) is a 74-item yes-no questionnaire designed to measure DSM-III-R schizotypal personality disorder. The SPQ has been the most frequently used scale in studies examining the factor structure of schizotypy traits (e.g., Stefanis, et al., 2004).

A third paranoia measure was the Suspiciousness subscale of the Dimensional Assessment of Personality Pathology-Basic Questionnaire (DAPP-BQ; Livesley & Jackson, 2002), which includes 14 items (e.g., When people do something nice for me, I wonder what their real motives are). Participants answer on a scale from 1 = very unlike me to 5 = very like me. The DAPP-BQ Suspiciousness subscale has been shown to be highly correlated with a count of DSM-IV paranoid personality disorder symptoms (r = .67; Bagge & Trull, 2003).

The fourth paranoia measure was the Paranoid Personality Disorder Features Questionnaire (PPDFQ; Useda & Trull, 2002), a 23-item questionnaire (e.g., I am careful about the way I act around other people because they may take advantage of me). Participants rate statements on a scale from 0 = strongly disagree to 4 = strongly agree. Two items are reverse coded, with higher scores reflecting higher paranoid personality disorder characteristics. The scale contains six subscales measuring suspiciousness/mistrust, antagonism, autonomy, hypersensitivity, hypervigilence, and rigidity. Useda and Trull (2002) found that the PPDFQ is highly correlated (r = .78) with the DAPP-BQ Suspiciousness Subscale. Since the paranoia scales were highly correlated with each other (rs ranged from 0.61 to 0.76), a composite paranoia score was calculated by taking the mean of the standardized z-score for all four measures.

Explicit Self-Esteem

Explicit self-esteem was measured with the Rosenberg Self-Esteem Scale (RSES; Rosenberg, 1965), a 10-item Likert scale ranging from 1 = strongly agree to 4 = strongly disagree (e.g., I feel that I am a person of worth, at least on an equal plane with others). Several items are reverse scored. The RSES has been shown to have high internal consistency and test-retest reliability (Rosenberg, 1965), is highly associated with other measures of explicit self-esteem (e.g., Demo, 1985; Tafarodi & Swann, 1995), and may be the most commonly used measure of trait self-esteem (Leary, Tambor, Terdal, & Downs, 1995).

Implicit Self-Esteem

Implicit self-esteem was measured with the self-esteem Implicit Association Test (IAT; Greenwald & Farnham, 2000). The self-esteem IAT has been found to have the highest test-retest reliability of all existing measures of implicit self-esteem (Bosson, Swann, & Pennebaker, 2000). Moreover, implicit self-esteem, as measured with the IAT, has been found to predict different outcomes than self-esteem assessed with explicit measures (Bosson, Brown, Zeigler-Hill, & Swann, 2003; de Jong, 2002; Schimmack & Diener, 2003).

Self-esteem was measured both explicitly and implicitly because previous research suggests that explicit and implicit self-esteem may be differentially related to facets of schizotypy, particularly paranoia. For example, some research suggests that paranoia is associated with a discrepancy between high explicit self-esteem and low implicit self-esteem (e.g., Bentall, Kaney, & Dewey, 1991; Bentall, Kinderman, & Kaney, 1994), while other research suggests that paranoia is associated with both decreased explicit and implicit self-esteem (e.g., Freeman, 2007). No previous research has examined whether referential thinking is associated with implicit self-esteem. The current research did not measure other variables on an implicit level because previous research has not suggested that they are associated with paranoia, referential thinking, or other facets of schizotypy on an implicit level.

Self-Consciousness

Self-consciousness was measured using the 23-item Self-Consciousness Scale (SCS; Fenigstein, Scheier, & Buss, 1975). The scale was administered as a true-false questionnaire. It contains subscales for public self-consciousness (e.g., I’m concerned about what other people think of me) and private self-consciousness (e.g., I’m always trying to figure myself out). This self-consciousness scale has been used in previous research examining associations between self-consciousness and paranoia (e.g., Combs & Penn, 2004; Lenzenweger et al., 1997).

Narcissism

Narcissism represents relatively normal but disordered self-processing characterized by a pattern of grandiosity and entitlement, and is strongly associated with self-esteem (Rodebaugh, Woods, & Heimberg, 2007; Sedikides, Rudich, Gregg, Kumashiro, & Rusbult, 2004). The Narcissistic Personality Inventory (NPI; Raskin & Terry, 1988) was used to measure narcissism. The NPI is a 40-item questionnaire (e.g., If I ruled the world it would be a much better place), that is commonly used to measure narcissism. It has been found to be correlated with staff and clinician ratings of narcissism in clients, ratings of narcissistic behavior in an experimental discussion task, and with dominance and sociability scores (two personality characteristics thought to be strongly related to narcissism; Raskin & Terry, 1988). Previous research suggests that the NPI may be multidimensional and composed of at least two factors (Corry, Merritt, Mrug, & Pamp, 2008; Rodebaugh, et al., 2007). A recent item-level confirmatory factor analysis of the NPI compared the fit of competing 2-, 3-, 4-, and 7-factor models and concluded that a two-factor model was the most parsimonious and provided the best fit to the data (Corry, et al., 2008). Additionally, subscale scores based on the two factor model have high internal consistency and are recommended for use by Corry et al. These two factors have been termed Leadership/Authority and Exhibitionism/Entitlement. Previous research has found that leadership/authority may be a more covert facet of narcissism and is strongly related to extraversion, dominance, social boldness, and high self-esteem. Conversely, exhibitionism/ entitlement may represent more overt narcissism and may be more strongly related to achievement, tension, anxiety, and suspiciousness (Corry et al., 2008). If unpleasant referential thinking is associated with low self-esteem and maladaptive personality, then we would expect to find that it would not be associated with leadership/authority but would be associated with exhibitionism/entitlement. In contrast, if pleasant referential thinking is associated with high self-esteem, then we would expect to find that it would be associated with both leadership/authority and exhibitionism/entitlement. As can be seen in Table 3, these two subscales of the NPI were highly correlated with each other and had high internal reliability.

Table 3.

Correlations among unpleasant and pleasant referential thinking and other variables in Study 2.

	1	2	3	4	5	6	7	8	9	10
1. Unpleasant Referential Thinking	.75
2. Pleasant Referential Thinking	.61*	.75
3. Paranoia	.62*	.49*	-
4. Rosenberg Self- Esteem	−.36*	.08	−.47*	.88
5. Implicit Self-Esteem	−.01	.12*	.02	.04	.93
6. Self-Consciousness	.31*	.18*	.29*	−.27*	.04	.60
7. NPI- Leadership/ Authority	−.10	.16*	−.01	.40*	.01	.13*	.78
8. NPI- Entitlement/ Exhibition	.19*	.46*	.24*	.17*	.02	.17*	.47*	.76
9. Magical Ideation Scale	.43*	.44*	.46*	−.17*	.03	.04	.15*	.30*	.83
10. Perceptual Aberration Scale	.45*	.29*	.48*	−.33*	.01	.06*	.01	.19*	.67*	.83
Mean	2.30	3.05	0	32.14	0.12	4.96	5.95	5.63	5.45	3.73
Standard Deviation	2.42	2.93	1	4.75	0.07	1.77	2.33	3.21	4.63	4.07
Range	0-13	0-14	−1.66- 2.89	13-40	−0.10- 0.31	0-8	0-9	0-13	0-25	0-27
Skewness	1.59	1.22	0.62	−0.36	0.01	−0.53	−0.69	0.32	1.34	2.49
Kurtosis	2.78	1.26	0.08	0.17	0.04	−0.59	−0.41	−0.65	2.25	8.82

^*p < .05. The numbers on the diagonal are Cronbach’s Alpha.

Other Schizotypal Personality Characteristics

There were two scales used to measure other schizotypal personality characteristics. One scale was the Magical Ideation Scale (Eckbald & Chapman, 1983), a 30-item true-false questionnaire designed to measure “beliefs in forms of causation that by conventional standards are invalid” (Eckbald & Chapman, 1983, p.215). For example, “I have worried that people on other planets may be influencing what happens on Earth.” The Magical Ideation scale has considerable support for its reliability and validity ( for a review, see Edell, 1995). A second schizotypal personality scale was the Perceptual Aberration Scale (Chapman, Chapman, & Raulin, 1978), a 35-item true false scale that includes 28 items designed to measure schizophrenic-like distortions in perception of one’s own body and seven items for other perceptual distortions (e.g., my hearing is sometimes so sensitive that ordinary sounds become uncomfortable). The Perceptual Aberration Scale also has considerable support for its reliability and validity (for a review, see Edell, 1995). The authors of these scales also referred to them as measures of “psychosis-proneness,” and both measures have been found to predict future onset of psychosis (e.g., Chapman, Chapman, Kwapil, Eckblad, & Zinser, 1994).

Big-Five Personality Characteristics

If unpleasant referential thinking is associated with decreased self-esteem and with increased paranoia, then we would expect to find that unpleasant referential thinking would be associated with maladaptive personality characteristics. Conversely, if pleasant referential thinking is associated with increased self-esteem and less strongly associated with paranoia, then we would expect that pleasant referential thinking would be associated with adaptive personality characteristics. Big-five personality characteristics were measured with the 100-item International Personality Item Pool (IPIP; Goldberg, 1999) with five 20-item subscales for each of the five factors of personality: neuroticism (e.g., I get stressed out easily), extroversion (e.g., I am the life of the party), openness to experience (e.g., I have a vivid imagination), agreeableness (e.g., I sympathize with other people’s feelings), and conscientiousness (e.g., I am always prepared). Participants rate their agreement with items on a 5 item Likert scale from 1 = very inaccurate to 5 very accurate.

Procedure

Participants first completed the Self-Esteem Implicit Association Test. Then they completed the Referential Thinking Scale, the Public Self-Consciousness Subscale of the Self-Consciousness Scale, and the Paranoia and Suspiciousness Questionnaire randomly mixed together. Then participants completed the Paranoid Personality Disorders Features Questionnaire, Survey of Attitudes and Experiences (Composed of the Magical Ideation Scale, Perceptual Aberration Scale, Revision Social Anhedonia Scale, and Infrequency Scale), DAPPBQ Suspiciousness subscale, and the Rosenberg Self-Esteem Scale. Participants completed the study in one occasion in an isolated room, with the entire study taking approximately 90 minutes.

Results

Discriminability of Unpleasant and Pleasant Referential Thoughts

To further test whether unpleasant referential thinking could be discriminated from pleasant referential thinking, we conducted item-level confirmatory factor analyses on the REF items. Items that were rated as more unpleasant than pleasant in Study 1 were specified to load on the first factor (i.e., the unpleasant factor) and items that were rated as more pleasant than unpleasant were specified to load on the second factor (i.e., the pleasant factor). Then, the fit of this model was compared to the fit of a single factor model in which all of the items loaded on a single “referential thinking” factor. Using Mplus’ categorical variable option and weighted least square mean and variance (WLSMV) estimation, the fit of these two models were compared. We used WLSMV as opposed to ML as in Study 1 because ML estimation cannot be used with categorical variables. The latent variables were allowed to correlate freely because we expected unpleasant and pleasant referential thinking to be correlated with each other. Both the two-factor model (χ²/df = 2.02, CFI = 0.86, RMSEA = 0.06, SRMR = 0.12) and the one-factor model fit the data reasonably well (χ²/df= 2.05, CFI = 0.86, RMSEA = 0.06, SRMR = 0.16). The unpleasant and pleasant referential thinking factors were strongly correlated with each other ( r = .93). A standard chi-square difference test cannot be used with WLSMV estimation because the difference between chi-square values for two models is not distributed as chi-square using this estimation method. Thus, the difftest command in Mplus, which uses derivatives to correct for this distribution (Asparouhov & Muthen, 2006), was used to compare the fit of the more restrictive model (i.e., the one factor model) to the fit of the less restrictive model (i.e., the two factor model). The resulting value can be interpreted like a standard chi-square difference test. The resulting χ² was significant (χ²diff (1) = 11.48, p < .001), which suggests that restricting all the items to load on a single factor, as opposed to two factors, worsened the fit of the model. In turn, this suggests that referential thinking may be composed of both an unpleasant-valence factor and a pleasant-valence factor that are distinct but highly correlated.

One explanation for the finding that a factor model with separate pleasant and unpleasant factors fit the data better than a single factor model could be that instead of tapping different latent constructs, our factors represent groups of items with different levels of item difficulty. If this were the case, we would expect there to be a significant difference in the percentage of the population endorsing the unpleasant items than the percentage of participants endorsing the pleasant items. There was not a significant difference in the percentage of participants endorsing the unpleasant items compared to percentage of participants endorsing the pleasant items ( M = 26.2%, SD = 0.16 vs. M = 25.2%, SD = 0.16, t (32) = .20, p = .85).

Paranoia

As can be seen in Table 3, unpleasant referential thinking was more strongly correlated with paranoia than was pleasant referential thinking. To test whether the difference between the correlations was significant, we computed a Z-score for the difference between correlated correlation coefficients as suggested by Meng, Rosenthal, and Rubin (1992). Unpleasant referential thinking was more strongly correlated with paranoia than was pleasant referential thinking (Z = 3.19, p = .001). To further test whether unpleasant referential thinking was more strongly correlated with paranoia than was pleasant referential thinking, unpleasant referential thinking and pleasant referential thinking were simultaneous ly entered into a multiple regression equation predicting paranoia. These results can be interpreted as the relation between unpleasant referential thinking and paranoia after removing shared variance with pleasant referential thinking and the relation between pleasant referential thinking and paranoia after removing variance shared with unpleasant referential thinking. In this regression analysis, unpleasant referential thinking seemed even more strongly predictive of paranoia than was pleasant referential thinking (β = .55 vs. .17).

Explicit Self-Esteem

As can also be seen in Table 3, unpleasant referential thinking was associated with low explicit self-esteem, while pleasant referential thinking was not significantly associated with explicit self-esteem (Z = 8.41, p < .001). When entered separately into a multiple regression, unpleasant referential thinking was negatively associated with explicit self-esteem (β = −.44, p < .001) and pleasant referential thinking was associated with increased explicit self-esteem (β = .19, p < .01).

Implicit Self-Esteem

Unpleasant referential thinking was not significantly associated with implicit self-esteem, but pleasant referential thinking was associated with increased implicit self-esteem (Z = 2.51, p = .01). Moreover, when removing shared variance with pleasant referential thinking, unpleasant referential thinking was still unassociated with implicit self-esteem (β = −.09, p = .25) and pleasant referential thinking was still positively associated with implicit self-esteem (β = .16, p = .04).²

Self-Consciousness

As shown in Table 3, both unpleasant and pleasant referential thinking were associated with increased public self-consciousness, but unpleasant referential thinking was more strongly correlated with public self-consciousness than was pleasant referential thinking (Z = 2.61, p = .008). When entered simultaneously into a multiple regression, unpleasant referential thinking was associated with increased public self-consciousness (β = .32, p < .001), while pleasant referential thinking was not significantly associated with public self-consciousness (β = −.02, p = .82).

Facets of Narcissism

As can be seen in Table 3, unpleasant referential thinking was significantly correlated with the exhibition/entitlement facet of narcissism, but not the leadership/authority facet. Pleasant referential thinking was more strongly associated with both the exhibitionism/entitlement facet (Z = 5.51, p < .001) and the leadership/authority facet (Z = 4.99, p < .001) than was unpleasant referential thinking. When entered simultaneously into a multiple regression equation, unpleasant referential thinking was negatively associated with exhibitionism/entitlement (β = −.17, p = .01), while pleasant referential thinking was positively associated with exhibitionism/entitlement (β = .56, p < .001). Similarly, unpleasant referential thinking was negatively associated with leadership/authority (β = −.34, p < .001) and pleasant referential thinking was positively associated with leadership/authority (β = .37, p < .001)

Schizotypal Personality

Unpleasant and pleasant referential thinking were both positively correlated with magical ideation and perceptual aberration. However, unpleasant referential thinking was more strongly correlated with perceptual aberration than was pleasant referential thinking (Z = 3.64, p < .001). When entered simultaneously into a multiple regression, both unpleasant (β = .28, p < .001) and pleasant referential thinking were still associated with magical ideation (β = .24, p < .001). Unpleasant referential thinking was still associated with perceptual aberration (β = .38, p < .001) when removing variance shared with pleasant referential thinking, but pleasant referential thinking was not (β = .05, p = .45).

Big Five Personality

As can be seen in Table 4, unpleasant referential thinking was associated with decreased extraversion, agreeableness, conscientiousness, and openness to experience, but increased neuroticism. In contrast, pleasant referential thinking was only associated with increased neuroticism, although not as strongly as was unpleasant referential thinking. These correlations were significantly different for neuroticism (Z = 2.93, p = .003), extraversion (Z = 5.15, p < .001), agreeableness (Z = 2.11, p = .04), conscientiousness (Z = 2.11, p = .04), and openness to experience (Z = 4.04, p < .001). When removing variance shared with unpleasant referential thinking, pleasant referential thinking was associated with increased extraversion (β = .29, p < .001) and openness to experience (β = .26, p < .001). After removing variance with pleasant referential thinking, unpleasant referential thinking was still associated with decreased extraversion (β = −.42, p < .001), agreeableness (β = −.22, p < .001), conscientiousness (β = −.24, p < .001), and openness to experience (β = −.32, p < .001), but increased neuroticism (β = .39, p < .001).

Table 4.

Zero-order correlations among referential thinking scales and Big Five personality characteristics in Study 2

	1	2	3	4	5	6	7
1. Unpleasant Referential Thinking	.75
2. Pleasant Referential Thinking	.61*	.75
3. Neuroticism	.42*	.28*	.93
4. Extraversion	−.26*	.01	−.35*	.91
5. Agreeableness	−.20	−.09	−.31*	.44*	.86
6. Openness to Experience	−.16*	.05	−.20	.41*	.42*	.87
7. Conscientiousness	−.18*	−.07	−.30*	.26*	.37*	.22*	.88
Mean	2.30	3.05	3.22	3.37	3.79	3.54	3.16
Standard Deviation	2.42	2.93	0.66	0.71	0.50	0.53	0.37

^*p < .05. The numbers on the diagonal are Cronbach’s Alpha.

Study 2 Discussion

Study 2 provided further evidence that unpleasant and pleasant referential thoughts could be discriminated from each other in a separate sample from Study 1. A confirmatory factor analysis with two factors, in which items rated as being more pleasant than unpleasant loaded on one factor and items rated as more unpleasant than pleasant loaded on a second factor, fit the data better than a CFA in which all the items loaded on a single factor. This suggests that unpleasant and pleasant items may be correlated but distinct.

Additionally, the results of Study 2 largely conformed to our hypotheses about the relations between unpleasant and pleasant referential thinking with paranoia, self-processing, other schizotypal personality characteristics, and Big 5 Personality Traits. As hypothesized, unpleasant referential thinking was more strongly correlated with paranoia than was pleasant referential thinking, which was found when shared variance was and was not removed. Moreover, unpleasant referential thinking was associated with lower explicit self-esteem and higher public self-consciousness than was pleasant referential thinking. In contrast, pleasant referential thinking was associated with increased implicit self-esteem, whereas unpleasant referential thinking was not. Unpleasant referential thinking was associated with personality traits that are generally considered to be maladaptive, while there was some evidence that pleasant referential thinking was associated with personality traits that are generally considered to be adaptive. Overall, these results suggest that unpleasant referential thinking is associated with more unpleasant biases in self-relevant information processing and maladaptive personality traits, while pleasant referential thinking is more associated with pleasant or the absence of biases in self-relevant information processing and more adaptive personality traits. In the current research, only self-esteem was measured on an implicit level. Future research could examine the relations among unpleasant referential thinking, pleasant referential thinking, other schizotypal traits, and Big Five personality measured implicitly.

Finally, Study 2 found that both unpleasant and pleasant referential thinking were associated with measures of schizotypal personality, but that unpleasant referential thoughts may be more strongly associated with some other schizotypal personality characteristics than are pleasant referential thoughts. The finding that both unpleasant and pleasant referential thoughts were correlated with schizotypal traits suggests that people with schizotypy may have elevated referential thoughts regardless of the valence of these thoughts. However, one limitation of Study 2 is that they involved unselected college student samples. Thus, it is not clear how unpleasant and pleasant referential thoughts are experienced among people with more clinically meaningful schizotypal symptoms. In Study 3, we administered the Referential Thinking Scale to a sample of participants with extreme levels of schizotypy and compared them with a control group.

Study 3

The main goal of Study 3 was to examine whether a group of psychometrically identified participants who have elevated schizotypy and are at increased risk for psychosis (Chapman, et al., 1994) would have more unpleasant and pleasant referential thoughts than a control group. In addition to examining risk for psychosis dimensionally by correlations with the Magical Ideation and Perceptual Aberration Scales as in Study 2, schizotypy researchers have often used a “high risk” approach to examining the correlates of schizotypy (e.g., Chapman, et al., 1994; Gooding, Tallent, & Matts, 2005; Lenzenweger, 1994; Miller, 1995). This approach consists of identifying participants with extremely high scores on the Magical Ideation and Perceptual Aberration Scales and comparing these participants to a control group of participants with relatively low scores on both of these scales. In Study 3, we used this high risk approach to complement and extend the results of Study 2.

If unpleasant and pleasant referential thinking are both associated with other schizotypal personality characteristics, then we would expect to find that a group of participants with extreme levels of schizotypy would have increased unpleasant and pleasant referential thoughts. However, if only unpleasant or only pleasant referential thoughts are associated with other schizotypal personality characteristics, then we would expect to find that only unpleasant or only pleasant referential thoughts would be elevated in the schizotypal sample.

Method

Participants

Participants were 55 (24 Schizotypal and 31 Control) undergraduate college students at the University of Missouri who were recruited from a large pool of participants (n = 1,901) who had completed a screening battery of questionnaires in partial fulfillment of a course requirement. The questionnaires included abbreviated versions of the Magical Ideation Scale (Eckbald & Chapman, 1983) and Perceptual Aberration Scale (Chapman, Chapman, & Raulin, 1978). Participants completed this battery online during a one week period. Based on the results of the screening measure, we recruited people who scored 1.96 standard deviations above the mean on the abbreviated versions of the Magical Ideation Scale or Perceptual Aberration Scale or a combined 3 standard deviations above the mean on the Magical Ideation and Perceptual Aberration Scale to participate in an individual testing session. We also recruited control participants who scored below 0.5 standard deviations above the mean on the Magical Ideation Scale, Perceptual Aberration Scale, and Social Anhedonia Scale (Chapman, Chapman, & Raulin, 1976) to take part in the individual testing session. Given that the Social Anhedonia Scale also predicts schizophrenia-spectrum disorders (Gooding et al., 2005; Kwapil, 1998), following previous research the Social Anhedonia Scale was also included to identify a control group (Gooding, et al., 2005; Kerns, 2005; Kwapil, 1998).

In the individual testing session, recruited participants completed the full versions of the Magical Ideation Scale, Perceptual Aberration Scale, and Social Anhedonia Scale. From these full versions of the scale, participants were assigned into two groups: Schizotypal and Control. Classifications were made based on previous norms from large samples of similar populations (Kerns & Berenbaum, 2003).

Schizotypal Group

There were 24 participants in the schizotypal group ranging from 18-20 years old, with an average age of 18.27 (SD = 0.55). Participants were 54.2% female, 70.8% White, 4.2% African-American, and 4.2% Asian-American, and 20.9% other.

Control Group

There were 31 participants in the control group ranging from 18-21 years old, with an average age of 18.35 (SD = 0.66). Participants were 71.0% female, 80.6% White, 3.2% African-American, 6.7% Asian-American, and 9.7% other.

Procedure

As part of a larger study, participants first completed the Magical Ideation Scale, Perceptual Aberration Scale, and Social Anhedonia Scale mixed together and titled the Survey of Attitudes and Experiences. In a separate session, participants completed the Referential Thinking Scale.

Results

Pleasant and unpleasant referential thinking scores were calculated in Study 3 as they were in Study 2. Participants in the schizotypal group had both higher unpleasant referential thoughts (M = 4.88, SD = 2.80 vs. M = 1.70, SD = 1.44, t (52) = 5.39, p < .001, effect size d = 1.43) and higher pleasant referential thoughts (M = 5.96, SD = 2.76 vs. M = 3.10, SD = 2.04, t (52) = 4.38, p < .001, d = 1.18) than participants in the control group. Next we tested whether in either of these groups they were more likely to experience unpleasant than pleasant referential thoughts. Since the unpleasant referential thinking subscale had 14 items and the pleasant item subscale had 20 items, unpleasant referential thinking scores were divided by 14 and pleasant item scores were divided by 20 to allow for a comparison between scales. Then, a paired-samples t-test was run to test whether there was a difference between the number of unpleasant and pleasant referential thoughts experienced by schizotypal or control participants. There was not a significant difference in the number of unpleasant and pleasant referential thoughts experienced by the schizotypal (t (23) = 1.33, p = .20) or control groups (t (29) = 1.53, p = .14).

Discussion

Study 3 found further evidence that both unpleasant and pleasant referential thoughts are related to other schizotypal personality characteristics. The schizotypal group had both elevated unpleasant and pleasant referential thinking scores compared to control participants.

General Discussion

The current research extended previous work on referential thinking in several ways (Lenzenweger et al., 1997). Study 1 was the first study to empirically examine whether referential thoughts can be experienced as both unpleasant and pleasant. Moreover, the CFA in Study 1 found that unpleasant and pleasant referential thoughts could be discriminated from each other and from paranoia, with a three-factor model with separate unpleasant referential thinking, pleasant referential thinking, and paranoia factors providing the best fit to the data. Study 2 provided further evidence that unpleasant and pleasant referential thoughts could be discriminated from each other in two ways. First, an item-level CFA with unpleasant items and pleasant items on separate factors fit the data better than a CFA with all items loading on a single factor. Second, unpleasant and pleasant referential thoughts displayed differential associations with self-processing, facets of narcissism, and schizotypal and normal personality traits. Finally, study 3 found that people with elevated schizotypal characteristics had both elevated unpleasant and elevated pleasant referential thoughts. This suggests that both unpleasant and pleasant referential thinking might be important for schizophrenia-spectrum disorders.

The current research found that unpleasant and pleasant referential thinking appear to be correlated but distinct constructs. These traits exhibited very different associations with self-processing, paranoia, and personality. In addition, the CFAs in both Study 1 and Study 2 found that unpleasant and pleasant referential thinking loaded on different factors. Hence, these results suggest that there could be important differences between unpleasant and pleasant referential thinking. At the same time, unpleasant and pleasant referential thinking still exhibited moderate to strong associations with each other, and both constructs were associated with other schizotypal personality characteristics.

Taken together, these results suggest that unpleasant and pleasant referential thinking might share some important common mechanisms, but other variables may moderate the expression of referential thinking. For example, one mechanism that might be in common between unpleasant and pleasant referential thinking is aberrant salience. Aberrant salience is the over-attribution of salience to personally irrelevant objects or events and has been conjectured to be a critical psychological mechanism in the development of psychosis (Kapur, 2003). Increased aberrant salience might foster the occurrence of either unpleasant or pleasant referential thinking. This might explain why both unpleasant and pleasant referential thinking are associated with other schizotypal characteristics associated with psychosis. However, whether referential thoughts are experienced as unpleasant or pleasant might depend on other moderating factors. The current research suggests that one moderating factor might be self-esteem. Potentially, the combination of high aberrant salience and low self-esteem results in the occurrence of unpleasant referential thoughts. In contrast, the combination of high aberrant salience and high self-esteem might result in the occurrence of pleasant referential thoughts. Hence, unpleasant and pleasant referential thinking might both share a common mechanism such as aberrant salience, but the valence of referential thinking might be moderated by self-esteem.

The current research may also have implications for the assessment and conceptualization of personality disorders, particularly cluster A or odd and eccentric personality disorders (American Psychiatric Association, 2000). Some research has suggested that the Big Five do not adequately account for personality characteristics associated with schizotypal PD, and that measures of “oddity” or “peculiarity” may do a better job of representing schizotypal PD (Tackett, Silberschmidt, Krueger, & Sponheim, 2008). Researchers have recently called for more work investigating these constructs (e.g., Watson, Clark, & Chmielewski, 2008). These aspects of personality may be separate from Big Five Personality characteristics, but may be strongly related to Cluster A personality disorders such as Schizotypal PD. Unpleasant and pleasant referential thoughts may be facets of oddity, and may be useful in identifying traits and dimensions underlying personality disorders. For example, the current research found that an elevated schizotypal group had more unpleasant and pleasant referential thinking than the control group. Since the schizotypal group in Study 3 would be thought to be at least somewhat similar to a group of participants with schizotypal personality disorder, this suggests that both unpleasant and pleasant referential thinking could be related to personality disorders. Future research could continue to examine the relations among unpleasant referential thinking, pleasant referential thinking, and other facets of oddity (e.g., odd or disorganized speech), which could lead to a better understanding of odd and eccentric personality disorders.

The finding that there may be different types of referential thoughts is consistent with previous theories of referential thinking. For example, some previous research has suggested that referential thinking may be multifaceted, with differences between “guilty” and “simple” ideas of reference (Wing, et al., 1974). From this perspective, guilty ideas of reference involve a feeling that others are holding an individual accountable for a unpleasant outcome while simple ideas of reference represent referential thoughts without an obvious unpleasant or pleasant affective component (e.g., thinking people are taking special notice of you could be unpleasant or pleasant). Guilty ideas of reference may be subsumed within the broader construct of unpleasant referential thinking, and simple ideas of reference could fall into either category depending on the valence of the thought. Thus, the current research is consistent with previous research that suggests that there may be different types of referential thoughts related to the valence of these thoughts.

The current research also provides evidence suggesting that referential thinking is distinct from paranoia. Previous research has found that referential thinking and paranoia load on the same schizotypy factor (e.g., Compton, Goulding, Bakeman, & McClure-Tone, 2009; Stefanis, et al., 2004). However, none of this research directly examined whether referential thinking might load on a factor separate from paranoia. In a study that was able to directly examine this, we found that referential thinking and paranoia load on distinct schizotypy factors (Cicero & Kerns, 2010). However, this research did not examine unpleasant versus pleasant referential thinking. The current research examined whether unpleasant referential thinking and paranoia might load on the same factor. In current Study 1 and Study 2, unpleasant referential thinking and paranoia loaded on distinct factors. Furthermore, there was some evidence of differential associations between unpleasant referential thinking and paranoia, as unpleasant referential thinking was more strongly associated with pleasant referential thinking and less strongly associated with neuroticism than was paranoia. Hence, the current research suggests that even specifically unpleasant referential thinking appears to be at least somewhat distinct from paranoia. This suggests that attempts to measure odd and eccentric personality disorders should include distinct referential thinking and paranoia symptom dimensions. One issue for future research would be to examine whether a CFA using additional unpleasant referential thinking scales also finds that unpleasant referential thinking and paranoia load on distinct factors. In addition, another issue for future research would be to further examine psychological mechanisms that might distinguish referential thinking and paranoia. For example, it is possible that referential thinking might exhibit stronger associations with aberrant salience than paranoia does.

The results of this research may also have implications for the treatment and prevention of schizophrenia. Previous research suggests that cognitive behavioral therapy may be an effective treatment for schizophrenia (see Rathod, Phiri, & Kingdon, 2010, for a review). In the current studies, we found that some psychotic-like experiences are experienced as pleasant while others were experienced as unpleasant. This suggests that clinicians may be able to focus on certain beliefs (i.e., the unpleasant ones) in cognitive therapy. Additionally, recent research has suggested that the identification and treatment of individuals in prodromal, or early pre-psychotic, stages of schizophrenia may lessen the severity of the disorder and potentially prevent its onset altogether (Compton, McGlashan, & McGorry, 2007). Future research could examine whether unpleasant and pleasant referential thoughts could be used to better identify people at risk for the development of the disorder in order to provide treatment for those individuals.