Factor Structure, Convergent, and Divergent Validity of the Prodromal Questionnaire-Negative Symptom Subscale.

Abstract

Negative symptoms such as anhedonia are associated with psychosis risk and poorer outcomes. The Prodromal Questionnaire (PQ) is a self-report questionnaire used to screen for psychosis spectrum symptoms. However; the convergent and divergent validity and underlying factor structure of the PQ-negative symptom subscale (PQ-N) have yet to be examined. Undergraduates (N=1,556) completed the PQ, Temporal Experience of Pleasure Scale, and measures assessing anxiety, depression, and motivation. An exploratory factor analysis conducted on the PQ-N yielded a two-factor solution, reflecting sub-dimensions of social expression and dissociative-depressive experiences, contrary to previous research examining the factor structure of negative symptoms. Associations between the PQ-N, its two factors, and measures of negative symptoms and other psychopathology were examined. Results indicated that the PQ-N and its factors were more strongly correlated with measures of depression and anxiety than with measures of negative symptoms relating to motivation and pleasure, suggesting poor convergent and divergent validity.

Introduction

Negative symptoms, such as anhedonia and avolition, are associated with psychosis risk and poorer functional outcomes among patients with psychotic disorders, such as schizophrenia (Kurtz, 2005; Milev, Ho, Arndt, & Andreasen, 2005; Sarkar, Hillner, & Velligan, 2015). As such, the development of measures that accurately assess negative symptoms has become imperative for conducting research in this domain. Negative symptoms generally are agreed to consist of five subdomains: blunted affect (a decrease in emotional expression), alogia (poverty of speech), anhedonia (decreased ability to feel pleasure), asociality (decrease in motivation to participate in social activities), and avolition (decrease in motivation for self-directed activity) (Kirkpatrick, Fenton, Carpenter, & Marder, 2006). Negative symptoms typically emerge prior to the onset of positive symptoms in the course of psychotic disorders (Häfner et al., 2005; Lencz, Smith, Auther, Correll, & Cornblatt, 2004; Lyne et al., 2015; Velthorst et al., 2010) and research has suggested that negative symptoms impact functional outcomes and quality of life to a greater degree than positive symptoms (e.g. hallucinations and delusions) (Mohamed et al., 2008; Rabinowitz et al., 2012; Milev et al., 2005; Narvaez, Twamley, Mckibbin, Heaton, & Patterson, 2008; Wegener et al., 2005). Additionally, individuals with a shorter duration of untreated negative symptoms tend to perform better on cognitive tasks than their counterparts with longer durations of untreated negative symptoms (Cuesta et al., 2012). These findings suggest that accurate, early identification and treatment of negative symptoms may not only assist in early identification of those at risk for developing psychosis, it also may improve their overall functional outcomes in the short term as well as the long term. Given the importance of negative symptoms in the course of psychotic disorders, a needed area of research is developing valid and reliable questionnaires that can aid in quickly identifying these dimensions across the psychosis spectrum.

Due to the breadth of research indicating that negative symptoms are an important prognostic factor of functioning for those at risk for psychosis, a variety of semi-structured interviews and self-report measures have been created to measure negative symptoms (e.g., Cooper, Lavayasse, & Gard, 2015; Deci & Ryan, 1985; Kay, Fiszbein, & Opler, 1987; Gard et al., 2006; Kring, Gur, Blanchard, Horan, & Reise, 2013). Although there have been some recent developments and improvements in the measurement of negative symptoms, many factors have impeded the development of accurate scales. First, the conceptualization of negative symptoms has changed over time (Kirkpatrick et al., 2006), which has led to changes in the way that negative symptom scales are designed. Although many newer measures are cognizant of and account for the changing definition of negative symptoms, many commonly used scales were created prior to the Kirkpatrick et al. (2006) consensus statement, wherein experts convened under the endorsement of the National Institute of Mental Health (NIMH) to, among other topics, review data and make a determination as to which domains of negative symptoms were empirically supported. Thus, older negative symptom scales potentially include remnants of items that are no longer widely classified as negative symptoms, such as social cognition (Blanchard, Kring, Horan, & Gur, 2011; Foussias & Remington, 2010; Kirkpatrick et al., 2006). Second, many negative symptom scales only measure one of the five subdomains of negative symptoms. This is problematic, because if negative symptom domains respond variably to treatment, domain-specific treatment responses may be obscured (Kirkpatrick et al., 2006). Furthermore, many negative symptom scales lack dimensions that tap into the behavioral aspects of negative symptoms (e.g., the Temporal Experience of Pleasure Scale (TEPS) assesses different facets of pleasure by asking participants to rate either how much they enjoy activities in the present moment or how much they look forward to them, but does not ask how often the participant either anticipates or engages in said behaviors [Gard, Gard, Kring, & John, 2006]), or fail to assess symptoms that are typically based on rater observation (e.g., alogia).

Similarly, some negative symptom measures are not structured in a way that adequately distinguishes between negative symptoms unique to psychosis and similar symptoms associated with related psychopathologies. There is a good deal of overlap between depressive symptoms and negative symptoms, although they are generally accepted to be separate constructs (Häfner et al., 1999; Herbener, Harrow, & Baker, 2002; Selten, Gernaat, Nolen, Wiersma, & Van Den Bosch, 1998). For example, both individuals with depressive symptoms and those with negative symptoms commonly experience anhedonia and social withdrawal and, as such, negative symptoms are often mistaken as transient symptoms of depression (Horan, Kring, & Blanchard, 2006). Many measures are not designed in a way that is conducive to examining whether experiences are primary or secondary to depression or to negative symptoms. In addition, many negative symptom measures do not ask participants about medication status (e.g., motivation and pleasure scale, self-report (MAP-SR) Llerena et al., 2013) and, as such, one cannot fully assess whether symptoms are secondary to medication effects. Thus, it is crucial to carefully examine the validity of measures intended to measure negative symptoms to determine whether negative symptoms are being accurately evaluated.

Although negative symptoms consist of five subdomains and consistently load onto factors that are separable from anxiety, depression, and other psychosis spectrum symptoms, such as positive symptoms (Blanchard, & Cohen, 2006; Liddle, 1987; Thompson & Meltzer, 1993), it is widely recognized that negative symptoms are multidimensional. That is, they are comprised of multiple, related yet distinct components. Due to the multidimensionality of negative symptoms, previous studies have examined their underlying factor structure and found that multi-factor solutions may be appropriate (Blanchard, & Cohen, 2006). The factor solution that has garnered the most support is a two-factor model that consists of a diminished or poor emotional expression factor and an avolition-apathy factor (Ergül & Üçok, 2015; Galderisi et al., 2013; Kimhy, Yale, Goetz, McFarr, & Malaspina, 2006; Nakaya, & Ohmori, 2008; Peralta & Manuel, 1995; Strauss et al., 2013). Although the majority of these factor analytic studies have been conducted on patient samples, recent evidence has also supported this 2-factor model in clinical high risk populations (Azis et al., 2018), although more evidence demonstrating the factor structure of negative symptoms in non-patient populations is necessary. Many studies supporting the two-factor model have utilized common measures of negative symptoms such as the scale for assessment of negative symptoms (SANS; Andreasen, 1982), schedule for the deficit syndromes (SDS; Kirkpatrick et al., 1989), and the brief psychiatric rating scale (BPRS; Overall & Gorham, 1962). However, methodologically some studies have opted to conduct factor analyses of items (e.g. Kimhy et al., 2006; Peralta & Manuel, 1995), whereas others have opted to conduct factor analyses of scales (e.g. Ergül & Üçok, 2015; Strauss et al., 2013); the latter of which assumes that the items within each scale or sub-scale homogenously reflects a single construct.

Despite the clear importance of accurate screening for negative symptoms early in the course of psychosis, only a few self-report questionnaires have been shown to be psychometrically sound in multiple validation studies. For example, to date some self-report measures have displayed good internal validity and reliability in multiple samples, such as the TEPS (Chan et al., 2012; Gard et al., 2006; Garfield, Cotton, & Lubman, 2016; Loas et al., 2009; Simon et al., 2018; Geaney, Treadway, & Smillie, 2015). However, other measures display good internal consistency but some measures had questionable discriminant validity, such as the positive and negative symptoms questionnaire (PNS-Q; Kilian et al., 2015; Iancu, Poreh, Lehman, Shamir, & Kotler, 2005). In other instances, measures have good internal consistency when administered as semi-structured interviews, such as the clinical assessment interview for negative symptoms (CAINS) and weaker internal consistency when administered as self-report versions (e.g. the CAINS-Self Report) (Kilian et al., 2015; Kring et al., 2013; Park et al., 2012). More recently developed measures, such as the General Causality Orientation Scale for Clinical Populations (GCOS-CP), have displayed excellent convergent and divergent validity (Cooper, Lavaysse, & Gard, 2015), but have not yet had their reliability and validity assessed by independent researchers.

Some broader measures of psychosis, such as the Prodromal Questionnaire (PQ), have not had the validity and reliability of their negative symptom subscales assessed (Loewy, Bearden, Johnson, Raine, & Cannon, 2005; Loewy, Johnson, & Cannon, 2007). The PQ, a commonly used measure in both research and clinical settings (Loewy et al., 2005), is a 92-item self-report screener that measures psychotic-spectrum symptoms and is utilized to predict the future onset of psychosis. In the original scale development study, Loewy and colleagues (2005) found that the PQ positive symptom items predicted diagnostic status of psychosis risk syndromes (as assessed via semi-structured interviews designed to assess risk for psychosis), while the other item subsets (e.g., negative symptoms and disorganized symptoms) did not. Although the PQ and its short version, the Prodromal Questionnaire-Brief (PQ-B), have been examined for validity in many studies (Ising et al., 2012; Loewy et al., 2005; Loewy, Pearson, Vinogradov, Bearden, & Cannon, 2011), these studies primarily focused on validating the positive symptom subscale of the PQ, given its reported predictive validity for psychosis-risk syndromes.

To date, no studies have examined the underlying factor structure of the PQ negative symptom subscale (PQ-N). Thus, it is unclear how many domains of negative symptoms are captured by this subscale. Although some items (see Table 3) appear to probe asociality (e.g., item 21), avolition (e.g. item 85), or anhedonia (e.g. item 82), not all items appear to align with a specific domain (e.g. item 16), and no items appear to examine blunted affect or alogia.

Table 3.

Factor Loadings based on an Exploratory Factor Analysis Results with a Geomin rotation for the 19 negative symptom items on the Prodromal Questionnaire (PQ)

Item	Question	Factor 1 Social Dysfunction	Factor 2 Dissociative/Depressive
39	I have gotten very nervous when I had to make polite conversation.	0.930*
80	I have been extremely anxious when meeting people for the first time.	0.929*
87	I have avoided going to places where there were many people because I get anxious.	0.774*
33	I have had nothing to say or very little to say.	0.708*
21	I have been very quiet and have kept in the background on social occasions.	0.687*
43	I have avoided social activities with other people.	0.629*
59	I have tended to keep my feelings to myself.	0.595*
66	I have not been very good at returning social courtesies and gestures.	0.593*
78	I have had little interest in getting to know other people.	0.581*
58	People have found me to be aloof and distant.	0.469*	0.345*
42	I have found it hard to be emotionally close to other people.	0.421*	0.337*
48	I have felt unable to enjoy things that I used to enjoy.		0.932*
89	I have felt uninterested in the things I used to enjoy.		0.916*
81	I have felt like I was at a distance from myself, as if I were outside my own body or that a part of my body did not belong to me.		0.678*
82	I have found that when something sad happened, I was no longer able to feel sadness, or when something joyful happened, I could not feel happy.		0.669*
16	I have noticed a sense of not knowing who I am.		0.555*
71	I have felt like I was looking at myself as in a movie, or that I was a spectator in my own life.		0.529*
85	I have felt unable to carry out everyday tasks because of fatigue or lack of motivation.		0.493*
14	I have had difficulty expressing my feelings as well as I used to.		0.437*

Note: N=1,556 Factor loadings <.3 are suppressed. Probability

^*p<.05

These limitations may be in part due to the measure development strategies utilized to construct the PQ. Although some novel items were added to the PQ by its authors (Loewy et al., 2015) the majority of prompts included in the PQ were drawn from the Schizotypal Personality Questionnaire (SPQ; Raine, 1991) and the Structured Interview for Prodromal Syndromes (SIPS; Miller et al., 2002). Notably, the SIPS negative symptom subscale has yet to be validated; and the SPQ was modeled not on negative symptoms, but on the DSM-III-R criteria for schizotypial personality disorder (Raine, 1991). Although the concurrent validity of the PQ has been demonstrated, the PQ’s development was not informed by the use of scale development tools such as CFA and EFA. Thus, due to these important limitations of the scales the PQ was constructed from, there may be a lack of content validity.

Although the PQ-N has been shown to have good internal consistency in multiple samples (Kotzalidis et al., 2017; Loewy et al., 2005), the factor structure and external validity of the measure are unexamined. Because the PQ is widely used and can be utilized in both clinical and research samples (Loewy, Johnson, & Cannon, 2007), it is imperative to determine the validity of its negative symptom subscale. The present study sought to determine 1) the factor structure of the PQ-N and 2) the discriminant and convergent validity with other measures of negative symptoms, as well with measures of other psychopathology. As the PQ-N appears to investigate multiple domains of negative symptoms, a multi-factor solution is expected. Furthermore, based on the content of the PQ-N items and their similarity to other measures of negative symptoms utilized in support of the 2-factor model, it is not expected that the resultant factors will align with previous research indicating that the underlying factor structure of negative symptoms consists of both a diminished or poor emotional expression factor, and an avolition-apathy factor, given that there are not items that fully capture theses dimensions (Ergül & Üçok, 2015; Galderisi et al., 2013; Kimhy et al., 2006; Nakaya, & Ohmori, 2008; Peralta & Manuel, 1995; Strauss et al., 2013). After determining the underlying factor structure of the PQ-N, its convergent and divergent validity will be examined by comparing the PQ-N and any resultant factors to well-established measures of negative symptoms and related psychopathology that have been previously determined to be highly valid and reliable.

Methods

Participants

The present protocol was approved by the Institutional Review Board at Temple University. Written informed consent was obtained from all participants prior to their enrollment in the study. We report how we determined our sample size, all data exclusions, all manipulations, and all measures in the study. In order to ensure a full range of psychotic-like experiences was represented, the present study consisted of a large sample of undergraduates (N=1556) who were identified via a multidisciplinary, online subject pool (sample characteristics are provided in Table 1). Undergraduates were selected in order to obtain a sample consisting of young adults; as the late teens to mid-20s represent the typical ages wherein subthreshold psychotic symptoms tend to emerge (Cannon et al., 2008). The present sample therefore represents an appropriate population for the assessment of prodromal symptoms as measured by the PQ. Notably, participants in the present sample were primarily female. However, evidence is mixed regarding biological sex differences in symptom expression in CHR populations, although some evidence suggests males may have more severe negative symptoms than females (Barajas, Ochoa, Obiols, & Lalucat-Jo, 2015). Finally, previous studies examining psychotic-like experiences in undergraduate samples have demonstrated that obtaining a full range of psychosis spectrum symptoms in this population is feasible (e.g. Cooper, Kring, & Ellman, 2018). Participants completed computerized versions of questionnaires online in the laboratory, with trained lab staff available to both provide instructions and answer questions. Students who did not complete all measures were excluded from this study. Data collection has been ongoing (e.g., based on doctoral student projects); therefore, measures were added during different time periods based on initiation of new projects (i.e., data were not missing, but rather the given questionnaire had not been added to the protocol).

Table 1.

Participant Demographics (N=1,556)

Male, n (%)	388 (24.9%)
Age, mean (SD)	20.86 (2.44)
Ethnicity, Hispanic, n (%)	102 (6.6%)
Race, n (%)
White	910 (58.5%)
Black/African American	256 (16.5%)
Asian	229 (14.7%)
Biracial or Multiracial	73 (4.7%)
American Indian/Alaska Native	5 (0.3%)
Native Hawaiian/Pacific Islander	3 (0.2%)
Other	80 (5.1%)

Instruments

The Prodromal Questionnaire

The PQ consists of 92 true/false questions that measure the presence of positive symptoms, negative symptoms, disorganized symptoms (e.g. decreased hygiene and odd behavior), and general symptoms (e.g. impaired role functioning) (Loewy et al., 2005). The PQ-N is a 19-item subscale of the PQ meant to identify the presence of negative symptoms and consists of questions such as, “I have found that when something sad happened, I was no longer able to feel sadness,” or “When something joyful happened, I could not feel happy”. These items, as well as the items from the other subscales, are mixed together randomly throughout the PQ so that the items are neither clumped together nor are they presented in a meaningful order. Similar to the other PQ subscales, the PQ-N was found to have good internal consistency (α =.88) by Loewy et al., 2005.

The Temporal Experience of Pleasure Scale

Anticipatory pleasure (the ability to anticipate a future hedonic experience) and consummatory pleasure (the in-the-moment hedonic experience of a given activity or event) were examined utilizing the TEPS. The TEPS is an 18-item scale that includes items such as, “I love the sound of rain on the windows when I’m lying in my warm bed” and “When something exciting is coming up in my life, I really look forward to it” which are rated on a scale from 1 (very false for me) to 6 (very true for me) (Gard et al., 2006). To date, the TEPS has been validated in a number of samples, its factor structure has been well documented (Chan et al., 2012; Garfield, Cotton, & Lubman, 2016; Loas et al., 2009; Simon et al., 2018; Geaney, Treadway, & Smillie, 2015), and its internal consistency for its subscales has been shown to range from α=.74−90 for anticipatory pleasure, and α=.54−.84 for consummatory pleasure in previous studies (Gard et al., 2006; Garfield, Cotton, & Lubman, 2016; Geaney, Treadway, & Smillie, 2015). In the present study, α=.75 for anticipatory pleasure, and α=.67 for consummatory pleasure.

General Causality Orientation Scale for Clinical Populations

Motivation deficits were examined utilizing the General Causality Orientation Scale for Clinical Populations (GCOS-CP). The GCOS-CP is a measure of motivational orientations derived from the original 51-item General Causality Orientation Scale (GCOS) developed by Deci and Ryan (1985). The GCOS-CP examines three motivation orientations: autonomy (i.e., intrinsic motivation), control (i.e., external motivation), and impersonal (i.e., amotivation or lack of agency) (Cooper, Lavaysse, & Gard, 2015; Deci & Ryan, 1985). This measure consists of 8 vignettes that begin with a situational prompt (e.g., “You feel that your friend or roommate is being inconsiderate. You would probably…”) then provides participants with three possible ways to respond, and asks them to rate how likely they would be to respond in that manner on a scale from 1 (very unlikely) to 7 (very likely) (e.g., 1) “Find an opportunity to explain why it bothers you; your friend or roommate may not even realize how much it is bothering you.”, 2) “Say nothing; if your friend or roommate cared about you she would understand how you felt.”, 3) “Demand that your friend or roommate start being more considerate; otherwise you’ll respond similarly”). The GCOS-CP has been shown to have good convergent and divergent validity, and its respective sub-scales display adequate to good fit, with an internal consistency of α=.74 for autonomy α=.65 for control, and α=.67 for impersonal orientations (Cooper et al., 2015). In the present study, α=.82 for autonomy, α=.60 for control, and α=.70 for impersonal orientations.

The Center for Epidemiologic Studies Depression Scale - Short Form

The presence of depressive symptoms was measured utilizing the Center for Epidemiologic Studies Depression Scale Short Form (CES-D-10). The CES-D-10 is a psychometrically sound (Björgvinsson, Kertz, Bigda-Peyton, McCoy, & Aderka, 2013), shortened form of the original 20-item Center for Epidemiologic Studies Depression Scale (CES-D) developed by Radloff (1977). To date, the CES-D-10 has been validated in a number of samples, and has consistently demonstrated good reliability (Andersen, Malmgren, Carter, & Patrick, 1994; Björgvinsson, et al., 2013; Irwin, Artin, & Oxman, 1999; Zhang et al., 2012) with an internal consistency of α=.89 in other studies (Björgvinsson, et al., 2013), and α=.82 in the present study.

The State-Trait Anxiety Inventory Trait Form

The presence of current anxiety symptoms was measured utilizing the State-Trait Anxiety Inventory Trait Form (STAI-T), a subscale of the State-Trait Anxiety Inventory that eliminates items that load highly onto depression (Spielberger, 1983; Spielberger, Vagg, Barker, Donham, & Westberry, 1980; Bieling, Antony, & Swinson, 1998). This version of the scale was selected in order to optimize our ability to compare the PQ-N to a measure of anxiety without conflating depression and anxiety. The STAI-T consists of 7-items and has been shown to be valid (Maynard et al., 2010; Oei, Evans, & Crook, 1990) as well as reliable and internally consistent α=.81–89 in other samples (Bieling, Antony, & Swinson, 1998; Maynard et al., 2010; Westberry et al., 1988), with α=.86 in the present study.

The Social Phobia Scale

The presence of social anxiety was measured utilizing the Social Phobia Scale (SPS). The SPS is a 20-item, psychometrically sound scale that taps into constructs distinct from general anxiety symptoms, with internal consistency values ranging from α=88−.94 in previous studies (De Beurs, Tielen, & Wollmann, 2014; Mattick & Clarke, 1989; Peters, 2000), α=.93 in the present study, and has been validated in a number of samples (Brown et al., 1997; De Beurs et al., 2014; Olivares, García-López, & Hidalgo, 2001).

Data Analysis

Several analyses were utilized in the present study in order to fully assess the factor structure, discriminant, and convergent validity of the PQ-N items, utilizing SPSS version 24 (IBM Corp., 2013) MPlus version 7 (Muthén & Muthén, 1998), and R. First, an exploratory factor analysis (EFA) was conducted on the PQ-N items utilizing a geomin rotation method using Mplus, Version 7 (Muthén & Muthén, 1998). As the PQ relies on binary responses, the robust weighted least squares estimator (WLSMV) was used. The geomin rotation method is an oblique rotation method, which allows resultant factors to be correlated with one another. This method was preferred as there is evidence that utilizing an orthogonal rotation method may result in an inadvertent loss of information if factors are correlated, and if factors are uncorrelated, oblique and orthogonal rotation methods return near-identical results (Costello & Osborne, 2005; Fabrigar, Wegener, MacCallum, & Strahan, 1999; Mulaik, 2009). The number of factors retained was determined by multiple indices of fit. We examined the results of a scree plot, wherein the number of factors retained is the point at which the plot reaches a natural ‘break point’, and the eigenvalue curve flattens (Cattell, 1966; Velicer & Jackson, 1990). In addition to investigating the scree plot, we also conducted a parallel analysis (Çokluk, & Koçak, 2016; Horn, 1965) and Velicer’s (1976) Minimum Average Partial (MAP) using Principal Components Analysis (PCA) extraction in the psych package in R. Parallel analysis has been demonstrated to perform well with unidimensional binary data (Weng & Chen, 2005). Furthermore, parallel and MAP analyses tend to be more robust to the over-extraction of variables (Zwick & Velicer, 1986) than the scree plot. Both Parallel and MAP analyses have been demonstrated to be appropriate for use with categorical and binary data (Cho, Li, & Bandalos, 2009; Garrido, Abad, & Ponsoda, 2011). Goldbergs (2006) ‘bass-ackwards’ approach was also utilized in order to further clarify the number of factors to retain. This approach may be utilized in PCA and EFA frameworks and is designed to provide further elucidation regarding the number of factors to retain. Specifically, it involves conducting successive factor analyses and organizing them by their respective factor score correlations from level to level.

In addition, no factors containing fewer than three items were retained, as this is in indication of poor fit (Costello & Osborne, 2005). Specific items were retained on a given factor if a minimum factor loading of .32 was attained, which is consistent with previous research (Tabachnick, & Fidell, 2001). If multiple items returned cross loadings of .32 or higher on more than one factor, then the item demonstrating the cross loadings were removed from subsequent analyses to assess their overall influence on the model (Costello & Osborne, 2005). If the items had no influence over the model, they would be retained and remain in the factor to which they had the highest loading; if they significantly altered the model, they would be removed from subsequent analyses (Worthington & Whittaker, 2006). We also examined fit indices obtained from the EFAs. We relied on the comparative fit index (CFI) and root mean square error of approximation (RMSEA), with CFI values greater than 0.90 (Hu & Bentler, 1999) and RMSEA values less than 0.08 indicating good fit (Chen, Curran, Bollen, & Paxton, 2008; Marsh, Hau, & Wen, 2004). We report chi-square statistics for models, but do not interpret them as this index may be overly sensitive to large samples (Markland, 2007; Marsh, Hau, & Wen, 2004). In order to obtain additional fit indices, after the initial EFA was conducted; a secondary EFA was performed using a maximum likelihood estimator in Mplus in order to obtain Akaike Information Critera (AIC; Akaike, 1973) and Bayesian information criterion (BIC; Schwarz, 1978). Lower AIC and BIC scores indicate better fit.

The EFA aimed to ascertain whether the PQ-N items truly belonged to a single ‘negative symptom’ factor, or whether the items fed into multiple factors. In order to assess the consistency of any resultant factors from the PQ-N with previous literature, the variables in each resulting factor were examined to ascertain whether they appeared to capture either diminished emotional expression or avolition-apathy (Ergül & Üçok, 2015; Galderisi et al., 2013; Kimhy et al., 2006; Nakaya, & Ohmori, 2008; Peralta & Manuel, 1995; Strauss et al., 2013).

Next, two-tailed bivariate Pearson’s correlations and correlation coefficient difference tests were conducted on the PQ-N, measures of negative symptoms, and other psychopathology to ascertain the convergent and discriminant validity of the PQ-N. Partial correlations were conducted on the TEPS, GCOS-CP, and PQ-N while controlling for depression and anxiety as measured by the CES-D-10, STAI-T, and SPS to ascertain whether depression and anxiety may account for the observed relationships between the negative symptom scales. Two-tailed bivariate correlations were deemed significant at the p<.05 level. Given that statistically significant positive correlations are common between measures assessing anxiety, negative, and depressive symptoms (Wigman et al., 2012), differences in magnitude of associations were evaluated using Fisher’s r-to-z transformation (Fisher, 1915; Fisher, 1921).

Results

Results from EFAs utilizing the WLSMV and ML estimators indicated relatively good fit for solutions with one to four factors (Table 2). With each addition of a new factor, fit indices improved. Examining the natural ‘break point’ of the scree plot (Figure 1) indicated a 2-factor solution. Results from the parallel analysis indicated that a 7-factor model should be retained; however, given that the PQ-N consists of only 19-items, retaining a 7-factor solution would result in multiple factors with less than 3 items and, thus, result in unstable factors and diminish theoretical interpretability. Results from the MAP analysis indicated that a 2-factor solution is optimal. Notably, MAP analysis have been demonstrated to typically be more accurate and less variable than scree tests (Zwick & Velicer, 1986). Similarly to the scree plot and MAP analysis, results from a parallel analysis with a PCA extraction also supported the retention of a 2-factor solution. The bass-ackwards approach revealed a relatively high degree of downstream stability (Figure 2) that indicates that the inclusion of additional factors is not adding a large degree of theoretically substantive information.

Table 2.

Fit Statistics based on an Exploratory Factor Analysis Results with a Geomin rotation for the 19 negative symptom items on the Prodromal Questionnaire (PQ)

Factors	Eigenvalue	chi-square	p-value	DF	CFI	RMSEA [CI]	SRMR	MAP	AIC	BIC
1 factor	9.49	1325.98	>.000	152	0.935	.07 [.067–.074]	0.08	0.028	29763.31	29966.61
2 factor	1.56	582.55	>.000	134	0.975	.046 [.043–.05]	0.05	0.025	29281.08	29580.67
3 factor	1.02	396.13	>.000	117	0.984	.039 [.035–.043]	0.04	0.027	29147.85	29538.39
4 factor	0.86	240.176	>.000	101	0.992	.030 [.025–.035]	0.031	0.027	29033.96	29510.09

Note: N=1,556 AIC and BIC were obtained utilizing the ML estimator; all other statistics are drawn from the WLSMV estimator in Mplus.

Figure 1.

Scree Plot of Eigenvalues for PQ-N Factors

Figure 2:

Goldberg’s Backasswards Model

Note: N=1,556; Model presents 5 factor solutions

The 1-factor solution had the poorest overall fit. Notably, the eigenvalue for the first factor is quite high; which could support a 1-factor solution. However, the 1-factor solution is highly heterogeneous; and consists of items spanning multiple domains such as social dysfunction and dissociative experiences that are difficult to meaningfully summarize from a theoretical perspective. The 2-factor model returned a factor solution that was more theoretically interpretable; Factor 1 included items relating to social dysfunction, and factor two included items associated with dissociative and depressive experiences (Table 3). The 2-factor solution’s factors were highly correlated (r=.79, p<.05). However, although dissociative and depressive symptoms are highly related; they arguably represent separate constructs. The 3-factor model (Supplemental Table 1) had improved fit in comparison to the 1 and 2 factor models, but was less theoretically interpretable. In this 3-factor model, a strong factor consisting of social dysfunction was still retained. However, both factors 2 and 3 consisted of a mixture of items related to diminished enjoyment and motivation as well dissociative experiences.

Finally, in the 4-factor model (Supplemental Table 2) the previously highly stable social dysfunction factor splits into two factors that are highly related to one another (r=.61, p<.05). Although these factors are difficult to separate theoretically, factor 2 notably consists of all items explicitly using the words “anxious” and “nervous”. In the 4-factor model, it is also notable that factors 3 and 4 remained stable, with the exception of item 16 moving its highest loading to factor 4. With this change, factor 3 emerges as more related to diminished emotional experience and factor 4 consists of items relating to dissociative experiences. However, although factors 3 and 4 improve from an interpretability standpoint, factors 1 and 2 become less interpretable.

Additional supplementary analyses were also conducted to examine the factor structure of the PQ-N. In one analysis, the original sample was randomly split into two sub-samples; an EFA was conducted in one sub-sample and a CFA was conducted on the remaining sample. Outcomes from this analysis provided mixed results that approximated our original analysis results (Supplementary Tables 3–4). However, it is notable that conducting a CFA on categorical item-level data has limitations, such as the potential overestimation of factors (Bernstein & Teng, 1989; Nunnally & Bernstein, 1994). Due to the variability in internal consistencies of scales utilized in the present study, additional supplementary analyses were conducted in R and disattenuating correlation coefficients (Osborne, 2003; Zimmerman, & Williams, 1997) were obtained for 1, 2, 3, and 4 factor solutions. Results indicated that correlations obtained in the present study were not strongly influenced by measurement error (Supplemental Tables 5–7).

Taken together, the statistical and theoretically informed results from the EFA are mixed. In the present study, we retained a 2-factor solution; χ2 (134) =582.55, p<.0001; CFI=.98; RMSEA=.046 [90% CI = .04, .05]. This selection was based on statistical support from the Scree plot, Parallel Analysis, and MAP analysis; as well as evidence of high stability obtained from utilizing the bass-ackwards approach and examining raw and disattenuated correlations between factors. The high level of stability across factors found in these analyses are indicative that although model fit may improve with model complexity, the 3 and 4 factor solutions had less overall support theoretically and from the various analyses. The 2-factor model was also selected due to its theoretical interpretability. Specifically, both the 3 and 4 factor solutions returned factors that were difficult to interpret theoretically, and the 1-factor model was highly heterogeneous. Thus, although there are some reservations regarding potential heterogeneity in the second factor, the 2-factor model was retained.

Some cross-loadings were present in the 2-factor model; however, omitting items with cross-loadings from the analysis did not diminish model fit nor change the substantive interpretation of the model solution. As such, all items were retained in subsequent analyses. The internal consistency of the PQ-N factor 1 was α=.84, and factor 2 was α=.79, which indicated good internal consistency for both factors. However, neither of the resultant factors from the PQ-N appeared to fully align with the factors that negative symptoms appear to be comprised of based off previous literature; namely diminished emotional expression and avolition-apathy.

In order to examine the potential distributional effects resulting from the frequency of item endorsement, average thresholds for each factor were estimated and the frequency of item endorsement was examined. Results indicated that for the 2-factor model, the threshold for factor 1 (social dysfunction) and factor 2 (dissociative/depressive) were 0.09 and 0.59 respectively. Although the threshold for factor 2 was higher compared to that of factor 1; these results are not unexpected given the content of the factors. For example, factor 1 contains multiple items that are indicative of anxiousness or nervousness, which is expected to be endorsed more frequently than dissociative experiences such as being an outside observer of one’s own actions (Bandelow & Michaelis, 2015; Loewenstein, 2018). Thus, although the average threshold for factor 2 is higher, we attribute this difference to content rather than distributional properties. Notably, the mean number of symptoms endorsed on the PQ-N was 7.45 with a standard deviation of 5; which is indicative of the presence of a full range of experiences.

Pearson’s correlations were conducted on the full PQ-N, the two resultant PQ-N factors identified in the EFA, negative symptom scales, and measures of other psychopathology (Table 4). We compared all associations between PQ-N items with measures of other psychopathology and negative symptoms and found that associations with the former criteria were stronger than those with the latter (z’s ranged from 15.1 to 26.3, all p’s<.01). This suggests that the 19-item PQ-N does not display adequate convergent or divergent validity with other measures of negative symptoms. The internal consistency of the full PQ-N was α=.88, which is the exact value identified by Loewy et al, 2005. We conducted similar comparisons of associations between the PQ-N EFA-based factors with measures of other psychopathology and negative symptoms. Similar to the results focusing on the total PQ-N score, PQ-N factors were more strongly correlated with measures of other psychopathology than with measures of negative symptoms for both PQ-N factor 1 (all z’s ranged from 6.9 to 18.5, all, ps<.01) and PQ-N factor 2 (all z’s ranged from 15.3 to 22.2, all ps<.01). When controlling for anxiety and depression, the relationship between the two PQ-N factors was greatly reduced (z=19.67, p<.001) as compared with their initial correlation, although the relationship remained significant (Table 5).

Table 4.

Descriptive Statistics and Correlations

Measure	PQ Negative	PQ Factor 1	PQ Factor 2	TEPS-ANT	TEPS-CON	GCOS-A	GCOS-C	GCOS-I	CES-D-10	STAI-T	SPS
PQ Negative	1
PQ Factor 1	.94**	1
PQ Factor 2	.86**	.64**	1
TEPS-ANT	−.20**	−.20**	−.15**	1
TEPS-CON	−.09**	−.10**	−.05*	.46**	1
GCOS-A	−.11**	−.09**	−.11**	.27**	.28**	1
GCOS-C	−0.01	−0.02	0.01	.17**	0.03	.22**	1
GCOS-I	.30**	.30**	.22**	−0.04	−0.02	−.11**	.24**	1
CES-D-10	.67**	.57**	.67**	−.23**	−.12**	−.15**	0.01	.26**	1
STAI-T	.60**	.49**	.62**	−.12**	−.07**	−.11**	0.05	.25**	.70**	1
SPS	.58**	.57**	.47**	−.11**	−.06*	−.12**	0.03	.34**	.54**	.55**	1
Mean	7.45	5.1	2.36	47.48	37.55	43.05	27.04	28.76	7.82	12.63	14.53
(SD)	5	2.36	2.21	7.15	6.39	8.53	7.34	7.78	5.28	4.84	13.13

Note: N= 1,556. PQ: Prodromal Questionnaire; TEPS-ANT: Anticipatory pleasure; TEPS-CON: consummatory pleasure; GCOS-I,A,C: Impersonal, Autonomy, Control; CES-D-10: the Center for Epidemiologic Studies Depression Scale Short Form; STAI-T: State-Trait Anxiety Inventory Trait-Form; SPS: Social Phobia Scale; Probability:

^*p<.05;

^**p<.01

Table 5.

Partial Correlations: Controlling for Anxiety and Depression

Measure	PQ Negative	PQ Factor 1	PQ Factor 2	TEPS-ANT	TEPS-CON	GCOS-A	GCOS-C	GCOS-I
PQ Negative	1
PQ Factor 1	0.91**	1
PQ Factor 2	0.74**	0.39**	1
TEPS-ANT	−0.08**	−0.10**	−0.015	1
TEPS-CON	−0.021	−0.05*	0.034	0.44**	1
GCOS-A	−0.009	−0.011	−0.002	0.25**	0.26**	1
GCOS-C	−0.042	−0.049	−0.014	0.18**	0.03	0.22**	1
GCOS-I	0.091**	0.12**	0.016	0.02	0.01	0.06*	0.24**	1

Note: N= 1,556. PQ: Prodromal Questionnaire; TEPS-ANT: Anticipatory pleasure; TEPS-CON: consummatory pleasure; GCOS-I, A,C: Impersonal, Autonomy, Control; CES-D-10: the Center for Epidemiologic Studies Depression Scale Short Form; STAI-T: State-Trait Anxiety Inventory Trait-Form; Probability:

^*p<.05;

^**p<.01

Discussion

The present study examined the factor structure of the PQ-N and assessed its discriminant and convergent validity with other measures of negative symptoms, as well with measures of other psychopathology. Results from the present study suggest that the PQ-N and its underlying factors neither converge with other measures of negative symptoms, nor do they diverge from other, related measures of psychopathology. Confidence in these findings are further strengthened as the magnitude of the associations between PQ-N and negative symptom scales either remain consistent or are reduced when controlling for anxiety and depression. As such, the PQ-N subscale may not be an optimal measure for accurately assessing the presence of negative symptomatology in non-clinical samples. Our study suggested that a two-factor solution better fit the PQ-N scale than other factor solutions, with the two factors appearing to be related to social dysfunction (factor 1) and dissociative-depressive experiences (factor 2). However, while a 2-factor solution was retained for analysis in the present manuscript, the mixed nature of the indices of model fit indicate that the factor structure of the PQ-N warrants additional research. Although we selected the model with the highest degree of support based on the scree plot, MAP, and supplemental disattenuated correlation analyses, an argument for either unidimensionality or a 4-factor model could be made. Nevertheless, given the importance of accurately assessing negative symptoms, the findings from the present study could have implications for a variety of questionnaire-based protocols. Further, our findings indicate that the PQ-N items may be better at tapping into non-negative symptom related constructs.

As discussed previously, multiple studies have indicated that a two-factor solution may be optimal for capturing the heterogeneity of negative symptoms as assessed by measures such as the SDS, BPRS, and SANS (Ergül & Üçok, 2015; Galderisi et al., 2013; Kimhy et al., 2006; Nakaya, & Ohmori, 2008; Peralta & Manuel, 1995; Strauss et al., 2013). Although the present study identified a two-factor solution, the content of the resultant factors from the PQ-N do not align with the two-factor solutions of negative symptoms found in previous literature. For example, in the present study, the social dysfunction factor consisted of many items that may be more related to social anxiety (e.g. item 87: “I have avoided going to places where there were many people because I get anxious.”) or social cognition (e.g. item 66: “I have not been very good at returning social courtesies and gestures.”), and the dissociative-depressive factor contained items that may indicate dissociative processes (e.g. item 71: “I have felt like I was looking at myself as in a movie, or that I was a spectator in my own life.”) and depression (e.g. item 48: “I have felt unable to enjoy things that I used to enjoy.”). Although the PQ-N factors contained items that could potentially tap into either diminished emotional expression or avolition-apathy (e.g. item 14), neither factor appears to fully capture either of these constructs. Notably, measures assessing alogia and blunted affect were not included in analyses examining convergent and divergent validity as the PQ-N does not contain items relating to these domains.

There are multiple possible explanations for our findings that should be considered. First, many of the PQ-N items are broad (e.g. “I have felt unable to carry out everyday tasks because of fatigue or lack of motivation”). Including such items ensures that the PQ-N is inclusive of many experiences, but potentially increases difficulties in differentiating negative symptoms from affective symptoms. Second, the PQ-N items do not assess the five negative symptom subdomains. Third, the finding that the PQ-N and its factors relate more to measures of other psychopathology may be partially related to overlap between negative symptoms and affective symptoms (e.g., depression), which is a common difficulty in these types of measures (Chemerinski, Bowie, Anderson, & Harvey, 2008; Krynicki, Upthegrove, Deakin, & Barnes, 2018; Marneros & Akiskal, 2007; Müller, Szegedi, Wetzel, & Benkert, 2001). Interestingly, neither the TEPS nor the GCOS-CP overlapped as strongly with measures of other psychopathology as the PQ-N. This suggests that these questionnaires tap into constructs separate from affective symptoms and depression, indicating that their continued use would be beneficial in evaluating negative symptoms.

Furthermore, there may be significant discrepancies in whether items accurately distinguish between primary negative symptoms (i.e., experiences that are fully attributable to negative symptoms) and secondary negative symptoms (i.e., symptoms more attributable to medication effects, positive symptoms, or other depressive symptoms; Flaum, & Andreasen, 1995; Kirschner, Aleman, & Kaiser, 2017). For example, social isolation may be more related to either social anxiety or positive symptoms such as paranoia than to negative symptoms (e.g., asociality). As such, some PQ-N items may inadvertently capture paranoia-related, isolative behaviors or other psychopathological symptoms under negative symptoms.

As mentioned previously, there is mixed research regarding reliability between interview-based and self-report measures of negative symptoms (Blanchard, Kring, Horan, & Gur, 2011; Foussias & Remington, 2010; Kirkpatrick et al., 2006). These potential differences may converge or diverge based on the level of symptom severity that the reporter is experiencing, and warrant investigation in future studies. For example, it may be the case that those experiencing moderate or severe symptoms may be less able to accurately assess and self-report their own levels of negative symptomatology, whereas those with mild symptoms may have more insight into this domain. Better understanding the nuances and potential interactions between symptom severity and type of measure may provide some elucidation as to whether certain types of measures are more accurate or valid within certain sub-populations, or whether they may be used interchangeably between multiple settings.

There are some limitations to the present paper. First, this study utilized a large undergraduate sample for analyses, thus its findings may not be generalizable to other samples, such as help-seeking populations. However, negative symptoms are common, with an estimated 12–20% of non-help-seeking individuals endorsing at least one negative symptom (Dominguez et al., 2010; Werbeloff et al., 2015). Furthermore, the PQ has been previously examined in undergraduate populations (Loewy, Johnson, & Cannon, 2007) with results indicating that undergraduates performed similarly to help-seeking populations experiencing non-psychotic spectrum symptoms, although they have significantly lower scores than their high-risk counterparts. Second, because the GCOS-CP and TEPS are self-report measures, they depend on participants’ level of insight for accurate ratings. Third, although the present study examined the PQ-N in relation to the TEPS and GCOS-CP, we did not examine the PQ-N in relation to other well-supported negative symptom scales or interviews, such as the CAINS (Kring et al., 2013), thus it is possible the PQ-N would have higher convergent validity with other measures, and warrants investigation in future research. There are also some notable strengths to the present study, such as the inclusion of well-established measures of negative symptoms and of related psychopathology as comparison groups as well as sampling from a large undergraduate population; which has been demonstrated to be appropriate for the examination of a full range of psychosis spectrum symptoms in previous studies. Although modern negative symptom measures have seen many improvements in recent years, there is a need to develop measures that accurately assess the five domains of negative symptoms, and to validate and replicate the validity of negative symptom measures in both clinical and non-clinical populations. Our findings highlight that some widely used measures of negative symptoms, such as the PQ, need further investigation as to whether they display adequate convergent and divergent validity, and whether their specified models are adequately reflected by their factor structures. Because previous research has shown that negative symptoms occur prior to the onset of positive symptoms (Häfner et al., 2005), and are a risk factor for the development of psychosis (Cooper, Kring, & Ellman, 2017; Schlosser et al., 2012), it is crucial to develop succinct, valid questionnaire-based assessments of negative symptoms for the early identification of individuals who may be at a heightened risk for psychosis.