Skip to main content
PMC home page
International Journal of Methods in Psychiatric Research logoLink to International Journal of Methods in Psychiatric Research
. 2018 Sep 10;27(4):e1740. doi: 10.1002/mpr.1740

Validation of the Prodromal Questionnaire–Brief in a representative sample of adolescents: Internal structure, norms, reliability, and links with psychopathology

Eduardo Fonseca‐Pedrero 1,3,✉, Felix Inchausti 2, Alicia Pérez‐Albéniz 1, Javier Ortuño‐Sierra 1
PMCID: PMC6877156  PMID: 30198201

Abstract

Objectives

The main purpose of the present study was twofold: to validate the Prodromal Questionnaire–Brief (PQ‐B) in a community‐derived sample of adolescents and to examine the links between psychotic‐like experiences and emotional and behavioral problems, prosocial behavior, suicidal ideation, and bipolar‐like experiences.

Method

One thousand five hundred eighty‐eight students selected by random sampling participated in a cross‐sectional survey. The PQ‐B, Paykel Suicide Scale, Strengths and Difficulties Questionnaire, Mood Disorder Questionnaire, Penn Matrix Reasoning Test, Family Affluence Scale‐II, and Oviedo Infrequency Scale were used.

Results

The unidimensional factorial structure of the PQ‐B items showed adequate goodness of fit index. This model also showed configural and strong invariance across gender. The internal consistency of the PQ‐B total frequency score was 0.92. A high degree of overlap was found between psychotic‐like experiences and emotional and behavioral problems, suicidal ideation, and bipolar‐like experiences.

Conclusion

The PQ‐B is a brief, easy, and reliable tool for screening psychotic‐like experiences in adolescents from the general population. The assessment of risk for psychosis and its relationship with other psychopathological risk factors in a close‐in strategy or two‐stage process model may help us to enhance the early identification for youths at heightened risk for serious mental disorders.

Keywords: adolescents, early detection, psychotic‐like experiences, risk for psychosis, screening

1. INTRODUCTION

Psychotic‐like experiences (PLEs) are often viewed as phenotypic indicators of liability for psychotic spectrum disorders (Linscott & van Os, 2013). Follow‐up studies carried out show that those adolescents who report PLEs or schizotypal traits have a greater probability of psychiatric outcome, particularly nonaffective psychotic disorders as well as other mental problems (e.g., depression; Debbané et al., 2015; Zammit et al., 2013). PLEs are relatively common during adolescence, with the mean annual prevalence among adolescents aged 13 to 18 around 7.5% (Kelleher et al., 2012a). In addition, adolescents with PLEs reported a high prevalence of mental Axis I diagnoses and psychopathology symptoms (e.g., depression, anxiety, bullying, and suicide attempts and ideation; Fonseca Pedrero & Debbané, 2017; Fonseca‐Pedrero et al., 2017; Kelleher et al., 2012b, 2013; Strauss, Raugh, Mittal, Gibb, & Coles, 2018) as well as social impairments, neurocognitive deficits, and structural brain abnormalities (Calkins et al., 2014; Satterthwaite et al., 2016). These findings converge to suggest that subclinical expression of psychosis phenotype may be useful in the understanding of the pathogenesis of psychosis as well as for the early identification of adolescents potentially at risk for psychotic spectrum before reaching clinical outcome.

In the last two decades, early detection of psychosis has become an extensive focus of research and debate, although it remains a significant challenge. Methods to screen and identify individuals at risk to develop psychotic disorders are essential for prevention strategies. Reliable identification of individuals at high risk and timely prophylactic intervention may delay, ameliorate, or prevent the onset of frank psychotic symptoms, as well as reduce its possible impact on many levels (e.g., personal, familiar, educational, and occupational; Fusar‐Poli, Carpenter, Woods, & McGlashan, 2014; Fusar‐Poli, McGorry, & Kane, 2017). In short, if the possibility of psychosis prevention is opened, it is necessary to have adequate tools to identify those individuals at high risk; however, the psychosis risk assessment is a complex topic that is not exempt from dilemmas and difficulties (e.g., stigmatization and false positives; Fonseca‐Pedrero, 2018). Moreover, the costs and risks related to the screening processes must be clearly defined, and benefits must overcome them (Polanczyk, Fatori, & Matijasevich, 2018).

At present, there are several measurement instruments available for clinicians and researchers to document the risk for psychosis (e.g., PLE, schizotypal traits, and at‐risk mental states; for review, see Fonseca Pedrero & Debbané, 2017; Kline & Schiffman, 2014; Mason, 2015). The Structured Interview for Prodromal Syndromes (SIPS; Miller et al., 2003), the Comprehensive Assessment of At‐Risk Mental States (Yung et al., 2005), and the Schizophrenia Proneness Instrument (Schultze‐Lutter, Addington, Ruhrmann, & Klosterkötter, 2007) are well‐known interview instruments to assess ultra‐high risk (UHR) states for psychosis. Within the self‐reports arena, the PRIME Screen (Miller, Cicchetti, Markovich, McGlashan, & Woods, 2004), the Youth Psychosis At‐Risk Questionnaire (Ord, Myles‐Worsley, Blailes, & Ngiralmau, 2004), and the Prodromal Questionnaire (PQ; Loewy, Bearden, Johnson, Raine, & Cannon, 2005), or its brief version (PQ‐B; Loewy, Pearson, Vinogradov, Bearden, & Cannon, 2011), are some good examples.

In particular, recent revision supports the use of the PQ and PQ‐B as screening tools for UHR in different settings (Savill, D'Ambrosio, Cannon, & Loewy, 2018). Particularly, the PQ items are based partly on the SIPS (Miller et al., 2003) in addition to the Schizotypal Personality Questionnaire (Raine, 1991). The PQ‐B was developed in order to improve its efficiency and accuracy. The authors retained only the items related with the positive dimension of psychosis phenotype as those are the basis for the prodromal syndromes (e.g., attenuated psychosis syndrome). Previous studies conducted have shown that PQ‐B is a good measure in the screening of mental high‐risk states (sensibility and specificity) in adolescents and young adults (Savill et al., 2018). This tool has demonstrated adequate reliability coefficients (e.g., internal consistency and temporal stability) and several sources of validity evidence (Brodey et al., 2018; Cicero, Krieg, & Martin, 2017; Fonseca‐Pedrero, Gooding, Ortuño‐Sierra, & Paino, 2016a; Kline et al., 2015; Kline & Schiffman, 2014; Loewy, Therman, Manninen, Huttunen, & Cannon, 2012; Okewole et al., 2015; Savill et al., 2018; Xu et al., 2016). For example, Fonseca‐Pedrero, Gooding, et al. (2016a) found an essentially unidimensional structure using a nonrandom sample of adolescents and an internal consistency of the PQ‐B total score of 0.93. In another study, Cicero et al. (2017), using a large U.S. college sample, found that the one‐factor model for the Distress scale fits the data well and this model also showed configural, scalar, and strong invariance across Asian, White, Hispanic, and Multiracial samples.

To date, we have little information about the validity of PQ‐B scores as a psychosis risk screener outside of mental health services or young adult samples. To the best of our knowledge, no previous studies have validated the PQ‐B in large and representative samples of nonclinical youths. For instance, no earlier studies have developed specific test norms for this age group, relevant to compare cut‐off scores and standardized testing practices. Moreover, due to low positive predictive values of subclinical psychotic experiences (Kaymaz et al., 2012; Livny et al., 2017) and the low transition rates of those at‐risk mental state for psychosis (e.g., transition rate of 36% at 3 years; Fusar‐Poli et al., 2012), it is interesting to use these tools in combination to other mental health indicators (e.g., emotional disturbance and suicide behaviors) and additional factors (e.g., child maltreatment, cannabis use, and polymorphism) in order to reduce the duration of untreated psychosis (Oliver et al., 2018), improve the prediction of later hospitalization for psychotic disorder (Radua et al., 2018), and ameliorate the negative outcomes associated with first‐episode psychosis (Fusar‐Poli et al., 2017).

Within this research framework, the main purpose of the present study was to validate the PQ‐B as well as to examine the relationship between PLEs and behavioral and emotional problems, prosocial behavior, suicidal ideation, and bipolar‐like experiences in a community‐derived sample of nonclinical adolescents. This main goal included the study of the (a) test norms of PQ‐B scores; (b) internal structure of the PQ‐B scores and measurement invariance by gender using Confirmatory Factor Analysis (CFA); (c) reliability of the PQ‐B scores as well as test information function using Item Response Theory (IRT) framework; (d) Differential Item Functioning (DIF) by gender; and (e) associations of PQ‐B scores with psychopathology.

2. METHOD

2.1. Participants

Stratified random cluster sampling was conducted at classroom level, in an approximate population of 15,000 students selected from La Rioja (region located in northern Spain). The students belonged to different public and concerted Educational Centers of Compulsory Secondary Education and Vocational Training, as well as to different socioeconomic levels. The layers were created as a function of the geographical zone and the educational stage.

The initial sample consisted of 1,881 students, eliminating those participants who presented a high score on the Oviedo Infrequency Response Scale (more than 3 points; n = 104), an age older than 19 (n = 170), or those who did not complete the test (n = 76). A total of 1,588 students, 739 men (46.5%) and 849 (53.5%) women, belonging to 34 schools and 98 classrooms participated in the study. The mean age was 16.13 years (SD = 1.36), ranging from age 14 to 19 years (14 years, n = 213; 15 years, n = 337; 16 years, n = 400; 17 years, n = 382; 18 years, n = 180; 19 years, n = 76).

With regards to nationality, the distribution was as follows: 89.9% Spanish, 3.7% Latin American (Bolivia, Argentina, Colombia, and Ecuador), 0.7% Portuguese, 2.4% Romanian, 1% Moroccan, 0.7% Pakistani, and 2% other nationalities.

2.2. Instruments

2.2.1. The PQ‐B

The PQ‐B is a psychosis risk screening measure containing 21 items that are answered in a dichotomous response format (true/false; Loewy et al., 2011). The PQ‐B asks additional questions regarding the extent/severity of impairment and distress, rated on Likert‐type (1 = Strongly disagree to 5 = Strongly agree). The Spanish adaptation of the PQ‐B has demonstrated adequate psychometric properties (Fonseca‐Pedrero, Ortuno‐Sierra, et al., 2016b).

2.2.2. The Paykel Suicide Scale (PSS)

The PSS is a self‐report tool designed for the evaluation of suicidal ideation. It consists of five items with a dichotomous response system Yes/No (score, 1 and 0, respectively; Paykel, Myers, Lindenthal, & Tanner, 1974). The total scores range from 0 to 5. The time frame to which the questions refer is the last year. Higher scores are related with high severity on suicidal ideation. The Spanish adaptation of the PSS has demonstrated adequate psychometric properties (Bousoño et al., 2017; Fonseca‐Pedrero et al., 2018).

2.2.3. The Strengths and Difficulties Questionnaire (SDQ)

The SDQ is a self‐report questionnaire that is widely used in the assessment of different emotional and behavioral problems related to mental health in adolescents (Goodman, 1997). The SDQ is made up of 25 statements distributed across five subscales: emotional symptoms, conduct problems, hyperactivity, peer problems, and prosocial behavior. The first four subscales yield a total difficulties score. In this study, we used a Likert‐type response format with three options (0 = Not true, 1 = Somewhat true, 2 = Certainly true). The validated Spanish version of the SDQ was used in the present study (Ortuño‐Sierra, Aritio & Fonseca‐Pedero, 2018; Ortuño‐Sierra, Chocarro, Fonseca‐Pedrero, Riba, & Muñiz, 2015).

2.2.4. The Mood Disorder Questionnaire (MDQ)

The MDQ consists of 13 yes/no items based on the DSM‐IV criteria for bipolar disorder (Hirschfeld et al., 2000). A result is considered positive if the participant replies affirmatively to seven or more items of the 13 proposed and if, in addition, the symptoms described occurred during the same time period (Criterion 2) and represented moderate or severe problems (Criterion 3). In this study, we used the version validated in Spanish adolescents and young adults (Fonseca‐Pedrero, Ortuno‐Sierra, Paino, & Muniz, 2016b).

2.2.5. The Penn Matrix Reasoning Test

This is a task of the Penn Computerized Neurocognitive Battery‐Child version. The Penn Matrix Reasoning Test was developed to measure nonverbal reasoning (using matrix reasoning problems as used in the Raven's Progressive Matrices Test), within complex cognition domain. This task, composed of 20 items, may be considered as IQ estimated. The battery includes different neurobehavioral indicators with different tasks adapted to guarantee psychometric properties for children (Gur et al., 2012; Moore, Reise, Gur, Hakonarson, & Gur, 2015).

2.2.6. The Family Affluence Scale‐II

Socioeconomic status was measured using a four‐item child‐appropriate measure of family wealth with score ranging from 0 to 9. Previous international studies have demonstrated its adequate psychometric properties (Boyce et al., 2006).

2.2.7. The Oviedo Infrequency Scale (INF‐OV)

INF‐OV was administered to the participants to detect those who responded in a random, pseudorandom, or dishonest manner (Fonseca‐Pedrero, Lemos‐Giráldez, Paino, Villazón‐García, & Muñiz, 2009). The INF‐OV instrument is a self‐report composed of 12 items in a 5‐point Likert scale format (1 = Completely disagree; 5 = Completely agree). Items of the INF‐OV included questions such as the following: “The distance between Madrid and New York is higher that the distance between Madrid and Barcelona.” Students with more than three incorrect responses on the INF‐OV scale were eliminated from the sample. This criterion is based in previous works (Fonseca‐Pedrero, Gooding, et al., 2016a) and in the Chapman Infrequency Scales (Chapman & Chapman, 1983).

2.3. Procedure

The research was approved by the Educational Government of La Rioja and the Ethical Committee of Clinical Research of La Rioja (CEICLAR). The tests and neurocognitive battery were administered collectively, through personal computers, in groups of 10 to 30 students, during normal school hours, and in a classroom specially prepared for this purpose. Administration took place under the supervision of the researchers trained in a standard protocol. No incentive was provided for their participation. For participants under 18, parents were asked to provide a written informed consent in order for their child to participate in the study. Participants were informed of the confidentiality of their responses and of the voluntary nature of the study.

2.4. Data analyses

First, we calculated the prevalence of the self‐reported PLEs as well as the descriptive statistics PQ‐B items and the tools used. We also developed specific test norms of PQ‐B in order to determine the relative standing of an individual who had taken the test.

Second, we examined the psychometric properties of the PQ‐B scores. In order to analyze the internal structure of the PQ‐B, a CFA was performed. Based on previous studies that have tested the factorial structure of this measure that assess positive items of psychosis, a unidimensional model was tested. The weighted least squares means and variance adjusted (WLSMV) estimator for ordinal items was used. The following goodness‐of‐fit indices were employed: chi‐square (χ2), comparative fit index (CFI), Tucker‐Lewis index, root mean square error of approximation, 90% confidence interval, and weighted root mean square residual. Hu and Bentler (1999) suggested that root mean square error of approximation should be 0.06 or less for a good model fit and CFI and Tucker‐Lewis index should be 0.95 or more, though any value over 0.90 tends to be considered acceptable. For weighted root mean square residual, values less than 0.95 indicate good model fit (Yu & Muthén, 2002). In order to test measurement invariance across gender, successive multigroup CFAs were conducted. The ΔCFI was used to determine cases where nested models were practically equivalent. Internal consistency of the PQ‐B scores were estimated using McDonald's Omega. In addition, from the IRT framework, IRT parameters from two‐parameter model (2pl) were estimated and the test information function was computed.

Third, DIF analysis was carried out according to gender. DIF is considered as validity evidence based on internal structure. The presence of DIF presumes that the probability of a person (or group) obtaining a correct response does not only depend solely on that person's level in the object of measurement but rather is also conditioned by whether the person belongs to a certain social, cultural, linguistic group, and so on, generating a lack of metric equivalence among scores (Gómez‐Benito, Sireci, Padilla, Hidalgo, y Benítez, 2018). In order to detect DIF by gender, the Mantel–Haenszel method for dichotomous items were employed.

Finally, the associations between PQ‐B scores, emotional and behavioral problems, prosocial behavior, suicide ideation, and bipolar‐like experiences were calculated using two types of data analyses: (a) Pearson's correlations between measures and (b) logistic regression analyses. Based in previous studies, the PQ‐B scores were dichotomized using a cut‐off score of ≥9 points in frequency. Age, gender, socioeconomic status, and estimated IQ were used as covariates.

SPSS 22.0 (IBM Corp Released, 2013), FACTOR 10.5.01 (Ferrando & Lorenzo‐seva, 2017), DifR Package 5.0 of R environment (Magis, Beland, & Raiche, 2018), and Mplus 7.4 (Muthén & Muthén, 1998–2015) were used for data analyses.

3. RESULTS

3.1. Descriptive statistics

Descriptive statistics for the subscales or total scores of the measuring instruments used are shown in Table 1. Descriptive statistics of the PQ‐B frequency items are depicted in Table 2. A total of 91.4% of the sample responded affirmatively to at least one item of the PQ‐B. A total of 27.3% of the adolescents scored ≥9 (standard cut‐off for frequency score).

Table 1.

Descriptive statistics of the measures

Mean SD Skewness Kurtosis Minimum Maximum
PQ‐B frequency 6.03 4.39 0.62 −0.28 0 20
PQ‐B distress 11.10 11.47 1.64 3.48 0 77
MDQ 5.03 2.83 0.14 −0.61 0 13
PSS 0.90 1.33 1.42 1.01 0 5
SDQ emotional problems 3.54 2.45 0.50 −0.53 0 10
SDQ conduct problems 2.00 1.68 0.96 1.00 0 10
SDQ peer problems 1.54 1.57 1.33 1.97 0 9
SDQ hyperactivity 4.33 2.18 0.10 −0.49 0 10
SDQ prosocial behavior 8.56 1.49 −1.31 2.01 0 10
SDQ total difficulties 11.40 5.24 0.44 0.02 0 33
FAS‐II 6.14 1.69 −0.27 −0.38 1 9
PMRT 5.37 4.51 0.35 −0.96 0 19
Open in a new tab

Note. PQ‐B: Prodromal Questionnaire‐Brief; MDQ: Mood Disorder Questionnaire; PSS: Paykel Suicide Scale; SDQ: Strengths and Difficulties Questionnaire; FAS‐II: Family Affluence Scale‐II; PMRT: Penn Matrix Reasoning Test.

Table 2.

Descriptive statistics, standardized factor loadings, and Item Response Theory parameters for PQ‐B frequency scores

CFA Standardized factor loadings IRT
Items Mean SD Discrimination index Total sample Male Female a b
1 0.16 0.36 0.316 0.499 0.531 0.475 1.130 1.817
2 0.43 0.50 0.441 0.614 0.570 0.662 1.322 0.288
3 0.14 0.35 0.371 0.594 0.593 0.600 1.419 1.678
4 0.15 0.36 0.272 0.442 0.463 0.424 0.964 2.081
5 0.26 0.44 0.465 0.667 0.669 0.654 1.596 0.959
6 2.00 0.44 0.358 0.521 0.550 0.493 1.046 1.152
7 0.16 0.37 0.312 0.497 0.495 0.549 1.081 1.821
8 0.48 0.50 0.412 0.572 0.573 0.560 1.173 0.113
9 0.21 0.41 0.394 0.587 0.567 0.599 1.343 1.336
10 0.31 0.46 0.454 0.632 0.609 0.648 1.453 0.757
11 0.32 0.47 0.477 0.663 0.701 0.630 1.572 0.723
12 0.45 0.50 0.457 0.635 0.652 0.611 1.407 0.221
13 0.15 0.35 0.392 0.619 0.652 0.589 1.550 1.568
14 0.45 0.50 0.485 0.668 0.644 0.687 1.554 0.226
15 0.21 0.41 0.392 0.580 0.557 0.599 1.336 1.296
16 0.30 0.46 0.334 0.472 0.471 0.467 0.944 1.029
17 0.22 0.41 0.447 0.654 0.652 0.655 1.596 1.138
18 0.55 0.50 0.367 0.521 0.569 0.465 1.037 −0.196
19 0.21 0.41 0.373 0.567 0.522 0.632 1.203 1.382
20 0.13 0.34 0.471 0.753 0.724 0.807 2.204 1.407
21 0.47 0.50 0.457 0.633 0.593 0.669 1.356 0.136
Open in a new tab

Note. PQ‐B: Prodromal Questionnaire–Brief; CFA: Confirmatory Factor Analysis; IRT: Item Response Theory.

Statistically significant differences were found by gender in the PQ‐B total frequency score (M (SD)male = 5.55 (4.29); M (SD)female = 6.44 (4.46); t (1586) = −4.035; p < 0.05, d = 0.203) and total distress score (M (SD)male = 9.87 (11.22); M (SD)female = 12.17 (11.59); t (1586) = −4.012; p < 0.05, d = 0.201). Although age was negatively associated with PQ‐B total score, Pearson coefficient value was irrelevant (r = −0.078, p < 0.05). The norms for the PQ‐B scores are showed in Table 3.

Table 3.

Percentiles of the PQ‐B frequency and distress scores

Total sample Male Female
Pc Frequency Distress Frequency Distress Frequency Distress
1 0 0 0 0 0 0
5 0 0 0 0 0 0
10 1 0 0 0 1 0
20 2 2 1 1 2 2
30 3 3 2 2 3 4
40 4 5 4 4 5 7
50 5 8 5 6 6 9
60 7 11 6 9 7 12
70 8 14 8 12 8 15
80 10 19 9 18 10 20
90 12 26 12 25 13 27
95 14 35 14 31 15 36
99 17 51 16 48 18 52
Open in a new tab

Note. Pc: percentile; PQ‐B: Prodromal Questionnaire–Brief.

3.2. Factorial structure and measurement invariance by gender

Table 4 shows the goodness‐of‐fit indices for the one‐factor model tested. This model yielded adequate goodness‐of‐fit indices for the total sample. The standardized factor loadings for the total sample and by gender are shown in Table 2. All standardized factor loadings estimated were statistically significant (p < 0.01).

Table 4.

Goodness‐of‐fit indices for the hypothetical models tested and measurement invariance across gender

Model χ 2 df CFI TLI RMSEA (90% CI) WRMR ΔCFI
One‐factor 564.25 189 .961 .957 .035 (.032–.039) 1.391
Measurement invariance
Male (n = 739) 307.42 189 .971 .968 .029 (.023–.035) 1.042
Female (n = 849) 395.22 189 .961 .956 .036 (.031–.041) 1.164
Configural invariance 699.02 378 .966 .962 .033 (.029–.036) 1.562
Strong invariance 714.72 397 .966 .964 .032 (.028–.035) 1.620 −.01
Open in a new tab

Note. χ 2: chi square; df: degrees of freedom; CFI: comparative fit index; TLI: Tucker‐Lewis index; RMSEA: root mean square error of approximation; CI: confidence interval; WRMR: weighted root mean square residual; ΔCFI: change in comparative fix index.

Given that the one‐factor model evidenced good model fit, we therefore tested the measurement invariance of the PQ‐B scores across gender. Goodness‐of‐fit indices for males and females were adequate (see Table 4). The configural invariance and strong invariance models showed an adequate fit to the data. The ΔCFI between the constrained and unconstrained models was under 0.01, and the DIFFTEST chi‐square was nonsignificant (χ 2 (19) = 29.694; p = 0.0558), indicating that strong measurement invariance across gender for this unidimensional model was supported.

From latent trait measurement models for binary responses, it is easy to compute IRT parameters (a = Item discrimination; b = Item difficulty or intercept) from CFA. An item factor model for binary outcomes is the same as a two‐parameter IRT model (e.g., Kamata & Bauer, 2008). In Table 2, item parameters of the two‐parameter IRT model are depicted.

3.3. DIF by gender

Analysis of the DIF revealed that a total of 10 items (2, 5, 7, 8, 12, 13, 18, 19, 20, and 21) showed differential functioning according to gender. However, only two items revealed large effect size (Items 7 and 20).

3.4. Reliability of the PQ‐B scores

The internal consistency of the PQ‐B total frequency score, estimated with McDonald's Omega, was 0.92. Discrimination indices of the PQ‐B items were higher than 0.30, except for Item 4 (0.27; see Table 2).

According to the IRT framework, the test information function provides an optimal estimation at the high latent trait (values between 0 and 2.5; see Figure 1). The tool reduces its accuracy around the lowest level of the latent trait (<−2).

Figure 1.

Figure 1

Open in a new tab

Test information function of the Prodromal Questionnaire–Brief

3.5. Links between PLEs, emotional and behavioral problems, suicidal ideation, and bipolar‐like experiences

Pearson's correlations between measures are shown in Table 5. All associations were statistically significant (p < 0.01). PLE, measured by PQ‐B, were positively associated with emotional and behavioral problems, suicidal ideation and attempts, and bipolar‐like experiences, and negatively associated with prosocial behavior.

Table 5.

Pearson's correlations between measures

1 2 3 4 5 6 7 8 9 10
PQ‐B frequency (1)
PQ‐B distress (2) .838**
MDQ total (3) .394** .332**
PSS total (4) .446** .471** .217**
SDQ emotional problems (5) .438** .463** .202** .466**
SDQ conduct problems (6) .320** .324** .314** .249** .177**
SDQ peer problems (7) .355** .383** .140** .375** .350** .238**
SDQ hyperactivity (8) .296** .272** .309** .184** .231** .380** .122**
SDQ prosocial behavior (9) −.084** −.097** −.068** −.114** −.021** −.300** −.190** −.110**
PMRT (10) .008 .072** .001 .040 .05 .109** .000 .103** −.031
Open in a new tab

Note. PQ‐B: Prodromal Questionnaire‐Brief; MDQ: Mood Disorder Questionnaire; PSS: Paykel Suicide Scale; SDQ: Strengths and Difficulties Questionnaire; PMRT: Penn Matrix Reasoning Test.

**

p < 0.01.

We then employed a forward stepwise logistic regression analysis (controlling by gender, socioeconomic status, and estimated IQ) with MDQ, PSS, and SDQ scores as independent variables and dichotomized PQ‐B scores (cut‐off score 9 or higher) as dependent variables. The results are presented in Table 6. Using a conditional forward stepwise method, all variables, except SDQ Prosocial Behavior, were entered into the regression model with a statistically significant power (p < 0.05). The percentage of correct diagnoses using this model for predicting PQ‐B dichotomized scores was 78.3%. The adjusted R 2 for this regression model was 0.326, and the Cox and Snell's R 2 was 0.225.

Table 6.

Logistic regression analyses

OR 95% CI
B SE Wald df p OR Lower Upper
Total MDQ 0.196 0.026 56.723 1 <0.001 1.217 1.156 1.28
Total PSS 0.278 0.052 28.763 1 <0.001 1.320 1.193 1.462
SDQ emotional problems 0.185 0.032 34.361 1 <0.001 1.203 1.131 1.280
SDQ conduct problems 0.160 0.044 12.881 1 <0.001 1.173 1.075 1.280
SDQ peer problems 0.198 0.044 19.955 1 <0.001 1.220 1.118 1.331
SDQ hyperactivity 0.070 0.034 4.287 1 0.038 1.072 1.004 1.145
Constant −4.436 0.557 63.409 1 <0.001 0.012
Open in a new tab

Note. PSS = Paykel Suicide Scale; SDQ = Strengths and Difficulties Questionnaire; MDQ = Mood Disorder Questionnaire; OR: odds ratio; CI: confidence interval.

4. DISCUSSION

There have been few attempts to identify adolescents potentially at high risk for psychosis using brief psychosis risk screening measures as PQ‐B (Loewy et al., 2011). Although the PQ‐B is potentially a good candidate to serve as a psychosis risk screening tool, its psychometric properties or standardized norms have not been extensively examined in large and representative samples of adolescents derived from the general population. According to Kline and Schiffman (2014), identifying high‐risk individuals for psychosis through low‐cost, brief methods is essential for the success of scalable prevention efforts. The current study therefore focused on the validity of PQ‐B to assess self‐reported PLEs and its associations with psychopathological psychometric indicators in a developmental stage of special risk for the development of the first symptoms of psychosis and other mental health disorders.

First, the results showed that PLEs are common in nonclinical samples of adolescents. A percentage of 27.3 of those adolescents scored higher using the standardized cut‐off score (>9 points). Prior studies have found similar results using self‐reports and clinical interviews (Calkins et al., 2014; Dolphin, Dooley, & Fitzgerald, 2015; Fonseca Pedrero & Debbané, 2017; Kelleher et al., 2012a). For instance, Calking et al. (2014) in a U.S. representative sample of 7,054 adolescents found that between 3.8% and 17.6% of the sample endorsed definitely agree on some psychotic symptom items. These set of experiences fall within a spectrum of “normal” developmental experience and, in most cases, disappear over time without progression to a clinical disorder. These findings offer support to the existence of a psychometric phenotypic continuity between the clinical and subclinical psychosis phenotypes (Linscott & van Os, 2013).

The results have demonstrated that the psychometric properties of the PQ‐B scores were adequate in this sample. The internal consistency of the scores was good. Using IRT as a modern psychometric approach, the tool measured with high accuracy individuals who scored at the high latent trait. IRT models have potentially significant roles to play in the advancement of measurement practice in this arena as well as for test development, scale construction, score reporting, and test evaluation. The results also suggest that the unidimensional factorial structure of the PQ‐B items showed adequate goodness of fit index. This model demonstrated measurement invariance across gender. Previous studies conducted in adolescents and young adults have found similar results (Cicero et al., 2017; Fonseca‐Pedrero, Ortuno‐Sierra, et al., 2016b). For instance, Cicero et al. (2017), in a U.S. college sample, found that this dimensional structure was equivalent across several ethnic groups. In another study, Fonseca‐Pedrero, Gooding, et al. (2016a), using a convenience sample of nonclinical adolescents, found a unidimensional factor structure. A large body of research showed that PQ‐B is a good measure for screening clinical high‐risk states revealing, in addition, adequate levels of predictive validity and convergence with other psychosis screening measures (e.g., SIPS and Comprehensive Assessment of At‐Risk Mental States; Brodey et al., 2018; Cicero et al., 2017; Fonseca‐Pedrero, Ortuno‐Sierra, et al., 2016b; Kline et al., 2015; Kline & Schiffman, 2014; Loewy et al., 2012; Okewole et al., 2015; Savill et al., 2018; Xu et al., 2016). This empirical evidence suggests that this tool could be useful for more comprehensive identifying and recruiting strategies of individuals from both nonclinical and clinical high‐risk samples. In addition, these kind of measures have to expand its use to the educational and primary care settings, where a required specialized training (i.e., costs and time) is not possible (Kline et al., 2015). To date, however, the psychometric validity of psychosis risk measures is markedly variable across studies because it is in its nascent stage. Thus, more research is needed in order to establish reliable norms and screening thresholds for different settings, representative samples, and cultures.

Analysis of DIF revealed that two PQ‐B items functioned differentially according to gender with large effect. These results are partially contradictory with those found in the measurement invariance study, however, the combination of quantitative methods, in validation studies, can help to improve the validity of test score interpretations (Gómez et al., 2018). Although these results are preliminary and future studies must replicate other samples of the population, it is worth mentioning that the presence of DIF does not guarantee equity in the measurement process, so that some consideration of ethical and legal aspects is relevant (e.g., the appropriateness of selecting at‐risk individuals based on their PQ‐B scores). Likewise, it is advisable for DIF analysis to be incorporated as routine practice in statistical analyses on instruments for assessing extended psychosis phenotype and UHR states.

PLEs were moderately associated with a range of psychopathology symptoms such as emotional and behavioral problems, suicidal ideation, bipolar‐like experiences, and poor prosocial skills. In addition, these psychopathology indicators predicted higher PQ‐B scores. Previous studies conducted in adolescent and young adults from both clinical and nonclinical populations have shown similar results to those found in this study. For instance, adolescents who reported PLEs, indicated a wide range of mental health problems, such as depressive symptoms (Armando et al., 2010; Fonseca‐Pedrero, Paino, Lemos‐Giráldez, & Muñiz, 2011), emotional and behavioral problems (Fonseca‐Pedrero et al., 2017; Wigman et al., 2011), and suicidal behavior (ideation and attempts; Kelleher et al., 2013; Kelleher, Cederlöf, & Lichtenstein, 2014). To date, however, few studies have demonstrated the relationship between PLEs and bipolar‐like experiences at a subclinical level in nonclinical samples. Here, PQ‐B and MDQ scores were positively and strongly associated. These results may provide valuable means to examine the dimensional relationships between schizophrenia and bipolar phenotypes at a nonclinical level, beyond traditional phenomenological diagnostic systems. These results also may be convergent with the increasing evidence for phenomenological, biological, and genetic overlap between schizophrenia and bipolar disorder (Keshavan et al., 2011; Tamminga et al., 2013). In sum, the presence of PLEs during adolescence is associated with multiple concurrent psychopathology symptoms, and it appears to be a fundamental part of the psychosis risk picture.

The results obtained in the present study must be interpreted in light of the following limitations. First, adolescence is a developmental period in which the brain, cognition, and personality are still consolidating. It is possible that these bio‐psychological changes affect the results found. In this regard, it is important to bear in mind that unusual experiences are relatively frequent in adolescence and may be related to the emotional turbulence and tensions generated by the quest for independence and by the family itself, as well as to other characteristic processes of adolescence (e.g., egocentrism, fantasy and imaginary audiences, or feelings of uniqueness). Second, in the present study, we only investigated the PQ‐B as a psychosis risk screening measure. There is some inherent problems in the use of self‐reports as indirect indicators of this phenomena (e.g., acquiescence; Suárez et al., 2018). In addition, these measures have been associated with stigmatization, negative labelling, and stereotypes. Finally, it should be borne in mind that this study was of a cross‐sectional nature, so that we cannot make cause–effect inferences. These limitations may undermine the validity and generalizability of the results found.

In summary, the current research provides further support for the reliability and validity of PQ‐B scores in a representative sample of nonclinical adolescents. The knowledge of evidences of validity of this tool is relevant in order to allow an early identification of those at risk for psychotic spectrum disorders in community or educational settings. In addition, it allows for the implementation of close‐in strategies or a two‐stage process model in order to obtain a further comprehensive evaluation of mental state or an early intervention in order to improve the outcome (Rietdijk et al., 2012).

Future studies should continue analyzing the protective and risk factors for psychosis (Fumero, Marrero, & Fonseca‐Pedrero, 2018). Furthermore, it is relevant to add new psychometric procedures (network theory;e.g., Borsboom, 2017), technology (digital phenotyping; Insel, 2017), and framework (RDoC; Insel et al., 2010) in order to improve our predictive capacity and to find etiological mechanisms to prevent serious mental health disorders in young people.

FUNDING

Spanish Ministry of Science and Innovation (MICINN); Instituto Carlos III, Center for Biomedical Research in the Mental Health Network (CIBERSAM); “Ayudas Fundación BBVA a Investigadores y Creadores Culturales, convocatoria 2015”; and “Ayudas Fundación BBVA a equipos de investigación científica 2017”.

CONFLICT OF INTEREST

All the authors have declared that there are no conflicts of interest in relation to this study.

Fonseca‐Pedrero E, Inchausti F, Pérez‐Albéniz A, Ortuño‐Sierra J. Validation of the Prodromal Questionnaire–Brief in a representative sample of adolescents: Internal structure, norms, reliability, and links with psychopathology. Int J Methods Psychiatr Res. 2018;27:e1740 10.1002/mpr.1740

REFERENCES

  1. Armando, M. , Nelson, B. , Yung, A. R. , Ross, M. , Birchwood, M. , Girardi, P. , & Nastro, P. F. (2010). Psychotic‐like experiences and correlation with distress and depressive symptoms in a community sample of adolescents and young adults. Schizophrenia Research, 119, 258–265. [DOI] [PubMed] [Google Scholar]
  2. Borsboom, D. (2017). A network theory of mental disorders. World Psychiatry, 16, 5–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bousoño, M. , Al‐Halabí, S. , Burón, P. , Garrido, M. , Díaz‐Mesa, E. M. , Galván, G. , … Bobes, J. (2017). Uso y abuso de sustancias psicotrópicas e internet, psicopatología e ideación suicida en adolescentes. [Substance use or abuse, internet use, psychopathology and suicidal ideation in adolescents]. Adicciones, 29, 97–104. [DOI] [PubMed] [Google Scholar]
  4. Boyce, W. , Torsheim, T. , Currie, C. , & Zambon, A. (2006). The Family Affluence Scale as a measure of national wealth: Validation of an adolescent self‐report measure. Social Indicators Research, 78, 473–487. [Google Scholar]
  5. Brodey, B. , Girgis, R. , Favorov, O. , Addington, J. , Perkins, D. , Bearden, C. , … Brodey, I. (2018). The Early Psychosis Screener (EPS): Quantitative validation against the SIPS using machine learning. Schizophrenia Research.. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Calkins, M. E. , Moore, T. M. , Merikangas, K. R. , Burstein, M. , Satterthwaite, T. D. , Bilker, W. B. , … Gur, R. E. (2014). The psychosis spectrum in a young U.S. community sample: Findings from the Philadelphia Neurodevelopmental Cohort. World Psychiatry, 13(3), 296–305. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Chapman, L. J. , & Chapman, J. P. (1983). Infrequency scale. WI: Unpublished test Madison. [Google Scholar]
  8. Cicero, D. C. , Krieg, A. , & Martin, E. A. (2017). Measurement invariance of the Prodromal Questionnaire–Brief among White, Asian, Hispanic, and Multiracial populations. Assessment. 10.1177/1073191116687391, 107319111668739 [DOI] [PubMed] [Google Scholar]
  9. Corp Released, I. B. M. (2013). IBM SPSS statistics for Windows, version 22.0. Armonk, NY: IBM Corp. [Google Scholar]
  10. Debbané, M. , Eliez, S. , Badoud, D. , Conus, P. , Flückiger, R. , & Schultze‐Lutter, F. (2015). Developing psychosis and its risk states through the lens of schizotypy. Schizophrenia Bulletin, 41, S396–S407. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Dolphin, L. , Dooley, B. , & Fitzgerald, A. (2015). Prevalence and correlates of psychotic like experiences in a nationally representative community sample of adolescents in Ireland. Schizophrenia Research, 169, 217–241. [DOI] [PubMed] [Google Scholar]
  12. Ferrando, P. J. , & Lorenzo‐seva, U. (2017). Program FACTOR at 10: Origins, development and future directions. Psicothema, 29(2), 236–240. 10.7334/psicothema2016.304 [DOI] [PubMed] [Google Scholar]
  13. Fonseca Pedrero, E. , & Debbané, M. (2017). Schizotypal traits and psychotic‐like experiences during adolescence: An update. Psicothema, 29, 5–17. [DOI] [PubMed] [Google Scholar]
  14. Fonseca‐Pedrero, E. (2018). Evaluación de los trastornos del espectro psicótico [Assessment of psychotic spectrum disorders]. Madrid: Pirámide. [Google Scholar]
  15. Fonseca‐Pedrero, E. , Gooding, D. C. , Ortuño‐Sierra, J. , & Paino, M. (2016a). Assessing self‐reported clinical high risk symptoms in community‐derived adolescents: A psychometric evaluation of the Prodromal Questionnaire–Brief. Comprehensive Psychiatry, 66, 201–208. 10.1016/j.comppsych.2016.01.013 [DOI] [PubMed] [Google Scholar]
  16. Fonseca‐Pedrero, E. , Inchausti, F. , Pérez, L. , Aritio, A. , Ortuño‐Sierra, J. , Sánchez‐García, A. , … Pérez de Albéniz, A. (2018). Suicidal ideation in a community‐derived sample of Spanish adolescents. Revista de Psiquiatria Y Salud Mental, 11, 76–85. 10.1016/j.rpsm.2017.07.004 [DOI] [PubMed] [Google Scholar]
  17. Fonseca‐Pedrero, E. , Lemos‐Giráldez, S. , Paino, M. , Villazón‐García, U. , & Muñiz, J. (2009). Validation of the Schizotypal Personality Questionnaire Brief form in adolescents. Schizophrenia Research, 111, 53–60. [DOI] [PubMed] [Google Scholar]
  18. Fonseca‐Pedrero, E. , Ortuño‐Sierra, J. , Chocarro, E. , Inchausti, F. , Debbané, M. , & Bobes, J. (2017). Psychosis risk screening: Validation of the Youth Psychosis At‐Risk Questionnaire–Brief in a community‐derived sample of adolescents. International Journal of Methods in Psychiatric Research, 26(4)e1543. 10.1002/mpr.1543 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Fonseca‐Pedrero, E. , Ortuno‐Sierra, J. , Paino, M. , & Muniz, J. (2016b). Screening the risk of bipolar spectrum disorders: Validity evidence of the Mood Disorder Questionnaire in adolescents and young adults. Rev Psiquiatr Salud Ment, 9(1), 4–12. 10.1016/j.rpsm.2015.04.003 [DOI] [PubMed] [Google Scholar]
  20. Fonseca‐Pedrero, E. , Paino, M. , Lemos‐Giráldez, S. , & Muñiz, J. (2011). Schizotypal traits and depressive symptoms in nonclinical adolescents. Comprehensive Psychiatry, 52, 293–300. [DOI] [PubMed] [Google Scholar]
  21. Fumero, A. , Marrero, R. J. , & Fonseca‐Pedrero, E. (2018). Well‐being in schizotypy: The effect of subclinical psychotic experiences. Psicothema, 30, 177–182. 10.7334/psicothema2017.100 [DOI] [PubMed] [Google Scholar]
  22. Fusar‐Poli, P. , Bonoldi, I. , Yung, A. R. , Borgwardt, S. , Kempton, M. J. , Valmaggia, L. , … McGuire, P. (2012). Predicting psychosis: Meta‐analysis of transition outcomes in individuals at high clinical risk. Archives of General Psychiatry, 69, 220–229. [DOI] [PubMed] [Google Scholar]
  23. Fusar‐Poli, P. , Carpenter, W. T. , Woods, S. W. , & McGlashan, T. H. (2014). Attenuated psychosis syndrome: Ready for DSM‐5.1? Annual Review of Clinical Psychology, 10, 155–192. [DOI] [PubMed] [Google Scholar]
  24. Fusar‐Poli, P. , McGorry, P. , & Kane, J. (2017). Improving outcomes of first‐episode psychosis: An overview. World Psychiatry, 16, 251–265. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Gómez‐Benito, J. , Sireci, S. , Padilla, J. L. , Hidalgo, M. D. , & Benítez, I. (2018). Differential Item Functioning: Beyond validity evidence based on internal structure. Psicothema, 30, 104–109. 10.7334/psicothema2017.183 [DOI] [PubMed] [Google Scholar]
  26. Goodman, R. (1997). The strengths and difficuties questionnaire: A research note. Journal of Child Psychology and Psychiatry, 38, 581–586. [DOI] [PubMed] [Google Scholar]
  27. Gur, R. C. , Richard, J. , Calkins, M. E. , Chiavacci, R. , Hansen, J. A. , Bilker, W. B. , … Gur, R. E. (2012). Age group and sex differences in performance on a computerized neurocognitive battery in children age 8‐21. Neuropsychology, 26(2), 251–265. 10.1037/a0026712 [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Hirschfeld, R. M. , Williams, J. B. , Spitzer, R. L. , Calabrese, J. R. , Flynn, L. , Keck, P. E. J. , … Zajecka, J. (2000). Development and validation of a screening instrument for bipolar spectrum disorder: The Mood Disorder Questionnaire. American Journal of Psychiatry, 157, 1873–1875. [DOI] [PubMed] [Google Scholar]
  29. Hu, L.‐T. , & Bentler, P. M. (1999). Cut off criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. [Google Scholar]
  30. Insel, T. , Cuthbert, B. , Garvey, M. , Heinssen, R. , Pine, D. S. , Quinn, K. , … Wang, P. (2010). Research domain criteria (RDoC): Toward a new classification framework for research on mental disorders. The American Journal of Psychiatry, 167(7), 748–751. [DOI] [PubMed] [Google Scholar]
  31. Insel, T. R. (2017). Digital phenotyping: Technology for a new science of behavior. Jama, 318(13), 1215–1216. [DOI] [PubMed] [Google Scholar]
  32. Kamata, A. , & Bauer, D. J. (2008). A note on the relation between factor analytic and Item Response Theory models. Structural Equation Modeling, 15, 136–153. [Google Scholar]
  33. Kaymaz, N. , Drukker, M. , Lieb, R. , Wittchen, H. U. , Werbeloff, N. , Weiser, M. , … van Os, J. (2012). Do subthreshold psychotic experiences predict clinical outcomes in unselected non‐help‐seeking population‐based samples? A systematic review and meta‐analysis, enriched with new results. Psychological Medicine, 20, 1–15. [DOI] [PubMed] [Google Scholar]
  34. Kelleher, I. , Cederlöf, M. , & Lichtenstein, P. (2014). Psychotic experiences as a predictor of the natural course of suicidal ideation: A Swedish cohort study. World Psychiatry, 13, 184–188. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Kelleher, I. , Connor, D. , Clarke, M. C. , Devlin, N. , Harley, M. , & Cannon, M. (2012a). Prevalence of psychotic symptoms in childhood and adolescence: A systematic review and meta‐analysis of population‐based studies. Psychological Medicine, 9, 1–7. [DOI] [PubMed] [Google Scholar]
  36. Kelleher, I. , Corcoran, P. , Keeley, H. , Wigman, J. T. , Devlin, N. , Ramsay, H. , … Cannon, M. (2013). Psychotic symptoms and population risk for suicide attempt: A prospective cohort study. JAMA Psychiatry, 70, 940–948. [DOI] [PubMed] [Google Scholar]
  37. Kelleher, I. , Murtagh, A. , Molloy, C. , Roddy, S. , Clarke, M. C. , Harley, M. , & Cannon, M. (2012b). Identification and characterization of prodromal risk syndromes in young adolescents in the community: A population‐based clinical interview study. Schizophrenia Bulletin, 38, 239–246. 10.1093/schbul/sbr164 [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Keshavan, M. S. , Morris, D. W. , Sweeney, J. A. , Pearlson, G. , Thaker, G. , Seidman, L. J. , … Tamminga, C. (2011). A dimensional approach to the psychosis spectrum between bipolar disorder and schizophrenia: The Schizo‐Bipolar Scale. Schizophrenia Research, 133(1–3), 250–254. 10.1016/j.schres.2011.09.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Kline, E. , & Schiffman, J. (2014). Psychosis risk screening: A systematic review. Schizophrenia Research, 158, 11–18. [DOI] [PubMed] [Google Scholar]
  40. Kline, E. , Thompson, E. , Demro, C. , Bussell, K. , Reeves, G. , & Schiffman, J. (2015). Longitudinal validation of psychosis risk screening tools. Schizophrenia Ressearch, 165, 116–122. [DOI] [PubMed] [Google Scholar]
  41. Linscott, R. J. , & van Os, J. (2013). An updated and conservative systematic review and meta‐analysis of epidemiological evidence on psychotic experiences in children and adults: On the pathway from proneness to persistence to dimensional expression across mental disorders. Psychological Medicine, 43, 1133–1149. [DOI] [PubMed] [Google Scholar]
  42. Livny, A. , Reichenberg, A. , Fruchter, E. , Yoff, R. , Goldberg, S. , Fenchel, D. , … Weiser, M. (2017). A population‐based longitudinal study of symptoms and signs before the onset of psychosis. Amercian Journal of Psychiatry. [DOI] [PubMed] [Google Scholar]
  43. Loewy, R. L. , Bearden, C. E. , Johnson, J. K. , Raine, A. , & Cannon, T. D. (2005). The Prodromal Questionnaire (PQ): Preliminary validation of a self‐report screening measure for prodromal and psychotic syndromes. Schizophrenia Research, 79, 117–125. [PubMed] [Google Scholar]
  44. Loewy, R. L. , Pearson, R. , Vinogradov, S. , Bearden, C. E. , & Cannon, T. D. (2011). Psychosis risk screening with the Prodromal Questionnaire–Brief version (PQ‐B). Schizophrenia Research, 129, 42–46. 10.1016/j.schres.2011.03.029 [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Loewy, R. L. , Therman, S. , Manninen, M. , Huttunen, M. O. , & Cannon, T. D. (2012). Prodromal psychosis screening in adolescent psychiatry clinics. Early Intervention in Psychiatry, 6, 69–75. [DOI] [PubMed] [Google Scholar]
  46. Magis, D. , Beland, S. , & Raiche, G. (2018). Package ‘difR” (verion 5.0). Retrieved from https://cran.r-project.org/web/packages/difR/difR.pdf
  47. Mason, O. (2015). The assessment of schizotypy and its clinical relevance. Schizophrenia Bulletin, 41, S374–S385. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Miller, T. J. , Cicchetti, D. , Markovich, P. J. , McGlashan, T. H. , & Woods, S. W. (2004). The SIPS screen: A brief self‐report screen to detect the schizophrenia prodrome. Schizophrenia Research, 70(Suppl. 1), 78. [Google Scholar]
  49. Miller, T. J. , McGlashan, T. H. , Rosen, J. L. , Cadenhead, K. , Ventura, J. , McFarlane, W. , … Woods, S. W. (2003). Prodromal assessment with the structured interview for prodromal syndromes and the scale of prodromal symptoms: Predictive validity, interrater reliability, and training to reliability. Schizophrenia Bulletin, 29(4), 703–715. [DOI] [PubMed] [Google Scholar]
  50. Moore, T. M. , Reise, S. P. , Gur, R. E. , Hakonarson, H. , & Gur, R. C. (2015). Psychometric properties of the Penn Computerized Neurocognitive Battery. Neuropsychology, 29(2), 235–246. 10.1037/neu0000093 [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Muthén, L. K. , & Muthén, B. O. 1988(2015). Mplus user's guide (Seventh ed.). Los Angeles, CA: Muthén & Muthén. [Google Scholar]
  52. Okewole, A. O. , Ajogbon, D. , Adeniji, A. A. , Omotoso, O. O. , Awhangansi, S. S. , Agboola, A. A. , & Oyekanmi, A. K. (2015). Psychosis risk screening among secondary school students in Abeokuta, Nigeria: Validity of the Prodromal Questionnaire–Brief Version (PQ‐B). Schizophrenia Research, 164, 281–282. [DOI] [PubMed] [Google Scholar]
  53. Oliver, D. , Davies, C. , Crossland, G. , Lim, S. , Gifford, G. , McGuire, P. , & Fusar‐Poli, P. (2018). Can we reduce the duration of untreated psychosis? A systematic review and meta‐analysis of controlled interventional studies. Schizophrenia Bulletin. 10.1093/schbul/sbx166 [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Ord, L. , Myles‐Worsley, M. , Blailes, F. , & Ngiralmau, H. (2004). Screening for prodromal adolescents in an isolated high‐risk population. Schizophrenia Research, 71, 507–508. [DOI] [PubMed] [Google Scholar]
  55. Ortuño‐Sierra, J. , Aritio‐Solana, R. , & Fonseca‐Pedrero, E. (2018). Mental health difficulties in children and adolescents: The study of the SDQ in the Spanish National Health Survey 2011‐2012. Psychiatry Research, 259, 236–242. [DOI] [PubMed] [Google Scholar]
  56. Ortuño‐Sierra, J. , Chocarro, E. , Fonseca‐Pedrero, E. , Riba, S. S. I. , & Muñiz, J. (2015). The assessment of emotional and Behavioural problems: Internal structure of The Strengths and Difficulties Questionnaire. International Journal of Clinical and Health Psychology, 15(3), 265–273. 10.1016/j.ijchp.2015.05.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Paykel, E. S. , Myers, J. K. , Lindenthal, J. J. , & Tanner, J. (1974). Suicidal feelings in the general population: A prevalence study. The British Journal of Psychiatry, 214, 460–469. [DOI] [PubMed] [Google Scholar]
  58. Polanczyk, G. V. , Fatori, D. , & Matijasevich, A. (2018). Integrating child and adolescent psychiatry and the field of early childhood development. European Child and Adolescent Psychiatry, 27, 137–138. [DOI] [PubMed] [Google Scholar]
  59. Radua, J. , Ramella‐Cravaro, V. , Ioannidis, J. P. A. , Reichenberg, A. , Phiphopthatsanee, N. , Amir, T. , … Fusar‐Poli, P. (2018). What causes psychosis? An umbrella review of risk and protective factors. World Psychiatry, 17, 49–66. 10.1002/wps.20490 [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Raine, A. (1991). The SPQ: A scale for the assessment of schizotypal personality based on DSM‐III‐R criteria. Schizophrenia Bulletin, 17, 555–564. [DOI] [PubMed] [Google Scholar]
  61. Rietdijk, J. , Klaassen, R. , Ising, H. , Dragt, S. , Nieman, D. H. , van de Kamp, J. , … van der Gaag, M. (2012). Detection of people at risk of developing a first psychosis: Comparison of two recruitment strategies. Acta Psychiatrica Scandinavica, 126(1), 21–30. 10.1111/j.1600-0447.2012.01839.x [DOI] [PubMed] [Google Scholar]
  62. Satterthwaite, T. D. , Wolf, D. H. , Calkins, M. E. , Vandekar, S. N. , Erus, G. , Ruparel, K. , … Gur, R. E. (2016). Structural brain abnormalities in youth with psychosis spectrum symptoms. JAMA Psychiatry, 73, 515–524. [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. Savill, M. , D'Ambrosio, J. , Cannon, T. D. , & Loewy, R. L. (2018). Psychosis risk screening in different populations using the Prodromal Questionnaire: A systematic review. Early Intervention in Psychiatry, 12, 3–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Schultze‐Lutter, F. , Addington, J. , Ruhrmann, S. , & Klosterkötter, J. (2007). Schizophrenia proneness instrument, adult version (SPI‐A). Rome: Giovanni Fioriti Editore s.r.l. [Google Scholar]
  65. Strauss, G. P. , Raugh, I. M. , Mittal, V. A. , Gibb, B. E. , & Coles, M. E. (2018). Bullying victimization and perpetration in a community sample of youth with psychotic like experiences. Schizophrenia Research, 195, 534–536. [DOI] [PubMed] [Google Scholar]
  66. Suárez, J. , Pedrosa, I. , Lozano, L. , García‐Cueto, E. , Cuesta, M. , & Muñiz, J. (2018). Using reversed items in Likert scales: A questionable practice. Psicothema, 30, 149–158. [DOI] [PubMed] [Google Scholar]
  67. Tamminga, C. A. , Ivleva, E. I. , Keshavan, M. S. , Pearlson, G. D. , Clementz, B. A. , Witte, B. , … Sweeney, J. A. (2013). Clinical phenotypes of psychosis in the bipolar‐schizophrenia network on intermediate phenotypes (B‐SNIP). American Journal of Psychiatry, 170(11), 1263–1274. 10.1176/appi.ajp.2013.12101339 [DOI] [PubMed] [Google Scholar]
  68. Wigman, J. T. , Vollebergh, W. A. , Raaijmakers, Q. A. , Iedema, J. , van Dorsselaer, S. , Ormel, J. , … van Os, J. (2011). The structure of the extended psychosis phenotype in early adolescence—A cross‐sample replication. Schizophrenia Bulletin, 37, 850–860. [DOI] [PMC free article] [PubMed] [Google Scholar]
  69. Xu, L. H. , Zhang, T. H. , Zheng, L. N. , Li, H. J. , Tang, Y. Y. , Luo, X. G. , … Wang, J. J. (2016). Psychometric properties of prodromal uestionnaire‐brief version among Chinese help‐seeking individuals. PLoS One, 11(2), e0148935 10.1371/journal.pone.0148935 [DOI] [PMC free article] [PubMed] [Google Scholar]
  70. Yu, C. Y. , & Muthén, B. O. (2002). Evaluation of model fit indices for latent variable models with categorical and continuous outcomes (technical report). Los Angeles: UCLA, Graduate School of Education and Information Studies. [Google Scholar]
  71. Yung, A. R. , Yuen, H. P. , McGorry, P. D. , Phillips, L. J. , Kelly, D. , Dell'Olio, M. , … Buckby, J. (2005). Mapping the onset of psychosis‐the Comprehensive Assessment of At Risk Mental States (CAARMS). Australian and New Zealand Journal of Psychiatry, 39, 964–971. [DOI] [PubMed] [Google Scholar]
  72. Zammit, S. , Kounali, D. , Cannon, M. , David, A. S. , Gunnell, D. , Heron, J. , … Lewis, G. (2013). Psychotic experiences and psychotic disorders at age 18 in relation to psychotic experiences at age 12 in a longitudinal population‐based cohort study. American Journal of Psychiatry, 170, 742–750. [DOI] [PubMed] [Google Scholar]

Articles from International Journal of Methods in Psychiatric Research are provided here courtesy of Wiley

RESOURCES