Skip to main content
NCBI home page
International Journal of Methods in Psychiatric Research logoLink to International Journal of Methods in Psychiatric Research
. 2021 Aug 31;30(4):e1893. doi: 10.1002/mpr.1893

The Community Assessment of Psychic Experiences: Optimal cut‐off scores for detecting individuals with a psychotic disorder

Edo S Jaya 1,2,✉, Therese van Amelsvoort 3, Agna A Bartels‐Velthuis 4, Richard Bruggeman 4,5, Wiepke Cahn 6,7, Lieuwe de Haan 8,9, Rene S Kahn 10,11, Jim van Os 6,12, Frederike Schirmbeck 8,9, Claudia J P Simons 3,13, Tania M Lincoln 2
PMCID: PMC8633944  PMID: 34464487

Abstract

Objectives

The need for a brief screening tool for psychosis is widely recognized. The Community Assessment of Psychic Experiences (CAPE) is a popular self‐report measure of psychosis, but a cut‐off score that can detect those most likely to fulfill diagnostic criteria for psychotic disorder is not established.

Methods

A case–control sample from the Genetic Risk and Outcome of Psychosis Project study (N = 1375, healthy individuals, n = 507, and individuals with a psychotic disorder, n = 868), was used to examine cut‐off scores of the CAPE with receiver operating curve analyses. We examined 27 possible cut‐off scores computed from a combination of scores from the frequency and distress scales of the various factors of the CAPE.

Results

The weighted severity positive symptom dimension was most optimal in detecting individuals with a psychotic disorder (>1.75 cut‐off; area under the curve = 0.88; sensitivity, 75%; specificity, 88%), which correctly identified 80% of the sample as cases or controls with a diagnostic odds ratio of 22.69.

Conclusions

The CAPE can be used as a first screening tool to detect individuals who are likely to fulfill criteria for a psychotic disorder. The >1.75 cut‐off of the weighted severity positive symptom dimension provides a better prediction than all alternatives tested so far.

Keywords: early detection, psychotic experiences, psychotic symptoms, ROC, schizophrenia

1. INTRODUCTION

The need for a brief instrument for the screening of psychotic disorder is widely recognized. There are comprehensive clinical diagnostic instruments, such as the Structured Clinical Interview for DSM Disorders, but these take a lot of time to administer. A brief screening instrument that can detect individuals with psychosis with a satisfactory reliability would be useful for researchers interested in the prevalence of psychotic disorder among the general population or in screening out individuals with a psychotic disorder in community samples. Furthermore, it could also be a relevant tool for clinicians to use during initial assessments.

One frequently used self‐report measure that may address this issue is the Community Assessment of Psychic Experiences (CAPE, Stefanis et al., 2002). The CAPE was developed to measure both the frequency and distress levels of psychotic experiences (PEs) within the positive (e.g., “Do you ever hear voices when you are alone?”), negative (e.g., “Do you ever feel that your emotions are blunted?”), and depressive symptom dimension in community settings (Stefanis et al., 2002), and has often been used in clinical samples (e.g. Hanssen et al., 2003; Pignon et al., 2019). The items are answered with a Likert scale ranging from 1 to 4 for both the frequency (“never” to “nearly always”) and the distress (“not distressed” to “very distressed”) scale. The CAPE has been shown to have good psychometric properties in many languages, including Dutch, French, English, German, Indonesian, Spanish, and Swedish (Brenner et al., 2007; Fonseca‐Pedrero et al., 2012; Jaya, 2017; Schlier et al., 2015; Ziermans, 2013). The original factorial structure of the CAPE includes three dimensions (positive symptoms, negative symptoms, and depressive symptoms (Stefanis et al., 2002)). However, the most recent psychometric examination of the CAPE shows that a nine‐cluster factorial structure has better model fit (Jaya et al., 2020; Schlier et al., 2015). This structure consists of a positive symptom dimension with five subdimensions (bizarre experiences, hallucinations, grandiosity, magical thinking, and paranoia), a negative symptom dimension with three subdimensions (affective flattening, avolition, and social withdrawal), and a depressive symptom dimension. The nine‐cluster factorial structure has been shown to have partial scalar measurement invariance between participants from low‐and‐middle‐income countries and high‐income countries (Jaya et al., 2020), and the binary version of the three‐dimensional factorial structure has been shown to have partial scalar measurement invariance too in Brazil, France, the Netherlands, Italy, Spain, and the UK participants (Pignon et al., 2019).

We identified three studies that investigated the usefulness of the CAPE as a screening tool, either for prodromal individuals in a help‐seeking sample (Bukenaite et al., 2017; Mossaheb et al., 2012) or for individuals with first‐episode psychosis in a help‐seeking sample (Boonstra et al., 2009). Bukenaite et al. (2017) examined a 15‐item version of the frequency scale of the positive symptom dimension of the CAPE and found a sensitivity of 77% and specificity of 58% with a cut‐off score of 1.47 (based on a scoring procedure of the CAPE that uses the average score, with a range from 1.00 to 4.00). This was an improvement on the previous study that used the complete 20‐item frequency scale of the positive symptom dimension in the same sample, finding a 67% sensitivity and 73% specificity with a cut‐off score of 3.20 (Mossaheb et al., 2012). In detecting individuals with first‐episode psychosis from a sample of referred patients, Boonstra et al. (2009) suggested a cut‐off score of 50 on the frequency and distress scale of the positive symptom dimension of the CAPE (based on using the sum scoring procedure with a possible range from 20 to 80) which showed a 77% sensitivity and 71% specificity. However, all of these studies used specific samples of help‐seeking prodromal or first‐episode patients. Thus, so far, no study has investigated the usefulness of the CAPE to distinguish individuals with psychotic disorders from healthy controls. However, this type of study is necessary to estimate the usefulness of the CAPE as a general screening tool and be able to compare its detection rates with those of other measures.

Furthermore, previous studies on cut‐off scores were conducted in 2009 and 2012 before the publication of factorial structure studies of the CAPE and have thus were not able to make use of the complete dimensions and subdimensions of the CAPE that are published later. Utilizing the subdimensions of the CAPE is important because a meta‐analysis of studies using the CAPE in 2016 recommended that the three‐dimensional factorial structure should be further divided into subdimensions to improve psychometric properties (Mark & Toulopoulou, 2016). Here is a demonstration on how improved psychometric properties result in better cut‐off score. In 2012, Mossaheb et al. (2012) published a study in which they examined cut‐off scores from four sets of scores by summing the items of the positive symptom dimension, negative symptom dimension, depressive symptom dimension and the CAPE total score. A year later, in 2013, Capra et al. published a study on the factorial structure of the CAPE and recommended the use of only 15 of the 20 items positive symptom dimension (CAPE‐P15, Capra et al., 2013). Four years later, using the newer CAPE‐P15 factorial structure of the CAPE, Bukenaite et al. (2017) re‐examined the sample from Mossaheb et al. (2012) and found improved specificity and sensitivity over the older study (Mossaheb et al., 2012). This demonstrates that incorporating the latest findings on the factorial structure of the CAPE can improve the detection accuracy of individuals with psychotic disorders.

In the present study, we used a holdout cross‐validation procedure to compare the discriminative ability of the CAPE cut‐off scores based on the dimensions from published factorial structures of the CAPE to select those that best differentiate between individuals with a diagnosis of a psychotic disorder and healthy controls.

2. METHOD

2.1. Participants and procedure

We analyzed data (database version 5.0) from the “Genetic Risk and Outcome of Psychosis Project” (GROUP) study (Korver et al., 2012). The GROUP study was designed to investigate risk and protective factors of non‐affective psychotic disorders. The baseline sample of the longitudinal GROUP study consisted of individuals with non‐affective psychotic disorders, their siblings, their parents, and a healthy control group. Participants were recruited from 36 mental healthcare institutions in the Netherlands and Belgium including four academic medical centers (Amsterdam, Groningen, Maastricht, and Utrecht). The diagnoses were made based on the DSM‐IV criteria and were obtained through a consensus of an independent psychiatrist and a structured interview delivered by a trained psychologist or psychiatrist. In case of incongruence, the site coordinators made a final decision on the diagnosis of the participants. The structured interview in three recruitment sites was the Comprehensive Assessment of Symptoms and History (CASH, Andreasen et al., 1992) and one recruitment site used the Schedules for Clinical Assessment for Neuropsychiatry (SCAN 2.1, Wing et al., 1990). Detailed information of the sample characteristics and recruitment methods has been previously published (Korver et al., 2012). Here, we used data from the sample of patients with a psychotic disorder (PD), including schizophrenia, schizoaffective disorder, schizophreniform disorder, delusional disorder, and brief psychotic disorder Not Otherwise Specified (NOS) and from healthy controls (HC) who did not fulfill criteria for any mental health diagnosis at the time of the assessment from the first wave (baseline) of the GROUP study who completed the CAPE. This data selection procedure resulted in 868 PD (78% of the original 1120 clinical sample) and 507 HC (86% of the original 590 controls), thus the combined sample used in this study was 1375 participants (80% of the original 1710 participants).

2.2. Measure: Community Assessment of Psychic Experience

The CAPE is a self‐report measure assessing lifetime PEs consisting of 42 items that include positive symptoms (20 items, e.g., “Have you ever felt that you were being persecuted in any way”, “Do you ever hear voices when you are alone?”), negative symptoms (14 items, e.g., “Do you ever feel that you experience few or no emotions at important events?”, “Do you ever feel that you have no interest to be with other people?”) and depressive symptoms (8 items, e.g., “Do you ever feel sad?”, “Do you ever feel pessimistic about everything?”; Stefanis et al., 2002). Each item's responses are recorded on 4‐point Likert scales from 1 to 4 indicating frequency (“never”, “sometimes”, “often”, “nearly always”) and distress (“not distressed”, “a bit distressed”, “quite distressed”, “very distressed”). The item's distress scale is not answered and recorded as “not distressed” if the item's corresponding frequency scale is answered “never”.

Based on the frequency and distress scales of the CAPE, four possible methods of score computations have been used in the literature. The most common computation is to use a single frequency or distress score, which creates an average score ranging between 1 and 4. A second option is to sum the frequency and distress scales to create an average score ranging from 2 to 8 (e.g., Mossaheb et al., 2012). The third option is to multiply each item's frequency score with its associated distress score. This creates a weighted severity score, with a mean total score ranging from 1 to 16 (e.g., Jaya et al., 2018). For example, an individual who reports to “sometimes” hear voices (frequency score of 2) and to feel “quite distressed” by this experience (distress score of 3) would receive a severity score of 6. The fourth option is to create a binary score of absence versus presence of each item (e.g., Pignon et al., 2021; Wigman et al., 2011; Yung et al., 2009). Absence was coded when participants answered “never” or “not distressed.” Presence was coded when participants answered “sometimes” to “nearly always” or “a bit distressed” to “very distressed.” However, we do not use this fourth option because we aimed to keep as much variance as possible.

We computed a total of 27 different scores of the CAPE measuring positive symptoms, negative symptoms, and depressive symptoms constructs. Six scores were computed from each of the frequency and distress scales of the original three‐dimensional factorial structure, which consists of the average of the frequency and distress scales of the 20 positive symptoms items, 14 negative symptoms items, and 8 depressive symptoms items (Stefanis et al., 2002). Sixteen scores were computed from the frequency and distress scales of the nine‐cluster factor structure (Schlier et al., 2015), which consists of the average of the frequency and distress scales of five subdimensions of positive symptoms (i.e. seven items were used to compute bizarre experiences subdimension, four items were used to compute hallucinations subdimension, two items were used to compute grandiosity subdimension, two items were used to compute magical thinking subdimension, and five items were used to compute paranoia subdimension) and three subdimensions of negative symptoms (i.e. four items were used to compute affective flattening subdimension, seven items were used to compute avolition subdimension, and four items were used to compute social withdrawal subdimension). Three scores were computed from the combination of frequency and distress scales for the positive symptom, negative symptom, and depression dimension as suggested by Mossaheb et al. (2012). In addition, Mossaheb et al. (2012) also suggest the creation of a score based on the sum of frequency and distress of three selected positive symptoms items. One score was computed from multiplying frequency and distress items from the positive symptom dimension to create a weighted severity score of positive symptoms (Jaya et al., 2018).

2.3. Analyses

Receiver‐operating characteristic (ROC) analysis was used to compute cut‐off scores of the dimension scores. The ROC curves along with area under the curve (AUC) statistic and its 95% confidence interval were reported. The AUC statistic is a measure of the diagnostic power of a test. A perfect test will have an AUC of 1.0 and an AUC of 0.5 means the test performs no better than chance. Cut‐off scores were selected based on the Youden index (Youden, 1950). Additionally, we computed the true positive rate (sensitivity), false negative rate (type II error rate), false positive rate (type I error rate), true negative rate (specificity), positive likelihood ratio, negative likelihood ratio, and diagnostic odds ratio for each dimensional cut‐off score. To aid interpretation, we also interpret our finding with a clinical significance guidance by Cicchetti et al. (1995) on diagnostic accuracy for sensitivity, specificity, and accuracy (i.e., below 70% is poor, 70%–79% is fair, 80%–89% is good, 90%–100% is excellent). ROC analyses were computed using the ROCIT package version 1.1.1 (Khan & Brandenburger, 2019).

In addition, to reduce the chance of overfitting, we used holdout cross‐validation (Koul et al., 2018). We randomly partitioned our dataset into a training dataset consisting of two‐thirds of the sample (n = 917, PD, n = 564, HC, n = 353) and a testing dataset consisting of one‐third of the sample (n = 458, PD, n = 304, HC, n = 154). Participants' characteristics and descriptive statistics of the CAPE scores of the training and testing dataset are reported in the supplementary. Furthermore, psychometric properties of the CAPE in the total, training, and testing dataset were examined by conducting confirmatory factor analyses on the original three‐dimensional and nine‐cluster factorial structure using the lavaan package ver. 0.6‐8 (Rosseel, 2012). They were estimated using maximum likelihood with robust standard errors. Model fit was assessed using the following criteria: Comparative Fit Index (CFI) > 0.95, Root Mean Square Error of Approximation (RMSEA) < 0.06, and Standardized Root Mean Square Residual (SRMR) < 0.08 (Hu & Bentler, 1999). The χ2 is reported but not used as a fit criterion because it tends to reject models that are based on a large sample size (Bentler & Bonett, 1980). Omega reliability estimates of each subdimension and dimension were computed with the semTools package ver. 0.5‐4. All analyses were conducted in R version 4.1.0 (R Core Team, 2021).

3. RESULTS

3.1. Participant characteristics

We present demographic characteristics of the total, PD and HC samples in Table 1. The participant characteristics of the testing and training dataset are available in the supplementary (Tables S1 and S2). The diagnostic information of the sample is presented in Table 2. The mean and standard deviations of all computed CAPE dimension scores of each sample group are presented in Table 3. Tables on the CAPE dimension scores of the testing and training dataset are available in the supplementary (Tables S3 and S4). The missing data of the total sample was minimal ranging from 88% (n = 1211) from the weighted severity positive symptoms score to 99% (n = 1374) from several variables such as frequency of positive symptoms, frequency of negative symptoms, and 10 other variables. Outliers were not removed because psychotic symptoms scores vary greatly in the field and this should be reflected in our data.

TABLE 1.

Participants' characteristics

Characteristic Total sample (N = 1375) Individuals with a psychotic disorder (n = 868) Healthy individuals (n = 507)
n/M %/SD n/M %/SD n/M %/SD
Age (M, SD) 28.67 9.03 27.98 7.99 29.87 10.48
Sex (n, %)
Male 900 65.45% 655 75.46% 245 48.32%
Female 475 34.55% 213 24.54% 262 51.68%
Highest educational degree (n, %)
No education (0) 7 0.51% 7 0.81% 0 0.00%
Primary school (1) 114 8.29% 101 11.64% 13 2.56%
Secondary school (2–3) 343 24.95% 264 30.41% 79 15.58%
High school (4–5) 381 27.71% 220 25.35% 161 31.76%
Vocational education (6–7) 429 31.20% 231 26.61% 198 39.05%
University 89 6.47% 34 3.92% 55 10.85%
Marital status (n, %)
Not married 1028 74.76% 741 85.37% 287 56.61%
Married/living together 265 19.27% 80 9.22% 185 36.49%
Divorced/Widowhood 40 2.91% 26 3.00% 14 2.76%
Open in a new tab

Note: Secondary school consists of the following Dutch system of secondary school, LBO/HH/LHNO/VBO and MAVO/VMBO; high school consists of HAVO and VWO; vocational education consists of MBO and HBO. The PD sample was younger than the HC sample (t[850.83] = 3.51, p < 0.001), had fewer females (χ2[1, N = 1375] = 103.04, p < 0.001), had lower education level (χ2[8, N = 1375] = 151.93, p < 0.001), and fewer were married (χ2[3, N = 1375] = 161.52, p < 0.001).

TABLE 2.

Diagnostic information of the total sample (N = 1375)

Characteristic Diagnostic code DSM‐IV‐TR Individuals with a psychotic disorder (n = 868) Healthy individuals (n = 507)
n % n %
Schizophrenia, disorganized type 295.1 35 4.03% 0 0%
Schizophrenia, catatonia type 295.2 1 0.12% 0 0%
Schizophrenia, paranoid type 295.3 487 56.11% 0 0%
Schizophreniform disorder 295.4 43 4.95% 0 0%
Schizophrenia, residual type 295.6 19 2.19% 0 0%
Schizoaffective disorder 295.7 102 11.75% 0 0%
Schizophrenia, undifferentiated type 295.9 52 5.99% 0 0%
Delusional disorder 297.1 17 1.96% 0 0%
Brief psychotic disorder 298.8 23 2.65% 0 0%
Psychotic disorder NOS 298.9 89 10.25% 0 0%
Diagnosis deferred 799.9 0 0.00% 3 0.59%
Bereavement V62.82 0 0.00% 1 0.20%
No diagnosis V71.09 0 0.00% 503 99.21%
Open in a new tab

Abbreviation: NOS, not otherwise specified.

TABLE 3.

Means and standard deviations of the CAPE dimension scores

Measure Total sample (N = 1375) Individuals with a psychotic disorder (n = 868) Healthy individuals (n = 507)
n M SD n M SD n M SD Possible score range
Three dimensions
F.POS 1374 1.50 0.47 867 1.68 0.50 507 1.19 0.17 1–4
F.NEG 1374 1.82 0.54 867 2.02 0.53 507 1.47 0.32 1–4
F.DEP 1374 1.84 0.54 867 2.00 0.58 507 1.56 0.33 1–4
D.POS 1374 1.43 0.52 867 1.64 0.55 507 1.07 0.12 1–4
D.NEG 1373 1.69 0.61 866 1.91 0.62 507 1.32 0.34 1–4
D.DEP 1371 1.85 0.70 864 2.06 0.72 507 1.49 0.48 1–4
Nine‐cluster
F.BIZ 1374 1.40 0.52 867 1.58 0.56 507 1.07 0.16 1–4
F.HAL 1369 1.30 0.49 864 1.47 0.55 505 1.02 0.09 1–4
F.PAR 1374 1.66 0.57 867 1.84 0.61 507 1.35 0.30 1–4
F.MAG 1373 1.74 0.81 866 1.92 0.86 507 1.43 0.59 1–4
F.GRA 1371 1.61 0.75 864 1.79 0.82 507 1.30 0.48 1–4
F.SOW 1373 1.91 0.62 866 2.12 0.62 507 1.57 0.43 1–4
F.AFF 1373 1.73 0.72 866 1.98 0.74 507 1.30 0.43 1–4
F.AVO 1373 1.80 0.55 866 1.99 0.57 507 1.49 0.33 1–4
D.BIZ 1374 1.40 0.59 867 1.61 0.65 507 1.03 0.11 1–4
D.HAL 1367 1.32 0.56 862 1.51 0.64 505 1.01 0.06 1–4
D.PAR 1373 1.62 0.69 866 1.87 0.74 507 1.20 0.28 1–4
D.MAG 1368 1.34 0.66 862 1.52 0.77 506 1.04 0.18 1–4
D.GRA 1364 1.34 0.71 857 1.52 0.84 507 1.06 0.22 1–4
D.SOW 1369 1.66 0.67 862 1.88 0.69 507 1.28 0.41 1–4
D.AFF 1370 1.63 0.76 863 1.87 0.81 507 1.22 0.41 1–4
D.AVO 1370 1.74 0.65 863 1.95 0.67 507 1.38 0.40 1–4
Mossaheb et al. (2012)
DF.POS 1374 1.46 0.48 867 1.66 0.50 507 1.13 0.14 1–8
DF.NEG 1374 1.76 0.55 867 1.97 0.55 507 1.39 0.31 1–8
DF.DEP 1374 1.84 0.60 867 2.03 0.62 507 1.52 0.39 1–8
3 items 1374 1.50 0.67 867 1.77 0.71 507 1.05 0.15 1–4
Jaya et al. (2018)
SEV.POS 1211 2.57 1.96 742 3.35 2.14 469 1.34 0.40 1–16
Open in a new tab

Note: Presented scores are average scores (i.e., the mean of item scores).

Abbreviations: AFF, affective flattening (3 items); AVO, avolition (7 items); BIZ, bizarre experiences dimension (7 items); CAPE, community assessment of psychic experiences; D, distress scale; DEP, depression dimension (8 items); DF, sum of frequency and distress scale following Mossaheb et al. (2012); F, frequency scale; GRA, grandiosity dimension (2 items); HAL, hallucinations dimension (4 items); MAG, magical thinking dimension (2 items); NEG, negative symptoms dimension (14 items); PAR, paranoia (5 items); POS, positive symptoms dimension (20 items); SSEV.POS, severity positive symptom dimension scale is computed by multiplying (weighting) each frequency answer with its corresponding distress score following Jaya et al. (2018)OW, social withdrawal dimension (4 items); Three items, sum of frequency and distress scale of Q7, Q31, and Q33 recommended in Mossaheb et al. (2012).

The average age of the total sample was 28.7 years (SD = 9.0), ranging from 15 to 61 years and 65% were male. The PD sample was younger than the HC sample (t (850.83) = 3.51, p < 0.001), had fewer females (χ2 [1, N = 1375] = 103.04, p < 0.001), had lower education level (χ2 [8, N = 1375] = 151.93, p < 0.001), and fewer were married (χ2 [3, N = 1375] = 161.52, p < 0.001).

3.2. Reliability and factorial structure of the CAPE

In the total dataset, the three‐dimensional factorial structure met the fit criteria for RMSEA and SRMR, but not CFI (χ[816] = 3263.278, p < 0.001, CFI = 0.857, RMSEA = 0.057 [90% CI 0.055, 0.059], SRMR = 0.052, AIC = 108,576). The Omega reliability estimates of the three‐dimensional factorial structure were good (positive symptom dimension, ω = 0.92; negative symptom dimension, ω = 0.91; depressive symptom dimension, ω = 0.87). Similarly, the nine‐cluster factorial structure met the fit criteria for RMSEA and SRMR, but not CFI (χ[808] = 2171.591, p < 0.001, CFI = 0.921, RMSEA = 0.043 [90% CI 0.041, 0.045], SRMR = 0.053, AIC = 106,932). The Omega reliability estimates of the nine‐cluster factorial structure were acceptable (positive symptom dimension, ω = 0.88; bizarre experiences subdimension, ω = 0.85, hallucinations subdimension, ω = 0.81, grandiosity subdimension, ω = 0.81, magical thinking subdimension, ω = 0.65, paranoia subdimension, ω = 0.79; negative symptom dimension, ω = 0.92; affective flattening subdimension, ω = 0.82, avolition subdimension, ω = 0.83, social withdrawal subdimension, ω = 0.78; depressive symptom dimension, ω = 0.88).

A similar pattern of results was found in the training and testing dataset. In the training dataset the three‐dimensional (χ[816] = 2278.401, p < 0.001, CFI = 0.860, RMSEA = 0.058 [90% CI 0.056, 0.061], SRMR = 0.058, AIC = 62,455) and nine‐cluster factorial structure (χ[808] = 1617.551, p < 0.001, CFI = 0.923, RMSEA = 0.043 [90% CI 0.040, 0.047], SRMR = 0.058, AIC = 61,424) met the fit criteria for RMSEA and SRMR, but not CFI. The Omega reliability estimates of the three‐dimensional factorial structure were good (positive symptom dimension, ω = 0.92; negative symptom dimension, ω = 0.91; depressive symptom dimension, ω = 0.88), and likewise for the nine‐cluster factorial structure (positive symptom dimension, ω = 0.89; bizarre experiences subdimension, ω = 0.86, hallucinations subdimension, ω = 0.81, grandiosity subdimension, ω = 0.82, magical thinking subdimension, ω = 0.69, paranoia subdimension, ω = 0.78; negative symptom dimension, ω = 0.93; affective flattening subdimension, ω = 0.85, avolition subdimension, ω = 0.84, social withdrawal subdimension, ω = 0.79; depressive symptom dimension, ω = 0.89).

Again, in the testing dataset, both factorial structures met the fit criteria for RMSEA and SRMR, but not CFI (three‐dimensional, χ[816] = 1583.627, p < 0.001, CFI = 0.840, RMSEA = 0.059 [90% CI 0.054, 0.063], SRMR = 0.059, AIC = 32,638; nine‐cluster, χ[808] = 1205.801, p < 0.001, CFI = 0.918, RMSEA = 0.042 [90% CI 0.037, 0.047], SRMR = 0.062, AIC = 32,076). The Omega reliability estimates in the testing dataset were similar to those in the training dataset (three‐dimensional: positive symptom dimension, ω = 0.91; negative symptom dimension, ω = 0.90; depressive symptom dimension, ω = 0.86; nine‐cluster: positive symptom dimension, ω = 0.87; bizarre experiences subdimension, ω = 0.83, hallucinations subdimension, ω = 0.82, grandiosity subdimension, ω = 0.80, magical thinking subdimension, ω = 0.54, paranoia subdimension, ω = 0.80; negative symptom dimension, ω = 0.92; affective flattening subdimension, ω = 0.82, avolition subdimension, ω = 0.82, social withdrawal subdimension, ω = 0.78; depressive symptom dimension, ω = 0.87).

3.3. The cut‐off scores of the CAPE

The ROC analyses on the cut‐off scores of the 27 dimensional scores from the total dataset (N = 1375) are presented in Table 4, and a similar table from the training (n = 917) and testing (n = 458) dataset are presented in Tables S5 and S6, respectively).

TABLE 4.

Cut‐off points of various dimensional scores of the CAPE (N = 1375)

Measure Cut‐off points AUC 95% CI: Lower 95% CI: Upper True positive rate (sensitivity) False negative rate (type II error rate) True negative rate (1‐specificity) False positive rate (type I error rate) Positive likelihood ratio Negative likelihood ratio Diagnostic odds ratio Accuracy
Three dimensions
F.POS 1.44 0.836 0.815 0.856 64.24% 35.76% 91.32% 8.68% 7.40 0.39 18.91 74.24%
F.NEG 1.77 0.815 0.793 0.836 68.51% 31.49% 82.45% 17.55% 3.90 0.38 10.22 73.65%
F.DEP 1.80 0.747 0.722 0.773 57.55% 42.45% 82.64% 17.36% 3.32 0.51 6.46 66.81%
D.POS 1.25 0.866 0.847 0.884 70.47% 29.53% 92.11% 7.89% 8.93 0.32 27.86 78.46%
D.NEG 1.57 0.800 0.777 0.823 69.28% 30.72% 79.68% 20.32% 3.41 0.39 8.85 73.12%
D.DEP 1.71 0.745 0.719 0.770 64.70% 35.30% 75.15% 24.85% 2.60 0.47 5.54 68.56%
Nine cluster
F.BIZ 1.29 0.833 0.813 0.854 67.13% 32.87% 88.17% 11.83% 5.67 0.37 15.21 74.89%
F.HAL 1.25 0.800 0.777 0.823 62.31% 37.69% 94.67% 6.14% 10.41 0.38 25.49 74.95%
F.PAR 1.80 0.750 0.725 0.775 52.60% 47.40% 86.98% 13.02% 4.04 0.54 7.41 65.28%
F.MAG 2.00 0.669 0.641 0.698 51.96% 48.04% 74.75% 25.25% 2.06 0.64 3.20 60.38%
F.GRA 2.00 0.684 0.656 0.712 46.18% 53.82% 83.23% 16.77% 2.75 0.65 4.26 59.88%
F.SOW 2.00 0.765 0.741 0.790 63.97% 36.03% 77.71% 22.29% 2.87 0.46 6.19 69.05%
F.AFF 1.50 0.782 0.758 0.806 69.98% 30.02% 73.57% 26.43% 2.65 0.41 6.49 71.30%
F.AVO 1.83 0.772 0.748 0.797 60.39% 39.61% 83.83% 16.17% 3.73 0.47 7.90 69.05%
D.BIZ 1.14 0.833 0.812 0.854 73.13% 26.87% 87.97% 12.03% 6.08 0.31 19.89 78.60%
D.HAL 1.25 0.764 0.740 0.789 54.29% 45.71% 97.82% 2.18% 24.93 0.47 53.34 70.37%
D.PAR 1.50 0.784 0.761 0.808 61.43% 38.57% 86.19% 13.81% 4.45 0.45 9.94 70.58%
D.MAG 1.50 0.689 0.661 0.717 42.34% 57.66% 94.27% 5.73% 7.39 0.61 12.08 61.55%
D.GRA 1.50 0.658 0.629 0.687 37.69% 62.31% 92.70% 7.30% 5.16 0.67 7.68 58.14%
D.SOW 1.33 0.773 0.749 0.797 71.69% 28.31% 71.60% 28.40% 2.52 0.40 6.38 71.66%
D.AFF 1.67 0.754 0.729 0.779 59.44% 40.56% 82.45% 17.55% 3.39 0.49 6.88 67.96%
D.AVO 1.67 0.764 0.740 0.789 64.19% 35.81% 78.30% 21.70% 2.96 0.46 6.47 69.42%
Mossaheb et al. (2012)
DF.POS 1.78 0.865 0.847 0.884 36.68% 63.32% 99.80% 0.20% 185.96 0.63 293.09 59.97%
DF.NEG 2.13 0.817 0.795 0.839 36.79% 63.21% 96.65% 3.35% 10.97 0.65 16.78 58.88%
DF.DEP 1.86 0.753 0.728 0.779 57.44% 42.56% 83.83% 16.17% 3.55 0.51 6.99 67.18%
3 items 1.20 0.838 0.818 0.859 71.63% 28.37% 89.74% 10.26% 6.98 0.32 22.09 78.31%
Jaya et al. (2018)
SEV.POS 1.75 0.875 0.856 0.894 75.47% 24.53% 88.06% 11.94% 6.32 0.28 22.69 80.35%
Open in a new tab

Note: Cut‐off points are based on average scores.

Abbreviations: AUC, area under the curve; AVO, avolition (7 items); BIZ, bizarre experiences dimension (7 items); CAPE, community assessment of psychic experiences; D, distress scale; DEP, depression dimension (8 items); DF, sum of frequency and distress scale following Mossaheb et al. (2012); F, frequency scale; GRA, grandiosity dimension (2 items); HAL, hallucinations dimension (4 items); MAG, magical thinking dimension (2 items); NEG, negative symptoms dimension (14 items); PAR, paranoia (5 items); POS, positive symptoms dimension (20 items); SSEV.POS, severity positive symptom dimension scale is computed by multiplying (weighting) each frequency answer with its corresponding distress score following Jaya et al. (2018) SOW, social withdrawal dimension (4 items); Three items, sum of frequency and distress scale of Q7, Q31, and Q33 recommended in Mossaheb et al. (2012).

The ROC analyses from the training dataset revealed the top five highest Youden index cut‐off scores to be distress weighted severity positive symptom dimension (>1.75, J = 0.66), distress positive symptom dimension (>1.25, J = 0.64), sum of frequency and distress of the positive symptom dimension (>1.30, J = 0.63), distress of bizarre experience dimension (>1.14, J = 0.62), and sum of frequency and distress of three selected items from the positive symptom dimension (>1.20, J = 0.60). Of these cut‐off scores, the weighted severity positive symptom dimension had the highest accuracy (82%). Therefore, the weighted severity positive symptom dimension was selected for further examination. The ROC curve of the weighted severity positive dimension score from the total sample is presented in Figure 1.

FIGURE 1.

FIGURE 1

Open in a new tab

Receiver‐operating characteristic curve, sensitivity versus 1‐specificity for severity positive symptom dimension. Severity positive symptom dimension scale is computed by multiplying (weighting) each frequency answer with its corresponding distress score following Jaya et al. (2018)

With the cut‐off point of >1.75 of the weighted severity positive symptom dimension in the training dataset, we found that 80% of the training sample was correctly identified as belonging to the PD or the HC. Specifically, 318 participants were correctly identified as HC, which was 90% of the total healthy sample (n = 345), and 370 participants were correctly identified as PD, which was 66% of the total PD sample (n = 566). We found 35 healthy individuals to be incorrectly identified as PD (10% of the total HC sample) and 194 individuals with a PD to be incorrectly identified as HC (34% of the total PD sample). The diagnostic odds ratio for the cut‐off point of >1.75 of the weighted severity positive symptom dimension was 26.84.

With the cut‐off point of >1.75 of the weighted severity positive symptom dimension in the testing dataset, we found that 78% of the total sample was correctly identified as belonging to the PD or the HC. Specifically, 133 participants were correctly identified as HC, which is 86% of the total healthy sample (n = 154), and 190 participants were correctly identified as PD, which was 63% of the total PD sample (n = 304). We found 21 healthy individuals to be incorrectly identified as PD (14% of the total HC sample) and 114 individuals with a PD to be incorrectly identified as HC (37% of the total PD sample). The diagnostic odds ratio for the cut‐off point of >1.75 of the weighted severity positive symptom dimension was 16.3.

4. DISCUSSION

The aim of the study was to discover which cut‐off score from the various dimensions of the CAPE can best differentiate between healthy individuals and individuals with a psychotic disorder. We found from the literature that there are 27 ways to produce scores from the various dimensions and subdimensions of the CAPE. Among these, the weighted severity positive symptom dimension with a cut‐off score of >1.75 out of 16.00 performed best in differentiating between healthy individuals and individuals with a psychotic disorder in our study. The weighted severity positive symptom dimension is computed by summing the multiplication of each positive symptom item's frequency score with its corresponding distress score. The cut‐off score had 75% sensitivity, 88% specificity, and 80% accuracy, which can be interpreted as fair sensitivity, good specificity, and good accuracy in clinical usage (Cicchetti et al., 1995).

Our proposed cut‐off score thus has better accuracy in comparison to previously suggested cut‐off scores that use the frequency or distress scale of the positive symptom dimension with a 77% sensitivity and 71% specificity (Boonstra et al., 2009) or the frequency scale of the CAPE‐P15 with a 77% sensitivity and 58% specificity (Bukenaite et al., 2017). However, it is important to note that our cut‐off score is appropriate to be used in non‐help‐seeker populations such as community samples to differentiate between healthy individuals and individuals with a psychotic disorder, while the previously published cut‐off score is more appropriate to be used in clinical settings to differentiate between help‐seekers without and with psychosis.

Even though our suggested cut‐off score from the weighted severity positive symptoms dimension performs better than previously published cut‐off scores and can be considered good in terms of clinical significance, the sensitivity and specificity statistics do not perform as well as commonly used screening tools for physical disorders, such as cancer, or for depression. Mammography, a well‐known screening tool for breast cancer can detect breast cancer with sensitivity of 87% and specificity of 97% in the Million Women Study (Banks et al., 2004). Furthermore, the Patient Health Questionnaire‐9 can detect major depressive disorder with a sensitivity of 88% and a specificity of 88% (Kroenke et al., 2001). More research into screening tools for psychosis is needed to improve the detection rate in order for it to be on par with these high precision tools.

Interestingly, we found a trend that the distress scale of positive symptoms dimension of the CAPE plays an important role in discriminating PD and HC sample whereas the frequency scale is less helpful. The emphasize on distress in evaluating the clinical relevance of psychotic symptoms is also found in our analysis of psychotic experiences in participants from 13 countries (Wüsten et al., 2018) and other studies looking at the relationship between religiosity and psychotic symptoms (e.g., Brito et al., 2021; Oh et al., 2018). However, there are exceptions in our results. The hallucination and grandiosity dimensions had higher AUC in the frequency than the distress scale. This may have methodological reasons, for example, the items in the hallucination dimensions appear to reflect stronger severity than in the other dimensions (e.g., hallucinations, “Do you ever hear voices when you are alone?”, compared to paranoia, “Do you ever feel as if you are being persecuted in some way?”). Although individuals with psychotic disorders have high level of depressive symptoms (Etchecopar‐Etchart et al., 2021), we found positive symptoms rather than depressive symptoms were the best indicator in discriminating individuals with a psychotic disorder from healthy individuals.

4.1. Strengths and limitations

The cut‐off score recommended in this study was based on the paper‐and‐pencil version of the CAPE using the Dutch version of the CAPE. Hence, how well the cut‐off is suited for different versions of the CAPE (e.g., the online version) or across different countries needs to be investigated in combination with issues related to measurement invariance of the CAPE. Furthermore, the CAPE is a self‐report scale and limitations to self‐report assessment such as social desirability and other response styles apply. Demographic characteristic of the sample such as the majority being male and single may have influenced the computation of the cut‐off. Additionally, because the case‐control sample was not drawn from a representative population sample, in which a base psychosis prevalence rate of 0.17% can be expected (Evensen et al., 2015), we were not able to compute the additional statistics (i.e., positive predictive value and negative predictive value) that would be helpful to further judge the accuracy of the CAPE in distinguishing between healthy individuals and individuals with a psychotic disorder. Future studies also need to examine the accuracy of the suggested cut‐off in a representative population sample consisting of healthy individuals, individuals with non‐psychotic mental disorders, and individuals with a psychotic disorder.

4.2. Conclusion

The distress weighted positive symptom dimension from the CAPE can be used as a preliminary screening tool to detect or screen out individuals with a high likelihood of having a psychotic disorder. It performs better than previously published cut‐off scores but does not reach the high precision of screening tools for depression. Thus, it cannot replace validated diagnostic assessment tools if the aim is to make a diagnosis.

CONFLICT OF INTERESTS

Edo S. Jaya, Therese van Amelsvoort, Agna A. Bartels‐Velthuis, Richard Bruggeman, Wiepke Cahn, Lieuwe de Haan, Rene S. Kahn, Jim van Os, Frederike Schirmbeck, Claudia J.P. Simons, and Tania M. Lincoln declared no conflict of interest.

Supporting information

Supporting Information S1

Click here for additional data file. (56.3KB, docx)

ACKNOWLEDGMENTS

We thank Md Riaz Ahmed Khan who assists us with the ROC analysis. We are grateful for the generosity of time and effort by the patients, their families, and healthy subjects. Furthermore, we would like to thank all research personnel involved in the GROUP project, in particular, Joyce van Baaren, Erwin Veermans, Ger Driessen, Truda Driesen, and Erna van’t Hag.

Edo S. Jaya was supported by Universitas Indonesia's internal research grants (“Hibah Publikasi Artikel di Jurnal Internasional Kuartil Q1 dan Q2 Tahun Anggaran 2019”, Grant No: NKB‐0281/UN2.R3.1/HKP.05.00/2019 and PINTER‐MIDI).

The infrastructure for the GROUP study is funded through the Geestkracht programme of the Dutch Health Research Council (Zon‐Mw, grant number 10‐000‐1001), and matching funds from participating pharmaceutical companies (Lundbeck, AstraZeneca, Eli Lilly, Janssen Cilag) and universities and mental health care organizations (Amsterdam: Academic Psychiatric Centre of the Academic Medical Center and the mental health institutions: GGZ Ingeest, Arkin, Dijk en Duin, GGZ Rivierduinen, Erasmus Medical Centre, GGZ Noord Holland Noord. Groningen: University Medical Center Groningen and the mental health institutions: Lentis, GGZ Friesland, GGZ Drenthe, Dimence, Mediant, GGNet Warnsveld, Yulius Dordrecht and Parnassia psycho‐medical center The Hague. Maastricht: Maastricht University Medical Centre and the mental health institutions: GGzE, GGZ Breburg, GGZ Oost‐Brabant, Vincent van Gogh voor Geestelijke Gezondheid, Mondriaan, Virenze riagg, Zuyderland GGZ, MET ggz, Universitair Centrum Sint‐Jozef Kortenberg, CAPRI University of Antwerp, PC Ziekeren Sint‐Truiden, PZ Sancta Maria Sint‐Truiden, GGZ Overpelt, OPZ Rekem. Utrecht: University Medical Center Utrecht and the mental health institutions Altrecht, GGZ Centraal and Delta).

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Open Access funding enabled and organized by Projekt DEAL. Blended DEAL: Projekt DEAL.

Open access funding enabled and organized by Projekt DEAL.

Jaya, E. S. , van Amelsvoort, T. , Bartels‐Velthuis, A. A. , Bruggeman, R. , Cahn, W. , de Haan, L. , Kahn, R. S. , van Os, J. , Schirmbeck, F. , Simons, C. J. P. , & Lincoln, T. M . (2021). The Community Assessment of Psychic Experiences: Optimal cut‐off scores for detecting individuals with a psychotic disorder. International Journal of Methods in Psychiatric Research, 30(4), e1893. 10.1002/mpr.1893

Genetic Risk and Outcome of Psychosis [GROUP] Investigators comprise Therese van Amelsvoort, Agna A. Bartels‐Velthuis, Richard Bruggeman, Wiepke Cahn, Lieuwe de Haan, Rene S. Kahn, Jim van Os, Frederike Schirmbeck, Claudia J.P. Simons.

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available from the Genetic Risk and Outcome of Psychosis (GROUP). Restrictions apply to the availability of these data, which were used under licence for this study. Data are available from the authors with the permission of the Genetic Risk and Outcome of Psychosis (GROUP).

REFERENCES

  1. Andreasen, N. C. , Flaum, M. , & Arndt, S. (1992). The Comprehensive Assessment of Symptoms and History (CASH): An instrument for assessing diagnosis and psychopathology. Archives of General Psychiatry, 49(8), 615–623. 10.1001/archpsyc.1992.01820080023004 [DOI] [PubMed] [Google Scholar]
  2. Banks, E. , Reeves, G. , Beral, V. , Bull, D. , Crossley, B. , Simmonds, M. , Hilton, E. , Bailey, S. , Barrett, N. , Briers, P. , English, R. , Jackson, A. , Kutt, E. , Lavelle, J. , Rockall, L. , Wallis, M. G. , Wilson, M. , & Patnick, J. (2004). Influence of personal characteristics of individual women on sensitivity and specificity of mammography in the Million Women Study: Cohort study. BMJ British Medical Journal, 329(7464), 477. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC515195/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bentler, P. M. , & Bonett, D. G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88(3), 588–606. 10.1037/0033-2909.88.3.588 [DOI] [Google Scholar]
  4. Boonstra, N. , Wunderink, L. , Sytema, S. , & Wiersma, D. (2009). Improving detection of first‐episode psychosis by mental health‐care services using a self‐report questionnaire. Early Intervention in Psychiatry, 3(4), 289–295. 10.1111/j.1751-7893.2009.00147.x [DOI] [PubMed] [Google Scholar]
  5. Brenner, K. , Schmitz, N. , Pawliuk, N. , Fathalli, F. , Joober, R. , Ciampi, A. , & King, S. (2007). Validation of the English and French versions of the Community Assessment of Psychic Experiences (CAPE) with a Montreal community sample. Schizophrenia Research, 95(1–3), 86–95. 10.1016/j.schres.2007.06.017 [DOI] [PubMed] [Google Scholar]
  6. Brito, M. A. , Amad, A. , Rolland, B. , Geoffroy, P. A. , Peyre, H. , Roelandt, J.‐L. , Benradia, I. , Thomas, P. , Vaiva, G. , Schürhoff, F. , & Pignon, B. (2021). Religiosity and prevalence of suicide, psychiatric disorders and psychotic symptoms in the French general population. European Archives of Psychiatry and Clinical Neuroscience. 10.1007/s00406-021-01233-3 [DOI] [PubMed] [Google Scholar]
  7. Bukenaite, A. , Stochl, J. , Mossaheb, N. , Schäfer, M. R. , Klier, C. M. , Becker, J. , Schloegelhofer, M. , Papageorgiou, K. , Montejo, A. L. , Russo, D. A. , Jones, P. B. , Perez, J. , & Amminger, G. P. (2017). Usefulness of the CAPE‐P15 for detecting people at ultra‐high risk for psychosis: Psychometric properties and cut‐off values. Schizophrenia Research, 189, 69–74. 10.1016/j.schres.2017.02.017 [DOI] [PubMed] [Google Scholar]
  8. Capra, C. , Kavanagh, D. J. , Hides, L. , & Scott, J. (2013). Brief screening for psychosis‐like experiences. Schizophrenia Research, 149(1–3), 104–107. 10.1016/j.schres.2013.05.020 [DOI] [PubMed] [Google Scholar]
  9. Cicchetti, D. V. , Volkmar, F. , Klin, A. , & Showalter, D. (1995). Diagnosing autism using ICD‐10 criteria: A comparison of neural networks and standard multivariate procedures. Child Neuropsychology, 1(1), 26–37. 10.1080/09297049508401340 [DOI] [Google Scholar]
  10. Etchecopar‐Etchart, D. , Korchia, T. , Loundou, A. , Llorca, P.‐M. , Auquier, P. , Lançon, C. , Boyer, L. , & Fond, G. (2021). Comorbid major depressive disorder in schizophrenia: A systematic review and meta‐analysis. Schizophrenia Bulletin, 47(2), 298–308. 10.1093/schbul/sbaa153 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Evensen, S. , Wisløff, T. , Lystad, J. U. , Bull, H. , Ueland, T. , & Falkum, E. (2015). Prevalence, employment rate, and cost of schizophrenia in a high‐income welfare society: A population‐based study using comprehensive health and welfare registers. Schizophrenia Bulletin, sbv141. 10.1093/schbul/sbv141 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Fonseca‐Pedrero, E. , Paino, M. , Lemos‐Giráldez, S. , & Muñiz, J. (2012). Validation of the Community Assessment Psychic Experiences‐42 (CAPE‐42) in Spanish college students and patients with psychosis. Actas Españolas de Psiquiatría, 40(4), 169–176. [PubMed] [Google Scholar]
  13. Hanssen, M. , Peeters, F. , Krabbendam, L. , Radstake, S. , Verdoux, H. , & van Os, J. (2003). How psychotic are individuals with non‐psychotic disorders? Social Psychiatry and Psychiatric Epidemiology, 38(3), 149–154. 10.1007/s00127-003-0622-7 [DOI] [PubMed] [Google Scholar]
  14. Hu, L. , & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. 10.1080/10705519909540118 [DOI] [Google Scholar]
  15. Jaya, E. S. (2017). Confirmatory factor analysis of the Indonesian version of community assessment of psychic experiences. Makara Human Behavior Studies in Asia, 9(1), 1–12. 10.7454/mssh.v21i1.3495 [DOI] [Google Scholar]
  16. Jaya, E. S. , Ascone, L. , & Lincoln, T. M. (2018). A longitudinal mediation analysis of the effect of negative‐self‐schemas on positive symptoms via negative affect. Psychological Medicine, 48(8), 1299–1307. 10.1017/S003329171700277X [DOI] [PubMed] [Google Scholar]
  17. Jaya, E. S. , Wüsten, C. , Alizadeh, B. Z. , van Amelsvoort, T. , Bartels‐Velthuis, A. A. , van Beveren, N. J. , Bruggeman, R. , Cahn, W. , de Haan, L. , Delespaul, P. , Luykx, J. J. , Myin‐Germeys, I. , Kahn, R. S. , Schirmbeck, F. , Simons, C. J. P. , van Haren, N. E. , van Os, J. , van Winkel, R. , Fonseca‐Pedrero, E. , & Lincoln, T. M. (2020). Comparing psychotic experiences in low‐and‐middle‐income‐countries and high‐income‐countries with a focus on measurement invariance. Psychological Medicine, 1–8. 10.1017/S0033291720003323 [DOI] [PubMed] [Google Scholar]
  18. Khan, M. R. A. , & Brandenburger, T. (2019). ROCit: Performance Assessment of binary Classifier with visualization (R package version 1.1.1) [Computer software]. https://CRAN.R‐project.org/package=ROCit [Google Scholar]
  19. Korver, N. , Quee, P. J. , Boos, H. B. M. , Simons, C. J. P. , de Haan, L. , & GROUP Investigators . (2012). Genetic Risk and Outcome of Psychosis (GROUP), a multi‐site longitudinal cohort study focused on gene–environment interaction: Objectives, sample characteristics, recruitment and assessment methods. International Journal of Methods in Psychiatric Research, 21(3), 205–221. 10.1002/mpr.1352 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Koul, A. , Becchio, C. , & Cavallo, A. (2018). Cross‐validation approaches for replicability in psychology. Frontiers in Psychology, 9. 10.3389/fpsyg.2018.01117 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Kroenke, K. , Spitzer, R. L. , & Williams, J. B. W. (2001). The PHQ‐9. Journal of General Internal Medicine, 16(9), 606–613. 10.1046/j.1525-1497.2001.016009606.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Mark, W. , & Toulopoulou, T. (2016). Psychometric properties of “Community Assessment of Psychic Experiences”: Review and meta‐analyses. Schizophrenia Bulletin, 42(1), 34–44. 10.1093/schbul/sbv088 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Mossaheb, N. , Becker, J. , Schaefer, M. R. , Klier, C. M. , Schloegelhofer, M. , Papageorgiou, K. , & Amminger, G. P. (2012). The Community Assessment of Psychic Experience (CAPE) questionnaire as a screening‐instrument in the detection of individuals at ultra‐high risk for psychosis. Schizophrenia Research, 141(2–3), 210–214. 10.1016/j.schres.2012.08.008 [DOI] [PubMed] [Google Scholar]
  24. Oh, H. , Waldman, K. , & Koyanagi, A. (2018). Psychotic experiences and religiosity: Findings from the collaborative psychiatric epidemiological surveys. Schizophrenia Research, 201, 435–436. 10.1016/j.schres.2018.05.037 [DOI] [PubMed] [Google Scholar]
  25. Pignon, B. , Lajnef, M. , Kirkbride, J. B. , Peyre, H. , Ferchiou, A. , Richard, J.‐R. , Baudin, G. , Tosato, S. , Jongsma, H. , de Haan, L. , Tarricone, I. , Bernardo, M. , Velthorst, E. , Braca, M. , Arango, C. , Arrojo, M. , Bobes, J. , Del‐Ben, C. M. , Di Forti, M. , … Schürhoff, F. (2021). The independent effects of psychosocial stressors on subclinical psychosis: Findings from the multinational EU‐GEI study. Schizophrenia Bulletin. sbab060. 10.1093/schbul/sbab060 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Pignon, B. , Peyre, H. , Ferchiou, A. , van Os, J. , Rutten, B. P. F. , Murray, R. M. , Morgan, C. , Leboyer, M. , Schürhoff, F. , & Szöke, A. , & EU‐GEI WP2 Group Author . (2019). Assessing cross‐national invariance of the Community Assessment of Psychic Experiences (CAPE). Psychological Medicine, 49(15), 2600–2607. 10.1017/S0033291718003574 [DOI] [PubMed] [Google Scholar]
  27. R Core Team . (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R‐project.org/ [Google Scholar]
  28. Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36. http://www.jstatsoft.org/v48/i02/ [Google Scholar]
  29. Schlier, B. , Jaya, E. S. , Moritz, S. , & Lincoln, T. M. (2015). The community assessment of psychic experiences measures nine clusters of psychosis‐like experiences: A validation of the German version of the CAPE. Schizophrenia Research, 169(1–3), 274–279. 10.1016/j.schres.2015.10.034 [DOI] [PubMed] [Google Scholar]
  30. Stefanis, N. C. , Hanssen, M. , Smirnis, N. K. , Avramopoulos, D. A. , Evdokimidis, I. K. , Stefanis, C. N. , Verdoux, H. , & van Os, J. (2002). Evidence that three dimensions of psychosis have a distribution in the general population. Psychological Medicine, 32(2), 347–358. [DOI] [PubMed] [Google Scholar]
  31. Wigman, J. T. W. , Vollebergh, W. A. M. , Raaijmakers, Q. A. W. , Iedema, J. , van Dorsselaer, S. , Ormel, J. , Verhulst, F. C. , & van Os, J. (2011). The structure of the extended psychosis phenotype in early adolescence—a cross‐sample replication. Schizophrenia Bulletin, 37(4), 850–860. 10.1093/schbul/sbp154 [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Wing, J. K. , Babor, T. , Brugha, T. , Burke, J. , Cooper, J. E. , Giel, R. , Jablenski, A. , Regier, D. , & Sartorius, N. (1990). SCAN. Schedules for Clinical Assessment in Neuropsychiatry. Archives of General Psychiatry, 47(6), 589–593. 10.1001/archpsyc.1990.01810180089012 [DOI] [PubMed] [Google Scholar]
  33. Wüsten, C. , Schlier, B. , Jaya, E. S. , Genetic Risk and Outcome of Psychosis (GROUP) Investigators . Fonseca‐Pedrero, E. , Peters, E. , Verdoux, H. , Woodward, T. S. , Ziermans, T. B. , Lincoln, T. M. , Delespaul, P. , Meijer, C. J. , Myin‐Germeys, I. , Kahn, R. S. , Schirmbeck, F. , Simons, C. J. P. , van Haren, N. E. , van Os, J. , … Lincoln, T. M. (2018). Psychotic experiences and related distress: A cross‐national comparison and network analysis based on 7141 participants from 13 countries. Schizophrenia Bulletin, 44(6), 1185–1194. 10.1093/schbul/sby087 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Youden, W. J. (1950). Index for rating diagnostic tests. Cancer, 3(1), 32–35. [DOI] [PubMed] [Google Scholar]
  35. Yung, A. R. , Nelson, B. , Baker, K. , Buckby, J. A. , Baksheev, G. , & Cosgrave, E. M. (2009). Psychotic‐like experiences in a community sample of adolescents: Implications for the continuum model of psychosis and prediction of schizophrenia. Australian and New Zealand Journal of Psychiatry, 43(2), 118–128. 10.1080/00048670802607188 [DOI] [PubMed] [Google Scholar]
  36. Ziermans, T. (2013). Working memory capacity and psychotic‐like experiences in a general population sample of adolescents and young adults. Frontiers in Psychiatry, 4, 161. 10.3389/fpsyt.2013.00161 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supporting Information S1

Click here for additional data file. (56.3KB, docx)

Data Availability Statement

The data that support the findings of this study are available from the Genetic Risk and Outcome of Psychosis (GROUP). Restrictions apply to the availability of these data, which were used under licence for this study. Data are available from the authors with the permission of the Genetic Risk and Outcome of Psychosis (GROUP).

Articles from International Journal of Methods in Psychiatric Research are provided here courtesy of Wiley

RESOURCES