The Validity of the 16-Item Version of the Prodromal Questionnaire (PQ-16) to Screen for Ultra High Risk of Developing Psychosis in the General Help-Seeking Population

Abstract

In order to bring about implementation of routine screening for psychosis risk, a brief version of the Prodromal Questionnaire (PQ; Loewy et al., 2005) was developed and tested in a general help-seeking population. We assessed a consecutive patient sample of 3533 young adults who were help-seeking for nonpsychotic disorders at the secondary mental health services in the Hague with the PQ. We performed logistic regression analyses and CHi-squared Automatic Interaction Detector decision tree analysis to shorten the original 92 items. Receiver operating characteristic curves were used to examine the psychometric properties of the PQ-16. In the general help-seeking population, a cutoff score of 6 or more positively answered items on the 16-item version of the PQ produced correct classification of Comprehensive Assessment of At-Risk Mental State (Yung et al., 2005) psychosis risk/clinical psychosis in 44% of the cases, distinguishing Comprehensive Assessment of At-Risk Mental States (CAARMS) diagnosis from no CAARMS diagnosis with high sensitivity (87%) and specificity (87%). These results were comparable to the PQ-92. The PQ-16 is a good self-report screen for use in secondary mental health care services to select subjects for interviewing for psychosis risk. The low number of items makes it quite appropriate for screening large help-seeking populations, thus enhancing the feasibility of detection and treatment of ultra high-risk patients in routine mental health services.

Introduction

The proposed inclusion of a diagnostic category for psychosis risk to be called “attenuated psychosis syndrome” in the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition, has led to an ongoing and fierce debate.^1,2 The criteria for this category have been developed in an attempt to identify individuals who are likely to develop schizophrenia and other psychotic disorders within the next few years. They rely heavily on the criteria that have been found to identify individuals at ultra high-risk (UHR) for developing psychosis using clinical interviews.^3–6 However, research has been limited to resource-consuming diagnostic measures among highly selected samples in tertiary specialized research clinics.² A key question remains whether and how the criteria for the psychosis risk syndrome can be applied in populations with a lower a priori probability of psychosis risk, outside the specialized clinics.² As the incidence of UHR is low and the early signs and symptoms are nonspecific, this approach is only likely to succeed if an adequate diagnostic screening instrument is available, with which individuals with suspected UHR symptoms can be identified for further in-depth diagnostic interviews. The test should have a high specificity (to maximize the proportion of actual negatives which are correctly identified as such) and also a good sensitivity (to maximize the proportion of actual positives which are correctly identified as such). In addition, the screening should be acceptable to clinicians and the population to be screened, and the procedure should be cost effective and feasible in routine mental health care.⁷

Several self-report screening measures for psychosis risk have been developed to screen more efficiently for the attenuated psychosis syndrome, including the Prodromal Questionnaire (PQ).⁸ The PQ seems to be a good resource because it is designed to function as the first step in a two-stage screening process⁹ with a subsequent clinical assessment.

The relatively low specificity of the PQ^8,10 may be increased if it is not used in the general population but in an enriched sample with proportionally more UHR patients, reducing the number of false positives.¹¹ A general help-seeking population of young adults who seek treatment for nonpsychotic mental disorders in secondary health care settings, combining multiple risk factors,^9,11–13 seems to be an optimal trade-off between the general population and the specialty clinic–referred samples. However, the 92-item PQ may be too time consuming for routine screening because of the long administration time. Hence, in this study, we developed a short version of the PQ and tested this instrument as a diagnostic screening test for the attenuated psychosis syndrome in secondary mental health care for nonpsychotic disorders.

Methods

Sample and Procedure

As part of the Dutch Early Detection and Intervention Evaluation study (EDIE-NL),¹⁴ we administered a Dutch-language version of the PQ in the context of a two-stage screening procedure in the consecutive help-seeking population accessing a secondary mental health care service in the Hague area. The EDIE trial is a randomized controlled trial comparing treatment as usual with an add-on cognitive behavioral therapy targeted at the prevention of psychosis. The trial is registered at Current Controlled Trials: number ISRCTN21353122. Ethics approval was received from the “Centrale Commissie Mensgebonden Onderzoek (CCMO)” Central Committee for Research Involving Human Subjects (CCMO number NL17123.097.07). The main outcome measure is the number of participants who develop a first episode of psychosis within 18 months. The secondary outcome is the reduction of the persistence of subclinical psychotic symptoms.

“PsyQ Haaglanden” is the provider of general adult mental healthcare in The Hague. The catchment area covers approximately 600 000 inhabitants. About 5000 (0.8%) inhabitants seek help for nonpsychotic disorders every year; 2200 (44%) are between 18 and 35 years old and almost 70% of them actually complete the intake procedure. Care is provided by nine health care programs: attention deficit hyperactivity disorder, anxiety disorders, posttraumatic stress disorder, mood disorders, eating disorders, personality disorders, relational problems, sexual disorders, and psychosomatic disorders. Help-seeking individuals, who are referred by their general practitioner, are first screened by a 20-min telephone interview. Following this initial screening, patients are referred to one of the care programs for further assessment and diagnosis. Patients with a first episode of psychosis are referred to the early intervention center.

Study participants were 3671 individuals aged 18–35 years who were screened with the PQ between February 2008 and February 2010. In line with other researchers,¹⁵ we aimed to select people with PQ-positive symptom scores in the top 20% of the distribution for further investigation, as the approximate risk of UHR is probably lower in a general mental health setting in comparison to an UHR referral setting. Therefore, we assumed that the risk of missing true-positive subjects should be very low with inclusion of 20% of the highest scores. To achieve this, the criterion was 18 or more PQ-positive symptom items. Patients with scores above the cutoff were assessed with the Comprehensive Assessment of At-Risk Mental States (CAARMS) to investigate their status, ie, psychosis risk, psychosis, or neither.

Exclusion criteria were current or previous use of antipsychotic medication amounting to more than 15 mg haloperidol equivalents, severe learning disability, psychiatric symptoms due to somatic etiology, insufficient fluency in the Dutch language, living outside the catchment area, a history of psychosis, and a global assessment of functioning score of 65 points or above. As a result, 138 subjects were excluded from further analysis.

A total of 639 individuals were eligible for interviewing. Another 219 subjects that scored above the cutoff were not assessed with the CAARMS due to no contact/refusal or repeated no-show (see figure 1). Eventually, 420 people (response rate of 66%) were interviewed with the CAARMS. Fifty-two patients were identified with a psychotic disorder, 147 with an UHR, and 221 received no diagnosis on the CAARMS. The PQ is less sensitive for distinguishing between the threshold of psychosis risk and clinical psychosis because it only measures the prevalence and not the intensity of symptoms.⁸ Therefore, we decided to combine CAARMS psychosis risk and psychosis in one group for our analyses.

Fig. 1.

Flowchart of the help-seeking population.

The CAARMS was also completed by a randomly selected test sample of subjects with scores under the criterion of 18 positive PQ symptom items (n = 147) in order to generate a receiver operating characteristic (ROC) curve that examines the ability of the PQ to discriminate between the diagnostic groups. No patient was detected with a psychosis status in this sample, and a score of 17 or higher on the positive PQ subscale detected the most patients with an UHR. These results of the interviewed subjects with low scores were extrapolated to the sample that was not interviewed with the CAARMS.

Measures

92-Item PQ.

Participants were assessed with the 92-item PQ (authorized Dutch translation by M. van der Gaag, R. Klaassen and L. Wunderink),⁸ a self-report screening questionnaire that assesses the presence of attenuated psychotic symptoms on a two-point scale (true/false), assessed over participants’ lifetimes for the purposes of this study. On average, it takes 20 min to complete. The items are divided into four major subscales: positive symptoms (45 items), negative symptoms (19 items), disorganized symptoms (13 items), and general symptoms (15 items).

Positive symptoms are grouped in three subscales: (1) unusual thought content, delusional ideas and paranoia (22 items); (2) perceptual abnormalities and hallucinations (17 items); and (3) conceptual disorganization (6 items).

CAARMS.

The criteria for UHR or at risk mental state for psychosis were assessed using the semi-structured CAARMS; authorized Dutch version by M.v.d.G., J. van der Werf, L.W., A. Malda, R.M.C.K.)³ as a gold standard. Yung, who is one of the developers of the CAARMS criteria, extensively trained the researchers during 2 days. The CAARMS was employed to determine the intensity/severity (0–6), frequency/duration (0–6), and fluctuation of attenuated psychotic symptoms (0–2). The CAARMS consists of seven subscales that include: four Positive Symptom items, two Cognitive and three Emotional Disturbances items, three Negative Symptoms items, four Behavioral Change items, four Motor/Physical Changes items, and eight General Psychopathology items.

Symptomatic criteria for psychosis risk are exclusively based on positive symptom items: disorder of thought content, perceptual abnormalities, and conceptual disorganization. The CAARMS distinguishes between three UHR groups:

• (i) Genetic group: family history of any psychotic disorder in a first-degree relative or a diagnosis of schizotypy in the participant.

• (ii) “Attenuated” psychotic symptoms: subthreshold, attenuated positive psychotic symptoms developed or worsened during the past 12 months, such as ideas of reference, odd beliefs, magical thinking, or unusual perceptual experiences.

• (iii) Brief limited intermittent psychotic symptoms: having experienced episodes of frank psychotic symptoms that lasted less than 1 week within the previous 12 months and that resolved spontaneously without treatment.

CAARMS assessment meetings with experienced clinical researchers were held every week to reach diagnostic consensus.

SOFAS.

To reach inclusion criteria of UHR, there has to be an impairment in social functioning as assessed with the SOFAS,¹⁶ ie, a SOFAS score of 50 points (on a scale ranging from 0 to 100) or less and/or a drop in SOFAS score of 30%, both in the last 12 months. The SOFAS is scored by the assessor and discussed in weekly diagnostic meetings.

Statistical Analyses

Data were analyzed using SPSS version 18.0.3¹⁷ and MedCalc version 11.6.1.0.¹⁸

Logistic Regression Analysis.

We performed stepwise logistic regression analyses. This is an automatic procedure based on the change in likelihood resulting from including or excluding the variable (the cutoff point for significance being 0.05).¹⁹ Backward stepwise elimination was used in the logistic regression of CAARMS diagnosis (dependent variable) on the PQ-92 items (independent variables), resulting in excluding 40 items that were not significantly associated with UHR and psychosis. Subsequently, we performed a forward stepwise inclusion of the 52 items that were left, which involved starting with no variables in the model, trying out the variables one by one and including them if they were significantly associated with psychosis risk or psychosis. After this step, we performed a CHi-squared Automatic Interaction Detector decision tree analysis over the 30 remaining items. This analysis was carried out using SPSS Answer Tree version 2.1.¹⁷ It generated binary decision trees and was used to find which independent variables of the remaining items were most strongly associated with the dependent variable. Eleven items were found to be good predictors. We added three more positive symptom items to assess visual hallucinations (1 item), perplexity/ delusional mood (1 item), and ideas of reference (1 item). Finally, we added two items on negative symptoms (item number 1 “excessive social anxiety” and item number 7 “avolition” as shown in the Appendix) to assess these as well and to increase the sensitivity of the questionnaire by almost 2% while maintaining the specificity (see Appendix for the 16-item version of the PQ).

ROC Analyses.

In order to investigate the validity and reliability of the newly developed questionnaire, we compared it to the PQ-92. For this purpose, we conducted ROC analyses in the total sample of 3533 individuals and determined sensitivity, specificity, positive predictive values (PPV), and negative predictive values. The PPV reflects the probability that an individual has a CAARMS diagnosis when a positive test result on the PQ is observed. Pairwise comparison of ROC curves was performed to test the statistical significance of the difference between the areas under the ROC curves of the PQ-16 and the PQ-92. The accuracy of the test depends on how well the test separates the group being tested into those with and without the disease in question and is measured by the area under the curve (AUC). An AUC test value equal to 0.5 means that there is no difference and when the value equals 1 there is perfect separation of the values of the two groups.

Cronbach Alpha Validity Indices.

The clinical validity/internal consistency of the PQ-16 in the general help-seeking population was examined, using Cronbach’s alpha statistics. A score above .65 is considered sufficient internal consistency. A biserial correlation was used to examine the association of the total PQ-16 score with CAARMS diagnosis.

Results

Sample Characteristics

The final population included 3533 patients. Age and intake diagnoses are presented in table 1. The overall sample was predominantly female and the majority (45%) was diagnosed with an anxiety or mood disorder.

Table 1.

Gender, Age, and Intake Diagnoses (ie, reported by the caretaker) of the General Help-Seeking Sample (n = 3533)

Characteristic
	Mean	SD
Age (years)	26.2	4.9
	N	%
Female	2419	68.5
Intake diagnosis
Anxiety disorder	903	25.6
Mood disorder	686	19.4
Attention deficit hyperactivity disorder	444	12.6
Relational problems	394	11.2
Personality disorder	350	9.9
Posttraumatic stress disorder	221	6.3
Eating disorder	176	5.0
No diagnosis	169	4.8
Psychosomatic disorder	96	2.7
Sexual disorder	84	2.4
Other	10	0.3

A 16-item list was selected (see Appendix). For use as a screening instrument, a cutoff score of 6 or more symptom items was found to identify “caseness” (UHR/psychosis) best with a sensitivity of 87%, resulting in a specificity of 87% and PPV of 44%. Table 2 presents further details of this analysis.

Table 2.

Classification Accuracy of PQ-16 (ie, Sensitivity, Specificity, Predictive Values, Likelihood Ratio, Area Under the Curve, and Confidence Intervals) vs CAARMS Diagnosis of UHR/Psychosis vs No CAARMS Diagnosis

PQ-16 Cutoff, Total Score	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)	LR+ (%)	AUC	95% CI	P
≥0	100	0	11	—	1.00	0.93	0.92–0.94	.0001
≥1	100	16	12	100	1.19	0.93	0.92–0.94	.0001
≥2	100	34	15	100	1.51	0.93	0.92–0.94	.0001
≥3	100	52	19	100	2.06	0.93	0.92–0.94	.0001
≥4	97	67	26	99	2.93	0.93	0.92–0.94	.0001
≥5	91	79	34	99	4.30	0.93	0.92–0.94	.0001
≥6a	87	87	44	98	6.75	0.93	0.92–0.94	.0001
≥7	76	92	53	97	9.61	0.93	0.92–0.94	.0001
≥8	58	95	59	95	12.48	0.93	0.92–0.94	.0001
≥9	43	97	63	94	14.37	0.93	0.92–0.94	.0001
≥10	31	99	72	92	21.47	0.93	0.92–0.94	.0001
≥11	22	99	75	92	25.34	0.93	0.92–0.94	.0001
≥12	16	100	81	91	35.42	0.93	0.92–0.94	.0001
≥13	10	100	86	90	51.29	0.93	0.92–0.94	.0001
≥14	5	100	86	90	54.14	0.93	0.92–0.94	.0001
≥15	3	100	92	90	94.04	0.93	0.92–0.94	.0001
≥16	1	100	100	90	—	0.93	0.92–0.94	.0001

Note: PQ, Prodromal Questionnaire.

^aHighest average of sensitivity and specificity.

ROC curves were plotted for PQ-16 total score to predict no CAARMS diagnosis vs CAARMS UHR/psychosis (see figure 2). The AUC was significant for the total score (AUC = 0.93, SE = 0.006, 95% CI = 0.92–0.94, P = .0001).

Fig. 2.

Receiver operating characteristic curves of PQ-92 and PQ-16 scores predicting UHR/psychotic diagnosis vs no CAARMS diagnosis

The PQ-92 was found to identify “caseness” best with a sensitivity of 90% and a specificity of 90% at cutoff of 18 or more positive PQ symptom items against CAARMS diagnosis. The PPV was good with 52% (table 3).

Table 3.

Classification Accuracy of PQ-92 (ie, Sensitivity, Specificity, Predictive Values, Likelihood Ratio, Area Under the Curve, and Confidence Intervals) vs CAARMS Diagnosis of UHR/Psychosis vs No CAARMS Diagnosis

PQ-92 Cutoff Total Score	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)	LR+ (%)	AUC	95% CI	P
≥8	100	44	17	100	1.77	0.95	0.94–0.95	.0001
≥14	96	76	32	99	3.95	0.95	0.94–0.95	.0001
≥18a	90	90	52	99	9.30	0.95	0.94–0.95	.0001
≥19	74	93	56	97	10.82	0.95	0.94–0.95	.0001

Note: PQ, Prodromal Questionnaire.

^aHighest average of sensitivity and specificity.

The difference between areas under the curve of the PQ-92 compared to the PQ-16 was 0.0142. The AUC of the PQ-92 was significantly (p = .0001) larger than the AUC of the PQ-16 (see figure 2).

Internal Consistency and Concurrent Validity of the PQ-16

Total score on the PQ-16 was significantly correlated with the CAARMS diagnosis (UHR/psychosis or neither; r = .572) at the level of p = .000 (two tailed). Cronbach’s alpha for the total score on the PQ-16 was .774. All item-total correlations were greater than .31.

Discussion

To the best of our knowledge, our study is the first to develop a short screening test for the attenuated psychosis syndrome with good psychometric properties in an adult population seeking help for nonpsychotic disorders in routine mental health care. Overall, the newly developed PQ-16 has good concurrent validity with both the interview-based CAARMS diagnoses in our population and also in comparison to the original PQ. A cutoff of 6 or more symptoms on the PQ-16 has a high true positive rate (87%) and high specificity (87%) when differentiating UHR/psychosis from those with no CAARMS diagnosis. The PQ-16 eliminated most of the people that had no psychotic spectrum diagnosis; it correctly rejected 9 out of every 10, comparable to the PQ-92. The PPV is 44% in this sample and reflects the probability that an individual has a psychosis risk syndrome or clinical psychosis as defined by the CAARMS, if a positive test result on the PQ is observed. Furthermore, we found a Cronbach’s alpha of about .8 for the internal consistency of the PQ-16, which we regard as satisfactory for a screening measure that has to be followed by a clinical interview.

The 16-item PQ consists of 9 items out of the perceptual abnormalities/hallucinations subscale, 5 items including unusual thought content/delusional ideas/paranoia, and 2 negative symptoms. The perceptual aberrations seem to be overrepresented; however, this confirms the observation of Maher²⁰ that perceptual aberrations are primary psychotic phenomena and that delusions are explanations of these abnormal sensory experiences. Perceptual aberrations could therefore be seen as a first phenomenon leading to psychosis.²¹ Furthermore, Kelleher et al²² assented the sensitivity of self-reported auditory hallucinations with regard to interview-verified attenuated psychotic symptoms. Moreover, a large birth cohort study in Northern Finland showed that perceptual abnormalities were the best predictor of developing psychosis during the 11-year follow-up.²³

Meeting the Criteria of a Screening Instrument

As discussed previously, a screening instrument needs to have practical benefits and acceptable psychometric properties.⁷ The PQ-16 predicted UHR/psychosis with a high sensitivity and high specificity, comparable to the original version of the PQ. Furthermore, the PQ-16 seems a suitable and practical screening measure as it takes only a few minutes to administer and it does reliably separate UHR/psychosis from no CAARMS diagnosis, which reduces the number of in-depth interviews to an acceptable amount. In our experience, the PQ-16 is acceptable to the patient-population and clinicians.

Another important point of consideration by screening a help-seeking population on risk of emergent psychotic disorders is the role of stigma and resources. False detection of individuals could lead to unnecessary concern and anxiety.²⁴ However, in our setting, the individuals that we screened were all seeking treatment and were quite motivated to get help for their mental health problems. Screening for potential future mental health problems appeared to be plausible for each individual, and they did not report to the researchers that they felt stigmatized. Lack of inducing stigma could also have been associated with our terminology. We chose to speak in terms of “at risk of developing mental health problems in the future” and never mentioned psychosis risk in the screening phase.

In addition, a systematic screening process might allow for the identification and treatment of patients to possibly prevent or delay transition to psychosis. Furthermore, if necessary, a quick referral to a first episode service was made in order to shorten the duration of untreated psychosis (DUP), which in turn improves prognosis and can save costs in the long term.²⁵

Prevalence of Prodromal and Psychotic Symptoms

This study found 1.4% of the help-seeking population presenting with a nonpsychotic disorder to actually have a psychotic disorder. An UHR syndrome was diagnosed in 4.0% of the help-seeking population. The high rate of the interviewed subjects that met the criteria for a psychotic disorder, but that had not been detected by clinicians, suggests that mental health care workers are often unaware of the presence of psychotic symptoms in patients who seek treatment for other mental disorders, as has also been demonstrated by other authors.^26–28 Therefore, screening with the PQ-16 in the general help-seeking population in secondary mental health care may aid in reducing DUP.

Strengths and Limitations

The strength of this study is that to our knowledge, this is the first study that showed that it is possible to detect young people with attenuated psychotic symptoms in a general help-seeking sample, using a two-stage screening process. The reduction of the 92-item PQ was successful, resulting in a 16-item version with similar and good psychometric properties. Our questionnaire was developed using data from a large epidemiological help-seeking sample (n = 3533), unlike the recently developed brief version of the Prodromal Questionnaire (PQ-B).⁹ The PQ-B showed reasonable psychometric properties (89% sensitivity and 58% specificity) in a relative small specialty clinic-referred sample (n = 141) but has not yet been tested in a general mental health treatment setting. Another advantage of the PQ-16 is that it is contains fewer items than the PQ-B, which has 21 items, each of which contains a subquestion. The PQ-16 has not yet been tested for its accuracy in predicting actual transition to psychosis in our study sample but we will be able to do so in the near future.

A limitation of this study is that the PQ-16 is performed in a sample that is “enriched” for the target diagnoses with young adults seeking help for nonpsychotic disorder and an impairment in social functioning and therefore not useable in the general population. Psychotic experiences occur commonly in the general population and are not always characterized by distress or treatment seeking;²⁹ too many false positives would be detected. However, the PQ-16 could be tested in patients younger than 18 years (due to logistic problems, we were unable to do so) in mental health care and in primary health. The results of the current study suggest that pursuing such a study is warranted.

Another limitation is that the PQ is not sensitive enough to distinguish between UHR syndromes and psychosis. Intensity of symptoms and suffering may be more important than presence of symptoms in distinguishing between psychosis risk and psychosis. However, because the PQ has the function of a screening instrument and in the second stage of assessment the CAARMS interview can make this discrimination well, it is less relevant.

Another possible weakness is the relatively small number of interviewed screen negatives (n = 147) on which we based the extrapolations in our study. We randomly selected 20% with a positive symptom score of 12 on the PQ-92 for the CAARMS-interview, 20% with a score of 13 and so on, through a PQ-92 score of 17. No patients were detected with a psychosis status in this sample, and a score of 17 or higher on the positive PQ subscale detected the most patients with an attenuated psychosis syndrome.

The PPV in this sample is 8% lower than in the 92-item version. This is a small disadvantage, and as a result, detection of UHR and psychosis needs slightly more interviews. On the other hand, a short screening instrument is much more acceptable for patients and services. Moreover, we included four Likert scale follow-up questions on distress following each individual PQ item and we will be able to investigate in the near future if this increases the PPV of the PQ-16.

Conclusion

The PQ-16 is a good screening instrument for routine use in secondary mental health care. The low number of items makes it feasible to screen large help-seeking populations. With this instrument, the DUP of undetected patients with a psychosis can be shortened and detection and treatment of patients with an attenuated psychosis syndrome in routine mental health services can be greatly improved.

Funding

The Netherlands Health Research Council (ZonMW), The Hague (120510001; and NTR1085 to MvdG); The Brod Group for a study funded by Genentech to R.L.L.; Jansen-Cilag and speakers fees from Astra-Zeneca, Bristol Myers Squibb, and Eli Lilly NL (to L.W.); Janssen-Cilag B.V (speakers bureau honorarium to D.H.L.).

Appendix. The 16-item Version of the Prodromal Questionnaire (PQ-16)

			If TRUE: how much distress did you experience?
			No	Mild	Moderate	Severe
1.	I feel uninterested in the things I used to enjoy.	□ True □ False	□ 0	□ 1	□ 2	□ 3
2.	I often seem to live through events exactly as they happened before (déjà vu).	□ True □ False	□ 0	□ 1	□ 2	□ 3
3.	I sometimes smell or taste things that other people can’t smell or taste.	□ True □ False	□ 0	□ 1	□ 2	□ 3
4.	I often hear unusual sounds like banging, clicking, hissing, clapping or ringing in my ears.	□ True □ False	□ 0	□ 1	□ 2	□ 3
5.	I have been confused at times whether something I experienced was real or imaginary.	□ True □ False	□ 0	□ 1	□ 2	□ 3
6.	When I look at a person, or look at myself in a mirror, I have seen the face change right before my eyes.	□ True □ False	□ 0	□ 1	□ 2	□ 3
7.	I get extremely anxious when meeting people for the first time.	□ True □ False	□ 0	□ 1	□ 2	□ 3
8.	I have seen things that other people apparently can't see.	□ True □ False	□ 0	□ 1	□ 2	□ 3
9.	My thoughts are sometimes so strong that I can almost hear them.	□ True □ False	□ 0	□ 1	□ 2	□ 3
10.	I sometimes see special meanings in advertisements, shop windows, or in the way things are arranged around me.	□ True □ False	□ 0	□ 1	□ 2	□ 3
11.	Sometimes I have felt that I’m not in control of my own ideas or thoughts.	□ True □ False	□ 0	□ 1	□ 2	□ 3
12.	Sometimes I feel suddenly distracted by distant sounds that I am not normally aware of.	□ True □ False	□ 0	□ 1	□ 2	□ 3
13.	I have heard things other people can't hear like voices of people whispering or talking.	□ True □ False	□ 0	□ 1	□ 2	□ 3
14.	I often feel that others have it in for me.	□ True □ False	□ 0	□ 1	□ 2	□ 3
15.	I have had the sense that some person or force is around me, even though I could not see anyone.	□ True □ False	□ 0	□ 1	□ 2	□ 3
16.	I feel that parts of my body have changed in some way, or that parts of my body are working differently than before.	□ True □ False	□ 0	□ 1	□ 2	□ 3