Abstract
Psychotic experiences are prevalent across a wide variety of psychiatric, neurological, and medical conditions. Yet current assessments are often designed for one disorder, or are limited in their examination of phenomenological features; this has hindered transdiagnostic research. This article describes an examination of the validity and reliability of the English version of a new assessment, the Questionnaire for Psychotic Experiences (QPE). This study aimed to use the QPE to examine hallucinations and delusions across a number of different conditions, and to ensure that the QPE had acceptable psychometric properties. An International Consortium on Hallucination Research working group, along with consumer groups, developed the 50-item QPE to assess the presence, severity, and phenomenology of hallucinations and delusions. Participants in the study who reported psychotic experiences included those with schizophrenia, schizoaffective disorder, bipolar affective disorder, and major depressive disorder, and those without a need for care (ie, nonclinical participants). There were 173 participants in total. Convergent and discriminant validity were assessed. Reliability was examined in terms of stability, equivalence, and internal consistency. The data confirmed that the QPE had good psychometric properties and could be put forward as an accepted measure of the transdiagnostic evaluation of psychotic experiences. Further validation is recommended with neurological and medical populations. Given its validity and reliability, comprehensive evaluation of psychotic phenomena, and relatively quick administration time, we propose that the QPE is a valuable instrument for both clinical and research settings.
Keywords: Questionnaire for Psychotic Experiences (QPE), scale validation, psychotic experiences, transdiagnostic
Introduction
Psychotic experiences, such as hallucinations and delusions, are common in psychiatric disorders, eg, schizophrenia, bipolar disorder, and major depressive disorders. They also occur in a proportion of the general population, as well as in neurological illnesses and medical conditions.1 There is a comprehensive literature that has presented the phenomenological characteristics of hallucinations and delusions in schizophrenia,2 yet there is a considerable paucity of investigation across most other conditions in psychiatry, neurological disorders, as well as medical illnesses; see Burghaus et al,3 Merrett et al,4 and Toh et al5 for reviews in the field.
The most commonly reported psychotic experiences are hallucinations in the auditory and visual modalities, as well as delusions. Yet most instruments that are used to examine phenomenology tend to focus solely on one modality, typically auditory hallucinations (AHs; eg, Psychotic Symptom Rating Scales [PSYRATS6]), or even one delusional theme, eg, paranoia (Green Paranoid Thinking Scale7). A number of other instruments, such as the Positive and Negative Syndrome Scale (PANSS8), assess delusions and hallucinations, but do not independently rate individual delusional themes or hallucination modalities with only global scores calculated. Further, there has been a paucity of study with regard to other psychotic phenomenon, eg, olfactory and somatic hallucinations, as well as both auditory and visual illusions. Sensed presence(s) is another sensory domain that has been rarely investigated, with some limited evidence that such experiences are overrepresented in bereaved populations.9 In addition, existing measures were primarily developed with singular or limited populations under consideration, with the majority of these instruments suited to examine psychotic experiences in schizophrenia spectrum disorders (ie, the PSYRATS), and a handful specifically designed for visual hallucinations (VHs) in neurodegenerative disorders (ie the Northeast Visual Hallucinations Interview [NEVHI]10). This limits a comprehensive understanding of psychotic experiences both within and across conditions. Therefore, to facilitate transdiagnostic comparisons of the broad range of psychotic phenomenon, a new measure was developed to assess psychotic experiences independent of diagnosis. The measure is called the Questionnaire for Psychotic Experiences (QPE; see www.qpeinterview.com/en), and it assesses the presence and phenomenology of auditory, visual, tactile, and olfactory hallucinations (OHs); sensed presence (SP), and concurrent multimodal hallucinations; and 9 common types of delusions. The QPE was designed to be used within the context of an interview.
The current study represents the first published validation of the QPE, and aimed to determine whether the English version offers accurate, valid, and interpretable data, by examining its psychometric properties, ie, completion rates, validity, and reliability. Construct validity will be explored via convergent and discriminant validity. Reliability was examined in terms of stability, equivalence, and internal consistency. This was to ensure that the QPE was showing similar patterns in the phenomenology of psychotic experiences to the considerable literature already published in the field in relation to schizophrenia spectrum disorders.
Methods
The Questionnaire for Psychotic Experiences
An International Consortium on Hallucination Research working group consisting of a panel of experts of different backgrounds (psychology, psychiatry, neurology) and consumer representatives developed the 50-item QPE. The QPE assesses the presence, severity, and phenomenological characteristics of hallucinations and delusions. The aim was to ensure that the QPE was transdiagnostic; therefore, existent instruments were screened for relevant items. These included PSYRATS,6 NEVHI,10 Scale for the Assessment of Positive Symptoms (SAPS11), Launay and Slade Hallucinations Scale12 and Cardiff Anomalous Perceptions Scale13, the Schedule for Clinical Assessment in Neuropsychiatry14, Community Assessment of Psychic Experiences,15 and Peters Delusional Inventory16. Fifty items were selected via our expert working group and via consumer consultation in the Melbourne (NT) and Sussex (CS) Voices Clinics, and modified into a standard format. The items that examined hallucinations gathered thorough information on auditory, visual, olfactory, tactile, and multimodal hallucinations and SP(s). The items on delusions covered the most frequent themes of delusions: paranoid, reference, guilt, control, religious, grandeur, somatic, Cotard’s syndrome (the conviction that one was dead or that part of the body has already died), and Capgras syndrome (the conviction that a known person has been replaced by an imposter). The latter 2 delusions were included as they are more common in neurological populations and are not typically employed in psychiatric assessments. The items were arranged into 4 modules or subscales: AHs, VHs, hallucinations in other modalities, and delusions (D). The questionnaire was completed during a semi-structured interview. The majority of items were initially explored using a series of predetermined questions; participants were subsequently required to narrow down their answers to a forced-choice response with regard to severity (6-point numerical scale see Supplementary Material) or phenomenological characteristics (alphabetical descriptives across 5–7 possibilities depending on the item see Supplementary Material). In addition, <10% of the items were open and required a short description of the experience, which was written verbatim. The QPE takes between 20 and 40 min to complete depending on the number of psychotic experiences a participant endorses. We assessed psychotic experiences across 2 time frames: (1) lifetime experiences (lifetime) and (2) experiences in the past 7 days (current). Thus, in this report there are two scores for each of the 50 items: lifetime scores and current scores.
Overall, the QPE permits analyses across the 4 subscales: AH, VH, total hallucinations (TotH), and D as well as giving a total severity of psychotic experiences, or total QPE score (thus 5 scores in total). For each of the subscales, the severity forced-choice items were summed. This included the following items for AH and VH: frequency, duration, distress, and impact; and for D: conviction, preoccupation, distress, and impact (with each of the AH, VH, and D items having a 6-point numerical scale from 0 to 5, the items relating to phenomenological characteristics were not included in these subscale scores). TotH and total QPE also included frequency of tactile hallucinations (THs), frequency of OHs, frequency of multimodal hallucinations, and frequency of SP (other severity information, ie, distress and impact, for these latter 4 hallucination modalities was not included in the QPE to ensure brevity).
Other Measures
To examine convergent validity, a series of current “gold standard” measures in their class were included in the assessment to investigate whether the QPE was in accordance with instruments that measured similar constructs. A semi-structured interview format was required for each of the measures selected as gold standard to ensure maximal compatibility. Self-report measures were not selected. Our measures included the PSYRATS,6 PANSS,8 SAPS,11 and the NEVHI.10 The PSYRATS has two subscales relevant to the current study: AH and delusions. The PANSS has two items from the positive scale that were relevant: an item score for delusions (P1) and an item score for hallucinations (P3). The SAPS has two subscales relevant: hallucinations and delusions. The delusion subscale of the SAPS was used in the convergent validity analyses to correlate with the D of the QPE. As the SAPS hallucination subscale sums across auditory and visual items, we used the individual SAPS item scores for auditory and VHs in the convergent validity analyses to correlate with the relevant subscales of the QPE. The NEHVI provides a total VH severity score, which was correlated with the VH subscale of the QPE. Psychotic and mood symptom severity were also obtained using the PANSS, Beck Depression Inventory (BDI-II17) and Beck Anxiety Inventory (BAI18). For the correlations performed during the discriminant function analyses, the total scores of BDI and BAI were used, as well as the negative and general subscales of the PANSS, as the positive PANSS subscale assesses for hallucinations and delusions and was included in the convergent validity correlations. The measures chosen to investigate discriminant validity represent common symptoms experienced in psychosis, ie, negative symptoms, general symptoms, depression, and mood, but are not expressions of psychotic experiences. For both convergent and discriminant validity, only 4 of the QPE subscales were used, AH, VH, D, and total QPE. When designing the battery, there were no other instruments available in the literature that assessed hallucinations in other modalities, ie, THs, OHs, multimodal hallucinations, or SP in a semi-structure interview format (SAPS does have olfactory and tactile but not multimodal hallucinations or SP).
Participants
Participants were recruited across in- and outpatient services of The Alfred Hospital, St Vincent’s Hospital, and The Melbourne Clinic, as well as community support groups and forums (the latter was particularly important for recruiting the nonclinical participants), all in Melbourne, Australia. Inclusion criteria for all participants were that they needed to be ≥18 years of age, be fluent English language speakers, and have the capacity to adequately answer questions and communicate their thoughts (ie, an absence of gross thought disorder). With regard to hallucinations, participants must have experienced AH at least 3 times over their lifetime to be included (this needed to be a significant or notable experience each time, ie, >2 min, and could not be a hearing of the name or very fleeting experience, ie, <20 s). If these symptoms were experienced within the past 7 days, a designation of current AH was given. When participants were assessed on lifetime AH, they were asked to reflect upon their most prominent or “worst ever” episode. Four clinical group were recruited: schizophrenia, schizoaffective disorder, bipolar disorder, and major depressive disorder. An additional nonclinical group was also recruited; these included persons with a history of current or lifetime AHs without a need for care, ie, no diagnosis of a mental health disorder. All participants completed the Mini International Neuropsychiatric Interview (MINI19) to confirm their primary Diagnostic and Statistical Manual of Mental Disorders - IV - Text Revision psychiatric diagnosis (the nonclinical group was verified as not having any current psychiatric diagnosis on the MINI). Basic demographic characteristics were obtained for the sample including age, gender, and education (see table 1).
Table 1.
Participant Characteristics of the Sample
| Schizophrenia | Schizoaffective Disorder | Bipolar Affective Disorder | Major Depressive Disorder | Nonclinical Participants | |
|---|---|---|---|---|---|
| n | 50 | 26 | 31 | 34 | 32 |
| Age (M [SD]) | 40.3 (11.1) | 43.4 (10.7) | 32.1 (12.8) | 30.2 (12.8) | 28.6 (8.9) |
| Sex (male:female) | 22:28 | 8:18 | 16:15 | 14:20 | 15:17 |
| Educationa(M [SD]) | 13.8 (3.4) | 13.7 (2.8) | 15.6 (3.9) | 14.1 (2.1) | 15.5 (2.2) |
| WTAR IQ (M [SD]) | 102.2 (12.8) | 104.5 (11.4) | 107.6 (11.5) | 106.4 (9.4) | 102.8 (8.6) |
| PANS P (M [SD]) | 19.4 (5.9) | 16.3 (5.7) | 16.0 (5.3) | 12.6 (3.7) | 10.7 (2.9) |
| PANS N (M [SD]) | 13.5 (5.5) | 11.9 (4.8) | 10.2 (3.4) | 10.8 (3.7) | 8.3 (1.4) |
| PANSS G (M [SD]) | 34.1 (8.3) | 34.2 (9.7) | 30.8 (8.2) | 30.5 (8.4) | 22.3 (4.1) |
| PANSS T (M [SD]) | 67.0 (15.9) | 62.5 (16.2) | 57.0 (14.9) | 54.0 (14.3) | 41.3 (6.5) |
| BDI (M [SD]) | 17.0 (11.7) | 20.7 (13.0) | 21.2 (15.2) | 21.4 (15.7) | 6.6 (7.4) |
| BAI (M [SD]) | 16.7 (12.0) | 20.1 (12.2) | 25.5 (17.9) | 22.4 (13.3) | 9.1 (8.2) |
| Medicationb | None = 13.6% | None = 8.0% | None = 22.6% | None = 55.9% | None = 93.8% |
| AP = 47.7% | AD = 4.0% | AD = 16.1% | AD = 23.5% | AD = 6.2% | |
| AP + AD = 22.7% | AP = 16.0% | MS = 6.5% | AX = 5.9% | ||
| AP + MS = 6.8% | AP + AD = 52.0% | AP = 12.9% | AP + AD = 5.8% | ||
| AP + AX = 9.1% | AP + MS = 8.0% | AP + AD = 19.4% | AD + MS = 8.8% | ||
| AP + AX = 8.0% | AP + MS = 16.1% | ||||
| AD + AX = 4.0% | AD + MS = 6.5% |
Note: M, mean; SD, standard deviation; WTAR, Weschler Adult Reading Test (used to calculate premorbid IQ); PANS P, Positive and Negative Syndrome Scale Positive; PANS N, Positive and Negative Syndrome Scale Negative; PANNS G, Positive and Negative Syndrome Scale General; PANSS T, Positive and Negative Syndrome Scale Total; BDI, Beck Depression Inventory; BAI, Beck Anxiety Inventory; MS, mood stabilizer; AD, antidepressant; AP, antipsychotic; AX, anxiolytics.
aEducation calculated as the number of years in formal education (includes school and university education).
bMedications as follows: MS, AD, AP, and AX; the percentage of each disorder taking each medication or combination of medications was calculated.
Procedure
All measures were administered in the same order, in accordance with the specific instructions outlined for each individual measure. All raters (W.L.T., M.S., M.R., S.L.R.: psychologists or trained higher degree students) were trained to administer the assessment. To enable us to examine test–retest reliability, a random 10% (N = 16) of the total sample were assessed again 1–14 days after the original assessment (range 1–14 days, mean 5.8 [SD = 4.4]). Session 1 included the full assessment described earlier, and session 2 repeated the QPE for the last 7 days only. The same 10% subsample was also used to calculate inter-rater reliability, with two raters assessing the same participant on session 2. The study was performed in accordance with the Declaration of Helsinki20 and received ethical approval from the Alfred Hospital Human Research Ethics Committee (HREC 341/14), Melbourne, Australia. Each participant provided written informed consent prior to assessment.
Statistical Analyses
Acceptability.
Data quality was considered acceptable if more than 95% of participants completed the interview.
Validity.
Six principal component analyses (PCA) were performed to complete a cross-validation of the factor structure of the QPE in line with preliminary analyses performed during the scale development phase (www.qpeinterview.com/en): one PCA per subscale (AH, VH, and D) for each time frame (lifetime and current). For each subscale all items, 15 each for AH and VH and 5 for D were entered into the analyses. This included all forced-choice items for severity and phenomenological characteristics. During development, 3-factor models for the AH and VH subscales and a 1-factor solution for D were demonstrated. The current analyses sought to confirm these factor solutions. Promax rotations were used, given the ordinal nature of the items and expected correlations between them. Eigen values greater than 1 and factor loadings of greater than 0.4 were retained and considered satisfactory (see Mokkink et al21). To assess for convergent validity, inter-scale correlations were performed between the QPE and diagnosis-specific instruments measuring similar constructs; this included the PSYRATS, PANSS hallucination and delusion items, SAPS and NEVHI. To assess for discriminant validity, instruments that assessed general psychopathology and mood were used. These included PANSS negative and general, BDI and BAI. All inter-scale correlations were conducted using Kendall’s tau due to non-normality.
Reliability.
For the stability and equivalence analyses, the 6 subscale scores for current experiences were calculated using the forced-choice severity items only. First, stability (how consistent or stable an assessment is over time) was examined using the test–retest intra-class correlation coefficients (ICC; for all ICC analyses 2-way mixed, average measures, consistency models were used) for all 4 subscale scores and the total QPE across the 2 sessions. When performing our stability analyses, an error was noted with regard to our strategy of retest a maximum of 14 days later. The QPE interview requires participants to reflect upon the previous 7 days. Two of our participants were retested 13 and 14 days after their initial interview (all of the other 14 participants were retested within a 7-day window). Their status for the VH subscale changed from present to absent across the 2 time periods; thus they were removed from the VH ICC analysis. Their data were included for all the other subscale scores and for the total QPE subscale score (just removing the visual subscale) as their presence status for other psychotic experiences remained the same over the 2 assessment points (NB whether we removed all of these 2 participants data or just the visual subscales made no difference to the final results presented). Second, equivalence was determined using ICC for inter-rater reliability. Third, internal consistency of each subscale was determined using Cronbach’s α coefficient. Inter-scale correlations between subscales of the QPE were also calculated. A Cronbach’s α between .70 and .95 is argued to reflect good internal consistency.22
Results
Participants
A total of 173 participants with schizophrenia (n = 50), schizoaffective disorder (n = 26), bipolar disorder (n = 31), and major depressive disorder (n = 34) as well as nonclinical participants (n = 32) were administered the QPE. Basic participant characteristics are presented in table 1.
Figure 1 shows the variety of psychotic experiences endorsed across the different groups. As per our recruitment strategy, all participants (100%) had experienced AH in their lifetime, with AH currently experienced in 38%–76% of the participant subgroups. The schizophrenia and schizoaffective groups had similar endorsement rates across all the other psychotic experiences, with a large majority also experiencing delusions (lifetime ~90%, current ~60%), and approximately 50% of both samples reporting hallucinations in other modalities. The two mood disorder groups showed similar rates of endorsement (both lifetime and current) across all the psychotic experiences except delusions, which the bipolar group reported approximately twice as often, both for lifetime and current. The nonclinical group showed the lowest endorsement across the other psychotic experiences. All groups reported SP currently and across their lifetime (19%–45% of cases).
Fig. 1.
Symptom profiles of all participants who endorsed psychotic experiences, lifetime (N = 173: 100%) and currently (N = 126: 72.8%), display by diagnosis: (a) schizophrenia, (b) schizoaffective disorder, (c) bipolar affective disorder, (d) major depressive disorder, and (e) nonclinical controls. AH, auditory hallucination; VH, visual hallucination; TH, tactile hallucinations; OH, olfactory hallucinations, SP, sensed presence; D, delusions.
Acceptability
After consent was obtained, 100% of the sample reached the end of the QPE interview, illustrating high completion rates. One participant declined to answer any questions on the VH subscale but completed all other items. The average time to complete the interview was 30 min (range 20–60 min), depending on the number of psychotic experiences and response speed.
Validity
A 3-factor solution was supported for the AH and VH subscales for both current and lifetime (table 2) data. For the AH subscale, the analyses on the current experiences yielded the following dimensions: impact on functioning, incidence, and illusions. These factors accounted for 63% of the variance with a total fit of 0.99. For the lifetime AH subscale, a slightly different 3-factor solution was ascertained relating to impact on functioning, insight, and illusions. The solution accounted for 49% of the variance with 0.93 fit. Only one item “past experiences” failed to load on any of the 3 factors for both current and lifetime data. For the current VH subscale, the same dimensions as for AH subscale were found: impact on functioning, incidence, and illusions. These factors accounted for 76% of the variance with a total fit of 0.97. For the lifetime VH subscale, a slightly different 3-factor solution was ascertained to lifetime AH but the same as current AH relating to incidence, impact on functioning, and illusions. The solution accounted for 65% of the variance with 0.99 fit. All items loaded on at least one of the factors for both current and lifetime data. A unidimensional model was present for delusions (with all items loading on this factor), labeled impact on functioning. For the current data, the solution accounted for 73% of the variance with a 0.97 fit and for the lifetime data, 80% of the variance with 0.9 fit.
Table 2.
Principal Component Analysis With Promax Rotations Carried Out Within Three of the QPE Subscales
| Auditory Hallucinations | Visual Hallucinations | Delusions | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| No | Item | 1 | 2 | 3 | No | Item | 1 | 2 | 3 | No | Item | 1 |
| Current Experiences | ||||||||||||
| AH6 | Impact | 0.86 | VH13 | Commands | 0.92 | D10 | Preoccupation | 0.93 | ||||
| AH5 | Distress | 0.84 | VH4 | Emotional | 0.87 | D11 | Conviction | 0.88 | ||||
| AH13 | Commands | 0.82 | VH3 | Duration | 0.87 | D12 | Distress | 0.88 | ||||
| AH12 | Interaction | 0.79 | VH11 | Insight | 0.84 | D13 | Impact | 0.85 | ||||
| AH8 | Complexity | 0.75 | 0.62 | VH12 | Interaction | 0.84 | D14 | Impact hallucinations | 0.71 | |||
| AH4 | Emotional | 0.69 | VH2 | Past event | 0.73 | |||||||
| AH11 | Insight | 0.61 | VH9 | Location | 0.95 | |||||||
| AH7 | Repetition | 0.93 | VH8 | Complexity | 0.92 | |||||||
| AH10 | Time day | 0.86 | VH7 | Repetition | 0.89 | |||||||
| AH9 | Location | 0.80 | VH10 | Time day | 0.88 | |||||||
| AH1 | Frequency | 0.79 | VH1 | Frequency | 0.86 | 0.33 | ||||||
| AH3 | Duration | 0.50 | 0.63 | 0.33 | VH15 | Illusions | 0.85 | |||||
| AH14 | Music | 0.75 | VH14 | Passage | 0.84 | |||||||
| AH15 | Illusions | 0.59 | VH5 | Distress | 0.54 | 0.82 | ||||||
| A2 | Past event | 0.00 | VH6 | Impact | 0.55 | 0.70 | ||||||
| Lifetime Experiences | ||||||||||||
| AH6 | Impact | 0.88 | VH2 | Past event | 0.88 | D10 | Preoccupation | 0.93 | ||||
| AH3 | Duration | 0.83 | VH8 | Complexity | 0.88 | D12 | Distress | 0.92 | ||||
| AH1 | Frequency | 0.81 | VH13 | Commands | 0.86 | D13 | Impact | 0.91 | ||||
| AH5 | Distress | 0.79 | VH7 | Repetition | 0.83 | D11 | Conviction | 0.91 | ||||
| AH4 | Emotional | 0.75 | VH9 | Location | 0.79 | D14 | Impact hallucinations | 0.81 | ||||
| AH13 | Commands | 0.65 | VH3 | Duration | 0.75 | |||||||
| AH8 | Complexity | 0.57 | VH11 | Insight | 0.74 | |||||||
| AH7 | Repetition | 0.50 | VH10 | Time day | 0.71 | |||||||
| AH12 | Interaction | 0.39 | 0.48 | VH12 | Interaction | 0.62 | ||||||
| AH10 | Time day | 0.65 | VH6 | Impact | 0.87 | |||||||
| AH11 | Insight | 0.60 | VH5 | Distress | 0.87 | |||||||
| AH9 | Location | 0.51 | VH4 | Emotional | 0.61 | 0.74 | ||||||
| AH14 | Music | 0.76 | VH15 | Illusions | 0.82 | |||||||
| AH15 | Illusions | 0.64 | VH14 | Passage | 0.67 | |||||||
| AH2 | Past event | 0.40 | VH1 | Frequency | 0.38 | 0.66 | ||||||
Note: All loadings were >0.4; where loadings did not meet this threshold, they have been included but shaded in light gray. A total of N = 173 participants who reported auditory hallucinations either currently or lifetime were included. No, item number; AH, auditory hallucination subscale; VH, visual hallucination subscale; D, delusion subscale.
For convergent validity (all instruments expected to measure a similar construct as the QPE), we performed 84 correlations in n = 173 participants, with 19 predicted to show a significant correlation at P < .0001 (this P value is conservatively corrected for multiple comparisons using Bonferroni23), with r > .55 at this P-value. All, but one, of the 19 predicted correlations were significant, and the remaining 65 correlations were not significant. The data are displayed in table 3. In sum, QPE total AH subscale correlated with PSYRATS AH but not with the delusion subscale of the PSYRATS. Similarly the QPE D subscale correlated with the delusion score of the PSYRATS, but not the AH subscale. The PANSS hallucination item was correlated with QPE AH, but not VH, with PANSS scores only rated currently and not for lifetime in this study. This result illustrates that the PANSS hallucination item does not adequately assess VH. The PANSS delusions item was correlated with QPE D. The SAPS AH and VH items as well as the delusion subscales significantly correlated with the equivalent subscales on the QPE. Finally, the NEVHI correlated with the QPE VH subscale and nothing else. For discriminant validity (all instruments expected to measure a different construct as compared to the QPE), we performed 32 correlations with none expected to show a significant correlation at P < .0001 with r > .55. This data are displayed in table 3 and was as predicted.
Table 3.
Validity Assessments
| Alternate Measures | Subscale | QPE Subscales | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| AH | VH | D | QPE | |||||||
| L | C | L | C | L | C | L | C | |||
| Convergent Validity Inter-scale Correlations | ||||||||||
| PSYRATS | AH | L | 0.84*** | 0.36 | 0.44 | 0.18 | 0.56*** | 0.27 | ||
| C | 0.31 | 0.86*** | 0.22 | 0.35 | 0.29 | 0.67*** | ||||
| D | L | 0.48 | 0.21 | 0.93*** | 0.35 | 0.59*** | 0.30 | |||
| C | 0.18 | 0.36 | 0.34 | 0.95*** | 0.25 | 0.60*** | ||||
| PANSS | H | C | 0.47 | 0.58*** | 0.06 | 0.11 | 0.26 | 0.44 | ||
| D | C | 0.44 | 0.37 | 0.10 | 0.07 | 0.48 | 0.73*** | |||
| SAPS | AH | L | 0.69*** | 0.29 | 0.10 | 0.04 | 0.46 | 0.26 | ||
| C | 0.44 | 0.61*** | 0.05 | 0.17 | 0.23 | 0.36 | ||||
| VH | L | 0.11 | 0.08 | 0.68*** | 0.41 | 0.12 | 0.09 | |||
| C | 0.13 | 0.23 | 0.37 | 0.66*** | 0.07 | 0.21 | ||||
| D | L | 0.53 | 0.23 | 0.15 | 0.09 | 0.72*** | 0.44 | |||
| C | 0.33 | 0.41 | 0.09 | 0.16 | 0.36 | 0.75*** | ||||
| NEVHI | VH | L | 0.09 | 0.06 | 0.63*** | 0.41 | 0.09 | 0.05 | ||
| C | 0.07 | 0.21 | 0.39 | 0.65*** | 0.07 | 0.14 | ||||
| Discriminant Validity Inter-scale Correlations | ||||||||||
| PANSS | Neg. | 0.22 | 0.29 | 0.03 | 0.04 | 0.24 | 0.37 | 0.16 | 0.29 | |
| Gen. | 0.29 | 0.42 | 0.01 | 0.16 | 0.27 | 0.53 | 0.23 | 0.47 | ||
| BDI | 0.19 | 0.25 | 0.13 | 0.17 | 0.14 | 0.23 | 0.21 | 0.29 | ||
| BAI | 0.18 | 0.27 | 0.06 | 0.13 | 0.14 | 0.18 | 0.17 | 0.28 | ||
Note: Shaded boxes predicted significant correlations. AH, auditory hallucination; VH, visual hallucination; D, delusions; QPE, total severity psychotic experiences on the Questionnaire for Psychotic Experiences; PSYRATS, Psychotic Symptom Rating Scales; SAPS, Scale for the Assessment of Positive Symptoms; NEVHI, Northeast Visual Hallucinations Interview; PANSS, Positive and Negative Syndrome Scale; H, Hallucinations; Neg., negative subscale; Gen, general subscale; BDI, Beck Depression Inventory; BAI, Beck Anxiety Inventory; L, lifetime; C, current.
***P < .0001.
Reliability
Reliability data are presented in table 4, representing stability and equivalence. The ICC for stability ranged between 0.70 and 0.92, and for equivalence 0.99 and 1.00. Table 4 also displays the internal consistency. The QPE subscales showed good internal consistency with Cronbach’s α ranging from .75 to .90 for lifetime experiences, and 0.70 to 0.89 for current experiences. The inter-scale correlations were low and nonsignificant across the subscales (ie, AH, VH, and D), with the exception of AH with D, which was significant. Individual subscales were correlated with total subscale scores (ie, AH with TotH and QPE). Overall, this indicates that the different subscales measured different types of psychotic experiences.
Table 4.
Reliability Assessments
| Stability and Equivalence | |||||
|---|---|---|---|---|---|
| QPE Subscales | Stability (Test–Retest) N = 16 | Equivalence (Inter-rater) N = 16 | |||
| AH | 0.88 | 0.99 | |||
| VH | 0.92a | 1.00 | |||
| TotH | 0.82 | 1.00 | |||
| D | 0.70 | 1.00 | |||
| QPE | 0.81 | 1.00 | |||
| Internal Consistency b and Inter-subscale Correlations | |||||
| AH | VH | TotH | D | QPE | |
| AH | L 0.78 | ||||
| C 0.70 | |||||
| VH | L 0.07 | L 0.86 | |||
| C 0.29 | C 0.75 | ||||
| TotH | L 0.48** | L 0.60** | L 0.75 | ||
| C 0.75** | C 0.55** | C 0.76 | |||
| D | L 0.51** | L 0.10 | L 0.58** | L 0.90 | |
| C 0.39** | C 0.14 | C 0.55** | C 0.89 | ||
| QPE | L 0.55** | L 0.47** | L 0.79** | L 0.58** | L 0.85 |
| C 0.71** | C 0.46** | C 0.82** | C 0.55** | C 0.86 | |
Note: AH, auditory hallucination total score; VH, visual hallucination total score; TotH, total hallucinations; D, delusions; QPE, total severity psychotic experiences on the Questionnaire for Psychotic Experiences; L, lifetime; C, current.
aReduced sample of 14 (see “Methods” section).
bInternal consistency is represented in bold on the diagonal and are reported as Cronbach’s α. Inter-subscale correlations ** significant at 0.001 using Kendall’s Tau.
Discussion
The current study aimed to report the psychometric properties, ie, the validity and reliability, of the English version of the QPE. We recruited and administered the instrument to a large cohort of participants, including those with schizophrenia spectrum disorders, affective psychoses, and nonclinical participants. The completion rates were extremely high, ie, 100%, illustrating that the items were accepted by participants, and they were easily able to provide responses within the numerical and alphabetic forced-choice outlined for each item. In most cases, the interviews were brief, approximately 30 min; longer interviews were only needed when multiple experiences were present or patients were acutely unwell, necessitating longer response times.
The QPE gathered data on a wide range of psychotic experiences present in our cohort. Auditory hallucinations were the most frequent phenomenon reported upon. High rates of delusions were also present in 3 of the groups (schizophrenia, schizoaffective disorder, and bipolar disorder), a finding that is supported by the literature for these diagnostic conditions.24 Inclusion of a nonclinical group was useful as it established that the QPE can detect psychotic experiences in participants that do not have frequent contact with health professionals and have perhaps not over medicalized their experiences. Further, there was significant endorsement of tactile and olfactory hallucinations as well as SP in all of the cohorts; these experiences are not typically captured on current assessments, thus confirming the need for the QPE.
Our analyses established that the psychometric properties of the QPE are excellent. The PCA confirmed a 3-factor solution for both auditory and VHs, and a unidimensional solution for delusions. All but one of our items showed a high loading onto a single factor and was thus not redundant. Of note, the underlying dimensions of each subscale represented sensible phenomenological categories within hallucinations research. For both auditory and visual hallucinations, factors of incidence, illusions, and impact on functioning were evident, with insight emerging as a factor for lifetime AH but not VH. Indeed, previous research has noted similar physical, cognitive, emotional, and impact on functioning dimensions.2
Our convergent validity predictions were 96% accurate. The QPE was significantly correlated with the current most widely used measures in the field on the appropriate corresponding items or subscales. This was true for psychotic experiences reported currently and over the lifetime. None of the correlations outside our predictions were significant. Further, the data demonstrated that there were no significant correlations between the QPE subscales and general psychopathology or mood measures. Both these results confirmed that the QPE has specificity, and is measuring psychotic experiences, and not general features of mental illness.
Our reliability was excellent, with 100% accuracy for inter-rater assessments and 70%–92% accurate for test–retest. Such high rates of equivalence speak to accurate wording of the items as well as clearly defined forced-choice responses, allowing participants to easily select their category of response. Both raters were subsequently easily able to detect the response selected. Stability of the results may be more conservative, as they assume that a person’s response remains the same over time. However, psychotic experiences are known to be remarkably unstable over the course of a persons’ illness, and wax and wane over acute and chronic periods. The current data indeed illustrate this, with 2 patients showing different presentations of VH over the 2 sessions due to a slightly longer gap between sessions (thus excluding them from the analyses). Overall, the current data do suggest that when tested within a short <7 day time window, there is stability of responses on the QPE. Finally, there was good internal consistency with Cronbach’s α >.7 for all subscales.22
A particular strength of this study was that all of the 173 participants completed exactly the same assessment, including the extra measures used to determine validity. This ensured that we can be confident about our convergent and discriminant validity correlations (Terwee et al22 state that greater than n = 50 participants are required for such inter-scale correlations).
This study has several limitations. The cohort overall is a sizeable sample, however, modest for each diagnostic groups. Given that the analyses reported for the current study were always completed on the entire cohort, this reduces the impact of this limitation. In addition, our focus was on symptoms and not diagnostic groups, the symptom profile across the cohort being homogeneous. The sample size was small for our test–retest and interrater reliability components. Ideally, we would have preferred to assess more participants, but funding and time restrictions limited these reliability assessments to 10% of the cohort. Finally, the current project was only able to recruit a psychiatric sample, as well as a nonclinical sample. This limits the generalizability of the psychometrics to alternate groups. Further work will need to confirm whether the QPE has the same favorable psychometrics properties in neurological samples and those with a medical disorder.
In conclusion, the English version of the QPE is a psychometrically sound measure of psychotic experiences. It offers a comprehensive phenomenological assessment, which includes hallucinations across several modalities and delusions across multiple themes. It was easy to administer and well-accepted by participants. The QPE fulfils a need that the research and clinical communities have for a transdiagnostic assessment of psychotic experiences. In addition, the QPE could be used as an outcome measure for intervention research.
Funding
National Health and Medical Research Council (NHMRC) project grant (APP1060664 to Professor Susan Rossell and A/Professor Neil Thomas); a project grant from The Barbara Dicker Brain Sciences Foundation (to Professor Susan Rossell, Dr Wei Lin Toh, and A/Professor Neil Thomas).
Supplementary Material
Acknowledgments
The authors would like to thank Monique Scott and Michelle Robertson for their help with the data collected for this article. In addition, the authors would like to thank Monash Alfred Psychiatry Research Centre for their support during data collection. The Questionnaire for Psychotic Experiences is free for use in all public health and nonprofit organizations, and can be administered online or downloaded from www.qpeinterview.com/en. The authors have declared that there are no conflicts of interest in relation to the subject of this study.
References
- 1. Sommer IE, Koops S, Blom JD. Comparison of auditory hallucinations across different disorders and syndromes. Neuropsychiatry. 2012;2:1–12. [Google Scholar]
- 2. McCarthy-Jones S, Thomas N, Strauss C, et al. . Better than mermaids and stray dogs? Subtyping auditory verbal hallucinations and its implications for research and practice. Schizophr Bull. 2014;40 (suppl 4):S275–S284. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Burghaus L, Eggers C, Timmermann L, Fink GR, Diederich NJ. Hallucinations in neurodegenerative diseases. CNS Neurosci Ther. 2012;18:149–159. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Merrett Z, Rossell SL, Castle DJ. Comparing the experience of voices in borderline personality disorder with the experience of voices in a psychotic disorder: a systematic review. Aust N Z J Psychiatry. 2016;50:640–648. [DOI] [PubMed] [Google Scholar]
- 5. Toh WL, Thomas N, Rossell SL. Auditory verbal hallucinations in bipolar disorder (BD) and major depressive disorder (MDD): a systematic review. J Affect Disord. 2015;184:18–28. [DOI] [PubMed] [Google Scholar]
- 6. Haddock G, McCarron J, Tarrier N, Faragher EB. Scales to measure dimensions of hallucinations and delusions: the psychotic symptom rating scales (PSYRATS). Psychol Med. 1999;29:879–889. [DOI] [PubMed] [Google Scholar]
- 7. Green CE, Freeman D, Kuipers E, et al. . Measuring ideas of persecution and social reference: the Green et al. Paranoid Thought Scales (GPTS). Psychol Med. 2008;38:101–111. [DOI] [PubMed] [Google Scholar]
- 8. Kay SR, Fiszbein A, Opler LA. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr Bull. 1987;13:261–276. [DOI] [PubMed] [Google Scholar]
- 9. Castelnovo A, Cavallotti S, Gambini O, D’Agostino A. Post-bereavement hallucinatory experiences: a critical overview of population and clinical studies. J Affect Disord. 2015;186:266–274. [DOI] [PubMed] [Google Scholar]
- 10. Mosimann UP, Collerton D, Dudley R, et al. . A semi-structured interview to assess visual hallucinations in older people. Int J Geriatr Psychiatry. 2008;23:712–718. [DOI] [PubMed] [Google Scholar]
- 11. Andreasen NC. Scale for the Assessment of Positive Symptoms (SAPS). Iowa City, Iowa: Department of Psychiatry, College of Medicine, University of Iowa; 1994. [Google Scholar]
- 12. Bentall RP, Slade PD. Reliability of a scale measuring disposition towards hallucination: a brief report. Pers Individ Diff. 1985;6:527–529. [Google Scholar]
- 13. Bell V, Halligan PW, Ellis HD. The Cardiff Anomalous Perceptions Scale (CAPS): a new validated measure of anomalous perceptual experience. Schizophr Bull. 2006;32:366–377. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Wing JK, Babor T, Brugha T, et al. . SCAN. Schedules for clinical assessment in neuropsychiatry. Arch Gen Psychiatry. 1990;47:589–593. [DOI] [PubMed] [Google Scholar]
- 15. Konings M, Bak M, Hanssen M, van Os J, Krabbendam L. Validity and reliability of the CAPE: a self-report instrument for the measurement of psychotic experiences in the general population. Acta Psychiatr Scand. 2006;114:55–61. [DOI] [PubMed] [Google Scholar]
- 16. Peters ER, Joseph SA, Garety PA. Measurement of delusional ideation in the normal population: introducing the PDI (Peters et al. Delusions Inventory). Schizophr Bull. 1999;25:553–576. [DOI] [PubMed] [Google Scholar]
- 17. Beck AT, Ward CH, Mendelson M, Mock J, Erbaugh J. An inventory for measuring depression. Arch Gen Psychiatry. 1961;4:561–571. [DOI] [PubMed] [Google Scholar]
- 18. Beck AT, Epstein N, Brown G, Steer RA. An inventory for measuring clinical anxiety: psychometric properties. J Consult Clin Psychol. 1988;56:893–897. [DOI] [PubMed] [Google Scholar]
- 19. Sheehan DV, Lecrubier Y, Sheehan KH, et al. . The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998;59 (suppl 20):22–33;quiz 34. [PubMed] [Google Scholar]
- 20. World Medical Association. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA. 2013;310:2191–2194. [DOI] [PubMed] [Google Scholar]
- 21. Mokkink LB, Terwee CB, Patrick DL, et al. . The COSMIN checklist for assessing the methological quality of studies on measurement properties of healthy status measurement instruments: an international Delphi study. Qual Life Res. 2010;19:539–549. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Terwee CB, Bot SD, de Boer MR, et al. . Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60:34–42. [DOI] [PubMed] [Google Scholar]
- 23. Bonferroni CE. Teoria statistica delle classi e calcolo delle probabilit `a. Pubblicazioni del R Istituto Superiore di Scienze Economiche e Commerciali di Firenze. 1936;8:3–62. [Google Scholar]
- 24. Bebbington P, Freeman D. Transdiagnostic extension of delusions: schizophrenia and beyond. Schizophr Bull. 2017;43:273–282. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.