Abstract
Introduction:
The Structured Interview of Psychosis Risk Syndromes (SIPS) was created to identify patients with Clinical High Risk for psychosis (CHR). This study aimed i) to translate and validate the Scale of Prodromal Syndromes (SOPS) in Turkish adolescents, ii) to explore the factor structure of the SIPS/SOPS in the adolescent population, especially focusing on those under the age of 15, iii) to generate a brief version of SIPS (SIPS-B).
Methods:
A total of 150 adolescents aged between 12 and 18 years, were consecutively interviewed using SIPS/SOPS. Patients with psychotic syndrome (n=20), psychosis risk syndrome (PRS) (n=59), and clinical controls (CC) (n=71) were included in the study.
Results:
Principal component analysis (PCA) yielded three latent factors, explaining 62.7% of the total variance in the whole clinical sample, including positive symptom factor, disorganized symptom factor, and negative symptom factor. The area under curve calculated in ROC analyses involving PRS and CC supported the four-item form of the SIPS-B (optimal cut-off=12.5, sensitivity=87%, specificity=80%).
Conclusion:
Our study results support the notion that the Turkish translation of SIPS/SOPS meets the reliability and validity criteria in Turkish adolescents. The SIPS-B could aid clinicians in their routine clinical practice to expedite referral procedures.
Keywords: Prodrome, psychosis, reliability, SIPS, Turkish validity
INTRODUCTION
Psychosis spectrum disorders are debilitating psychiatric illnesses leading to an impairment in emotional, cognitive, and social functioning. Psychosis spectrum disorders are often associated with poor outcomes (1). Early recognition and intervention strategies to identify psychosis prodrome have been a focus of research for the last three decades (2).
The definition of Clinical High Risk for psychosis (CHR) includes three different psychosis risk criteria. Attenuated positive syndrome (APS) is characterized by subthreshold symptoms that have emerged or deteriorated within the past year. Brief-limited intermittent psychosis syndrome (BLIPS) refers to full psychotic symptoms that do not meet the criteria of psychotic disorder due to subthreshold frequency and duration. Finally, subjects with family high-risk and reduced functioning meet the criteria of genetic risk and deterioration syndrome (GRDS). These three syndromes are not mutually exclusive, and one might meet the criteria of different psychosis risk syndromes at the same time (3).
Prodromal psychotic syndromes consist of clinical high risk (CHR) or ultra-high risk (UHR) criteria, more commonly seen in help-seeking patients compared to the general population. This difference also encourages research on various sampling strategies (4). Increased awareness regarding CHR led to an increase in self-referral subjects from the general population. Nevertheless, patients with CHR were not generally referred from primary-care and secondary-care institutions in Turkey. Rather, most cases applied with various psychiatric complaints such as depression or anxiety symptoms. Accordingly, increased rates of self-referral individuals in the help-seeking population give rise to a proportional dilution of pretest risk for psychosis, associated with decreased transition rates. There is an unmet need for a practical, timesaving, and reproducible strategy to increase the number of cases referred by clinicians (5).
Highlights
Structured Interview of Psychosis Risk Syndromes (SIPS) is a valid and reliable tool for diagnosing prodrome in Turkish adolescents.
SIPS is the first gold standard interview validated in the Turkish language in this field.
Adolescents with clinical high risk can be diagnosed with prodromal syndromes with SIPS.
SIPS-B that provides rapid screening in clinical conditions is a four-item short version.
CHR frequency is nearly 3.3% (0–12%) in the general population, more common in the child and adolescent age group (2). The recent meta-analysis indicated similar transition rates in children and adolescent samples to those found in adults (6). CHR prevalence reaches 9% under the age of 16, and it was found to be as high as 20.3% in clinical settings (5.5%-38.3%). Also, in the previous literature, individuals who met the criteria of CHR had a 20% probability of developing a psychotic disorder over two years (2).
The most commonly used diagnostic tools to detect three prodromal syndromes (i. e. APS, BLIPS, GRDS), namely Comprehensive Assessment of At-Risk Mental States (CAARMS) and the Structured Interview of Psychosis Risk Syndromes (SIPS) are similar in terms of structure. The predictive validity of these two tools shows that approximately 29–36% of those diagnosed with CHR/UHR syndrome develop full psychosis within 2–3 years. The CAARMS had excellent inter-rater reliability (Intraclass Correlation Coefficients [ICC] in the range of 0.62–0.93 with only one subscale below 0.7); sensitivity was 83% at six months, and specificity was 74% at six months. SIPS indicates excellent inter-rater reliability and predictive validity. The developers of the SIPS found an inter-rater reliability of 93%. The sensitivity of SIPS was 100% at the sixth, 12th, and 24th months; and the specificity was 71%, 74%, and 73% at the sixth, 12th, and 24th months, respectively (2). However, as it is known, SIPS is the unique validated structured psychosis risk interview, examining the subsyndromal and prodromal symptoms of psychosis. Related to this interview, the Scale of Prodromal Symptoms (SOPS) was rated to assess psychotic symptomatology. The SOPS consists of four subscales, including positive symptoms, negative symptoms, disorganized symptoms, and general symptoms, with 19 items (7). Previous studies provided empirical evidence for the factor structure of the SOPS yet, revealed inconsistent results and could not identify disorganization symptoms separately (8–10). Also, across these studies, the age of participants mostly covered late-adolescence and young adulthood (10–12). Considering the importance of the high prevalence of CHR in the adolescent population, it is crucial to show the factor structure of the SOPS to increase the feasibility of SIPS for clinical practice. However, no study specifically has focused on the items that detect CHR for psychosis during the early phases of adolescence. Since we aimed to assess the evidence concerning the applicability of the SIPS for the early– and middle-adolescents under the age of 15 were quite limited, the importance of developing a time-efficient and useful pretest enrichment method has come to the fore.
The prognostic accuracy of the risk calculators has reached quite good accuracy levels, yet, these rates are still sample-dependent (5). Thus, the need to better characterize CHR symptoms in the adolescent sample remains a research topic (13). The European Psychiatric Association suggests CHR criteria should only be used with the utmost care for children and young adolescents, considering the high rates of false positivity (14). Therefore, there is an unmet need for physicians who should effectively screen and identify youth with CHR. The concept of pretest risk enrichment has come to the fore to define subjects who need further evaluation (5). Previous studies also have reported that some sociodemographic variables and recruitment strategies can be used as enrichment strategies (15). The semi-structured interviews to detect prodromal cases require time and trained interviewers, which seems more suitable for selected patients. Accordingly, self-report screening tools have been developed to stratify patients before evaluation procedures. However, there is an ongoing dilemma in identifying patients with CHR, which might be either too late (e.g., after multiple assessments by several health professionals) or too early (e.g., due to the increased rates of self-awareness and self-referrals) (5). Thus, there was no established selection criterion or procedure for the pre-assessment screening of patients with CHR (5).
In line with the growing evidence regarding sampling concerns, improving screening for clinical practice is still needed for adolescents (14). Thus, this study aimed i) to show reliability and validity analyses of SIPS/SOPS in the Turkish community and ii) to examine the psychometric features of SOPS in the adolescent population, especially focusing on those under the age of 15, to develop a fast and probabilistic evaluation strategy in this age group. To this end, we also investigated items that could be a candidate for the pretest risk enrichment to develop a brief version of SIPS (SIPS-B) that could distinguish patients with a psychotic disorder or CHR from clinical controls.
METHODS
Structured Interview of Psychosis Risk Syndromes (SIPS)
The SIPS/SOPS was developed by Miller and McGlashan et al. at Yale University in 1997 (3). The SIPS has been translated into many languages, such as English, Spanish, Italian and Korean (2). The SIPS includes i) a scale for subthreshold psychotic symptoms (i. e. SOPS), ii) the criteria of schizotypal personality disorder, iii) a comprehensive assessment of medical and family history, and iv) a revised version of the Global Assessment of Functioning (GAF).
The SOPS, the major scale of the assessment, has four subscales: positive symptoms, negative symptoms, disorganization symptoms, and general symptoms. These four symptom domains are Likert-type scales ranging between 0=absent, 1=questionably, 2=mild, 3=moderate, 4=moderately severe, 5=severe but not psychotic, 6=severe and psychotic. The questions are structured for the interview, and a detailed description of symptoms is needed for each severity anchor. Although 19 items were rated during the SIPS/SOPS, only five items categorized under the heading of positive symptoms domain are considered for at-risk criteria (3).
Psychosis Risk Syndromes
Brief-limited Intermittent Psychotic Syndrome (BLIPS)
BLIPS is defined by the presence of severe psychotic symptoms (i. e. rated as 6=severe and psychotic level). The symptoms seen in BLIPS appear for a very short time or momentarily. For the diagnosis of BLIPS, the psychotic symptoms must start within the last three months and occur for a few minutes at least once a month. Finally, psychotic symptoms do not cause urgency, and the possibility of psychotic disorders should be ruled out.
Attenuated Psychotic Syndrome (APS)
APS is defined by the presence of subthreshold positive symptoms (i.e., rated between 3=moderate to 5=severe but not psychotic) that have started or worsened within the last 12 months. In addition, these positive symptoms must occur at least once a week in the past month. APS is the most common psychosis-risk syndrome, which consists of the vast majority (85%) of UHR cases (16).
Genetic Risk and Deterioration Syndrome (GRDS)
GRDS requires the combination of genetic risk for psychosis and a recent impairment in functioning. Subjects with GRDS either have a first-degree relative diagnosed with a psychotic disorder or meet the criteria of schizotypal personality disorder. GRDS is only diagnosed when subjects with family high-risk have a %30 reduction in GAF score within the last year (3).
Participants
Individuals aged between 12–18 years, who were seeking help for any mental complaint, were consecutively recruited from a child and adolescent psychiatry outpatient clinic between October 2019 and May 2020. Briefly, Bakirkoy Mazhar Osman Mental Health Hospital is one of the tertiary-care centers for children with psychotic spectrum disorders in Turkey, serving a population of approximately 20 million. Exclusion criteria were intellectual disability, chronic medical, and severe neurological illness that would give rise to psychotic-like symptoms. All patients and their caregivers gave written informed consent to participate in the research. The local ethics committee of the institution approved the study protocol (Protocol number: 28082/347).
Translation Procedures and Prolinguistic Validation
In line with the World Health Organization approaches and the International Test Commission (ITC), the translation was done by two child psychiatrists who know Turkish and English. Whenever needed, cultural adaptions were also made according to Turkish culture. Back-translation was conducted by two medical doctors. Another independent child psychiatrist compared the original text and back-translated form. Discrepancies between the original text and the back-translated version were solved by the authors. Also, an explorative pilot study was conducted prior to the validation study. Ten youth were interviewed to improve the final text. All authors reached a consensus for the final text (17).
Psychometric Assessment
Sociodemographic, illness, and treatment characteristics were collected using a sociodemographic data form. Kiddie Schedule of Affective Disorders and Schizophrenia Present and Lifetime Version (K-SADS-PL) was administered to determine participants’ past and present psychiatric disorders per DSM-5 criteria (18, 19). The Positive and Negative Symptom Scale (PANSS) was administered by child and adolescent psychiatrists (20, 21).
Kiddie Schedule for Affective Disorders and Schizophrenia for School-Aged Children (6–18 Years) – Present and Lifetime Version (DSM-5)
It is a semi-structured interview form developed by Kaufman et al. (1997s) to determine past and current diagnoses of children and adolescents according to DSM-5 criteria (18). The first part of the interview screened the main symptoms of major psychiatric disorders (Rated between 0=no information, 1=not present, 2=subthreshold level, 3=threshold level). In the second part, the symptoms at the threshold level should be further questioned for additional signs/symptoms in the second part. The interview could be implemented to informants, including children, parents, or caregivers. The validity and reliability of the scale in Turkish sample was conducted by Ünal et al. (2018) (19).
Positive and Negative Symptom Scale (PANSS)
PANSS consists of a total of 30 items and three subscales, including positive symptoms (7 items), negative symptoms (7 items), and general psychopathology symptoms (16 items). Items are scored from 1=absent to 7=extreme, according to the symptom severity. The total PANSS score is calculated with the total scores of positive, negative, and general psychopathology scales. The scale was developed by Kay et al., and the Turkish validation study was conducted by Kostakoglu et al. (20, 21).
Clinical Procedures
The SIPS/SOPS involves a semi-structured interview and a prodromal symptom rating scale, implemented along with the modified Global Assessment of Functioning (GAF) (22) and the criteria of schizotypal personality disorder (23). The SOPS consists of four subscales to measure positive, negative, disorganized, and general psychopathology symptoms, which might occur during the early phases of psychotic illness. The Turkish version of SIPS was carried out by adequately trained child psychiatrists in our study. All raters were trained and supervised by an experienced interviewer (last author). According to the SIPS interview performed with participants and their families, patients were included in one of the following three diagnostic subgroups (7).
Group 1-Psychotic syndrome (PS): The PS subgroup consists of patients who received 6 points from any positive symptom item and met the impairment criteria for the presence of psychotic symptoms (POPS) according to the SIPS interview. These individuals were also diagnosed with a DSM-defined psychotic disorder according to K-SADS-PL (i. e. schizophrenia, schizophreniform disorder, mania with psychotic features, depression with psychotic features, schizoaffective disorder according to DSM criteria).
Group 2-Psychosis Risk Syndrome (PRS): According to the SIPS interview, PRS involves at least one of the CHR diagnoses; brief intermittent psychotic syndrome (BLIPS), attenuated positive symptom syndrome (APS), genetic risk and deterioration syndrome (GRD).
Group 3-Clinical Control Group (CC): Patients who were seeking help for a psychiatric disorder but did not meet the criteria of the first or second group.
Statistical Analysis
Reliability and validity analyses
The total sample size was determined as 150 participants by reviewing the previous literature (24). The chi-square test was used to compare categorical variables, and the ANOVA test was used to compare continuous variables. Cronbach’s alpha coefficients were calculated to show the internal consistency of SOPS. For interrater reliability, ICC were measured. Also, correlations with total PANSS subscale scores were used for convergent validity. The construct validity of the SOPS was evaluated by Explanatory factor analysis (EFA). EFA was implemented using Principal component analysis (PCA), with direct oblimin rotation since correlations between factors were high (25). Keiser-Meyer-Olkin (KMO) sampling adequacy criterion and Bartlett test were also performed.
The development of SIPS-B
The highest items were determined by evaluating the Area Under Curve (AUC) for each item to decide on SIPS-B items. As an indicator of overall predictive validity, AUC=0.5 was considered to be an indicator of random prediction, and the minimum acceptable value was considered 0.75 (26). The Receiver Operating Curves (ROC) analysis was performed to distinguish subjects with PS or PRS from CCs. A subgroup analysis was also conducted for those aged under 15 years. The items with the highest loadings in positive symptom dimensions and highest AUC values in repeated ROC analyzes were selected. As a universal and well-known scale, CGI-Severity (CGI-S) was added to enrich the screening with the clinician’s opinion (27).
The brief interview’s sensitivity and specificity were also tested using ROC curves. It has been reported in previous studies that the minimum sensitivity value for screening scales should be between 70% and 80% (28). A lower threshold for specificity was set at 80% similar to other prevention tests in clinical medicine to reduce over-referral.
Additionally, the predictive validity of the total SIPS-B score was evaluated by using binary logistic regression analysis to distinguish subjects with a syndromal or prodromal psychosis spectrum diagnosis (PS+or PRS+) from CCs. The level of statistical significance was considered as p<0.05. IBM Statistical Package for Social Sciences (SPSS version 25 Armonk, New York, USA) and SPSS AMOS statistical package program were used for data analysis.
RESULTS
The study included 150 adolescents from the outpatient unit. The case group consisted of 79 adolescents (52.7%) diagnosed with a full PS (n=20, 13.3%) or a PRS (n=59, 39.3%), and the CC group consisted of 71 adolescents (47.3%) without PS and/or PRS. The mean age of the adolescents was 14.7±1.6 years, with no significant difference among study groups (p=0.1) (Table 1). Sixty-eight participants were under the age of 15, two of whom (2.9%) were in the PS group; 27 (39.7%) were in the PRS group, and 39 (57.3%) were in the CC group (F=1.588, p=0.212). There were 69 males (45.7%) and 81 females (53.6%) in the sample without any significant difference among study groups (p=0.49).
Table 1.
Demographic and clinical data of subjects of the study
| Study sample, n=150 | n (%) or mean ± SD |
|---|---|
| Age (years), mean ± SD | 14.7±1.6 |
| Patients under the age of 15, n (%) | 68 (45.3) |
| Sex, n (%) | |
| Male n (%) | 69 (45.7) |
| Female n (%) | 81 (53.6) |
| Education, n (%) | |
| None | 21 (14) |
| Primary school | 1 (0.7) |
| Secondary school | 50 (33.3) |
| High school | 78 (52) |
| Research diagnosis, n (%) | |
| PS | 20 (13.3) |
| PRS | 59 (39.3) |
| CC | 71 (47.3) |
| DSM-5 diagnoses, n (%) | |
| Mood disorders | 67 (44.7) |
| Anxiety disorders | 107 (71.3) |
| Obsessive-compulsive disorders | 20 (13.3) |
| Attention-deficit hyperactivity disorders | 63 (42.0) |
| Post-traumatic stress disorder | 10 (6.7) |
| Other | 60 (40) |
| Medications, n (%) | |
| Antidepressants | 55 (36.7) |
| Antipsychotics | 44 (29.3) |
| Sedatives/Hypnotics | 28 (18.7) |
| Mood stabilizers | 6 (4) |
| Other | 25 (16.8) |
| Clinical Assessment, mean ± SD | |
| The total score of SOPS | 45.78±21.91 |
| The total score of PANSS | 77.32±24.32 |
| Global assessment of functioning | 45.89±19.27 |
| Beck Depression Inventory | 19.77±12.99 |
CC, clinical controls; DSM-5, the Diagnostic and Statistical Manual of Mental Disorders; GAF, global assessment of functioning; PANSS, the Positive and Negative Symptoms Scale; PRS, psychosis risk syndrome; PS, psychotic syndrome; SD, standard deviation; SOPS, the Scale of Prodromal Symptoms.
Reliability and Validity Results
The mean item score of the whole sample was 2.4±1.1, and the distribution of SOPS items in the whole sample was demonstrated in Table 2. All items were significantly different among the three groups (p<0.001). In posthoc comparisons, there was no significant difference between the PS and PRS groups in terms of Perceptual Abnormalities/Hallucinations (P4), Social Anhedonia (N1), Avolition (N2), Experience of Emotions/Self (N4), Sleep Disturbance (G1), Dysphoric Mood (G2), Motor Disturbances (G3), and Impaired Tolerance to Normal Stress (G4) items (Table 2).
Table 2.
SOPS item means and standard deviations in the whole sample and PS, PRS, CC groups
| PS, n=20 | PRS, n=59 | CC, n=71 | p value* | Post hoc** | |
|---|---|---|---|---|---|
| Positive symptoms, mean ± SD | |||||
| P1. Unusual Thought Content/Delusional Ideas | 5.7±0.9 | 4.3±1.4 | 2.5±1.5 | <0.001 | 1>2>3 |
| P2. Suspiciousness/Persecutory Ideas | 5.1±1.7 | 3.4±1.4 | 1.5±1.3 | <0.001 | 1>2>3 |
| P3. Grandiosity | 2.4±2.1 | 1.3±1.7 | 0.4±0.7 | <0.001 | 1>2>3 |
| P4. Perceptual Abnormalities/Hallucinations | 4.8±1.7 | 3.9±1.8 | 1.9±1.5 | <0.001 | 1=2>3 |
| P5. Disorganized Communication | 3.1±2.1 | 1.6±1.6 | 0.6±0.9 | <0.001 | 1>2>3 |
| Negative symptoms, mean ± SD | |||||
| N1. Social Anhedonia | 4.7±1.8 | 3.5±2.1 | 2±1.9 | <0.001 | 1=2>3 |
| N2. Avolition | 4.7±1.5 | 4.1±1.4 | 2.3±1.5 | <0.001 | 1=2>3 |
| N3. Expression of Emotion | 3.9±1.3 | 2.7±1.6 | 1.2±1.5 | <0.001 | 1>2>3 |
| N4. Experience of Emotions and Self | 3.4±1.8 | 3±1.7 | 1.3±1.6 | <0.001 | 1=2>3 |
| N5. Ideational Richness | 3.6±1.1 | 2.3±1.5 | 1.4±1.5 | <0.001 | 1>2>3 |
| N6. Occupational Functioning | 5.2±0.7 | 3.7±1.4 | 2.6±1.5 | <0.001 | 1>2>3 |
| Disorganization Symptoms, mean ± SD | |||||
| D1. Odd Behavior or Appearance | 3.7±1.8 | 1.7±1.5 | 0.5±0.8 | <0.001 | 1>2>3 |
| D2. Bizarre Thinking | 2.8±2 | 1.8±1.7 | 0.3±0.8 | <0.001 | 1>2>3 |
| D3. Trouble with Focus and Attention | 3.6±1.3 | 2.7±0.9 | 2±1.1 | <0.001 | 1>2>3 |
| D4. Personal Hygiene | 3±1.7 | 1.4±1.3 | 0.4±0.8 | <0.001 | 1>2>3 |
| General Symptoms, mean ± SD | |||||
| G1. Sleep Disturbance | 3.5±1.4 | 2.9±1.2 | 1.5±1.3 | <0.001 | 1=2>3 |
| G2. Dysphoric Mood | 4.7±1.3 | 4.5±1.3 | 2.8±1.9 | <0.001 | 1=2>3 |
| G3. Motor Disturbances | 1.9±1.5 | 1.4±1.2 | 0.6±1.1 | <0.001 | 1=2>3 |
| G4. Impaired Tolerance to Normal Stress | 4.9±1.6 | 4.2±1.4 | 2.7±1.7 | <0.001 | 1=2>3 |
CC, clinical controls; PRS, psychosis risk syndrome; PS, psychotic syndrome; SD, standard deviation; SOPS, the Scale of Prodromal Symptoms..
One-Way ANOVA.
Bonferroni’s test.
Cronbach’s α coefficient of the SOPS was excellent (α=0.928). The Cronbach alpha coefficient for the population aged under 15 years was good (α=0.891). Interrater reliability of the SIPS/SOPS was excellent for the whole scale (ICC=0.947), positive symptom subscale (ICC=0.934), negative symptom subscale (ICC=0.976), and general symptoms subscale (ICC=0.915), and the ICC of the disorganized symptom subscale was good (ICC=0.831).
Factor structure of SOPS in the whole sample
Three factors with Eigenvalues exceeding one were obtained without assigning a factor number. Nineteen items of the SOPS explained 62.7% of the total variance as the result of PCA. The items of negative and general subscales were generally in the first dimension (Social Anhedonia [N1], Avolition [N2], Expression of Emotion [N3], Experience of Emotions/Self [N4], Occupational Functioning [N6], Sleep Disturbance [G1], Dysphoric Mood [G2] and Impaired Tolerance to Normal Stress [G4] items) (negative symptom factor, NSF). The items of the positive symptom subscale (Unusual Thought Content/Delusional Ideas [P1], Suspiciousness/Persecutory Ideas [P2], Grandiosity [P3], Perceptual Abnormalities/Hallucinations [P4], Bizarre Thinking [D2], Motor Disturbances [G3]) were mostly placed in the second dimension (positive symptom factor, PSF). Items of disorganized symptoms (Disorganized Communication [P5], Ideational Richness [N5], Odd Behavior or Appearance [D1], Trouble with Focus and Attention [D3], Personal Hygiene [D4]) were located in the third dimension (disorganization symptom factor, DSF).
Factor structure of SOPS in adolescents aged under 15 years
The factor structure of the SOPS in adolescents younger than 15 years revealed a four-factor model, which explains 63.2% of the total variance. Sleep Disturbance (G1), Dysphoric Mood (G2), Impaired Tolerance to Normal Stress (G4), Social Anhedonia (N1), and Occupational Functioning (N6) were in the first dimension (distress); Avolition (N2), Expression of Emotion (N3), Experience of Emotions and Self (N4), Ideational Richness (N5), and Personal Hygiene (D4) were located in the third dimension (negative). Similarly, Unusual Thought Content/Delusional (P1), Suspiciousness/Persecutory Ideas (P2), Grandiosity (P3), Perceptual Abnormalities/Hallucinations (P4), and Bizarre Thinking (D2) composed the second dimension (Positive), Disorganized Communication (P5), Trouble with Focus/Attention (D3), Odd Behavior/Appearance (D1), and Motor Disturbances (G3) were in the fourth dimension (Disorganization) (Table 3).
Table 3.
SOPS principal component analysis in the whole sample and adolescents under 15 years
| Study Sample, n=150 | Patients under 15 years of age, n=68 | ||||||
|---|---|---|---|---|---|---|---|
| SOPS Negative Symptom Factor | SOPS Positive Symptom Factor | SOPS Disorganization Symptom Factor | SOPS Distress Symptom Factor | SOPS Positive Symptom Factor | SOPS Negative Symptom Factor | SOPS Disorganization Symptom Factor | |
| P1. Unusual Thought Content/Delusional Ideas | 0.414 | 0.622 | -0.014 | 0.237 | 0.704 | 0.195 | -0.016 |
| P2. Suspiciousness/Persecutory Ideas | 0.521 | 0.552 | -0.098 | 0.408 | 0.671 | 0.077 | -0.139 |
| P3. Grandiosity | -0.161 | 0.787 | 0.136 | -0.236 | 0.714 | -0.067 | 0.178 |
| P4. Perceptual Abnormalities/Hallucinations | 0.282 | 0.672 | -0.088 | 0.175 | 0.811 | -0.031 | -0.168 |
| P5. Disorganized Communication | -0.075 | 0.321 | 0.671 | -0.204 | 0.267 | 0.276 | 0.584 |
| N1. Social Anhedonia | 0.663 | -0.08 | 0.271 | 0.557 | -0.167 | 0.279 | 0.266 |
| N2. Avolition | 0.744 | -0.43 | 0.227 | 0.479 | 0.02 | 0.521 | 0.113 |
| N3. Expression of Emotion | 0.561 | -0.178 | 0.506 | 0.213 | -0.021 | 0.803 | -0.122 |
| N4. Experience of Emotions and Self | 0.8 | -0.122 | 0.117 | 0.436 | 0.186 | 0.58 | -0.278 |
| N5. Ideational Richness | -0.055 | 0.792 | -0.331 | 0.111 | 0.81 | 0.208 | |
| N6. Occupational Functioning | 0.516 | -0.017 | 0.381 | 0.45 | -0.111 | 0.199 | 0.39 |
| D1. Odd Behavior or Appearance | 0.08 | 0.406 | 0.546 | -0.06 | 0.362 | 0.259 | 0.416 |
| D2. Bizarre Thinking | 0.018 | 0.79 | 0.135 | -0.066 | 0.814 | -0.034 | 0.228 |
| D3. Trouble with Focus and Attention | 0.102 | 0.112 | 0.626 | 0.065 | 0.009 | 0.131 | 0.62 |
| D4. Personal Hygiene | 0.202 | 0.222 | 0.519 | 0.169 | -0.047 | 0.419 | 0.26 |
| G1. Sleep Disturbance | 0.709 | 0.264 | -0.146 | 0.679 | 0.249 | -0.162 | 0.133 |
| G2. Dysphoric Mood | 0.845 | 0.16 | -0.176 | 0.699 | 0.239 | 0.215 | -0.094 |
| G3. Motor Disturbances | -0.067 | 0.432 | 0.268 | 0.286 | 0.098 | -0.312 | 0.633 |
| G4. Impaired Tolerance to Normal Stress | 0.78 | 0.042 | 0.025 | 0.863 | 0.058 | -0.028 | 0.046 |
| KMO (Kaiser-Meyer-Olkin) | 0.913 | 0.787 | |||||
| Bartlett’s Test p value | <0.001 | <0.001 | |||||
SOPS, the Scale of Prodromal Symptoms.
Correlation analysis
Pearson’s correlation coefficients were calculated among the SOPS positive/negative/disorganized symptom factor scores and the subscales scores of PANSS to show the convergent validity of the SOPS symptoms factors. Results showed a positive correlation between scores of the SOPS-PSF and the PANSS-positive symptom subscale (r=0.91); between scores of the SOPS-NSF and the PANSS-negative symptom subscale (r=0.806); between scores of the SOPS-DSF and the PANSS-positive symptom subscale (r=0.67). The SIPS-NSF was also highly correlated with GAF (r=-0.764) and the CGI-S (r=0.750).
Psychometric Features of SIPS-B
Since the discriminative validity of each item between the PRS and CC groups was significant, all items were initially included in the analysis. Table 4 demonstrates AUC values between PRS and CC groups.
Table 4.
AUC values and confidence intervals in the whole sample and adolescents under 15 years for Case vs CC groups and PRS vs CC groups
| Study Sample, n=150 | Patients under 15 years of age, n=68 | |||||||
|---|---|---|---|---|---|---|---|---|
| Case vs CC | PRS vs CC | Case vs CC | PRS vs CC | |||||
| SOPS item | AUC | CI | AUC | CI | AUC | CI | AUC | CI |
| P1. Unusual thought content/delusional ideas | 0.844 | 0.782–0.907 | 0.808 | 0.733–0.883 | 0.823 | 0.721–0.925 | 0.811 | 0.703–0.918 |
| P2. Suspiciousness/persecutory ideas | 0.849 | 0.786–0.912 | 0.826 | 0.753–0.899 | 0.818 | 0.712–0.924 | 0.804 | 0.692–0.916 |
| P3. Grandiosity | 0.671 | 0.585–0.757 | 0.641 | 0.543–0739 | 0.672 | 0.537–0.808 | 0.673 | 0.533–0.812 |
| P4. Perceptual abnormalities/hallucinations | 0.809 | 0.739–0.879 | 0.787 | 0.706–0.867 | 0.777 | 0.661–0.892 | 0.762 | 0.641–0.883 |
| P5. Disorganized communication | 0.718 | 0.636–0.799 | 0.687 | 0.594–0.779 | 0.671 | 0.537–0.805 | 0.700 | 0.566–0.834 |
| N1. Social anhedonia | 0.740 | 0.660–0.819 | 0.704 | 0.613–0.794 | 0.626 | 0.490–0.761 | 0.623 | 0.484–0.762 |
| N2. Avolition | 0.818 | 0.751–0.884 | 0.802 | 0.727–0.878 | 0.737 | 0.615–0.859 | 0.724 | 0.597–0.851 |
| N3. Expression of emotion | 0.781 | 0.707–0.856 | 0.746 | 0.661–0.831 | 0.688 | 0.560–0.816 | 0.670 | 0.536–0.803 |
| N4. Experience of emotions and self | 0.771 | 0.695–0.846 | 0.762 | 0.680–0.845 | 0.735 | 0.613–0.859 | 0.717 | 0.589–0.846 |
| N5. Ideational richness | 0.705 | 0.621–0.789 | 0.657 | 0.563–0.751 | 0.606 | 0.470–0.743 | 0.598 | 0.458–0.739 |
| N6. Occupational functioning | 0.754 | 0.678–0.831 | 0.693 | 0.603–0.783 | 0.612 | 0.474–0.750 | 0.589 | 0.447–0.731 |
| D1. Odd behavior or appearance | 0.773 | 0.699–0.848 | 0.723 | 0.633–0.812 | 0.722 | 0.598–0.846 | 0.724 | 0.596–0.852 |
| D2. Bizarre thinking | 0.800 | 0.728–0.871 | 0.777 | 0.694–0.859 | 0.828 | 0.721–0.935 | 0.838 | 0.731–0.945 |
| D3. Trouble with focus and attention | 0.683 | 0.599–0.767 | 0.646 | 0.553–0.740 | 0.582 | 0.446–0.717 | 0.597 | 0.460–0.734 |
| D4. Personal hygiene | 0.764 | 0.688–0.840 | 0.716 | 0.625–0.806 | 0.673 | 0.543–0.804 | 0.673 | 0.538–0.808 |
| G1. Sleep disturbance | 0.788 | 0.715–0.861 | 0.770 | 0.688–0.852 | 0.735 | 0.615–0.855 | 0.736 | 0.613–0.858 |
| G2. Dysphoric mood | 0.773 | 0.697–0.848 | 0.767 | 0.685–0.848 | 0.718 | 0.594–0.842 | 0.697 | 0.568–0.826 |
| G3. Motor disturbances | 0.697 | 0.612–0.781 | 0.679 | 0.585–0.773 | 0.735 | 0.610–0.859 | 0.749 | 0.623–0.875 |
| G4. Impaired tolerance to normal stress | 0.763 | 0.686–0.839 | 0.739 | 0.653–0.824 | 0.715 | 0.595–0.836 | 0.705 | 0.581–0.829 |
AUC, area under curve; CC, clinical controls; PRS, psychosis risk syndrome; PS, psychotic syndrome, SOPS, the scale of prodromal symptoms.
First, eight items as Unusual Thought Content/Delusional Ideas (P1), Suspiciousness/Persecutory Ideas (P2), Perceptual Abnormalities/Hallucinations (P4), Avolition (N2), Experience of Emotions and Self (N4), Bizarre Thinking (D2), Sleep Disturbance (G1), and Dysphoric Mood (G2) yielded an AUC >0.75.
Secondly, items with the highest loadings in PCA and highest AUC values to distinguish subjects with PRS from CCs were selected for SIPS-B. Items included in the SIPS-B were Unusual Thought Content/Delusional Ideas (P1), Suspiciousness/Persecutory Ideas (P2), Perceptual Abnormalities/Hallucinations (P4), Bizarre Thinking (D2). Finally, the CGI-S score was added to SIPS-B to improve the final version.
The AUC value of the SIPS-B was 0.905 and can be considered excellent with a cut-off of 12 (its sensitivity was 87%, and specificity was 80%). The Cronbach alpha value of the SIPS-B was considered as good (α=0.87) (Figure 1).
Figure 1.
ROC curves of SIPS-B item scores and the total score
The total score of SIPS-B items was evaluated by using binary logistic regression analysis, which was statistically significant [p<0.001, Exp[B]=1.4, 95% Confidence Interval (CI)=1.2–1.5]. The Hosmer and Lemeshow test indicated goodness of fit (α2=2.02, p=0.98). The model explained (Nagelkerke R2=60.6%) a significant portion of the variance in the CC and PRS groups. The odds of getting PRS classification increased 1.4 times for each one-unit increase in the SIPS-B total score.
DISCUSSION
To the best of our knowledge, this study is the first study that evaluates the factor structure and psychometric properties of the SOPS in the adolescent age group, especially among those younger than 15-year-old. Also, the results suggested a shorter evaluation implemented by clinicians, which could be a novel referral strategy. Results suggest the factor structure of the SOPS within the adolescent population yielded good reliability and validity.
In our study, three sub-dimensions (i. e. positive, negative, and disorganization factors) were obtained in parallel with the current knowledge about phenomenology. 62.7% of the total variance was explained in the model, and this structure was appropriate for the adolescent population. Also, we demonstrated a more homogenous factor structure compared to some of the previous research (8, 9). We assume this finding was somewhat associated with the sample characteristics. Previous factor analyses conducted in English, Spanish and Italian versions revealed the three-factor solutions as positive, negative, and general symptoms (9, 11, 12). On the other hand, Tso et al. demonstrated a four-dimension solution with the direct oblimin rotation, including positive symptoms, distress, negative symptoms, and deteriorated thought processes. In our study, in addition to obtaining a theoretically favorable structure, the distribution of positive items was relatively homogeneous, quite similar to those demonstrated by Hawkins and colleagues (2002) (9). The homogenous dimension of positive symptoms gains importance, considering CHR criteria only involve the scores obtained from the positive symptom items.
In addition to the first four items classified in the positive symptom index, bizarre thinking (D2) was in the positive symptom factor. This finding is consistent with the previous factor analysis demonstrated by Hawkins et al. (9). Since the CHR criteria are based on P items, bizarre thinking could increase the predictive validity of SOPS. Despite the methodologic concerns to define bizarreness of thought content, given its specificity to detect schizophrenia, bizarre thinking could increase the diagnostic accuracy for the screening criteria (29). Moreover, disorganized symptoms might be mildly present in the psychosis prodrome, and screening these symptoms could help in predicting transition to psychosis (30). On the other side, previous studies indicated that disorganized items yielded a heterogeneous dispersion into other dimensions (9, 10, 12). The disorganization items represented a relatively homogenous distribution and were under the “disorganization symptom factor”. This finding is an important result of our study.
Adolescence and early adulthood differed in terms of both cognitive development and the presentation of schizophrenia (31). Considering that early-onset schizophrenia is a distinct clinical entity regarding its epidemiological, clinical, and etiologic features (1), the assessment of symptomatology should be modified for different age groups, especially for adolescents. A previous study examining the characteristics of psychosis prodrome in children and adolescents suggested non-specific mental complaints occurred a long time before the detectable early prodromal period (32). In some studies, the factor structures of the SOPS and the PANSS in the UHR group involved a symptom dimension, which represents “distress” or “anxiety/depression” (8, 33). However, in our sample, all negative items except ideational richness, together with some general items were under the same factor supporting a single-component negative symptom structure. Nevertheless, under the age of 15, this unidimensional negative symptom factor was divided into two constructs. Of both, the distress factor included dysphoric mood, social anhedonia, occupational functioning, impaired tolerance to normal stress, and sleep disturbances. This condition was in line with the current knowledge since younger adolescents with CHR may represent non-specific mental complaints before the emergence of specific prodromal signs and symptoms (32). Our study revealed that the SIPS interview is a reliable and valid tool to diagnose CHR for those older and younger than 15 years. In the light of these findings, it can be argued that prodromal symptomatology varies according to age. Thus, revising the current tools for different age categories may increase the diagnostic accuracy rather than applying the same assessment to all age groups.
The European Psychiatric Association guideline recommends using three assessment methods i) CAARMS, ii) SIPS/SOPS, iii) the assessment of basic symptoms, for the early detection of psychosis (14). The SIPS-B is aimed to be a candidate for a time-saving, practical guide for routine clinical practice to detect patients who require further evaluation for a CHR diagnosis in the help-seeking population. Accordingly, results also supported using the SIPS-B to differentiate psychosis prodrome from clinical controls with high sensitivity (87%) and specificity (80%), even among early adolescents. Although a previous meta-analysis has reported that the assessment of UHR criteria may vary according to sampling; therefore, the risk assessment depends on different recruitment strategies (5). This uncertainty regarding referral criteria prevents the standardization of the studies, and the “help-seeking population” is not clearly defined. Also, the application of the gold standard test to “everyone” from the community decreases the time– and cost-effectivity. Fusar-Poli et al. indicated the proportion of self-referrals shows an inverse correlation with pretest risk of psychosis as well as a decrease in the rate of subjects from the health-services referrals in the help-seeking patients, which leads to a decreased prognostic accuracy as well as the transition risk (5). Especially in adolescents, the presence of standardized criteria for referral becomes more crucial due to the low pretest risk and transition rates in this population (13). Therefore, the SIPS-B may provide a more generalizable and feasible procedure to increase pretest risk enrichment which might be a solution for the sampling problem in the adolescent age group. Accordingly, the SIPS-B would be a time-saving strategy for clinicians in routine clinical practice.
Limitations
Several limitations have to be acknowledged to interpret our results. Since the study is cross-sectional, it was not possible to perform a predictive validity analysis. Therefore, our results also require an independent validation for predictive validity. Also, SIPS-B was investigated in the adolescent group, and psychometric properties in the adult population were not examined. Participants were help-seeking patients, some of whom were referred from primary or secondary centers. Accordingly, our sample did not cover community settings. Nevertheless, considering the need for a fast screening tool as an enrichment strategy for outpatient clinics, the preliminary results of our study could pave the way to implement SIPS-B in clinical practice.
CONCLUSION
Our study results support the notion that the Turkish translation of SIPS/SOPS meets the reliability and validity criteria in Turkish adolescents. Proposed SIPS-B items might be a practical and time-effective solution for sampling problems and may provide a pretest risk enrichment to enhance the prognostic accuracy. Prospective studies with larger samples are needed to re-evaluate the effectiveness of SIPS-B administered in routine clinical practice.
Footnotes
Ethics Committee Approval: Local ethics committee of Bakirkoy Prof Dr. Mazhar Osman Mental Health and Disorders Training and Research Hospital approved the protocol (Protocol number: 28082/347, Date: 03.09.2019).
Informed Consent: All patients and their caregivers gave written informed consent to participate in the research.
Peer-review: Externally peer-reviewed.
Author Contributions: Concept- AT, GK, AK, EE, ÖU, OÇ, BÖ; Design- AT, GK, AK, EE, UK, BÖ; Supervision- AT, GK, BÖ; Resource- AT, GK; Materials- AT, GK, BÖ; Data Collection and/or Processing- AT, GK, AK, EE, UK, ÖU, OÇ, BÖ, ÇE; Analysis and/or Interpretation- AT, GK, AK, EE, UK, ÖU, OÇ, BÖ, ÇE; Literature Search- AT, GK, OÇ, BÖ, ÇE; Writing- AT, GK, BÖ, ÇE; Critical Reviews- AT, GK, AK, EE, UK, ÖU, OÇ, BÖ, ÇE.
Conflict of Interest: No.
Financial Disclosure: CE received a grant from the Scientific and Technological Research Council of Turkey (TÜBİTAK) – Directorate of Science Fellowships and Grant Programmes (BIDEB) (2214A – International Research Fellowship Programme, Grant No: 1059B141800278).
REFERENCES
- 1.Rey J International Association for Child and Adolescent Psychiatry and Allied Professions. IACAPAP textbook of child and adolescent mental health. Geneva: International Association for Child and Adolescent Psychiatry and Allied Professions; 2012. [Google Scholar]
- 2.Woods SW, Walsh BC, Powers AR, McGlashan TH. Reliability, validity, epidemiology, and cultural variation of the Structured Interview for Psychosis-Risk Syndromes (SIPS) and the Scale of Psychosis-Risk Symptoms (SOPS) In: Li H, Shapiro DI, Seidman LJ, editors. Handbook of Attenuated Psychosis Syndrome Across Cultures. Springer; 2019. pp. 85–113. [Google Scholar]
- 3.McGlashan TH, Johannessen JO. Early detection and intervention with schizophrenia:rationale. Schizophr Bull. 1996;22:201–222. doi: 10.1093/schbul/22.2.201. [DOI] [PubMed] [Google Scholar]
- 4.Fusar-Poli P. Why ultra high risk criteria for psychosis prediction do not work well outside clinical samples and what to do about it. World Psychiatry. 2017;16:212–213. doi: 10.1002/wps.20405. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Fusar-Poli P, Schultze-Lutter F, Cappucciati M, Rutigliano G, Bonoldi I, Stahl D, et al. The dark side of the moon:meta-analytical impact of recruitment strategies on risk enrichment in the clinical high risk state for psychosis. Schizophr Bull. 2016;42:732–743. doi: 10.1093/schbul/sbv162. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Raballo A, Poletti M, Preti A, McGorry P. Clinical high risk for psychosis in children and adolescents:A meta-analysis of transition prevalences. Schizophr Res. 2020 doi: 10.1016/j.schres.2020.03.063. S0920-9964(20)30174-2. [DOI] [PubMed] [Google Scholar]
- 7.McGlashan TH, Walsh B, Woods S. The Psychosis-Risk Syndrome Handbook for Diagnosis and Follow-up. 1st ed. US: Oxford University Press; 2010. [Google Scholar]
- 8.Tso IF, Taylor SF, Grove TB, Niendam T, Adelsheim S, Auther A, et al. Factor analysis of the Scale of Prodromal Symptoms:data from the Early Detection and Intervention for the Prevention of Psychosis Program. Early Interv Psychiatry. 2017;11:14–22. doi: 10.1111/eip.12209. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Hawkins KA, McGlashan TH, Quinlan D, Miller TJ, Perkins DO, Zipursky RB, Addington J, et al. Factorial structure of the Scale of Prodromal Symptoms. Schizophr Res. 2004;68:339–347. doi: 10.1016/S0920-9964(03)00053-7. [DOI] [PubMed] [Google Scholar]
- 10.Jung MH, Jang JH, Kang D-H, Choi J-S, Shin NY, Kim HS, et al. The reliability and validity of the Korean version of the structured interview for prodromal syndrome. Psychiatry Investig. 2010;7:257–263. doi: 10.4306/pi.2010.7.4.257. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Comparelli A, Savoja V, Kotzalidis G, Woods S, Mosticoni S, Vassallo F, et al. Factor-structure of the Italian version of the Scale Of Prodromal Symptoms (SOPS):a comparison with the English version. Epidemiol Psychiatr Sci. 2011;20:45–54. doi: 10.1017/s2045796011000114. [DOI] [PubMed] [Google Scholar]
- 12.Fernández P, Ortega J, García P, Gutiérrez A, García A, Bobes J, et al. Predictive validity of the Scale of Prodromal Symptoms (SOPS) Actas Esp Psiquiatr. 2006;34:216–223. https: //pubmed.ncbi.nlm.nih.gov/16823681/ [PubMed] [Google Scholar]
- 13.Schimmelmann BG, Michel C, Martz-Irngartinger A, Linder C, Schultze-Lutter F. Age matters in the prevalence and clinical significance of ultra-high-risk for psychosis symptoms and criteria in the general population:Findings from the BEAR and BEARS-kid studies. World Psychiatry. 2015;14:189–197. doi: 10.1002/wps.20216. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Schultze-Lutter F, Michel C, Schmidt SJ, Schimmelmann BG, Maric NP, Salokangas RK, et al. EPA guidance on the early detection of clinical high risk states of psychoses. Eur Psychiatry. 2015;30:405–416. doi: 10.1016/j.eurpsy.2015.01.010. [DOI] [PubMed] [Google Scholar]
- 15.Fusar-Poli P, Rutigliano G, Stahl D, Schmidt A, Ramella-Cravaro V, Hitesh S, et al. Deconstructing Pretest Risk Enrichment to Optimize Prediction of Psychosis in Individuals at Clinical High Risk. JAMA Psychiatry. 2016;73:1260–1267. doi: 10.1001/jamapsychiatry.2016.2707. [DOI] [PubMed] [Google Scholar]
- 16.Fusar-Poli P, Raballo A, Parnas J. What Is an Attenuated Psychotic Symptom?On the Importance of the Context. Schizophr Bull. 2017;43:687–692. doi: 10.1093/schbul/sbw182. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.International Test Commission. The ITC Guidelines on Computer-Based Testing. https: //www.intestcom.org/page/16# .
- 18.Kaufman J, Birmaher B, Brent D, Rao U, Flynn C, Moreci P, et al. Schedule for affective disorders and schizophrenia for school-age children-present and lifetime version (K-SADS-PL):initial reliability and validity data. J Am Acad Child Adolesc Psychiatry. 1997;36:980–988. doi: 10.1097/00004583-199707000-00021. [DOI] [PubMed] [Google Scholar]
- 19.Ünal F, Öktem F, Çetin Çuhadaroğlu F, Çengel Kültür SE, Akdemir D, Foto Özdemir, et al. Okul ÇağıÇocuklarıIçin Duygulanım BozukluklarıVe Şizofreni Görüşme Çizelgesi-Şimdi Ve Yaşam Boyu Şekli-Dsm-5 Kasım 2016-Türkçe Uyarlamasının (Çdşg-Şy-Dsm-5-T) Geçerlik Ve Güvenirliği. Turk Psikiyatri Dergisi. 2019;30(1) [PubMed] [Google Scholar]
- 20.Kay SR, Fiszbein A, Opler LA. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr Bull. 1987;13:261–276. doi: 10.1093/schbul/13.2.261. [DOI] [PubMed] [Google Scholar]
- 21.Kostakoğlu A, Batur S, Tiryaki A, Göğüş A. Pozitif ve negatif sendrom ölçeğinin (PANSS) Türkçe uyarlamasının geçerlilik ve güvenilirliği. Türk Psikoloji Derg. 1999;14:23–32. https: //toad.halileksi.net/sites/default/files/pdf/pozitif-ve-negatif-sendrom-olcegi-toad.pdf . [Google Scholar]
- 22.Hall RC. Global assessment of functioning:a modified scale. Psychosomatics. 1995;36:267–275. doi: 10.1016/S0033-3182(95)71666-8. [DOI] [PubMed] [Google Scholar]
- 23.Association AP. Diagnostic and Statistical Manual of Mental Disorders (DSM-5®) 5th ed. USA: American Psychiatric Publishing; 2013. https: //www.psychiatry.org/psychiatrists/practice/dsm . [Google Scholar]
- 24.Anthoine E, Moret L, Regnault A, Sébille V, Hardouin J-B. Sample size used to validate a scale:a review of publications on newly-developed patient reported outcomes measures. Health Qual Life Outcomes. 2014;12:176. doi: 10.1186/s12955-014-0176-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Tabachnick BG, Fidell LS, Ullman JB. Using multivariate statistics. 7th ed. MA: Pearson Boston; 2007. https: //www.pearsonhighered.com/assets/preface/0/1/3/4/0134790545.pdf . [Google Scholar]
- 26.Douglas KS, Guy LS, Reeves KA, Weir J. HCR-20 violence risk assessment scheme:Overview and annotated bibliography. Implementation Science and Practice Advances Research Center Publications. 2005 [Google Scholar]
- 27.Busner J, Targum SD. The clinical global impressions scale:applying a research tool in clinical practice. Psychiatry (Edgmont) 2007;4:28–37. https: //pubmed.ncbi.nlm.nih.gov/20526405/ [PMC free article] [PubMed] [Google Scholar]
- 28.Power M, Fell G, Wright M. Principles for high-quality, high-value testing. Evid Based Med. 2013;18:5–10. doi: 10.1136/eb-2012-100645. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Cermolacce M, Sass L, Parnas J. What is bizarre in bizarre delusions?A critical review. Schizophr Bull. 2010;36:667–679. doi: 10.1093/schbul/sbq001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Demjaha A, Valmaggia L, Stahl D, Byrne M, McGuire P. Disorganization/Cognitive and Negative Symptom Dimensions in the At-Risk Mental State Predict Subsequent Transition to Psychosis. Schizophr Bull. 2010;38:351–359. doi: 10.1093/schbul/sbq088. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Marshall C, Lu Y, Lyngberg K, Deighton S, Cadenhead KS, Cannon TD, et al. Changes in symptom content from a clinical high-risk state to conversion to psychosis. Early Interv Psychiatry. 2019;13:257–263. doi: 10.1111/eip.12473. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Schultze-Lutter F, Ruhrmann S, Berning J, Maier W, Klosterkötter J. Basic Symptoms and Ultrahigh Risk Criteria:Symptom Development in the Initial Prodromal State. Schizophr Bull. 2008;36:182–191. doi: 10.1093/schbul/sbn072. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Emsley R, Rabinowitz J, Torreman M RIS-INT-35 Early Psychosis Global Working Group. The factor structure for the Positive and Negative Syndrome Scale (PANSS) in recent-onset psychosis. Schizophr Res. 2003;61:47–57. doi: 10.1016/s0920-9964(02)00302-x. [DOI] [PubMed] [Google Scholar]