Abstract
Background
Paranoia—incorrectly thinking that others are deliberating trying to harm you—causes distress, undermines social interactions and leads to withdrawal. It presents across multiple psychiatric diagnoses.
Objective
The primary aim was to determine the extent that cognitive and social processes may explain paranoia. The secondary aim was to identify explanatory factors that distinguished paranoia and social anxiety.
Methods
10 382 UK adults, quota sampled to match the population for age, gender, ethnicity, income and region, participated in a non-probability survey. All participants completed a paranoia measure and assessments of cognitive and social processes. Structural equation modelling was conducted.
Findings
2586 (24.9%) participants described being mistrustful of other people. 1756 (16.9%) participants wanted help to trust more. 66.7% of variance in paranoia was explained by a model comprising (in descending order of importance): within-situation defence behaviours, negative images, negative self-beliefs, discrimination, dissociation, aberrant salience, anxiety sensitivity, agoraphobic distress, worry, less social support, agoraphobic avoidance, less analytical reasoning and alcohol use. All explanatory factors were associated with paranoia and social anxiety. Ten factors were more closely associated with paranoia than social anxiety, including discrimination, hallucinations, negative images, aberrant salience and alcohol use. Nine factors were more closely associated with social anxiety, including less positive self-belief, an external locus of control, worry and less analytical reasoning.
Conclusions
Multiple causes are likely to be involved in paranoia. Cognitive and social processes may explain a high degree of paranoia.
Clinical implications
Multiple clear targets for intervention to reduce paranoia are identified.
Keywords: Schizophrenia & psychotic disorders, Anxiety disorders
WHAT IS ALREADY KNOWN ON THIS TOPIC.
A number of social and cognitive factors have been linked with paranoia but they have not been studied together in a large representative population.
WHAT THIS STUDY ADDS
This is the most comprehensive investigation of paranoia yet conducted. Multiple causes are likely to be involved in paranoia. It also helps explain what may particularly lead to paranoia and what may lead to the related but distinct issue of social anxiety.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
This research identifies key factors that can be used to help formulate a patient’s presentation. Treatment targets for paranoia are identified.
Introduction
Paranoia is excessive mistrust or suspicion of people -thinking incorrectly, for example, that someone is trying to hurt you, that people are doing things to deliberately annoy you or that there is a conspiracy against you. Its severest form is persecutory delusion where there is a high degree of belief in the thoughts, distress and disruption. Persecutory delusions are a key symptom of psychotic diagnoses such as schizophrenia but paranoid thinking is also higher in patients with common mental health disorders such as anxiety and depression.1–3 Just as for emotional difficulties there is a spectrum of severity of paranoia in the general population.4 Many people have a few paranoid thoughts; a few people have many paranoid thoughts.5 Mistrust of other people brings a wariness to social interactions and in its wake many difficulties. Paranoia has increasingly become a topic of study in its own right, especially from a social and psychological perspective. In this paper, we bring together the learning from our own work on paranoia to test the extent to which paranoia may now be explained.
The theoretical understanding of paranoia
Our cognitive model of paranoia was used to guide the factors chosen to be assessed in the survey.6 It is a multifactorial view of causation. Each cause is considered an ‘inus condition’7—‘an insufficient but non-redundant part of an unnecessary but sufficient condition’. A single factor only increases the probability of paranoia occurring. It is hypothesised that paranoia—inaccurate threat cognitions—builds on feelings of vulnerability. This includes social anxiety-related fears of rejection8 and negative self-beliefs9 that lead to a sense of inferiority. Negative images or memories,10 and a general sense of less control over events,11 may further feed the sense of vulnerability. Positive self-beliefs may be weak and unable to counteract the sense of vulnerability.12 Importantly, fears of harm are elaborated and consolidated by the ruminative process of worry.13 Odd internal sensations and perceptions—such as unexplained anxiety symptoms, dissociation, aberrant salience and hallucinations—provoke fearful explanations.14 15 A person may react to the fears by protective defensive actions, safety-seeking behaviours, that lock the fears into place by preventing the processing of disconfirmatory evidence.16 These can include avoidance of feared places but also more subtle strategies used when within situations of perceived threat (eg, vigilance, keeping a low profile, leaving as quickly as possible). Less use of analytical reasoning will further mean that the ideas remain unchallenged.17 Sleep disruption exacerbates most of these processes such as negative self-beliefs and anomalous experiences.18 Social factors are accounted for in the model by their effects on the cognitive processes. For example, cannabis has been found to increase paranoia by causing exacerbations in negative self-beliefs, worry and the occurrence of anomalous experiences.15 Negative life events, discrimination and bullying will raise the likelihood of paranoia via affecting views of the self.19 20
The study aims
The aim was to test the theoretical model in a large representative group of the general population. We set out to determine the potential association of each cognitive and social factor with paranoia. But we also wished to quantify the extent to which the model explains paranoia and whether there are particularly key individual processes. The focus was on cognitive and social processes that could be of direct benefit in informing the treatment of paranoia. We also wished to inspect sociodemographic associations of paranoia. Further, we wanted to identify factors that may have stronger relationships with the prediction of paranoia than social anxiety. Anxiety and paranoia are closely connected and share many of the same causal factors,21 22 but we wanted to identify factors that may particularly lead to one of those symptom outcomes above another. Previously, we have found in experimental studies that perceptual anomalies such as hallucination-like experiences increase the risk of paranoid thoughts occurring but not social anxiety.21 23 The current study was conducted to serve as a guide to the strengths and weaknesses of our current understanding of paranoia and to inform further development of the Feeling Safe programme for the treatment of persecutory delusions.24
Methods
Participants
An online survey with a quota sampled UK participant group of 10 382 adults (18+ years old) who fully completed all questionnaire items (without speeding) was conducted from 15 to 27 March 2023 via a market research company (Lucid). The quotas were based on UK Office for National Statistics population estimate data for gender, age, ethnicity, income and region (see online supplemental table S42). Only individuals who fully completed the survey were considered as participants, and anyone who completed the survey in less than one-third of the median completion time was removed and not considered a participant.
bmjment-2023-300880supp001.pdf (629.5KB, pdf)
Assessments
All the assessment items included in the survey, the details of the factor analyses, reliability and the items that comprise each factor are provided in the online supplemental materials (supplemental files 1 and 2).
Revised Green et al Paranoid Thoughts Scale (R-GPTS). 25 The R-GPTS comprises an 8-item ideas of reference scale and a 10-item ideas of persecution scale. Each item is rated for the past month on a 5-point scale. Higher scores indicate greater levels of paranoia. For descriptive purposes, we also added eight new ‘Part C’ mistrust items that concerned a person thinking their fears about other people may be exaggerated and that they may like help.
Social Phobia Inventory.26 Seventeen items assessing social anxiety are rated for the past week on a 5-point scale. Higher scores indicate greater levels of social anxiety.
Dunn Worry Questionnaire.27 Ten worry-related items are rated for the past month on a 5-point scale. Higher scores indicate higher levels of worry.
Oxford Positive Self Scale–Short Form.12 Eight positive self-items are rated for the past week on a 5-point scale. Higher scores indicate higher levels of positive self-belief.
Brief Core Schema Scale–Negative Self.28 The 6-item negative subscale was used. Each item is rated on a scale from 0 (do not believe it) to 4 (believe it totally). Higher scores indicate greater endorsement of items.
Global Felt Sense of Anomaly Scale. 29 Five items assess a sense of strangeness experienced over the past fortnight on a 5-point scale. Higher scores indicate higher levels of dissociative experiences.
Cardiff Anomalous Perceptions Scale-Hallucinations (CAPS).30 This scale comprises 11 hallucination items taken from the CAPS. Each item is rated on a 0 (not at all) to 5 (daily) scale. Higher scores indicate greater levels of hallucinatory experiences.
Intrusive Images. Three new items were used, concerning experiencing negative images of self in relation to others. Each item was rated for frequency on a 6-point scale. Higher scores indicate more frequent negative imagery.
Aberrant Salience Inventory.31 Twenty-nine items are rated for whether they have ever been experienced (No/Yes). Higher scores indicate greater levels of aberrant salience experiences.
Anxiety Sensitivity Index. 32 Sixteen items are rated on a 5-point scale. Higher scores indicate higher levels of anxiety sensitivity.
Oxford Agoraphobic Avoidance Scale (O-AS).33 The O-AS lists eight simple tasks progressing in difficulty. Participants are asked whether they could do the task now or whether they could not because of anxiety (avoidance scale) and how much distress each task would cause (distress scale). Higher scores indicate greater agoraphobic symptoms.
Oxford Paranoia Defences Scale. Twenty within-situation safety-seeking behaviours are assessed for their presence over the past fortnight on a 4-point scale. Higher scores indicate greater use of defence behaviours.
Rational Experiential Inventory–Rational Reasoning. 34 The 20-item rational reasoning scale was used. Items are rated on a 5-point Likert scale. Higher scores indicate greater use of rational reasoning.
Rotter’s Locus of Control Scale–Adapted Short Form.35 Four items were rated on a 4-point scale from strongly disagree to strongly agree. Higher scores indicate an external locus of control.
Insomnia Severity Index (ISI). 36 The ISI is a 7-item self-report insomnia questionnaire. Higher scores indicate the presence of symptoms of insomnia.
Alcohol Use Disorders Identification Test.37 Three items rated on 5-point scales assess alcohol consumption. Higher scores indicate higher intake of alcohol.
Cannabis Use Disorder Identification Test (CUDIT).38 The first item from the CUDIT was used to assess how often the person used cannabis on a 5-point scale. Higher scores indicate higher cannabis use.
Adult Psychiatric Morbidity Survey Discrimination Scale. 39 Seven items assess whether a person has experienced various types of discrimination in the past year. Higher scores indicate a greater number of experiences of discrimination.
Stressful Life Events. 39 A list of eight stressful life events was presented and participants answered whether they had experienced each over the past 12 months. Higher scores indicate the occurrence of a greater number of stressful events.
California Bully-Victimization Scale. 40 Eight items assessed bullying when the person was at school (Yes/No). Higher scores indicate a greater experience of bullying.
Childhood Mistreatment.41 Four items assessed childhood (before aged 18 years old) mistreatment on a Yes/No response format. Higher scores indicate higher levels of childhood mistreatment.
Multidimensional Scale of Perceived Social Support. 42 Twelve items concerning social support are rated on a 7-point scale. Higher scores indicate higher levels of perceived social support.
Analysis
The full details of the statistical methods and results are provided in the online supplemental materials. Statistical analyses were conducted in the R software environment for statistical computing and graphics, V.4.2.3. The ‘psych’ package42 was employed to generate descriptive statistics and perform exploratory factor analysis (EFA). The ‘lavaan’ package43 was used to conduct both confirmatory factor analysis (CFA) and structural equation modelling (SEM).
A total of 22 social and cognitive measures were considered. There were no missing data. An EFA was implemented on measures for which no prior EFA had been performed. Prior to EFAs, the Kaiser-Meyer-Olkin measure of sampling adequacy was employed to assess the suitability of factor recovery using the observed dataset.44 Parallel analysis was used to identify the number of factors to retain. CFA was then conducted. Measures that had previously been factor analysed were only evaluated using CFA. We performed higher order CFAs on multidimensional measures that had substantial factor correlations. We used the composite reliability index derived from the estimated factor model parameters as a report of a scale’s internal consistency.45 First, separate SEMs were performed, regressing paranoia (R-GPTS Part B) onto each explanatory factor. Second, we integrated the factors into a unified SEM to account for the correlation between the predictors, with paranoia as the response variable, taking the entire dataset into account. SEM was used since it can provide a more comprehensive and nuanced analysis of data than traditional regression.46 SEM is particularly valuable for investigating complex relationships among variables and dealing with measurement error and issues of multicollinearity. In assessing the relationships of the constructs in the structural model and eliminating irrelevant variables, we employed a backward elimination strategy, specifically targeting items that were either non-significant statistically or demonstrated a suppressor effect.47 Third, the process of evaluating the differential relationships of the explanatory factors to paranoia and social anxiety used separate SEMs for each factor including both paranoia and social anxiety as the response variables. Having both outcomes in a single SEM accounts for the shared variance between paranoia and social anxiety, allowing examination of the relationship between the explanatory variable and each outcome while accounting for the relationship between the outcomes. A Wald test was employed to compare statistically the two regression coefficients of the explanatory variable to paranoia and social anxiety. Finally, we used the sociodemographic information as predictors of paranoia in both separate SEMs and a unified SEM.
Results
A summary of the responses to the paranoia questions is provided in table 1. A total of 1427 (13.7%) participants described themselves as very trusting of other people, 6371 (61.4%) as generally trusting of other people, 2034 (19.6%) as generally mistrustful of other people and 552 (5.3%) as very mistrustful of other people. Seven hundred and eighteen (5.9%) participants reported that they were often more fearful of what other people may do to them than they should, 1501 (14.5%) sometimes felt this, 2155 (20.8%) occasionally felt this and 6008 (57.9%) did not feel this. Levels of persecutory ideation (R-GPTS Part B) were significantly higher (p<0.001) in those who were often more fearful of what other people may do to them than they should (mean score=26.8, SD=9.6, n=718) than those who were sometimes more fearful (mean score=21.6, SD=9.2, n=1501), which was significantly higher (p<0.001) than levels in individuals who occasionally had such fears (mean score=13.5, SD=10.1, n=2155), which was significantly higher (p<0.001) than levels in those who did not have such fears (mean score=3.4, SD=6.6, n=6008), F(df=3, 10 378)=3564.8, p<0.001. Eighty-eight participants (0.8%) reported having a current diagnosis of schizophrenia and 164 participants (1.6%) reported having a diagnosis of bipolar disorder.
Table 1.
Responses on the Revised Green et al Paranoid Thoughts Scale (R-GPTS) for the past month
| Not at all (%) | Somewhat (%) | Totally (%) | ||||
| 0 | 1 | 2 | 3 | 4 | ||
| Part A | ||||||
| 1 | I spent time thinking about friends gossiping about me. | 5167 (49.8) | 1673 (16.1) | 1628 (15.7) | 1262 (12.2) | 652 (6.3) |
| 2 | I often heard people referring to me. | 5309 (51.1) | 1874 (18.1) | 1535 (14.8) | 1176 (11.3) | 488 (4.7) |
| 3 | I have been upset by friends and colleagues judging me critically. | 4985 (49.0) | 1723 (16.6) | 1625 (15.74) | 1326 (12.8) | 723 (7.0) |
| 4 | People definitely laughed at me behind my back. | 5236 (50.4) | 1668 (16.1) | 1395 (13.4) | 1206 (11.6) | 877 (8.4) |
| 5 | I have been thinking a lot about people avoiding me. | 5055 (48.7) | 1621 (15.6) | 1525 (14.7) | 1293 (12.5) | 888 (8.6) |
| 6 | People have been dropping hints for me. | 5864 (65.5) | 1575 (15.2) | 1346 (13.0) | 1045 (10.1) | 552 (5.3) |
| 7 | I believed that certain people were not what they seemed. | 3574 (34.4) | 1660 (16.0) | 1985 (19.1) | 1748 (16.8) | 1415 (13.6) |
| 8 | People talking about me behind my back upset me. | 4592 (44.2) | 1504 (14.5) | 1554 (15.0) | 1428 (13.8) | 1304 (12.6) |
| Part B | ||||||
| 1 | Certain individuals have had it in for me. | 5130 (49.4) | 1615 (15.6) | 1609 (15.5) | 1169 (11.3) | 859 (8.3) |
| 2 | People wanted me to feel threatened, so they stared at me. | 6229 (60.0) | 1279 (12.3) | 1282 (12.3) | 1043 (10.0) | 549 (5.3) |
| 3 | I was certain people did things in order to annoy me. | 4798 (46.2) | 1727 (16.6) | 1658 (16.0) | 1321 (12.7) | 878 (8.5) |
| 4 | I was convinced there was a conspiracy against me. | 6313 (60.8) | 1221 (11.8) | 1206 (11.6) | 987 (9.5) | 655 (6.3) |
| 5 | I was sure someone wanted to hurt me. | 6390 (61.5) | 1217 (11.7) | 1153 (11.1) | 930 (9.0) | 692 (6.7) |
| 6 | I couldn’t stop thinking about people wanting to confuse me. | 6180 (59.5) | 1336 (12.9) | 1240 (11.9) | 1002 (9.7) | 624 (6.0) |
| 7 | I was distressed by being persecuted. | 6339 (61.1) | 1237 (11.9) | 1197 (11.5) | 983 (9.5) | 626 (6.0) |
| 8 | It was difficult to stop thinking about people wanting to make me feel bad. | 5815 (56.0) | 1394 (13.4) | 1384 (13.3) | 1079 (10.4) | 710 (6.8) |
| 9 | People have been hostile towards me on purpose. | 5608 (54.0) | 1487 (14.3) | 1391 (13.4) | 1102 (10.6) | 794 (7.6) |
| 10 | I was angry that someone wanted to hurt me. | 6069 (58.5) | 1223 (11.8) | 1275 (12.3) | 1013 (9.8) | 802 (7.7) |
A total of 1756 (16.9%) participants wanted help to be more trusting of other people, 4019 (38.7%) reported that they might want help to be more trusting and 4607 (44.4%) did not want help to be more trusting. Levels of persecutory ideation (R-GPTS Part B) significantly differed between the responses for wanting help to be more trusting, F(df=2, 10 379)=1720.6, p<0.001. Persecutory ideation was significantly higher (p<0.001) in those who wanted help to be more trusting (mean score=19.5, SD=11.8, n=1756) than those who might want help (mean score=12.0, SD=11.2, n=4016), which was significantly higher (p<0.001) than the levels in those who did not want help (mean score=4.0, SD=7.71, n=4607). Totalling the eight new Part C mistrust items (see online supplemental materials file 2) that concerned the person thinking their fears about other people were exaggerated produced a mistrust scale (Cronbach’s alpha=0.83) that was highly correlated with R-GPTS Part A (r=0.76, p<0.0.001, n=10 382) and Part B (r=0.77, p<0.001, n=10 382) total scores. This indicates that higher scores on the paranoia measure were associated with participants considering themselves to have excessive fear of others.
In the individual SEMs, all the social and cognitive variables were significantly associated with paranoia (see table 2). The factors most strongly associated (standardised coefficients ≥0.60) with higher levels of paranoia (in descending order) were: within-situation defence behaviours, negative images, hallucinations, dissociation, negative self-beliefs, discrimination, anxiety sensitivity, social anxiety, aberrant salience and agoraphobic distress. In the combined analysis, the final SEM model indicated a good fit to the data (robust χ2 (10 627, n=10 382)=83 150.970, p<0.001, RMSEA (Root Mean Square Error of Approximation)=0.03; SRMR (Standardized Root Mean Squared Residual)=0.07; CFI (Comparative Fit Index)=0.92; TLI (Tucker Lewis Index)=0.92) and the variance explained by the structural model was 66.7%. Thirteen explanatory factors were significant (figure 1). Within-situation defence behaviours had the strongest association with paranoia and alcohol use had the lowest association (table 2 for standardised coefficients).
Table 2.
Parameter estimates of the SEMs with paranoia as the response variable
| Explanatory variable | B | SE | Adjusted P value | β | R2 |
| Individual SEMs for each social and cognitive factor | |||||
| Worry | 0.584 | 0.012 | <0.001 | 0.504 | 0.254 |
| Defence behaviours | 0.928 | 0.009 | <0.001 | 0.783 | 0.612 |
| Positive self-beliefs | −0.316 | 0.014 | <0.001 | −0.262 | 0.069 |
| Negative self-beliefs | 0.791 | 0.013 | <0.001 | 0.678 | 0.460 |
| Dissociative experiences | 0.801 | 0.011 | <0.001 | 0.699 | 0.488 |
| Hallucinations | 0.698 | 0.011 | <0.001 | 0.707 | 0.499 |
| Negative images | 0.662 | 0.010 | <0.001 | 0.730 | 0.533 |
| Aberrant salience | 0.657 | 0.011 | <0.001 | 0.610 | 0.372 |
| Anxiety sensitivity | 0.759 | 0.013 | <0.001 | 0.632 | 0.399 |
| Agoraphobic avoidance | 0.625 | 0.014 | <0.001 | 0.574 | 0.329 |
| Agoraphobic distress | 0.266 | 0.005 | <0.001 | 0.604 | 0.365 |
| Analytical reasoning | −0.618 | 0.034 | <0.001 | −0.233 | 0.054 |
| Insomnia | 0.702 | 0.014 | <0.001 | 0.549 | 0.302 |
| Locus of control | 0.622 | 0.015 | <0.001 | 0.432 | 0.187 |
| Social anxiety | 0.775 | 0.013 | <0.001 | 0.616 | 0.379 |
| Alcohol use | 0.343 | 0.019 | <0.001 | 0.252 | 0.063 |
| Discrimination | 0.692 | 0.014 | <0.001 | 0.666 | 0.443 |
| Bullying | 0.537 | 0.013 | <0.001 | 0.462 | 0.213 |
| Childhood mistreatment | 0.602 | 0.014 | <0.001 | 0.542 | 0.294 |
| Social support | −0.228 | 0.010 | <0.001 | −0.261 | 0.068 |
| Cannabis use | 0.344 | 0.013 | <0.001 | 0.317 | 0.100 |
| Stressful life events | 0.372 | 0.020 | <0.001 | 0.279 | 0.078 |
| Unified SEM model | |||||
| Worry | 0.044 | 0.011 | <0.001 | 0.038 | 0.67 (across the model) |
| Defence behaviours | 0.427 | 0.028 | <0.001 | 0.250 | |
| Negative self-beliefs | 0.119 | 0.019 | <0.001 | 0.103 | |
| Dissociative experiences | 0.087 | 0.019 | <0.001 | 0.076 | |
| Negative images | 0.209 | 0.016 | <0.001 | 0.229 | |
| Aberrant salience | 0.239 | 0.053 | <0.001 | 0.053 | |
| Anxiety sensitivity | 0.050 | 0.016 | <0.01 | 0.042 | |
| Agoraphobic avoidance | 0.153 | 0.056 | <0.01 | 0.029 | |
| Agoraphobic distress | 0.018 | 0.006 | <0.01 | 0.041 | |
| Analytical reasoning | −0.070 | 0.023 | <0.01 | −0.026 | |
| Alcohol use | 0.028 | 0.011 | <0.01 | 0.020 | |
| Discrimination | 0.521 | 0.061 | <0.001 | 0.102 | |
| Social support | −0.027 | 0.007 | <0.001 | −0.031 | |
B denotes unstandardised coefficient; β indicates standardised coefficient.
SEM, structural equation modelling.
Figure 1.
Structural equation modelling for the social and cognitive explanatory factors (showing unstandardised coefficients). *P<0.01, **p<0.01, ***p<0.001. ABE SAL, aberrant salience; AGO AVO, agoraphobic avoidance; AGO DIS, agoraphobic distress; ALC USE, alcohol use; ANA REA, analytical reasoning; ANX SEN, anxiety sensitivity; DEF BEH, defensive behaviours; DIS, discrimination; DIS EXP, dissociative experiences; NEG BEL, negative beliefs; NEG IMG, negative images; SOC SUP, social support; WOR, worry.
In the differential analysis, all the social and cognitive variables were significantly associated with both social anxiety and paranoia (table 3). That is, all explanatory factors explained variance in both social anxiety and in paranoia. The Wald test analyses for standardised coefficients indicated that all but the associations of agoraphobic avoidance and social support were found to be statistically different between paranoia and social anxiety. Ten factors were more strongly associated with paranoia than social anxiety (in descending order of discrepancy): discrimination, hallucinations, negative images, aberrant salience, alcohol use, stressful life events, cannabis use, within-situation defence behaviours, childhood mistreatment and dissociation. Nine factors were more strongly associated with social anxiety than paranoia (in descending order of discrepancy): lower levels of positive self-belief, an external locus of control, worry, less use of analytical reasoning, anxiety sensitivity, insomnia, agoraphobic distress, bullying and negative self-beliefs.
Table 3.
Individual SEMs with both paranoia and social anxiety as the response variables
| Individual SEM results | Wald test analysis to compare the two regression coefficients | |||||||||||
| Response variable | Explanatory variable | B | SE | Adjusted P value | β | R2 | B | SE B | Adjusted P value | β | SE β | Adjusted P value |
| Paranoia Social anxiety |
Worry Worry |
0.584 0.604 |
0.012 0.010 |
<0.001 <0.001 |
0.504 0.657 |
0.254 0.432 |
−0.019 | 0.011 | 0.18 | −0.153 | 0.007 | <0.001 |
| Paranoia Social anxiety |
Defences Defences |
0.919 0.812 |
0.009 0.009 |
<0.001 <0.001 |
0.782 0.716 |
0.612 0.513 |
0.107 | 0.008 | <0.001 | 0.066 | 0.006 | <0.001 |
| Paranoia Social anxiety |
Positive beliefs Positive beliefs |
−0.317 −0.474 |
0.014 0.011 |
<0.001 <0.001 |
−0.262 −0.495 |
0.069 0.245 |
0.157 | 0.012 | <0.001 | 0.233 | 0.010 | <0.001 |
| Paranoia Social anxiety |
Negative beliefs Negative beliefs |
0.794 0.654 |
0.013 0.012 |
<0.001 <0.001 |
0.678 0.701 |
0.459 0.492 |
0.141 | 0.013 | <0.001 | −0.023 | 0.009 | <0.05 |
| Paranoia Social anxiety |
Dissociation Dissociation |
0.802 0.605 |
0.011 0.011 |
<0.001 <0.001 |
0.699 0.662 |
0.488 0.439 |
0.197 | 0.012 | <0.001 | 0.036 | 0.008 | <0.001 |
| Paranoia Social anxiety |
Hallucinations Hallucinations |
0.698 0.446 |
0.011 0.009 |
<0.001 <0.001 |
0.707 0.566 |
0.499 0.321 |
0.252 | 0.011 | <0.001 | 0.140 | 0.008 | <0.001 |
| Paranoia Social anxiety |
Negative images Negative images |
0.664 0.442 |
0.010 0.008 |
<0.001 <0.001 |
0.730 0.610 |
0.533 0.372 |
0.222 | 0.010 | <0.001 | 0.120 | 0.008 | <0.001 |
| Paranoia Social anxiety |
Aberrant salience Aberrant salience |
0.667 0.518 |
0.011 0.011 |
<0.001 <0.001 |
0.611 0.504 |
0.373 0.254 |
0.149 | 0.010 | <0.001 | 0.107 | 0.009 | <0.001 |
| Paranoia Social anxiety |
Anxiety sensitivity Anxiety sensitivity |
0.758 0.723 |
0.013 0.012 |
<0.001 <0.001 |
0.632 0.760 |
0.399 0.577 |
0.035 | 0.013 | <0.05 | −0.128 | 0.008 | <0.001 |
| Paranoia Social anxiety |
Agoraphobic avoidance Agoraphobic avoidance |
0.625 0.585 |
0.014 0.014 |
<0.001 <0.001 |
0.574 0.571 |
0.329 0.326 |
0.040 | 0.012 | <0.01 | 0.003 | 0.011 | 0.836 |
| Paranoia Social anxiety |
Agoraphobic distress Agoraphobic distress |
0.266 0.236 |
0.005 0.004 |
<0.001 <0.001 |
0.604 0.672 |
0.364 0.452 |
0.030 | 0.005 | <0.001 | −0.069 | 0.009 | <0.001 |
| Paranoia Social anxiety |
Analytical reasoning Analytical reasoning |
−0.581 −0.728 |
0.031 0.023 |
<0.001 <0.001 |
−0.234 −0.369 |
0.055 0.136 |
0.147 | 0.028 | <0.001 | 0.135 | 0.011 | <0.001 |
| Paranoia Social anxiety |
Insomnia Insomnia |
0.699 0.649 |
0.014 0.012 |
<0.001 <0.001 |
0.549 0.643 |
0.413 0.413 |
0.050 | 0.013 | <0.001 | −0.093 | 0.008 | <0.001 |
| Paranoia Social anxiety |
Locus of control Locus of control |
0.621 0.740 |
0.015 0.013 |
<0.001 <0.001 |
0.433 0.651 |
0.187 0.423 |
−0.119 | 0.014 | <0.001 | −0.218 | 0.008 | <0.001 |
| Paranoia Social anxiety |
Alcohol use Alcohol use |
0.344 0.169 |
0.019 0.014 |
<0.001 <0.001 |
0.252 0.156 |
0.063 0.024 |
0.175 | 0.013 | <0.001 | 0.096 | 0.010 | <0.001 |
| Paranoia Social anxiety |
Discrimination Discrimination |
0.712 0.516 |
0.015 0.014 |
<0.001 <0.001 |
0.664 0.511 |
0.440 0.262 |
0.196 | 0.011 | <0.001 | 0.152 | 0.010 | <0.001 |
| Paranoia Social anxiety |
Bullying Bullying |
0.564 0.556 |
0.014 0.013 |
<0.001 <0.001 |
0.464 0.490 |
0.216 0.240 |
0.008 | 0.012 | 0.49 | −0.025 | 0.010 | <0.05 |
| Paranoia Social anxiety |
Childhood mistreatment Childhood mistreatment |
0.622 0.511 |
0.015 0.014 |
<0.001 <0.001 |
0.541 0.475 |
0.293 0.226 |
0.110 | 0.012 | <0.001 | 0.066 | 0.010 | <0.001 |
| Paranoia Social anxiety |
Social support Social support |
−0.228 −0.175 |
0.010 0.008 |
<0.001 <0.001 |
−0.261 −0.252 |
0.068 0.064 |
−0.053 | 0.009 | <0.001 | −0.008 | 0.010 | 0.836 |
| Paranoia Social anxiety |
Cannabis use Cannabis use |
0.345 0.202 |
0.013 0.009 |
<0.001 <0.001 |
0.311 0.229 |
0.100 0.055 |
0.143 | 0.010 | <0.001 | 0.081 | 0.009 | <0.001 |
| Paranoia Social anxiety |
Stressful life events Stressful life events |
0.373 0.222 |
0.020 0.015 |
<0.001 <0.001 |
0.336 0.252 |
0.078 0.044 |
0.150 | 0.012 | <0.001 | 0.084 | 0.009 | <0.001 |
B denotes unstandardised coefficient; β indicates standardised coefficient. Adjusted p value based on Holm.50
SEM, structural equation modelling.
In the sociodemographic analysis (online supplemental tables S39–S41), the final SEM model indicated an excellent fit to the data (robust χ2 (179, n=10 309)=2359.650, p<0.001, RMSEA=0.03; SRMR=0.009; CFI=0.97; TLI=0.97). The variance explained by the structural model using sociodemographic factors was 21.1%. Paranoia was associated with lower age, male gender, lower income, relationship status and region. In the individual SEMs, age had by far the strongest association of the sociodemographic variables with paranoia (standardised coefficient=−0.42, p<0.001, R2=0.186).
Discussion
One in four of this general population group described themselves as mistrustful of other people. Approximately one in five people were having regular persecutory thoughts. Approximately 5–8% of the population may be having very strong paranoia. We show for the first time that people scoring higher on a standard assessment of paranoia were also recognising that their fears were exaggerated. This indicates that when completing the paranoia questionnaire participants were reporting their inaccurate fears rather than actual threats. A significant minority of the population would like help to become more trusting of other people. Difficulties with paranoia are not confined to people diagnosed with serious mental health conditions. Scoring higher on the paranoia assessment was also associated with wanting help to increase trust.
This was a large study that could estimate with precision the degree of association between putative explanatory variables and paranoia. Encouragingly, a high proportion of the population variance in levels of paranoia was explained by the social and cognitive factors. Much is known about the occurrence of paranoia. Importantly, there are not one or two factors likely to lead to paranoia but multiple. The most influential factor was the use of protective strategies—limiting time in social situations, watching out for danger and trying to be inconspicuous—that the person believes have kept the harm from happening. Acting as if the world is unsafe prevents learning that the world is safe. Indeed, avoidance at agoraphobic levels may severely limit the potential for any receipt of disconfirmatory evidence. Intrusive negative images of other people laughing or physically harming were also an important explanatory factor. Such images may add to the sense of vulnerability that paranoia builds on. It is likely a range of negative affective processes, including low self-esteem, worry and social-evaluative concerns, and anomalous states, including dissociation, hallucinations and aberrant salience, that drive the development and persistence of paranoia. Experiencing discrimination was the strongest social predictor of paranoia. It has previously been found to be associated with paranoia.48 Being mistreated by people will foster mistrust. Many of the explanatory factors associated with paranoia are those targeted in the Feeling Safe programme for the treatment of persecutory delusions.24 The survey results support our new programme of work, called Feeling Safer, that develops the therapeutic approach by expanding the range of mechanistic targets and techniques that can be offered to patients.
As would be expected there was a great degree of potential shared causation for paranoia and social anxiety. All the social and cognitive factors were associated with both paranoia and social anxiety, often explaining a high degree of variance in each. The study was sufficiently large to detect differences, even small, in the sizes of associations between explanatory factors and paranoia and social anxiety. The overwhelming majority of explanatory factors differed at least to a degree in their level of association with paranoia and social anxiety. Perhaps most surprising were that intrusive negative images were more strongly associated with paranoia than social anxiety and that less use of analytical reasoning was more strongly associated with social anxiety than paranoia. It is more expected that discrimination, hallucinations, aberrant salience and cannabis use were more strongly associated with paranoia than social anxiety. The study clearly indicates that paranoia and social anxiety may arise from similar causes and co-occur but it also identifies a range of factors that may make paranoia or social anxiety more prominent in a presentation.
There are a number of study limitations. The research was cross-sectional and therefore the presence of causal effects or the direction of most associations cannot be determined. The value of each factor in understanding paranoia may be best determined from causal-interventionist approaches in which the effect of targeting a mechanism in intervention is examined.49 We used a non-probability online quota sampling method, rather than random sampling, which will have introduced bias as to who was approached to take part. We do know that, taken as a whole, the respondents in this survey were broadly representative of the adult general population on a number of basic sociodemographic features but not that individual respondents were representative of the general population. There were no attention checks in the survey so it is unknown the degree to which unconsidered responses to questions lowered the quality of the data. It is also the case that there is unexplained variance in paranoia, meaning that identification of additional factors truly independent of those assessed in the current study is needed. Nevertheless, the study identifies multiple routes by which paranoia might be lessened. At a social level there can be targeting of discrimination, bullying, childhood mistreatment and insufficient social support. At an individual level it will be helpful to reduce the sense of vulnerability caused by negative self-beliefs and images. Lessening anxiety, worry and sleep difficulties and enabling greater toleration of anomalous experiences such as dissociation, aberrant salience and anxiety sensitivity should prove valuable. Such work may then facilitate the crucial step of the lowering of defence behaviours, including avoidance, so a person can experience safety directly. Experiences of safety can build the trust that counteracts paranoia.
bmjment-2023-300880supp002.pdf (832.1KB, pdf)
Acknowledgments
We thank Sinéad Lambe and Felicity Hudson for assistance in setting up the survey.
Footnotes
Contributors: DF conceived and designed the study and wrote the paper. DF is responsible for the overall content as guarantor. BSL carried out the statistical analysis and contributed to the paper.
Funding: The study was supported by an NIHR Senior Investigator Award to DF, an NIHR Programme Grant for Applied Research (NIHR204013) and the NIHR Oxford Health Biomedical Research Centre.
Disclaimer: The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.
Competing interests: None declared.
Provenance and peer review: Not commissioned; externally peer reviewed.
Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
Data availability statement
Data are available upon reasonable request. Deidentified participant data will be available in anonymised form from the corresponding author (DF) on reasonable request (including a study outline), subject to university approval, following the publication of results.
Ethics statements
Patient consent for publication
Not applicable.
Ethics approval
This study involves human participants and was approved by the University of Oxford Medical Sciences Interdivisional Research Ethics Committee (R84937/RE001). Participants gave informed consent to participate in the study before taking part.
References
- 1. Wigman JTW, van Nierop M, Vollebergh WAM, et al. Evidence that psychotic symptoms are prevalent in disorders of anxiety and depression, impacting on illness onset, risk, and severity--implications for diagnosis and ultra-high risk research. Schizophr Bull 2012;38:247–57. 10.1093/schbul/sbr196 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Knight C, Russo D, Stochl J, et al. Prevalence of and recovery from common mental disorder including psychotic experiences in the UK primary care improving access to psychological therapies (IAPT) programme. J Affect Disord 2020;272:84–90. 10.1016/j.jad.2020.04.015 [DOI] [PubMed] [Google Scholar]
- 3. Bird JC, Fergusson EC, Kirkham M, et al. Paranoia in patients attending child and adolescent mental health services. Aust N Z J Psychiatry 2021;55:1166–77. 10.1177/0004867420981416 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Zavos HMS, Freeman D, Haworth CMA, et al. Consistent etiology of severe, frequent psychotic experiences and milder, less frequent manifestations: a twin study of specific psychotic experiences in adolescence. JAMA Psychiatry 2014;71:1049–57. 10.1001/jamapsychiatry.2014.994 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Freeman D, Garety PA, Bebbington PE, et al. Psychological investigation of the structure of paranoia in a non-clinical population. Br J Psychiatry 2005;186:427–35. 10.1192/bjp.186.5.427 [DOI] [PubMed] [Google Scholar]
- 6. Freeman D. Persecutory delusions: a cognitive perspective on understanding and treatment. Lancet Psychiatry 2016;3:685–92. 10.1016/S2215-0366(16)00066-3 [DOI] [PubMed] [Google Scholar]
- 7. Mackie JL. The Cement of the Universe: A Study of Causation. Oxford: Oxford University Press, 1974. [Google Scholar]
- 8. Gilbert P, Boxall M, Cheung M, et al. The relation of paranoid Ideation and social anxiety in a mixed clinical population. Clin Psychol Psychother 2005;12:124–33. 10.1002/cpp.438 [DOI] [Google Scholar]
- 9. Freeman D, Evans N, Lister R, et al. Height, social comparison, and paranoia: an Immersive virtual reality experimental study. Psychiatry Res 2014;218:348–52. 10.1016/j.psychres.2013.12.014 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Schulze K, Freeman D, Green C, et al. Intrusive mental imagery in patients with persecutory delusions. Behav Res Ther 2013;51:7–14. 10.1016/j.brat.2012.10.002 [DOI] [PubMed] [Google Scholar]
- 11. Mirowsky J, Ross CE. Paranoia and the structure of powerlessness. Am Sociol Rev 1983;48:228–39. [PubMed] [Google Scholar]
- 12. Freeman D, Rosebrock L, Loe BS, et al. The Oxford Positive Self Scale: psychometric development of an assessment of cognitions associated with psychological well-being. Psychol Med 2023:1–9. 10.1017/S0033291723000624 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Freeman D, Garety PA. Worry, worry processes and dimensions of delusions: an exploratory investigation of a role for anxiety processes in the maintenance of delusional distress. Behav Cogn Psychother 1999;27:47–62. 10.1017/S135246589927107X [DOI] [Google Scholar]
- 14. Freeman D, Dunn G, Murray RM, et al. How cannabis causes paranoia: using the intravenous administration of ∆9-Tetrahydrocannabinol (THC) to identify key cognitive mechanisms leading to paranoia. Schizophr Bull 2015;41:391–9. 10.1093/schbul/sbu098 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Černis E, Evans R, Ehlers A, et al. Dissociation in relation to other mental health conditions: an exploration using network analysis. J Psychiatr Res 2021;136:460–7. 10.1016/j.jpsychires.2020.08.023 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Freeman D, Garety PA, Kuipers E. Persecutory delusions: developing the understanding of belief maintenance and emotional distress. Psychol Med 2001;31:1293–306. 10.1017/s003329170100455x [DOI] [PubMed] [Google Scholar]
- 17. Freeman D, Lister R, Evans N. The use of intuitive and analytic reasoning styles by patients with persecutory delusions. J Behav Ther Exp Psychiatry 2014;45:454–8. 10.1016/j.jbtep.2014.06.005 [DOI] [PubMed] [Google Scholar]
- 18. Reeve S, Emsley R, Sheaves B, et al. Disrupting sleep: the effects of sleep loss on psychotic experiences tested in an experimental study with mediation analysis. Schizophr Bull 2018;44:662–71. 10.1093/schbul/sbx103 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Gracie A, Freeman D, Green S, et al. The association between traumatic experience, paranoia and hallucinations: a test of the predictions of psychological models. Acta Psychiatr Scand 2007;116:280–9. 10.1111/j.1600-0447.2007.01011.x [DOI] [PubMed] [Google Scholar]
- 20. Bentall RP, Wickham S, Shevlin M, et al. Do specific early-life adversities lead to specific symptoms of psychosis? A study from the 2007 the adult psychiatric morbidity survey. Schizophr Bull 2012;38:734–40. 10.1093/schbul/sbs049 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Freeman D, Gittins M, Pugh K, et al. What makes one person paranoid and another person anxious? the differential prediction of social anxiety and persecutory Ideation in an experimental situation. Psychol Med 2008;38:1121–32. 10.1017/S0033291708003589 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Freeman D, Thompson C, Vorontsova N, et al. Paranoia and post-traumatic stress disorder in the months after a physical assault: a longitudinal study examining shared and differential predictors. Psychol Med 2013;43:2673–84. 10.1017/S003329171300038X [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Freeman D, Garety PA, Bebbington P, et al. The psychology of persecutory Ideation II: a virtual reality experimental study. J Nerv Ment Dis 2005;193:309–15. 10.1097/01.nmd.0000161686.53245.70 [DOI] [PubMed] [Google Scholar]
- 24. Freeman D, Emsley R, Diamond R, et al. Comparison of a theoretically driven cognitive therapy (the Feeling Safe Programme) with befriending for the treatment of persistent persecutory delusions: a parallel, single-blind, randomised controlled trial. Lancet Psychiatry 2021;8:696–707. 10.1016/S2215-0366(21)00158-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Freeman D, Loe BS, Kingdon D, et al. The revised Green et al., Paranoid Thoughts Scale (R-GPTS): Psychometric properties, severity ranges, and clinical cut-offs . Psychol Med 2021;51:244–53. 10.1017/S0033291719003155 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Connor KM, Davidson JR, Churchill LE, et al. Psychometric properties of the Social Phobia inventory (SPIN): new self-rating scale. Br J Psychiatry 2000;176:379–86. 10.1192/bjp.176.4.379 [DOI] [PubMed] [Google Scholar]
- 27. Freeman D, Bird JC, Loe BS, et al. The Dunn Worry Questionnaire and the Paranoia Worries Questionnaire: new assessments of worry. Psychol Med 2020;50:771–80. 10.1017/S0033291719000588 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Fowler D, Freeman D, Smith B, et al. The Brief Core Schema Scales (BCSS): Psychometric properties and associations with paranoia and grandiosity in non-clinical and psychosis samples. Psychol Med 2006;36:749–59. 10.1017/S0033291706007355 [DOI] [PubMed] [Google Scholar]
- 29. Černis E, Beierl E, Molodynski A, et al. A new perspective and assessment measure for common dissociative experiences: ‘felt sense of anomaly' PLoS One 2021;16:e0247037. 10.1371/journal.pone.0247037 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Bell V, Halligan PW, Ellis HD. The Cardiff Anomalous Perceptions Scale (CAPS). Schizophr Bull 2006;32:366–77. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Cicero DC, Kerns JG, McCarthy DM. The Aberrant Salience Inventory: a new measure of psychosis proneness. Psychol Assess 2010;22:688–701. 10.1037/a0019913 [DOI] [PubMed] [Google Scholar]
- 32. Reiss S, Peterson RA, Gursky DM, et al. Anxiety sensitivity, anxiety frequency and the prediction of fearfulness. Behav Res Ther 1986;24:1–8. 10.1016/0005-7967(86)90143-9 [DOI] [PubMed] [Google Scholar]
- 33. Lambe S, Bird JC, Loe BS, et al. The Oxford Agoraphobic Avoidance Scale. Psychol Med 2023;53:1233–43. 10.1017/S0033291721002713 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Pacini R, Epstein S. The relation of rational and experiential information processing styles to personality, basic beliefs, and the ratio-bias phenomenon. J Pers Soc Psychol 1999;76:972–87. 10.1037//0022-3514.76.6.972 [DOI] [PubMed] [Google Scholar]
- 35. Barnett T, Lanier PA. Comparison of alternative response formats for an abbreviated version of Rotter’s locus of control scale. Psychol Rep 1995;77:259–64. 10.2466/pr0.1995.77.1.259 [DOI] [Google Scholar]
- 36. Bastien CH, Vallières A, Morin CM. Validation of the Insomnia Severity Index as an outcome measure for insomnia research. Sleep Med 2001;2:297–307. 10.1016/s1389-9457(00)00065-4 [DOI] [PubMed] [Google Scholar]
- 37. Bush K, Kivlahan DR, McDonell MB. The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking [DOI] [PubMed]
- 38. Adamson SJ, Sellman JD. A prototype screening instrument for Cannabis use disorder: the Cannabis Use Disorders Identification Test (CUDIT) in an alcohol-dependent clinical sample. Drug Alcohol Rev 2003;22:309–15. 10.1080/0959523031000154454 [DOI] [PubMed] [Google Scholar]
- 39. McManus S, Bebbington P, Jenkins R, et al. Mental health and wellbeing in England: Adult Psychiatric Morbidity Survey 2014. Leeds: NHS Digital, 2016. [Google Scholar]
- 40. Felix ED, Sharkey JD, Green JG, et al. Getting precise and pragmatic about the assessment of bullying: the development of the California Bullying Victimization Scale. Aggress Behav 2011;37:234–47. 10.1002/ab.20389 [DOI] [PubMed] [Google Scholar]
- 41. Zimet GD, Powell SS, Farley GK, et al. Psychometric characteristics of the Multidimensional Scale of Perceived Social Support. J Pers Assess 1990;55:610–7. 10.1080/00223891.1990.9674095 [DOI] [PubMed] [Google Scholar]
- 42. Revelle W. psych: Procedures for psychological, psychometric, and personality research. Evanston, Illinois: Northwestern University, 2017: 1–10. [Google Scholar]
- 43. Rosseel Y. Lavaan: an R package for structural equation modeling and more. version 0.5–12 (BETA). J Stat Softw 2012;48:1–36. 10.18637/jss.v048.i02 [DOI] [Google Scholar]
- 44. Kaiser HF. A second generation little jiffy. Psychometrika 1970;35:401–15. 10.1007/BF02291817 [DOI] [Google Scholar]
- 45. McDonald RP. Test theory. In: Test theory: A unified treatment. Psychology press, 2013. 10.4324/9781410601087 [DOI] [Google Scholar]
- 46. Schreiber JB, Nora A, Stage FK, et al. Reporting structural equation modeling and confirmatory factor analysis results: A review. The Journal of Educational Research 2006;99:323–38. 10.3200/JOER.99.6.323-338 [DOI] [Google Scholar]
- 47. Smith RL, Ager JW, Williams DL. Suppressor variables in multiple regression/correlation. Educational and Psychological Measurement 1992;52:17–29. 10.1177/001316449205200102 [DOI] [Google Scholar]
- 48. Pearce J, Rafiq S, Simpson J, et al. Perceived discrimination and psychosis: a systematic review of the literature. Soc Psychiatry Psychiatr Epidemiol 2019;54:1023–44. 10.1007/s00127-019-01729-3 [DOI] [PubMed] [Google Scholar]
- 49. Kendler KS, Campbell J. Interventionist causal models in psychiatry: repositioning the mind–body problem. Psychol Med 2009;39:881–7. 10.1017/S0033291708004467 [DOI] [PubMed] [Google Scholar]
- 50. Holm S. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics 1979:65–70. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
bmjment-2023-300880supp001.pdf (629.5KB, pdf)
bmjment-2023-300880supp002.pdf (832.1KB, pdf)
Data Availability Statement
Data are available upon reasonable request. Deidentified participant data will be available in anonymised form from the corresponding author (DF) on reasonable request (including a study outline), subject to university approval, following the publication of results.