Psychosis, Delusions and the “Jumping to Conclusions” Reasoning Bias: A Systematic Review and Meta-analysis

Abstract

We did a systematic review and meta-analysis to investigate the magnitude and specificity of the “jumping to conclusions” (JTC) bias in psychosis and delusions. We examined the extent to which people with psychosis, and people with delusions specifically, required less information before making decisions. We examined (1) the average amount of information required to make a decision and (2) numbers who demonstrated an extreme JTC bias, as assessed by the “beads task.” We compared people with psychosis to people with and without nonpsychotic mental health problems, and people with psychosis with and without delusions. We examined whether reduced data-gathering was associated with increased delusion severity. We identified 55 relevant studies, and acquired previously unpublished data from 16 authors. People with psychosis required significantly less information to make decisions than healthy individuals (k = 33, N = 1935, g = −0.53, 95% CI −0.69, −0.36) and those with nonpsychotic mental health problems (k = 13, N = 667, g = −0.58, 95% CI −0.80, −0.35). The odds of extreme responding in psychosis were between 4 and 6 times higher than the odds of extreme responding by healthy participants and participants with nonpsychotic mental health problems. The JTC bias was linked to a greater probability of delusion occurrence in psychosis (k = 14, N = 770, OR 1.52, 95% CI 1.12, 2.05). There was a trend-level inverse association between data-gathering and delusion severity (k = 18; N = 794; r = −.09, 95% CI −0.21, 0.03). Hence, nonaffective psychosis is characterized by a hasty decision-making style, which is linked to an increased probability of delusions.

Introduction

Delusions are fixed beliefs that are not amenable to change in light of counterevidence. Grandiosity and paranoia are common themes¹ and such beliefs are a hallmark feature of diagnoses like schizophrenia and delusional disorder. Delusions are often preoccupying, distressing, and disabling to the holder or those around the person, and show only modest response to medication² or talking therapies such as Cognitive Behavioural Therapy.³ In order to improve treatment effectiveness, research has focussed on identifying and understanding psychological processes that may underlie delusion formation and maintenance. A key finding to emerge is that people with delusions appear to make decisions on the basis of less evidence than healthy individuals or people with nonpsychotic mental health problems.⁴ This decision-making style has been referred to as the “jumping to conclusions” (JTC) bias and there is preliminary evidence that it is demonstrated by nonclinical participants who score highly for delusional ideation⁵; in those at risk of developing psychosis⁶; as well as in first-degree relatives of people with psychosis.⁷

Given these findings, JTC plays a central role in numerous psychological and neuropsychological theories of delusions as this bias may lead to the premature acceptance of implausible ideas and perhaps prevent consideration of more realistic alternative explanations of events.⁸ The JTC bias has been targeted in specific reasoning interventions, in the hope that modifying it may reduce delusional ideation.⁹ However, the magnitude and prevalence of the JTC bias in psychosis has yet to be systematically quantified. Moreover, despite almost 3 decades of research, the specificity of the JTC bias to delusions, rather than illness severity or psychosis per se, remains unclear.

The experimental paradigm most commonly used to assess the JTC bias is known as the “beads task.”¹⁰ This task involves showing participants 2 jars of beads in equal but opposite ratios, such as 85 red and 15 blue and vice versa. Both jars are then hidden, and the participant is told that individual beads are drawn consecutively from 1 jar. The beads are actually presented in a prespecified sequence. The task of the participant is to decide from which jar beads are being drawn. Two key outcome variables from this type of study are the number of beads drawn before a decision is made (“draws to decision”; DTD) and the proportion of “extreme responders.” This is the number of people who make a decision on the basis of 2 or fewer beads, which is taken to indicate the presence of a marked JTC bias. Variations of this task have presented the materials on computer,¹¹ made the task harder (using 60:40 ratio for instance,¹²), and have manipulated the content to make it less abstract,^13,14 more personally relevant,¹⁵ socially relevant,¹⁶ or emotionally laden.¹²

The role of the JTC bias in psychosis has been subject to numerous narrative reviews (ie, ¹,⁴,¹⁷) but only 2 systematic meta-analyses. The first, by Fine and colleagues,¹⁸ answered several important questions regarding the nature of the JTC bias in an established psychosis population, and usefully established the importance of JTC and DTD as key outcome variables in this field but the precision of their estimates was limited by inclusion of only 12 small studies, and there was no systematic assessment of study or outcome quality. Since this time, there have been a large number of studies investigating decision-making and delusions published (for a narrative review see Freeman and Garety⁸) meaning a definitive estimate of any psychosis or delusion-attributable differences in data-gathering is both possible and timely.

A second meta-analysis, by Ross and colleagues,¹⁹ was published in early 2015. This analysis of participant-level data from 23 nonclinical samples (N = 1754) and 8 samples with current delusions (N = 262) studies found a small relationship between DTD and self reported delusional ideation measured using the Peters Delusion Inventory (PDI; ²⁰) (nonclinical r = −.10; clinical current delusions r = −.12). Although Ross and colleagues examined the moderating role of study design variables, they did not report a systematic assessment of study or outcome quality, and did not examine whether extreme responding was associated with greater delusion proneness or was more prevalent in people with delusions. The high ratio of nonclinical to clinical studies and the relatively low number of clinical participants means we are cautious in generalising their findings to people with established psychosis and a clinically focussed analysis of the research is clearly indicated.

The aim of the current systematic review and meta-analysis was to provide an assessment of (1) the extent to which people with nonaffective psychosis demonstrate an increased data gathering bias, as measured by both average “draws to decision” and the number of extreme responders, and (2) whether and to what extent this bias is specific to delusions rather than psychosis or illness severity. We used published and unpublished data to compare people with psychosis to those with and without other mental health problems, and we compared people with psychosis who had and did not have current delusions. We examined whether reduced data-gathering was associated with delusion severity, and we carried out a number of pre-specified moderator analyses to examine the effect of study quality and design variables on the overall estimates of effect.

Methods

Search Strategy

Publication databases and previous reviews were searched independently by 2 reviewers (PT and PH) up to March 2014, and updated by SW in January 2015. Studies were eligible for inclusion if they measured DTD or JTC using the beads tasks or one conceptually similar (such as the fishes in lakes task, ¹³) in (1) people with established psychosis and (2) people with nonpsychotic mental health problems or healthy participants. Studies comparing people with psychosis who had current delusions to people with psychosis who did not have delusions were also eligible. The psychosis and delusions groups consisted only of those participants who had a diagnosis of nonaffective psychosis. Cross-sectional baseline data were extracted from longitudinal studies, experimental manipulation studies, or reports of randomized controlled trials, and authors were contacted in every case where usable but unpublished data were thought to exist. Studies without control group data were eligible for inclusion in the analyses of DTD/JTC and delusion severity. Further details of our search strategy and inclusion and exclusion criteria are provided in our protocol in the supplementary material and a previous publication.²¹

Data Extraction and Outcomes

Two reviewers independently extracted data from each study using a spreadsheet. Our primary outcomes were (1) the average draws to decision and (2) the number of people who demonstrated an extreme JTC bias. We defined the latter as the number of people choosing to make a decision after either 2 beads or 1 bead (where this is what is reported by the authors), with 2 beads being preferable. For both outcomes we prioritized performance on trials where the ratio of visible bead colour (or equivalent) was 85:15. If not available we used, in order of preference, 80:20, 90:10, or 60:40. We made an a priori decision to select 85:15 as our preference for inclusion into the meta-analysis as we anticipated this would be the most prevalent ratio used across studies. Where results for an 85:15 ratio were provided these were always used rather than results for other ratios. Also, where data from multiple trials all using 85:15 were provided we used the average result.

We analysed data from trials where the information to be gathered was emotionally neutral (eg, number of beads from a jar, or number of fish from a lake) but if this was not available we used data from “salient” trials, where participants are asked to make decisions regarding the source of emotionally salient material (eg, negative or positive survey responses, ¹²).

For the analysis of differences in DTD, we extracted mean DTD scores per group and the associated SD to allow calculation of the weighted (WMD) and standardized mean difference (SMD). Missing SDs were, where possible, calculated from t test values, P-values, F-values, SEs or confidence intervals using equations specified in the Cochrane Handbook²² or Borenstein and colleagues.²³ If neither means and SDs were reported, then we used the reported effect size measures to calculate SMD.

For the analyses of extreme-responding, we used the number of participants who jumped to conclusions as the numerator, and number who entered the study group as the denominator. Correlation coefficients and related variance parameters were extracted for the analysis of DTD and delusion severity. For the analysis of JTC and delusions we used, for each study: (1) mean delusions ratings and associated SDs for extreme and nonextreme responders and (2) number of people with psychosis and delusions who show extreme responding and number of people with psychosis without current delusions who show extreme responding. Both sets of figures were then converted to ORs and included in the meta-analysis together.

Meta-Analytic Calculations

Both Hedges’s g SMD and the WMD were used as metrics of effect size for group differences in DTD. For group differences in extreme responding, odds ratio (OR) and absolute risk difference were used. When a study had 2 or more relevant groups, we combined the data using methods described in the Cochrane Handbook.²²

Random-effects meta-analyses were undertaken using the DerSimonian and Laird²⁴ estimator. For all outcomes with I ² <40%, we carried out a sensitivity analysis using a fixed effects analysis but this made no substantive difference to the results. Notably, the Profile-Likelihood estimator may have better properties than the DerSimonian and Laird method in some contexts (cases of extreme nonnormal data, ²⁵). We repeated all main analyses where heterogeneity was present with this method and did not observe any substantial deviations from the reported results. Analyses were undertaken using MetaXL²⁶ and STATA version 9.²⁷

As mentioned, a number of studies provided mean delusion ratings and SDs for extreme and nonextreme responders. For these, we calculated the SMD and converted this to an OR following the steps outlined by Borenstein and colleagues.²³ These ORs were then included in the meta-analysis of JTC and delusions, which also included ORs derived from studies reporting rates of extreme responding in people with psychosis who had delusions vs those who were delusion-free.

For the analysis of the relationship between DTD and delusion severity, Pearson’s correlations were converted into Fisher’s Z.²³ Spearman’s correlations were first converted into approximate Pearson’s correlations as outlined in Rupinski and Dunlap.²⁸ In 3 cases where correlations were not available in the original paper or provided, they were taken from Ross and colleagues.¹⁹ Notably, Ross and colleagues derive their correlations by combining results across difficulty conditions (ie, 85:15 and 60:40 ratio), which is in contrast to the other correlations reported here which are based on a particular version of the task. We repeated the analysis comparing correlations between DTD and delusions exchanging the effect sizes obtained from the original papers with those reported in Ross et al¹⁹ and found this made minimal difference to the final aggregated effect size obtained (F _z = −.09 in both cases).

Moderator and Sensitivity Analyses

Three prespecified methodological moderators of effect size were examined: (1) matching of groups, (2) use of blinding, (3) use of practice trials. These moderators were examined for the 2 largest analyses, (1) DTD for psychosis vs nonclinical controls and (2) the number who demonstrate a JTC bias in samples with experiences of psychosis vs controls. The numbers of studies contributing to the other 4 comparisons were smaller and so it would not be possible to look at moderators with the same level of precision. It was noticed that for the studies comparing groups by the number of participants who demonstrated a JTC bias there was variation between using either 1 bead or 2 beads as a cut-off for indicating a JTC bias. We therefore included this variable (1 vs 2 beads) as an additional moderator. Random effects meta-regression was used to test these moderator effects via STATA’s METAREG command, which adopts a restricted maximum-likelihood estimator.²⁹

Risk of Bias and Study Quality

The methodological quality of all studies was assessed independently by 2 raters using an adapted version of the Agency for Healthcare Research and Quality assessment tool (AHRQ; see supplementary material or Taylor et al²¹). This tool allows for the consistent and transparent judgement of study quality parameters such as degree of participant matching, sample size, recruitment procedures and adequate reporting.

The overall quality of the individual outcomes was assessed using an adapted version of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach. The overall GRADE rating (whether high, moderate, low or very low quality) incorporated considerations of quality of data, publication bias, inconsistency and imprecision. Overall data quality was assessed by using the distribution of AHRQ ratings for studies contributing to that particular outcome. Publication bias was assessed using funnel plots for all outcomes with ≥10 studies, following recommendations by Sterne, Egger and Moher.³⁰ Inconsistency was assessed via inspection of heterogeneity and overall direction and magnitude of effect, and imprecision by inspection of effect size, confidence intervals and overall N contributing to the analyses. The specific criteria we used for making AHRQ and GRADE ratings are detailed in the supplementary material.

Registration and Peer-Review of Protocol and Subsequent Changes

The review protocol was peer-reviewed, published²¹ and registered in the public domain (PROSPERO Registration CRD42014007603) before searches, data extraction and analysis were conducted. Subsequent changes included the decision to analyse the comparison between the psychosis sample with nonclinical and clinical control groups separately rather than combining data from these nonclinical and clinical control groups. This decision was made prior to analyses being undertaken, as it was felt it would provide a greater insight into the JTC effect. Furthermore an additional moderator analysis (whether JTC based on 1 or 2 beads) was included. Some planned moderator analyses were dropped due to insufficient data, including consideration of differences in phase of psychosis (ie, first episode vs more established and chronic difficulties) and whether delusions were current or remitted. The effect of different versions of the task (salient vs nonsalient; easy vs hard) will be the focus of a subsequent paper. Further changes are detailed in the supplementary material.

Results

As shown in figure 1, we identified 55 relevant studies. Twelve researchers provided additional information that was not included in their original article and new, unpublished data were provided by 4 authors. Characteristics of included studies and participant demographics are provided in the supplementary material. The majority of studies took place in the United Kingdom, with the remainder occurring in Australia, Canada, Germany, United States, and Hong Kong. A list of excluded studies, with reasons for exclusion, is provided in the supplementary material. In many cases the studies were excluded if they did not use a beads type task but another reasoning task, or they reported data that had already been reported as part of another study.

Fig. 1.

PRISMA flowchart.

Risk of Bias

The overall AHRQ assessment is presented in table 1, and overall GRADE ratings for each outcome are provided in the right-hand column of table 2. A consistent problem with the studies was a lack of a priori sample size justification. Many studies were small and underpowered to detect with adequate power effect sizes of seemingly theoretical significance. With notable exceptions,^31–33 the vast majority of studies did not take steps to mask participant diagnoses from raters, thus increasing the risk of rater bias and expectancy effects. We judged this to be more problematic for studies using the manual rather than computerized versions of the beads task, where experimenter expectations could more easily influence responding.

Table 1.

Overview of Assessment of Study Methodological Quality

Study Ref	Unbiased Selection of Cohort?	Selection Minimises Baseline Differences in Prognostic Factors? a	Sample Size Calculation?	Adequate Description of the Cohort?	Validated Method for Ascertaining Psychosis or Delusions?	Validated Methods for Ascertaining jumping to conclusions?	Outcome Assessments Blind to Clinical Status?	Missing Data Low or Adequately Handled?
Andreou (2013)	Partial	Yes	No	Yes	Partial	Partial	Partialc	Yes
Andreou (unpublished)	Partial	Yes	Nob	Yes	Yes	Partial	Partialc,d	No
Balzan (2012)	Unclear	Yes	No	Yes	Yes	Yes	Partialc	Yes
Balzan (unpublished)	Unclear	No	No	Yes	Yes	Unclear	No	Unclear
Baskak (2015)	Partial	Yes	No	Yes	Partial	Yes	Partialc	Yes
Bentall (2009)	Yes	Partial	No	Yes	Yes	Yes	No	Yes
Buck (2012)	Yes	—	No	Yes	Yes	Yes	No	Yes
Colbert (2010)	Partial	No	No	Yes	Partial	Partial	Partialc	Yes
Conway (2002)	Unclear	Yes	No	No	Partial	Yes	No	Yes
Dudley (1997a)	Yes	Yes	No	Partial	No	Yes	Partialc	Yes
Dudley (1997b)	Partial	Yes	No	Partial	No	Yes	Partialc	Yes
Dudley (2011)	Yes	Yes	No	Partial	Partial	Yes	Partialc	Yes
Evans (2012)	Yes	Yes	No	Partial	Partial	Yes	Partialc	Yes
Falcone (2014)	Yes	No	No	Yes	Yes	Partial	Partialc	Yes
Fear (1997)	Partial	No	No	Partial	Partial	Partial	Partialc	Partial
Fraser (2006)	Unclear	Partial	Yes	Partial	Partial	Yes	No	Yes
Garety (1991)	Partial	Yes	No	Partial	No	Partial	No	Unclear
Garety (2013a)	Partial	—	No	Partial	Yes	Partial	Partialc	Yes
Garety (2013b)	Yes	Yes	No	Yes	Yes	Yes	Partialc	Yes
Huq (1988)	Partial	Partial	No	Partial	No	Partial	No	Yes
Jacobsen (2012)	Yes	Yes	Yes	Yes	Yes	Partial	No	Yes
Krug (2014)	Yes	No	No	Partial	Yes	Yes	Partialc	Unclear
Langdon (2010)	Yes	Yes	No	Partial	Yes	Partial	Partialc	Yes
Langdon (2014)	Partial	Yes	No	Yes	Yes	Yes	No	Yes
Lim (2012)	Yes	Yes	No	Yes	Partial	Partial	No	Yes
Lincoln (2010)	Partial	Yes	No	Yes	Yes	Yes	No	Yes
McKay (2007)	Yes	Yes	No	Yes	Partial	Yes	No	Yes
Menon (2006a, Study 1)	Partial	Yes	No	Yes	Yes	Yes	Partial	Partial
Menon (2008)	Yes	—	No	Yes	Yes	Partial	No	Yes
Menon (2013)	Partial	Yes	No	Yes	Yes	Partial	No	Unclear
Moore (2006)	Partial	Yes	No	Yes	Yes	Partial	No	Partial
Moritz (2005)	Partial	Yes	No	Yes	Yes	Partial	Partialc	Yes
Moritz (2007)	Partial	Yes	No	Yes	Yes	Yes	Yes	Yes
Moritz (2011)	Partial	Yes	No	Yes	Yes	Partial	Partialc	No
Mortimer (1996)	Partial	—	No	No	Yes	Partial	Yes	Yes
Ochoa (2014)	Yes	Yes	No	Partial	Yes	Partial	No	No
Ochoa (unpublished)	Unclear	Unclear	Unclear	Unclear	Unclear	Unclear	Unclear	Unclear
Orem (2009)	Yes	Yes	Yes	Yes	Yes	Yes	No	Yes
Ormrod (2012)	Yes	Partial	No	Partial	Partial	Partial	Partialc	Yes
Peters (2006)	Yes	No	No	Partial	Partial	Partial	No	Yes
Peters (2008)	Partial	Yes	No	Partial	Partial	Partial	No	Yes
Rausch (2014)	Partial	Yes	No	Yes	Yes	Yes	Partialc	Yes
Ross (2011)	Yes	Yes	Yes	Yes	Yes	Partial	Partialc	Yes
So (2012, PRP Study 4)	Yes	—	Nob	Yes	Yes	Partial	Partialc	No
So (2014)	Partial	—	No	No	Yes	Partial	Partialc	Yes
So (2015)	Yes	No	No	Yes	Partial	Partial	No	Yes
Startup (2008)	Partial	Yes	No	Yes	Yes	Partial	No	Yes
Van Dael (2006, CoP Study 1)	Yes	No	Yes	Yes	Yes	Partial	Partialc	Yes
Van der Gaag (2013)	Unclear	Unclear	Yes	Yes	Unclear	Partial	Unclear	Yes
Westermann (unpublished)	Unclear	Partial	Unclear	Partial	Unclear	Unclear	Unclear	Unclear
Waller (2011)	Yes	Unclear	No	Yes	Yes	Partial	No	Yes
Warman (2007)	Partial	No	No	Yes	Partial	Yes	No	Yes
Warman (2013)	Partial	—	No	Yes	Yes	Partial	No	Yes
Wittorf (2012)	Yes	Partial	No	Yes	Yes	Partial	Partialc	Yes
Young (1997)	Partial	Yes	No	Yes	No	Yes	No	Yes

Note: ^aGroup comparison studies only.

^bJTC a secondary outcome so power calculation would not be expected.

^cComputerized beads classified as partial blindness, since relatively less researcher involvement in comparison to manual beads tasks.

^dRaters were blind to treatment allocation, but not clinical status.

Table 2.

Summary of Meta-Analysis Results

Outcome	No. of Included Studies	Psychosis	Control N, events/N	Hedges’s g or r (95% CI)	Weighted Mean Difference (95% CI)	Odds Ratio (95% CI)	Absolute Risk Difference (95% CI)	Heterogeneity for g, r, or OR: I 2 , Chi 2 P-value	Quality (GRADE)
		N, events/N
Difference in draws to decision: psychosis vs healthy individuals.	33	962	973	−0.53 (−0.69, −0.36)	−1.37 (−1.82, −0.92)a	—	—	66%, P < .001	Moderate
									-1 quality (lack of blinding & power calculations)
Difference in DTD: psychosis vs other mental health problems.	13	349	318	−0.58 (−0.80, −0.35)	−1.66 (−2.27, −1.04)	—	—	46%, P = .033	Moderate
									-1 quality (lack of blinding & power calculations)
Difference in DTD: psychosis with delusions vs psychosis without delusions	8	256	200	−0.29 (−0.48, −0.09)	−0.59 (−1.01, −0.09)	—	—	0%, P = .720	Low
									-1 imprecision
									-1 quality (lack of power calculations)
Correlation between DTD and delusion severity in people with psychosis and delusions.	18	794	—	r = −.09 (−.21, .03)	—	—	—	54%, P = .003	Very low
									-1 imprecision
				g = −0.18 (−0.43, 0.06)
									- 1 inconsistency
									-1 quality (lack of blinding & power
									calculations)
Difference in jumping to conclusions: psychosis vs healthy individuals.	22	393/817	177/614	—	—	3.82 (2.69, 5.43)	0.27 (0.20, 0.34)	44%, P = .01	Moderate
									-1 quality (lack of power calculations)
Difference in JTC: psychosis vs other mental health problems.	4	60/99	29/103	—	—	5.55 (2.32, 13.28)	0.38 (0.27, 0.49)	20%, P = .29	Low
									-1 imprecision
									-1 quality (lack of power calculations)
JTC and presence or severity of delusions in psychosis.	14	338	432	—	—	1.52 (1.12, 2.05)	—	13%, P = .31	Moderate
									-1 quality
									(lack of power calculations)

Note: ^a k = 32, psychosis N = 942, healthy individuals N = 958

Most studies matched participants on key variables such as age and gender, provided adequate information regarding sample characteristics and used valid and reliable measures to rate psychotic symptoms and establish diagnosis. Although all trials used the beads task or equivalent to assess JTC (this was an inclusion criteria), most did not provide participants with a practice trial, which is recommended to minimise the confounding effects of confusion regarding task instructions.³⁴

Draws to Decision

As shown in Table 2, there was moderate quality evidence that people with psychosis required significantly less information to form conclusions than both healthy participants and participants with nonpsychotic mental health problems. The differences were moderate in magnitude (see figure 2), and corresponded to people with psychosis requiring, on average, 1·4 to 1·7 fewer beads to make a decision.

Fig. 2.

Forest plots for analyses of draws to decision (DTD).

(A) Forest plot for comparison of psychosis group vs nonclinical control on DTD.

(B) Forest plot for comparison of psychosis group vs clinical control on DTD.

(C) Forest plot for comparison of individuals with psychosis and delusions vs those with without current delusions on DTD (mean differences).

(D) Correlation between draws to decisions and delusion severity in people with psychosis and delusions.

People with delusions required significantly less information than people with psychosis who did not have delusions, but the difference was small and the estimate imprecise. Those with current delusions required on average 0·6 fewer beads to make a decision than those without. A small trend-level inverse correlation was observed between delusion severity and DTD for those with delusions, but this was affected by significant heterogeneity and imprecision.

Extreme Responding

Moderate quality evidence suggested the odds of extreme responding in people with psychosis were, respectively, almost 4 times and 6 times higher than the odds of extreme responding by healthy participants and participants with nonpsychotic mental health problems. Approximately 29% of healthy participants and 38% of people with nonpsychotic mental health problems made a decision after 1–2 beads, compared to 48% to 60% of people with psychosis. There was also moderate quality evidence of a small significant association between extreme responding and delusions in people with psychosis (figure 3).

Fig. 3.

Forest plots for analyses of extreme responding (JTC).

(A) Forest plot for comparison of psychosis group vs nonclinical control on number who demonstrate extreme responding.

(B) Forest plot for comparison of psychosis group vs clinical control on number who demonstrate extreme responding.

(C) Forest plot for comparison of individuals with psychosis and delusions vs those with without current delusions on number who demonstrate extreme responding.

Moderator Analyses

Whilst blinding of researcher to participant status (eg, psychosis or control) had been pre-specified as a potential moderator of interest, only 3 studies^31–33 reported using blinding therefore this analysis was not possible. Results of the moderator analyses for the other variables are presented in table 3. None were significant with the exception of the presence of multiple or practice trials when the outcome was extreme responding (psychosis vs nonclinical control). The OR changed from 3.01 to 6.69 between studies without practice or multiple trials and those with practice or multiple trials.

Table 3.

Result of Meta-Regression Moderator Analyses

Analysis	Moderator	N	B (SE)	P
DTD: psychosis vs. nonclinical controls	Matching of groups	22/32	−0.18 (0.19)	.36
	Multiple/ practice trials	17/32	−0.06 (0.18)	.74
		N	OR (SE)	p
Number who JTC: psychosis vs nonclinical controls	Matching of groups	13/21	1.32 (0.44)	.41
	Multiple/ practice trials	8/21	2.17 (0.67)	.02
	One vs 2 bead cut-off	15/21	0.63 (0.28)	.32

Note: All moderators binary, 0 = No matching, multiple/practice trials or one-bead cut-off. N represents number of studies where the moderator = 1.

Publication Bias

Inspection of funnel plots did not reveal evidence of publication bias (see supplementary material).

Discussion

Our analysis confirms that people with nonaffective psychosis require less evidence to make decisions, and are more likely to make extreme decisions, than both healthy participants and participants with other types of mental health problems. This latter finding provides good evidence that the JTC bias is specifically associated with psychosis. That this bias also involves a degree of delusion-specificity is supported by the finding that those with current delusions require less information to make decisions than those without, and by the significant association between extreme responding and delusions. Although only small delusion-specific effects were observed, this is not unexpected if we consider people with psychosis to have a generally increased vulnerability to forming delusions, even when not currently delusional. Alternatively, that a large number of healthy participants demonstrate an extreme responding style,^8,18 and that 40%–50% of those with psychosis do not,⁸ indicates JTC is neither a sufficient or necessary cause of psychosis or delusions. Overall, the evidence is largely consistent with Garety and Freeman’s suggestion that the JTC bias may act as a “trait representing liability to delusions” (p.329: 17).

The inverse correlation between delusion severity and draws to decision was small and of only trend-level significance. This may be a result of studies requiring participants to have a certain severity of delusions before study entry that may have artificially constrained variance in delusion ratings. Nonetheless, our results replicate those of Ross and colleagues who found only a small nonsignificant association between self reported delusional ideation and draws to decision in their meta-analysis of clinical samples who reported current delusional beliefs.¹⁹

The results of the moderator analyses indicated the importance of practice, and multiple trials in the likelihood of people with psychosis demonstrating JTC. Previous work has shown that people with psychosis may perform poorly on the beads task owing to miscomprehension of the tasks demands.³⁴ Our finding may mean that with more practice or multiple trials there is less chance of miscomprehension and we are seeing a more representative picture of performance. There was no evidence for the contribution of other hypothesized moderators, and whether 1 or 2 beads is used seems to be of little importance, and both are measures of extreme responding.

Does Hasty Decision Making Cause Psychosis and Delusions?

According to Bradford Hill’s influential criteria, claims of causation are more likely to be justified if the association between the relevant variables is specific, strong and consistent across different populations, settings and paradigms.³⁵ Although the relationship between JTC and psychosis appears to largely satisfy these criteria, the relationship with delusions is weaker and arguments both for and against specificity can be made. That the odds of extreme responding in psychosis (OR 3.82, 95% CI 2.69, 5.43 vs healthy participants) are higher than those for childhood adversity in psychosis (OR 2.78, 95% CI 2.34, 3.31,³⁶) illustrates the strength of the relationship. That almost 90% of the 33 studies included in our largest analysis found data-gathering to be reduced in psychosis, despite variation in country, population and methodologies, is evidence for consistency. The central role given to JTC in several cognitive and neuropsychological models of psychotic symptoms may satisfy Bradford Hill’s additional proposals that claims of causality are more likely to be justified if they fit with existing theory and facts.

Bradford Hill³⁵ also noted that a causal claim is more likely to be true if the hypothesized causal factor precedes the event of interest, if there is a dose-response relationship between the two, and if the claim is supported by experimental evidence. Whether JTC precedes psychosis or delusions remains unclear since no studies, as far as we know, have examined the long-term outcome of healthy individuals who demonstrate a JTC bias. Findings of temporal relationships are mixed in clinical populations, with JTC being associated with psychosis or delusion severity at follow-up in some studies^37–41 but not others.^42,43 Three experimental studies have found that JTC interventions led to reductions in both JTC and delusion severity,^9,44,45 but one found no evidence that the reduction in JTC mediated the change in delusional ideation after adjusting for baseline confounders.⁹

Recommendations

Future research should consider the causes of the JTC bias (see ref.⁸ for a review) and whether and how it is linked to negative symptoms,³³ insight,⁴⁶ neurocognitive factors such as IQ,⁴⁷ working memory,⁴⁸ prognosis and treatment response. More broadly, studies investigating the role of JTC in relation to other known causes and mediators of psychotic symptoms such as childhood adversity, urbanicity, cannabis use, and genetic predisposition would be valuable.

Firmer conclusions about the potential causal role of JTC in psychosis and delusions require adequately powered longitudinal studies focused on understanding both the temporal sequence of JTC and psychotic symptoms and the development and consequences of hasty decision-making in the general population. Suitably-powered randomized controlled trials of JTC manipulation are essential for testing causal claims, and these should consider comparing the effects on delusions of different intensities of JTC-focused interventions. Although such studies will require large samples, this could be offset by enriching the sample—ie, randomising only those who display both delusions and extreme responding.

Future observational and experimental studies of the JTC bias in psychosis would benefit from pre-registration of their hypotheses and methods in the public domain. As with clinical trials, this will allow greater transparency and assessment of selective reporting bias, as well as differentiation of a priori and post hoc hypothesis testing. In our main analysis 31/33 studies were underpowered to detect small (d = 0.2) and medium (d = 0.5) effect sizes (n = 394 and 64 per group respectively). We therefore recommend that future studies implement and adhere to a priori calculations of sample size, and that these provide sufficient power to detect effects of theoretical significance. To minimise risk of bias, JTC studies should incorporate rater blinding and automated JTC tasks where possible. Practice trials should be used to avoid participant miscomprehension and greater attention should be given to careful matching of participants on key demographic variables (age, gender, education, ethnicity). Multiple measures (ideally self report and observer completed) of delusional beliefs, and psychotic symptoms would be helpful.

Strengths and Limitations

A particular strength of our study was our success in receiving unpublished and new data. We were able to contact all authors except 2 from initial searches in order to obtain extra information. Twelve authors provided unpublished extra details from published articles (eg, clarification over task used, correlations, and study design). Four authors were able to provide new, currently unpublished data. In addition to this, the benefit of having the protocol for this systematic review and meta-analysis published²¹ was such that several authors contacted the corresponding author with their work that was either in print or soon to be published.

However, resource constraints meant we were limited to English language studies. The variety of different hypotheses being assessed in the included studies meant that usable JTC data were not always adequately reported. We were able to resolve queries in a number of cases via correspondence with authors, but not all. We think it is unlikely that data excluded for this reason would have had a substantial impact upon the reported effect sizes, since most of these were relatively robust and precise.

Of course, our analysis is a synthesis and summary of reported effect sizes. We did not carry out a meta-analysis of individual participant data, which is an approach that may help reduce the risk of selective reporting bias.¹⁹ It is important to note as well that our review has not been able to raise and address the theoretical or methodological issues associated with understanding reasoning and delusions. The discussion of these points is important and is given fuller consideration elsewhere.⁴⁹ We have been unable to comment on whether the findings in the present study support a particular theory as to what leads people with psychosis and delusions to exhibit this data gathering bias. Moreover, we have been unable to engage in discussion on the challenges of undertaking research investigating reasoning in people with and without delusions. Briefly, in terms of context, the value of the beads task was that it involves reasoning about content that is seemingly unconnected to the content of the delusional belief. However, this has limitations as well, in that it may only to a limited degree reflect the actual reasoning involved in delusion formation and maintenance.

We recognise as well that the measurement of psychosis and delusions differed between studies. We considered this in our assessment of study and outcome quality. Importantly, 6 out of 33 included studies (18%) contributing to our largest analysis relied on chart assessment or diagnosis of referring clinician rather than a standardized observer interview based assessment.

Finally, there were an insufficient number of studies with required methodological features (eg, researcher blinding), which meant we were unable to conduct a number of planned moderator analyses.

Conclusion

Our review provides definitive evidence that people with psychosis make decisions on the basis of less evidence, and have a more extreme reasoning style than people with other mental health conditions or healthy controls. This is linked to a greater probability of having delusions. Although hasty decision making is not necessary or sufficient for delusional beliefs, it may still play a causal or mediating role, and given the clear association targeting this style may help reduce delusional ideation in psychosis.

Supplementary Material

Supplementary material is available at http://schizophreniabulletin.oxfordjournals.org.

Funding

The authors received financial support from their employers for travel and accommodation. The Lena Teague Bequest fund supported costs of open access publication of our review protocol²¹ and for the current article.