A Bias Against Disconfirmatory Evidence Is Associated With Delusion Proneness in a Nonclinical Sample

Abstract

Previous work has suggested that a bias against disconfirmatory evidence (BADE) may be associated with the schizophrenia spectrum. The current investigation focused on whether a BADE (1) overlaps with traditional measures of memory and executive functions or selectively taps into a unique aspect of cognition and (2) is correlated with delusional ideation but not with other aspects of schizotypy. Sixty-eight undergraduate students were administered the Schizotypal Personality Questionnaire (SPQ), the BADE test, the Rey Auditory Verbal Learning Test (RAVLT), the Wisconsin Card Sorting Test (WCST), the Trail Making Tests A and B (TMT), and tests used to estimate IQ. Factor analysis of all cognition measures resulted in a 6-factor solution, 4 of which reflected the 4 domains of neuropsychological tests (WCST, RAVLT, TMT, and IQ), and 2 of which reflected different aspects of the BADE test: Initial Belief and Integration of Disconfirmatory Evidence. This solution suggests that BADE measures were independent from the other cognitive domains measured. Integration of Disconfirmatory Evidence was the only factor that correlated with delusion-content subscales of the SPQ, providing support for the contribution of a BADE to delusional ideation.

Introduction

Delusions are defined as fixed false beliefs not amenable to contrary evidence, and are hallmark symptoms of schizophrenia spectrum disorders. Although early theorists conceptualized delusions as irreducible,¹ cognitive neuropsychiatric investigations have exposed aspects of the cognitive architecture that may contribute to the formation of delusions, such as deviances of attributional style and a data-gathering bias.^2–4 In past work, we have repeatedly demonstrated that a bias against disconfirmatory evidence (BADE) may contribute to the formation and/or maintenance of delusions in schizophrenia.^5–7

In these previous studies, a BADE was measured by gradually presenting participants with a sequence of either 3 pictures or sentences that increasingly disambiguated a scenario (eg, initial information: a girl cannot fall asleep; it gradually becomes clear that this is due to excitement over the impending Christmas morning). Participants were asked to rate the plausibility of 4 interpretations of each scenario. Relative to control groups, people with schizophrenia and/or delusions were less willing to downrate their originally favored interpretations (eg, the girl is nervous about an examination) when presented with information incompatible with this interpretation (ie, she is thinking about her presents). In contrast, no group differences emerged for integration of “confirmatory” evidence (ie, patients and controls rated true interpretations similarly). It is important to note that these studies were conducted using pictures or words that did not refer to delusional themes, and in some studies, the magnitude of the BADE corresponded with the severity of delusions.^6,7

Patient studies invoke a number of confounds, in that groups differing on delusions and/or the presence of schizophrenia may differ in other systematic ways, including medication status, and effects of hospitalization and/or stigmatization. Moreover, impairments in memory, intelligence, and executive functions characterize schizophrenia,⁸ and may also differ across groups. An alternative approach that circumvents these concerns is to study delusion proneness in a nonclinical sample. In a recent article, we reported that groups of university students scoring at opposite extremes on the total score for Schizotypal Personality Questionnaire (SPQ)⁹ differed on the BADE, with the BADE being increased in the group scoring high on the SPQ.¹⁰ Although this group-comparison approach provided a clear schizotypy-based group comparison of BADE measures, the multivariate correlational approach utilized in the current article allows assessment of whether a BADE is secondary to more general neuropychological deficits (eg, memory, intelligence, and executive functions), and whether the overlapping or unique aspects of a BADE are directly linked to the delusional ideation aspect of schizotypy.

To meet this purpose, we employed a multivariate correlational design in a nonclincal sample. Undergraduate students completed a questionnaire on schizotypal traits and completed a neuropsychological test battery involving measures of memory, executive function, IQ, and the BADE test. The cognitive measures were analyzed by an exploratory factor analysis, and the resulting factors were correlated with 9 aspects of schizotypy. Our hypotheses were that (1) the BADE measures would load together instead of loading with other aspects of cognition and (2) the BADE measures would correlate with delusional ideation but not other aspects of schizotypy.

Methods

Participants

In total, 535 undergraduate students were screened for schizotypal traits with the SPQ.⁹ In accordance with previous schizotypy studies, individuals with SPQ scores greater than the 90th percentile and those with SPQ scores less than the 10th percentile were requested to participate in the second session in exchange for course credits or a small remuneration. Of the 103 subjects contacted, 75 agreed to participate. Data were lost for 6 participants due to technical difficulties. Thirty-seven high-schizotypal and 32 low-schizotypal individuals completed the full neuropsychological battery. The high- and low-schizotypal groups were found to differ on the BADE measures, which is reported elsewhere.¹⁰ In the current article, using the same subjects, we focus on potential overlap between BADE and more widely studied neuropsychological measures, and whether overlapping or unique aspects of a BADE correlate with delusional ideation subscales of the SPQ (as opposed to groupings based on total SPQ score).

A medical questionnaire was administered and all participants claimed to be free of previous severe head trauma and mental illness. All participants gave written informed consent before the screening and test sessions in accordance with the Simon Fraser University Research Ethics Board.

Measures

The SPQ is a 74-item yes-no self-report inventory consisting of 9 subscales modeled after the 9 DSM-III-R criteria of schizotypal personality disorder.¹¹ The Quick Test¹² and the North American Adult Reading Test (NAART)¹³ served as estimates of intelligence. Executive function was measured using the computerized version of the Wisconsin Card Sorting Test (WCST),¹⁴ and the Trail Making Tests A and B (TMT).¹⁵ The Rey Auditory Verbal Learning Test (RAVLT)¹⁶ was used to assess verbal memory. The core parameters for the WCST are the number of categories completed and perseverative errors that were employed to assess overlap with the “switching set” aspect of integrating disconfirmatory evidence. Both TMT completion times were entered as a measure of psychomotor speed and cognitive flexibility. The RAVLT measures of total recall, delayed recall, and recognition were entered as memory measures. With 2 IQ screens, this resulted in 9 variables being entered into the factor analysis for traditional neuropsychological tests.

The BADE test employed here was based on previous versions of this test, and the details regarding administration are available elsewhere.^6,7 The current version consisted of 30 written delusion-neutral scenarios. The plausibility of 4 interpretations was rated for each scenario. Each scenario had 1 “true” interpretation, 2 “lure” interpretations, and 1 “absurd” interpretation. The true interpretation appeared implausible initially but became more reasonable as additional information was revealed. The lure interpretations appeared plausible initially but became implausible as additional information was revealed, and consisted of one lure designed to refer to an “emotional” interpretation, and one designed to refer to a “neutral” interpretation (examples below). The absurd interpretation was designed to be implausible at all stages. Disconfirmatory and confirmatory evidence was introduced by 3 successive sentences (eg, “Jenny can't fall asleep”; “Jenny can't wait until it is finally morning”; and “Jenny wonders how many presents she will find under the tree”). After presentation of each sentence, ratings for each interpretation “Jenny is nervous about her exam the next day” (neutral lure); “Jenny is worried about her ill mother” (emotional lure); “Jenny is excited about Christmas morning” (true); and “Jenny loves her bed” (absurd) would be updated based on all evidence.

The BADE scenarios were presented via Microsoft Internet Explorer 6.0. A 0–10 rating scale with a scroll bar was positioned beneath each interpretation, with nominal ratings “Poor,” “Possible,” “Good,” and “Excellent” centered on the following rating scale numbers: 0, 3.5, 6.5, and 9.5, respectively. Ratings were entered via mouse click. One practice trial was given to allow participants to become familiar with the rating scale and scroll bar. Test duration was approximately 20 minutes.

A total of 6 BADE measures were entered into the exploratory factor analysis: the plausibility ratings for emotional and neutral lure items after sentence 1, 2, and 3 (3 emotional and 3 lure items = 6 measures). In previous work, the ratings after sentence 2 and/or 3 were subtracted from those after sentence 1,^6,7 but in the current study all ratings were entered into the factor analysis to explore for independence between ratings following all 3 sentences.

Results

For the factor analysis, the BADE test (6 measures) and the neuropsychological variables (9 measures) were submitted to an exploratory principal component analysis with varimax rotation. A 6-factor solution was suggested by both a scree plot and the Kaiser-Guttman criterion (eigenvalues > 1). The results are listed in table 1. A clear simple structure is evident with independent factors emerging for BADE Initial Belief (BADE ratings primarily after sentence 1 and partly after 2; 20.25% of total variance), BADE Integration of Disconfirmatory Evidence (BADE ratings partly after sentence 2 and primarily after sentence 3; 17.19% of total variance), Memory (RAVLT; 13.06% of total variance), Executive Functioning (WCST; 11.93% of total variance), IQ (11.92% of total variance), and TMT (10.75% of variance). Factor scores were saved for correlation with the SPQ. In order to ensure that the cognitive constructs were not artificially uncorrelated by an orthogonal component rotation, correlations between factor-based composite scores (ie, the sum of the measures that dominated each factor, in place of factor scores) were computed using normalized and averaged measures corrected for directionality. This resulted in a significant correlation between Memory and TMT, r(66) = −0.27, P < .05, but all other correlations were not significant (all P’s > .10), justifying the orthogonal component solution, and providing further evidence that the BADE measures are independent from other cognitive domains. Correlation of the BADE scores with the neuropsychological variables provided evidence that the BADE does not reflect a more general cognitive function (see table 2).

Table 1.

Factor Loadings for Neuropsychological and BADE Variables

	Initial Belief	Integration of Disconfirmatory Evidence	Memory	Executive	IQ	Trails
NAART (errors)	−0.05	−0.05	−0.08	0.04	−0.94	−0.03
Quick Test	0.13	0.00	0.08	0.09	0.92	−0.11
RAVLT total recall	0.01	−0.10	0.83	0.06	0.09	−0.14
RAVLT delayed recall	−0.03	0.01	0.84	0.12	0.01	−0.05
RAVLT recognition	0.04	0.09	0.69	0.02	0.05	−0.08
Trail A time	0.02	−0.02	−0.06	−0.04	−0.01	0.90
Trail B time	−0.03	−0.01	−0.21	−0.06	−0.07	0.86
WCST perseverative	0.00	0.05	−0.09	−0.94	−0.13	−0.01
WCST categories	−0.02	0.10	0.12	0.93	−0.09	−0.13
Neutral lure 1	0.96	−0.12	0.04	−0.02	0.06	0.03
Neutral lure 2	0.72	0.60	0.05	0.08	0.00	−0.08
Neutral lure 3	−0.02	0.97	0.04	0.02	0.00	−0.02
Emotional lure 1	0.97	−0.03	−0.04	−0.04	0.09	0.02
Emotional lure 2	0.79	0.53	0.00	0.01	0.09	−0.04
Emotional lure 3	0.10	0.97	−0.04	0.02	0.04	0.01

Note: All loadings above 0.50 are set in bold font.

Table 2.

Correlations of Neuropsychological and BADE Variables

	Neutral Lure 1	Neutral Lure 2	Neutral Lure 3	Emotional Lure 1	Emotional Lure 2	Emotional Lure 3
NAART (errors)	−0.11	−0.01	−0.05	−0.13	−0.11	−0.08
Quick Test	0.14	0.08	0.00	0.15	0.16	0.02
RAVLT total recall	0.02	0.05	−0.06	−0.03	0.01	−0.13
RAVLT delayed recall	0.01	0.06	0.05	−0.07	−0.01	−0.03
RAVLT recognition	0.07	0.03	0.09	0.02	0.02	0.06
Trail A time	0.03	−0.08	−0.03	0.05	−0.03	0.00
Trail B time	−0.01	−0.06	−0.05	−0.02	−0.05	−0.01
WCST perseverative	0.01	−0.02	0.01	0.04	0.02	0.02
WCST categories	−0.07	0.02	0.09	−0.08	−0.03	0.06

Note: No correlations were significant at the P < .05 level.

When correlating the factor scores with 9 subscales of the SPQ, the only factor that correlated with delusional aspects of the SPQ was Integration of Disconfirmatory Evidence (see table 3). Specifically, Integration of Disconfirmatory Evidence correlated with the SPQ subscales Ideas of Reference, Excessive Social Anxiety, Odd Beliefs or Magical Thinking, Unusual Perceptual Experiences, and Odd or Eccentric Behavior. The only other significant correlation between any of the factors and the cognitive variables was between IQ and the SPQ subscale No Close Friends.

Table 3.

Correlation of Schizotypal Personality Questionnaire (SPQ) Subscales with Neuropsychological and Integration of Disconfirmatory Evidence Factors

	Initial Belief	Integration of Disconfirmatory Evidence	Memory	Executive	IQ	Trails
Ideas of Reference	−0.07	0.24*	0.12	−0.10	0.02	0.03
Excessive Social Anxiety	0.01	0.33**	0.17	−0.08	−0.08	0.11
Odd Beliefs or Magical Thinking	0.04	0.35**	0.08	−0.02	−0.05	0.02
Unusual Perceptual Experiences	0.10	0.30*	0.20	−0.08	−0.06	0.13
Odd or Eccentric Behavior	−0.01	0.25*	0.17	−0.13	0.09	0.03
No Close Friends	−0.02	0.17	0.15	−0.14	−0.27*	0.02
Odd Speech	0.04	0.17	0.05	−0.14	−0.05	−0.02
Constricted Affect	0.07	0.19	0.02	−0.09	−0.21	0.04
Suspiciousness	−0.02	0.12	0.08	−0.10	0.01	0.13

^*P < .05, **P < .01, 2 tailed.

Discussion

Previous work has suggested that a generalized BADE may be associated with the schizophrenia spectrum disorders. As outlined in the introduction, most studies compared patient groups with healthy control groups, invoking interpretational difficulties. The current investigation of schizotypy focused on whether the BADE (1) taps into an aspect of cognition not measured by other more traditional neuropsychological tests and (2) is correlated with delusional ideation and not other aspects of schizotypy. The results suggest that BADE measures are independent from the other cognitive domains measured here, and that Integration of Disconfirmatory Evidence correlates with aspects of delusional ideation as measured by the SPQ. The measures of memory, IQ, executive functions, and mental flexibility did not correlate with any SPQ subscales.

The 2 most prominent factors emerging from our multivariate analysis were dominated by BADE measures; together, Initial Belief and Integration of Disconfirmatory Evidence accounted for 37.44% of the total variance in all cognitive measures. In our past studies, the separation of Initial Belief and Integration of Disconfirmatory Evidence were achieved manually by subtracting ratings for sentence 2 and/or 3 from ratings after sentence 1. In the current-factor solution, Initial Belief and Integration of Disconfirmatory Evidence emerged as independent constructs, and the pattern of loadings suggests that responses on lure items after sentence 1 are independent from those that determine responses after all information is presented after sentence 3, and that ratings after sentence 2 are a mixture of both. We interpret the ratings of lure items following sentence 3—when all disconfirming information is available—as primarily determined by the cognitive operations involved in Integration of Disconfirmatory Evidence.

The finding that WCST perseverative responses and categories completed did not load onto the same factor as the BADE, and did not correlate with the BADE measures, suggests that, as we previously assumed, a BADE does not reflect a more general integration deficit, or mere perseveration of responses.^6,7 In other words, the present Integration of Disconfirmatory Evidence effect is not likely better accounted for by a deficit in global executive functioning, or inability to override a previously learned rule.

Although the BADE component did not correlate with the “negative” aspects of schizotypy (eg, No Close Friends and Constricted Affect), the specificity of its association with delusional ideation was not as strong as expected, and association with more general “positive” schizotypy was evident. For example, correlations with Excessive Social Anxiety and Unusual Perceptual Experiences were observed as well as with Ideas of Reference and Odd Beliefs or Magical Thinking. In this regard, it is helpful to focus on the individual SPQ items that correlate most highly with that factor, rather than referring to the suggested subscale structure of the SPQ. In table 4 we have listed the individual SPQ items that showed the highest correlations with the Integration of Disconfirmatory Evidence factor, sorted by r value. All can be classified as either discomfort with strangers (SPQ items 38, 2, 29, and 71) or belief in the paranormal (SPQ items 12, 30, 13, and 55). One way to interpret this result is that a BADE may lead to a belief in (and comfort with) “paranormal” beings and forces, and a reciprocal rejection of physically present but unfamiliar people. Another possibility is that a BADE, discomfort with strangers, and belief in the paranormal, are all indicators of being at risk to develop psychosis,¹⁷ and are clustering together with people in a “healthy” sample who are at risk.

Table 4.

Individual SPQ Items Correlating Most Highly with the Integration of Disconfirmatory Evidence Factor, Sorted by r Value

r	SPQ Scale	P Value	SPQ Item
0.37	ESA	0.002	38. Do you often feel nervous when you are in a group of unfamiliar people?
0.34	ESA	0.004	2. I sometimes avoid going to places where there will be many people because I will get anxious.
0.33	ESA	0.006	29. I get anxious when meeting people for the first time.
0.32	OBMT	0.008	12. Do you believe in telepathy (mind reading)?
0.30	OBMT	0.011	30. Do you believe in clairvoyancy (psychic forces, fortune telling)?
0.30	UPE	0.014	13. Have you ever had the sense that some person or force is around you, even though you cannot see anyone?
0.29	OBMT	0.017	55. Have you ever felt that you are communicating with another person telepathically (by mind reading)?
0.27	ESA	0.024	71. I feel very uneasy talking to people I do not know well.

Note: ESA, Excessive Social Anxiety; OBMT, Odd Beliefs or Magical Thinking; UPE, Unusual Perceptual Experiences.

The Integration of Disconfirmatory Evidence factor did not correlate with the Suspiciousness subscale, which may appear to be contrary to expectations, as BADE is expected to be associated with (paranoid) delusions. However, the Suspiciousness subscale includes items such as “I feel I have to be on my guard even with friends,” perhaps relating more to antisocial behavior than to delusions (r = 0.03 with the Integration of Disconfirmatory Evidence factor for this item). Better parallels to paranoid delusions for schizotypal traits may be located on the Excessive Social Anxiety subscales (“Do you ever get nervous when someone is walking behind you?”) and the Ideas of Reference subscale (eg, “When you see people talking to each other, do you often wonder if they are talking about you”), items that correlated with the Integration of Disconfirmatory Evidence factor at 0.24 and 0.21, respectively.

A limitation of the present study is that the sample was specifically selected to include individuals with only high- and low-schizotypal traits. This also may have led to the nonspecific moderate correlations for the Integration of Disconfirmatory Evidence factor with all the subscales of the SPQ, and a significant correlation with Odd or Eccentric Behavior, the latter typically being considered more reflective of the disorganization syndrome than delusional ideation.^11,18 People with high SPQ scores did display high delusional ideation, but may also display other aspects of schizotypy. Future research will be necessary to determine if the reported relationships hold over the full range of schizotypal phenomenology, or whether a BADE is associated with more general “schizotypy” or “positive schizotypy” factors than with delusional ideation proper.

Another limitation was the relatively small sample size. With 15 variables and 69 subjects, the subject-to-variable ratio was 4.6:1. Although this ratio falls just under the recommended minimum criterion of 5:1,¹⁹ and short of the more conservative and widely cited 10:1 ratio,²⁰ the factor structure reported in table 1 agrees with theoretical expectation, and is a good demonstration of simple structure,²¹ making it unlikely to have arisen due to spurious correlations between items. The 4 factors reflecting neuropsychological constructs appear to reflect variance shared between test-specific measures, which facilitated domain-specific correlations with the SPQ subscales. To alleviate concerns that the BADE factors were affected by shared method variance in the overall factor structure, the analysis was rerun using only single indicators from each neuropsychological domain. As may be expected from observation of table 2 this did not affect the BADE factors or the overall set of results reported above. However, future work with larger samples will be required to determine whether the reported factor structure is stable.

These data place BADE among other cognitive biases, such as “jumping to conclusions,”² “liberal acceptance,”²² “attributional bias,”³ “theory of mind,”⁴ and “knowledge corruption,”²³ as candidates for dysfunctional aspects of the cognitive architecture that underlie delusions. In agreement with the fully dimensional account of the schizophrenia spectrum,²⁴ many of these biases have also been demonstrated in healthy people displaying schizotypy.^25–28 The implication is that in their most severe forms, these cognitive biases may combine to contribute to the formation of the delusional aspects of psychosis, and future studies may investigate combinations of these measures that may emerge as clear prodromal features of schizophrenia. Our group has begun to translate knowledge of these cognitive biases to patients with schizophrenia in the framework of a group intervention,²⁹ with the goal of ameliorating positive symptoms. However, overlaps and redundancies between these cognitive biases remain unknown. Multivariate designs investigating associations between these measures and overlap with the symptoms of schizophrenia are required to assess this potentially providing insight into the nature of the underlying neurocognitive systems.