Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Aug 1.
Published in final edited form as: Dev Psychopathol. 2018 Mar 6;31(1):379–392. doi: 10.1017/S0954579417001912

Core Beliefs in Healthy Youth and Youth at Ultra High-Risk for Psychosis: Dimensionality and Links to Depression, Anxiety, and Attenuated Psychotic Symptoms

Henry R Cowan 1, Dan P McAdams 2, Vijay A Mittal 3
PMCID: PMC6127011  NIHMSID: NIHMS923975  PMID: 29506584

Abstract

Cognitive theory posits that core beliefs play an active role in developing and maintaining symptoms of depression, anxiety, and psychosis. This study sought to comprehensively examine core beliefs, their dimensionality, and their relationships to depression, anxiety, and attenuated psychotic symptoms in two groups of community youth: a group at ultra-high risk for psychosis (UHR; n = 73, mean age = 18.7), and a matched healthy comparison group (HC; n = 73, mean age = 18.1). UHR youth reported significantly more negative beliefs about self and others, and significantly less positive beliefs about self and others. HC youth rarely endorsed negative self-beliefs. Indeed, exploratory factor analyses found that HC negative self-beliefs did not cohere as a single factor. We hypothesized specific links between core beliefs and symptoms based on cognitive models of each disorder, and tested these links through regression analyses. Results in the HC group were consistent with the proposed models of depression and anxiety. Results in the UHR group were consistent with proposed models of depression and negative psychotic symptoms, somewhat consistent with a proposed model of positive psychotic symptoms, and not at all consistent with a proposed model of anxiety. These findings add to a growing developmental literature on core beliefs and psychopathology, with important clinical implications.

Keywords: Psychosis prodrome, core beliefs, depression, anxiety, self-concept

Introduction

Cognitive theory posits that core beliefs are key factors in the development and persistence of psychopathology (Beck, 2011). These rigid, inflexible, strongly-held beliefs about the self, others, the world, or the future underlie appraisals of moment-to-moment experience. When core beliefs are negative (e.g., “I am worthless”), these appraisals can be dysfunctional (e.g., “I failed the test because I'm stupid”). In this model, core beliefs create a positive feedback loop. When a person interprets a situation in a dysfunctional manner, this biases his or her experience of the situation, which provides further evidence for the core belief and makes it more accessible in the future (see Figure 1). Although core beliefs are often rooted in early experience in childhood and adolescence, this self-reinforcing property means that dysfunctional core beliefs tend to become much more prominent with the onset of psychopathology (Beck, 2011; e.g., Stowkowy et al., 2016). This makes core beliefs an important topic for developmental research on cognitive processes in psychopathology.

Figure 1.

Figure 1

Positive feedback loops between core beliefs and dysfunctional appraisals, adapted from J. S. Beck (2011) Cognitive therapy: Basics and beyond. New York, NY: The Guildford Press.

Traditionally, core beliefs have been emphasized in cognitive theories of mood and anxiety disorders (Beck, 1979; Beck, Emery, & Greenberg, 1985). However, broad disruptions in self-concept, or the way in which a person understands him- or herself, have long been recognized in schizophrenia (e.g. Rogers, 1958). In recent years, researchers have begun to define those disruptions in detail. Low self-esteem and negative beliefs about self and others have been linked to positive psychotic symptoms including delusions, hallucinations, paranoia, and grandiosity (e.g., Fowler et al., 2006; Freeman et al., 1998; Smith et al., 2006).1 Moreover, delusions may be strongest when they conform to previously-held negative beliefs about self and others (Garety, Kuipers, Fowler, Freeman, & Bebbington, 2001). These results speak to the importance of core beliefs in the development and maintenance of positive psychotic symptoms. Indeed, these results are consistent with a positive feedback loop in which negative beliefs (e.g. “others are hostile”) bias a person toward delusional thoughts (e.g., “the people in this restaurant are laughing at me”) which strengthen the core belief (see Figure 1). Research on negative psychotic symptoms (e.g., affective flattening, avolition) suggests that these too are supported by specific schemas: low expectancies for pleasure, success, and acceptance, and a perception of limited psychological resources (Rector, Beck, & Stolar, 2005). Again, when these schemas are activated, they consolidate underlying core beliefs such as worthlessness and personal failure (Rector, Beck, & Stolar, 2005). For adults who experience psychosis, core beliefs seem to play a role in both positive and negative psychotic symptoms.

Psychosis tends to emerge after a prodromal period marked by attenuated psychotic symptoms, mood symptoms, and functional decline (Fusar-Poli, et al., 2013). Roughly 75% of schizophrenia cases are preceded by this prodromal phase (Häfner et al., 1995). This has led to an extension of psychosis research into the prodromal or high-risk period, before a clinically-diagnosable psychotic disorder has emerged. Youth at high risk for psychosis are typically identified by the presence of attenuated psychotic symptoms. These represent a noticeable change from normative experience but do not rise to the level of a diagnosable psychotic symptom. For example, a psychotic paranoia symptom might be an unshakeable conviction that secret agents have placed cameras behind the mirrors in a person's house. By contrast, an attenuated suspiciousness symptom might be a vague feeling of being the centre of negative attention, or a preoccupation with the idea that others cannot be trusted. By identifying and studying youth at clinical high risk or ultra high-risk for psychotic disorders (UHR), researchers hope to better understand how psychosis develops, and establish interventions to mitigate, delay, or even prevent psychotic disorders (Bechdolf et al., 2012; Fusar-Poli et al., 2013; Niendam, Jalbrzikowski, & Bearden, 2009).

UHR youth are commonly identified in late adolescence or during the transition to early adulthood. Normatively, this is a key developmental phase for self-referential cognition (Cole, et al., 2001). Brain regions involved in self-referential cognition, particularly the dorsomedial pre-frontal cortex (dmPFC), significantly develop during this time (Sebastian, Burnett, & Blakemore, 2008). New abilities emerge to reason about the remembered past and imagined future (McAdams, 2013) and about the inner states of others (Sebastian, Burnett, & Blakemore, 2008). Youth experience a corresponding increase in self-conscious emotions (Sebastian, Burnett, & Blakemore, 2008), and these emotions are accompanied by existential questions about how to balance the needs of self and others and how to proceed into an uncertain personal future (Adamson, Hartman, & Lyxell, 1999). In this context of rich self-reflective cognitive development, UHR youth experience several notable impairments. UHR youth experience deficits in memory and autobiographical reasoning (Berna, et al., 2016; Fusar-Poli, et al., 2012), social dysfunction and withdrawal (Addington, Penn, Woods, Addington, & Perkins, 2008), and a disturbance in their basic sense of self, feeling alienated or disconnected rather than immersed in their own experience (Berna et al., 2016; Nelson, et al., 2009). These impairments could have important consequences for how core beliefs develop. Memory deficits could make it more difficult to retrieve autobiographical information to disconfirm negative appraisals. Likewise, social dysfunction could limit opportunities for others to challenge dysfunctional beliefs. Most of all, disruption in the basic sense of self could have far-reaching consequences for development of beliefs about the self and others. Although UHR youth are at an age that is typically characterized by development of core beliefs through self-reflective cognition, they also tend to experience important deficits that may alter or interrupt this process.

Indeed, recent research has found direct associations between core beliefs and UHR experience. Core belief profiles differentiate healthy youth from UHR youth, with UHR youth tending to have more negative and less positive beliefs about self and others (Stowkowy, et al., 2016; Taylor et al., 2014). Among UHR youth, negative core beliefs correlate with severity of attenuated psychotic symptoms (Addington & Tran, 2009; Stowkowy et al., 2016; Taylor et al., 2014), and negative beliefs about the self increase as participants transition to psychosis (Stowkowy et al., 2016). Physiologically, negative self-beliefs have been associated with hypothalamic-pituitary-adrenal axis dysregulation (Carol & Mittal, 2015) and unusual neural activity in cortical midline structures including the dmPFC (Modinos, Renken, Ormel, & Aleman, 2011). Given putative feedback loops between core beliefs and psychiatric symptoms, this is an important emerging area for study in UHR populations. Several related research questions stem from this line of inquiry. First, is the finding that core beliefs differentiate normatively healthy youth from UHR youth robust across various samples? Second, is the dimensional structure of core beliefs similar for youth and adults, and for various samples of healthy and UHR youth? Third, what are the specific links between core beliefs and symptoms, and the mechanisms by which they influence one another?

There is a small but growing body of research on maladaptive schemas in youth following Young's schema-focused approach (Young, 1994). Young proposes fifteen schemas (e.g., Defectiveness/Shame; Vulnerability to Harm/Illness) that refer to specific perceptions and expectations about the world. Various latent variable models with three, four, or five higher-order factors have been proposed for these fifteen schemas. In youth, some specific links between schemas and psychopathology have been identified, and it is somewhat unclear which higher-order factor model best fits youth data (Van Vlierberghe, Braet, Bosmans, Rosseel, & Bögels, 2010).

However, research with UHR youth has generally eschewed Young's model in favour of the Brief Core Schema Scales (BCSS; Fowler et al., 2006). The BCSS assesses general schemas about self and others (e.g., “I am weak”; “others are trustworthy”). Schemas are organized on four subscales that represent global positive and negative evaluations of self and others. The BCSS was developed specifically for adults who experience psychotic disorders, and has been a useful tool in defining how self-concept relates to psychotic symptoms in adults (e.g., Tiernan, Tracey, and Shannon, 2014). This scale also has several properties that make it amenable to youth research. First, it is relatively simple to administer and score, with 24 items laid out on a single page. Second, because it was developed for psychosis patients who experience cognitive impairments, it is written in accessible language. Third, its explicit hierarchical structure affords three levels of analysis: specific core beliefs, captured by single items; evaluative dimensions of core beliefs, captured by subscale scores; and overall profiles of core beliefs, captured by relationships between subscale scores.

However, this last point assumes that the dimensional structure of the BCSS is equivalent for healthy adults, healthy youth, adults who experience psychosis, and UHR youth. The BCSS was validated by comparing a clinical group (psychosis patients experiencing a relapse of positive symptoms; mean age = 38) with a non-clinical group of young adults (students at two prestigious London universities; mean age = 24; Fowler et al., 2006). Orthogonal principal components analyses found that these two groups' core beliefs loaded onto four principal components: Positive-Self, Negative-Self, Positive-Other, and Negative-Other. However, there are two reasons to suspect that this structure might not fit all samples of youth participants. First, as discussed above, youth experience self-referential cognitive maturation throughout adolescence and early adulthood. The dimensional structure of BCSS subscales may be affected by these maturation processes. Furthermore, the unusual patterns of self-referential cognitive development experienced by UHR youth could complicate this picture. There is, in fact, preliminary evidence that the BCSS may function somewhat differently in youth and adult samples. For instance, positive beliefs about self and others seem to differentiate UHR youth from healthy youth (Stowkowy et al., 2016; Taylor et al., 2014), but do not differentiate adult psychotic patients from non-clinical young adults (Fowler et al., 2006). Second, Fowler and colleagues (2006) modelled BCSS data orthogonally, which assumes that the four BCSS subscales are uncorrelated with one another. This is unlikely to be the case in real-world relationships between core beliefs. In fact, the authors explicitly designed the BCSS subscales as an analogue for self-esteem, which is often considered to be a single bipolar dimension (Leary & Baumeister, 2000; Rosenberg, 1979). It may be the case that oblique rotation (i.e., allowing the factors to correlate with one another) would uncover core belief structures somewhat different from those proposed by Fowler and colleagues (2006). Interpretations of BCSS results in youth depend on psychometric assumptions about the BCSS's hierarchical structure. For this reason, the current study's first goal was to examine BCSS subscale scores, internal consistency, and dimensionality in two samples of UHR and healthy comparison (HC) youth.

Our second goal was to examine relationships between core beliefs and various symptom measures in these two youth samples. As mentioned above, core beliefs are implicated in cognitive models of depression (Beck, 1979; Dozois & Rnic, 2015; Kaslow, Adamson, & Collins, 2000), anxiety (Beck, Emery, & Greenberg, 1985), and psychosis (Garety, Kuipers, Fowler, Freeman, & Bebbington, 2001; Rector, Beck, & Stolar, 2005). Depression and anxiety share common developmental pathways in youth (Kreuger, Caspi, Moffitt, & Silva, 1998), and many mood and anxiety disorders emerge in late adolescence and early adulthood (Beesdo, Pine, Lieb, & Wittchen, 2010). Among UHR youth, internalizing symptoms and attenuated psychotic symptoms often interact with one another. In a recent longitudinal twin study of adolescents, Zavos et al. (2016) found a high genetic overlap between positive or cognitive psychotic experiences and depression, and found that these symptoms interact to reinforce one another over time. It is not surprising, then, that internalizing problems are quite common in UHR samples. In a meta-analysis of 1683 high-risk participants, Fusar-Poli and colleagues (2014) reported that 41% were diagnosed with a depressive disorder and 15% were diagnosed with an anxiety disorder.

However, even for normatively healthy youth it is important to understand relationships between core beliefs and internalizing problems. At this age, internalizing problems that do not meet clinical disorder cutoffs are still problematic (Graber, 2013). These subclinical symptoms are relatively common, with depressed mood reported by as many as 40% of some youth samples (Petersen et al., 1993). These symptoms impair functioning and predict progression to clinical disorders later in life (Graber, 2013). Maladaptive schemas and core beliefs are known to play a role in these subclinical internalizing symptoms as well as diagnosable internalizing disorders (Shah & Waller, 2000; Van Vlierberghe, Braet, Bosmans, Rosseel, & Bögels, 2010). In fact, Fowler and colleagues (2006) reported that self-beliefs were associated with depression and anxiety in their non-clinical sample. Cognitive models linking core beliefs to depression and anxiety are relevant for both healthy and UHR youth.

In the current study, we examined these relationships through simultaneous multiple linear regression models. This allowed us to parcel out shared and unique variance to ask two parallel questions: do core beliefs as a set predict symptom scores, and do any of the BCSS subscales uniquely predict symptom scores? The cognitive literature is clear that we should expect negative self-beliefs (e.g., “I am worthless”, “I am vulnerable”) to be uniquely associated with both depression and anxiety. Depression is also strongly linked to low self-esteem, which should be reflected in part by a lack of positive self-beliefs (e.g., “I am not successful”). This was borne out in the BCSS validation study, in which the authors reported higher correlations between positive self-beliefs and depression (r = -.45) than positive self-beliefs and anxiety (r = -.33; Fowler et al., 2006). We hypothesized that a cognitive model of depression would be supported if positive and negative self-beliefs uniquely predicted depression symptoms, and a cognitive model of anxiety would be supported if negative self-beliefs uniquely predicted anxiety symptoms. Because internalizing experiences are relevant for HC and UHR youth, we tested both models in both groups.

Finally, we predicted specific links between core beliefs and attenuated psychotic symptoms in UHR youth. Models of positive psychotic symptoms emphasize negative core beliefs (Garety, Kuipers, Fowler, Freeman, & Bebbington, 2001), and previous youth research has found links between positive symptoms and negative beliefs about self and others (Addington et al., 2009; Stowkowy et al., 2016; Taylor et al., 2014). We therefore hypothesized that a cognitive model of positive psychotic symptoms would be supported if negative beliefs about self and others uniquely predicted attenuated positive psychotic symptoms.

Negative symptoms have received less attention in this literature. However, The severity and persistence of negative attenuated symptoms are known risk factors for conversion to psychosis (Piskulic, et al., 2012). This highlights the importance of understanding their development in UHR youth. Cognitive models of negative psychotic symptoms posit that these symptoms are driven by perceptions of limited psychological resources, negative social and performance attitudes, and low expectancies for success (Rector, Beck, & Stolar, 2005). The BCSS Self-Negative and Self-Positive scales contain items that appear to tap these cognitive factors (e.g. limited psychological resources = “I am weak”; social and performance attitudes = “I am not talented”; success expectancies = “I am a failure”). Although there is some evidence that negative performance attitudes impact negative symptoms in UHR youth (Perivoliotis, Morrison, Grant, French, & Beck, 2009), one recent study found no direct associations between the BCSS and attenuated negative symptoms (Stowkowy et al., 2016). Evidence on this front is equivocal. However, we hypothesized that a cognitive model of negative psychotic symptoms would be supported if negative self-beliefs and a lack of positive self-beliefs uniquely predicted attenuated negative psychotic symptoms.

In summary, the current study explored the structure of core beliefs in a community sample of youth at ultra-high risk for psychosis, and a matched sample of healthy comparison community youth. The BCSS was administered to both groups along with clinical and self-report measures. We first examined BCSS scores, internal consistency, and dimensionality, modeled by factor analysis with oblique rotation. We then examined relationships between core beliefs, depression, anxiety, and attenuated psychotic symptoms through a series of simultaneous multiple linear regressions.

Method

Participants

Two groups of community youth participants were recruited through the Adolescent Development and Preventative Treatment (ADAPT) research program via newspaper, bus, and Craigslist ads, e-mail postings, and community professional referrals. One group of participants (n = 73) was classified as being at ultra-high risk for psychosis based on the Structured Interview for Prodromal Symptoms (SIPS; Miller et al., 1999). Ultra-high risk (UHR) participants with DSM-IV-TR Axis I diagnoses were included in the study, as these disorders are common in psychosis-risk populations. Comorbid DSM-IV-TR Axis I disorders in the UHR group included 21 mood disorders (29% of the sample), 6 PTSD (8%), 6 OCD (8%), 25 other anxiety disorders (34%), 7 ADHD (10%), and 1 eating disorder (1%). These comorbidity rates are comparable to those found in other studies (Fusar-Poli et al., 2014).

The second group was a matched sample of healthy comparison participants (HC; n = 73) recruited from the community. Exclusion criteria for the HC group included any psychotic disorder in a first-degree relative, a prodromal syndrome as assessed by the SIPS, or any current DSM-IV-TR Axis I disorder. In both groups, participants with a history of head injury, neurological disorder, substance dependence, or any DSM-IV-TR psychotic disorder were excluded from the study. The protocol and informed consent procedures were approved by the Institutional Review Board (IRB). Data from a subgroup of this sample (including UHR and HC participants) were previously reported in a study focusing on familial environment, self-concept, and HPA-axis abnormalities (Carol & Mittal, 2015).

Descriptive statistics for study participants are shown in Table 1. Two-tailed t-tests (continuous variables) and χ2 tests (categorical variables) found no significant differences between demographic variables in the two groups, with the exception of gender, χ2 (1, n = 146) = 3.96, p = .046). However, gender was not correlated with any clinical or self-report measures (BCSS, BDI, BAI, or SIPS), except for a weak correlation with the positive SIPS subscale (r = .19, p = .02). Overall, the two groups were closely-matched on demographic variables.

Table 1. Demographic, Clinical, and Self-Report Measures.

Ultra-high risk Healthy comparison Test statistic p
N 73 73
Gender Female 29 (40%) 41 (56%)
Male 44 (60%) 32 (44%) 3.95 .047
Race White 50 (68%) 46 (63%)
Hispanic 11 (15%) 14 (19%)
Other 12 (17%) 13 (18%) 7.89 n.s.
Age 18.7 (1.8) 18.1 (2.6) -1.50 n.s.
Years of education 12.4 (1.7) 12.2 (2.6) -0.40 n.s.
BCSS Self-Negative 5.35 (5.3) 1.01 (1.5) -6.73 <.001
Self-Positive 12.1 (6.1) 16.1 (5.2) 4.33 <.001
Other-Negative 6.43 (5.6) 2.75 (3.4) -4.79 <.001
Other-Positive 10.2 (5.4) 13.8 (5.3) 4.04 <.001
BDI 17.1 (11.8) 3.7 (5.0) -8.79 <.001
BAI 23.9 (13.0) 5.5 (6.5) -10.6 <.001
SIPS Positive 12.0 (4.6) 0.45 (1.0) -21.1 <.001
Negative 9.90 (6.7) 0.41 (0.9) -11.5 <.001

Test statistics were χ2 tests for categorical variables and two-tailed t-tests for continuous variables. Numbers in parentheses are percentages for categorical variables and standard deviations for continuous variables. BCSS = Brief Core Schema Scale; BDI = Beck Depression Inventory-II; BAI = Beck Anxiety Inventory; SIPS = Structured Interview of Prodromal Symptoms

Measures

All participants completed two standardized clinical interviews. First, to diagnose the presence and severity of attenuated psychotic symptoms, the Structured Interview for Prodromal Symptoms (SIPS; Miller et al., 1999) was administered to all study participants. The SIPS is a semi-structured clinical interview that assesses positive (e.g. perceptual abnormalities), negative (e.g. emotional expression), disorganized (e.g. personal hygiene), and general symptoms (e.g. sleep disturbance), with a total score calculated for each category. In this study, SIPS scores were used in two separate contexts. First, participants were screened for inclusion in the UHR group and exclusion from the HC group based on whether they met SIPS criteria for an at-risk status. Second, SIPS scores were used as dependent variables to examine associations between core beliefs and attenuated psychotic symptoms in the UHR group (see “Data Analysis” below for details). The Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-IV; First et al., 1995) was also administered to all participants to rule out formal psychotic disorders, and to rule out any DSM-IV Axis I psychopathology in the HC group. Clinical interviews were conducted in person by advanced doctoral students. Diagnostic decisions were made in team meetings (directed by VAM), and incorporated data from the SIPS/SOPS as well as material, when available, from corroborative sources including parents, relatives, and treatment providers. Raters, consisting of advanced doctoral students, were trained with videos and then live cases until a high level of reliability was met (Kappas ≥ .80), and then this was regularly maintained through in-group consensus meetings and regularly held training meetings.

Participants completed several self-report scales in person at the ADAPT research lab. The Brief Core Schema Scale (BCSS) is a self-report scale developed to capture core evaluative schemas in psychotic disorders (Fowler et al., 2006). The BCSS is composed of 24 items, each of which is a brief positive or negative statement about the self or others (e.g. “I am valuable”, “Others are hostile”) rated on a 5-point scale ranging from “0 – I do not believe this” to “4 – I believe this totally”. Items are grouped into Positive-Self, Negative-Self, Positive-Other, and Negative-Other subscales, with six items on each subscale. The BCSS has been validated in adult psychosis populations (Fowler, et al., 2006). It has also been used increasingly in research with UHR youth. Researchers have observed that profiles of core beliefs differentiate healthy youth from UHR youth (UHR youth tend to have more negative and less positive beliefs), that core beliefs correlate with symptom measures, and that negative self-beliefs tend to increase as participants transition to psychosis (Addington & Tran, 2009; Stowkowy et al., 2016, Taylor et al., 2014).

Participants also completed self-report measures of depression and anxiety. The Beck Depression Inventory-II (BDI; Beck, Steer, & Brown, 1996) is a commonly-used self-report scale for depression. It consists of 21 items assessing DSM-IV depressive symptoms (e.g. sadness, irritability). Each item is rated from not present (0) to severe (3), and a total score indicates subclinical, mild, moderate, or severe depressive symptomatology. The BDI has good psychometric properties and high validity standards in various populations (Richter, Werner, Heerlein, Kraus, & Sauer, 1998). In UHR youth, the BDI shows good internal consistency as well as construct and criterion validity (i.e., identifying a previously-diagnosed major depressive disorder; DeVylder, et al., 2014). The Beck Anxiety Inventory (BAI; Beck & Steer, 1993) is a 21-item self-report measure of anxiety symptom severity, with each item rated from not present (0) to severe (3), and a total score indicating subclinical, mild, moderate, or severe anxious symptomatology. The BAI has seen increased use in youth populations in recent years (Bardhoshi, Duncan, & Erford, 2016), including among youth at clinical high risk for psychosis (e.g., Hui, et al., 2013).

Statistical Analyses

We used chi-squared tests and independent two-tailed t-tests to examine group differences in demographics and study variables (BCSS, BDI, BAI, and SIPS). Reliability and factor analyses were carried out in R version 3.2.4 Revised (R Core Team, 2016) using the psych package (Revelle, 2016). We measured the reliability of BCSS subscales by computing alpha and hierarchical omega (McDonald, 1999). We determined the optimal numbers of factors to represent BCSS items by parallel analysis (PA; Horn, 1965), minimum average partial tests (MAP; Velicer, 1976), and Bayesian information criteria (BIC; Schwartz, 1978). After determining the optimal number of factors, we ran exploratory factor analyses and rotated factors via oblique (oblimin) rotation to allow for the possibility that factors would correlate with one another. The sample sizes in this study are small for factor analyses. We therefore present these factor analyses as descriptive data for these particular samples. These factor analyses are not intended to be representative of broader youth populations.

Simultaneous multiple linear regression analyses were carried out in JMP® Pro version 13.0.0 (SAS Institute Inc., 2013) to examine relationships between core beliefs and various clinical measures (SIPS, BDI, and BAI). In both UHR and HC groups, BCSS subscale means were regressed on BDI and BAI scores (i.e. all four BCSS subscales were entered as predictors in each regression model). In the UHR group, BCSS subscale means were regressed on SIPS positive and negative scores. We conducted three supplemental analyses to more closely examine (a) the Self-Negative scale in HC youth, (b) associations between core beliefs and attenuated positive symptoms in the UHR group, and (c) associations between self-beliefs and various symptoms in the UHR group (see “Results” below for details of each of these analyses). We corrected for multiple correlations in these supplemental analyses using the Holm method, also known as the “sequential Bonferroni method”, a relatively conservative procedure (Holm, 1979). Missing data were fit separately in all analyses.

Four HC participants (5% of the sample) endorsed clinically significant anxiety symptoms (defined as a BAI score ≥ 16; Beck & Steer, 1993), and one of these four participants (1% of the sample) also endorsed clinically significant depressive symptoms (defined as a BDI score ≥ 20; Dozois, Dobson, & Ahnberg, 1998). None of these participants met criteria for a clinical diagnosis on the SCID-IV. No other HC participants endorsed significant depressive symptoms. In keeping with a dimensional approach to internalizing psychopathology, analyses are reported here with the moderately-symptomatic HC participants included.

Results

UHR and HC participants reported significantly different profiles of core beliefs on the BCSS. UHR participants reported significantly more negative beliefs about self and others than HC participants, and significantly less positive beliefs about self and others (see Table 1). As shown in Figure 2, both groups reported more positive than negative beliefs overall, and this pattern was more pronounced for HC participants. Endorsements of negative self-beliefs were notably rare in the HC group.

Figure 2.

Figure 2

Brief Core Schema Scale total scores in healthy comparison and ultra-high risk youth. Error bars indicate the standard error of the mean.

In general, BCSS subscales showed good internal consistency in both UHR and HC groups. Cronbach's alpha and McDonald's hierarchical omega are shown in Table 2 for the BCSS as a whole, and for each subscale. Zinbarg, Revelle, Yovel, and Li (2005), among others, have pointed out that Cronbach's alpha is not ideal for measuring internal consistency. It assumes that all item correlations are equal, and therefore overestimates the internal consistency of “lumpy” scales that include two or more relatively independent factors. Hierarchical omega, which estimates the proportion of variance accounted for by a single unrotated general factor, is recommended for this purpose instead (Zinbarg et al., 2005). In the current study, hierarchical omega values indicated that a single factor accounted for the majority of observed variance in three of the four BCSS subscales. The Self-Negative subscale was a notable exception to this pattern. This subscale showed poor internal consistency, particularly in the HC group.

Table 2. Brief Core Schema Scales Internal Consistency.

α ωhierarchical
BCSS Total (all 4 subscales) UHR .92 .62
HC .88 .52
Self-Negative UHR .86 .67
HC .42 .62
Self-Positive UHR .88 .81
HC .84 .72
Other-Negative UHR .89 .79
HC .88 .73
Other-Positive UHR .89 .79
HC .91 .83

We used three statistical tests to determine the appropriate number of factors for BCSS data in each group (see “Data Analysis” above). In the UHR group, parallel analysis (PA) and minimum average partial tests (MAP) suggested that four factors were optimal, while Bayesian information criteria (BIC) suggested three factors. We determined that four factors were appropriate for the UHR group, based on two out of three tests suggesting a four-factor solution, BIC's bias toward more parsimonious models, and the four-component solution that has previously been reported for the BCSS (Fowler, et al., 2006). Four factors were extracted and rotated via oblimin rotation. Factors and factor correlations are shown in Table 3, and item loadings are shown in Table 4. The four factors cumulatively accounted for 60% of the variance in BCSS scores. These factors matched the four BCSS subscales, with all six items from each subscale loading onto the expected factor. Factors correlated with one another somewhat. In particular, the Self-Negative factor correlated with all other factors at r = .34 or greater. There was little cross-loading of individual items onto multiple factors. Overall, this oblique factor structure matches the four BCSS subscales.

Table 3. Factors and Factor Intercorrelations from Exploratory Factor Analyses of the Brief Core Schema Scales.

UHR group HC group

Other Pos Other Neg Self Pos Self Neg Other Pos Other Neg Self Pos
Factor SS loadings 3.99 4.01 3.30 3.11 4.31 4.03 2.97
Factor % variance .17 .17 .14 .13 .18 .17 .12
Cumulative % var. .17 .33 .47 .60 .18 .35 .47
Factor correlations
 Other-Positive 1 1
 Other-Negative -.22 1 -.13 1
 Self-Positive .43 -.11 1 .48 -.23 1
 Self-Negative -.34 .36 -.40 1

Table 4. Item Loadings from Exploratory Factor Analyses of the Brief Core Schema Scales.

UHR Group HC Group

Other Pos Other Neg Self Pos Self Neg λ2 Other Pos Other Neg Self Pos λ2
1. Self-unloved .28 -.16 .57 .42 .19 .04
2. Self-worthless -.18 .84 .87 -.37 .14
3. Self-weak -.16 .72 .68 .37 .18
4. Self-vulnerable .20 -.27 .38 .35 .26 -.18 .11
5. Self-bad .44 .22 .51 .61 -.22 .49 .30
6. Self-failure -.21 .64 .63 .28 -.51 .23
7. Self-respected .26 .54 .50 .27 .39 .32
8. Self-valuable .72 .68 .25 .57 .59
9. Self-talented .81 .68 .78 .58
10. Self-successful .67 -.17 .66 .22 .53 .45
11. Self-good .37 -.18 .47 .58 .17 -.30 .49 .51
12. Self-interesting .61 -.15 .47 .77 .57
13. Others-hostile .85 .67 .70 .46
14. Others-harsh .77 .60 .72 .51
15. Others-unforgiving .69 .57 .73 .53
16. Others-bad .75 .18 .64 .67 .43
17. Others-devious .69 .54 .81 .66
18. Others-nasty .69 .15 .57 .80 .66
19. Others-fair .67 .27 -.15 -.30 .52 .65 .26 .51
20. Others-good .65 .46 .77 .68
21. Others-trustworthy .82 -.21 .79 .88 .69
22. Others-accepting .81 .74 .85 .73
23. Others-supportive .61 .23 .56 .76 .69
24. Others-truthful .77 .65 .77 .65

Factor loadings greater than .30 shown in bold. Factor loadings less than .15 omitted.

In the HC group, all three tests (PA, MAP, and BIC) suggested a three-factor model. Three factors were extracted and rotated via oblimin rotation. Factors and factor correlations are shown in Table 3, and item loadings are shown in Table 4. The three factors cumulatively accounted for 47% of the variance in BCSS scores, and matched the Other-Positive, Other-Negative, and Self-Positive BCSS subscales. Self-Negative items did not show a cohesive pattern in their factor loadings, with items split between the Self-Positive and Other-Negative subscales, and “I am vulnerable” and “I am unloved” failing to load onto any of the three factors. The three factors accounted for hardly any variance in these items—communalities for “I am vulnerable” and “I am unloved” were .11 and .04, respectively. The Self-Positive and Other-Positive factors correlated with one another substantially (r = .48). In contrast to the UHR group, BCSS scores in the HC group seem to be best represented by a three-factor model.

Because Fowler and colleagues (2006) reported a four-component solution for the BCSS with healthy young adults, we also extracted a four-factor model with oblimin rotation. This solution was a Heywood case due to one item loading greater than 1.0, suggesting that it may have been over-factored. More importantly, the oblique four-factor structure did not match the BCSS subscales. The fourth factor mostly captured variation in two items, “I am bad” (loading = .92) and “I am weak” (loading = .53), and the next highest item loadings were “others are bad” (.36) and “others are hostile” (.32). Details of this four-factor model are available from the first author on request. This four-factor model did not match the BCSS's subscale structure, nor was it statistically optimal. The three-factor model better represents BCSS data in the current HC sample.

We computed simultaneous multiple linear regression models to examine relationships between core beliefs, depression, and anxiety in the UHR and HC groups. In each model, we entered BDI or BAI scores as the dependent variable, and the four BCSS subscales as predictor variables (see Table 5). Core beliefs predicted depression in both groups, with unique main effects for positive and negative self-beliefs in both groups. Core beliefs predicted anxiety in the HC group but not in the UHR group. Negative self-beliefs uniquely predicted anxiety in the HC group. By contrast, none of the core beliefs subscales uniquely predicted anxiety in the UHR group. In interpreting core beliefs in the HC group, however, recall that Self-Negative items were rarely endorsed in the and failed to group together as a single factor in our EFA. One advantage of the BCSS is its hierarchical structure, which allows us to examine scores at the level of individual items (individual core beliefs), scales (broad sets of core beliefs), and relationships between scales (overall patterns of core beliefs). Because the Self-Negative subscale did not make a cohesive factor in our HC group, we can instead examine associations between core beliefs, depression, and anxiety at the level of individual core beliefs. As a supplemental analysis, we calculated bivariate correlations between individual Self-Negative items, depression, and anxiety in the HC group. We found that depression and anxiety were specifically associated with schemas of one's own worthlessness (depression r = .64, p < .001; anxiety r = .54, p < .001), vulnerability (depression r = .29, p = .01; anxiety r = .36, p = .002), and weakness (depression r = .24, p = .04; anxiety r = .26, p = .03). After applying a Holm correction for multiple comparisons, the correlations between worthlessness and depression, worthlessness and anxiety, and vulnerability and anxiety remained statistically significant at p < .05.

Table 5. Regression Models Predicting Depression and Anxiety Symptoms from Core Beliefs.

Beck Depression Inventory Beck Anxiety Inventory

UHR group HC group UHR group HC group

B Std Er β B Std Er β B Std Er β B Std Er β
Self-Negative 4.42 1.65 .34** 5.71 2.36 .28* 0.56 2.32 .04 10.8 3.33 .40**
Self-Positive -4.41 1.44 -.37** -2.39 0.86 -.39** -2.23 2.07 -.17 -0.23 1.21 - .03
Other-Negative -1.25 1.47 -.09 1.18 0.97 .14 1.71 2.03 .11 0.14 1.36 .01
Other-Positive -2.16 1.51 -.16 .57 0.73 .10 -1.74 2.18 -.12 -1.02 1.10 - .14
Full model F(4, 60) = 14.64, p < .001, R2 = .49 F(4, 66) = 7.65, p < .001, R2 = .32 F(4, 63) = 2.07, p = .09, R2 = .12 F(4, 66) = 4.15, p = .005, R2 = .20
*

p < .05;

**

p < .01;

UHR = Ultra high-risk; HC = Healthy comparison

Finally, we computed simultaneous multiple linear regression models to examine associations between core beliefs and attenuated psychotic symptoms in the UHR group. In these models, we entered SIPS Positive or SIPS Negative scores as the dependent variable, and all four BCSS subscales as predictor variables. As shown in Table 6, core beliefs predicted negative attenuated psychotic symptoms, with unique main effects for positive and negative self-beliefs. However, core beliefs did not predict positive attenuated psychotic symptoms, and none of the core beliefs subscales uniquely predicted these symptoms. To follow up on this unexpected finding, we examined the correlation matrix of BCSS subscales and individual positive symptoms. Two positive symptoms were associated with core beliefs. Suspiciousness/persecutory ideas were associated with negative beliefs about self (r = .36, p = .002) and others (r = .24, p = .04), and perceptual abnormalities were associated with a lack of positive beliefs about self (r = -.25, p = .03) and others (r = -.26, p = .03). However, after applying a Holm correction for multiple comparisons, only the link between negative self-beliefs and suspiciousness/persecutory ideas remained statistically significant at p < .05.

Table 6. Regressions Models Predicting Attenuated Psychotic Symptoms from Core Beliefs in the Ultra-High Risk Group.

SIPS Positive SIPS Negative

B Std Er β B Std Er β
Self-Negative 0.64 0.79 .12 2.17 1.07 .28*
Self-Positive -0.24 0.71 -.05 -2.45 0.96 -.36*
Other-Negative 0.12 0.68 .02 0.15 0.92 .02
Other-Positive -0.55 0.77 -.11 0.58 1.04 .08
Full model F(4, 68) = 1.06, p = .38, R2 =.06 F(4, 68) = 5.80, p < .001, R2 =.25
*

p < .05;

SIPS = Structured Interview for Prodromal Symptoms

Note that, in the UHR group, a similar profile of more negative self-beliefs and less positive self-beliefs was associated with both depression and attenuated psychotic symptoms. However, depression and attenuated psychotic symptoms correlated with one another (BDI— SIPS Neg, r = .47; SIPS Pos—SIPS Neg, r = .45; SIPS Pos—BDI, r = .26). This raises the question of whether these core beliefs relate to common variance shared among symptoms, or unique variance for each symptom scale. This study's data were collected concurrently, so we can answer this question by reversing our regression models, and entering core beliefs as dependent variables with symptom scores as predictors. We entered BCSS Self-Negative and Self-Positive scores as dependent variables in two simultaneous multiple linear regressions, and entered all four outcome measures (BDI, BAI, SIPS Positive, and SIPS Negative) as predictors. In each model, we examined beta weights to determine which symptom scales were uniquely associated with core beliefs. We found that depression was uniquely associated with less positive self-beliefs (β = -.52, p < .001) and more negative self-beliefs (β = .49, p < .001), and that negative attenuated psychotic symptoms were uniquely associated with less positive self-beliefs (β = -.37, p = .002). This suggests that depression may have mediated the relationship between negative self-beliefs and negative psychotic symptoms.

Discussion

This study had two broad goals. The first was to describe core belief profiles and dimensionality among the current samples of healthy comparison youth (HC) and youth at ultra-high risk for psychotic disorders (UHR). The BCSS is an excellent scale for this purpose due to its simple hierarchical structure. Note, however, that this was not a validation study of the BCSS per se. We were agnostic to the orthogonal four-component BCSS structure defined in adults, and examined our data through exploratory analyses. Moreover, our sample size was quite small by factor analytic standards. For this reason, we cannot claim that our exploratory factor analysis results are representative of a broader population. This first study goal was simply to describe in detail how the current sample of participants responded on the BCSS. Building on past research (Addington & Tran, 2009; Stowkowy, et al., 2016), we found notable differences between core beliefs in the two groups. While UHR and HC youth both reported more positive and less negative schemas of self and other, this pattern was much less pronounced in the UHR group (see Figure 2). Compared to the HC group, youth in the UHR group endorsed more negative beliefs about self and others and less positive beliefs about self and others.

Notably, the HC youth reported very few negative self-beliefs (e.g., “I am worthless”, “I am weak”). In fact, we were unable to recover a factor matching this scale in our exploratory factor analyses. This sample of healthy youth (mean age 18.1) did not seem to have a global set of negative beliefs about the self. The non-clinical young adults in the BCSS validation study (Fowler et al., 2006) reported more negative self-beliefs, and the authors found that these grouped together into a Self-Negative principal component. It seems that there may be significant variation between non-clinical samples in terms of negative self-beliefs, possibly related to demographic differences or cohort effects. It is also worth noting that the mean age of Fowler and colleagues' sample was 24. That sample was comprised of university students, so it is unlikely that the entire group was older than the HC group in the current study, but there may be meaningful age-related differences between these two samples. Taking Fowler and colleagues' results into account, it may be most accurate to say that our HC youth have not yet developed a cohesive set of negative self-beliefs. These are important considerations for developmental researchers when using the BCSS or similar scales. Core beliefs evolve throughout childhood, adolescence, and into adulthood (Beck, 2011). It is advisable to confirm a four-factor core belief structure in any one sample before interpreting BCSS subscales. We recommend examining subscales' internal consistency using McDonald's hierarchical omega, or if it is unavailable then using Cronbach's alpha 2. In situations where alpha or hierarchical omega are particularly low, as they were for the Self-Negative scale in our HC participants, it is feasible to analyze core beliefs at the level of individual items (i.e., individual core beliefs). In this situation, a correction for multiple comparisons (e.g., Bonferroni, Holm, procedures for estimating the false discovery rate) can be used to avoid unduly inflating the Type I error rate.

The fact that we recovered different factor structures in our HC and UHR groups highlights two reasons to test for internal consistency. First, if we had simply compared subscale scores between the two groups, these subscales would not have been psychometrically equivalent. We were able to detect and correct for this in our analyses. Second, the fact that we see different structures in the two groups could have interesting implications. Cognitive or schema theories predict that core beliefs and attenuated psychotic symptoms should mutually reinforce one another. The putative cognitive mechanism that links core beliefs to psychiatric symptoms is a feedback loop: core beliefs bias how a person interprets an event; the person experiences the event in a way consistent with the core belief; and this provides further evidence to strengthen the core belief and make it more accessible in the future. This theory predicts that dysfunctional core beliefs (e.g., negative beliefs about the self) should intensify as symptoms increase, which is consistent with our data as well as others'. In a recent longitudinal study, Stowkowy and colleagues (2016) reported that self-schemas at baseline did not predict conversion to psychosis, but that participants who transitioned to psychosis had significantly more negative self-schemas at the time of transition. These participants' negative self-evaluations increased as their psychotic experiences intensified, and vice versa. Our data, which show a global set of negative self-beliefs present in our UHR participants and absent in our HC participants, are consistent with the same effect. Due to the putative feedback loop between self-beliefs and attenuated psychotic symptoms, self-beliefs could even play a causal role in the development of attenuated psychotic symptoms. Longitudinal relationships between negative self-beliefs and attenuated psychotic symptoms are an important topic for future developmental research.

The second goal of our study was to test relationships between core beliefs and depression, anxiety, positive psychotic symptoms, and negative psychotic symptoms. We posited specific links between core beliefs and various symptoms based on cognitive theories of mood, anxiety, and psychotic disorders (see “Introduction” above). We hypothesized that a cognitive model of depression would predict unique associations between depression, more negative self-beliefs, and less positive self-beliefs. We hypothesized that a cognitive model of anxiety would predict unique associations between anxiety and more negative self-beliefs. Finally, we hypothesized that a cognitive model of psychosis would predict unique associations between attenuated positive psychotic symptoms and more negative self- and other-beliefs, and between attenuated negative psychotic symptoms, more negative self-beliefs, and less positive self-beliefs. Conceptually, we designed this section of the study as a series of tests to see which of these models were supported in our data. We did so through a series of simultaneous multiple linear regressions, with symptom scales entered as dependent variables and all four BCSS subscales entered as predictors (see Tables 5 and 6).

Data from our HC participants were consistent with the proposed cognitive models for both depression and anxiety. As a set, core beliefs significantly predicted both depression and anxiety. Negative self-beliefs uniquely predicted both depression and anxiety, and a lack of positive self-beliefs uniquely predicted depression. This is precisely the pattern predicted by our cognitive models. However, as noted in our discussion of the EFA results, the BCSS Self-Negative subscale did not form a meaningful factor in the HC group. We therefore examined bivariate correlations between specific negative self-beliefs (i.e., individual items on the Self-Negative subscale), depression, and anxiety in this group. After adjusting for multiple comparisons using the Holm method (a relatively conservative method), we found robust relationships between worthlessness beliefs and depression (r = .64), worthlessness beliefs and anxiety (r = .54), and vulnerability beliefs and anxiety (r = .36).

By comparison, data from our UHR participants were consistent with a cognitive model of depression but not a cognitive model of anxiety. We found strong support for links between core beliefs and depression in the UHR youth. BCSS scores accounted for roughly half of the total observed variance in BDI scores (R2 = .49). Moreover, in line with our specific hypotheses, negative self-views and a lack of positive self-views both uniquely predicted depression scores. However, we did not find any links between core beliefs and anxiety symptoms in the UHR group. Although the overall model predicting BAI scores approached statistical significance (p = .09), it did not predict a substantial amount of observed variance (R2 = .12), and none of the core beliefs subscales uniquely predicted anxiety. Overall, we found support for a cognitive model of depression in HC and UHR youth, and support for a cognitive model of anxiety in HC but not UHR youth.

Finally, with respect to attenuated psychotic symptoms, data from our UHR participants were clearly consistent with the proposed model for negative psychotic symptoms, and marginally consistent with the proposed model for positive psychotic symptoms. As a set, core beliefs significantly predicted attenuated negative psychotic symptoms, and negative self-beliefs and a lack of positive self-beliefs both uniquely predicted negative attenuated psychotic symptoms. By contrast, core beliefs as a set did not uniquely predict positive attenuated psychotic symptoms, and none of the core belief subscales uniquely predicted these symptoms.

To follow up on this surprising finding, we examined bivariate correlations between BCSS subscales and specific attenuated positive symptoms. After adjusting for multiple comparisons using the Holm method, we found a significant correlation between negative self-beliefs and suspiciousness/persecutory ideas (r = .36). This association has been reported in previous research (Addington & Tran, 2009; Fowler et al., 2006), and may be a particularly robust link between core beliefs and attenuated positive symptoms. Moreover, negative self-beliefs are implicated in the maintenance of persecutory delusions among adults with a psychotic disorder diagnosis (Vorontsova & Garety, 2013). If future research were to look for causal links between core beliefs and attenuated positive symptoms, this would be a promising relationship to explore.

Surprisingly, we failed to replicate previously-reported associations between self-beliefs and unusual thought content (Addington & Tran, 2009). In our sample, no relationships between core beliefs and unusual thought content were significant even without correcting for multiple comparisons. The highest correlation was with negative self-beliefs, r = .19, p = .11. This is puzzling. It may be that relationships between core beliefs and unusual thought content are mediated by as-yet unknown processes or variables. However, we should note that no methods currently exist for identifying UHR youth at the population level. It is a challenge to find and recruit UHR youth, and sites often recruit only one or two participants per month (Addington, et al., 2008). Various UHR samples are likely to be somewhat idiosyncratic, varying based on recruitment procedures and demographic factors of a given community. Even assuming that all procedures in the current study (e.g., training and supervision of interviewers) were implemented adequately, it is possible that this result is Type II error based on sampling variation.

Table 7 shows a summary of the evidence for our proposed cognitive models of depression, anxiety, and psychosis. The HC results were in line with models of depression and anxiety. In our UHR group, we found support for cognitive models of depression and negative psychotic symptoms, qualified support for a model of positive psychotic symptoms, and no support for a cognitive model of anxiety. This mixed picture in the UHR group raises two questions. First, our symptom measures were moderately correlated with one another. Were core beliefs associated with shared variance between symptoms, or were they associated with specific symptoms? Second, the only model for which we observed no support was the anxiety model in the UHR group. Why would this be the case?

Table 7. Summary of Evidence for Predicted Links Between Symptoms and Core Beliefs.

Cognitive model Predicted link HC UHR
Depression Neg. self + +
Pos. self + +
Anxiety Neg. self +
Positive psychotic Neg. self n/a (+)
symptoms Neg. other n/a
Negative psychotic Neg. self n/a +
symptoms Pos. self n/a +

+ = support; (+) = qualified support; – = weak or no support; HC = healthy comparison youth; UHR = youth at ultra high-risk for psychosis

The first question is relatively straightforward to answer. We computed two supplemental regression models, and found that depression was uniquely associated with more negative self-beliefs, and depression and negative attenuated psychotic symptoms were both uniquely associated with less positive self-beliefs. The observed relationship between negative self-beliefs and negative psychotic symptoms may have been mediated by depressive symptoms. Perhaps negative beliefs about the self contribute to depressed mood and cognition, which activate cognitive mechanisms that contribute to negative psychotic symptoms (i.e., disengagement, low expectancies for success, negative social and performance attitudes, and perceptions of limited psychological resources; Rector, Beck, & Stolar, 2005). Alternatively, scales of depression and negative psychotic symptoms may overlap with one another somewhat. Previous research has challenged the distinction between depression and negative psychotic symptoms in the psychosis prodrome. Depressed mood can be the first observable sign of a prodromal syndrome (Häfner, et al., 2005), and acute depressive symptoms tend to remit somewhat after a first psychotic episode (Häfner, et al., 2005; Koreen, Siris, Chakos, & Alvir, 1993). Abnormal striatal dopaminergic function is a key process in the development of psychosis (Fusar-Poli et al., 2010; 2011; Howes et al., 2009; 2011). Depressive symptoms may be, in part, an early manifestation of abnormal striatal dopaminergic function. We may not have observed direct links between negative self-beliefs and negative attenuated psychotic symptoms because our symptom measures captured two aspects of a single syndrome in which negative self-beliefs play a role.

In fact, this could also begin to explain why we did not observe links between core beliefs and anxiety in the UHR group. If the BDI and SIPS are picking up on two shades of a single process that impacts mood, cognition, and functioning, the same cannot be said for the BAI. In our data, anxiety correlated moderately with depression (r = .46) but not with positive (r = -.05) or negative (r = .20) attenuated psychotic symptoms. Our data are consistent with the hypothesis that anxiety in UHR youth (as measured by the BAI) is relatively distinct from attenuated psychotic symptoms. In both youth (Carragher et al., 2015) and adults (Tackett, Quilty, Sellbom, Rector, & Bagby, 2008), depression and anxiety are typically understood as two facets of an internalizing psychopathology dimension. Part of the variance in our UHR participants' depression scores was likely related to this internalizing dimension, and a separate part of the variance was likely related to negative attenuated psychotic symptoms. To test this interpretation, we ran a post hoc simultaneous multiple linear regression with BDI scores entered as the dependent variable and BAI and SIPS Negative scores entered as predictors. Standardized betas for BAI (.41) and SIPS Negative scores (.37) were both significant at p < .001, suggesting that our UHR participants' experience of anxiety symptoms was relatively distinct from their experience of attenuated psychotic symptoms, and that their experience of depressive symptoms was independently related to both symptom clusters. These participants' self-beliefs related to their experiences of depression and negative psychotic symptoms, with some amount of overlap between these two symptom clusters. However, their self-beliefs did not relate to a distinct cluster of anxiety symptoms. This is interesting, given the prevalence of anxiety symptoms in UHR populations and the face-validity of items like “I am vulnerable” in predicting anxiety. Unfortunately, negative results are less easily interpretable than positive results due to the possibility of Type II error discussed above. We hesitate to offer a strong explanation for this negative result until it is replicated in other studies.

We should note that our study design did not include a matched clinical group of patients diagnosed with depression or anxiety who do not experience attenuated psychotic symptoms. This would have strengthened our conclusions. For instance, we cannot rule out the possibility that relationships between anxiety and core beliefs differed based on the severity of reported anxiety symptoms (recall that our UHR participants endorsed significantly more anxiety symptoms than our HC participants). Despite this limitation, we can make several recommendations. There has been strong research interest in relationships between depression and negative attenuated psychotic symptoms in the ultra-high risk period. It may be fruitful to explore cognitive links between these symptoms in more depth. In addition, schema- or self esteem-focused therapy has been suggested as a promising treatment for UHR youth (Stowkowy, et al., 2016; Taylor et al., 2014). Our data suggest that these types of interventions may be more effective for negative psychotic symptoms and mood symptoms, and less effective for anxiety symptoms that are also common in UHR youth. Finally, as mentioned above, we recommend that developmental researchers using the BCSS or similar scales of core beliefs confirm the internal consistency of all subscales (particularly the Self-Negative subscale) before drawing conclusions based on subscale means or total scores. Because of the BCSS's simple hierarchical structure, in scenarios where a subscale does not show internal consistency in a particular sample, it is feasible to run analyses at the level of single items (i.e., at the level of individual core beliefs) rather than the level of subscale means or total scores. In doing so, we strongly recommend statistically correcting for multiple comparisons to avoid inflating Type I error rates.

In summary, this study found several important differences in core beliefs between two samples of healthy comparison youth and youth at ultra-high risk for psychosis. The two groups reported different profiles of core beliefs, with UHR youth reporting less positive beliefs and more negative beliefs. Exploratory factor analyses revealed that UHR youths' core beliefs were best represented by an oblique four-factor structure of positive and negative beliefs about the self and others. HC youth, who rarely reported negative self-beliefs, were best represented by an oblique three-factor structure of positive beliefs about the self, and positive and negative beliefs about others. We also found that HC youths' core beliefs were linked to subclinical symptoms of both depression and anxiety, as predicted by cognitive models of depression and anxiety. In contrast, we found that UHR youths' core beliefs were linked to attenuated psychotic symptoms (particularly negative symptoms and suspiciousness) and depressive symptoms, but not anxiety symptoms. This study contributes to a growing literature on cognitive factors in youth at high-risk for psychotic disorders. We hope to stimulate further research on developmental relationships between core beliefs, psychotic symptoms, and internalizing symptoms.

Acknowledgments

This work was supported in part by the National Institutes of Health RO1MH094650 and R21/R33MH103231 to V.A.M. and the Canadian Institutes of Health Research DFS-152268 to H.R.C. We would like to thank our many supportive colleagues, and the participants without whom this research would not have been possible.

Footnotes

Dr. Mittal is a consultant for Takeda. No other authors have any disclosures.

1

Note that in this model, negative beliefs contribute to positive psychotic symptoms. Beliefs are described as positive or negative when they evaluate something in a positive or negative way (e.g., “I am good” vs. “I am bad”). Psychotic symptoms are described as positive or negative when they either add something unusual to normative human experience (e.g., hallucinations, delusions), or remove something from normative human experience (e.g., anhedonia, avolition). These terms emerged from unrelated research traditions and do not always fit together neatly. For instance, positive psychotic symptoms frequently include negative evaluations (e.g., a self-critical voice is a common type of auditory hallucination).

2

McDonald's hierarchical omega can be computed easily in R using the omega() function in the psych package (Revelle, 2016). Cronbach's alpha can be calculated in most statistical software packages.

Contributor Information

Henry R. Cowan, Psychology, Northwestern University

Dan P. McAdams, Psychology, School of Education and Social Policy, Northwestern University

Vijay A. Mittal, Psychology, Psychiatry, Institute for Policy Research, Medical Social Sciences, Institute for Innovations in Developmental Sciences, Northwestern University

References

  1. Adamson L, Hartman SG, Lyxell B. Adolescent identity—a qualitative approach: Self-concept, existential questions and adult contacts. Scandinavian Journal of Psychology. 1999:21–3a. 1. doi: 10.1111/1467-9450.00094. [DOI] [PubMed] [Google Scholar]
  2. Addington J, Tran L. Using the Brief Core Schema Scales with individuals at clinical high risk of psychosis. Behavioral and Cognitive Psychotherapy. 2009;37:227–31. doi: 10.1017/s1352465809005116. [DOI] [PubMed] [Google Scholar]
  3. Addington J, Epstein I, Reynolds A, Furimsky I, Rudy L, et al. Zipursky R. Early detection of psychosis: Finding those at clinical high risk. Journal of Early Intervention in Psychiatry. 2008;2:147–153. doi: 10.1111/j.1751-7893.2008.00078.x. [DOI] [PubMed] [Google Scholar]
  4. Addington J, Penn D, Woods SW, Addington D, Perkins DO. Social functioning in individuals at clinical high risk for psychosis. Schizophrenia Research. 2008;99(1-3):119–124. doi: 10.1016/j.schres.2007.10.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Bardhoshi G, Duncan K, Erford BT. Psychometric meta-analysis of the English version of the Beck Anxiety Inventory. Journal of Counseling & Development. 2016;94:356–373. doi: 10.1002/jcad.12090. [DOI] [Google Scholar]
  6. Bechdolf A, Wagner M, Ruhrmann S, Harrigan S, Putzfeld V, Pukrop R, et al. Klosterkötter J. Preventing progression to first-episode psychosis in early initial prodromal states. The British Journal of Psychiatry. 2012;200:22–29. doi: 10.1192/bjp.bp.109.066357. [DOI] [PubMed] [Google Scholar]
  7. Beck AT. Cognitive therapy and the emotional disorders. New York, NY: Penguin Books; 1979. [Google Scholar]
  8. Beck AT, Steer RA. Beck Anxiety Inventory manual. San Antonio, TX: Psychological Corporation; 1993. [Google Scholar]
  9. Beck AT, Emery G, Greenberg RL. Anxiety disorders and phobias: A cognitive perspective. New York: Basic Books; 1985. [Google Scholar]
  10. Beck AT, Steer RA, Brown GK. Manual for the Beck Depression Inventory-II. San Antonio, TX: Psychological Corporation; 1996. [Google Scholar]
  11. Beck JS. Cognitive behavior therapy: Basics and beyond. New York, NY: The Guildford Press; 2011. [Google Scholar]
  12. Beesdo K, Pine DS, Lieb R, Wittchen HU. Incidence and risk patterns of anxiety and depressive disorders and categorization of generalized anxiety disorder. Archives of General Psychiatry. 2010;67(1):47–57. doi: 10.1001/archgenpsychiatry.2009.177. [DOI] [PubMed] [Google Scholar]
  13. Berna F, Göritz AS, Schröder J, Martin B, Cermolacce M, Allé MC, et al. Moritz S. Self-disorders in individuals with attenuated psychotic symptoms: Contribution of a dysfunction of autobiographical memory. Psychiatry Research. 2016;239:333–341. doi: 10.1016/j.psychres.2016.03.029. [DOI] [PubMed] [Google Scholar]
  14. Carol EE, Mittal VA. Resting cortisol level, self-concept, and putative familial environment in adolescents at ultra high-risk for psychotic disorders. Psychoneuroendocrinology. 2015;57:26–36. doi: 10.1016/j.psyneuen.2015.03.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Carragher N, Teesson M, Sunderland M, Newton NC, Krueger RF, Conrod PJ, et al. Slade T. The structure of adolescent psychopathology: A symptom-level analysis. Psychological Medicine. 2015;46(5):981–994. doi: 10.1017/s0033291715002470. [DOI] [PubMed] [Google Scholar]
  16. Cole DA, Maxwell SE, Martin JM, Peeke LG, Seroczynski AD, Tram JM, et al. Maschman T. The development of multiple domains of child and adolescent self-concept: A cohort sequential longitudinal design. Child Development. 2001;72:1723–1746. doi: 10.1111/1467-8624.00375. [DOI] [PubMed] [Google Scholar]
  17. DeVylder JE, Yang LH, Harkavy-Friedman JM, Azimov N, Walder DJ, Corcoran CM. Assessing depression in youth at clinical high risk for psychosis: A comparison of three measures. Psychiatry Research. 2014;215:323–328. doi: 10.1016/j.psychres.2013.12.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Dozois DJ, Rnic K. Core beliefs and self-schematic structure in depression. Current Opinion in Psychology. 2015;4:98–103. doi: 10.1016/j.copsyc.2014.12.008. [DOI] [Google Scholar]
  19. Dozois DJ, Dobson K, Ahnberg JL. A psychometric evaluation of the Beck Depression Inventory-II. Psychological Assessment. 1998;10(2):83–89. doi: 10.1037/1040-3590.10.2.83. [DOI] [Google Scholar]
  20. First M, Spitzer RL, Gibbon M, Williams JB. Structured Clinical Interview for the DSM-IV Axis I Disorders (SCID-I). Patient Edition. American Psychiatric Press; Washington, DC: 1995. [Google Scholar]
  21. Fowler D, Freeman D, Smith B, Kuipers E, Bebbington P, Bashforth H, et al. Garety P. The Brief Core Schema Scales (BCSS): Psychometric properties and associations with paranoia and grandiosity in non-clinical and psychosis samples. Psychological Medicine. 2006;36:749–759. doi: 10.1017/s0033291706007355. [DOI] [PubMed] [Google Scholar]
  22. Freeman D, Garety P, Fowler D, Kuipers, Dunn G, Bebbington P, Hadley C. The London-East Anglia randomised controlled trial of cognitive behaviour therapy for psychosis IV: Self-esteem & persecutory delusions. British Journal of Clinical Psychology. 1998;37:415–430. doi: 10.1111/j.2044-8260.1998.tb01399.x. [DOI] [PubMed] [Google Scholar]
  23. Fusar-Poli P, Borgwardt S, Bechdolf A, Addington J, Riecher-Rössler A, Schultze-Lutter F, et al. Yung A. The psychosis high-risk state: A comprehensive state-of-the-art review. JAMA Psychiatry. 2013;70:107–120. doi: 10.1001/jamapsychiatry.2013.269. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Fusar-Poli P, Deste G, Smieskova R, Barlati S, Yung AR, Howes O, et al. Borgwardt S. Cognitive functioning in prodromal psychosis: A meta-analysis. Archives of General Psychiatry. 2012;69(6):562–571. doi: 10.1001/archgenpsychiatry.2011.1592. [DOI] [PubMed] [Google Scholar]
  25. Fusar-Poli P, Howes OD, Allen P, Broome M, V I, Asselin MC, et al. McGuire P. Abnormal prefrontal activation directly related to pre-synaptic striatal dopamine dysfunction in people at clinical high risk for psychosis. Molecular Psychiatry. 2011;16(1):67–75. doi: 10.1038/mp.2009.108. [DOI] [PubMed] [Google Scholar]
  26. Fusar-Poli P, Howes OD, Allen P, Broome M, Valli I, Asselin M, et al. MP K. Abnormal frontostriatal interactions in people with prodromal signs of psychosis: A multimodal imaging study. Arch Gen Psychiatry. 2010;67(7):683–691. doi: 10.1001/archgenpsychiatry.2010.77. [DOI] [PubMed] [Google Scholar]
  27. Fusar-Poli P, Nelson B, Valmaggia L, Yung AR, McGuire PK. Comorbid depressive and anxiety disorders in 509 individuals with an at-risk mental state: Impact on psychopathology and transition to psychosis. Schizophrenia Bulletin. 2014;40:120–131. doi: 10.1093/schbul/sbs136. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Garety PA, Kuipers E, Fowler D, Freeman D, Bebbington PE. A cognitive model of the positive symptoms of psychosis. Psychological Medicine. 2001;31:189–195. doi: 10.1017/s0033291701003312. [DOI] [PubMed] [Google Scholar]
  29. Graber JA. Internalizing problems during adolescence. In: Lerner RM, Steinberg L, editors. Handbook of Adolescent Psychology. 2nd. Hoboken, NJ: John Wiley & Sons, Ltd; 2013. pp. 587–626. [DOI] [Google Scholar]
  30. Häfner H, Maurer R, Löffler W, Bustamante S, an der Heiden W, Reicher-Rössler, Nowotny B. Onset and early course of schizophrenia. In: Häfner H, Gattaz WF, editors. Search for the causes of schizophrenia. Vol. 3. Berlin: Springer-Verlag; 1995. pp. 43–66. [DOI] [Google Scholar]
  31. Häfner H, Maurer K, Trendler G, an der Heiden W, Schmidt M, Könnecke R. Schizophrenia and depression: Challenging the paradigm of two separate diseases—a controlled study of schizophrenia, depression and healthy controls. Schizophrenia Research. 2005;77(1):11–24. doi: 10.1016/j.schres.2005.01.004. [DOI] [PubMed] [Google Scholar]
  32. Holm S. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics. 1979;6(2):65–70. http://www.jstor.org/stable/4615733. [Google Scholar]
  33. Horn JL. A rationale and test for the number of factors in factor analysis. Psychometrica. 1965;30(2):179–185. doi: 10.1007/BF02289447. [DOI] [PubMed] [Google Scholar]
  34. Howes OD, Montgomery AJ, Asselin MC, Murray RM, Valli I, Tabraham P, et al. McGuire PK. Elevated striatal dopamine function linked to prodromal signs of schizophrenia. Arch Gen Psychiatry. 2009;66(1):13–20. doi: 10.1001/archgenpsychiatry.2008.514. [DOI] [PubMed] [Google Scholar]
  35. Howes O, Bose S, Turkheimer F, Valli I, Egerton A, Stahl D, et al. McGuire P. Progressive increase in striatal dopamine synthesis capacity as patients develop psychosis: A PET study. Molecular Psychiatry. 2011;16(9):885–886. doi: 10.1038/mp.2011.20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Hui C, Morcillo CR, Stochi J, Shelley GF, Painter M, Jones PB, Perez J. Psychiatric morbidity, functioning and quality of life in young people at clinical high risk for psychosis. Schizophrenia Research. 2013;148:175–180. doi: 10.1016/j.schres.2013.05.026. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Kaslow NJ, Adamson LB, Collins MH. A Developmental Psychopathology Perspective on the Cognitive Components of Child and Adolescent Depression. In: Sameroff AJ, Lewis M, Miller SM, editors. Handbook of Developmental Psychopathology. Springer; Boston, MA: 2000. [Google Scholar]
  38. Koreen AR, Siris SG, Chakos M, Alvir J. Depression in first-episode schizophrenia. The American Journal of Psychiatry. 1993;150(11):1643–48. doi: 10.1176/ajp.150.11.1643. [DOI] [PubMed] [Google Scholar]
  39. Kreuger RF, Caspi AC, Moffitt TE, Silva PA. The structure and stability of common mental disorders (DSM-III-R): A longitudinal-epidemiological study. Journal of Abnormal Psychology. 1998;107:216–227. doi: 10.1037/0021-843x.107.2.216. [DOI] [PubMed] [Google Scholar]
  40. Leary MR, Baumeister RF. The nature and function of self-esteem: Sociometer theory. Advances in Experimental Social Psychology. 2000;32:1–62. doi: 10.1016/s0065-2601(00)80003-9. [DOI] [Google Scholar]
  41. McAdams DP. The psychological self as actor, agent, and author. Perspectives on Psychological Science. 2013;8:272–295. doi: 10.1177/1745691612464657. [DOI] [PubMed] [Google Scholar]
  42. McDonald RP. Test theory: A unified treatment. Mahwah, N.J: L. Erlbaum Associates; 1999. [DOI] [Google Scholar]
  43. Miller T, McGlashan T, Woods S, Stein K, Driesen N, Corcoran C, et al. Davidson L. Symptom assessment in schizophrenia prodromal states. Psychiatric Quarterly. 1999;70:273–287. doi: 10.1023/A:1022034115078. [DOI] [PubMed] [Google Scholar]
  44. Modinos G, Renken R, Ormel J, Aleman A. Self-reflection and the psychosis-prone brain: An fMRI study. Neuropsychology. 2011;25:295–305. doi: 10.1037/a0021747. [DOI] [PubMed] [Google Scholar]
  45. Nelson B, Fornito A, Harrison BJ, Yücel M, Sass LA, Yung AR, et al. McGorry PD. A disturbed sense of self in the psychosis prodrome: Linking phenomenology and neurobiology. Neuroscience & Biobehavioral Reviews. 2009;33(6):807–817. doi: 10.1016/j.neubiorev.2009.01.002. [DOI] [PubMed] [Google Scholar]
  46. Niendam TA, Jalbrzikowski M, Bearden CE. Exploring predictors of outcome in the psychosis prodrome: Implications for early identification and intervention. Neuropsychology Review. 2009;19:280–293. doi: 10.1007/s11065-009-9108-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Perivoliotis D, Morrison AP, Grant PM, French P, Beck AT. Negative performance beliefs and negative symptoms in individuals at ultra-high risk of psychosis: A preliminary study. Psychopathology. 2009;42(6):375–379. doi: 10.1159/000236909. [DOI] [PubMed] [Google Scholar]
  48. Peterson AC, Compas BE, Brooks-Gunn J, Stemmler M, Ey S, Grant KE. Depression in Adolescence. American Psychologist. 1993;48(2):155–68. doi: 10.1037//0003-066x.48.2.155. [DOI] [PubMed] [Google Scholar]
  49. Piskulic D, Addington J, Cadenhead KS, Cannon TD, Comblatt BA, Heinssen R, et al. McGlashan TH. Negative symptoms in individuals at clinical high risk of psychosis. Psychiatry Research. 2012;196:220–224. doi: 10.1016/j.psychres.2012.02.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. R Core Team. Vienna, Austria: 2016. R: A langauge and environment for statistical computing. https://www.R-project.org. [Google Scholar]
  51. Rector NA, Beck AT, Stolar N. The negative symptoms of schizophrenia: A cognitive perspective. Canadian Journal of Psychiatry. 2005;50:247–257. doi: 10.1177/070674370505000503. [DOI] [PubMed] [Google Scholar]
  52. Revelle W. psych: Procedures for Personality and Psychological Research (1.6.9) Northwestern University; Evanston, IL, USA: 2016. [Google Scholar]
  53. Richter P, Werner J, Heerlein A, Kraus A, Sauer H. On the validity of the Beck Depression Inventory: A review. Psychopathology. 1998;31:160–168. doi: 10.1159/000066239. [DOI] [PubMed] [Google Scholar]
  54. Rogers AH. The self concept in paranoid schizophrenia. Journal of Clinical Psychology. 1958;14:365–366. doi: 10.1002/1097-4679(195810)14:4<365::aid-jclp2270140406>3.0.co;2-0. 10.1002/1097-4679(195810)14:4<365∷aid-jclp2270140406>3.0.co;2-0. [DOI] [PubMed] [Google Scholar]
  55. Rosenberg M. Conceiving the self. New York: Basic Books; 1979. [Google Scholar]
  56. SAS Institute Inc. JMP® Pro, Version 13.0.0. Cary, NC: 1989-2013. [Google Scholar]
  57. Schwartz G. Estimating the dimension of a model. The Annals of Statistics. 1978;6:461–464. doi: 10.1214/aos/1176344136. [DOI] [Google Scholar]
  58. Sebastian C, Burnett S, Blakemore SJ. Development of the self-concept during adolescence. Trends in Cognitive Science. 2008;12(11):441–446. doi: 10.1016/j.tics.2008.07.008. [DOI] [PubMed] [Google Scholar]
  59. Shah R, Waller G. Parental style and vulnerability to depression: The role of core beliefs. Journal of Nervous & Mental Disease. 2000;188(1):19–25. doi: 10.1097/00005053-200001000-00004. [DOI] [PubMed] [Google Scholar]
  60. Smith B, Fowler DG, Freeman D, Bebbington P, Bashforth H, Garety P, et al. Kuipers E. Emotion and psychosis: Links between depression, self-esteem, negative schematic beliefs and delusions and hallucinations. Schizophrenia Research. 2006;86:181–188. doi: 10.1016/j.schres.2006.06.018. [DOI] [PubMed] [Google Scholar]
  61. Stowkowy J, Liu L, Cadenhead KS, Cannon TD, Cornblatt BA, Perkins DO, et al. Addington J. Core schemas in youth at high risk for psychosis. Behavioural and Cognitive Psychotherapy. 2016;44:203–213. doi: 10.1111/j.1751-7893.2011.00297.x. [DOI] [PubMed] [Google Scholar]
  62. Tackett JL, Quilty LC, Sellbom M, Rector NA, Bagby RM. Additional evidence for a quantitative hierarchical model of mood and anxiety disorders for DSM–V: The context of personality structure. Journal of Abnormal Psychology. 2008;117:815–825. doi: 10.1037/a0013795. [DOI] [PubMed] [Google Scholar]
  63. Taylor HE, Stewart SL, Dunn G, Parker S, Fowler D, Morrison AP. Core schemas across the continuum of psychosis: A comparison of clinical and non-clinical groups. Behavioural and Cognitive Psychotherapy. 2014;42(6):718–730. doi: 10.1017/s1352465813000593. [DOI] [PubMed] [Google Scholar]
  64. Tiernan B, Tracey R, Shannon C. Paranoia and self-concepts in psychosis: A systematic review of the literature. Psychiatry Research. 2014;216:303–313. doi: 10.1016/j.psychres.2014.02.003. [DOI] [PubMed] [Google Scholar]
  65. Van Vlierberghe L, Braet C, Bosmans G, Rosseel Y, Bögels S. Maladaptive schemas and psychopathology in adolescence: On the utility of Young's schema theory in youth. Cognitive Therapy and Research. 2010;34:316–332. doi: 10.1007/s10608-009-9283-5. [DOI] [Google Scholar]
  66. Velicer WF. Determining the number of components from the matrix of partial correlations. Psychometrika. 1976;41:321–327. doi: 10.1007/BF02293557. [DOI] [Google Scholar]
  67. Vorontsova N, Garety P. Cognitive factors maintaining persecutory delusions in psychosis: The contribution of depression. Journal of Abnormal Psychology. 2013;122:1121–1131. doi: 10.1037/a0034952. [DOI] [PubMed] [Google Scholar]
  68. Young JE. Cognitive therapy for personality disorders: A schema-focused approach. Sarasota, FL: Professional Resource Press; 1994. [Google Scholar]
  69. Zavos HM, Eley TC, McGuire P, Plomin R, Cardno AG, Freeman D, Angelica R. Shared etiology of psychotic experiences and depressive symptoms in adolescence: A longitudinal twin study. Schizophrenia Bulletin. 2016;42:1197–1206. doi: 10.1093/schbul/sbw021. [DOI] [PMC free article] [PubMed] [Google Scholar]
  70. Zinbarg RE, Revelle W, Yovel I, Li W. Cronbach's α, Revelle's β, and McDonald's ωH: Their relations with each other and two alternative conceptualizations of reliability. Psychometrika. 2005;70:123–133. doi: 10.1007/s11336-003-0974-7. [DOI] [Google Scholar]

RESOURCES