Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Aug 1.
Published in final edited form as: J Anxiety Disord. 2020 Jun 12;74:102259. doi: 10.1016/j.janxdis.2020.102259

The Skidmore Anxiety Stigma Scale (SASS): A covert and brief self-report measure

Casey A Schofield a,*, Gabriella T Ponzini b
PMCID: PMC7780731  NIHMSID: NIHMS1657070  PMID: 32585425

Abstract

Shame and concerns about stigma are salient barriers to treatment for people with anxiety disorders, and yet very little stigma research has focused on this class of disorders. One explanation for this research gap is the absence of a brief, psychometrically sound measure for assessing public stigma for the anxiety disorders as a class. This (three-study) paper presents the psychometric properties of a 7-item scale that covertly assesses anxiety stigma by presenting as a test of knowledge. Items for the measure were derived from a mixed-methods project (Study 1) which assessed patient (N = 47) experiences with stigma. Subsequently, exploratory factor analysis (N = 270) demonstrated that the scale fit a one-factor solution (Study 2). Study 3 comprehensively evaluated the measure’s psychometric properties, including confirming the one-factor solution. Results further demonstrated test-retest reliability, convergent and discriminant validity, and internal consistency. This brief measure fills an important gap by providing means for covertly assessing public stigma encountered by individuals with anxiety disorders and thus subverts social desirability concerns that plague self-report measures of stigma. Thus, the SASS increases the feasibility of work capturing the nature and impact of anxiety stigma – a highly relevant barrier to treatment.

Keywords: Anxiety disorder, Stigma, Scale, Measure, Covert

1. Introduction

Stigma arises when an identity is “marked” as different, and subsequently labeled as discrediting or inferior (Link & Phelan, 2001). As Goffman noted decades ago, this process effectively serves to dehumanize individuals with such a mark, thus activating prejudicial attitudes and discriminatory acts (Goffman, 1963). The consequences of stigma are real in terms of status loss and discrimination, and include health disparities (Chaudoir, Earnshaw, & Andel, 2013), challenges to obtaining housing (Corrigan, Larson, Watson, Boyle, & Barr, 2006), and employment discrimination (Corbiere et al., 2011; Hipes, Lucas, Phelan, & White, 2016). In short, “stigma plays an underrecognized role in the distribution of life chances” (Link & Hatzenbuehler, 2016, p. 653).

In terms of psychological suffering, stigma is frequently identified as a barrier to timely intervention, and thus effective treatment (e.g., Clement et al., 2014; Corrigan, Druss, & Perlick, 2014; Gulliver, Griffiths, & Christensen, 2010; Thornicroft, 2008). Of importance, the construct of stigma has multiple dimensions, and the clarity of previous work in this area has suffered from inconsistencies in definition and measurement across laboratories and disciplines (see Fox, Earnshaw, Taverna, & Vogt, 2018 for a review). That said, the cascade of negative consequences associated with mental health stigma arguably begins with negative stereotypes held by the public about individuals with psychological disorders (sometimes referred to as “public stigma”; Chaudoir et al., 2013; Corrigan, 2018). For people with psychological disorders, fear of encountering these negative attitudes, concerns about the consequences of receiving a label of a negatively stereotyped identity, and discomfort pursuing a course that amplifies one’s own sense of shame, all seem to undermine willingness to seek treatment (Corrigan et al., 2014). In this way, stigmatizing attitudes are directly responsible for some amount of the enormous costs and consequences of untreated psychological disorders. That said, the majority of research about the nature and impact of stigma for psychological disorders focuses on a small number of disorders (i.e., primarily schizophrenia and depression). In fact, a systematic review in the area of stigma research and treatment barriers has identified that the anxiety disorders in particular are under-represented (comprising less than 3% of the studies in their review; Clement et al., 2014). In light of this limited range in stigma research, researchers have called for a broadening of the focus of this area of research (Hinshaw, 2007).

Despite little work, there is good reason to believe that people with anxiety disorders do encounter stigma (Calear, Batternham, Griffths, & Christensen, 2017), and that stigma undermines treatment seeking. For one, anxious patients report that shame (Goetter et al., 2018) and fear of what others will think (Olfson et al., 2000) represent some of the most salient barriers to seeking treatment. Further, some evidence suggests that stigma encountered by patients with anxiety disorders is commensurate with other psychological disorders (Patten et al., 2016). In some ways, the stereotypes of patients with anxiety disorders may be similar to those of other patient groups. For example, previous work has demonstrated that, in terms of personal weakness, participants evaluate people with an anxiety disorder comparably to people with depression, post-traumatic stress disorder, and psychosis (Yap, Reavley, Mackinnon, & Jorm, 2013). Further, participants rate vignettes depicting an anxiety disorder similarly to vignettes depicting depression on patient blameworthiness and prognostic pessimism (Wood, Birtel, Alsawy, Pyle, & Morrison, 2014). Still, research demonstrates that the form of stigmatizing attitudes varies some as a function of the disorder that produces the discrediting mark, and the nature of negative stereotypes applied to people with anxiety disorders may be unique relative to disorders that have received more research attention. For example, anxiety may be considered more embarrassing than depression or generic “mental illness” (Anderson, Jeon, Blenner, Wiener, & Hope, 2015), and people may be more inclined to behave dismissively toward anxious symptoms (Robinson, Turk, Sagar, & Celia, 2019). Further, research participants tend to endorse less concern about dangerous behavior for patients with anxiety disorders relative to other psychological disorders (e.g., Wood et al., 2014; Yap et al., 2013). As a result, stigma measures that incorporate perceptions of dangerousness while failing to assess invalidating and trivializing attitudes may systematically underestimate the scope and impact of stigma for people with anxiety disorders. In brief, just as psychological disorders themselves are highly differentiated, so too mental health stigma is not a monolith.

There is a large body of measures for assessing the various forms of stigma encountered by people with psychological disorders (Fox et al., 2018; Wei, McGrath, Hayden, & Kutcher, 2015). As recent systematic reviews highlight, the psychometric quality of these measures spans a rather large range, with many measures presenting weak to no psychometric support (Fox et al., 2018; Wei, McGrath, Hayden, & Kutcher, 2017). Further, the overwhelming majority of self-report measures of stigma are direct measures in which participants respond to items where they rate (i.e., on a Likert scale) attitudes towards people with a psychological disorder on some undesirable characteristic (e.g., “people with an anxiety disorder do not make suitable employees”). As noted by others (Stier & Hinshaw, 2007; Hinshaw & Stier, 2008; Wei et al., 2017), in light of evidence that direct measurement of sensitive topics yield responses that are systematically biased towards socially acceptable attitudes (Maccoby & Maccoby, 1954), there are serious concerns about the validity of findings that rely exclusively on face-valid measures of stigma. Notably, only 1 of the 101 measures evaluated in the systematic review conducted by Wei et al. (2017) took an approach designed to circumvent socially desirable responding (cf. Hepperlen et al., 2002, measuring ADHD stigma).

To our knowledge, no indirect self-report measure exists for capturing anxiety disorder stigma. The anxiety disorders represent the most common category of psychological suffering (Bandelow & Michaelis, 2015) and upwards of 50 % of individuals affected by an anxiety disorder will not access treatment at any point in their lifetime (2007, Wang et al., 2005). For these reasons, such a measure would have important implications in terms of understanding barriers to treatment for these common and undertreated (but treatable; Cuijpers et al., 2013) psychological disorders. The series of three studies described herein catalogue the development and evaluation of a covert measure of stigmatizing stereotype beliefs held about individuals with anxiety disorders. The central goal of Study 1 was to characterize the scope and nature of experiences with stigma among patients with anxiety disorders, as well as to ensure we had sufficient coverage of the relevant forms of anxiety stigma according to stakeholders (i.e., patients). To do this, we recruited participants who self-identified as having an anxiety disorder from the anxiety subreddit (/r/anxiety) of reddit.com (cf. Jamnik & Lane, 2017) to complete a mixed-methods survey. Participants responded to a series of items assessing whether they ever encountered stigma, the impact stigma had on their life, and the frequency with which they encountered particular potential stereotype beliefs. These beliefs were generated from a combination of previous empirical work as well attention to comments made online about people with anxiety disorders. We expected to observe that encounters with stigmatizing attitudes were common, and that patients found that stigma negatively impacted a range of domains in their life. Results from this initial study informed the development of items designed to covertly measure anxiety stigma. The subsequent two studies evaluated the psychometric properties of this measure.

2. Study 1 method

2.1. Participants

Reddit.com is a web-based discussion board that describes itself as “the front page of the internet” and is, in fact, one of the most popular websites on the internet (Nguyen, 2018). There are over one million “subreddits” which cover an expansive range of interests. The anxiety subreddit (/r/anxiety) is devoted to “discussion and support for sufferers and loved ones of any anxiety disorder” and includes over 221,000 subscribers. Participants (N = 47) for this study were recruited from this subreddit from a post explaining that “This survey will help us understand common stigmas that people with anxiety disorders experience.” Participants’ ages ranged from 18 to 48 (M = 26.50, SD = 6.89) and had completed an average of 15.2 years of formal education (range: 10–23 years). The majority of the sample (74.5 %) identified as female (23.4 % male, 2.1 % non-binary). The sample was homogenous both in terms of race (95.5 % White, 4.4 % Asian) and ethnicity (90.9 % not Hispanic/Latinx, 9.1 % Hispanic or Latinx). On average, the sample reported self-reported symptoms of anxiety (M = 12.39, SD = 3.78) that well-exceeded the clinical threshold for the measure (OASIS > 8; Campbell-Sills et al., 2009).

2.2. Measures

2.2.1. Anxiety symptoms

Symptoms were measured with the Overall Anxiety Severity and Impairment Scale (Norman, Cissell, Means-Christensen, & Stein, 2006). This brief measures assesses symptoms across anxiety disorders and has demonstrated both reliability and validity (Norman et al., 2006) as well as the capacity to detect clinically significant symptoms (Campbell-Sills et al., 2009). Internal consistency in the current sample was excellent (α = .92).

2.2.2. Experience with and impact of stigma

In order to assess experience with stigma, participants responded to the prompt “have you ever experienced stigma?”. In order to assess the impact stigma has had on participants’ lives, they reported about whether stigma negative impacted each of the following domains: Professional life, Romantic relationships, Family relationships, Friendships, Self-esteem, Willingness or comfort seeking treatment, Academics. Each item was answered on a scale with the following options: 0: “Definitely not,” 1: “Probably not,” 2: “Might or might not,” 3: “Probably yes,” and 4: “Definitely yes.”

2.2.3. Perception of specific stigmatizing attitudes

Based on previous literature on stigma enacted towards individuals with anxiety disorders (Griffiths, Batterham, Barney, & Parsons, 2011) as well review of comments on websites (e.g., YouTube videos in which people describe their struggles with anxiety) seven statements were presented that captured presumed stigmatizing attitudes held against people with anxiety disorders. This measure assessed stakeholder perceptions of how common each of these attitudes are (based on the following scale −2: “Very uncommon view,” −1: “Somewhat uncommon view,” 0: “Neither common or uncommon”, 1: “Somewhat common view,” 2: “Very common view”). Finally, we provided an open-ended question where participants were prompted to share whether they had suggestions about additional common stigmatizing attitudes that were not captured by the list.

2.3. Procedure

Participants provided informed consent and then were asked to describe their encounters with stigma. Next, participants reported the impact of stigma on their life and whether they believed a series of stigmatizing statements represented common beliefs about individuals with anxiety. They were also invited to offer other stigmatizing beliefs that the statements listed did not capture. Participants then completed a series of items asking about their previous experience with mental health treatment, diagnosis of anxiety disorder(s), and current symptoms of anxiety. Lastly, participants completed demographic questions and were debriefed.

3. Study 1 results

3.1. Experience with and impact of stigma

The overwhelming majority of respondents indicated that they “probably” (23.4 %) or “definitely” (70.2 %) experienced stigma (the remaining 6.4 % reported they “might or might not” have). Furthermore, participants reported that stigma had a notable impact across life domains, with a particularly striking impact on self-esteem (see Fig. 1).

Fig. 1.

Fig. 1.

Open in a new tab

Self-report of whether stigma affects various life domains (0 = definitely not and 4 = definitely yes).

3.2. Perception of specific stigmatizing stereotypes

On average, each of the beliefs were rated as “somewhat common” to “very common.” The beliefs that were considered most common were that people with anxiety disorders exaggerate their symptoms of anxiety (M = 1.51, SD = 0.75) and that anxiety disorders are nothing but everyday stress (M = 1.45, SD = 0.15) (see Fig. 2). Open ended responses generally indicated that the items captured the nature of stigma that was familiar to them (e.g., “y’all really hit the nail on the head”), although participants did offer suggestions of additional stereotypes, including perceptions/assumptions of weakness, assumptions that anxiety is a “performance” in order to avoid responsibilities, and dismissive attitudes reflected in suggestions that small lifestyle changes will sufficiently address symptoms of an anxiety disorder (e.g., just drink more tea). These suggestions motivated the inclusion of three additional items to the scale, for a total of 10 statements.

Fig. 2.

Fig. 2.

Open in a new tab

Self-reported perceptions of how common various negative stereotypes about people with anxiety disorders are.

4. Interim discussion

Results from this study provide initial evidence for the pervasive and negative impacts of stigma for individuals with anxiety disorders. Although our Reddit-based sample was small and homogenous, this stake-holder data is consistent with findings from previous literature (Davies, 2000) and informal commentary across websites frequented by patients. Indeed, these data highlight the utility of a valid and reliable measure to capture these attitudes. Such measurement will be particularly important for researchers who aim to test and implement anxiety-specific stigma reduction strategies. In the absence of such a measure, targeted interventions aimed at reducing public stigma for anxiety will either rely on insufficient measures developed for a single study, inappropriate measures of stigma that are not anxiety-specific or are strongly affected by social desirability, and perhaps most concerning: researchers may avoid stigma-reduction work for the anxiety disorders altogether.

As mentioned previously, self-report measures of stigma have been rightly criticized for their potential to elicit socially desirable responding (Steir & Hinshaw, 2007). And, although there is a measure designed to capture stigma enacted towards patients with generalized anxiety disorder (Generalized Anxiety Stigma Scale; Griffiths et al., 2011) to our knowledge there are no measures to assess anxiety-specific stigma across anxiety disorders that also employs an indirect approach. One approach to assessing stigma in a covert manner is to frame the measure as a knowledge test, rather than a test of attitudes (see, for example the “error choice” technique; Hammond, 1948). The advantage of disguising the intent of the scale is that participants are presumably more likely to provide responses that reflect true attitudes (as compared to what they believe is socially acceptable) when they understand the measure to be assessing what they know, rather than what the feel (cf., Corrigan & Shapiro, 2010).

The aim of study 2 was to develop a measure that assesses stereotypes the public holds about people with anxiety disorders (reflected in both study one and previous research) while masking the true purpose of the scale as a measure assessing knowledge. In order to do so, each item requires participants to provide a percentage estimate, and all scale items were written so that higher percentage estimates reflect more negative perceptions of people with anxiety disorders. Notably, all items appear as though they are assessing a known correct answer; for example, given that patients indicated that one common stigmatizing attitude they encounter is the belief that people with anxiety disorders exaggerate their symptoms, we asked people to indicate “What percentage of people with an anxiety disorder exaggerate their symptoms?”. In addition to the 7 domains of stigma evaluated in study one, we added three additional items based on the qualitative suggestions of participants (i.e., dismissive perceptions that anxiety can be wholly managed by diet and exercise, anxiety as a means for avoiding responsibility, and assumptions that people with anxiety disorders are a burden to friends/family). Further, we modified the language for one item to improve readability (i.e., changed the language from “lack resilience” to “weak”). Further, acknowledging the value of especially brief measures (given that brevity increases the feasibility of including a measure in research), we aimed to create a scale that was as short as possible while still psychometrically sound. Study 2 comprised an exploratory factor analysis designed to inform item reduction.

5. Study 2 method

5.1. Participants

Participants (N = 283) for this study were recruited from mTurk. Of the recruited sample, 13 participants failed to pass the attention check (described below) and thus were removed from analyses, yielding a final sample of N = 270. Ages ranged from 19 to 71 (M = 35.5, SD = 10.6) and participants completed an average of 15.2 years of education (range: 10–24). Approximately half (50.6 %) of the sample identified as female (48.7 % male, 0.4 % transgender, 0.4 % non-binary). Participants reported the following racial identities: 83.8 % White, 7.9 % Black or African American, 4.9 % Asian, 3.0 % Bi/Multiracial, and 0.4 % American Indian. In terms of ethnicity, 5.7 % of the sample identified as Hispanic or Latinx.

5.2. Measure

5.2.1. Anxiety stigma measure

This scale consisted of 10 statements designed to capture commonly held stigmatizing attitudes held about people with anxiety disorders based on the results from Study 1. The measure included instructions explaining that the measure aims to test anxiety knowledge (see instructions in Appendix A). Each item was written such that participants were expected to provide estimates of how common they believe a particular characteristic of anxiety disorders (or people with anxiety disorders) is/are. As such, each question was answered on a 0 %–100 % scale. For all items, a higher estimated percentage reflects stronger endorsement of a stigmatizing attitude (e.g., “What percentage of people with an anxiety disorder are too weak to overcome their symptoms?”).

5.2.2. Attention check

In addition to these items, the scale included an attention check item (i.e., “What percentage of people are paying attention to this test? Please move the slider to 50″). Responses that ranged from 48 to 52 were considered evidence of attention to the measure (i.e., when scores were outside that range, data was not included).

5.3. Procedure

Following informed consent, participants responded to the 10-item measure. Then, all participants responded to demographic questions and were debriefed and paid $0.50.

5.4. Analysis plan

We conducted Exploratory Factor Analysis (EFA) with principal axis factoring in SPSS v25. At each stage of the EFA data were assessed for sphericity (Bartlett, 1950; all ps < .001) and the Kaiser-Meyer-Olkin measure of sampling adequacy (all KMO values > .89; Kaiser, 1974). We employed promax rotation (i.e., allowed factors to correlate).

6. Study 2 results

6.1. Exploratory factor analysis

An Eigenvalue cut-off of 1.0 yielded two factors (where factor one explained 49.45 % of variability and factor two explained 10.5 % of variability). We reduced the scale to one factor (i.e., dropped the three items that loaded onto factor two) in light of a number of measure characteristics. First, there is an established tendency for Kaiser’s eigenvalue > 1 rule tending to overestimate the number of factors (Field, 2009) and the scree plot revealed a clear inflection point after one factor. Further, the second factor of three items was poorly defined; it had insufficient “marker items” (Brown & Kenny, 2006) yielding lack of clarity/coherence for this factor. Finally, we did not have theoretical justification for a two-factor measure. As such, the items that loaded onto factor two (e.g., “what percentage of people with anxiety have friends and/or family that report feeling burdened by their anxiety?”) were dropped from the scale. A subsequent EFA of the remaining seven items indicated that all items loaded meaningfully onto a single factor (all loadings > 0.50; see Table 1 and explained 58.74 % of total variance. Lastly, these seven items demonstrated good internal consistency (α = .88).

Table 1.

Summary of Exploratory Factor Analysis Results for Skidmore Anxiety Stigma Scale Using Principal Axis Factoring (N = 270).

Item Summary Factor Loading
1. Exaggerate symptoms .779
2. Do not require treatment .501
3. Too weak to overcome symptoms .698
4. Nothing but everyday stress .777
5. Use symptoms to attract attention .857
6. Treated by changing diet and exercise .552
7. Use symptoms to avoid responsibilities .837
Open in a new tab

7. Interim discussion

Data from Study 2 provided initial evidence for this scale as a brief measure of stigmatizing attitudes held about people with anxiety disorders. EFA provided support for a one-factor solution (retaining seven of the original ten items), and this shortened measure accounted for significant variance in our data and demonstrated strong internal consistency. The goal of the subsequent study was to evaluate the psychometric properties of this 7-item scale.

Study 3 included a series of psychometric tests; confirmatory factor analysis to test the one-factor solution and convergent validity with an established stigma measure (Attribution Questionnaire; Corrigan, Markowitz, Watson, Rowan, & Kubiak, 2003, using a version adapted for an anxiety focus and utilized in previous work; cf. Ponzini & Schofield, 2019). Further, since stigma is an established barrier to helpseeking, we assessed convergent validity with a measure of attitudes towards seeking professional help (ATTSPH; Fischer & Farina, 1995). In addition, given that symptoms of anxiety are a distinct construct from stigmatizing views about anxiety, we tested discriminant validity with a measure of anxiety symptoms (OASIS; Campbell-Sills et al., 2009).

Additionally, to establish whether the SASS captures stigma specific to anxiety disorders (versus stigma generally enacted against people with a health condition), we conducted a test of contrasting groups validity. Participants were randomly assigned to complete either this anxiety stigma measure or a measure of the same items characterizing perceptions of diabetes (e.g., “What percentage of people with diabetes exaggerate their symptoms?”). We chose diabetes as a comparison condition here given that diabetes is distinct from the psychological disorders but is a well-recognized medical condition that provokes stigmatizing attitudes (Schabert, Browne, Mosely, & Speight, 2013). Lastly, Study 3 assessed both internal consistency and test-retest reliability (with one-week delay). All study hypotheses were preregistered with the Open Science Framework (https://osf.io/4kqru/). We hypothesized that the factor structure of the measure (i.e., the Skidmore Anxiety Stigma Scale; SASS) would be confirmed in an independent sample, and that the measure would demonstrate reliability (via test-retest and internal consistency). In addition, we expected that if the SASS is a valid measure of public stigma for anxiety it will be significantly positively correlated with an anxiety-focused version of an established measure (i.e., Attribution Questionnaire), and significantly negatively correlated with positive attitudes towards seeking professional help for emotional distress while unrelated to symptoms of anxiety. Lastly, we expected that if the SASS measures public stigma for anxiety in particular, respondents will endorse more stigma on the SASS than when the same items are rated for a condition for which condition for which they are not especially relevant (i.e., diabetes).

8. Study 3 method

8.1. Participants

Four hundred and two participants were initially recruited for this study via mTurk. Of this initial sample, 28 participants were removed from subsequent analyses either for dropping out of the study immediately following consent (n = 8) or for failing to pass the attention check defined in Study 2 (n = 20). Time one analyses were conducted on the remaining (N = 374) participants. Of this total sample n = 185 were randomly assigned to condition in which they completed the SASS, and thus psychometric characteristics of the SASS were conducted on this subgroup. Ages for this subsample ranged from 19 to 71 (M = 37.99, SD = 11.75) and participants completed an average of 15.2 years of education (range: 12–25). Approximately half (52.2 %) of the sample identified as female (47.3 % male, 0.3 % transgender, 0.3 % non-binary). Participants reported the following racial identities: 82.1 % White, 10.5 % Black or African American, 5.2 % Asian, 0.6 % Bi/Multiracial, and 0.3 % Native Hawaiian or Pacific Islander. In terms of ethnicity, 6.1 % of the sample identified as Hispanic or Latinx. There were no significant differences in terms of measured demographic characteristics between participants who completed the SASS versus those who completed the diabetes version of the SASS (d-SASS), all ps > .05.

8.2. Measures

8.2.1. Skidmore anxiety stigma scale (SASS)

The seven items derived from Study 2 were included in this measure. The total score for the measure was calculated by summing the 7 items of the SASS and dividing by 10 (such that scores range from 0 to 70, with higher scores reflecting greater stigma; Ponzini & Schofield, 2019). Internal consistency for this measure in the current sample was good (α = .86). Scale details are included in Appendix A.

8.2.2. Diabetes version of SASS (d-SASS)

This measure was identical to the SASS except that the words “anxiety disorders” were replaced with “diabetes” throughout. Internal consistency for this measure in the current sample was good (α = .84).

8.2.3. Revised attribution questionnaire – 9 (rAQ-9)

A revised version of the AQ-9 (Corrigan et al., 2003) was included in order to assess public stigma. This measure was revised consistent with previous research assessing anxiety stigma (Ponzini & Schofield, 2019). Internal consistency for this measure in the current sample was questionable (α = .62).

8.2.4. Attitudes towards seeking professional psychological help (ATSPPH)

Participants’ perceptions about help-seeking for psychological distress were measured by the ATSPPH (Fischer & Farina, 1995). Each of the 10 items was rated on a 0 (disagree) to 3 (agree) scale. Per scale instructions, five items were reverse coded; higher scores indicate more positive attitudes towards psychological help. Internal consistency in the current sample was good (α = .86).

8.2.5. Overall anxiety severity and impairment scale (OASIS)

General symptoms of anxiety were assessed via the OASIS (Campbell-Sills et al., 2009). This five-item measure has demonstrated reliability and validity of this measure in both nonclinical (Norman et al., 2006) and clinical samples (Campbell-Sills et al., 2009). Internal consistency in the sample was strong (α = .93).

8.3. Procedure

Following informed consent, participants were randomized to complete either the SASS or the d-SASS.Participants who completed the SASS then responded to the rAQ-9 (public stigma) and the ATSPPH (attitudes towards help-seeking) to assess convergent validity and the OASIS (self-reported anxiety symptoms) to assess discriminant validity. After, all participants responded to demographics questions and were informed that they may be contacted to complete a brief (i.e., less than two-minute) follow-up assessment. At one-week follow-up (Time 2), participants who completed the SASS at Time 1 were invited to complete the measure again and were debriefed.

9. Data analysis

Confirmatory factor analysis (CFA) was used to examine a measurement model in which all seven items loaded on a single latent variable. This analysis was conducted using the lavaan package (Rosseel, 2012) for the R statistical computing environment (R Core Team, 2018). The model was estimated using robust maximum likelihood with standardized latent and observed variables. Model fit was evaluated using several incremental and absolute fit indexes: the Comparative Fit Index (CFI), the Tucker-Lewis Index (TLI), the Root Mean Squared Error of Approximation (RMSEA), and the Standardized Root Mean Squared Residual (SRMR). Cutoff criteria for these indexes were adopted from Hu and Bentler (1999), i.e., values of CFI and TLI at 0.95 or more, values of RMSEA at 0.06 or less, and values of SRMR at 0.08 or less were taken as indicative of good model fit. Interpreting and reporting multiple fit indexes can help compensate for the shortcomings of any individual index (Hu & Bentler, 1999). Finally, the reliability of the total scale score was assessed using omega (Revelle & Condon, 2018).

All other analyses were conducted using SPSS v25 and included independent samples t-tests and correlations.

9.1. Results

9.1.1. Test-retest reliability

Of the initial sample of participants who completed the SASS at Time 1, 76.2 % of participants completed the SASS at Time 2 (Mdelay to retest = 7.04 days, SD = 0.02) Consistent with study hypotheses, the SASS demonstrated good test-retest reliability (r = 0.84, p < 0.001).

9.1.2. Convergent validity

Consistent with study hypotheses, public stigma as measured by the SASS was positively and significantly correlated with public stigma as measured by the rAQ-9 (r = 0.33, p < 0.001). Furthermore, consistent with study hypotheses, public stigma as measured by the SASS was negatively and significantly correlated with attitudes towards seeking professional help as measured by the ATSPPH (r = −0.42, p < .001).

9.1.3. Discriminant validity

Consistent with study hypotheses, public stigma as measured by the SASS was not significantly correlated with symptoms of anxiety as measured by the OASIS (r = .12, p = 0.10).

9.1.4. Contrasting groups

Consistent with study hypotheses, participants endorsed higher levels of stigma on the SASS when it identified people with anxiety disorders as the focus of stigma (M = 23.74, SD = 19.63) compared to when the measure identified people with diabetes as the focus of stigma (M = 19.63, SD = 11.64), t(372) = 3.33, p = 0.001, d = 0.34.

9.1.5. Confirmatory factor analysis

All seven items had significant loadings on the single latent variable (Table 2). Furthermore, the standardized loadings were quite large in magnitude (M = 0.68), which means that the latent variable accounted for much of the variance in item responses. Although still reasonable in magnitude (i.e., 0.42), it is worth mentioning that the standardized factor loading for the second item in the measure was low relative to the other items, which means that less of the variance in its responses was explained by the latent variable.

Table 2.

Standardized factor loadings from confirmatory factor analysis model.

Item Summary Loading p-value 95 % Cl
1. Exaggerate symptoms 0,733 < .001 [0.624, 0.842]
2. Do not require treatment 0.423 < .001 [0.261, 0.585]
3. Too weak to overcome symptoms 0.654 < .001 [0.521, 0.788]
4. Nothing but everyday stress 0.705 < .001 [0.593, 0.816]
5. Use symptoms to attract attention 0.865 < .001 [0.736, 0.994]
6. Treated by changing diet and exercise 0.581 < .001 [0,455, 0.708]
7. Use symptoms to avoid responsibilities 0.832 < .001 [0.718, 0.946]
Open in a new tab

Based on the four selected fit indexes, the CFA model fit the data well, i.e., all indexes met conventional cutoff criteria (CFI = 0.98, TLI = 0.97, RMSEA = 0.06, and SRMR = 0.04). The reliability of the scale was high (ω = 0.87), which means that only a small amount (i.e., 13 %) of the variance in total scale scores was residual (i.e., item-specific or error variance).

10. Discussion

In the 1970s mental health stigma research was emerging victorious – evidence suggested that stigmatizing beliefs had reduced remarkably, and the consequences of remaining beliefs were slight (Crocetti, Spiro, & Siassi, 1974). These claims were subsequently criticized as undue optimism. Critics noted that the data used to support these claims drew from non-experimental studies using direct self-report measures. As such, this “improvement” mostly likely reflected participants shifting their self-report towards more socially desirable responding, as cultural attitudes themselves had shifted to consider dehumanizing patients with mental health problems to be objectionable (Hinshaw, 2007). In short, the methodological challenges presented by social desirability bias have plagued stigma research for some time. And yet, despite the development of many self-report instruments of mental health stigma, very few are designed to circumvent this challenge. The SASS, which is both brief and covert, facilitates an important novel line of empirical work evaluating stigma enacted against people with anxiety disorders. The current series of studies present strong psychometric evidence for this instrument as a single-factor measure designed to capture a form of public stigma; endorsement of negative stereotypes about people with anxiety disorders. Notably, results from the studies reported herein provide evidence across a number of indices that the measure is both reliable and valid; analyses of construct validity yielded the expected pattern of (preregistered) results.

Legal scholar and public health ethicist Daniel Goldberg noted that, “at the same time that stigma is common, it is also invisible” (Goldberg, 2017, p. 477). The central goal of this project was to increase the visibility of the impact of stigma on the lives of patients with anxiety disorders. Despite recognizing that stigma serves as a barrier to recovery (Wahl, 2012), clinical science has arguably paid insufficient attention to this common and corrosive force in the lives of patients with anxiety disorders. One potential explanation for this research gap is the relative disconnect between the available stigma measurement tools and the form of negative stereotypes that patients with anxiety are most likely to encounter. That is, we have known for some time that different psychological disorders elicit different negative attitudes (Crisp, Gelder, Rix, Meltzer, & Rowlands, 2000), and anxiety disorders in particular may elicit a different set of negative stereotypes than those that established measures address. For example, probably in part because the history of mental health stigma research has focused on schizophrenia (a condition that is often wrongfully characterized by (mis)representations of violence), measures tend to incorporate items about dangerousness and violence. On the other hand, stereotypes about people with anxiety disorders are comprised primarily of invalidating (e.g., symptoms do not warrant treatment), dismissive (e.g., people with anxiety are weak), and skeptical (e.g., people with anxiety are exaggerating) reactions to symptoms (as predicted by Hinshaw & Stier, 2008). Such attitudes are less often addressed in established stigma measures (presumably given that they are less relevant to schizophrenia). Thus, to the extent that established measures of stigma are employed to assess stigma for the anxiety disorders, those measures are liable to yield poor reliability (as demonstrated by the performance of the rAQ in study three) and especially likely to underestimate of the scope and impact of stigma in the lives of anxious patients.

The SASS enables future work that will be important to informing (and ultimately addressing) stigma as a barrier to treatment for the anxiety disorders. First, the SASS allows us to evaluate the relative scope of stigma across different settings (e.g., medical, professional, academic, and interpersonal), communities (e.g. geographic regions, personal identities), and across time. Such information will be valuable in terms of identifying where to target efforts in terms of stigma reduction. The SASS may also address important challenges in testing the impact of campaigns designed to reduce stigma. That is, face-valid self-report measures are particularly vulnerable to demand effects in prepost designs (where participants are aware both of what is being measured, and aware of what they are being taught). These measures may initially underestimate the scope of stigma (due to social desirability) and simultaneously overestimate the impact of a program on these attitudes (due to demand effects). In addition, we see a natural connection between the important work happening in dissemination research (including the various “direct-to-consumer” efforts; Gallo, Comer, Barlow, Clarke, & Antony, 2015) and measuring stigma. We encourage clinical scientists working on such programs designed to improve awareness of evidence-based treatments to embed some of the findings from stigma science into these programs in an effort to affect both knowledge and stigma, given previous evidence suggesting that knowledge about the anxiety disorders and their treatment may be particularly lacking (Coles & Coleman, 2010; Paulus, Wadsworth, & Hayes-Skelton, 2015; Schofield, Moore, Hall, & Coles, 2016) and limited knowledge may be one factor contributing to the development and maintenance of public stigma. With that said, however, previous work suggests that the relation between knowledge and stigma is unlikely to be straightforward. Stigma for anxiety and related disorders is prevalent in even the most knowledgeable professionals (e.g., mental health providers; Steinberg & Wetterneck, 2017) and campaigns designed to improve anxiety knowledge appear to have more a more muted impact on stigma (Ponzini & Schofield, 2019). Further, systematic review and meta-analysis suggests that particular messages (e.g., about biological correlates) may have negative consequences for public attitudes (Schomerus et al., 2012). Accordingly, future work should evaluate the extent to which knowledge may reduce stigma, while also parsing why these constructs may function independent of each other. Ultimately, this work may help to approaches that simultaneously empower patients and dismantle stigma.

Initial psychometric evidence for the SASS suggests that it is reliable and valid, and thus makes research questions about the scope and nature of this potential barrier to treatment more accessible to research. With that said, there are notable limitations to the claims we can make about the measure at this time. Most importantly, there is no evidence for the predictive validity of the scale. Although masking the measure as a knowledge test as compared to an attitude measure addresses some concerns about the social desirability, it will be imperative that future work evaluates whether the SASS predicts adverse outcomes for patients, such as discriminatory behavior. As demonstrated by the literature evaluating implicit assessments of socially undesirable attitudes, predictive validity remains a particular challenge in this area of work (Greenwald, Banaji, & Nosek, 2015). Further, this measure was developed from a literature based in WEIRD countries (Henrich, Heine, & Norenzayan, 2010), and the patient sample was adults who were overwhelmingly White, and primarily female. In light of the heterogeneity of expressions of anxiety across culture (Hofmann & Hinton, 2014), it is unclear at this time whether this measure captures the nature and form of stigma encountered by people in communities that are not represented in this work. In particular, given disparities in terms of access to treatment for people from developing nations, rural settings, communities of color, children, and socioeconomically disadvantaged backgrounds, it will be especially important to know whether this measure is generally applicable to the experiences of people with anxiety disorders. Finally, the SASS was designed to capture stigma at the level of the source (e.g., public attitudes), and support for its use in the assessment of self-stigma (i.e., internalizations of public attitudes, Corrigan & Rao, 2012) remain unexplored. It is plausible, however, that when the SASS is completed by people anxiety disorder, the instrument is in fact capturing internalizations reflective of self-stigma. Likewise, while the SASS is designed to capture stigma as it impacts people with anxiety disorders broadly future research should explore whether adapting the SASS items to indicate particular anxiety disorders (or related disorders such as OCD) yields evidence for particular forms of stigma across conditions.

Nearly half of adults with an anxiety disorder will never access treatment (Wang et al., 2007), and yet there is relatively little work investigating the nature and impact of one of the best recognized barriers to treatment: stigma. In light of patient reports that concerns about encountering stigmatizing attitudes represent a particularly salient barrier to reaching treatment for anxiety, there is apparent need to be able to quantify stigmatizing beliefs about anxiety disorders. Arguably, untreated anxiety represents an especially discouraging missed opportunity because when left untreated, the course is chronic and is associated with the subsequent development of comorbid conditions. We hope that the SASS enables and encourages a growing interest in recognizing (and ultimately, working to address) the subversive impacts of stigma in the lives of our patients.

Appendix A

SASS

This is a test of your knowledge about anxiety. The questions on this test are taken from scientific research findings. You are not expected to have read the research reports, but by using your experience and general knowledge you should be able to estimate the correct answer. Some people will do much better than others because of their experience, or because of their training in medicine, rehabilitation, or psychology.

Read each question carefully and select the percentage that you consider to be the correct answer. THERE IS NO PENALTY FOR GUESSING. There is no time limit for the completion of this test, but you should work as rapidly as you can.

  1. What percentage of people with an anxiety disorder exaggerate their symptoms?

    0 ———— 25 ———— 50 ———— 75 ———— 100

  2. What percentage of anxiety disorders do not require treatment to reduce symptoms?

    0 ———— 25 ———— 50 ———— 75 ———— 100

  3. What percentage of people with an anxiety disorder are too weak to overcome their symptoms?

    0 ———— 25 ———— 50 ———— 75 ———— 100

  4. What percentage of anxiety disorders are nothing but everyday stress?

    0 ———— 25 ———— 50 ———— 75 ———— 100

  5. What percentage of people with an anxiety disorder use their symptoms to attract attention?

    0 ———— 25 ———— 50 ———— 75 ———— 100

  6. What percentage of anxiety disorders can be treated by changing one’s diet and exercise?

    0 ———— 25 ———— 50 ———— 75 ———— 100

  7. What percentage of people with an anxiety disorder use their symptoms to avoid certain responsibilities?

    0 ———— 25 ———— 50 ———— 75 ———— 100

References

  1. Anderson KN, Jeon AB, Blenner JA, Wiener RL, & Hope DA (2015). How people evaluate others with social anxiety disorder: A comparison to depression and general mental illness stigma. The American Journal of Orthopsychiatry, 85, 131–138. 10.1037/ort000004. [DOI] [PubMed] [Google Scholar]
  2. Bandelow B, & Michaelis S (2015). Epidemiology of the anxiety disorders in the 21st century. Dialogues in Clinical Neuroscience, 17, 327–335. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bartlett MS (1950). Tests of significance in factor analysis. British Journal of Statistical Psychology, 3, 77–85. 10.1111/j.2044-8317.1950.tb00285.x. [DOI] [Google Scholar]
  4. Introduction to CFA. In Brown TA, & Kenny DA (Eds.). Confirmatory factor analysis for applied research (pp. 40–102). New York, NY: Guilford Press. [Google Scholar]
  5. Calear AL, Batternham PJ, Griffths KM, & Christensen H (2017). Generalized anxiety disorder stigma in adolescents: Personal and perceived stigma levels and predictors. Stigma and Health, 2, 208–215. 10.1037/sah0000046. [DOI] [Google Scholar]
  6. Campbell-Sills L, Norman SB, Craske MG, Sullivan G, Lang AJ, Chavira DA, et al. (2009). Validation of a brief measure of anxiety-related severity and impairment: The Overall Anxiety Severity and Impairment Scale (OASIS). Journal of Affective Disorders, 112, 92–101. 10.1016/j.jad.2008.03.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Chaudoir SR, Earnshaw VA, & Andel S (2013). “Discredited” versus “discreditable”: Understanding how shared and unique stigma mechanisms affect psychological and physical health disparities. Basic and Applied Social Psychology, 35, 75–87. 10.1080/01973533.2012.746612. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Clement S, Schuaman O, Graham T, Maggioni F, Evans-Lacko S, Bezborodovs N, et al. (2014). What is the impact of mental health-related stigma on help-seeking? A systematic review of quantitative and qualitative studies. Psychological Medicine, 45, 11–27. 10.1017/S0033291714000129. [DOI] [PubMed] [Google Scholar]
  9. Coles ME, & Coleman SL (2010). Barriers to treatment seeking for anxiety disorders: Initial data on the role of mental health literacy. Depression and Anxiety, 27, 63–71. 10.1002/da.20620. [DOI] [PubMed] [Google Scholar]
  10. Corbiere M, Zaniboni S, Lecomte T, Bond G, Gilles P, Lesage A, et al. (2011). Job acquisition for people with severe mental illness enrolled in supported employment programs: A theoretically grounded empirical study. Journal of Occupational Rehabilitation, 21, 343–354. 10.1007/sl0926-011-9315-3. [DOI] [PubMed] [Google Scholar]
  11. Corrigan PW (2018). The stigma effect: Unintended consequences of mental health campaigns. New York, NY: Columbia University Press. [Google Scholar]
  12. Corrigan PW, & Rao D (2012). On the self-stigma of mental illness: Stages, disclosure, and strategies for change. The Canadian Journal of Psychiatry, 57(8), 464–469. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Corrigan PW, & Shapiro JR (2010). Measuring the impact of programs that challenge the public stigma of mental illness. Clinical Psychology Review, 30, 907–922. 10.1016/j.cpr.2010.06.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Corrigan P, Markowitz FE, Watson A, Rowan D, & Kubiak MA (2003). An attribution model of public discrimination towards persons with mental illness. Journal of Health and Social Behavior, 44, 162–179. 10.2307/1519806. [DOI] [PubMed] [Google Scholar]
  15. Corrigan PW, Larson JE, Watson AC, Boyle M, & Barr L (2006). Solutions to discrimination in work and housing identified by people with mental illness. The Journal of Nervous and Mental Disease, 194, 716–718. 10.1097/01.nmd.0000235782.18977.de. [DOI] [PubMed] [Google Scholar]
  16. Corrigan PW, Druss BG, & Perlick DA (2014). The impact of mental illness stigma on seeking and participating in mental health care. Psychological Science in the Public Interest, 15, 37–70. 10.1177/1529100614531398. [DOI] [PubMed] [Google Scholar]
  17. Crisp AH, Gelder MG, Rix S, Meltzer HI, & Rowlands OJ (2000). Stigmatisation of people with mental illnesses. The British Journal of Psychiatry, 177, 4–7. 10.1192/bjp.177.1.4. [DOI] [PubMed] [Google Scholar]
  18. Crocetti G, Spiro HR, & Siassi I (1974). Contemporary attitudes toward mental illness. Pittsburgh, PA: University of Pittsburgh Press. [Google Scholar]
  19. Cuijpers P, Berking M, Andersson G, Quigley L, Kleiboer A, & Dobson KS (2013). A meta-analysis of cognitive-behavioural therapy for adult depression, alone and in comparison with other treatments. The Canadian Journal of Psychiatry, 58, 376–385. 10.1177/070674371305800702. [DOI] [PubMed] [Google Scholar]
  20. Davies MR (2000). The stigma of anxiety disorders. International Journal of Clinical Practice, 54(1), 44–47. [PubMed] [Google Scholar]
  21. Field A (2009). Discovering statistics using SPSS: Introducing statistical method (3rd ed). Thousand Oaks, CA: Sage Publications. [Google Scholar]
  22. Fischer EH, & Farina A (1995). Attitudes toward seeking professional psychologial help: A shortened form and considerations for research. Journal of College Student Development, 36(4), 368–373. [Google Scholar]
  23. Fox AB, Earnshaw VA, Taverna EC, & Vogt D (2018). Conceptualizing and measuring mental illness stigma: The mental illness stigma framework and critical review of measures. Stigma and Health, 3(4), 348–376. 10.1037/sah0000104. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Gallo KP, Comer JS, Barlow DH, Clarke RN, & Antony MM (2015). Direct-to-consumer marketing of psychological treatments: A randomized controlled trial. Journal of Consulting and Clinical Psychology, 83, 994–998. 10.1037/a0039470. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Goetter EM, Frumkin MR, Palitz SA, Swee MB, Baker AW, Bui E, et al. (2018). Barriers to mental health treatment among individuals with social anxiety disorder and generalized anxiety disorder. Psychological Services. 10.1037/ser0000254. [DOI] [PubMed] [Google Scholar]
  26. Goffman E (1963). Stigma: Notes on the management of spoiled identity. New York, NY: Simon & Schuster. [Google Scholar]
  27. Goldberg DS (2017). On stigma & health. The Journal of Law, Medicine & Ethics, 45(4), 475–483. 10.1177/1073110517750581. [DOI] [Google Scholar]
  28. Greenwald AG, Banaji MR, & Nosek BA (2015). Statistically small effects of the Implicit Association Test can have societally large effects. Journal of Personality and Social Psychology, 108, 553–561. 10.1037/pspa0000016. [DOI] [PubMed] [Google Scholar]
  29. Griffiths KM, Batterham PJ, Barney L, & Parsons A (2011). The generalised anxiety stigma scale (GASS): Psychometric properties in a community sample. BMC Psychiatry, 11, 184 10.1186/1471-244X-11-184. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Gulliver A, Griffiths KM, & Christensen H (2010). Perceived barriers and facilitators to mental health help-seeking in young people: A systematic review. BMC Psychiatry, 10, 113 10.1186/1471-244X-10-113. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Hammond KR (1948). Measuring attitudes by error-choice: An indirect method. The Journal of Abnormal and Social Psychology, 43, 38–48. 10.1037/h0059576. [DOI] [PubMed] [Google Scholar]
  32. Henrich J, Heine SJ, & Norenzayan A (2010). Beyond WEIRD: Towards a broad-based behavioral science. Behavioral and Brain Sciences; New York, 33, 111–135. 10.1017/S0140525X10000725. [DOI] [PubMed] [Google Scholar]
  33. Hepperlen TM, Clay DL, Henly GA, Barké CR, Hehperlen TM, & Clay DL (2002). Measuring teacher attitudes and expectations toward students with ADHD: Development of the Test of Knowledge about ADHD (KADD). Journal of Attention Disorders, 5, 133–142. 10.1177/108705470200500301. [DOI] [PubMed] [Google Scholar]
  34. Hinshaw SP (2007). The mark of shame: Stigma of mental illness and an agenda for change. New York, NY: Oxford University Press. [Google Scholar]
  35. Hinshaw SP, & Stier A (2008). Stigma as related to mental disorders. Annual Review of Clinical Psychology, 4, 367–393. 10.1146/annurev.clinpsy.4.022007.141245. [DOI] [PubMed] [Google Scholar]
  36. Hipes C, Lucas J, Phelan JC, & White RC (2016). The stigma of mental illness in the labor market. Social Science Research, 56, 16–25. 10.1016/j.ssresearch.2015.12.001. [DOI] [PubMed] [Google Scholar]
  37. Hofmann SG, & Hinton DE (2014). Cross-cultural aspects of anxiety disorders. Current Psychiatry Reports, 16, 450 10.1007/s11920-014-0450-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Hu L, & Bentler PM (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. 10.1080/10705519909540118. [DOI] [Google Scholar]
  39. Jamnik MR, & Lane DJ (2017). The use of Reddit as an inexpensive source for high-quality data. Practical Assessment, Research & Evaluation, 22(5), 1–10. [Google Scholar]
  40. Kaiser HF (1974). An index of factorial simplicity. Psychometrika, 39, 31–36. 10.1007/BF02291575. [DOI] [Google Scholar]
  41. Link B, & Hatzenbuehler ML (2016). Stigma as an unrecognized determinant of population health: Research and policy implications. Journal of Health Politics, Policy and Law, 41, 653–673. 10.1215/03616878-3620869. [DOI] [PubMed] [Google Scholar]
  42. Link BG, & Phelan JC (2001). Conceptualizing stigma. Annual Review of Sociology, 27, 363–385. 10.1146/annurev.soc.27.1.363. [DOI] [Google Scholar]
  43. Maccoby EE, & Maccoby N (1954). The interview: A tool of social science In Lindzey G (Ed.). Handbook of social psychology: Vol. 1. Theory and method (pp. 449–487). Reading, MA: Addison-Wesley. [Google Scholar]
  44. Nguyen C (2018). Reddit beats out Facebook to become the third most popular site on the web. Digital Trends. May 30Retrieved from: https://www.digitaltrends.com/computing/reddit-more-popular-than-facebook-in-2018/. [Google Scholar]
  45. Norman SB, Cissell SH, Means-Christensen AJ, & Stein MB (2006). Development and validation of an overall anxiety severity and impairment scale (OASIS). Depression and Anxiety, 23, 245–349. 10.1002/da.20182. [DOI] [PubMed] [Google Scholar]
  46. Olfson M, Guardino M, Struening E, Schneier FR, Heilman F, & Klein DF (2000). Barriers to the treatment of social anxiety. American Journal of Psychiatry, 157, 521–527. 10.1176/appi.ajp.157.4.521. [DOI] [PubMed] [Google Scholar]
  47. Patten SB, Williams JVA, Lavorato DH, Bulloch AGM, Charbonneau M, Gautam M, et al. (2016). Perceived stigma among recipients of mental health care in the general Canadian population. The Canadian Journal of Psychiatry, 61, 480–488. 10.1177/0706743716639928. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Paulus DJ, Wadsworth LP, & Hayes-Skelton SA (2015). Mental health literacy for the anxiety disorders: How perceptions of symptom severity might relate to recognition of psychological distress. Journal of Public Mental Health, 14, 94–106. 10.1108/JPMH-09-2013-0064. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Ponzini GT, & Schofield CA (2019). Randomized controlled trials evaluating the effectiveness of a direct-to-consumer marketing campaign for social anxiety disorder. Stigma and Health, 10.1037/sah0000163 Advance online publication. [DOI] [Google Scholar]
  50. R Core Team (2018). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. [Google Scholar]
  51. Revelle W, & Condon D (2018). Reliability from α to ω: A Tutorial, 10.17605/OSF.IO/2Y3W9. [DOI] [PubMed] [Google Scholar]
  52. Robinson P, Turk D, Sagar J, & Celia M (2019). Meausring attitudes towards mental health using social media: Investigating stigma and trivialisation. Social Psychiatry and Psychiatric Epidemiology, 54, 51–58. 10.1007/s00127-018-1571-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Rosseel Y (2012). Lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48, 1–36. 10.18637/jss.v048.i02. [DOI] [Google Scholar]
  54. Schabert J, Browne JL, Mosely K, & Speight J (2013). Social stigma in diabetes. The Patient - Patient-Centered Outcomes Research, 6, 1–10. 10.1007/s40271-012-0001-0. [DOI] [PubMed] [Google Scholar]
  55. Schofield CA, Moore CD, Hall A, & Coles ME (2016). Understanding perceptions of anxiety disorders and their treatment. Journal of Nervous and Mental Disease, 204, 116–122. 10.1097/NMD.0000000000000433. [DOI] [PubMed] [Google Scholar]
  56. Schomerus G, Schwahn C, Holzinger A, Corrigan PW, Grabe HJ, Carta MG, & Angermeyer MC (2012). Evolution of public attitudes about mental illness: a systematic review and meta-analysis. Acta Psychiatrica Scandinavica, 125, 440–452. 10.1111/j.1600-0447.2012.01826.x. [DOI] [PubMed] [Google Scholar]
  57. Steinberg DS, & Wetterneck CT (2017). OCD taboo thoughts and stigmatizing attitudes in clinicians. Community Mental Health Journal 53, 275–280. 10.1007/s10597-016-0055x. [DOI] [PubMed] [Google Scholar]
  58. Stier A, & Hinshaw DSP (2007). Explicit and implicit stigma against indivls with mental illness. Australian Psychologist, 42, 106–117. 10.1080/00050060701280599. [DOI] [Google Scholar]
  59. Thornicroft G (2008). Stigma and discrimination limit access to mental health care. Epidemiology and Psychiatric Sciences, 17, 14–19. 10.1017/S1121189X00002621. [DOI] [PubMed] [Google Scholar]
  60. Wahl OF (2012). Stigma as a barrier to recovery from mental illness. Trends in cognitive Sciences, 16, 9–10. 10.1016/j.tics.2011.11.002. [DOI] [PubMed] [Google Scholar]
  61. Wang PS, Lane M, Olfson M, Pincus HA, Wells KB, & Kessler RC (2005). Twelve-month use of mental health services in the United States: Results from the national comorbidity survey replication. Archives of General Psychiatry, 62, 629–640. 10.1001/archpsyc.62.6.629. [DOI] [PubMed] [Google Scholar]
  62. Wang PS, Aguilar-Gaxiola S, Alonso J, Angermeyer MC, Borges G, Bromet EJ, … Wells JE (2007). Use of mental health services for anxiety, mood, and substance disorders in 17 countries in the WHO world mental health surveys. The Lancet, 370, 841–850. 10.1016/S0140-6736(07)61414-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. Wei Y, McGrath PJ, Hayden J, & Kutcher S (2015). Mental health literacy measures evaluating knowledge, attitudes and help-seeking: A scoping review. BMC Psychiatry, 15, 291 10.1186/s12888-015-0681-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Wei Y, McGrath PJ, Hayden J, & Kutcher S (2017). Measurement properties of mental health literacy tools measuring help-seeking: A systematic review. Journal of Mental Health, 26, 543–555. 10.1080/09638237.2016.1276532. [DOI] [PubMed] [Google Scholar]
  65. Wood L, Birtel M, Alsawy S, Pyle M, & Morrison A (2014). Public perceptions of stigma towards people with schizophrenia, depression, and anxiety. Psychiatry Research, 220, 604–608. 10.1016/j.psychres.2014.07.012. [DOI] [PubMed] [Google Scholar]
  66. Yap MBH, Reavley N, Mackinnon AJ, & Jorm AF (2013). Psychiatric labels and other influences on young people’s stigmatizing attitudes: Findings from an Australian national survey. Journal of Affective Disorders, 148, 299–309. 10.1016/j.jad.2012.12.015. [DOI] [PubMed] [Google Scholar]

RESOURCES