The Development and Psychometric Properties of a New Measure of Perceived Stigma Toward Substance Users

Abstract

A self-report measure of perceived stigma toward substance users was developed and studied. An initial measure was created based on a previously developed scale that was rated by experts for content validity and quality of items. The scale, along with other measures, was administered to 252 people in treatment for substance problems in the United States during 2006–2007. Refinement efforts resulted in an eight-item scale with good face validity, construct validity, and adequate levels of internal consistency. Most relationships with other constructs were as expected. Findings suggest that perceived stigma is distinct from other forms of stigma.

Introduction

Stigma is traditionally defined as the dehumanization of an individual based on their social identity or participation in a negative or undesirable social category (Goffman, 1963). A person who is stigmatized is considered devalued, unimportant, and flawed in important ways. Knowledge of or experiences with stigma can lead to the internalization of these beliefs for those who identify with a stigmatized group (Ritsher, Otilingam, and Grajales, 2003).

Research on stigma is vast and has focused on stigma directed toward members of stigmatized groups defined by race and ethnicity, homosexuality, religion, and mental illness, to name a few. Results of these studies indicate that experiences of stigma, whether enacted, perceived, or self-stigma, can have serious consequences for individuals. Some of the consequences of stigma for those with behavioral health problems include difficulties obtaining employment (Link, 1987; Penn and Martin, 1998; Penn, Ritchie, Francis, Combs, and Martin, 2002), housing (Page, 1983), and social relationships (Perlick et al., 2001). Previous research has also found various forms of stigma related to lower self-esteem/self-efficacy (Corrigan and Watson, 2002; Link, Struening, Neese-Todd, Asmussen, and Phelan, 2002; Wright, Gronfein, and Owens, 2000) and lower quality of life (Luoma et al., 2007; Rosenfield, 1997).

At least three conceptually distinct forms of stigma can be identified (Luoma et al., 2007). Enacted stigma refers to directly experienced discrimination based on membership in a stigmatized group, for example difficulty in obtaining employment, reduced access to housing, poor support for treatment, or interpersonal rejection. Perceived stigma refers to beliefs that members of a stigmatized group have about the prevalence of stigmatizing attitudes and actions in society (cf. Link, 1987). Self-stigma refers to negative thoughts and feelings (e.g., shame, negative self-evaluative thoughts, fear) that emerge from identification with a stigmatized group and their resulting behavioral impact—avoidance of treatment, failure to seek employment, and avoidance of intimate contact with others (Luoma et al., 2007). Several instruments exist for use with a mentally ill population measuring experiences with enacted stigma, perceived stigma, and internalized stigma (Link, Yang, Phelan, and Collins, 2004). However, the experiences of stigma in substance using populations are not as well researched. New measures of stigma specific to substance misuse are needed if research on this area is going to progress.

This paper focuses on the development of a new measure of perceived stigma toward those with substance use problems. Perceived stigma is important in that a number of studies have shown that people often report stigma as a barrier to entering treatment (Cunningham, Sobell, Sobell, Agrawal, and Toneatto, 1993; Hingson, Mangione, Meyers, and Scotch, 1982; Klingeman, 1991; Tuchfeld, 1981; Tucker, Vuchinich, and Gladsjo, 1994). While perceived stigma has been identified as one among many barriers, the strength of the relationship between perceived stigma and treatment-seeking behavior is unknown. A quantitative measure is needed if researchers want to examine the strength of the relationship between perceived stigma and treatment entry and persistence. No standardized measure of perceived stigma in a substance using population exists that we are aware of. Because prior studies have suggested that the forms of stigma targeting substance users¹ are often similar to those with mental illness (Corrigan, River, and Lundin, 2000; Crisp, Gelder, Rix, Meltzer, and Rowlands, 2000), we decided to adapt a scale that had already been created to measure perceived stigma toward serious mental illness (Link, Struening, Rahav, Phelan, and Nuttbrock, 1997).

In an effort to refine this new measure, termed the Perceived Stigma of Addiction Scale (PSAS), we administered the scale along with other self-report instruments to a sample of individuals in treatment for substance abuse. This paper describes the resulting psychometric analyses of the scale, including examination of the scales content, convergent and divergent validity, and measures of reliability. Convergent validity of the PSAS would be shown through moderate correlations with other stigma-related measures such as internalized stigma, internalized shame, stigma-related rejection, and a tendency toward self-concealment of one’s problems. Discriminant validity would be demonstrated through low or no relationship to measures less related to stigma such as depression, self-esteem, and social support.

Method

Participants

Participants were 252 adults, 145 male and 106 female (one did not report gender), in treatment for substance use related problems at an outpatient (n = 223) or inpatient (n = 29) addictions treatment program. Their average age was 30.5 (SD = 9.95, range 18–63) with 49% single, 12% married, 11% separated, 22% divorced, 3% widowed, and 3% not reporting. Participants identified as 4% Native American, 1% Asian/Pacific Islander, 4% African American, 79% Caucasian, 7% “other,” 6% not responding, and 12% of the total sample Latino.

Design and Procedure

Over a period of about one year, group therapy participants were alerted to the study by staff unaffiliated with the treatment center. Participants who indicated interest left group sessions to complete the questionnaire packet in a nearby room. They were given a $10 gift certificate to a national department store chain after completing the survey. After the first nine participants complained of fatigue, we split the packet of measures in half, with each packet containing the demographics form and PSAS along with half of the remaining measures. Approximately half the participants received one packet and half the other and this assignment was made at the time of assessment. Assignment to which packet they would complete was usually alternating, but this was not always consistent. Some participants did not fully complete all the items in a scale and when that occurred, they were excluded from analyses that related to that scale. Four other pilot self-report scales were also included in the packet, with those results reported elsewhere. The shorter packets with about half the scales summarized below took most participants less than an hour to complete. The order of the measures varied somewhat across assessments, but this variation was not systematic.

Measures

Perceived Stigma of Addiction Scale (PSAS)

An initial pool of 12 items was created by modifying the discrimination–devaluation measure perceived by Link et al. (1997) to refer to “someone who has been treated for substance use” instead of mental illness. The content validity of the preliminary scale was assessed by ratings of seven experts. Expert raters were considered to be professionals who had previously published an article in a peer-reviewed journal on the stigma of substance abuse. An electronic bibliography search identified a total of 23 such experts for whom e-mail addresses could be obtained.

The identified experts were contacted via e-mail, asked to participate, and provided an Internet link to our web-based rating task. Those who responded (n = 7) were provided with a description of stigma and perceived stigma and the initial 12-item version of the PSAS adapted from Link et al. (1997). Raters then rated each item on two dimensions. First, they rated each item for fit, whether each item reflected a component of stigma as directed toward those with substance problems using a 4-point Likert-type rating scale ranging from 1 (not at all) to 4 (very much). Second, they rated each item in terms of overall quality on a 4-point Likert-type scale ranging from 1 (poor) to 4 (excellent).

Across the 12 items, the mean rating for fit was 3.25 and for quality, 3.32. Three items had average fit ratings below a 3 (moderate fit) and were removed from the scale. One of these three items also had a mean quality rating below a 3. The remaining nine items all had average ratings above 3 with average fit ratings of 3.46 and average quality ratings of 3.44, indicating that these items fit the content of the scale and were clear and well written. Raters were also asked to suggest domains of stigma relevant to substance addiction that were not assessed by our scale. None of the raters suggested any additional domains for inclusion in the scale.

These initial efforts resulted in a nine-item scale that was used as part of a questionnaire packet. The items were rated on a 7-point Likert scale ranging from strongly disagree (1) to strongly agree (7). Six items were reverse-scored.

Demographics

The first few pages of the questionnaire contained 45 face valid questions regarding personal characteristics, substance use, social functioning, education, and employment.

Shame

The Internalized Shame Scale (ISS) is a highly reliable 24-item test that asks subjects to report how often they find themselves experiencing a variety of shame-related thoughts and feelings (Cook, 1989). For the original instrument, subjects rated each item on a 5-point scale ranging from 0 (never) to 4 (almost always). Due to a clerical error, the present study used a 7-point scale ranging from 1 (never) to 7 (always). Item scores were summed to achieve total scores ranging from 30 to 210.

Internalized stigma

The Internalized Stigma of Substance Abuse (ISSA) was adapted from the Internalized Stigma of Mental Illness Scale (Ritsher et al., 2003). It is designed to measure subjective experience of stigma related to substance abuse, with subscales measuring alienation, stereotype endorsement, perceived discrimination, social withdrawal, and stigma resistance. The scale consists of 29 items rated on a 4-point Likert scale ranging from strongly disagree (1) to strongly agree (4).

Stigma-related interpersonal rejection

The Stigma-Related Rejection Scale (SRS) is a survey of mental health consumer’s ongoing experiences of enacted interpersonal stigma that was originally developed by Wahl (1999). The term “mental health consumer” was changed to fit individuals with reported substance abuse problems, and items were scored on a 7-point Likert-type scale ranging from never (1) to always (7) with agreement indicating higher rejection. The scale includes nine statements asking about experiences such as being treated as less competent, hearing others say unfavorable things about people with substance abuse problems, and worrying that others will view one unfavorably.

Perceived social support

The Multidimensional Scale of Perceived Social Support (MSPSS) is a 12-item self-report inventory assessing the adequacy of the respondent’s perceived social support. It includes three subscales, inquiring about support from family, friends, and a significant other (Zimet, Dahlem, Ziment, and Farley, 1988). The items also combine to form an overall scale, with low scores indicating strong social support. Each item is rated on a 7-point Likert scale, ranging from very strongly agree (1) to very strongly disagree (7).

Self-esteem

The Rosenberg Self-Esteem Scale (SES) is the most commonly used measure of global self-esteem (Rosenberg, 1965). The scale consists of 10 items rated on a 4-point Likert scale from strongly disagree (0) to strongly agree (3). The use of this scale is well established in the literature (e.g., Blascovish and Tomaka, 1991).

Self-concealment

The Self-Concealment Scale (SCS) measures one’s tendency to conceal personal information that is distressing or negative (Larson and Chastain, 1990). Each question is answered on a Likert scale from strongly agree (1) to strongly disagree (5). The scale can yield a total possible score from 10 to 50; greater values indicate greater self-concealment.

Depression

The Beck Depression Inventory (Beck, Steer, and Garbin, 1988) is a widely used measure of depression consisting of 21 items measuring the severity of affective, cognitive, behavioral, and somatic symptoms of depression.

Results

Internal Consistency

All analyses described in this paragraph were conducted on the nine items of the PSAS remaining after receiving the content and quality ratings from experts in the field. Following the recommendations of Clark and Watson (1995), individual item distributions were examined. No items were highly skewed or unbalanced. Next we examined the correlation matrix for the nine items. We hoped to see most correlations being in the range of .15 −.5. All items except number 5 had mean item intercorrelations in that range. Item 5 did not correlate well with the other items, with a mean r = .08, suggesting that this item should be considered for removal. Next, we examined item-total correlations. Again, item 5 was the only item with a low item-total correlation (r = .099). Based on these analyses, item 5 was removed from the scale.

We then recalculated alpha and item-total correlations for the remaining eight items. Item-total correlations were all above .3. The alpha for this eight-item scale was .73, which is in the adequate range, especially for a short scale. The mean inter-item correlation for the eight items from the scale was r = .25.

Exploratory Factor Analysis

Responses to the remaining eight items were subject to an exploratory factor analysis using principal components analysis without rotation, as suggested by Cortina (1993). This analysis resulted in three factors with eigenvalues greater than 1.0, accounting for 63% of the variance. However, the scree plot clearly indicated a one factor solution, with factor 1 having a much higher eigenvalue than the other factors. We decided to follow the recommendation of Floyd and Widaman (1995) who argued that the eigenvalue cutoff of 1.0 often overestimates the number of factors and that the scree plot is a more useful guide. Therefore a second factor analysis was conducted forcing one factor, again with principal components analysis with no rotation. This second analysis resulted in one factor with an eigenvalue of 2.85, accounting for 36% of the variance. We examined the factor loadings for items loading below .4, indicating a weak correlation with the other items (Clark and Watson, 1995). No items loaded below .4 and thus all items were retained (see Table 1). We concluded that this eight-item scale would be our final scale to examine for construct validity.

Table 1.

Results of principal components analysis with no rotation for final eight-item scale

Item	Factor loadings
1. Most people would willingly accept someone who has been treated for   substance use as a close friend (r)	.497
2. Most people believe that someone who has been treated for substance   use is just as trustworthy as the average citizen (r)	.659
3. Most people would accept someone who has been treated for substance   use as a teacher of young children in a public school (r)	.689
4. Most people would hire someone who has been treated for substance   use to take care of their children (r)	.689
5. Most people think less of a person who has been in treatment for   substance use.	.466
6. Most employers will hire someone who has been treated for substance   use if he or she is qualified for the job (r)	.640
7. Most employers will pass over the application of someone who has   been treated for substance use in favor of another applicant.	.437
8. Most people would be willing to date someone who has been treated for   substance use (r)	.637

Note: (r) = reverse-scored item, N = 250.

Convergent and Discriminant Validity

Relationship between PSAS and demographics

Perceived stigma was not related to gender, age, education level, number of previous treatment episodes, ethnicity, employment status, or whether the person was having problems with the legal system. Those who indicated they were currently taking medication for a physical problem (n = 32) did perceive somewhat higher levels of substance use related stigma [t(232) = 3.0, p < .01].

Relationship between PSAS and other measures

As expected, the PSAS was moderately correlated with most other measures of stigma-related constructs. Perceived stigma was moderately correlated with internalized shame [r(111) = .39, p < .00001], self-concealment [r(131) = .48, p < .00001], and internalized stigma [r(129) = .48, p < .00001]. Contrary to expectations, perceived stigma was only slightly related to ongoing stigma-related rejection [r(131) = .22, p < .05]. Significant but small correlations were found between perceived stigma and experiential avoidance [r(237) = .27, p < .001] and depression [r(160) = .20, p < .05]. Perceived stigma was not significantly correlated with measures that were less related to stigma in the literature such as self-esteem [r(128) = .15] and social support [r(130) = .00].

Discussion

The overall purpose of these study was to develop a new measure of perceived stigma toward substance abuse and to examine its psychometric qualities. Results suggest that this effort was successful in creating a brief, unidimensional measure with good face validity, reliability, and construct validity.

Ratings by experts in the stigma of substance use provided independent validation of the face validity and quality of the content of scale items. Additional analyses of intercorrelations between scale items and internal consistency resulted in a final eight-item scale that was then subjected to psychometric analyses. Factor analysis indicated a one factor solution with strong item loadings.

Convergent validity was demonstrated through moderate correlations with measures of other stigma-related dimensions such as internalized stigma, internalized shame, and self-concealment. These results suggest that those who perceived higher levels of stigma tended to have higher levels of internalized shame, higher levels of internalized stigma, and more often use self-concealment as a coping method. Contrary to expectations, our measure of ongoing experiences with stigma-related rejection was only slightly related to perceived stigma. Perhaps this is because early experiences with stigma are more influential on perceptions of stigma than current experiences. Another possibility is that those who expect to be the target of stigma may work to conceal their substance use problems and may actually be somewhat successful in reducing direct experiences of rejection. This idea is supported by the fairly high relationship between perceptions of stigma and the use of secrecy as a coping method.

Divergent validity was shown by a lack of or limited correlation with other measures of other constructs such as depression, social support, and self-esteem. We find it interesting that perceptions of stigma do not correlate with self-esteem suggesting that ideas of the prevalence of stigma may be quite independent from a person’s more global self-appraisal.

Weaker results were found for internal consistency (α = .73), probably reflecting the brevity of this measure. Thus, developing a longer scale might result in higher internal consistency ratings. On the other hand, an alpha of .73 for such a short scale is fairly good. Another weakness is that this measure was adapted from a measure originally intended for the seriously mentally ill. As such, it may not adequately sample all the content domains that might be relevant to a substance abusing sample. However, our sample of experts did not identify any areas of content that were obviously not included.

Future research directions are numerous. Now that a quantitative measure of perceived stigma exists, researchers may more easily quantify the size of the relationship between perceived stigma and its role as a barrier to treatment attendance. Alternately, this measure could be used in studies of interventions intended to reduce stigma among those with addiction. One component of self-stigma is the fear of perceived stigma (Luoma et al., 2007), while a common intervention is education. It is possible that stigma education interventions could actually result in higher levels of perceived stigma, potentially increasing the sense of self-stigma and possibly impeding recovery. This measure could now be used to test such a hypothesis.

Stigma is a multifaceted construct and includes at least the conceptually distinct dimensions of perceived stigma, enacted stigma, and self-stigma. This study demonstrates the reliability and validity of a new measure of perceived stigma, but other measures are still needed. We hope that this measure will enable studies on stigma as a barrier to recovery from addiction as well as empowering others to develop scales related to other dimensions of the stigma of addiction.

Glossary

Perceived stigma

Beliefs that members of a stigmatized group have about the prevalence of stigmatizing attitudes and actions in society.

Self-stigma

Negative thoughts and feelings (e.g., shame negative self-evaluative thoughts fear) that emerge from identification with a stigmatized group and their resulting behavioral impact (e.g., avoidance of treatment, failure to seek employment, and avoidance of intimate contact with others).

Stigma

The dehumanization of an individual based on their social identity or participation in a negative or undesirable social category.

Biographies

Jason B. Luoma, Ph.D., is the director of the Portland Psychotherapy Clinic, Research, & Training Center, in Portland, Oregon, USA. His research focuses on stigma and shame, interventions for substance abuse disorders, and the dissemination and implementation of evidence-based therapies, particularly Acceptance and Commitment Therapy (ACT). He also is active in training and supervision and maintains a therapy practice as a clinical psychologist.

Alyssa K. O’Hair, M.A., is an associate director at the Center for the Application of Substance Abuse Technologies at the University of Nevada, Reno (UNR). In this position, she works with state agencies, tribal agencies, and pacific jurisdictions to translate research into practice and implement research-based initiatives related to substance abuse prevention. Previously, she worked as an evaluation manager at UNR’s Center for Program Evaluation where she provided external evaluation and consulting services to substance abuse and education-related programs and initiatives.

Barbara S. Kohlenberg, Ph.D., is an Associate Professor in the Department of Psychiatry and Behavioral Sciences at the University of Nevada School of Medicine. Her research has focused on acceptance-based treatments for addictive disorders, shame, and stigma. She teaches in the medical school and maintains a psychotherapy practice in addition to providing clinical supervision to psychiatry residents and psychology graduate students.

Steven C. Hayes, Ph.D., is Nevada Foundation Professor at the Department of Psychology at the University of Nevada. An author of 32 books and over 400 scientific articles, he has shown in his research how language and thought lead to human suffering. Dr. Hayes has been President of several scientific societies and has received several national awards, such as the Lifetime Achievement Award from the Association for Behavioral and Cognitive Therapy.

Lindsay Fletcher, M.A., is a doctoral candidate in the clinical psychology program at the University of Nevada, Reno (UNR). She currently works at the Center for Nutrition and Metabolic Disorders as a behavioral health specialist providing counseling for weight loss. She is interested in the application of acceptance- and mindfulness-based approaches, specifically in the areas of weight loss and stigma. She received her bachelor’s degree from the University of Pennsylvania.