Stigma Scale for Receiving Psychological Help (SSRPH): An Examination Among Adolescent Girls

Abstract

Stigma is one of the greatest obstacles to mental health care and is associated with more severe psychiatric symptoms, impaired daily functioning, and limited engagement in mental health treatment. The Stigma Scale for Receiving Professional Psychological Help (SSRPH) is a measure of public stigma, which has been used in young adults and has limited psychometric data in adolescents. This paper reports the reliability and validity of the SSRPH in adolescent girls (n=156, age 13–17). Discriminant validity was supported, but concurrent validity was not. The confirmatory factor analysis revealed excellent model fit and serves as beginning evidence for construct validity. Cronbach’s alpha for the SSRPH was .65. The SSRPH was stable over 8 weeks. Findings suggest that the SSRPH can serve as a foundation for further instrument development. Future studies may explore enhancing the reliability and validity of the SSRPH and employ advanced analytic techniques to examine the overall global construct of stigma, the latent constructs of public and private stigma, and associations of individual items to these constructs.

Stigma associated with mental illness is a serious and modifiable risk factor for poor long-term prognosis and increased morbidity and mortality (Yue-Tong & Xiao-Gang, 2012). In fact, the U.S. Surgeon General (1999) has described stigma as one of the greatest obstacles to accessing mental health treatment and hinders a person’s recovery from mental illness. Stigma is associated with more severe psychiatric symptoms, greater impaired functioning, and limited engagement in mental health treatment (Livingston & Boyd, 2010; Yanos et al., 2012).

The Effects of Mental Illness and Stigma

Mental illness is prevalent, afflicting 20% of individuals in the United States, and usually begins in adolescence and young adulthood (Kessler, McGonagle, Swartz, Blazer, & Nelson, 1993; Merikangas et al., 2010). The onset of psychiatric symptoms occurs at the same time as adolescents undergo significant changes in social and emotional growth that is characteristic of normal growth and development (Kessler et al., 1993; Merikangas et al., 2010). Stigma prevents adolescents from disclosing psychiatric symptoms to others in their social network who could potentially offer support (Merikangas et al., 2011). Stigma also impedes an adolescent’s engagement in mental health treatment (Merikangas et al., 2011). Regrettably, most adolescents do not receive mental health treatment at the onset of symptoms, and individuals may not receive treatment for 10 years after the onset of symptoms. (Kessler, Merikangas, & Wang, 2007). Without mental health treatment, adolescents will continue to suffer significant functional impairment and a myriad of adverse social, economic, health, and academic consequences throughout their most formative years of development (Kessler et al., 1993). Mental illness is devastating for millions of adolescents around the globe who struggle with psychiatric symptoms each year (WHO, 2014). Globally, depression is the leading cause of illness and disability among adolescents, and suicide is the third leading cause of death in this age group (WHO, 2014). Early detection of illness and new methods to promote entry into treatment are urgently needed for adolescents (Merikangas et al., 2010).

Major depression, one of the most prevalent mental illnesses, is stigmatized among adolescents (Chandra & Minkovitz, 2006; Rose, Joe, & Lindsey, 2011). Major depression is of particular concern for adolescent girls because it has an earlier age of onset and takes a more chronic course in adolescent girls than boys (Essau, Lewinsohn, Seeley, & Sasagawa, 2010). To improve early detection and treatment of major depression, adolescent girls must overcome stigma, feel comfortable in disclosing psychiatric symptoms to adults (i.e. teachers, athletic coaches, and school counselors) who they identify as “trusted adults,” and be skilled in garnering the support of peers (Jensen et al., 2011; Pinto-Foltz, Hines-Martin, & Logsdon, 2010).

National Initiatives and Need for Psychometric Data on Stigma Instruments

In June and September of 2013, The White House held two conferences on mental health. These two unprecedented conferences brought mental health onto the national stage, highlighting the necessity to address stigma and develop innovative and evidence-based strategies to improve the mental health of Americans. Over the last decade, there has been an outgrowth of anti-stigma interventions that use approaches like protest, education, and contact with individuals who have a mental illness (Corrigan, Morris, Michaels, Rafacz, & Rüsch, 2012). Roughly 25% of these studies focus on adolescents, and there are nearly 40 interventions available for this age group (Schachter et al., 2008). Despite this large number of interventions, the evidence base to support anti-stigma interventions for adolescents is not well established. Specifically, scientific progress has been hindered because studies overall have lacked experimental design, are low rigor, have have poor reporting quality of findings; in addition, studies do not consistently use validated instruments to measure stigma (Mukolo, Heflinger, & Wallston, 2010; Schachter et al., 2008). There are limited data on reliability and validity of stigma measures in adolescents (Schachter et al., 2008). To advance the science and evaluate the effectiveness of anti-stigma interventions with confidence, it is essential scientists have a strong psychometric evidence-base for instruments that measure stigma to use in adolescent studies (Brohan, Slade, Clement, & Thornicroft, 2010).

Public stigma is the perception of the endorsement of stereotypes about people with mental illness among society (Corrigan, 2004). Self-stigma is the feelings an individual has about himself or herself related to mental illness (Corrigan, 2004). Self-stigma is thought to be an internalization of public stigma and has been linked with low self-esteem and self-efficacy. Corrigan and colleagues have developed an excellent resource for anti-stigma interventions that includes a tool-kit of stigma measures, which can be found at http://www.scattergoodfoundation.org/sites/default/files/Evaluation%20Toolkit__Corrigan.pdf. The Attribution Questionnaire (AQ-8-C) is described as a measure of public stigma for adolescents, and this instrument is also referred to as the Revised Attribute Question or r-AQ in Watson et al. (2004). The r-AQ has been used frequently in large adolescent populations (Corrigan et al., 2005; Corrigan, 2007). Among adolescent studies, psychometric data on the r-AQ for adolescents has been limited to reporting of reliability only (Watson et al., 2004), and individual survey items have been used in the r-AQ represent a single attribute to examine a hypothesis using path analyses, but not the instrument as a whole (Corrigan et al., 2005; Corrigan et al., 2007). Psychometric evaluations of stigma scales that describe validity are beginning to emerge in the adolescent literature. For example, the r-AQ has recently been examined in adolescents by Pinto et al. (2012); items were eliminated and a modified five-item scale was found to be reliable and valid.

Corrigan and colleagues’ toolkit contains the Attribution Questionnaire AQ8-C (conceptualized as a measure of public stigma) and the Self-Stigma of Mental Illness scale (conceptualized as a measure of self-stigma). The Self-Stigma of Mental Illness Scale, which can also be found in Corrigan’s toolkit, has been noted in the literature and used to measure self-stigma. While some measures are more population than others, a recent systematic review identifies no “gold standard” measure of self-stigma; the Child Attitude Towards Illness Scale and Internalized Stigma of Mental Illness scales possess the promising evidence for measurement of self-stigma (Stevelink, Wu, Voorend, & van Brakel, 2012).

The Stigma Scale for Receiving Professional Psychological Help (SRRPH) (Komiya et al. 2000) is conceptualized to measure public stigma and has been consistently used in studies among adolescents and young adults (Bathje & Pryor, 2011; Vogel, Wester, Wei, & Boysen, 2005). Comprehensive assessment of the instrument’s psychometric properties in adolescent samples remains unavailable. Research with adolescent cohorts has been limited by cross-sectional design and reports of the SSRPH’s internal consistency only without reports of validity. Emphasis on anti-stigma intervention research highlights the need for rigorous assessments of reliability and validity of stigma instruments over-time in adolescent samples (Brohan et al., 2010; Link, Yang, Phelan, & Collins, 2004).

Description of the SSRPH

The Stigma Scale for Receiving Professional Psychological Help (SSRPH) was originally developed by Komiya, et al. (2000) in a cohort of college students and is conceptualized to measure public stigma. Recent studies have proposed that the SSRPH is may not measure public stigma, but instead the closely related construct of help-seeking (Tucker, Vogel, Wage, & Maier). Conceptual clarity of the measure is needed.

The SSRPH has demonstrated reliability in studies among young adults (Bathje & Pryor, 2011; Vogel, Wester, Wei, & Boysen, 2005). According to Komiya et al., (2000), instrument development began with the confirmation of content validity by psychological experts. The instrument was then tested in a sample (n=311) of college students (M_age = 18.4 years; SD = 1.32) who were enrolled at a large Midwestern university and predominantly Caucasian (87%) females (60%). An exploratory factor analysis, using maximum likelihood method, established a single factor solution. Factor loadings were .40–.81, accounted for 100% of the variance, and are further described by Komiya et al. (2000). Among young adults, the SSRPH has shown to be internally consistent, and initial support for validity was verified with a correlation between the SSRPH and the Attitude Toward Seeking Professional Psychological Help Short (ATSPPH) (r=−.40, p<.0001) (Komiya, et al., 2000). In another study involving young adults, a mediating pathway between the SSRPH and ATSPPH have been shown to predict young adults’ willingness to seek psychological treatment (Vogel, Wade, & Hackler, 2007).

Although the SSRPH has been used in young adult samples, there are only few studies that use the SSRPH without modification in adolescent cohorts (Logsdon, Usui, Pinto-Foltz, & Leffler Rakestraw, 2009). Studies among adolescents rarely use multiple measures of stigma concurrently and without modification, and modified versions of instruments cannot be assumed reliable and valid without extensive psychometric testing in the target population (Waltz, Strickland, & Lenz, 2005). With this in mind, the single factor structure proposed by Komiya et al. (2000) has not been verified through confirmatory factor analysis in adolescents. Furthermore, the psychometric properties of the SSRPH and stability of the SSRPH over time and how it compares with other measures of stigma have not been examined among adolescents. Thus, the purpose of this study is to explore the reliability and validity of the SSRPH in a sample of adolescent girls.

Methods

Design

Data were drawn from a randomized trial examining the effects of psychosocial anti-stigma intervention. Findings of the parent study are reported elsewhere (Pinto-Foltz, Logsdon, & Myers, 2011). In the parent project, adolescent girls (n=156) participated in an 8 week longitudinal intervention trial and completed paper-and-pencil measures of their attitude toward seeking professional psychological help, public stigma, self-stigma, and demographics at four time points. The parent study was a school-based intervention in which adolescent participants were assigned to complete measures only (control group) or receive a psychosocial anti-stigma intervention that used a story-telling approach (called “In Our Own Voice”) and was administered one-time by lay persons affiliated with the National Alliance on Mental Illness.

Setting and Sample

A community sample of 156 adolescent girls (13–15 years of age) was derived from two public high schools located in an urban area of Southern United States. Sampling was by convenience. In this community sample, the participant’s mental health status or history of mental health treatment was not reported. For adolescents to be included in the study they must have attended one of selected high schools and enrolled in the 9^th or 10^th grade.

Procedures

After receiving university and school system IRB approval, adolescent girls were approached in their health and physical education courses to determine their interest in study participation. During this time, the investigator provided a detailed description of the study and distributed consent forms to all interested adolescent girls. Adolescent girls were instructed to return the consent forms within a week to a designated school counselor. Signed adolescent assent and parental consent were required for study participation. Data were collected from adolescent girls at baseline, 2 weeks, 4 weeks, and 8 weeks post baseline. Participants received a $5 retail store gift card upon completion of the questionnaire at each data collection point.

Instruments

Stigma Scale for Receiving Psychological Help (SSRPH)

The SSRPH is conceptualized as a measure of public stigma Komiya et al. (2000). For this 5-item scale, respondents rate the degree in which they agree or disagree with each item using a likert-type format from 0 (strongly disagree) to 3 (strongly agree). Scores range from 0 to 15 with a higher score indicating a higher degree of public stigma. Cronbach’s alpha among adolescent girls in a previous study was .80 (Logsdon et al., 2009)

Attitude Toward Seeking Professional Psychological Help Scale-Short (ATSPPHS)

The ATSPPHS (Fischer & Farina, 1995) is a 10-item, 4-point unidimensional version of Fischer and Turner’s (1970) 29-item scale to measure attitudes toward seeking psychological help. For this 10-item scale respondents rate in a likert-type format the degree in which they agree or disagree with the item; alternatives are “agree,” “partly agree,” “partly disagree,” and “disagree.” Scores range from 0 to 30 with a higher score indicating a more positive attitude toward help seeking (Fischer & Farina, 1995). Cronbach’s alpha coefficient was .84, and the test-retest correlation of the revised scale was .80 (Fischer & Farina, 1995). In recent studies with young adults, Cronbach’s alpha coefficient ranged from .77 to .83 (Elhai, Schweinle, & Anderson, 2008; Shea & Yeh, 2008; Wallace & Constantine, 2005). Cronbach’s alpha for this study was .65.

Modified Revised Attribution Questionnaire (r-AQ)

The modified r-AQ is a shortened version of the Watson, et al.’s, (2004) 9-item Revised Attrition Questionnaire. Watson’s 9-item scale was constructed using the highest factor loading for each construct of the AQ (conceptualized to measure public stigma) and revising for middle school adolescents (A. Watson, personal communication, January 28, 2010). The modified r-AQ Pinto et al. (2012) is a shortened form of the revised attribution questionnaire used in Watson et al., (2004) and the psychometric properties have been supported through exploratory and confirmatory factor analyses previously in a cohort of adolescents (Pinto et al., 2012).

The modified r-AQ consists of 5 items and is proposed to measure the emotional reaction to individuals with mental illness. For this 5-item scale, respondents rate on 1 to 7 likert-type scale the extent in which they agree or disagree with each item in context of “a new student in your class that just came from another school …you have heard the student has a mental illness.” A summative score is generated, and scores range from 7 to 35, with a higher score indicating a higher level of self-stigma. The Cronbach’s alpha for this study was .70. The revised r-AQ has demonstrated construct validity in adolescents in other studies (Pinto et al., 2012).

Demographic data

A demographic data collection form was used to capture the participant’s age, race/ethnicity, grade, gender, living arrangement, and socioeconomic level.

Analytic Approach to Assessments of Validity and Reliability

Unless otherwise specified, all analyses were performed using participants’ baseline data, prior to exposure to the intervention described in the parent study (Pinto-Foltz et al., 2011). All analyses were performed using IBM-SPSS and Analysis of Moment Structures (AMOS) (Version 20, Chicago, IL) analytic software.

CFA to Test the Validity of Factor Structure

Determination of Model Fit

Use of a CFA is supported when the scale development phase is complete and hypothesized factors structures of instruments can be specified a priori (Hurley et al., 1997). Rarely do well constructed scales exhibit poor model fit in a CFA, and if model fit is found to be poor, it is then recommended that an exploratory factor analysis be performed (Hurley et al., 1997). Because the SSRPH has been previously specified by Komiya et al. (2000) as a single factor structure in an exploratory factor analysis, a first-order confirmatory factor analysis (CFA) using AMOS was performed. The following goodness-of-fit indices were used to assess the model fit: χ², Tucker Lewis Index (TLI: >.90 acceptable, >.95 excellent), the Comparative Fit Index (CFI: >.90 acceptable, >.95 excellent), and Root Mean Square Error of Approximation (RMSEA: <.08 acceptable, <.05 excellent) (Bentler, 1990; Bentler & Bonett, 1980). Based on the interpretation of modification indices, the addition of paths between error terms were considered to enhance the goodness-of-fit of the model to these data (Kline, 2004).

Concurrent and discriminant validity

To assess concurrent and discriminant validity, bivariate correlations were calculated between the SSRPH and Modified r-AQ and ATSPPHS.

Concurrent validity

To assess concurrent validity, the relationship between the Modified r-AQ, a measure of self-stigma, and the SSRPH was assessed by a correlation coefficient. To support concurrent validity, the correlation between the SSRPH and Modified r-AQ was expected to be positive and ≥ .45 (DeVon et al., 2007).

Discriminant validity

Similar to previous studies among young adults, we examined discriminant validity by calculating a correlation coefficient for the SSRPH and ATSPPHS. The ATSPPHS is a 10-item measure of attitude toward seeking professional psychological help. To support discriminant validity, the correlation between the SSRPH and ATSPPHS was expected to be negative and ≤ .45 (DeVon et al., 2007).

Internal consistency reliability

The internal reliability consistency was assessed using Cronbach’s alpha coefficient. The conventional standard for acceptable reliability is a Cronbach’s alpha of ≥ .70 for scales with at least a moderate number of items (Nunnally & Bernstein, 1994).

Test-retest reliability

The instrument’s stability over time or test-retest reliability was assessed among the subsample of 55 adolescent girls who were not exposed to the intervention in the parent study (Pinto-Foltz, et al., 2011) and were assigned to the control group. First a repeated measures Analysis of Variance (ANOVA) was employed to compare the mean SSRPH scores of the sample over the four data collection points (baseline, 2 weeks, 6 weeks and 8 weeks post baseline) in the 8-week study. Then Intraclass Correlation Coefficient (ICC) was computed to assess the stability of the instrument over 8 weeks. An ICC of .75 or greater is considered excellent reliability (Fleiss, 1986).

Results

Participants

Results of t-test and chi-square analyses support collapsing the data between the two schools sites; there were no significant differences at baseline on age, race/ethnicity, socioeconomic level, or scores on baseline measures. On average, participants (n=156) were 15 years of age (56%), in the 10^th grade (67%), Caucasian (69%), living in a two-parent home (64%), and of at least moderate socioeconomic status (66%). Participant characteristics are described in Table 1.

Table 1.

Demographic Characteristics of the Sample (n=156)

Demographic variable	n	%
Age in years (Mean 15, SD=.672)
14	35	22
15	86	56
16	35	22
Grade in school
9th	52	33
10th	104	67
Race/Ethnicity
Caucasian	108	69
African American	42	27
Asian	1	<1
Other	5	3
Living arrangements
Two-parent (including step-parents)	100	64
Mother only	50	33
Father only	2	1
Grandparent	2	1
Other family member	2	1
Socioeconomic level (receives free/reduced school lunch)
No (moderate to high)	103	66
Yes (low)	53	34

Assessments of Validity and Reliability

Confirmatory Factor Analysis

The single factor structure of the SSRPH provides the goodness-of-fit indices for the five-item single factor solution identified by (Komiya et al., 2000). The CFA revealed a structural model with excellent model fit (χ² =6.8, df =5, p =.24, TLI=.926, CFI=.975, RMSEA=.048), which is shown in Figure 1. The model fit could not be improved by deleting any items or correlating any items. Fourteen percent of variance in public stigma was explained by the SSRPH.

Figure 1.

Confirmatory Factor Analysis of SSRPH

Concurrent Validity

Concurrent validity was not supported in this sample with small correlation coefficient, r=.21, (p=.01) of the SSRPH and Modified r-AQ. Although there was a significant positive correlation between the SSRPH and Modified r-AQ, the magnitude of the correlation was insufficient to support concurrent validity.

Discriminant Validity

Discriminant validity was supported in this sample with a correlation coefficient of r= −.33 (p<.000) in the expected direction and sufficient magnitude between ATSPPHS and SSRPH.

Internal consistency reliability

Cronbach’s alpha of the SSRPH was .65, falling just short of the conventional standard of .70 needed for reliability. Cronbach’s alpha of the SSRPH could not be improved by deleting any items. Inter-item and item-total scale correlations were positive and are found in Table 2.

Table 2.

Inter-Item and Item-Total Scale Correlation for the Stigma Scale for Receiving Psychological Help (SSRPH)

Item	1	2	3	4	5	Item-Total
1	1.00
2	.33**	1.00
3	.32**	.24**	1.00
4	.26**	.20*	.36**	1.00
5	.15	.24**	.30**	.28**	1.00
Item-Total	.61**	.63**	.71**	.64**	.63**	1.00

^*p≤.05.

^**p≤.01.

Test-Retest reliability

In the subsample of adolescent girls (n=55) assigned to the control group, mean scores were calculated for each time point: M=5.54(3.03) at baseline, M=5.72(3.10) at 2 weeks, M=5.74(3.32) at 4 weeks, and M=5.51(3.05) at 8 weeks post baseline. Repeated Measures ANOVA revealed no significant differences in the SSRPH scores between the four time points F(3,162)=.186, p=.91. The ICC of the mean SSRPH scores over the 4 data collection points equaled .89 (95% CI [.83, .93, p=.00]. These results indicate stability of the SSRPH instrument over 4 time points, over the 8 week study.

Discussion

Study findings partially support the validity and reliability of the SSRPH in adolescent girls. The original factor structure of the 5-item SSRPH as proposed by Komiya et al. (2000) was verified in our sample through a CFA, which serves as beginning evidence to support construct validity. While the CFA yielded a model with excellent fit to this data, only 14% of public stigma was explained by the five-item SSRPH scale. Since a large portion of variance remains unexplained, it is possible that more items, as identified by Link and colleagues (2004), could be included that measure behavior, labeling, stereotyping, cognitive separating, emotional reactions, status loss/discrimination (expectations and experiences), structural discrimination, and behavioral responses to stigma.

Similar to the findings of Komiya et al. (2000), the SSRPH discriminated well when compared to the ATSPPHS. However, the SSRPH, originally conceptualized to measure public stigma. The modified r-AQ, which is based on the r-AQ (Watson, et al., 2004) (also called the AQ-8-C), which was conceptualized to measure public stigma, did not demonstrate concurrent validity with the SSRPH. The modified r-AQ was found by Pinto, et al (2012) to be a validated measure of stigma among adolescents in a previous study and was hypothesized to capture the emotional response to persons with mental illness.

There are several plausible explanations for the lack of concurrent validity between measures of SSRPH and the modified r-AQ. To date, most conceptual frameworks of stigma have been developed for adults and used in child and adolescent populations (Corrigan et al., 2007; Mukolo et al., 2010); there has been little attention to the conceptualization of stigma among adolescents. Transfer of adult frameworks to adolescent populations without sufficient evidence is problematic for two reasons. First, adolescence is a unique time in development and of particular concern is the effect of identity negotiation, characteristic of and unique to adolescent development (Stevelink et al., 2012). Adolescents are keenly focused on identity negotiation and do not wish to be different than their peer in-group; differences from peers can impact their identity, self-esteem, and self-image (Brown & Lohr, 1987). As such, when adolescents are asked about stigma and mental illness related to oneself, adolescents may choose responses that confirm their desired self-concept and consistent with peer in-group norms opposed to their true feelings (Swann, 1987). On the other hand, when adolescents are asked about a peer with mental illness, this may be less threatening to their self-identity, and in light of this, adolescents responses may be more reflective of true feelings about oneself (Swann, 1987). Adolescents are resistant to labeling themselves as having a mental illness (Moses, 2009). Further investigation related to how development impacts reports of public and self-stigma in adolescents is needed because of the unique identify negotiation. In adolescents, it is possible that public and self-stigma may not be as clearly differentiated as suggested in studies among adults.

Second, during adolescence, teens continue to undergo significant neurodevelopment, which is characterized by changes in neuroanatomical and neuronal connections (Giedd, 2008; Hwang, Hallquist, & Luna, 2013). For stigma to be perceived, adolescents must cognitively and affectively appraise and process observations, feelings, and behaviors of others and relate these observations, feelings, and behaviors to self. The adolescent brain is different than that of an adult (Giedd, 2008; Hwang et al., 2013). Ongoing neurodevelopment and the unique developmental stage of adolescence begs the question if teens may be experiencing stigma differently than adults. To date, there have not been rigorous assessments that illustrate similarities and differences in adolescent and adult differences in conceptualizations of stigma. Mukolo et al. (2010) developed one of the few frameworks of stigma associated mental illness for children and adolescence. Their framework describes factors related to stigma pertaining to self, general public, and institution (Mukolo et al., 2010). We are not aware of any framework that clearly explicates how children and adolescents understand and cognitively process stigma related to mental illness within the context of neurodevelopmental changes, unique to adolescents.

The modified r-AQ was used in this study and is a shortened form of Watson’s r-AQ (conceptualized to measure public stigma). The SSRPH was also conceptualized to measure public stigma (Komiya et al., 2000). Pinto et al. (2012) conducted a psychometric evaluation on the r-AQ and found a shortened form to be reliable and have construct validity. Findings of our current study did not support concurrent validity between the two measures of public stigma, the SSRPH and modified r-AQ. Although the two measures were related, the magnitude of the correlation between the measures was insufficient to support concurrent validity. Though the SSRPH was originally conceptualized as a measure of public stigma, recent studies among adults have shown the SSRPH has been hypothesized as a measure of help seeking stigma rather than mental illness stigma, in adults a mediating pathway has been shown between SSRPH and the ATSPPH through self-stigma in young adults (Vogel et al., 2007). The current study did not statistically model the relationships among variables. Future studies that use a more sophisticated statistical analytics to model relationships using path analysis or structural equation modeling are recommended to advance the science in this area.

Our study findings are different than previous work with adults, and indicate the need for further research to gain conceptual clarity of stigma among adolescents. These differences in findings illuminates a potential opportunity to modify child and adolescent frameworks of stigma that consider the perspective of teens as well as the unique aspects of their development. The omission of members of the target population in the development and conceptualization of measures are rare and have been noted in the literature and prompts the question, do the current measures accurately capture the real-world stigma experience of adolescents?

The modified r-AQ has been conceptualized to measure the emotional response to individuals with mental illness (Pinto et al., 2012). Both the SSRPH and the modified r-AQ are five item scales. Another potential explanation for the lack of concurrent validity is the two measures contain an insufficient numbers of items to comprehensively capture the construct its intended measure. Said another way, the stigma instruments in their current form, do not adequately share common attributes of both public stigma and self-stigma, indicating the need for further instrument development. Since there has been little investigation of concurrent validity and comparisons of stigma measures in studies among adolescents, we are unable to compare our results with other studies.

The SSRPH was only slightly lower (.65) than the conventional standard. The SSRPH contains five-items and reliability may be enhanced with additional items. Other studies involving adolescents and young adults (Logsdon et al., 2009; Rose et al., 2011; D. L. Vogel et al., 2005) had higher internal consistency than the current study among adolescent girls. To our knowledge, this is the first assessment of instrument stability of the SSRPH, using test-retest over 8 weeks. The SSRPH was stable and had excellent test-retest reliability. This finding has important implications for future studies, especially longitudinal observational studies and intervention trials with adolescents in which measurement over time is essential to evaluate the effects of interventions with confidence.

Although the SSRPH may be refined and improved, these findings contribute to the evidence-base for establishing the reliable and validity of stigma instruments among adolescent girls. The strengths of the SSRPH illuminated in this study are the discriminant validity and strong reliability over time. Despite the good model fit shown in the CFA, more variance in public stigma needs to be explained to support construct validity. In light of these positive findings, the existing psychometric properties of the SSRPH can serve as a foundation for further instrument development among adolescents.

Because conceptual clarity of stigma is needed in adolescents, employing sophisticated analytical techniques like hierarchical confirmatory analysis to examine the global construct of stigma, latent constructs of public and private stigma, and the associations of individual items of multiple stigma instruments is a logical first step to improve the conceptual clarity of stigma in adolescents. Ideally, analytic techniques would be employed in conjunction with adolescent qualitative research to inform the revision and addition of items to stigma measures.

Despite an adequate number of participants for our analyses, we recognize that generalizability of study findings are limited by use of a non-probability sample, and the demographic characteristics of the sample. This paper reports data from a larger study that involved a story-telling intervention about mental illness. We acknowledge that participants in this study many have self-selected into the study based on their interest in learning more about mental health. Adolescent girls in this study were predominantly Caucasian with moderate socioeconomic level, limiting the generalizability of findings to other more diverse populations. Specifically, concurrent, convergent, and predictive validity should be explored in larger samples of adolescents with different characteristics, such as clinical samples of adolescent boys and girls and minority and under-represented groups.

This study provides a foundation to build the science of measurement of stigma among adolescent girls. Use of reliable and valid measures of stigma will provide scientists with confidence in measuring the effects of evidence based anti-stigma interventions among adolescents. It will also assist clinicians, particularly in primary care and emergency room settings which often serve as frontline providers, in the identification of patients who are struggling with stigma and are at risk for poor prognosis and recovery from mental illness.