Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Sep 1.
Published in final edited form as: Psychol Assess. 2014 Apr 21;26(3):763–773. doi: 10.1037/a0036571

Rural Parents’ Perceived Stigma of Seeking Mental Health Services for their Children: Development and Evaluation of a New Instrument

Stacey L Williams 1, Jodi Polaha 1
PMCID: PMC4169144  NIHMSID: NIHMS586530  PMID: 24749752

Abstract

The purpose of this paper was to examine the validity of score interpretations of an instrument developed to measure parents’ perceptions of stigma about seeking mental health services for their children. The validity of the score interpretations of the instrument was tested in two studies. Study 1 examined confirmatory factor analysis (CFA) employing a split half approach, and construct and criterion validity using the entire sample of parents in rural Appalachia whose children were experiencing psychosocial concerns (N=347), while Study 2 further examined CFA, construct and criterion validity, as well as predictive validity of the scores on the new scale using a general sample of parents in rural Appalachia (N=184). Results of exploratory and confirmatory factor analyses revealed support for a two factor model of parents’ perceived stigma, which represented both self and public forms of stigma associated with seeking mental health services for their children, and correlated with existing measures of stigma and other psychosocial variables. Further, the new self and public stigma scale significantly predicted parents’ willingness to seek services for children.

Keywords: stigma, mental health, services, children, parents

Presently, in spite of empirically supported treatments that could effectively address children’s psychosocial concerns, less than half of those with diagnosable mental illnesses are receiving treatment (Wang, Lane, Olfson, Pincus, Wells, & Kessler, 2005). Access to effective treatments is particularly limited in rural areas, where several hypothesized barriers have been implicated including: financial concerns including inadequate insurance coverage (U.S.D.H.H.S., 1999), problems with travel and transportation (Fox, Merwin, & Blank, 1995), and beliefs about the relevance and need for (Girio-Herrera, Owens, & Langberg, 2013) or appropriateness of mental health treatment such as the stigma associated with it (Jameson & Blank, 2007; Mukolo, Heflinger, & Wallston, 2010). Further, recent evidence suggests that in rural areas parents of children at risk for behavioral difficulties are more likely to seek help from informal (e.g., spouse, family) than formal systems (e.g., medical professionals) for problems with their children (Girio-Herrera et al., 2013). Stigma of mental illness and service seeking is often cited as a top barrier for those in rural communities (Bray, Enright, & Easling, 2004; Jameson & Blank, 2007). Perceptions of stigma about services are important because they may be more directly linked with less service seeking (Vogel, Wade, & Haake, 2006). Parents presumably are the primary decision makers about seeking mental health services for children with psychosocial concerns. However, no studies have been conducted on parents in rural communities who have children with psychosocial concerns and who perceive stigma of seeking services. The purpose of this study was to examine the validity of test score interpretations of an instrument developed to measure the stigma the parents in rural communities perceive about seeking services for children, as a first step toward examining stigma as a barrier to mental health service delivery for rural children.

Defining Stigma

Definitions of stigma span multiple decades and disciplines. The classical sociological definition considers stigma as a deviant condition (e.g., physical disability, disease), or discrediting attribute (Goffman, 1963). While an overarching social psychological definition is similar in scope, viewing stigma as a characteristic, behavior, or attribute, the social-cognitive approach within social psychology depicts the attribute as reflecting a devalued social identity in a particular context (Crocker, Major, & Steele, 1998), and has led to extensive research on the perceiver of discredited individuals and their negative reactions or stereotypes toward such individuals (Crocker, Major, & Steele, 1998; Jones et al., 1984). Link and Phelan (2001) further defined stigma in a situation of power and include labeling, stereotyping, separation, status loss, and discrimination.

In addition to the objective devalued attribute or identity, stigma can be perceived by the person holding the particular attribute. In this vein, perceived stigma has been discussed as individuals’ reactions to holding a deviant condition, including personal beliefs about the self, such as shame, embarrassment, as well as projection of such beliefs onto others (Mickelson, 2001; Williams & Mickelson, 2008). This latter conceptualization was formally distinguished by Corrigan (2004) who defined public stigma as the negative stereotypes and unfair treatment from others, and self-stigma as the ways that individuals internalize the public view resulting in negative beliefs about the self. In the present study we defined perceived stigma by its public (expected negative treatment from others) and self stigma (shame, embarrassment) components, and applied it to parents of children with psychosocial concerns in the context of mental health service seeking.

Stigma of Service Seeking as a Barrier to Rural Children’s Services

The majority of empirical studies examining perceptions of public and self stigma about seeking services have focused on adults seeking services for themselves. For example, research in rural Appalachia shows residents cite stigma as a barrier to mental health treatment more often (28%) than in a comparison sample outside the region (22%; NORC, August, 2008). In another study, Hoyt, Conger, Valde, & Weihs (1997) found that adults in rural areas perceived more stigma of mental health services (e.g., anticipated embarrassment if friends or other people in the community were to find out about their seeking services) than their urban counterparts. Importantly, greater perceived stigma was associated with less willingness to seek services ranging from talking about a personal problem to getting help for a serious emotional problem (Hoyt et al.). However, the measure of stigma consisted of two items only and did not focus on children or perceptions of parents. Vogel, Wade, and Haake (2006) have constructed a scale to measure perceptions of self stigma of seeking services which demonstrates adequate validity and predicts service seeking; however, their instrument does not directly address rural communities or parents seeking mental health services for children experiencing psychosocial concerns.

Recently, more attention has been focused on perceptions of stigma about children’s mental illness and mental health services (see Mukolo et al., 2010 for a review). In particular, the National Stigma Study – Children (NSS-C) provided the first nationally representative, large-scale survey of adults’ public knowledge and beliefs about child mental health concerns. These groundbreaking data showed beliefs about public stigma and stereotypes for children parallel those found for adults. For example, the results of one study showed adults who labeled a child as “mentally ill” (e.g., depression, ADHD) were about twice as likely to describe the child as having a potential for violent behavior (Pescosolido, Fettes, Martin, Monahan, & McLeod, 2007). A similar study showed adults felt that providing psychiatric medications to children would cause them to be considered outsiders at school, and for their parents to feel like failures (Pescosolido, Perry, Martin, McLeod, & Jensen, 2007), indicators of public and self stigma, respectively. In a follow up study that reanalyzed the NSS-C data, Mukolo and Heflinger (2011) examined people’s likelihood to distance themselves from children or the families of children with health (e.g. asthma) or mental health (e.g., ADHD) conditions based on attribution of blame, using the vignettes. Results showed that social distance was preferred when the child was to blame for the condition. However, in general, more social distance was linked with mental illness conditions such as ADHD and depression than other conditions such as asthma.

In spite of the strengths of these national data that highlight the stigmatizing views held by the public about children with behavioral problems and who seek treatment, even this research focused on children did not directly assess perceptions of stigma held by parents about seeking services for their own children who are having psychosocial concerns. Parents are the primary decision makers regarding the mental health treatments sought for children. If parents perceive stigma surrounding seeking mental health care for their children, their children may not receive the services they need. The stigma perceived by those more likely in need of treatment may be especially important to understand as it may have a direct impact on their decisions to seek services for children (see Heflinger & Hinshaw, 2010, for review of the link between various forms of stigma and children’s mental health service utilization).

Moreover, rural Appalachia may be a particularly ripe context in which to study parents’ perceptions of stigma of seeking services for children with psychosocial concerns. It is likely that stigma surrounding mental illness and treatment seeking is embedded within a cultural context (Weiss, Jadhav, Raguram, Vounatsou, & Littlewood, 2001) in that across different cultures the meanings of stigma may differ, even though in general stigma is a universal experience (Yang et al., 2007). In this way, rural Appalachians may experience public and self stigma of seeking services for their children but the specific experience may be unique due to aspects of Appalachian culture. For example, although there is variability in culture across communities within the Appalachian region, particular cultural beliefs more common to rural Appalachia may engender stigma. Anthropological studies have shown rural Appalachians may be inclined toward independence and individualism, a strong sense of family ties and kinship, and a religious world view (e.g., Beaver, 1986; Jones, 1994; Keefe, 1998). Independence in rural Appalachia can manifest in self-reliance and stoicism, making individuals reluctant to seek services. Indeed, stoicism has been linked to less service seeking in rural locations other than Appalachia perhaps because individuals are reticent to admit they have any problems or because they do not recognize their symptoms are problematic until they are quite severe (see Judd et al., 2006). And, strong attachment to kinship may increase distrust of “outsiders” and translate into distrust of services and providers (see Goins, Spencer, & Williams, 2011, for review).

Further, individuals may live in locations where “everyone in town knows them” and their life history (see MacAvoy & Lippman, 2001). In this way, rural areas tend toward less confidentiality and information traveling fast (Jameson & Blank, 2007). Anecdotes of rural residents hesitant to park their vehicles outside of a mental health facility abound. Because everyone knows each other in rural communities it may be a goal to avoid being the center of gossip (Philo, Parr, & Burns, 2003) by avoiding a label (Link & Phelan, 2001). Individuals may anticipate greater risks of negative responses or stigma from family, friends, church members, and the community for seeking mental health services. For these cultural reasons many have considered stigma a particular challenge in rural areas, but to date it has rarely been documented and never in relation to parents’ service seeking for their children with psychosocial concerns.

The Present Research

The purpose of our research was to examine the validity of a measure developed to assess parents’ perceptions of public and self stigma about seeking mental health services for their children in two samples of parents in rural Appalachia residing in northeast Tennessee and southwest Virginia. The instrument tested in the present study addresses gaps in prior stigma-related instruments. Specifically, prior work has focused on the stigma about mental health concerns among children (e.g., depression, ADHD), and on the stigma of seeking services among adults (e.g., Vogel et al., 2006), but no studies have examined the perceived stigma about seeking services for children’s psychosocial concerns. At least theoretically, it may be that perceived stigma about the services represent beliefs more proximal to the decision to seek services than stigma surrounding the psychosocial concerns. Moreover, no prior work has focused on stigma perceived by parents living in rural areas about children who need services. As described above, widespread claims that residents in rural areas experience more stigma around getting mental health services are supported by only a few studies using adult samples. Stigma might be particularly relevant for rural communities due to their characterization as having strong social support networks, more conservative values, and a greater lack of privacy (e.g., word travels fast, everybody knows everybody; Bray et al., 2004; Jameson & Blank, 2007).

STUDY 1

The main aim of Study 1 was to examine the validity of score interpretations of the stigma instrument by conducting Confirmatory Factor Analysis (CFA), and examining the relations between parents’ stigma and existing measures of stigma discussed in the above literature (Self-Stigma of Seeking Psychological Help, Social Impact Scale, and National Stigma Study Items) and other constructs (low efficacy of help seeking, self-esteem, social desirability). As well, we examined whether parents’ stigma predicted willingness to seek mental health services among parents, beyond existing measures of stigma. We hypothesized that parents’ stigma would be positively associated with other measures of stigma and low efficacy of help seeking, and negatively associated with self-esteem, and social desirability. Further, we hypothesized that parents’ stigma would predict willingness to seek services above and beyond the existing measures of stigma. To address the gap in the work on stigma of children’s psychosocial concerns, we specifically targeted parents of children with psychosocial concerns (identified by a screening measure, the Pediatric Symptom Checklist (PSC)). Beliefs held by parents with children more likely in need of treatment may more directly and practically speak to decisions about seeking services.

Method

Sample & Procedure

Study 1 was conducted with 347 parents who reported psychosocial concerns about their children. In order to recruit the sample, we screened 2,672 parents in the waiting room of eight pediatric primary care clinics in rural Appalachia (northeast Tennessee and southwest Virginia). All waiting room parents completed a two-page screening survey, consisting of a measure of child psychosocial concerns (Pediatric Symptom Checklist (PSC); Jellinek, Murphy, & Robinson, 1988; described below) and questions about demographic characteristics (parents also provided contact information). Those parents of children that met our criteria for psychosocial concerns (i.e., scores of ≥20 on the PSC, N=727) were contacted and asked to participate in the study, comprised of a mail-back paper-and-pencil survey, described below. Of the 727 (27%) who had children with psychosocial concerns and who were contacted for participation, 156 were unreachable (phones disconnected or unanswered), 113 declined to participate, 105 agreed to participate but did not mail back the survey packet and 347 returned a completed survey, yielding an overall 48% response rate. Informed consent was collected via a participation letter for the screening, while written informed consent was included in the mailed packet for the study. Parents were compensated with $20.00 for their time to complete the study.

The 347 parents who participated in the study (see Table 1) were primarily (79.8%) mothers of the targeted child (versus 7.8% fathers, 12.4% “other” such as grandparent) and Caucasian (94.2% versus 2.9% Hispanic, .6% Black, .6% Asian, and 1.6% other race), having at least a high school education (88.7% mothers, 74.1% fathers), and an annual household income of $34,559.56 (SD = $27,516.88; range = $0 – $155,000). They also reported that the child about whom they were answering was, on average, 9 years of age (M = 9.21, SD = 3.43, range =4 – 16), had a PSC score of 28.87 (SD = 7.72, range = 20 – 54), and was male (57.9% versus 42.1% female).

Table 1.

Demographics of Total Samples of Parents in Studies 1 and 2

Demographic Characteristic Study 1 (N = 347) Study 2 (N = 184)
% n % n
Child Age (M, SD) 9.24 3.43 8.68 3.62
Income (M, SD) 34,763.08 27,624.67
Child Gender
 Female 41.9 143 42.6 78
 Male 58.1 198 57.4 105
Relation to Child
 Mother 80.0 272 82.1 151
 Father 7.9 27 8.7 16
 Other 12.1 41 9.2 17
Mother Education
 Less than high school 10.6 35 6.1 11
 High school 39.8 131 51.1 92
 2 year college/technical school 34.7 114 27.2 49
 4-year college 8.8 29 11.7 21
 Post college degree 5.2 17 3.3 6
Father Education
 Less than high school 15.7 52 13.0 23
 High school 51.7 171 49.7 88
 2 year college/technical school 15.4 51 18.1 32
 4-year college 6.6 22 9.6 17
 Post college degree 3.3 11 4.5 8
Race/Ethnicity
 White 94.4 322 89.6 164
 White Hispanic 2.3 8 2.2 4
 Black .6 2 3.3 6
 Black Hispanic .6 2 1.1 2
 Asian/Pacific Islander .3 1 --- ---
 Other 1.8 6 3.8 7
Ever talked – Counselor 52.5 179 31.5 58
Know someone – mental health tx 92.6 311 67.6 123
Know someone – psychiatric hospital 73.3 247 54.6 100
Open in a new tab

An attrition sub-analysis showed some significant differences between parents who completed the study and those who were unreachable or declined (either passively or actively, as described above) to participate. Specifically, those who did not participate had children that were slightly older (9.73 years versus 9.13 years, t = 2.33, p < 05) and had slightly fewer psychosocial concerns (27.56 versus 28.87, t = 2.23, p < .05). In addition, they were less likely to have talked with professionals about their child’s behavior in the past (M = 2.72 versus M = 3.04, t = 3.18, p<.05) and more likely to report a relationship to the child of “other” (e.g., grandparent, guardian) than those who participated. No other significant differences were found.

Measures

Pediatric Symptom Checklist (PSC)

The Pediatric Symptom Checklist (PSC; Jellinek, Murphy, & Burns, 1986) is a screening instrument designed to identify psychosocial concerns among parents of children ages 4–16 presenting to primary care settings. The PSC consists of 35 statements (e.g., “Complains of aches and pains,” “Spends more time alone,” “Distracted easily”) to be rated by the parent as occurring “Never” (0), “Sometimes” (1), and “Often” (2). In validity studies, classification using the PSC was in agreement with other well-established ratings as well as clinician ratings of psychiatric impairment in a variety of settings with children representing a wide range of socioeconomic backgrounds (see Jellinek, Murphy, & Little, 1999, for an overview). For the current study, we calculated a total PSC score (α=.85) at the screening stage to determine eligibility for the full validation study. Eligibility for the study was determined using a total score of at least 20 on the PSC. Although 20 is less than the cutoff for clinical significance (24–28 is cutoff depending on age), we chose this lower cutoff in order to assess perceptions of parents with children demonstrating elevated psychosocial concerns but that may not yet be at clinical levels. We presumed that a greater range of severity of child behavior would provide us with a sample for which services were relevant while increasing the number of parents who had not previously sought services for their children.

Demographics

These questions included: 1) the respondent’s relationship to the child (mother, father, or other), 2) the child’s age, 3) the child’s gender, 4) parent education, 5) history of prior services from a therapist or counselor for child’s psychosocial concerns, 6) parent name and telephone number; 7) annual income before taxes; 8) race/ethnicity (including White, White Hispanic, Black, Black Hispanic, Asian/Pacific Islander, and Other).

Parents’ Perceived Stigma of Service Seeking (PPSSS)

This new measure was developed for this study and comprised of items intended to represent both public and self stigma of seeking mental health services for children. We compiled an initial 33 items based on existing literature on stigma around mental health concerns and service-seeking for the adult population (e.g. Crandall, 1991; Levinson & Starling, 1981; Mickelson, 2001; Vogel et al., 2006), and on limited literature suggesting how parents in rural areas might feel if they took their child to a counselor, therapist, or psychologist for problems with his or her behaviors or emotions (addressed above). We further adapted these items based on the use of an expert panel that read the items after receiving a definition of public and self stigma. Specifically, the expert panel consisted of six professionals working in rural mental health or primary care. In a two-hour meeting, these experts were asked to review the items and to revise them to match with their perceptions of how parents of children with psychosocial concerns in rural Appalachia might feel based on their interactions with them in their practices. Experts provided direct editorial feedback on the items via paper and pencil editing and informal roundtable discussion. For example, an initial item described parents as feeling “inferior” if treatment were sought for the child, and the revised item described how parents would feel “less” about themselves. Finally, we conducted cognitive interviews with a subset of parents from the community in order to learn how future survey respondents might interpret questions and whether items were worded appropriately (for more information on cognitive interviews, see Willis, Royston, & Bercini, 1991, for example). The individual cognitive interviews with four volunteer parents from rural Appalachia were conducted to further refine the items based on their experiences and interpretations of the item wording. This combined process resulted in 18 items that tapped parents’ perceptions of self-stigma (e.g., I would be embarrassed; I would feel like a bad parent) and public stigma (e.g., people would say bad things about me behind my back; my child would be labeled in school) about seeking mental health services for their children, to which parents respond with how much they would agree (1= strongly disagree, 6=strongly agree) with the 18 statements (see Table 2).

Table 2.

Study 1 Factor Loadings from EFA and CFA of PPSSS

Perceived Stigma Items EFA (N=174) CFA (N=173)
Factor 1 Factor 2 Factor 1 Factor 2
1. It would make me feel strange .52 .65
2. It would make me feel embarrassed .66 .80
3. It would make me feel like a bad parent .88 .85
4. My view of myself would be less .98 .91
5. It would make me feel that I am weak .95 .85
6. It would make me feel like there is something wrong with me .87 .83
8. Some people might treat me unfairly .81 .72
9. Some people might look down on me .93 .72
10. Some people might say bad things about me behind my back .96 .70
11. Some people would treat me with less respect .92 .73
12. Some people would avoid me .86 .72
13. My child might be labeled at school .60 .60
14. People in my church might frown on my decision .62 .77
15. My child’s teacher would treat him or her unfairly .63 .71
16. I would be worried that people in town would find out .66 .77
17. I would try to hide that I was getting counseling for my child .55 .74
18. I would try to go to a counselor in another town so no one I know would find out .56 .72
Open in a new tab

Note: The EFA was conducted on a random half (developmental sample) and the CFA was conducted on the remaining random half.

Self-Stigma of Seeking Psychological Help (SSSPH)

This measure (Vogel, Wade, & Haake, 2006) consisted of 10 items that assessed the self-stigma associated with seeking psychological help. Parents responded to these items based on how much they agreed (1= disagree, 4= agree) with statements about self-stigma (e.g., I would feel inadequate if I went to a therapist for psychological help). Items were summed with higher scores indicating greater self stigma of seeking psychological help. In prior work this scale has evidenced adequate reliability as well as construct, criterion, and predictive validity (Vogel et al., 2006). Using present study data internal consistency was strong (α = .79).

Social Impact Scale

Twelve items were drawn from the Social Impact Scale developed by Fife and Wright (2000), and were adapted to fit with child behavior problems. These items represented the internalized shame and social isolation subscales that tapped the social psychological feelings regarding stigma. Parents responded to these items based on how much they agreed (1= disagree, 4= agree) with statements about the child’s behavior problems (e.g., I feel others think I am to blame for my child’s problems with behavior or emotions; I feel I need to keep my child’s problems with behavior or emotions a secret). In prior work the full scale has evidenced adequate reliability as well as item and person validity, sensitivity, and concurrent validity (Pan, Chung, Fife, & Hsiung, 2007). Using present study data internal consistency was strong (α = .88).

National Stigma Study Items

Nine items were drawn from the National Stigma Study, specifically for children (Pescosolido, Perry, Martin, McLeod, & Jensen, 2007) and used to assess parents’ perceptions of mental health treatments for children. Parents responded to these items based on how much they agreed (1= strongly disagree, 4=strongly agree) with statements about treatments for children (e.g., Getting mental health treatment would make a child an outsider at school; Regardless of laws protecting confidentiality, most people in the community still know which children have had mental health treatment). In prior work these items evidenced adequate reliability (Pescosolido et al., 2007). Using present study data internal consistency was strong (α = .83).

Rosenberg Self-Esteem Scale

This measure consisted of 10 items that assessed global self-esteem (Rosenberg, 1965). Parents responded by indicating how much they agreed (1=strongly agree, 4=strongly disagree) with each statement. Items were reversed scored and mean scores were calculated with higher scores to indicating greater self-esteem. In prior work the scale has evidenced adequate reliability, factor structure, and convergent and discriminant validity (Sinclair et al., 2010). Using present study data internal consistency was strong (α = .88).

Efficacy of Help Seeking

This 8-item scale assessed the extent to which individuals hold negative help seeking beliefs (Eckenrode, 1983). Parents responded to items based on how much they agreed (1= strongly disagree, 4=strongly agree) with statements such as “It is better to take care of your own problems than to rely on others for help” and “Admitting hardships to others is a sign of weakness”. Mean scores were calculated with higher scores indicating low efficacy of help seeking. Although prior work has not examined the psychometric properties of this scale, this measure has shown adequate reliability and associations with one’s ability to mobilize support to cope (Eckenrode, 1983; Riley & Eckenrode, 1986). Using present study data internal consistency was strong (α = .77).

Marlowe-Crowne Social Desirability

The 13-item short version of the Marlowe-Crowne Social Desirability Scale (Reynolds, 1982) was used to assess general tendency to respond in a socially desirable way to make favorable impressions. Parents responded to statements by indicating whether or not they are true or false for them. A total score was calculated by summing the number of socially favorable items endorsed. In prior work the scale has evidenced adequate reliability, factor loadings, and convergent validity (Reynolds, 1982). Using present study data internal consistency was strong (α = .74).

Mental Health Service Seeking

Willingness to seek services for children was assessed with one item developed for this study, which read “I have taken or would take my child to see a counselor, therapist, or psychologist in a center that is designated to provide mental/behavioral health services.” Parents responded by indicating how likely they would be (1=not at all, 6 = definitely). In addition, prior use was assessed with one item that asked parents to indicate whether they have ever “talked about concerns they have about their child’s behavior or emotions with a counselor, therapist, or psychologist. Parents responded by indicating “yes” or “no”.

Results

Construct Validity of the Self and Public Subscales of the PPSSS Scale

In order to examine the validity of the perceived stigma scale, a combination of Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) was employed. Prior to conducting these analyses, a random half of the sample was taken and considered a developmental sample, (n = 174) leaving the remaining random half as the confirmatory sample (n = 173).

Taking the developmental sample first, EFA was conducted on the 18-item PPSSS. Specifically, a principle axis factor analysis without specification of number of factors was conducted with promax (oblique) rotation, which acknowledges the correlation between the factors identified. The number of factors was determined by eigenvalues greater than 1.0 (Kaiser-Guttman rule; Guttman, 1954; Kaiser, 1960) and confirmed by a scree plot, and followed by a parallel analysis (Horn, 1965; which has been found consistently accurate – e.g., Zwick & Velicer, 1986). Factor loadings under .40 were suppressed. Initial inspection of EFA factor loadings revealed one loading below .40. This one item (I would think there was something wrong with my child) was subsequently removed. When EFA was recalculated based on the 17-item scale, a two-factor solution in line with public and self-stigma was revealed. A two-factor solution was consistently shown using parallel analysis (Horn, 1965). Six items loaded onto the self-stigma factor and 11 items loaded onto the public stigma factor. In addition, oblique rotation indicated that the two factors were correlated (r = .63). See Table 2 for factor loadings.

Using the confirmatory sample next, CFA was conducted on the 17 items (6 item self-stigma, 11 item public stigma) using Structural Equation Modeling (SEM). The model tested also included 24 correlated error terms between items determined by identification of similar question stems within public and self-stigma (but not between loadings across public and self-stigma). Results of the main test of the measurement model were evaluated using the following indices of good fit: a Comparative Fit Index (CFI) of .90 or higher, an RMSEA of less than .10, and an standardized RMR of less than .10 (as suggested for samples sizes under 500, see Weston & Gore, 2006). Although ideally a non-significant chi-square also represents good fit, this index is a test of exact fit of the model to the data, where an exact fit is rare. In addition, chi-square can be sensitive to sample size. A non-significant chi-square may not be indicated (see Weston & Gore, 2006, for review). Factor loadings also were examined to confirm items loaded with their respective factors. Results revealed support for the two-factor model of parents’ perceived public and self-stigma of seeking services for children (χ2 (94) = 164.35, p = .00, CFI = .97, SRMR = .04, RMSEA = .07 (90% CI = .05 – .08)). The chi-square was statistically significant, but the other indicators represented adequate fit. In addition, the ratio of chi-square to degrees of freedom was low (1.75). And, the factor loadings revealed that all items loaded on their respective factors (loadings ranged from .60 – .90).

Construct and Criterion Validity of the Self and Public Subscales of the PPSSS Scale

In order to further test the validity of the scores reflecting public and self stigma, bivariate relations with three existing measures of stigma were examined using the entire sample of parents (N = 347). We anticipated self and public stigma would be associated positively with other measures of stigma. Prior to analysis, scale reliability analysis indicated the internal consistency for self and public stigma, respectively, was strong (α = .93, α = .94). Results of main bivariate tests revealed that, respectively, parents’ self and public stigma of seeking services for their children were significantly and positively related to the SSSPH (r = .58, p < .001; r = .53, p < .001), Social Impact Scale (r = .48, p < .001; r = .60, p < .001), and National Stigma Study items (r = .40, p < .001; r = .42, p < .001).

In addition, we conducted bivariate correlations between self and public stigma and measures self-esteem, low efficacy of help seeking, and social desirability using the entire sample of parents (N = 347). We anticipated that self and public stigma would be associated negatively with self-esteem and social desirability and positively with low efficacy of help seeking. As hypothesized, results revealed that, respectively, parents’ self and public stigma of seeking services for their children were significantly and negatively related to self-esteem (r = −.24, p < .001; r = −.26; p < .001) and social desirability (r = −.15, p < .01; r = −.22, p < .001) and significantly and positively related to low efficacy of help seeking (r = .46, p < .001; r = .50, p < .001).

Predictive Validity of the Self and Public Subscales of the PPSSS Scale

Further, we conducted two multiple regression analyses to assess the extent to which self and public stigma predicted willingness to seek mental health services over and above existing measures of stigma. In the first analysis we entered prior use in the first step, SSSPH and social impact scale in the second step, followed by the self stigma PPSSS Scale. We chose this hierarchical analysis to show the incremental validity attributable to the new stigma measure. Results revealed the self stigma PPSSS Scale contributed uniquely to parents’ willingness to seek mental health services (ΔR2 = .016, p = .013); the more parents perceived self stigma, the less willing they were to seek services (b = −.16, SE = .06, p = .013). Similarly, in the second analysis we entered prior use in the first step, national stigma study items and social impact scale in the second step, followed by the public stigma PPSSS Scale. Results revealed the public stigma PPSSS Scale contributed uniquely to parents’ willingness to seek services (ΔR2 = .023, p = .003); the more parents perceived public stigma, the less willing they were to seek services (b = −.20, SE = .07, p = .003).

Discussion

Although empirically supported treatments abound for various psychosocial concerns among children, the majority of parents do not seek mental health services for their children (e.g., Wang et al., 2005; Girio-Herrera et al., 2013). Stigma is purported to be a significant barrier to such service seeking (Jameson & Blank, 2007; Mukolo et al., 2010); however, little empirical evidence for perceived stigma of service seeking exists among parents of children in rural areas of the U.S. As the first step in understanding stigma among parents of children with psychosocial concerns, this study sought to examine the score interpretations of an instrument developed to assesses parents’ perceived public and self stigma of seeking mental health services for their children living in rural communities. In assessing perceived stigma, we focused on parents from rural Appalachia whose children were experiencing psychosocial concerns because, for these parents, stigma as a potential barrier to service seeking may be particularly relevant. Results of exploratory and confirmatory factor analysis uncovered a two-factor solution in line with public and self-stigma about service seeking, and that the measure appropriately correlated with existing measures of stigma and predicted parents’ willingness to seek services.

This study was the first to evaluate the validity of score interpretations of an instrument that assesses perceived stigma about seeking mental health services for children among parents of children with psychosocial concerns. Our study resulted in initial evidence for the validity of public and self forms of parents’ perceived stigma of seeking services for their children. These findings are aligned with the longstanding idea that mental health services are associated with mental illness, a label that carries a legacy of stigma (e.g., Link et al., 1989), as well as prior research that has depicted both public and self components of stigma related to mental illness (e.g., Corrigan, 2004). Based on the scale validity, parents of children with psychosocial concerns may feel inadequate as parents and expect unfair treatment from others toward them and their children if they sought services. Moreover, our results showed the importance of public and self stigma of service seeking for children above and beyond other indicators of stigma and prior use.

Our capacity to fully understand parents’ perceptions of stigma may have been limited by our particular sample of parents from rural Appalachia. In this vein, the ability to generalize these findings to parents in rural areas may be limited by the relatively low response rate and the focus on parents of children exhibiting psychosocial concerns only. Presumably, low response is a problem to the extent that differences exist between those who fully participated and those who dropped out. Because we had demographic information and child behavior data for all parents who participated at the screening level, we were able to make attrition comparisons (see above). Those who did not participate in the full study had children who were slightly older and had slightly fewer psychosocial concerns. In addition, they were less likely to have talked with professionals about their child’s behavior in the past and more likely to report a relationship to the child of “other” (e.g., grandparent, guardian) than those who participated in the validation study. This latter difference may imply that those who did not complete the study perceived more stigma surrounding services for children, a possibility we are unable to test given stigma was not assessed as a screening measure. However, no significant differences were found for child’s gender, parent education, or race/ethnicity based on attrition. Future research examining parents’ perceptions of stigma about service seeking should maximize efforts to recruit parents who have not yet talked with professionals about child behavior.

Another potential limitation involves the inclusion of only parents of children exhibiting psychosocial concerns. Although these parents might be the most likely to need to seek mental health services for their children, parents in general might at some future point be faced with the need for services for their child. In addition it is unclear the extent to which our sample represents parents with children living in rural Appalachia. Comparing our sample to some of the demographic factors of individuals who reside in the counties most commonly represented by parents in our study, we find that our sample is somewhat less educated when highest education is averaged across mothers and fathers (i.e. lower percentage of people with at least a bachelors degree (11.95% versus 16.66%), and reports a slightly lower annual income (M = $34,763.08 versus median = $36,685.40) (see STATSAmerica, 2013). Future research should examine the self and public stigma of service seeking for children in parents more broadly, regardless of reported child behaviors and psychosocial concerns.

STUDY 2

The main aim of Study 2 was to further examine the validity of score interpretations of the instrument in a general sample of parents in rural Appalachia by replicating the CFA of Study 1, and by examining relations between self and public stigma with existing measures (Social Impact Scale and low efficacy of help seeking). As well, we examined whether self and public stigma predicted willingness to seek mental health services among parents, beyond child psychosocial concerns, prior use, or parents’ stigma related to the child’s problems. We hypothesized that self and public stigma would be positively associated with an existing measure of stigma and low efficacy of help seeking, and that the new stigma measures would predict service seeking. In this second study we directly recruited parents for the study in the waiting room of two pediatric primary care clinics as opposed to using a screening and follow up strategy in order to increase parent response rate. This change in strategy allowed us to recruit a broader range of parents, rather than to screen for those with children meeting criteria for potential psychosocial concerns using the Pediatric Symptom Checklist (PSC). As a result we presumably were better positioned to recruit parents who had not yet talked with professionals about their child’s behavior.

Method

Sample and Procedure

The Study 2 paper and pencil survey was conducted with 184 parents recruited in the waiting room of two pediatric primary care clinics in rural Appalachia (northeast Tennessee, southwest Virginia). Informed consent was collected via a participation letter. We approached a total of 238 eligible parents in the waiting rooms. We determined eligibility if parents had not participated in our previous study and were not otherwise engaged with clinic paperwork or on the phone during recruitment. Of these eligible parents approached, 54 declined participation. Thus, the overall response rate was 77%. Parents were compensated with $10.00 for their time to complete the survey.

Parents who participated in Study 2 (see Table 3) were primarily (82.1%) mothers of the targeted child (versus 8.7% fathers, 9.2% “other”), Caucasian (89.6% versus 3.3% Hispanic, 3.3% Black, and 3.8% other race), and having at least a high school education (93.3% mothers, 81.9% fathers). They also reported that the child about whom they were answering was, on average, 9 years of age (M = 8.68, SD = 3.62, range = 4 – 16), had a PSC of 17.67 (SD = 11.19, range = 0 – 54), and was male (57.4% versus 42.6% female).

Due to the change in recruitment, we examined demographic differences between parents in Study 1 and Study 2. There were no significant differences found between studies on child age, child gender, income, mother and father education, and race. Results revealed parents in Study 1 reported significantly more child psychosocial concerns as indicated by greater PSC scores than parents in Study 2 (M = 28.87 versus M = 17.67, t = 11.86, p < .001). Similarly, results revealed parents in Study 1 were more likely than parents in Study 2 to have talked previously with a counselor, therapist, or psychologist about their child’s behavior (51.9% versus 31.5%, respectively, χ2 = 20.14, p < .001). However, when considering only Study 2 parents whose children scored 20 or greater on the PSC, no significant difference in psychosocial concerns (M = 28.87 versus M = 28.38, respectively, t = .50, p = .621) or prior use between Studies 1 and 2 were indicated (51.9% versus 51.4%, respectively, χ2 = .01, p = .94).

Measures

Identical to Study 1, measures included demographics, the Pediatric Symptom Checklist (PSC; Jellinek, Murphy, & Burns, 1986; α = .85), Parents’ Perceived Stigma of Service Seeking (PPSSS; self (α = .94) and public (α = .93)); the Social Impact Scale (Fife &Wright, 2000; α = .87), and Efficacy of Help Seeking (α = .76). In addition, to measure Mental Health Service Seeking we assessed parents’ willingness to seek services for children using one item, which read “How likely would you be to talk with a counselor, therapist or psychologist?” This item differed from Study 1 by not specifying a location of the service. Parents responded by indicating how likely they would be (1 = not at all, 6 = definitely). Prior use was measured with one item that asked parents to indicate whether they have ever “talked about concerns they have about their child’s behavior or emotions with a counselor, therapist, or psychologist. Parents responded by indicating “yes” or “no”.

Results

Construct and Criterion Validity of the Self and Public Subscales of the PPSSS Scale

The main goal of Study 2 was to further examine the validity of the scores reflecting public and self stigma. First, we conducted the CFA examined in Study 1. Results of testing the two-factor model revealed additional support for the validity of scores reflecting parents’ perceived public and self-stigma of seeking services for children (χ2 (94) = 215.25, p = .00, CFI = .97, SRMR = .05, RMSEA = .09 (90% CI = .07 – .10)). Although the chi-square was statistically significant, other indicators represented adequate fit (particularly when sample sizes are less than 500; Weston & Gore, 2006). In addition, the ratio of chi-square to degrees of freedom was low (2.29). And, the factor loadings revealed that all items loaded on their respective factors (loadings ranged from .64 – .96).

Second, we calculated bivariate relations between public and self stigma and boththe Social Impact Scale and low efficacy of help seeking using the entire sample of parents (N = 184). We anticipated that self and public stigma would be associated positively with these two existing measures. As hypothesized, results revealed parents’ self and public (respectively) stigma of seeking services for their children were significantly and positively related to Social Impact Scale (r = .31, p < .001; r = .50, p < .001) and to low efficacy of help seeking (r = .43, p < .001; r = .47, p < .001).

Predictive Validity of the Self and Public Subscales of the PPSSS Scale

We conducted two multiple regression analyses using the entire sample of parents (N = 184), in order to assess the extent to which self and public stigma predicted willingness to seek mental health services over and above existing measure of stigma. Because there were significant differences between Study 1 and 2 in PSC and prior use, we entered these variables as covariates in the first step. Second, we entered the self stigma PPSSS Scale to determine the incremental change in R2. Results revealed that the self stigma PPSSS Scale contributed uniquely to parents’ willingness to seek mental health services (ΔR2 = .060, p = .001); the more parents perceived self stigma, the less willing they were to seek services (b = −.15, SE = .04, p = .001). Similarly, in the second analysis we entered prior use and PSC score in the first step, followed by the public stigma PPSSS Scale in the second step. Results revealed that the public stigma PPSSS Scale contributed uniquely to parents’ willingness to seek services in a mental health facility (ΔR2 = .023, p = .041); the more parents perceived public stigma, the less willing they were to seek services (b = −.10, SE = .05, p = .041). Of note, we also found similar relations after adding the Social Impact Scale as an additional covariate (self stigma: b = −.16, SE = .05, p = .001; public stigma: b = −.12, SE = .06, p = .047). We tested the Social Impact Scale as another covariate, because it was the existing measure of parents’ stigma related to the child’s problem and most strongly related to our self and public stigma subscales in Study 1.

Discussion

Study 2 showed additional validity of scores reflecting the self and public subscales of the PPSSS Scale, as evidenced by the CFA results and significant relations between the self and public stigma measure with an existing measure of parents’ stigma related to the child’s psychosocial problems (Social Impact Scale) and low efficacy of help seeking. As well, self and public stigma predicted parents’ willingness to seek mental health services for their child from a counselor, therapist, or psychologist, regardless of child behavioral or psychosocial concerns, prior use, and stigma of the child’s problem.

In this second study we recruited a broader range of parents as defined by a wider range of child PSC scores and prior service use. This permitted us to examine the self and public stigma subscales of the PPSSS Scale among parents who reported having children with psychosocial concerns as well as parents who did not. Results revealed that the scale functioned well among this general sample of parents from rural Appalachia as evidenced by the construct and predictive validity tests. Yet, parents did not differ in demographics such as income and education from those in Study 1, which when compared to the county-level statistics available showed less education and income. Future research may benefit from obtaining a larger and more diverse sample from varying rural locations to increase representativeness and generalizability. Indeed, although our recruitment strategy may have resulted in a broader range of parents of children, we recruited parents from a limited number of pediatric primary care clinics in rural Appalachia. Thus, the results of Study 2 might still be limited, which we further discuss below.

GENERAL DISCUSSION

Overall results of Studies 1 and 2 provide support for the validity of score interpretations of the self and public subscales of the PPSSS Scale. Thus, this paper contributes to the knowledge base in the area of parents living in rural communities in regard to seeking services for their children’s psychosocial concerns. This new scale could have implications for both clinicians and researchers working with such parents. For example, this instrument permits examination of the extent that parents’ stigma of service seeking may be a barrier to services for children with psychosocial concerns. Prior to the PPSSS scale, no measure existed to tap perceptions held by parents living in rural communities about stigma of seeking mental health services for their children. The 17-item measure is brief and therefore non-taxing to parents. The PPSSS may aid future studies that examine how parents’ perceptions of stigma might predict intentions to seek a variety of mental health services for their children. As translational research seeks to impact end-point use of specific treatments and service delivery mechanisms, measures such as this one may provide a barometer regarding parents’ perceptions of stigma as they predict willingness to seek services in particular contexts, such as community mental health agencies, schools, and physicians’ offices. Presumably, the more stigma parents perceive, the more appealing innovative service delivery mechanisms might seem – such as integrated primary care and school-based mental health care.

Although these studies expand prior knowledge on perceived stigma of service seeking by honing perceptions held by parents in rural Appalachia (both with and without children with psychosocial concerns), it remains unclear how this scale would function in other rural or urban locations. One future direction is to examine the PPSSS in urban counterparts, as well as those in other rural regions of the U.S. Particular items of the PPSSS may carry more or less weight in differing locations. For example, two of the items from the PPSSS discuss people in town finding out and wanting to seek treatment in another town, which highlight the rural focus and might not be found in other regions. Similarly, one item of the PPSSS taps the response from one’s church regarding service seeking, which may be more relevant for those in rural Appalachia, where people tend to be highly religious.

Future work in the area of parents’ stigma of service seeking for children might also benefit from a mixed-methods strategy whereby rich qualitative data on stigma is collected, in addition to quantitative survey based data (e.g., Creswell & Clark, 2011). This qualitative data involving community-based participatory research or ethnography might allow for an in-depth discussion about stigma in relation to services for children and likely would require a multidisciplinary team, which is aligned with a translational research approach.

In sum, findings support the validity of score interpretations of a new instrument that assesses stigma of seeking mental health services for children perceived by parents in rural communities. We believe that researchers and clinical providers alike might find this instrument useful in understanding the extent to which perceived stigma is playing a role in parents’ decision making regarding services for children. Thus, this work ultimately could have implications for the mental health services children receive.

Figure 1.

Figure 1

Open in a new tab

χ(94)2 = 164.35, p = .00, CFI = .97, SRMR = .04, RMSEA = .07

Figure 2.

Figure 2

Open in a new tab

χ(94)2 = 215.25, p = .00, CFI = .97, SRMR = .05, RMSEA = .09

Acknowledgments

This study was supported by the National Institute of Mental Health (R21 MH090539-01) and the Appalachian Center for Translational Research in Disparities at East Tennessee State University. Contents are solely the responsibility of the authors and do not necessarily represent the official views of the funding agencies. The authors would also like to thank Craig Anne Heflinger and Ana Maria Brannan who served as consultants, and Matthew McBee and Robert Price, who served as statistical consultants. Finally, we would like to thank the graduate and undergraduate students who assisted in data collection, and the many primary care practices and parents who participated, making this project possible.

References

  1. Beaver PD. Rural community in the Appalachian south. Lexington, KY: The University Press of Kentucky; 1986. [Google Scholar]
  2. Bray JH, Enright MF, Easling I. Psychological practice in rural primary care. In: Frank RG, McDaniel SH, Bray JH, Heldring M, editors. Primary Care Psychology. 2004. pp. 243–258. [Google Scholar]
  3. Corrigan PW. How stigma interferes with mental health care. American Psychologist. 2004;59:614–624. doi: 10.1037/0003-066X.59.7.614. [DOI] [PubMed] [Google Scholar]
  4. Crandall CS. AIDS-related stigma and the lay sense of justice. Contemporary Social Psychology. 1991;15:66–67. [Google Scholar]
  5. Creswell JW, Clark VLP. Designing and conducting mixed methods research. Thousand Oaks, CA: Sage; 2011. [Google Scholar]
  6. Crocker J, Major B, Steele C. Social stigma. In: Gilbert D, Fiske ST, Lindzey G, editors. The Handbook of Social Psychology. 4. Vol. 2. New York: McGraw Hill; 1998. pp. 504–553. [Google Scholar]
  7. Eckenrode S. The mobilization of social supports: Some individual constraints. American Journal of Community Psychology. 1983;11:509–520. doi: 10.1007/BF00896802. [DOI] [PubMed] [Google Scholar]
  8. Fife BL, Wright ER. The dimensionality of stigma: A comparison of its impact on the self of persons with HIV/AIDS and cancer. Journal of Health and Social Behavior. 2000;41:50–67. [PubMed] [Google Scholar]
  9. Fox J, Merwin E, Blank M. De facto mental health services in the rural south. Journal of Health Care for the Poor and Underserved. 1995;6(4):434–468. doi: 10.1353/hpu.2010.0003. [DOI] [PubMed] [Google Scholar]
  10. Girio-Herrera E, Owens JS, Langberg JM. Perceived barriers to help-seeking among parents of at-risk kindergarteners in rural communities. Journal of Clinical Child & Adolescent Psychology. 2013;42:68–77. doi: 10.1080/15374416.2012.715365. [DOI] [PubMed] [Google Scholar]
  11. Goffman E. Stigma: Notes on the management of spoiled identity. New York: Prentice Hall; 1963. [Google Scholar]
  12. Goins RT, Spencer SM, Williams K. Lay meanings of health among rural older adults in Appalachia. The Journal of Rural Health. 2011;27(1):13–20. doi: 10.1111/j.1748-0361.2010.00315.x. [DOI] [PubMed] [Google Scholar]
  13. Guttman L. Some necessary conditions for common factor analysis. Psychometrika. 1954;19:149–162. [Google Scholar]
  14. Heflinger CA, Hinshaw SP. Stigma in child and adolescent mental health services research: Understanding professional and institutional stigmatization of youth with mental health problems and their families. Administration and Policy in Mental Health and Mental Health Services Research. 2010;37:61–70. doi: 10.1007/s10488-010-0294-z. [DOI] [PubMed] [Google Scholar]
  15. Horn JL. A rationale and test for the number of factors in factor analysis. Psychometrika. 1965;30:179–185. doi: 10.1007/BF02289447. [DOI] [PubMed] [Google Scholar]
  16. Hoyt DR, Conger RD, Valde JG, Weihs K. Psychological distress and help seeking in rural America. American Journal of Community Psychology. 1997;25:449–470. doi: 10.1023/a:1024655521619. [DOI] [PubMed] [Google Scholar]
  17. Jameson JP, Blank MB. The role of clinical psychology in rural mental health services: Defining problems and developing solutions. Clinical Psychology: Science and Practice. 2007;14:283–298. [Google Scholar]
  18. Jellinek MS, Murphy JM, Burns B. Brief psychosocial screening in outpatient pediatric practice. Journal of Pediatrics. 1986;109:371–378. doi: 10.1016/s0022-3476(86)80408-5. [DOI] [PubMed] [Google Scholar]
  19. Jellinek MS, Murphy JM, Robinson J. Pediatric symptom checklist: Screening school-age children for psychosocial dysfunction. Journal of Pediatrics. 1988;112(2):201–209. doi: 10.1016/s0022-3476(88)80056-8. [DOI] [PubMed] [Google Scholar]
  20. Jellinek MS, Murphy JM, Little M. Use of the pediatric symptom checklist (PSC) to screen for psychosocial problems in pediatric primary care: A national feasibility study. Archives of Pediatric and Adolescent Medicine. 1999;153(3):254–260. doi: 10.1001/archpedi.153.3.254. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Jones EE, Farina A, Hastorf AH, Markus H, Miller DT, Scott RA. Social stigma: The psychology of marked relationships. New York: Freeman; 1984. [Google Scholar]
  22. Jones L. Appalachian values. Ashland, KY: Jesse Stuart Foundation; 1994. [Google Scholar]
  23. Judd F, Jackson H, Komiti A, Murray G, Fraser C, Grieve A, Gomez R. Help-seeking by rural residents for mental health problems: The importance of agrarian values. Australian and New Zealand Journal of Psychiatry. 2006;40:769–776. doi: 10.1080/j.1440-1614.2006.01882.x. [DOI] [PubMed] [Google Scholar]
  24. Kaiser HF. The application of electronic computers to factor analysis. Educational and Psychological Measurement. 1960;20:141–151. [Google Scholar]
  25. Keefe SE. Appalachian mental health. Lexington, KY: The University Press of Kentucky; 1998. [Google Scholar]
  26. Levinson RM, Starling DM. Retardation and the burden of stigma. Deviant Behavior. 1981;2:371–390. [Google Scholar]
  27. Link BG, Cullen FT, Struening EL, Shrout PE, Dohrenwend BP. A modified labeling theory approach to mental disorders: An empirical assessment. American Sociological Review. 1989;54:400–424. [Google Scholar]
  28. Link BG, Phelan JC. Conceptualizing stigma. Annual Review of Sociology. 2001;27:363–385. [Google Scholar]
  29. MacAvoy S, Lippman DT. Teaching culturally competent care: Nursing students experience in rural Appalachia. Journal of Transcultural Nursing. 2001;12(3):221–227. doi: 10.1177/104365960101200306. [DOI] [PubMed] [Google Scholar]
  30. Mickelson KD. Perceived stigma, social support, and depression. Personality and Social Psychology Bulletin. 2001;27:1046–1056. [Google Scholar]
  31. Mukolo A, Heflinger CA. Factors associated with attributions about child health conditions and social distance preference. Community Mental Health Journal. 2011;47:286–299. doi: 10.1007/s10597-010-9325-1. [DOI] [PubMed] [Google Scholar]
  32. Mukolo A, Heflinger CA, Wallston KA. The stigma of childhood mental disorders: A conceptual framework. Journal of the American Academy of Child and Adolescent Psychiatry. 2010;49:92–103. doi: 10.1097/00004583-201002000-00003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Appalachian Regional Commission and the National Opinion Research Center. Final report. 2008. An analysis of mental health and substance use disparities & access to treatment services in the Appalachian region. [Google Scholar]
  34. Pan AW, Chung L, Fife BL, Hsiung PC. Evaluation of the psychometrics of the Social Impact Scale: A measure of stigmatization. International Journal of Rehabilitation Research. 2007;30:235–238. doi: 10.1097/MRR.0b013e32829fb3db. [DOI] [PubMed] [Google Scholar]
  35. Pescosolido BA, Fettes DL, Martin JK, Monahan J, McLeod JD. Perceived dangerousness of children with mental health problems and support for coerced treatment. Psychiatric Services. 2007;58:1–7. doi: 10.1176/ps.2007.58.5.619. [DOI] [PubMed] [Google Scholar]
  36. Pescosolido BA, Perry BL, Martin JK, McLeod JD, Jensen PS. Stigmatizing attitudes and beliefs about treatment and psychiatric medication for children with mental illness. Psychiatric Services. 2007;58(5):613–618. doi: 10.1176/ps.2007.58.5.613. [DOI] [PubMed] [Google Scholar]
  37. Philo C, Parr H, Burns N. Rural madness: A geographical reading and critique of the rural mental health literature. Journal of Rural Studies. 2003;19:259–281. [Google Scholar]
  38. Reynolds WM. Development of reliable and valid short forms of the Marlowe-Crowne Social Desirability Scale. Journal of Clinical Psychology. 1982;38:119–125. [Google Scholar]
  39. Riley D, Eckenrode J. Social ties: Subgroup differences in costs and benefits. Journal of Personality and Social Psychology. 1986;51:770–778. doi: 10.1037//0022-3514.51.4.770. [DOI] [PubMed] [Google Scholar]
  40. Rosenberg M. Society and the Adolescent Self Image. Princeton, NJ: Princeton University Press; 1965. [Google Scholar]
  41. Sinclair SJ, Blais MA, Gansler DA, Sandberg E, Bistis K, LoCicero A. Psychometric properties of the Rosenberg Self-Esteem Scale: Overall and across demographic groups living within the United States. Evaluation & the Health Professions. 2010;33:56–80. doi: 10.1177/0163278709356187. [DOI] [PubMed] [Google Scholar]
  42. STATSAmerica. USA counties in profile. 2013 Retrieved on October 8, 2013, from http://www.statsamerica.org/profiles/us_profile_frame.html.
  43. U.S. Department of Health and Human Services. Mental health: A report of the surgeon general-executive summary. Rockville, MD: U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services Administration, Center for Mental Health Services, National Institutes of Health, National Institute of Mental Health; 1999. [Google Scholar]
  44. Vogel DL, Wade NG, Haake S. Measuring the self-stigma associated with seeking psychological help. Journal of Counseling Psychology. 2006;53:325–337. [Google Scholar]
  45. Wagenfeld MO. A snapshot of rural and frontier America. In: Stamm BH, editor. Rural behavioral health care: An interdisciplinary guide. Washington, DC: American Psychological Association; 2003. pp. 33–40. [Google Scholar]
  46. Wang PS, Lane M, Olfson M, Pincus HA, Wells KB, Kessler RC. Twelve-month use of mental health services in the United States: Results from the national comorbidity survey replication. Archives of General Psychiatry. 2005;62:629–640. doi: 10.1001/archpsyc.62.6.629. [DOI] [PubMed] [Google Scholar]
  47. Weiss MG, Jadhav S, Raguram RR, Vounatsou P, Littlewood R. Psychiatric stigma across cultures: Local validation in Bangalore and London. Anthropology & Medicine. 2001;8(1):71–88. [Google Scholar]
  48. Weston R, Gore PA. A brief guide to Structural Equation Modeling. The Counseling Psychologist. 2006;34:719–751. [Google Scholar]
  49. Williams SL, Mickelson KD. A paradox of support seeking and rejection among the stigmatized. Personal Relationships. 2008;15:493–509. [Google Scholar]
  50. Willis G, Royston P, Bercini D. The use of verbal report methods in the development and testing of questionnaires. Applied Cognitive Psychology. 1991;5:251–267. [Google Scholar]
  51. Yang LH, Kleinman A, Link BG, Phelan JC, Lee S, Good B. Culture and stigma: Adding moral experience to stigma theory. Social Science and Medicine. 2007;64(7):1524–1535. doi: 10.1016/j.socscimed.2006.11.013. [DOI] [PubMed] [Google Scholar]
  52. Zwick WR, Velicer WF. Comparison of five rules to determining the number of components to retain. Psychological Bulletin. 1986;99:432–442. [Google Scholar]

RESOURCES