The Barriers to Mental Health Services Scale Revised: Psychometric Analysis Among Older Adults

Abstract

Objectives

Older adults underutilize mental health services suggesting that significant barriers are operating. This study presents reliability and validity data for a revised version of the self-report Barriers to Mental Health Services Scale (BMHSS) designed to quantify 10 barriers to mental health service use, so that barriers can be examined collectively.

Methods

The Barriers to Mental Health Services Scale Revised (BMHSS-R) was revised to improve its reliability and validity, including adding items, eliminating poor items, and balancing the number of items across subscales. A sample of 100 older adults (M age = 72.1 years, SD = 17.8 years) completed the BMHSS-R, the Beliefs Toward Mental Illness Scale, and the Willingness to Seek Help Questionnaire.

Results

Internal consistency for the 10 subscales of the BMHSS-R ranged between .63 and .87, with 8 of the 10 values greater than .70. Correlational analyses indicated that many of the subscales overlap considerably but are still distinct. Convergent validity of the BMHSS-R subscales of help-seeking and stigma was partially supported, although correlations were modest.

Conclusion

Revisions to the BMHSS resulted in improved reliability estimates for use as a measure of perceived barriers to mental health services. We recommend when using the BMHSS-R to combine results with other information (e.g., service utilization data) to characterize a profile of barriers. We discuss directions for future research and further refinement of the BMHSS-R.

The prevalence of mental health conditions among older adults residing in the community ranges from 6.8% to 10.2% with only 10% of those in need actually receiving mental health services (Institute of Medicine [IOM], 2012). The gap between the number of older adults with mental health care needs and those receiving services is anticipated to grow as the population of older adults increases (IOM). A variety of barriers have been posed as reasons for this underuse of services. These include intrinsic barriers attributed to internal characteristics and beliefs, such as perceptions of stigma about receiving services and extrinsic barriers due to external social, economic, geographic, and other service-related factors, such as the number of clinicians available (Pepin, Segal, & Coolidge, 2009). To maximize appropriate, efficient, and effective use of mental health services for older adults, metrics to characterize and quantify these barriers are needed. In a prior report (Pepin et al.), we described a self-report instrument that aggregates the most commonly cited barriers into a multi-component scale (Barriers to Mental Health Services Scale, BMHSS). A preliminary study of the psychometric characteristics of the original version of the scale provided evidence of its promise as a measure of barriers to seeking services with the majority of subscales reaching acceptable values of internal consistency (Pepin et al.). The next step in the development of this instrument was to refine the scale to achieve the most parsimonious and balanced set of questions and to characterize the instrument’s reliability and validity. This report describes the revisions and reliability and validity analyses of the revised measure.

Previous research has identified several barriers that explain why older adults underutilize mental health services, despite their demonstrated need for such services. Determining which obstacles are most limiting for older adults has been challenging due to a lack of research that systematically examines which barriers are the most prevalent and most influential in limiting use of needed mental health services. Quantitative studies typically have examined only one or two barriers of interest (e.g., Mackenzie, Scott, Mather, & Sareen, 2008; Sarkisian, Lee-Henderson, & Mangione, 2003; Segal, Coolidge, Mincic, & O’Riley, 2005) whereas qualitative studies on barriers have included multiple barriers simultaneously, but were unable to quantify the comparative strength of different barriers (Choi & Kimbell, 2009; Palinkas et al., 2007; Solway, Estes, Goldberg, & Berry 2010).

Individual barriers to mental health services that older adults encounter have been described in numerous accounts, but it is unlikely that each barrier operates in isolation. For example, it is not uncommon for an older adult to have limited financial resources, to believe that feelings of sadness are normal for older people, and also to be unlikely to disclose their mental health symptoms to their primary care provider due to concerns related to the stigma of mental illness. An understanding of various barriers to mental health services can be used to identify points within a service delivery system where improvements may be made to decrease obstacles and expand access to services. Because more than one barrier may be present, there is a need for a measure that provides both individual and aggregate ratings of a broad array of potential barriers. This multidimensional measure will improve understanding of how intrinsic and extrinsic obstacles to care contribute to the low utilization of mental health services in older adults (Pepin et al., 2009). Our review of the literature failed to identify any instruments that included a comprehensive array of both of these types of barriers, which prompted our development of the original Barriers to Mental Health Services Scale.

The BMHSS is comprised of 10 subscales, each representing a barrier to mental health services that has been described in the literature. The BMHSS is designed to quantify the obstacles that are most prevalent within a given population of older adults (e.g., a subpopulation of older adults, a catchment area, or a geographic territory). The results from the BMHSS can be used to characterize a profile of barriers, underscoring the barriers that are higher and those that are lower, identifying points where changes can be made to improve a delivery system and decrease barriers.

The first version of the BMHSS was developed by conducting a review of the mental health services research literature on barriers to mental health services for older adults, followed by item generation, and expert review of the proposed measure (Pepin et al., 2009). We used information derived from our literature review to develop a measure with 10 subscales, each characterizing a barrier prominent in the literature. The instrument was organized into two domains, intrinsic barriers and extrinsic barriers. Five intrinsic barriers were identified, each representing a subscale of the BMHSS: help-seeking attitudes; stigma; knowledge and fear of psychotherapy; belief about inability to find a psychotherapist; and belief that depressive symptoms are normal. Five extrinsic barriers were also identified, each representing a subscale of the BMHSS: insurance and payment concerns; ageism; concerns about psychotherapist’s qualifications; physician referral; and transportation concerns. Over 200 items were initially developed and were presented to experts in geropsychology for review to ensure adequate domain coverage with minimal redundancy. Many of the items were redundant and thus deleted, resulting in a 63-item instrument. The BMHSS was administered to younger and older adults. Our initial pilot assessment of the performance of this measure found many of the subscales reached acceptable values of internal consistency, with 6 of the 10 subscales having Cronbach’s alpha values above .70, and two of the subscales having Cronbach’s alpha values of .69 (Pepin et al.). Nevertheless, the BMHSS had some notable limitations including:

• Subscale scores with low reliability estimates. Four subscales’ scores had reliability estimates below the acceptable value (.70 or greater [Streiner, 2003]).

• Inconsistent distribution of items across subscales. The number of items comprising subscales ranged from 3 to 12.

• Items with reverse scoring. Items with reversed scoring frequently have lower internal consistency values, particularly among older adults. Many popular instruments designed for use with older adults do not include reverse score items (e.g., Geriatric Anxiety Scale; Center for Epidemiological Studies – Depression Scale). It has been recommended that instruments for older adults be designed without reverse-scored items (Carlson et al., 2011), and the removal of reverse score items was described in an article reporting the development and validation of the Geriatric Anxiety Inventory (Pachana et al., 2007).

• Neutral response category. The original version of the BMHSS had a neutral mid-point of “neither agree nor disagree.” A neutral response option is somewhat artificial in this case as it suggests that respondents may be unsure whether or not a barrier a barrier has impacted their use of mental health services.

• Layout of the instrument. The first version of the BMHSS was not optimized to be consistent with the standard of being easy to use and appearing “clear and uncluttered” (Fowler, 1988, p.103).

Based on these findings and limitations, we identified the need to refine the initial version of the BMHSS to create a new version that addressed the limitations described above. We revised the scale based on procedures for survey development suggested by DeVellis (2012). The aim of this report is to describe the process used to develop the BMHSS-Revised (BMHSS-R), resulting in a more parsimonious and psychometrically robust instrument designed to reliably and validly assess barriers to mental health services for older adults.

Method

To evaluate the reliability and validity of the revised instrument, we administered the BMHSS-R to a sample of older adults and then conducted the following analyses: 1) reliability by internal consistency and correlations between subscales of the BMHSS-R and total score; and 2) convergent validity by correlations between the BMHSS-R subscales of stigma and help-seeking with similar existing measures.

Participants

The participant sample was comprised of 100 community dwelling older adults aged 60 to 98 years-old (M = 72.1 years, SD = 7.6; 75% women, 89% European American) with years of education ranging from 2 to 20 years (M = 14.5, SD = 3.4). See Table 1 for full demographic details. The self-reported measure of overall physical health ranged from 1 to 100 (M = 75.6, SD = 22.4). We selected a community dwelling sample of older adults to capitalize on the possibility of including both older adults who perceived barriers to services that prevented them from engaging in services, as well as older adults who had engaged in services but may have experienced obstacles related to seeking services.

Table 1.

Demographic Information of Participant Sample

Demographic
Age, mean (SD)	72.09 (7.56)
Women, % (n)	75.30% (73)
European American, % (n)	89.90% (89)
Completed high school or less, % (n)	32.00% (31)
Married or partnered, % (n)	48.50% (47)
Has children, % (n)	91.90% (91)
Religious, % (n)	81.30% (78)
Has health insurance, % (n)	96.00% (96)
Visited a mental health clinician, % (n)	43.40% (43)

Measures

Barriers to Mental Health Services Scale-Revised (BMHSS-R)

The Barriers to Mental Health Services Scale-Revised is a 43-item self-report questionnaire that measures 10 barriers preventing individuals from seeking mental health services. The survey is comprised of two domains, intrinsic barriers and extrinsic barriers. Five intrinsic barriers include: help-seeking attitudes; stigma; knowledge and fear of psychotherapy; belief about inability to find a psychotherapist; and belief that depressive symptoms are normal. Five extrinsic barriers include: insurance and payment concerns; ageism; concerns about psychotherapist’s qualifications; physician referral; and transportation concerns. The help seeking subscale addresses help seeking attitudes and behaviors. The stigma subscale addresses attitudes towards mental illness and treatment. The knowledge and fear subscale addresses knowledge and fear about engaging in psychotherapy. The belief about an inability to find a psychotherapist subscale addresses beliefs about difficulty finding a psychotherapist. The belief that depressive symptoms are normal subscale addresses the belief that depressive symptoms are normal for the participant and their cohort. The insurance and payment subscale addresses insurance and payment barriers. The ageism subscale addresses perceived attitudes towards older adults within mental health services. The concerns about psychotherapist’s qualifications subscale addresses the concern that specialty mental health providers are not qualified to help participants. The physician referral subscale addresses resistance to adhere to physical referrals to specialty mental health providers. The transportation concerns subscale addresses concerns about transportation to and from specialty mental health providers. For a more detailed descriptions and sample items please refer to Pepin and colleagues (2009). Participants rate the extent to which they agree with each item on a 4-point Likert-type scale ranging from 1 (strongly disagree) to 4 (strongly agree). The subscale scores are standardized; first the responses to the items from each subscale are summed and then divided by the number of items on each subscale. Additionally, for conceptual organization, subscales can be divided into two domains of barriers, intrinsic and extrinsic. The intrinsic barriers scale score is constructed by summing the items from the following subscales: help-seeking, stigma, knowledge and fear of psychotherapy, belief about inability to find a psychotherapist, and belief that depressive symptoms are normal. Similarly, the extrinsic barriers scale score is constructed by summing the items from the remaining subscales, including: insurance and payment concerns, ageism, concerns about psychotherapist’s qualifications, physician referral, and transportation concerns. Higher scores on all scales indicate a higher degree of perceived barriers.

Revisions to the BMHSS

All 10 subscales of the original version were retained and no new subscales were added. Revisions to the original version included changing the response format from a 5-point Likert-type scale to a 4-point scale and removing the neither agree nor disagree response option. We changed the response format from an odd to an even number of items to force respondents to make a decision whether they agree or disagree for each item included in order to assess which barriers have affected their use of mental health services. Additionally, for the revised measure, the number of items (5) was equalized across subscales. Some items from the first version were retained in the revised measure, based on item analyses showing that Cronbach’s alpha would decrease if the item were deleted. Likewise, those items that had poor item characteristics were deleted based on item analyses showing that Cronbach’s alpha would increase if the item were deleted (Pepin et al., 2009). Items were also added to increase domain coverage. Additional items were generated for 3 original subscales with fewer than 5 items (belief about inability to find a psychotherapist, belief that depressive symptoms are normal, and physician referral). Additional items were generated based on existing BMHSS items, literature review, and consultation with experts in clinical geropsychology. Two authors (RP & DLS) reviewed all additional items to ensure adequate domain coverage with minimal redundancy. As noted above, the 13 reverse scored items from the original version were changed to align with the scoring direction of the other items in the scale (i.e. higher scores indicating higher perception of barriers). Finally, the layout of the instrument was changed from circling responses below each question to checking boxes in a column to the right of all questions in order to make the measure more streamlined and to improve the ease of administration and scoring.

Beliefs Toward Mental Illness Scale (BMI)

The Beliefs Toward Mental Illness Scale (Hirai & Clum, 2000) is a self-report questionnaire consisting of 21 items designed to measure the degree to which individuals endorse negative stereotypes of mental illness. This measure was selected for use in convergent validity analyses due to its conceptual similarities to the BMHSS-R stigma subscale. Participants rate items on a six-point Likert-type scale ranging from 0 (completely disagree) to 5 (completely agree), with higher scores reflecting more negative stereotypes. There are three subscales: dangerousness, poor social and interpersonal skills, and incurability, as well as a total score. Reliability estimates as assessed with Cronbach’s alpha have been shown to be high (from .89 to .91) (Hirai & Clum, 2000). In the present sample, internal consistency estimates (Cronbach’s alpha) for the scores on the three BMI subscales and total score were .78 for dangerousness, .88 for poor social and interpersonal skills, .86 for incurability, and .94 total score.

Willingness to Seek Help Questionnaire (WSHQ)

The Willingness to Seek Help Questionnaire (Cohen, 1999) is a self-report questionnaire consisting of 25 items designed to measure the degree to which individuals are open to seeking mental health services. This measure was selected for use in convergent validity analyses due to its conceptual similarities to the BMHSS-R help-seeking subscale. Participants rate items on a four-point Likert-type scale ranging from 0 (do not identify at all) to 3 (strong identification). The items sum to a total score, with higher scores signifying more willingness to seek mental health services. Reliability as assessed with Cronbach’s alpha has been shown to be high (.85; Cohen, 1999). In the present sample, reliability assessed with Cronbach’s alpha was similar and excellent (.89).

Procedure

Participants were recruited through a local registry of older adults who had expressed interest in participating in research. This registry was developed in 2004 to create a centralized system for recruiting older adults at the institution where the study was conducted. Since its inception there have been ongoing efforts among researchers who conduct studies with older adults to share recruitment of participants. An IRB approved database is maintained by the gerontology center. Older adults in the community who are interesting in participating in studies are actively recruited to be part of the registry. We contacted registry members who were aged 60 years and older and invited them to participate. In our study, participants first read and signed an informed consent form before completing any other materials. Participants subsequently completed a packet containing the BMHSS-R, BMI, WSHQ, and a demographics questionnaire with the option to skip any items for any reason. Participants were scheduled in small groups and completed the packet individually at a local, community-based gerontology center. An author (RP) was present for each data collection session. To address missing data, we used the Expectation-Maximization (EM) algorithm to impute missing items. On the BMHSS-R, 28% of cases had missing data for a total of 1.2% missing data. Data were missing completely at random.

Results

Reliability

Means, standard deviations, and reliabilities (Cronbach’s alphas) of the BMHSS-R total scores and subscale scores were calculated. According to Streiner (2003), alpha values of .70 or greater are considered acceptable. The majority of the internal consistency estimates of the subscale scores were deemed to be adequate: eight of the 10 subscales had a Cronbach’s alpha above .70 whereas the help-seeking and physician referral subscales failed to meet the .70 cut-off (see Table 2). In addition, the Intrinsic Total Score, the Extrinsic Total Score, and the overall Total Score had excellent internal consistency values.

Table 2.

Internal Consistency, Mean and Standard Deviation, for Barriers to Mental Health Seeking Services Scale Revised: Subscales and Total after Revision

Subscale/Scale Items	alpha	M(SD)
Help-seeking	.67	2.77(.72)
11, 15, 39, 44
Stigma	.87	1.71(.61)
6, 7, 19, 48, 50
Knowledge and Fear of Psychotherapy	.85	1.84(.62)
26, 32, 33, 45, 46
Belief about Inability to Find a Psychotherapist	.86	1.95(.60)
5, 9, 23, 28, 34
Belief that Depressive Symptoms are Normal	.74	2.45(.55)
3, 18, 24, 27
Insurance/Payment Concerns	.79	2.27(.68)
22, 31, 40
Ageism	.82	1.61(.50)
2, 21, 36, 37, 38
Concerns about Psychotherapist’s Qualifications	.81	1.75(.48)
1, 13, 16, 25, 29
Physician Referral	.63	1.94(.63)
4, 10, 49
Transportation Concerns	.76	1.77(.55)
12, 20, 35, 41, 43
Intrinsic Total Score	.93	2.00(.48)
3, 5, 6, 7, 9, 11, 15, 18, 19, 23, 24, 26, 27, 28, 32, 33, 34, 39, 44, 45, 46, 48, 50
Extrinsic Total Score	.92	1.82(.46)
1, 2, 4, 10, 12, 13, 16, 20, 21, 22, 25, 29, 31, 35, 36, 37 38, 40, 41, 43, 49
Total Score	.96	1.91(.46)
All except for 8, 14, 17, 30, 42, 47

Note. alpha = Cronbach’s Alpha; M = mean; SD = standard deviation

Next, all the BMHSS-R items were examined to determine which had low response rates (i.e., participants who did not answer these items) using a database without imputed values. Two items from the insurance/payment concerns subscale had relatively high rates of missing data: items 40 (6% missing) and 42 (17% missing). Examination of these items revealed these as knowledge-based questions about insurance policies. Due to the low response rates of these items, item 42 removed from the measure, increasing Cronbach’s alpha of the scores on this subscale from .78 to .79.

Next, correlations between subscales of BMHSS-R and total score were computed (see Table 3). Consistent with our conceptualization, all subscales had statistically significant and positive correlations with other subscales and the total score. Each subscale had at least large strength correlations with one other subscale except for the belief that depressive symptoms are normal subscale. Several relationships were very strong (r > .69). The subscales of belief that depressive symptoms are normal, insurance/payment concerns, and transportation had the lowest correlations with other subscales.

Table 3.

Subscale and Total Score Correlations for the BMHSS-R Subscales and Total

	HS	S	KF	FT	BDN	IP	A	CTQ	PR	T	I	E	BTS
Help-seeking	1.00
Stigma	.64	1.00
Knowledge and Fear of Psychotherapy	.66	.86	1.00
Inability to Find a Therapist	.46	.63	.65	1.00
Belief that Depression is Normal	.48	.43	.45	.34	1.00
Insurance/Payment Concerns	.44	.57	.68	.65	.34	1.00
Ageism	.57	.79	.76	.63	.41	.58	1.00
Confidence in Therapist’s Qualifications	.54	.79	.80	.72	.43	.62	.81	1.00
Physician Referral	.60	.67	.64	.51	.36	.49	.60	.61	1.00
Transportation Concerns	.36	.55	.57	.60	.34	.66	.63	.59	.48	1.00
Intrinsic	.78	.91	.92	.78	.63	.68	.80	.83	.70	.61	1.00
Extrinsic	.60	.81	.83	.75	.45	.80	.88	.88	.75	.83	.87	1.00
BMHSS-R Total Score	.72	.89	.91	.79	.57	.76	.87	.88	.75	.74	.97	.96	1.00

r = .30 medium, r = .50 large

Note. HS = Help-seeking, S = Stigma, KF = Knowledge and Fear of Psychotherapy, FT = Finding a Therapist, BDN = Belief that Depression is Normal, IP = Insurance/Payment, A = Ageism, CTQ = Confidence in Therapist’s Qualifications, PR = Physician Referral, T = Transportation, I = Intrinsic, E = Extrinsic, and BTS = BMHSS-R Total Score

BMHSS and BMHSS-R Item Structure

The original BMHSS had a range of subscale items of 9: some subscales were comprised of 3 items while others had 12 (see Table 4). The BMHSS-R had a much narrower range of subscale items, 2, with number of items per subscale ranging from 3 to 5. Six of the subscales on the original BMHSS had internal consistency estimates above .70 whereas eight of the subscales’ internal consistency estimates on the BMHSS-R reached this cut-off. Although the subscale of help seeking’s internal consistency estimates reached the cut-off on the original BMHSS, it failed to meet this standard on the BMHSS-R.

Table 4.

Number of Items per Subscale and Internal Consistency Estimates for the Original and Revised BMHSS-R

Original BMHSS (Pepin et al., 2009)		BMHSS-R
Subscale/Number of Items	alpha	Subscale/Number of Items	alpha
Help-seeking (12)	.75*	Help-seeking (4)	.67
Stigma (5)	.73*	Stigma (5)	.87*
Knowledge and Fear of Psychotherapy (7)	.74*	Knowledge and Fear of Psychotherapy (5)	.86*
Belief about Inability to Find a Psychotherapist (3)	.69	Belief about Inability to Find a Psychotherapist (5)	.86*
Belief that Depressive Symptoms are Normal (3)	.48	Belief that Depressive Symptoms are Normal (4)	.75*
Insurance/Payment Concerns (5)	.78*	Insurance/Payment Concerns (3)	.80*
Ageism (4)	.73*	Ageism (5)	.82*
Concerns about Psychotherapist’s Qualifications (6)	.69	Concerns about Psychotherapist’s Qualifications (5)	.81*
Physician Referral (3)	.61	Physician Referral (3)	.63
Transportation Concerns (3)	.73*	Transportation Concerns (5)	.77*
Range of items	9 (3–12)	Range of items	2 (3–5)
Mean (SD) number of items	5.10 (2.01)	Mean (SD) number of items	4.4 (.84)

Note. alpha = Cronbach’s Alpha;

^*indicates Cronbach’s alpha above .70

Following revisions and psychometric analyses the following subscales are comprised of five items: stigma; knowledge and fear of psychotherapy; belief about inability to find a psychotherapist; ageism; concerns about psychotherapist’s qualifications; and transportation concerns. The following subscales are comprised of four items: help-seeking attitudes and belief that depressive symptoms are normal. The following subscales are comprised of three items: insurance and payment concerns and physician referral.

Convergent Validity

Convergent validity was examined for the BMHSS-R subscales of stigma and help-seeking. Most subscales of the BMHSS-R, such as concerns about transportation and concerns about insurance and payment, lack comparable measures that can be used to establish convergent validity. We focused on the subscales of stigma and help-seeking due to the availability of comparable measures (i.e., the BMI and WSHQ). The BMHSS-R subscale of stigma is most similar to the BMI as both measure stigma attitudes. The BMHSS-R subscale of help-seeking is most similar to the WSHQ as both measure help-seeking attitudes. As can be seen in Table 5, the measures were correlated significantly in the expected directions, although magnitudes of correlations were modest. The BMHSS-R subscale of stigma was significantly and positively correlated with the total BMI, r(98) = .25 p = .01 indicating a relationship between the scales of small strength (Pallant, 2003). Similarly, the BMHSS-R subscale of stigma was significantly and positively correlated with all of the BMI subscales: dangerousness subscale, r(98) = .21, p = .04; poor social and interpersonal skills, r(98) = .24 p = .02, and incurability, r(98) = .23, p = .02. The BMHSS-R subscale of help seeking was significantly and negatively associated with the WSHQ total score, r(98) = −.31, p = .002 indicating a moderate relationship between the scales. The relationship is in the negative direction because higher scores indicate greater perception of barriers using the BMHSS-R whereas on the WSHQ higher scores indicate greater help seeking.

Table 5.

Subscale and Total Score Correlations for the BMHSS-R, BMI, and WSHQ

	r	p
BMHSS-R Stigma and BMI Dangerousness	.21	.04
BMHSS-R Stigma and BMI Poor Social and Interpersonal Skills	.24	.02
BMHSS-R Stigma and BMI Incurability	.23	.02
BMHSS-R Stigma and BMI Total Score	.25	.01
BMHSS-R Help-seeking and WSHQ	−.31	.002

Discussion

The purpose of this report was to present the procedure and results of revisions to the original BMHSS resulting in an improved self-report measure of barriers to seeking mental health services to provide a framework to determine the obstacles most threatening older adults’ mental health service use. We attempted to equalize the number of items across subscales, and we were generally successful. We designed and administered a version of the BMHSS-R that had 5 items across all subscales. However, following analyses of internal reliability and review of response rates, some items were removed. Although we were unable to equalize the number of items for all scales, we did improve the general equality of items across subscales by reducing the range of items across subscales from nine to two.

In addition to addressing the limitation of having an imbalance of items across subscales, revising the items corrected 3 of the 4 subscales that had reliability estimates below the acceptable level on the original version. Overall, the internal consistency values of most subscales in the revised version were adequate (.7 or above), which is an improvement on the original version. On the other hand, internal consistency estimates for scores on two of the BMHSS-R subscales (i.e., help-seeking and physician referral) were low even after revisions. The low reliability of the help-seeking subscale scores was initially surprising given its adequate reliability in the first version of the BMHSS (.75). However, the deletion of eight scale items and the addition of a different type of item, that of religious help-seeking, make the revised help-seeking scale different from the original, perhaps contributing to lower subscale reliability estimates. Considering the original reliability estimates of the help-seeking subscale, which were adequate, it is a valuable subscale but may benefit from continued development.

The subscale of physician referral, or more generally the linkage between, or integration of, medical and mental health care, appears more complicated to develop and evaluate than other subscales. To our knowledge, this barrier has not been assessed in survey form in previous research and this subscale has low reliability estimates in both the original and revised versions of the BMHSS. The barrier of physician referral or poor integration of general medical and mental health services is consistently identified as a major impediment to service use (Bogner, de Vries, Maulik, & Unützer, 2009; Cairney, Corna, & Streiner, 2010; Harman, Veazie, & Lyness, 2005; Jeste et al. 1999; Kovess-Masféty et al., 2007; Mackenzie, Gekoski, & Knox, 2006). However, it has been difficult for us to effectively characterize this construct in a scale on our measure. When using the BMHSS-R to identify barriers within systems, it is likely that it will be necessary to augment the BMHSS-R results with medical record data, system referral patterns, and other more open-ended inquiries into perceived barriers whenever feasible.

Convergent validity was acceptable, but not as robust as expected. Although both BMHSS-R subscales analyzed for convergent validity were significantly correlated with comparable measures in the expected direction, the magnitudes of the correlations were generally modest. Although the BMI and WSHQ have been used in older adult samples before (Segal et al., 2005), and had good to excellent internal consistency values in our sample, both of these measures were developed and validated on younger samples, which may contribute to the lower than expected convergent validity values in our older adult sample. Perhaps more substantively, meaningful differences exist between the measures. For example, the BMI measures multiple facets of beliefs about mental illness and some facets are more closely related to the BMHSS-R stigma subscale than others. Additionally, the BMI asks respondents to rate their general impressions of people with psychiatric illnesses, whereas the BMHSS-R asks respondents to rate the degree to which a barrier (i.e., stigma) influences their own mental health service use. These factors may have led to the modest correlations. Similarly, the WSHQ examines a range of help-seeking attitudes and is not limited to attitudes about seeking services. It is likely that items unrelated to seeking services negatively impacted the strength of the relationships. The presence of significant correlations in the expected directions are promising, however, continued consideration of appropriate comparisons is necessary to further demonstrate convergent validity.

We developed the BMHSS-R to include 10 subscales that represent distinct barriers presented in previous research. We conducted correlation analysis to examine the relationship of BMHSS-R subscales with one another to guide our interpretation of the measure’s reliability and validity. As expected, the subscales of the BMHSS-R were strongly positively correlated with each other, suggesting that barriers to seeking mental health services are related and should be viewed concurrently. Despite significant relationships, in most cases subscales appeared to measure unique constructs. However, four of the correlations between subscales (e.g., stigma and knowledge and fear of psychotherapy) were very strong (r >.80) suggesting that these concepts are not well distinguished. Cronbach’s alpha and item-total correlations are not sufficient evidence of unidimensionality. Although we recommend scoring and interpreting the BMHSS-R based on the 10-barrier structure we developed based on theory, users should expect that some subscale scores may be very similar to one another (e.g., stigma and knowledge and fear of psychotherapy).

The current study had several limitations. Many the participants were highly educated, healthy, and active older adults who transported themselves to the research facility. Therefore, our findings may not generalize to more physically frail and less independent older adults. As noted previously, the participant sample was recruited through a database of older adults who had indicated their interest in participating in research studies. Older adults who volunteer to participate in research studies may not represent the broader older adult population, and they may in fact have a greater openness to psychotherapy than those who are not interested in psychological research. Overall, the sample was relatively homogenous with respect to ethnic diversity, and the sample also was not fully representative of the older adult population in the United States with regard to gender (with a higher percentage of women in the current sample). Future, larger studies should strive to more recruit samples that more closely reflect the demographics of the aging population in the US. Another limitation of the current study is that the sample was not large enough to conduct a Factor Analysis to examine the BMHSS-R’s internal structure and to determine the extent to which items load on their intended factor. Additionally, we selected a broad, community dwelling population as our study sample; therefore, it is possible that many of the older adults who participated in our study had not necessarily experienced significant mental health concerns.

For barriers to mental health services to operate as an obstacle for someone, he/she must experience symptoms associated with a mental health condition, must perceive a need for services and, finally, not seek services. Although a sample of older adults who were engaged in mental health services would have ensured the participants actually had mental health concerns, it would have made it impossible to capture the perspectives of older adults who are not engaged in mental health services because of barriers preventing them to do so. If possible, when using the BMHSS-R to identify barriers within systems or subpopulations, it would be useful to assess symptoms associated with a mental health condition as well as perceived need for services in addition to barriers to service use.

Future research on barriers to mental health services should specifically target those older adults who have a need for mental health services but do not use them, which can be approached in a number of ways. Community dwelling older adults may be recruited, provided the sample is large enough. Alternatively, a strategy could be to screen potential study participants for symptoms of mental health and to assess their engagement in mental health services. Another option would be to conduct retrospective studies with older adults who were admitted to acute inpatient psychiatric services but failed to seek outpatient services before an admission to explore the barriers that prevented them from engaging in services before admission.

The results of this study support the use of the BMHSS-R to identify barriers within a specific population or service delivery system. However, we recommend augmenting the BMHSS-R with other sources of data including psychiatric symptoms, service use data, and referral patterns when feasible to develop a comprehensive profile of barriers to seeking mental health services. Results also support the continued testing of the BMHSS-R. Factor analysis should be conducted to examine the underlying structure of the instrument and to evaluate unidimensionality. Researchers should also explore the test-retest reliability of the BMHSS-R. In this vein, we would anticipate that the scales would be relatively stable over time, but not immutable to change especially with targeted interventions. Other types of validity should also be measured, such as predictive validity, in future studies. Additionally, the BMHSS-R could be applied to other subgroups that underutilize mental heath services such as ethnic minorities or individuals with serious mental illness (SMI). The measure may require further modification to ensure that it captures barriers prevalent in these groups, such as the extent to which symptoms interfere with mental health service use in the case of those with SMI. Finally, we wish to emphasize that the current results apply most directly to the current cohort of older adults. Sadly, we know that many adults in the Baby Boomer generation, who are just now approaching later life, are bringing with them higher rates or mental health disorders but also more familiarity and experience with the mental health systems (Segal, Qualls & Smyer, 2011). As such, it is likely that some barriers to services will evolve over time, and having a sound and comprehensive measurement instrument is a prerequisite for these important types of longitudinal studies.

Despite the limitations, our findings contribute to mental health policy and clinical practice by providing a framework for the objective examination of diverse barriers to mental health services. Results from the subscale correlation analyses indicated many barriers to seeking mental health services are highly related and supported the need to examine multiple barriers concurrently. Additionally, the promising results of the reliability analyses suggest the BMHSS-R may be used along with other information (e.g., service utilization data) to develop a profile of barriers, underscoring the barriers that are higher and those that are lower in order to identify points where changes can be made to improve a delivery system and decrease barriers.