Impact of Skill-Based Approaches in Reducing Stigma in Primary Care Physicians: Results from a Double-Blind, Parallel-Cluster, Randomized Controlled Trial

Abstract

Objective:

Most interventions to reduce stigma in health professionals emphasize education and social contact–based strategies. We sought to evaluate a novel skill-based approach: the British Columbia Adult Mental Health Practice Support Program. We sought to determine the program’s impact on primary care providers’ stigma and their perceived confidence and comfort in providing care for mentally ill patients. We hypothesized that enhanced skills and increased comfort and confidence on the part of practitioners would lead to diminished social distance and stigmatization. Subsequently, we explored the program’s impact on clinical outcomes and health care costs. These outcomes are reported separately, with reference to this article.

Methods:

In a double-blind, cluster randomized controlled trial, 111 primary care physicians were assigned to intervention or control groups. A validated stigma assessment tool, the Opening Minds Scale for Health Care Providers (OMS-HC), was administered to both groups before and after training. Confidence and comfort were assessed using scales constructed from ad hoc items.

Results:

In the primary analysis, no significant differences in stigma were found. However, a subscale assessing social distance showed significant improvement in the intervention group after adjustment for a variable (practice size) that was unequally distributed in the randomization. Significant increases in confidence and comfort in managing mental illness were observed among intervention group physicians. A positive correlation was found between increased levels of confidence/comfort and improvements in overall stigma, especially in men.

Conclusions:

This study provides some preliminary evidence of a positive impact on health care professionals’ stigma through a skill-building approach to management of mild to moderate depression and anxiety in primary care. The intervention can be used as a primary vehicle for enhancing comfort and skills in health care providers and, ultimately, reducing an important dimension of stigma: preference for social distance.

A substantial portion of the global burden of disease is attributable to mental illness, yet funding for mental health care services is disproportionately low.^1–3 An international consensus holds that redirecting funding toward the integration of mental health care into primary care settings would allow for more cost-effective interventions, reduced wait times, improved continuity, and increased satisfaction with care.^1,4 Although this is a realistic proposition in theory, many primary care providers do not feel confident dealing with mental health concerns.⁵ This lack of confidence is often coupled with systemic issues (i.e., lack of time, inefficient or lack of reimbursement, and inadequate access to specialty services), all of which can be a barrier to better addressing mental health concerns.^5–10 Social distance is an important conceptual component of stigmatization, and it has been argued that when providers feel inadequately equipped in terms of knowing “what to say” or “what to do,” this contributes to stigmatization through greater social and clinical distance, especially when accompanied by negative attitudes.^5,11,12 Historically, most antistigma interventions have targeted negative attitudes. Although there is a dearth of evidence, intuition suggests that attitude-based interventions have not been confirmed to affect behavior because of too much social distance, poor communication, and a lack of comfort and confidence in dealing with these problems. The literature does suggest a clear, inverse relationship between providers’ stigma and patients’ willingness to seek help.^13–17 Also, there is an important spillover effect on the provision of care for physical ailments: It has been suggested that diagnostic overshadowing and pessimistic views about adherence to treatment often lead to an increased reluctance to refill prescriptions or initiate investigations and, ultimately, result in excess morbidity and mortality due to physical illness.^9,18–23

The British Columbia Adult Mental Health module of the Practice Support Program, designed by the General Practice Services Committee, has been developed as a contact-based education approach to enhance skills in primary health care providers, increasing levels of comfort and confidence in treating patients with mild to moderate depression and anxiety (although many of its principles are more broadly applicable). Taken together, these elements have proven to be a promising approach for combating stigma.^5,6,24–26 The program consists of three 3.5-hour interactive workshop-style sessions, each session interspersed with two 6- to 8-week implementation action periods providing learners the ability to practice what they have learned.¹¹ Social contact is encouraged during training sessions by people with lived experience of mental illness who share stories about their experiences and recovery. Practice support consists of 3 evidence-based supported self-management tools: the Cognitive Behavioral Interpersonal Skills workbook, which incorporates a comprehensive mental illness diagnostic assessment interview tool; the Canadian Mental Health Association Bounce Back program, a telephone-guided, evidence-based, cognitive behavioral therapy–based service for mild to moderate depression, with or without anxiety; and the Antidepressant Skills Workbook for patients with mild and moderate depression. Throughout the action periods, a practice support coordinator is readily accessible. The coordinator plays an integral role, offering on-site support during action periods, guidance on office redesign, and strategies for enhanced implementation of various program tools, skills, and processes. This individual serves as an important link between participants, facilitators, and contributors, encouraging the sharing of experiences, challenges, recommendations, and solutions.

The program has been identified by the antistigma initiative of the Mental Health Commission of Canada, “Opening Minds,” as a promising program to reduce stigma in primary care providers, one of the main groups targeted.⁵ While previous evaluations lend support to this theory, prior evidence is derived from nonrandomized studies, leaving its efficacy uncertain.^11,27

In 2013, as part of the province’s mental health and addictions strategy, “Together We Can,” the Nova Scotia Mental Health and Addiction Services branch of the Department of Health and Wellness, along with key stakeholders, supported a multicenter, double-blind, 2-parallel group, cluster randomized trial in order to rigorously evaluate the program’s effectiveness on Nova Scotia primary care providers’ stigma, perceived comfort level, and knowledge in providing care for mentally ill patients; patient clinical outcomes; and health economics. This article focuses specifically on the program’s impact on primary care physicians’ comfort level and skills in providing care and how these results affect providers’ stigma.

This study was approved by the Nova Scotia Multisite Research Ethics Board and the University of Calgary Research Ethics Board.

We hypothesized that enhanced skills and increased comfort and confidence on the part of practitioners who participated in the program would lead to lower stigma scores compared with practitioners who did not participate in the program. Intuitively, the largest impact is expected to be in the stigma dimension of social distance, but our a priori hypothesis was that a reduction in overall stigma would be seen. Our secondary hypothesis was that participation in the program would lead to greater confidence and comfort treating patients with mental illnesses, compared with nonparticipation in the program. An exploratory objective was to examine whether increases in physician confidence were associated with reductions in stigma.

Methods

Study Design and Participants

The study was a multicenter, double-blind, 2-parallel group, cluster randomized trial. Participants were identified by the Canadian College of Family Physicians Nova Scotia Chapter, Doctors Nova Scotia, or through an expression of interest following promotional letters, recruitment letters, telephone calls from the research team, or presentations at key primary care conferences. For reasons of feasibility, recruitment was restricted to family physicians practicing within a 2-hour radius of the Halifax Regional Municipality.

Procedures

We randomized the practices rather than individual physicians since the intervention might lead to changes in the practice patterns of physicians within practices. Participants were assigned a practice and a unique identifier number. A master file linking participant names and ID numbers was kept by the principal investigator in a secure location. Study databases contained deidentified information only. The research coordinator (data collector) and outcome assessors (independent researchers) were blinded to group assignment.

Randomization was stratified on the total number of physicians per practice, as well as urban or rural setting, to ensure equal distribution of practice clusters and urban and rural groups. STATA, version 12 (College Station, Texas) was used to generate the sequence for practice (cluster) randomization. Random numbers were generated from a binomial distribution with a probability of success of 0.5. The randomization sequence was assigned centrally by one of the investigators (S.P.) who was not involved in trial operations and had no interaction with the randomized practices.

In February 2014, the intervention group began participation in the program (lasting 15 weeks). The control group received the same training at study end, after patient outcome evaluation was completed. The program was delivered by a trained team from British Columbia, which included first voice advocates from the Nova Scotia Certified Peer Specialist Program.²⁸ Practices also received on-site support through a practice support coordinator as described above. Although our study focused on evaluating the impact of the program on family physicians, office staff members were also invited to attend the training and the Mental Health First Aid Program²⁹ in order to support providers in caring for patients with mental health concerns.

Outcomes Measures

All questionnaire were completed by both groups at comparable times prior to the intervention group’s first training session (February 2014) and after the end of the last training session (June 2014) (pre- and posttraining in the intervention group).

The Opening Minds Scale for Health Care Providers (OMS-HC) was used to measure stigma levels among participants. The OMS-HC is a 15-item validated scale that captures 3 main dimensions of stigma: negative attitudes, health professionals’ own willingness to disclose or seek help for a mental illness, and preference for greater social distance.³⁰ Items are rated on a 5-point scale from strongly agree to strongly disagree. Total scores can range from 15 to 75, with lower scores indicating less stigma. This scale has been widely validated and used in evaluations of antistigma interventions in Canada. In 2014, a rigorous examination of the psychometric properties and responsiveness of the OMS-HC was conducted by Modgill et al.³⁰ Results from this evaluation showed internal consistency as acceptable for all versions of the OMS (α = 0.74-0.79) and corresponding subscales (α = 0.67-0.68), further showing the OMS-HC to be an accurate and reliable instrument.³⁰

Modified versions of 3 ad hoc scales developed for a nonrandomized evaluation study in British Columbia were used to collect demographic information. The scales also were used to assess levels of confidence on a number of statements pertaining to the participants’ ability to diagnose, treat, and otherwise manage patients’ mental health concerns and were used to measure levels of familiarity, confidence, and comfort with program-specific and non-program-specific tools and skills. For these measures, decreases in score represent increases in confidence. The psychometric properties of these scales have not been assessed.³¹

Statistical Analysis

Our calculations indicated that 50 physicians would be required in each group. Under this assumption, 80% power would be achieved to detect a between-group difference in mean scores of 3 points on the OMS-HC with an alpha value of 5%. Clinically meaningful change was defined as a change of 3 points, because this is slightly better than what is usually seen in evaluations of brief interventions that have used the OMS-HC.

The preplanned primary analysis was to compare pre- minus postintervention OMS-HC scores by using a t test accounting for the clustering using STATA’s “svy” commands. However, given the limitation of this method in managing missing data and the occurrence of greater than expected attrition, the analysis was conducted using a multilevel mixed model in which physicians were clustered within practices and stigma ratings were clustered within physicians (1 or 2 observations per physician). The effect of the intervention was measured in this analysis as an intervention by time interaction. Differences in confidence and comfort were also analyzed with a multilevel mixed model. The question of whether increasing physician confidence is associated with reductions in stigma was examined through use of the slope of a regression line fit by using generalized estimating equations with an exchangeable correlation structure. A Spearman’s correlation coefficient was also calculated. The analysis assessed changes in confidence in the management of mental illness with changes in stigma ratings on the total OMS-HC as well as the 3 subscales.

Results

Completions and Response Rates

Seventy-seven practices with 111 community-based family physicians were recruited and randomized into intervention and control groups. As shown in Figure 1, 72 physicians (n = 38 intervention, n = 34 control) completed the full OMS-HC at both time points and could therefore be included in the primary analysis. A total of 101 participants (n = 51 intervention, n = 50 control) had data at 1 or more time points.

Figure 1.

CONSORT flow diagram: allocations, attrition. One participant completed both surveys but did not complete the OMS-HC. As such, paired analysis of OMS-HC stigma scores are based on n = 38 for the intervention group.

Sample Characteristics

As shown in Table 1, only 1 difference was observed between intervention and control groups: Intervention group participants were more likely to indicate working in small group practices. This occurred despite stratification for practice size due to differences in the size of practices within the assignment strata: A large practice stratum included one particularly large practice, which by chance was assigned to the control condition.

Table 1.

Physician sample characteristics: intervention group, control group, and overall.^a

	Intervention group (n = 39)b	Control group (n = 34)	Total	p c
Gender
Male	18 (46.2%)	13 (38.2%)	31 (42.5%)	0.64
Female	21 (53.8%)	21 (61.8%)	42 (57.5%)
Age group
20-29 years	—	2 (5.9%)	2 (2.7%)
30-39 years	8 (20.5%)	5 (14.7%)	13 (17.8%)	0.33
40-49 years	10 (25.6%)	14 (41.2%	24 (32.9%)
50-59 years	16 (41.0%)	10 (29.4%)	26 (35.6%)
60-69 years	5 (12.8%)	3 (8.8%)	8 (11.0%)
Years in practice
<1 year	—	2 (5.9%)	2 (2.8%)
1-4 years	7 (18.4%)	2 (5.9%)	9 (12.5%)
5-9 years	3 (7.9%)	3 (8.8%)	6 (8.6%)
10-14 years	3 (7.9%)	5 (14.7%)	8 (11.1%)
15-19 years	1 (2.6%)	6 (17.6%)	7 (9.7%)	0.09
20-24 years	9 (23.7%)	3 (8.8%)	12 (16.7%)
25-29 years	9 (26.3%)	7 (20.6%)	16 (23.6%)
≥30 years	5 (13.2%)	6 (17.6%)	11 (15.3%)
Pattern of work
Full time	34 (85.3%)	29 (85.3%)	63 (86.3%)	1.00
Part time	5 (12.8%)	5 (14.7%)	10 (13.7%)
Practice type
Solo practice	9 (23.7%)	5 (14.7%)	14 (19.4%)
Two-physician practice	8 (21.1%)	5 (14.7%)	13 (18.1%)	0.02
Small group practice (<5)	16 (42.1%)	8 (23.5%)	24 (33.3%)
Large group practice (≥5)	5 (13.2%)	16 (47.1%)	21 (29.2%)
Unique patients per year
<1000	6 (18.2%)	7 (21.9%)	13 (20.0%)
1000-1999	12 (36.4%)	12 (37.5%)	24 (36.9%)	0.19
2000-2999	2 (6.1%)	7 (21.9%)	9 (13.8%)
≥3000	11 (33.3%)	6 (18.8%)	17 (26.2%)
Other (hospitalist, locum, etc.)	2 (6.1%)	—	2 (3.1%)

^aValid percentage reported; nonresponses set to missing. As such, total for years in practice for intervention group = 37, total for practice type for intervention group = 38, and total for unique patients per year = 33 for the intervention the group and 32 for the control group. The sample characteristics reported in Table 1 are based on pre and post completers. However, the breakdown does not change when measured based on those who completed at least 1 data time point (n = 101).

^bOne participant completed both surveys but did not complete the OMS-HC. As such, demographic reporting is based on n = 39, but paired analysis of stigma scores is based on n = 38.

^cDifference between the intervention and control groups. Fisher’s exact test was used.

OMS-HC

For the intervention group, Cronbach’s alphas for the 15-item total scale were 0.70 at pretest and 0.87 at posttest, indicating an acceptable level of internal consistency at both time points. For the control group, Cronbach’s alphas were 0.52 at pretest and 0.82 at posttest. Alphas for the 6-item Attitudes subscale were 0.53 at pretest and 0.49 at posttest for the intervention group and 0.44 at pretest and 0.70 at posttest for the control group. For the 4-item Disclosure/Help-seeking subscale, alphas were 0.51 at pretest and 0.58 at posttest for the intervention group and 0.51 at pretest and 0.42 at posttest for the control group. Alphas for the 5-item Social Distance subscale were 0.57 at pretest and 0.70 at posttest for the intervention group and 0.61 at pretest and 0.78 at posttest for the control group.

In the primary analysis, a mixed model not adjusting for practice size found that changes in OMS-HC total and subscale scores were not significant at the 95% confidence level (total scale, t ₇₀ = –1.48, p = 0.15; negative attitudes, t ₇₀ = –0.37, p = 0.71; willingness to disclose/seek help, t ₇₀ = –0.32, p = 0.75; preference for social distance, t ₇₀ = –1.77, p = 0.08).

With adjustment for practice size, there was a significant difference between groups in the social distance score only (95% CI) (total scale, t ₇₀ = –1.91, p = 0.06; negative attitudes, t ₇₀ = –0.68, p = 0.50; willingness to disclose/seek help, t ₇₀ = –0.36, p = 0.72; preference for social distance, t ₇₀ = –2.17, p = 0.03). The independent samples effect size calculation (Cohen’s d) for the overall scale was 0.45, which is considered a weak to moderate effect. Table 2 displays the results of the multilevel mixed-model analysis, adjusted for practice size. In this analysis, the effect on overall stigma remained nonsignificant, but that for social distance remained significant (p = 0.03).

Table 2.

Multilevel mixed model: intervention effect as time by intervention interaction, adjusted for practice size^a.

OMS-HC	Coefficient	z	p
Total scale
Intervention	1.6553	1.23	0.217
Time	–0.3509	–0.44	0.662
Interaction	–2.1115	–1.90	0.058
Large practice indicator	–1.3068	–0.96	0.335
Model constant	31.9086
Negative attitudes
Intervention	0.1305	0.18	0.853
Time	–0.6438	–1.62	0.105
Interaction	–0.5003	–0.91	0.362
Large practice indicator	–1.1869	–1.78	0.076
Model constant	12.5371
Willingness to disclose/seek help
Intervention	0.3756	0.64	0.522
Time	0.0180	0.05	0.958
Interaction	–0.5026	–1.06	0.287
Large practice indicator	0.1122	0.18	0.855
Model constant	10.5480
Preference for social distance
Intervention	1.0067	1.95	0.051
Time	0.3320	0.85	0.395
Interaction	–1.1892	–2.20	0.028
Large practice indicator	–0.4361	–0.89	0.371
Model constant	8.9531

^aA total of 101 participants (n = 51 intervention, n = 50 control) had data at 1 or more time points.

Measures of Confidence, Comfort, and Familiarity

For the first confidence measure, Cronbach’s alphas were 0.84 at pretest and 0.87 at posttest. For the second and third confidence measures, alphas were 0.90 at pretest, 0.91 at posttest, and 0.98 at pretest, 0.98 at posttest, respectively. As shown in Table 3, the program effects on all 3 measures of physician confidence were strongly significant. Table 4 displays pretest and posttest mean scores and standard deviations for the OMS-HC and global confidence measures for the intervention and control groups.

Table 3.

Multilevel mixed model: intervention effect (physician confidence measures) as time by intervention interaction, adjusted for practice size.

Physician confidence	Coefficient	z	p
Confidence in the management of mental illness
Intervention	2.2361	2.45	0.014
Time	0.9530	1.72	0.086
Interaction	–4.6378	–6.07	>0.001
Large practice indicator	2.2277	2.43	0.015
Model constant	19.2340
Confidence, comfort, and familiarity using tools not specifically developed for the program
Intervention	2.9384	1.53	0.125
Time	1.0000	0.82	0.411
Interaction	–10.4474	–6.24	>0.001
Large practice indicator	4.9827	2.64	0.008
Model constant	38.5375
Confidence, comfort, and familiarity using tools specifically developed for the program
Intervention	1.8101	0.72	0.471
Time	–5.3824	–2.93	0.003
Interaction	–28.4334	–11.25	>0.001
Large practice indicator	3.9514	1.66	0.097
Model constant	90.6699

Table 4.

Pretest and posttest mean scores and standard deviations for the OMS-HC and global confidence measures for the intervention and control groups.

	Intervention group (n = 38)		Control group (n = 34)
	Mean	SD	Mean	SD
OMS-HCa
Total scale
Pretest score	2.21	0.40	2.11	0.35
Posttest score	2.07	0.46	2.07	0.38
Attitudes subscale
Pretest score	2.13	0.48	2.02	0.48
Posttest score	1.97	0.49	1.90	0.51
Disclosure/help-seeking
Pretest score	2.65	0.66	2.63	0.61
Posttest score	2.57	0.75	2.65	0.52
Social distance
Pretest score	1.95	0.46	1.77	0.39
Posttest score	1.79	0.55	1.82	0.48
Global confidence measures
Confidence in the management of mental illnessb
Pretest score	2.18	0.39	2.03	0.37
Posttest score	1.80	0.36	2.12	0.40
Confidence, comfort, and familiarity using tools not specifically developed for the programc
Pretest score	2.30	0.39	2.32	0.47
Posttest score	1.88	0.41	2.38	0.47
Confidence, comfort, and familiarity using tools specifically developed for the programc
Pretest score	2.86	0.34	2.87	0.21
Posttest score	1.96	0.53	2.92	0.18

^aMean scores can range from 1 to 5, with lower scores indicating less stigma.

^bMean scores can range from 1 to 4, with lower scores indicating higher confidence.

^cMean scores can range from 1 to 3, with lower scores indicating higher confidence.

Correlation between Changes in Physician Confidence and Stigma Scores

There was a weak but statistically significant association between improvements in confidence in the management of mental illness and reductions in stigma (beta coefficient for change in confidence = 2.90; z = 2.14; p = 0.03, and Spearman’s correlation coefficient between increases in confidence and decreases in stigma = 0.284). This relationship is depicted in Figure 2.

Figure 2.

Scatterplot of relationship between improvements in confidence and decreases in stigma. The red line is the regression line. Decreases in scores indicate increases in confidence. Decreases in scores indicate decreases in stigma.

No interaction effect was observed for baseline sigma scores (p = 0.12) in this analysis, but an interaction effect was observed for gender (p = 0.022), with the relationship being stronger in men than in women.

Discussion

Knaak et al⁵ suggested that a provider’s sense of helplessness often contributes to feelings of anxiety and a preference for avoidance that is likely to translate into stigmatization in primary mental health care. Our study provides some support for this hypothesis, emphasizing the potential importance of teaching practical skills to enhance provider confidence, also demonstrating an association between improvements in confidence and reduced stigma. While our primary analysis sought to examine the impact of this intervention on providers’ stigma overall, significant changes were observed only in relation to social distance. In retrospect, social distance and attitudes are components of stigma most closely linked to the type of intervention evaluated; however, we acknowledge that this was not a planned analysis and it therefore must be considered an exploratory result requiring replication. Also, the significant difference emerged only after adjustment for a variable that was unequally distributed in the randomization.

Another interesting finding is the occurrence of a gender by intervention interaction in the association of changes in confidence with stigma ratings. These results are consistent with existing evidence on gender and stigma, which suggest that attitudes toward mental illness differ between males and females.^32–36 This interaction suggests that there may be considerable value in exploring a “gender-specific” approach to developing antistigma initiatives. We view this as a hypothesis generated by our results. This finding also requires replication.

Limitations

As a result of lower than anticipated survey completion rates, the primary analysis likely lacked sufficient power to detect significant effects in total OMS-HC scores (primary outcome) across the 2 time points, contributing to the possibility of a Type II error for these primary analyses. Lack of time emerged as the primary reason for attrition. The overall assessment of the primary analysis was associated with a p value of 0.058. Additionally, Cronbach’s alphas for the control group at pretest fell below an acceptable range (0.52). Although the reason for this is unclear, a lower scale reliability rating at 1 time point has been occasionally observed in prior studies using the OMS-HC.³⁷ Regarding the low internal consistencies for 2 of the subscales, as proposed by Modgill et al,³⁸ it could be that the alphas are lower due to the smaller number of items on the subscales compared with the total scale. Other scales used in the study were ad hoc groupings of items that have been used in the evaluation of this intervention; the psychometric properties of these scales have not been assessed.

Conclusion

We believe that these results provide some preliminary evidence of a positive effect on stigma for a skill-building approach to management of mild to moderate depression and anxiety in primary care. Larger studies would be useful for confirming that this is the case and for further delineating possible gender-specific effects and underlying mechanisms. The program can be used as a primary vehicle for enhancing skills in primary health care providers, increasing levels of comfort and confidence, and ultimately reducing an important dimension of stigma: preference for social distance.