Abstract
Background
Depression and anxiety interfere with an individual's quality of life and result in substantial economic costs to the workplace through lost productivity. The internet offers an unparalleled opportunity for the large scale, cost-effective delivery of mental health awareness and destigmatisation programs in the workplace. However, to date high quality assessments of the effectiveness of such workplace programs have been confined to health professional settings. The current study reports the effectiveness of Mental Health Guru (MH-Guru), a two-module online mental health workplace induction program.
Methods
A total of 507 employees from a large multi-departmental government workplace were randomised to a 2-week online depression and anxiety educational program (Mental Health Guru; MH-Guru), or a Wait List Control condition. Participants completed online surveys at baseline, 1 week post-intervention and at 6-month follow-up. Primary outcome measures included depression and anxiety literacy, personal stigma about depression and anxiety, and help seeking intentions for anxiety and depression at post-test. Secondary measures included help-seeking attitudes and self-reported help seeking behaviour. Satisfaction measures were also collected.
Results
Dropout at post intervention was 26.7% and 21.0% for the MH-Guru and Control conditions respectively. Relative to Control, MH-Guru participants showed significantly greater improvements in depression and anxiety literacy at post intervention. Between group standardized effect sizes at post-test and 6-months were 0.78 and 0.81 for depression literacy and 0.80 and 0.79 for anxiety literacy. Compared to the Control participants, the MH-Guru group also showed significantly greater reductions in depression and anxiety personal stigma. Between group effect sizes in stigma for depression were − 0.56 and − 0.47 at post-test and 6-months respectively and − 0.42 at both time points for anxiety. The MH-Guru intervention was not associated with a differentially greater increase in anxiety or depression help-seeking intentions or improvement in help seeking attitudes compared to the Control group. However, self-reported help-seeking behaviour was significantly greater in the MH-Guru group at post-test. In addition, the MH-Guru group showed greater intentions to seek help for depression from the Internet at 6-month follow-up. Satisfaction items suggested that the program was acceptable to employees.
Conclusions
Brief online programs such as MH-Guru have the potential to play an important role in increasing mental health awareness and decreasing stigmatizing attitudes in the workplace, and by extension the general community.
Keywords: Internet, Work, Mental health, Intervention, Depression, Anxiety
Highlights
-
•
First RCT in the workplace to demonstrate the effectiveness of an online intervention for anxiety and depression in staff without health credentials.
-
•
The intervention (MH-Guru) was associated with reductions in stigma and improvements in literacy for depression and anxiety, sustained over 6 months.
-
•
Intentions to seek help from the Internet were greater at 6 months among those receiving the workplace intervention.
1. Introduction
Mental disorders are the leading cause of disability burden worldwide accounting for 23% of all years lived with disability (Whiteford et al., 2015). Globally the cost of mental illness in 2010 was $US 2.5 trillion, a figure which has been projected to increase to $US 6 trillion by 2030 (Bloom et al., 2011). A major component of these costs is attributable to lost work productivity due to absences from work and ‘presenteeism’. For example, lost work productivity and compensation claims due to mental illness have been estimated to account for $11 billion in Australia and $102 billion in the USA (Greenberg et al., 2015).
Currently mental illness is the subject of substantial stigma. For example, 20% of respondents in a national stigma survey in Australia reported that they would not employ a person with depression and a similar percentage indicated that they would not work closely with a person with depression (Griffiths et al., 2006). Stigma not only further compromises the mental health of people with a mental illness but also serves as a barrier to seeking help for mental ill-health (Barney et al., 2006). Poor mental health literacy (lack of knowledge about mental ill-health, its diagnosis, management, treatment and prevention (Jorm et al., 1997) also mitigates against help seeking (Gulliver et al., 2010).
In a recent meta-analysis we reported that online programs are as effective as face-to-face interventions in reducing stigma in the community (Griffiths et al., 2014). There is also evidence that online educational programs can improve mental health literacy (e.g., (Griffiths et al., 2004, Kiropoulos et al., 2011)) in the general community and there is increasing interest in the potential for online applications to promote help seeking for mental ill-health. Such programs are scalable in theory. However, the implementation of online programs among adults in the general community presents challenges given that there are few drivers to promote their uptake.
The workplace presents a unique opportunity to deliver public health programs which address stigma and mental health literacy and promote help seeking, for three reasons. First, a significant percentage of the adult population are members of the workforce. Secondly, since the employer pays the employee for their time, if the former determines that staff should undertake a work-place mental health awareness program it is likely that the employee will do so. Finally, the employer has strong financial motivation to engage with measures that might reduce the high economic cost of mental health problems in the workplace.
However, such programs, if they are to be implemented must be evidence-based. Surprisingly, to our knowledge, except for programs focused on educating health practitioners or health trainees (Rosen et al., 2002, Bayar et al., 2009, Irvine et al., 2012a, Irvine et al., 2012b) there have been no randomised controlled trials of the effectiveness of internet-based passive educational interventions for delivering improved mental health knowledge, attitudes and help seeking outcomes in the workplace.
Accordingly the current study sought to investigate the effectiveness of Mental Health Guru (MH-Guru), a brief online universal workplace educational program about depression and generalised anxiety. Based on the empirical evidence cited above, MH-Guru was designed to increase anxiety and depression literacy, decrease negative attitudes to these conditions, provide advice to supervisors and colleagues to assist co-workers with mental ill-health and promote help seeking.
2. Method
The study employed a two-armed randomised controlled trial design with outcome measurements at baseline, post-intervention and 6-month follow-up. The trial was approved by The Australian National University Human Research Ethics Committee (Protocol 2013/387) and registered with the Australian and New Zealand Clinical Trials registry (ANZCTRN 12613001083785).
2.1. Participant recruitment and setting
Participants were 507 employees from a large Australian multi-department government organisation in Australia comprising 20,921 employees. The organisation provided its written consent to participate in the trial with approval provided by the Executive of the organisation. Employee interest in the trial was generated through a recruitment strategy comprising emails, in-house newsletter articles, presentations and “top-down” management encouragement and support. All advertising materials contained a link to the study website which provided detailed information about the scope and nature of the trial. Employees who opted to proceed completed a series of online questions to indicate informed consent after which the employee was enrolled in the trial and referred to the online baseline assessment survey. Recruitment of individual employees took place over a 5-month period from July 2014 to November 2014.
All employees of the organisation who were aged at least 18 years were eligible to enrol in the trial. There were no other exclusion criteria.
2.2. The trial conditions
Participants were assigned either to the online Mental Health Guru intervention or a Wait-List control.
2.2.1. Mental Health Guru (MH-Guru)
The intervention program is an online psychoeducation workplace induction program scripted by the first author and developed by the Australian National University. MH-Guru comprises two modules, the first focused on depression and the second on generalised anxiety disorder. Each module takes approximately 30 min to complete and is comprised of information about the condition including its prevalence, symptoms, how to identify if a person is depressed, a symptom checker, treatments, risk factors, myth busting, advice to supervisors, advice to colleagues of a person with depression/anxiety and sources of help. The program is presented in a simple multi-media, interactive format containing graphics and in-program exercises. Video vignettes of consumers with lived experience of depression or anxiety reinforce the program content and were incorporated as a proxy form of contact since there is some evidence that contact may be an effective intervention for decreasing stigma (Cooper et al., 2003).
Participants were invited to complete one module of MH-Guru per week.
2.2.2. Wait-List control (Control)
During the trial period, participants in the control condition were asked to complete the study online surveys only. This group was free to access professional mental health care during the wait-list period. Control participants were offered the opportunity to complete the MH-Guru program after the 6-month trial period.
2.3. Procedure
Participants completed the baseline measure online in week 1 after which they were allocated to either the MH-Guru or the Wait List Control condition (see randomisation procedure below). Those allocated to the intervention condition were invited by automated e-mail to access the MH-Guru Depression module in Week 2. A further automated email was sent in Week 3 inviting intervention participants to access the MH-Guru generalised anxiety disorder module although participants were required to complete the depression module before they could access the anxiety module. In Week 4 participants in both the MH-Guru and Wait List control conditions were invited by automated email to access and complete the post-intervention assessment. Invitations to complete the 6-month follow-up assessments were also sent via automated email to participants in both conditions. All emails contained a link to the study portal where the participant was able to log on using their username and password to access the relevant program module or assessment. Participants who failed to access the post-intervention or follow-up assessment within one week of the initial request were sent a reminder email, followed if necessary by a further reminder email one week later.
Participants were advised that their employer had agreed that they could complete the surveys and intervention during work hours. However, they were also provided the option of completing the study at home, if they preferred. The program was delivered from a server at the Australian National University. All assessment data were collected online on the ANU server.
Participants could seek technical support from the ANU research team. The protocol required the trial manager to contact an in-house clinical psychologist should a participant contact them in distress. One participant eventually randomised to MH-Guru was followed up by the psychologist as part of this process.
2.4. Randomisation and allocation concealment
Participants were randomly allocated to either the MH-Guru or the Control condition using an automated computer generated random number assignment that employed a stratified block design procedure. Stratification variables were gender and management/general staff status.
2.5. Measures
2.5.1. Primary outcomes
The primary outcomes for this study were depression and anxiety literacy, stigma and help-seeking intentions 1 week after completion of the intervention (week 4)
2.5.1.1. Literacy
Depression and anxiety literacy were measured using (i) The Depression Literacy Scale (D-Lit) (Griffiths et al., 2004) a 22-item scale comprising statements to which the participant must respond true or false (e.g., “Most people with depression need to be hospitalised.” True/False); and (ii) The Generalised Anxiety Literacy scale (A-Lit) (Gulliver et al., 2013) a similar 22-item scale (e.g., “People with anxiety disorder often hear voices that are not there” True/False). The score for each measure comprises the total number of correct items (range 0 to 22 for each scale). In this study, the internal reliabilities of the D-Lit and A-Lit scales at baseline in the current study were 0.78 and 0.81 respectively. The 4-week test-retest reliabilities in the Control group for these scales were 0.61 and 0.60.
2.5.1.2. Personal stigma
Depression and anxiety stigma were measured using (i) The Depression Stigma Scale – Personal subscale (DSS-Personal) (Griffiths et al., 2004, Griffiths et al., 2008), a 9-item measure comprising statements rated on a 5-point Likert scale from 0 (strongly agree) to 4 (strongly disagree). The subscale measures the respondent's personal attitudes to stigma (e.g., “People with depression should snap out of it”). Scores range from 0 to 36 with high scores corresponding to greater personal stigma (ii) The Generalised Anxiety Stigma scale – Personal subscale (GASS-Personal) (Griffiths et al., 2011), a similar 10-item scale focused on the respondents attitudes to generalised anxiety disorder. Higher scores signify greater generalised anxiety stigma. The internal reliabilities of the DSS and GASS scales at baseline in the current study were 0.81 and 0.91 respectively. The 4-week test-retest reliabilities in the Control group for these scales were 0.67 and 0.65.
2.5.1.3. Help seeking intentions
Separate versions of the Generalised Help-Seeking Questionnaire were used to measure help seeking intentions for depression and anxiety. The internal reliabilities of the GHSQ at baseline in the current study were 0.78 (depression) and 0.79 (anxiety). The 4-week test-retest reliability of the GHSQ anxiety and depression in the Control group in the current study were 0.75 and 0.77 respectively.
2.5.2. Secondary outcomes
Secondary outcomes included literacy, stigma and help-seeking intentions at 6-months together with help-seeking attitudes and help-seeking behaviour as follows:
2.5.2.1. Help seeking attitudes
The Attitudes Towards Seeking Professional Psychological Help Scale-Short Form (ATSPPH-SF) (Elhai et al., 2008) was used to measure attitudes to seeking psychological help. The internal reliability of the ATSPPH-SF at baseline was 0.83 and the 4-week test-retest reliability was 0.82.
2.5.2.2. Help seeking behaviour
Self-reported help seeking was measured at baseline and at post-test using the question: “In the last two weeks, have you sought information or treatment for depression or anxiety”. It had been intended that help seeking behaviour for the prior 6-month period would be ascertained at the 6-month follow-up. However, the item was incorrectly included with a 2-week rather than a 6-month time frame; accordingly, the 6-month data was not analysed.
2.5.2.3. Other
Other data including symptom severity and perceived workplace stigma were collected but are not the focus of the current paper.
2.5.3. Satisfaction, acceptability and usability feedback
Satisfaction, acceptability and usability feedback were collected from participants in the MH-Guru condition at post-intervention. For ease of reporting some categories were collapsed (e.g., ‘very easy’ and ‘easy’ were combined as were ‘difficult’ and ‘very difficult’–see Table 5).
Table 5.
Satisfaction and reported helpfulness of MH-Guru at post-test.
| Item | Response | |||
|---|---|---|---|---|
| How useful was the information in MH-Guru | Very useful/useful 92.0% | Not very 6.8% |
Not 1.1% |
|
| How easy was the MH-Guru material to understand | Very easy/ easy 89.2% |
Neither easy nor difficult 10.2% |
Difficult/very difficult 0.6% |
|
| How much did you feel you could trust the information | Very/mostly 89.8% |
Somewhat/slightly 6.9% |
Not at all 0% |
N/A 3.4% |
| Yes | Not sure | No | ||
| Do you think the information in MH-Guru will be useful to you in the future | 75.6% | 19.9% | 4.5% | |
| Have you done anything differently because of MH-Guru | 27.8% | 19.9% | 52.3% | |
| Would you recommend other workplaces make MH-Guru available to their employees | 83% | 13.6% | 3.4% | |
| Very/somewhat confident | Not very confident/somewhat unsure/ | Very unsure/not know what to do | ||
| How confident would you feel in helping someone with depression | 87.0% | 12.5% | 0.6% | |
| How confident would you feel in helping someone with an anxiety disorder | 80.7% | 18.2% | 1.2% | |
| Very/mostly | Somewhat/slightly | Not at all | N/A | |
| How easy was MH-Guru to use | 89.2% | 8.0% | 0% | 2.0% |
| How convenient was MH-Guru to use | 86.4% | 10.8% | 0% | 2.8% |
| How much did MH-Guru keep your interest and attention | 73.9% | 21.0% | 2.3% | 2.8% |
| How much did you like the MH-Guru program | 72.7% | 21.6% | 3.4% | 2.3% |
| How much did you like the way the MH-Guru program looked | 80.1% | 16.5% | 1.1% | 2.3% |
| How worried were you about your privacy in using the MH-Guru program | 9.6% | 39.8% | 47.7% | 2.8% |
| How satisfied were you with the MH-Guru program | 77.9% | 18.7% | 0.6% | 2.8% |
| How good a fit was the MH-Guru program for you | 62.5% | 30.7% | 3.4% | 3.4% |
| How good a method was the internet for delivering this information | 84.1% | 12.5% | 0.6% | 2.8% |
| MH-Guru helped me learn about | Strongly agree/agree | Neither agree nor disagree | Disagree/strongly disagree | |
| Depression and how to recognise it | 79.5% | 17.6% | 2.8% | |
| Anxiety disorders and how to recognise them | 76.7% | 18.8% | 4.5% | |
| Treatments for depression | 79.0% | 17.0% | 3.9% | |
| Treatments for anxiety disorders | 76.2% | 18.2% | 5.7% | |
| How to seek help for depression or anxiety disorders | 80.7% | 14.8% | 4.5% | |
| Myths relating to depression and anxiety disorders | 82.4% | 13.6% | 3.9% | |
| How to assist someone with depression/anxiety | 78.4% | 17.6% | 3.9% | |
2.5.4. Contamination
Since participants were randomised at the individual rather than departmental level, there was potential for cross-contamination across trial conditions. Accordingly, contamination was assessed at post-test and at the 6-month follow-up using four items (Q1. Did anyone discuss the MH-Guru program with you? Please describe what was discussed about the MH-Guru program? Did anyone show the MH-Guru program to you? Please describe the sections of the MH-Guru program that you saw. Please provide any feedback that you have about those sections).
2.5.5. Demographic, workplace clinical status
Age, gender, education, relationship status, workplace role, duration of employment at current workplace, salary, workplace department, and hours worked. In addition to the administration of the CES-D and GAD-7, self-reported history of depression and anxiety (current and lifetime) were collected through self-report at baseline assessment.
2.6. Planned sample size
The target sample size was based on the smallest effect sizes for the primary outcomes and the level of attrition in an uncontrolled pilot trial (n = 114) of MH-Guru in another organisation. The smallest effect sizes for the pilot trial were 0.34 and 0.38 for help seeking intentions for depression and anxiety respectively as measured by the GHSQ. Attrition in the pilot trial was 61% at post-test. A priori it was calculated that a sample size of 352 employees was required to achieve a between group effect size of 0.3 following the intervention with power of 0.80, and an alpha of 0.05. Assuming 60% attrition at post-test, we calculated that a total of 880 employees (440 per condition) were required for randomisation. In fact, attrition was substantially lower than anticipated (26.7%) so that the required sample size to detect the above effect size was also lower (n = 482).
2.7. Analyses
Differences in the baseline characteristics of post-test and follow-up survey completers and non-completers were examined for each demographic, clinical and outcome variable data using parametric (ANOVAs, t-test) and non-parametric (Pearson chi-square) tests to investigate attrition bias and explore potential threats to internal study validity.
Outcome data were analysed on an intent-to-treat basis. Continuous outcome data were analysed by SPSS 22 (IBM Corp, 2013) using linear mixed model repeated-measures analysis (MMRM) and an unstructured covariance matrix. A mixed models analysis was used to accommodate missing data. Data were assumed to be missing at random. Custom contrasts were used to compare the difference between the intervention and the control group in change over time from baseline to post-test and also from baseline to 6-month follow-up. Dichotomous outcome data (self-reported help seeking behaviour) were analysed by SPSS 22 (IBM Corp, 2013) using Generalised Estimating Equations (GEE) and an unstructured working correlation matrix.
Standardized mean difference scores between groups (Cohen, 1988) were calculated at post-test and at follow-up from the observed (raw) outcome data by dividing the difference in the mean scores for the intervention and control groups by their pooled standard deviations at the relevant time point. For two variables for which randomization resulted in a small difference in scores across groups at baseline we also computed between effect size adjusted for the initial between group effect size at baseline (Durlak, 2009). The magnitude of each effect size is described in accordance with the guidelines suggested by Cohen (1988).
3. Results
The participant flow diagram is shown in Fig. 1.
Fig. 1.
Participant flow.
3.1. Baseline characteristics of the groups
Table 1 summarises the demographic and clinical characteristics of participants in the intervention and control groups. Over two-thirds of participants were women, the mean participant age was 45 years, approximately two-thirds of the sample were non-management staff and the majority of participants were in a married or de-facto relationship. Two-thirds of the participants self-reported they had suffered from depression and over 40% of the respondents reported a history of anxiety. Overall, 28.4% and 19.3% of the sample reported current depression and anxiety respectively. The average baseline depression symptom levels as measured by the CES-D exceeded the cut-off for clinically significant distress. The demographic and clinical characteristics of the control and intervention groups were broadly similar. However, there was some imbalance between the groups in the intention to seek help, with staff in the MH-Guru condition expressing a greater intention to seek help at baseline than those in the Control group (Table 3, depression: d = 0.19 anxiety d = 0.14).
Table 1.
Descriptive characteristics of the participants at baseline as a function of group.
| MH-Guru (n = 255) |
Control (n = 252) |
||
|---|---|---|---|
| n (%) | n (%) | ||
| Gender | Male | 74 (29.0) | 59 (23.4) |
| Female | 180 (70.6) | 193 (76.6) | |
| Position | Managerial | 99 (38.8) | 95 (37.7) |
| Non-managerial | 156 (61.2) | 157 (62.3) | |
| Education | Tertiary educated | 182 (71.4) | 175 (69.4) |
| Department | 1 | 16 (6.3) | 14 (5.6) |
| 2 | 15 (5.9) | 21 (8.3) | |
| 3 | 12 (4.7) | 17 (6.7) | |
| 4 | 20 (7.8) | 28 (11.1) | |
| 5 | 11 (4.3) | 12 (4.8) | |
| 6 | 17 (6.7) | 13 (5.2) | |
| 7 | 60 (23.5) | 53 (21.0) | |
| 8 | 52 (20.4) | 55 (21.8) | |
| 9 | 18 (7.1) | 16 (6.3) | |
| 10 | 23 (9.0) | 9 (3.6) | |
| 11 | 11 (4.3) | 14 (5.6) | |
| Relationship status | Married/de-facto | 171 (67.1) | 167 (66.3) |
| Other | 84 (32.9) | 85 (33.7) | |
| Ever experienced | Yes | 169 (66.3) | 170 (67.5) |
| depression | No | 44 (17.3) | 36 (14.3) |
| Not sure | 42 (16.5) | 45 (17.9) | |
| Ever experienced | Yes | 113 (44.3) | 111 (44.0) |
| anxiety disorder | No | 97 (38.0) | 104 (41.3) |
| Not sure | 45 (17.6) | 37 (14.7) | |
| Current depression | Yes | 78 (30.6) | 66 (26.2) |
| No | 101 (39.6) | 109 (43.3) | |
| Not sure | 76 (29.8) | 76 (30.2) | |
| Current anxiety | Yes | 49 (19.2) | 49 (19.4) |
| No | 135 (52.9) | 150 (59.5) | |
| Not sure | 71 (27.8) | 53 (21.0) | |
| Age | Mean (SD) | 44.4 (11.2) | 44.6 (11.5) |
| Tenure at work (yrs) | Mean (SD, range) | 7.2 (7.7, 1 mth–39 yrs) | 7.5 (7.6, 0 mth–38 yrs) |
| CES-D | Mean (SD) | 17.4 (11.8) | 17.6(13.0) |
| GAD-7 | Mean (SD) | 6.5 (5.4) | 6.1 (5.3) |
CES-D: Center for Epidemiological Studies - Depression Scale; and GAD-7: Generalised Anxiety Disorder Scale.
Table 3.
(i) Observed mean (standard deviation) and number of participants at each measurement time point (derived from raw data); (ii) Estimated marginal means (standard errors) at each measurement time point (from mixed models analysis); and (iii) interaction effect for mixed model repeated-measures analyses for each outcome.
| (i) Observed means (SD), n |
(i) Estimated marginal means |
(iii) Group × time interaction |
||||||
|---|---|---|---|---|---|---|---|---|
| Baseline | Post-test | Follow-up | Baseline | Post-test | Follow-up | Intent to treat mixed models analysis | ||
| DSS | MH-Guru | 7.2 (5.2), 255 | 4.2 (4.2), 187 | 4.5 (3.9), 125 | 7.2 (0.33) | 4.4 (0.32) | 5.3 (0.37) | F (2, 345.7) = 20.8, p < 0.001 |
| Control | 7.3 (5.2), 252 | 6.8 (5.0), 199 | 6.6 (5.2), 154 | 7.3 (0.33) | 6.9 (0.31) | 6.6 (0.35) | ||
| GASS | MH-Guru | 5.1 (5.4), 255 | 3.0 (4.2), 187 | 3.0 (4.0), 124 | 5.1 (0.34) | 3.1 (0.34) | 3.6 (0.39) | F (2, 338.9) = 16.0, p < 0.001 |
| Control | 4.9 (5.6), 252 | 5.0 (5.3), 196 | 5.1 (5.6), 154 | 4.9 (0.35) | 5.1 (0.33) | 5.2 (0.37) | ||
| D-Lit | MH-Guru | 14.9 (3.8), 255 | 17.4 (3.1), 186 | 17.5 (2.4), 121 | 14.9 (0.23) | 17.2 (0.24) | 17.1 (0.26) | F (2, 340.5) = 38.4, p < 0.001 |
| Control | 14.9 (3.6), 252 | 14.7 (3.7), 195 | 14.8 (4.0), 154 | 14.9 (0.24) | 14.7 (0.24) | 14.8 (0.24) | ||
| A-Lit | MH-Guru | 12.5 (4.6), 255 | 16.1 (4.3), 185 | 16.1 (3.7), 121 | 12.5 (0.27) | 15.9 (0.30) | 15.6 (0.32) | F (2, 332.9) = 52.1, p < 0.001 |
| Control | 12.7 (4.2), 252 | 12.6 (4.4), 194 | 12.7 (4.6), 154 | 12.7 (0.28) | 12.6 (0.30) | 12.9 (0.3) | ||
| GHSQ-Dep | MH-Guru | 44.2 (12.8), 255 | 45.6 (13.3), 186 | 46.4 (13.0), 122 | 44.2 (0.78) | 45.3 (0.90) | 46 (0.98) | F (2, 356.0) = 0.29, p = 0.75 |
| Control | 41.8 (12.1), 252 | 42.8 (12.9), 195 | 42.1 (12.4), 154 | 41.8 (0.79) | 42.6 (0.88) | 42.8 (0.92) | ||
| GHSQ-Anx | MH-Guru | 43.8 (13.4), 255 | 45.2 (14.1), 186 | 45.6 (13.6), 122 | 43.8 (0.83) | 44.8 (0.95) | 45.1 (1.0) | F (2, 342.9) = 0.35, p = 0.70 |
| Control | 41.9 (13.0), 252 | 42.3 (13.9), 195 | 42.2 (13.2), 154 | 41.9 (0.84) | 42.2 (0.94) | 42.7 (0.97) | ||
| ATSPPH-SF | MH-Guru | 22.2 (5.3), 255 | 22.7 (5.3), 186 | 23.4 (5.1), 122 | 22.2 (0.34) | 22.6 (0.36) | 23.1 (0.38) | F (2, 341.7) = 1.75, p = 0.18 |
| Control | 22.1 (5.6), 252 | 22.0 (5.7), 195 | 22.1 (5.3), 154 | 22.1 (0.34) | 22.1 (0.36) | 22.3 (0.35) | ||
DSS-Depression Stigma Scale – Personal; GASS: Generalised Anxiety Stigma Scale – Personal; D-Lit: Depression Literacy Scale; A-Lit: Anxiety Literacy Scale; GHSQ: General Help Seeking Questionnaire; Dep: Depression; Anx: Anxiety; and ATSPPH-SF: Attitudes Towards Seeking Professional Psychological Help Scale-Short Form. Significant findings (p < .05) are highlighted in bold.
The MH-Guru sample represented approximately 2.4% of the organisational population. The demographic characteristics of the sample were not markedly different from those of the organisational population. Thus, 51.3% of participants in the trial were aged between 30 and 49 years compared to 51% of the organisational population, a difference which was not statistically significant. Almost three-quarters (73.6%) of the participants were women compared to 65.0% in the organisation (χ2 (1, n = 506) = 16.25, p < 0.001), a difference which although statistically significant was not large (Cramer's V = 0.03). Similarly, 84% of participants compared to 77.5% in the population were permanent employees (χ2 (1, n = 507) = 11.58, p < 0.001, Cramer's V = 0.02) and 81.9% compared to 74% in the population were employed full-time (χ2 (1, n = 507) = 33.06, p < 0.001, Cramer's V = 0.04).
3.2. Attrition across conditions and characteristics of non-completers
Of the 507 valid cases who were randomised, 121 people (23.9%) did not complete the first primary outcome measure (DSS) at post-test and 228 (45.0%) failed to complete the measure at the 6-month follow-up. There was no significant difference in attrition at post-test between the groups (χ2 (1, n = 507) = 2.22, p = 0.14), but a significantly greater proportion of the MH-Guru participants completed the 6-month follow-up survey (χ2 (1, n = 507) = 7.49, p = 0.006).
There was little evidence of differences in baseline clinical or demographic characteristics or baseline outcome measure scores for post-test survey completers and non-completers (Table 2). Only one difference achieved statistical significance. Those employed in a managerial position were significantly more likely to drop out of MH-Guru than non-managerial staff.
Table 2.
Baseline characteristics and measures for post-test completers (C) and non-completers (NC).
| MH-Guru |
Control C |
||||||
|---|---|---|---|---|---|---|---|
| C (n = 187) n (%) | NC (n = 68) n (%) | C (n = 199) n (%) | NC (n = 53) n (%) | ||||
| Gender | Female | 136 (73.1)a | 44 (64.7) | χ2 (1) = 1.7, p = 0.19 | 157(78.9) | 36 (67.9) | χ2 (1) = 0.28, p = 0.09 |
| Tertiary educated | Yes | 132 (70.6) | 50 (73.5) | χ2 (1) = 0.21, p = 0.65 | 138 (69.3) | 37 (69.8) | χ2 (1) = 0.004, p = 0.95 |
| Relationship status | Married/de-facto | 125 (66.8) | 46 (67.6) | χ2 (1) = 0.12, p = 0.90 | 133 (66.8) | 34 (64.2) | χ2 (1) = 0.14, p = 0.71 |
| Position | Managerial | 65 (34.8) | 34 (50) | χ2(1) = 4.88, p = 0.03 | 73 (36.7) | 22 (41.5) | χ 2 (1) = 0.42, p = 0.52 |
| Ever experienced | Yes | 127 (67.9) | 42 (61.8) | χ2 (2) = 0.87, p = 0.65 | 135 (68.1)2 | 35 (66) | χ2 (2) = 0.39, p = 0.83 |
| depression | No | 31 (16.6) | 13 (19.1) | 27 (13.6)2 | 9 (17) | ||
| Not sure | 29 (15.5) | 13(19.1) | 36 (18.2)2 | 9 (17) | |||
| Ever experienced anxiety disorder | Yes | 84(44.9) | 29 (42.6) | χ2 (2) = 0.41, p = 0.81 | 86(43.2) | 25 (47.2) | χ2 (2) = 0.36, p = 0.84 |
| No | 69(36.9) | 28 (41.2) | 84(42.2) | 20 (37.7) | |||
| Not sure | 34(18.5) | 11 (16.2) | 29(14.6) | 8 (15.1) | |||
| Current depression | Yes | 60(32.1) | 18 (26.5) | χ2 (2) = 1.02, p = 0.60 | 49 (24.7)2 | 17 (32.1) | χ2 (2) = 2.54, p = 0.28 |
| No | 74(39.6) | 27 (39.7) | 91 (46)2 | 18 (33.9) | |||
| Not sure | 53(28.3) | 23 (33.8) | 58 (29.3)2 | 18 (34.0) | |||
| Current anxiety | Yes | 39(20.9) | 10 (14.7) | χ2 (2) = 1.32, p = 0.52 | 35(17.6) | 14 (26.4) | χ2 (2) = 2.59, p = 0.27 |
| No | 96(51.3) | 39 (57.4) | 123(61.8) | 27 (50.9) | |||
| Not sure | 52(27.8) | 19 (27.9) | 41(20.6) | 12 (22.6) | |||
| Mean (sd) | Mean (sd) | Mean (sd) | Mean (sd) | ||||
| Age | 44.5 (11.2)a | 44.4 (11.2) | t(252) = − 0.041, p = 0.97 | 44.5(11.7) | 44.8 (10.6) | t(250) = 0.15, p = 0.88 | |
| CES-D | 18.1(12.0) | 15.5 (11.1) | t(253) = − 1.5, p = 0.13 | 17.2(12.8) | 19.3 (13.9) | t(250) = 1.1, p = 0.29 | |
| GAD-7 | 6.7(5.6) | 5.8 (5.1) | t(253) = − 1.1, p = 0.26 | 6.0(5.3) | 6.3 (5.3) | t(250) = 0.41, p = 0.68 | |
| DSS | 6.9(4.9) | 7.9 (5.9) | t(253) = 1.23, p = 0.22 | 7.1(5.1) | 8.0 (5.5) | t(250) = 1.05, p = 0.30 | |
| GASS | 5.0(5.0) | 5.3 (6.5) | t(253) = 0.28, p = 0.78 | 4.7(5.4) | 5.6 (6.0) | t(250) = 1.1, p = 0.28 | |
| D-Lit | 15.2(3.5) | 14.3 (4.6) | t(253) = − 1.52, p = 0.13 | 14.9(3.6) | 15.0 (3.7) | t(250) = 0.19, p = 0.85 | |
| A-Lit | 12.7(4.4) | 11.8 (5.0) | t(253) = − 1.3, p = 0.18 | 12.7(4.3) | 12.9 (3.7) | t(250) = 0.29, p = 0.77 | |
| GHSQ-Dep | 44.4(12.3) | 43.8 (14.4) | t(253) = − 0.33, p = 0.74 | 41.9(12.2) | 41.4 (12.0) | t(250) = − 0.27, p = 0.79 | |
| GHSQ- Anx | 44.1(12.9) | 42.7 (14.9) | t(253) = − 0.74, p = 0.46 | 41.9(13.0) | 41.7 (13.2) | t(250) = − 0.14, p = 0.89 | |
| ATSPPH-SF | 22.2(5.3) | 22.2 (5.3) | t(253) = 0.05, p = 0.96 | 22.0(5.6) | 22.4 (5.7) | t(250) = 0.49, p = 0.63 | |
CES-D: Center for Epidemiological Studies - Depression Scale; GAD-7: Generalised Anxiety Disorder Scale; DSS: Depression Stigma Scale – Personal; GASS: Generalised Anxiety Stigma Scale – GASS-P; D-Lit: Depression Literacy Scale; A-Lit: Anxiety Literacy Scale; GHSQ: General Help Seeking Questionnaire; Dep: Depression; Anx: Anxiety; and ATSPPH-SF: Attitudes Towards Seeking Professional Psychological Help Scale-Short Form.
n = 186. 2n = 198.
With respect to the 6-month follow-up, non-completers in the MH-Guru group had a significantly higher baseline depression stigma score (Completer: Mean (SD) = 6.41 (4.68), Non completer: Mean (SD) = 7.94 (5.57), t(248.6) = 2.4, p = 0.02). There were no other statistically significant baseline differences between completers and non-completers for the MH-Guru condition at the 6-month follow-up. However, non-completers in the Control condition showed more severe baseline depressive symptoms as measured by the CES-D (Completer: Mean (SD) = 15.9 (12.6); Non-completer: Mean (SD) = 20.3 (13.3); t(250) = 2.62, p = 0.009), a higher level of self-reported current depression (33.1% non-completers compared with 21.6% completers; χ2 (2, n = 251) = 9.51, p = 0.009) and more severe anxiety symptoms (Completer: Mean (SD) = 5.54 (5.42); Non-completer: Mean (SD) = 6.86 (5.1), t(250) = 1.93, p = 0.055).
3.3. Adherence to the intervention
3.3.1. Logons
The mean number of logons across the 255 participants in the intervention condition was 1.77 (sd = 1.38; range 0 to 7). The modal number of logins was 2. A total of 21.6% of participants did not log in to the MH-Guru program, 27.0% logged in more than twice and the remaining participants (51.4%) logged in once or twice.
3.3.2. Module completion
During the intervention period (1 to 3 weeks after the baseline date stamp) 180 intervention participants (70.6%) completed module 1 and 127 (49.8%) completed module 2. These completion rates increased to 192 (75.3%) and 133 (52.2%) over the entire trial period (from baseline to 6-month follow-up).
3.4. Effect of MH-Guru on primary outcomes
3.4.1. Intent to treat
Table 3 summarises the results for the outcome measures for the two conditions at baseline, post-test (1 week after the final module) and at 6-month follow-up, together with the results of the mixed models repeated measures analysis. A statistically significant interaction between group and time indicates that there was a differential effect of the intervention and the control condition on an outcome.
3.4.2. Stigma
Significant interactions were found for both personal depression stigma (DSS-personal) and generalised anxiety disorder stigma (GASS-personal). The custom contrasts revealed a decline in depression and anxiety stigma scores for MH-Guru compared to control group participants from baseline to post-test and baseline to 6 month follow-up (Post-test: Depression (t(421) = 6.4, p < 0.001); Anxiety (t(416.6) = 5.5, p < 0.001). 6-months: Depression (t(339.8) = 2.8, p = 0.005); Anxiety: (t(326.3) = 4.1, p < 0.001). There were moderate between group effect sizes for depression and anxiety stigma at post-test (d = − 0.56 and d = − 0.42, respectively) and at 6 months (d = − 0.47 and d = − 0.42 respectively), where the negative effect corresponded to a reduction in stigma for the MH-Guru group.
3.4.3. Literacy
Similarly, the significant interactions between group and time indicate that employees completing the MH-Guru program demonstrated a greater increase in knowledge about depression and anxiety than staff who were not assigned to the MH-Guru program. The custom contrasts showed an improvement in knowledge for MH-Guru participants compared to the control from baseline to post-test and from baseline to 6-month follow-up (Post-test: Depression (t(415.2) = 8.5, p < 0.001); Anxiety (t(408.1) = 9.7, p < 0.001); 6-months: Depression: (t(349.4) = 7.2, p < 0.001); Anxiety (t(322.9) = 7.9 p < 0.001). There was a moderate-to-large between group effect size for both depression and anxiety literacy at post-test (d = 0.78 and d = 0.80 respectively) and at 6-month follow-up (d = 0.81 and d = 0.79 respectively).
3.4.4. Help seeking
3.4.4.1. Intentions and attitudes
There was no statistically significant interaction between group and time for the intention to seek help (GHSQ) for depression or anxiety or for attitudes to seeking help for a mental health problem (ATSPPH-SF). However, there was a main effect of time for help seeking intentions for depression (F(2, 356.0) = 3.8, p = 0.023) and for help seeking attitudes (F(2, 341.7) = 3.6, p = 0.027). The custom contrasts revealed, that across the combined groups, there was an increase in intention to seek help for depression from baseline to post-test (t(393.8) = 2.0, p = 0.043) and baseline to 6-month follow-up (t(309.4) = 2.5, p = 0.012). Attitudes towards help-seeking also improved at 6-month follow-up compared to baseline for the combined groups (t(317.4) = 2.7, p = 0.007). There were small between group effect sizes for GHSQ depression and anxiety at baseline (d = 0.19 and d = 0.14) as well as at post-test (d = 0.21 and d = 0.21 respectively) and 6-month follow-up (d = 0.34 and 0.25 respectively). After adjusting the post- and follow-up between group effect sizes for baseline differences, the post-test effect sizes were negligible (adjusted d = 0.02 and 0.07 respectively) and very small at the 6-month follow-up (adjusted d = 0.15 and 0.11 respectively). The between group effect size for attitudes towards seeking help was small at both post-test (d = 0.12) and at the 6-month follow-up (d = 0.25).
Since the intention to seek help might differ across individual sources of help, analyses were undertaken separately for each item. There was a significant interaction between group and time for the intention to seek help for depression from the Internet (F(2, 375.68) = 3.95, p = 0.02). A custom contrast revealed that at the 6-month post-intervention, participants exposed to the MH-Guru program showed a greater increase in the intention to seek help for depression from the Internet (EMM 4.35, SE 0.15) than the Control group (EMM 3.55, SE 0.14) (t(315.34) = 2.80, p = 0.005). There was a small between group effect size at baseline (d = 0.14) with effect sizes of 0.23 and 0.44 at post-test and 6-month follow-up. Adjusting the post-test and 6-month effect sizes for the baseline difference, yielded a very small effect size at post-test (adjusted d = 0.09) and a small effect at 6-month follow-up (adjusted d = 0.30).
A similar analysis was undertaken for the items in the attitudes to help seeking scale. There was a significant interaction between time and group for the item “Personal and emotional problems tend to work out by themselves” (F(2, 375.32) = 4.553, p = 0.011). A custom contrast showed that relative to baseline the MH-Guru participants were less likely to endorse this statement at 6-month follow-up than participants from the Control group (t(335.41) = 2.91, p = 0.004; d = − 0.26).
3.4.4.2. Behaviour
There was a statistically significant interaction between group and time for self-reported information/treatment seeking for depression with help seeking greater among the intervention group (MH-Guru: 14.9%, Control 8.3%, Wald Chi-Square (1) = 5.07, OR = 1.84, p = 0.024). There was also a significant difference in help seeking for anxiety in favour of the intervention (MH-Guru: 12.2%, Control 9.5%, Wald Chi-Square (1) = 4.99, OR = 1.90, p = 0.026). When the data were reanalysed excluding help seeking from the Internet, the effect for depression was of the same magnitude but failed to attain statistical significance (Depression: Wald Chi-Square (1) = 3.27, OR = 1.7, p = 0.07) and the effect for anxiety, although in the same direction, was no longer statistically significant (Wald Chi-Square(1) = 0.91, OR = 1.36, p = 0.34).
3.4.5. Module completers
Table 4 shows the findings for the subset of participants who completed both modules of MH-Guru. These broadly replicated the results for the Intent-to-treat analyses. However, in contrast to the latter, there was a statistically significant interaction effect for the attitudes to help seeking outcome. Custom contrasts showed that relative to baseline, the MH-Guru module completers demonstrated more positive attitudes to help seeking both at post-test and at the 6-month follow-up than the Control group (post-test: t(320.3) = 2.5, p = 0.013; 6-month follow-up: t(267.8) = 2.3, p = 0.02. The magnitude of the effect was negligible prior to adjustment for baseline difference (d = 0.05) and very small after adjustment (d = 0.18). Comparable figures at 6-months were 0.17 and 0.30 (adjusted) respectively.
Table 4.
Findings for module completers.
| (i) Observed means (SD), n |
(i) Estimated marginal means |
(ii) Group × time interaction |
||||||
|---|---|---|---|---|---|---|---|---|
| Baseline | Post-test | Follow-up | Baseline | Post-test | Follow-up | Intent to treat mixed models analysis | ||
| DSS | MH-Guru | 7.1 (4.9), 127 | 3.9 (3.8), 119 | 4.2 (3.8), 84 | 7.1 (0.45) | 3.8 (0.41) | 4.7 (0.47) | F (2, 294.1) = 25.2, p < 0.001 |
| Control | 7.3 (5.2), 252 | 6.8 (5.0), 199 | 6.6 (5.2), 154 | 7.3 (0.32) | 6.9 (0.31) | 6.7 (0.34) | ||
| GASS | MH-Guru | 5.1 (5.1),127 | 2.5 (3.9), 119 | 2.9 (3.9), 84 | 5.1 (0.48) | 2.4 (0.44) | 3.3 (0.51) | F (2, 286.1) = 19.8, p < 0.001 |
| Control | 4.9 (5.6), 252 | 5.0 (5.3), 196 | 5.1 (5.6), 154 | 4.9 (0.34) | 5.1 (0.33) | 5.2 (0.37) | ||
| D-Lit | MH-Guru | 15.0 (3.5),127 | 18.0 (2.5), 118 | 17.9 (2.2), 82 | 15.0 (0.32) | 18.0 (0.30) | 17.6 (0.33) | F (2, 289.4) = 52.5, p < 0.001 |
| Control | 14.9 (3.6), 252 | 14.7 (3.7), 195 | 14.8 (4.0), 154 | 14.9 (0.23) | 14.7 (0.22) | 14.9 (0.24) | ||
| A-Lit | MH-Guru | 12.7 (4.4),127 | 17.2 (3.6), 117 | 16.7 (3.0), 82 | 12.7 (0.38) | 17.2 (0.37) | 16.5 (0.40) | F (2, 289.6) = 74.2, p < 0.001 |
| Control | 12.7 (4.2), 252 | 12.6 (4.4), 194 | 12.7 (4.6), 154 | 12.7 (0.27) | 12.6 (0.30) | 12.9 (0.29) | ||
| GHSQ-Dep | MH-Guru | 42.9 (11.7),127 | 45.7 (13.0), 118 | 47.0 (12.7), 83 | 42.9 (1.1) | 45.1 (1.2) | 46.3 (1.2) | F (2, 289.7) = 2.4, p = 0.09 |
| Control | 41.8 (12.1), 252 | 42.8 (12.9), 195 | 42.1 (12.4), 154 | 41.8 (0.76) | 42.6 (0.86) | 42.8 (0.90) | ||
| GHSQ-Anx | MH-Guru | 42.8 (12.6),127 | 45.7 (13.6), 118 | 45.6 (13.0), 83 | 42.8 (1.1) | 45.1 (1.2) | 44.9 (1.3) | F (2, 278.3) = 1.8, p = 0.17 |
| Control | 41.9 (13.0), 252 | 42.3 (13.9), 195 | 42.2 (13.2), 154 | 41.9 (0.81) | 42.2 (0.92) | 42.7 (0.94) | ||
| ATSPPH-SF | MH-Guru | 21.4 (5.5),127 | 22.3 (5.3), 118 | 23.0 (5.4), 83 | 21.4 (0.49) | 22.4 (0.50) | 22.7 (0.50) | F (2, 281.8) = 4.0, p = 0.02 |
| Control | 22.1 (5.6), 252 | 22.0 (5.7), 195 | 22.1 (5.3), 154 | 22.1 (0.35) | 22.1 (0.37) | 22.3 (0.37) | ||
DSS-Depression Stigma Scale – Personal; GASS: Generalised Anxiety Stigma Scale – Personal; D-Lit: Depression Literacy Scale; A-Lit: Anxiety Literacy Scale; GHSQ: General Help Seeking Questionnaire; Dep: Depression; Anx: Anxiety; and ATSPPH-SF: Attitudes Towards Seeking Professional Psychological Help Scale-Short Form. Significant findings (p < .05) are highlighted in bold.
3.5. Satisfaction, acceptability and usability of MH-Guru
Overall, 92% of the respondents in the MH-Guru group indicated that the information in the program was very useful or useful with only 1% indicating that it was not. Further 90% felt the information could be trusted; no respondent felt the information was not trustworthy. Most respondents indicated the program material was easy to understand (89% with only one participant finding it difficult or very difficult), easy to use (89% with no participant finding it difficult) and convenient to use (86.4% with no participant finding it inconvenient). Most participants felt confident that they could help another person with depression (87%) or anxiety (81%). The majority would recommend it for use in another workplace (83%), with only 3.4% indicating that they would not recommend the program. Between 70 and 80% were: very or mostly satisfied with the program; indicated the program very much or mostly kept their interest; very much or mostly approved of the aesthetics of the program; and were very or mostly satisfied with and liked the program. Conversely only 0–3.4% of respondents self-reported that they were not at all satisfied with the latter attributes. The lowest level of endorsement of MH-Guru related to the ‘fit’ of the program to the respondent. Overall, 60% of respondents indicated a mostly or very good fit, although only 3.4% reported that it did not fit them at all. In response to the question “Have you done anything differently because of MH-Guru” almost one-quarter of participants indicated that they had (e.g., by assisting another staff or family member with a mental health problem, or by seeking help for a mental health problem); a further 20% were unsure if they had responded differently.
Participants in the MH-Guru group typically reported that the program assisted them to improve their mental health literacy. In particular, between 76% and 82% of participants agreed or strongly agreed that the intervention assisted them to learn about recognising, treating, seeking help and assisting another person with anxiety and depression and to recognise myths about the conditions (Table 5). Only a very small minority of participants (2.8–5.7%) disagreed with the propositions that the program was helpful in these respects.
It is notable that only a minority (10%) of participants were strongly concerned about their privacy while completing MH-Guru although 40% of respondents indicated that they were somewhat or slightly worried.
Overall, 84% of respondents indicated that the Internet was a very or mostly good method of delivering the mental health content; only one person responded ‘not at all’ to this item.
3.6. Contamination across conditions
3.6.1. Post-test
Of the 194 Control participants who responded to the question, 3% (n = 15) indicated that they had discussed the MH-Guru Program with someone but the follow-up detail indicated that this discussion related primarily to promotional material and encouragement to participate (e.g., “My workplace encouraged people to complete the survey”) and to the surveys (e.g., “We discussed the first survey and how difficult it was to give useful answers because many of the questions were only yes/no … “). None indicated that they had discussed the content of the program. In response to the item asking if another person had shown them the program, three participants (0.6%) indicated that they had and a further three indicated they were unsure. However, when asked to describe the section they saw the responses provided no evidence that these participants had seen the MH-Guru intervention itself as opposed to promotional material about the study.
3.6.2. Six-month follow-up
Of the 154 Control participants who responded to the question, 1.2% (n = 6) reported that they had discussed the MH-Guru Program and one was unsure. One participant reporting telling a colleague that they were disappointed that the survey did not provide more information about depression and anxiety treatments and how to manage staff and one indicated that someone visited the workplace to talk about depression and anxiety. Other responses related to the survey or to workplace promotion of the study. None of the participants mentioned that they had discussed the content of the program. With respect to the question asking whether another person had shown them Mental Health Guru program, only one person (0.6%) responded in the affirmative, explaining that they had been shown some survey responses by a participant in ‘another stream’. A second participant reported that they were ‘unsure’. There was no indication that any of the Control participants had sighted the Mental Health Guru intervention content.
4. Discussion
The study found that MH-Guru was effective in reducing depression and anxiety stigma and increasing depression and anxiety literacy. Moreover, the effectiveness of the intervention was sustained over a 6-month period. Self-reported information and treatment help-seeking behaviour was greater among participants in the MH-Guru relative to the Control condition at post-test for both anxiety and depression. We also observed that overall help seeking attitudes improved over time as did intentions to seek help for depression; however, there was no differential increase for MH-Guru. The exception was for intentions to seek help from the Internet which at 6-month follow-up and relative to baseline had increased more in the MH-Guru than the Control group. A similar benefit was conferred by MH-Guru for the help-seeking attitude item “Personal and emotional problems, tend to work out by themselves”. Participants who completed both modules of MH-Guru also demonstrated a significantly greater increase in positive attitudes (combined items) to help seeking at post-test and 6-month follow-up than the Control group, although the size of the effects were negligible to very small at post-test.
To our knowledge the current trial is the first to demonstrate the effectiveness of an online intervention for anxiety and depression stigma in a workplace comprising staff without health credentials. Research in other settings has shown that mental health educational programs can reduce stigma both when administered face-to-face and via the Internet (Griffiths et al., 2014). However, to date computerised or online programs have either focused on members of the general public or in the few workplace-based trials on health professionals rather than on lay employees.
Critically the magnitude of the effect of the program for depression stigma was moderate and compared favourably with the effect sizes reported in a recent meta-analysis of stigma interventions which found a pooled mean effect size for depression personal stigma of 0.36 (95% CI 0.1–0.6) (Griffiths et al., 2014). Further the fact that the effect was sustained over 6 months, suggests that an online workplace intervention may confer long-term, meaningful benefits. In fact, only a small number of face-to-face or online trials have previously demonstrated the sustainability of stigma reduction over 6 months or longer. In particular, Jorm et al. (2010b) demonstrated a significant reduction in depression stigma among teachers who received a face-to-face educational program, but the effect was present in only one of 7 stigma items. Consistent with the current findings, Jorm et al. (2010a) using an CD based intervention also reported a reduction in depression stigma at 6-month follow-up but the study was focused on members of the general community rather than staff in a workplace.
The current trial is also the first, to our knowledge, to demonstrate the effectiveness of an educational intervention in reducing anxiety stigma in the workplace, and the only trial to demonstrate long term sustainability of this effect over a 6 month period. The only other investigation of which we are aware that has studied the effect of a psychoeducational program on anxiety stigma was undertaken in a small pilot trial of an online educational program for elite athletes. Gulliver et al. (2012) found that the intervention was associated with a moderate reduction in anxiety stigma at 3-month follow-up (between group effect size = 0.5) but found no effect at post-intervention (effect size = 0.04). However, the elite athlete study was underpowered, with only 29 and 31 participants randomised to the Control and Intervention groups respectively.
The current study also demonstrated a sustained effect of online psychoeducation on depression and anxiety literacy. This extends the findings of Deitz et al. (2009) who investigated the short-term effect of a web-based educational program on depression, anxiety and treatment literacy among parents in a workplace. Relative to control and after adjusting for baseline knowledge levels, Deitz and colleagues reported that the web-based program was associated with greater depression and anxiety literacy at post-test. However, in contrast to the current study Deitz et al. (2009) did not investigate the long term effect of the program on mental health literacy and was focused on parents only.
The effect of the MH-Guru program on help seeking outcomes is not entirely clear. It is encouraging that MH-Guru participants were more likely to have engaged in help-seeking behaviour (either for mental health information or treatment) at post intervention. However, overall there was not a significant differential improvement in help seeking intentions for the intervention group. This is broadly consistent with the findings of previous randomised controlled studies of the effect of passive educational information on help seeking intentions in other (non-workplace) settings (Costin et al., 2009, Gulliver et al., 2013, Taylor-Rodgers and Batterham, 2014). In a study involving young people in the general community, Costin et al. (2009) reported an increase in help seeking intentions for depression for participants receiving e-cards, but there was a similar increase among the control group, a pattern that was also found in the current study. Taylor-Rodgers and Batterham (2014) reported a differential increase in intentions to seek help for anxiety or depression for only one of five sources of help in an Internet-based randomised controlled trial of a passive mental health information intervention among university students. Finally, Gulliver et al. (2013) failed to find an effect of online depression and anxiety information on the help seeking intentions of elite athletes. In the current study, there was evidence of a differential increase in the intentions of MH-Guru participants to seek help for depression from the Internet. The MH-Guru intervention provided an accessible and seamless connection to automated evidence-based programs for treating depression and anxiety. It is possible that this level of accessibility reduced the perceived barriers to help seeking and increased help seeking intentions for the Internet modality. Although somewhat mixed, the data also provided some support for the possibility that MH-Guru might contribute to an improvement in attitudes to help seeking. Two previous studies have evaluated the effect of online mental health education in the workplace on attitudes to help seeking; one program reported a positive effect but it delivered cognitive behaviour therapy along with help seeking information. The other was based on social cognitive therapy and also incorporated passive educational material; it found no effect (Deitz et al., 2009). Previous research targeting students suggests that attitudes to psychological help seeking can improve with exposure to mental health information (Sharp, 2007, Taylor-Rodgers and Batterham, 2014). There is a clear need for further research to investigate in more detail the effects of passive education on help seeking attitudes in the workplace.
There was a medium level of dropout at post-test in the current study and a significant level of attrition at 6 months. Further, although 75% of the staff assigned to MH-Guru completed module 1, a significant percentage of participants did not complete both modules (48%). Additional research is required to determine uptake rates of MH-Guru in practice outside a trial context and particularly when the load of baseline survey is removed, since the time demands associated with the latter may have decreased willingness to devote time to the intervention itself.
At post-test managerial staff were more likely to drop out than their non-managerial counterparts in the MH-Guru condition. It is possible that managerial staff were busier than other workers; alternatively the perceived value of the workplace program may have been lower among managerial staff. Whatever the explanation, if confirmed, the finding is of concern given that those in leadership may be best positioned to model positive approaches to mental health, promote mental health education in the workplace, and serve as important gatekeepers in the workplace. Moreover, there is evidence from controlled trials that supervisor mental health education is associated with improved mental health among staff (Tsutsumi, 2011). The finding that Control group but not MH-Guru participants were less likely to complete the 6-month follow-up survey if they had higher levels of depressive or anxiety symptoms is of interest. It suggests that such symptoms may have interfered with the participant's long-term motivation, but only in the absence of the active program.
Overall, the satisfaction items suggest that MH-Guru is an acceptable intervention for the workplace, with participants finding it useful, trustworthy, easy to understand and helpful in improving their mental health literacy and their capacity to assist others. The majority of participants indicated that they would recommend the program for use in other workplaces. A substantial minority of participants reported at post-test that exposure to MH-Guru had influenced their behaviour. It is encouraging that respondents considered that the Internet was an appropriate modality for delivering mental health training. However, the finding may not represent the views of workers as a whole since those for whom the Internet was not a convenient delivery modality may have declined to enrol or dropped out of the study. If there are barriers to some workers using online mental health training, consideration should be given to targeting and delivering mental health training to these workers through alternative modalities. Future research should investigate whether programs such as MH-Guru are more effective and suitable for some workers than others, to consider the value of tailoring programs such as MH-Guru to individual workers and to determine what the tailoring characteristics should be.
Finally, although most workers were not strongly concerned about their privacy, the latter should be a critical factor in implementing a program such as MH-Guru. For example, programs such as these might best be hosted externally on secure servers not accessible by the workplace, with the software service delivered by providers who are well trained in the relevant privacy and data protection issues.
4.1. Limitations
The limitations associated with study attrition at 6-months have been discussed above. Other limitations were the failure to separate out information and treatment help seeking outcomes and the absence of a measure of treatment help-seeking behaviour for the 6-month period prior to the final follow-up. It is also possible that participants in the MH-Guru group counted information from the MH-Guru program when responding to the question: “In the last two weeks, have you sought information or treatment for depression or anxiety”. However, this is unlikely to fully account for the findings given that excluding internet sources of help, the trend remained, although it was no longer statistically significant. Research is required to further explore the impact of MH-Guru on help seeking. Another potential limitation of the current study is that the design involved randomisation at the individual rather than departmental level. However, there was no evidence from questions designed to elicit such information that there was contamination across conditions and if even if there were, such contamination would not explain the positive findings for the Intervention group. A further limitation of the study was that it focused on a particular self-selected sample of volunteers. Although the demographic characteristics of the participants were broadly reflective of those of the workforce population, only a small minority of employees participated in the program, and a substantial proportion of the participants in the study reported a history of depression; thus, the participants might have been more receptive to mental health training. At the same time their stigma levels may have been lower and their literacy higher than their counterparts in the organisation limiting the magnitude of the potential effects of the intervention. An intervention and associated recruitment campaign incorporating a greater focus on the concept of workplace stress might have generated greater engagement and yielded a more representative sample and higher rates of uptake and reduced attrition. However, a broader aim of the intervention was to raise community awareness of depression and anxiety with the workplace being a convenient means of accessing a captive audience. Tailoring the intervention by focusing on workplace stress might reduce its generalizability. In practice, program uptake is likely to be greater outside a research trial environment, particularly if mandated as part of a workplace induction program. A further, related limitation of the current study is that it focused on one type of organisation, namely a government workplace. Finally, we acknowledge that the statistical analyses included multiple comparisons which may have generated spuriously significant results due to type 1 error.
4.2. Conclusions
Online mental health education in the workplace has the potential to deliver mental health training conveniently, with high fidelity and without the requirement for specialist expertise. This study demonstrated that a brief online educational mental health program was both acceptable and effective in reducing stigma and improving mental health literacy among employees of varying tenure in a workplace setting. Since the program was designed as a mental health induction course, research is required to investigate if these findings generalise to new employees at the time of induction and if effects are sustained in workplaces which mandate participation in the program. In addition, further research is required to evaluate the effectiveness and acceptability of the intervention in different types of workplace. Finally, there is a need to further investigate the impact of this intervention on help seeking for depression and anxiety and to investigate the effect of individual attributes such as mental health status on outcomes and acceptability of the program.
Conflict of interest
KG, KB, SG and AB were involved in the development of the MH-Guru program. KG is affiliated with and KB, JW and AB are employed by the Australian National University which now offers organisational subscriptions to the program on a commercial basis.
Acknowledgements
KG was supported by NHMRC Fellowship 1059620 at the time this study was undertaken. The trial was funded by a grant from the ANU and the ACT Government through the Discovery Translation Fund. The original development of MH-Guru was enabled by one-off funding from the Australian Commonwealth Department of Health. Ms. Neera Odhavji provided research assistance. Dr. Philip Batterham undertook the sample size calculations. Dr. Hwan-Jin Yoon provided statistical expertise.
Contributor Information
Kathleen M Griffiths, Email: kathy.griffiths@anu.edu.au.
Kylie Bennett, Email: kylie.bennett@anu.edu.au.
Jennie Walker, Email: jennie.walker@anu.edu.au.
Susan Goldsmid, Email: susan.goldsmid@aic.gov.au.
Anthony Bennett, Email: anthony.bennett@anu.edu.au.
References
- Barney L.J., Griffiths K.M., Jorm A.F., Christensen H. Stigma about depression and its impact on help-seeking intentions. Aust. N. Z. J. Psychiatry. 2006;40:51–54. doi: 10.1080/j.1440-1614.2006.01741.x. [DOI] [PubMed] [Google Scholar]
- Bayar M.R., Poyraz B.C., Aksoy-Poyraz C., Arikan M.K. Reducing mental illness stigma in mental health professionals using a web-based approach. Isr. J. Psychiatry Relat. Sci. 2009;46:226–230. [PubMed] [Google Scholar]
- Bloom D.E., Cafiero E.T., Jané-Llopis E., Abrahams-Gessel S., Bloom L.R., Fathima S., Feigl A.B., Gaziano T., Mowafi M., Pandya A., Prettner K., Rosenberg L., Seligman B., Stein A.Z., Weinstein C. World Economic Forum; Geneva: 2011. The Global Economic Burden of Noncommunicable Diseases. [Google Scholar]
- Cohen J. Lawrance Eribaum Association. 1988. Statistical power analysis for the behavior science. [Google Scholar]
- Cooper A., Corrigan P., Watson A. Mental illness stigma and care seeking. J. Nerv. Ment. Dis. 2003;191 doi: 10.1097/01.NMD.0000066157.47101.22. [DOI] [PubMed] [Google Scholar]
- Costin L.D., Mackinnon J.A., Griffiths M.K., Batterham J.P., Bennett J.A., Bennett K., Christensen H. Health e-cards as a means of encouraging help seeking for depression among young adults: randomized controlled trial. J. Med. Internet Res. 2009;11 doi: 10.2196/jmir.1294. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Deitz D.K., Cook R.F., Billings D.W., Hendrickson A. A web-based mental health program: reaching parents at work. J. Pediatr. Psychol. 2009;34:488–494. doi: 10.1093/jpepsy/jsn108. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Durlak J.A. How to select, calculate, and interpret effect sizes. J. Pediatr. Psychol. 2009 doi: 10.1093/jpepsy/jsp004. [DOI] [PubMed] [Google Scholar]
- Elhai J., Schweinle W., Anderson S. Reliability and validity of the attitudes toward seeking professional psychological help scale-short form. Psychiatry Res. 2008;159:320–329. doi: 10.1016/j.psychres.2007.04.020. [DOI] [PubMed] [Google Scholar]
- Greenberg P.E., Fournier A.A., Sisitsky T., Pike C.T., Kessler R.C. The economic burden of adults with major depressive disorder in the United States (2005 and 2010) J. Clin. Psychiatry. 2015;76:155–162. doi: 10.4088/JCP.14m09298. [DOI] [PubMed] [Google Scholar]
- Griffiths K.M., Christensen H., Jorm A.F., Evans K., Groves C. Effect of web-based depression literacy and cognitive-behavioural therapy interventions on stigmatising attitudes to depression: randomised controlled trial. Br. J. Psychiatry. 2004;185:342–349. doi: 10.1192/bjp.185.4.342. [DOI] [PubMed] [Google Scholar]
- Griffiths K.M., Nakane Y., Christensen H., Yoshioka K., Jorm A.F., Nakane H. Stigma in response to mental disorders: a comparison of Australia and Japan. BMC Psychiatry. 2006;6:21. doi: 10.1186/1471-244X-6-21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Griffiths K.M., Christensen H., Jorm A.F. Predictors of depression stigma. BMC Psychiatry. 2008;8:25. doi: 10.1186/1471-244X-8-25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Griffiths K.M., Batterham P.J., Barney L., Parsons A. The generalised anxiety stigma scale (GASS): psychometric properties in a community sample. BMC Psychiatry. 2011;11:184. doi: 10.1186/1471-244X-11-184. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Griffiths K.M., Carron-Arthur B., Parsons A., Reid R. Effectiveness of programs for reducing the stigma associated with mental disorders. A meta-analysis of randomized controlled trials. World Psychiatry. 2014;13:161–175. doi: 10.1002/wps.20129. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gulliver A., Griffiths K., Christensen H. Perceived barriers and facilitators to mental health help-seeking in young people: a systematic review. BMC Psychiatry. 2010;10:113. doi: 10.1186/1471-244X-10-113. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gulliver A., Griffiths K.M., Christensen H., Mackinnon A., Calear A.L., Parsons A., Bennett K., Batterham P.J., Stanimirovic R. Internet-based interventions to promote mental health help-seeking in elite athletes: an exploratory randomized controlled trial. J. Med. Internet Res. 2012;14 doi: 10.2196/jmir.1864. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gulliver A., Griffiths K., Christensen H., Mackinnon A., Calear A., Parsons A., Bennett1 A., Batterham P., Stanimirovic R. Internet-based interventions to promote mental health help-seeking in elite athletes: an exploratory randomized controlled trial. J. Med. Internet Res. 2013;14 doi: 10.2196/jmir.1864. [DOI] [PMC free article] [PubMed] [Google Scholar]
- IBM Corp . IBM Corp; Armonk, NY: 2013. IBM SPSS Statistics for Windows, Version 22. [Google Scholar]
- Irvine A., Billow M., Bourgeois M., Seeley J.R. Mental illness training for long term care staff. J. Am. Med. Dir. Assoc. 2012;81:e.7–e13. doi: 10.1016/j.jamda.2011.01.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Irvine A.B., Billow M.B., Eberhage M.G., Seeley J.R., Mcmahon E., Bourgeois M. Mental illness training for licensed staff in long-term care. Issues Ment. Health Nurs. 2012;33:181–194. doi: 10.3109/01612840.2011.639482. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jorm A.F., Korten A.E., Jacomb P.A., Christensen H., Rodgers B., Pollitt P. "Mental health literacy": a survey of the public's ability to recognise mental disorders and their beliefs about the effectiveness of treatment. Med. J. Aust. 1997;166:182–186. doi: 10.5694/j.1326-5377.1997.tb140071.x. [DOI] [PubMed] [Google Scholar]
- Jorm A.F., Kitchener B.A., Fischer J.A., Cvetkovski S. Mental health first aid training by e-learning: a randomized controlled trial. Aust. N. Z. J. Psychiatry. 2010;44:1072–1081. doi: 10.3109/00048674.2010.516426. [DOI] [PubMed] [Google Scholar]
- Jorm A.F., Kitchener B.A., Sawyer M.G., Scales H., Cvetkovski S. Mental health first aid training for high school teachers: a cluster randomized trial. BMC Psychiatry. 2010;10 doi: 10.1186/1471-244X-10-51. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kiropoulos L.A., Griffiths K.M., Blashki G. Effects of a multilingual information website intervention on the levels of depression literacy and depression-related stigma in Greek-born and Italian-born immigrants living in Australia: a randomized controlled trial. J. Med. Internet Res. 2011;13 doi: 10.2196/jmir.1527. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rosen J., Mulsant B., Kollar M., Kastango K., Mazumdar S., Fox D. Mental health training for nursing home staff using computer-based interactive video: a 6-month randomized trial. J. Am. Med. Dir. Assoc. 2002;3:291–296. doi: 10.1097/01.JAM.0000027201.90817.21. [DOI] [PubMed] [Google Scholar]
- Sharp W.G. University of Mississippi; 2007. Help-Seeking and Mental Health Education: An Evaluation of a Classroom-Based Strategy to Modify Help-Seeking for Mental Health Problems. Ph.D. [Google Scholar]
- Taylor-Rodgers E., Batterham P.J. Evaluation of an online psychoeducation intervention to promote mental health help seeking attitudes and intentions among young adults: randomised controlled trial. J. Affect. Disord. 2014;168:65–71. doi: 10.1016/j.jad.2014.06.047. [DOI] [PubMed] [Google Scholar]
- Tsutsumi A. Development of an evidence-based guideline for supervisor training in promoting mental health: literature review. J. Occup. Health. 2011;53:1–9. doi: 10.1539/joh.r10002. [DOI] [PubMed] [Google Scholar]
- Whiteford H.A., Ferrari A.J., Degenhardt L., Feigin V., Vos T. The global burden of mental, neurological and substance use disorders: an analysis from the global burden of disease study 2010. PLoS One. 2015;10 doi: 10.1371/journal.pone.0116820. [DOI] [PMC free article] [PubMed] [Google Scholar]