Reviewing the Evidence Base for Mental Health First Aid

Abstract

This article reviews the current state of evidence for Mental Health First Aid's effectiveness in key target populations, to help the California Mental Health Services Authority's planning for future prevention and early intervention activities.

Mental Health First Aid (MHFA) is an educational program designed to equip individuals to recognize the signs of mental illness and substance use disorders and to provide immediate support and assistance (Kitchener and Jorm, 2008). Originally developed in Australia, MHFA was launched in the United States in 2008 and has been spreading rapidly throughout the nation. President Barack Obama included $15 million for MHFA training in the 2014 fiscal year budget and requested another $15 million for fiscal year 2015 (National Council for Behavioral Health, 2014). At least 23 states have put forth legislation or appropriations in support of MHFA (National Council for Behavioral Health, 2014).

The California Mental Health Services Authority (Cal-MHSA), an independent administrative and fiscal intergovernmental structure that encourages counties to work together to develop, fund, and implement a variety of local, regional, and statewide programs and services to improve the mental health of California residents, has also invested in MHFA training. CalMHSA oversees a statewide prevention and early intervention program that includes initiatives addressing mental illness stigma and discrimination reduction, suicide prevention, and student mental health. CalMHSA funded MHFA training across counties in California and California State University campuses under the stigma and discrimination and student mental health initiatives, respectively.

The purpose of this study is to review the current state of evidence for the effectiveness of MHFA in order to help CalMHSA's planning efforts for future prevention and early intervention activities. To accomplish this, this article first provides a broad overview of the evidence base for MHFA, while noting key issues that should be taken into consideration when weighing the available body of evidence. Second, it examines outcomes among individuals trained to administer MHFA (referred to as “MHFAiders”) who are from key target populations of interest to CalMHSA. Finally, the article reviews the evidence base for outcomes among those whom MHFA is ultimately designed to reach—individuals who may be at risk for or are experiencing mental illness and are potential recipients of aid from someone who has participated in MHFA training.

What Is the Evidence Base for Mental Health First Aid?

One approach to integrating findings across a number of studies is through a statistical method known as meta-analysis. A meta-analysis provides a rigorous systematic process for quantifying the overall effect of a treatment or program by summarizing findings across independent studies (Egger, Smith, and Phillips, 1997). Given that a recent meta-analysis of MHFA was published during the writing of the present review, we summarize the findings from the meta-analysis to provide an overview of the evidence base for MHFA. We then highlight limitations of the studies included in the meta-analysis to aid in the interpretation of the strength of the evidence base for MHFA.

A recent meta-analysis of MHFA identified 15 studies that conducted a quantitative evaluation of the standard adult or youth version of MHFA (three of the 15 studies included the youth version) (Hadlaczky et al., 2014). Mean effect sizes, a measure of the magnitude of the intervention effect, were calculated for the following outcomes: knowledge, attitudes, and behaviors. Knowledge consisted of measures that tapped into participants' ability to identify mental health problems, as well as beliefs about effective treatments for mental health problems. Attitudes comprised measures of social distance, which asked participants about their willingness to interact with individuals with mental health problems in a variety of interpersonal situations (e.g., work, socializing). Behaviors were assessed according to the number of times participants self-reported providing help to individuals with a mental health problem. Generally, 0.2 is considered a small effect size, 0.5 a medium effect size, and 0.8 a large effect size (Cohen, 1998). Mean effect sizes were 0.56 for knowledge, 0.28 for attitudes, and 0.25 for behaviors, indicating that MHFA training had a medium effect on changing knowledge and small effects on influencing attitudes and behaviors. As a frame of reference, average effect sizes for effective medical treatments (e.g., chemotherapy for certain cancers, surgery for coronary heart disease) typically fall below 0.50 (Lipsey and Wilson, 1993). Moreover, successful prevention programs for smoking and alcohol use among school children have yielded mean effect sizes between 0.29 and 0.36 (Bruvold, 1993; Tobler, 1992). Thus, findings from the meta-analysis indicate promising gains among individuals trained in MHFA equivalent to or greater than those for other widely accepted interventions.

Despite these findings, it is important to consider the nature and characteristics of the studies that undergird this body of evidence. Below, we highlight key summative information about the 15 studies included in the meta-analysis:

• Study Design: Four studies employed a randomized controlled trial (RCT), which compares individuals randomly assigned to the intervention (i.e., MHFA training) or to a control group. Random assignment helps to ensure that there are no systematic differences between individuals in the intervention versus the control group (e.g., prior mental health knowledge, motivation) thereby increasing the confidence that any observed group differences are solely due to the intervention. Thus, RCTs are considered one of the most rigorous research study designs. Another two studies included a control group for comparison but did not randomly assign individuals to the intervention versus the control group, which is not as strong as an RCT. The remaining nine studies involved single-group pre-post designs, which assess for intervention effects by examining changes in individuals before and after the training. Compared with RCTs, single-group pre-post designs cannot establish as definitively that observed changes are due solely to the intervention; rather, changes could have been related to other factors, such as increased sensitivity to mental health issues resulting from completing the pre-test, exposure to other influences after the training but before the post-test (e.g., news stories about mental illness), or “response shift,” in which participants' understanding of a concept shifts after the training, resulting in changes in the interpretation of assessment questions (e.g., after learning about how to appropriately respond to suicide risk, participants may report feeling less confidence in supporting individuals with suicide risk after the training) (Harris et al., 2006).

• Time Between MHFA Training and Outcome Assessment: Ten studies assessed outcomes at five months or more following MHFA training, three studies assessed outcomes at one to three months after training, and two studies evaluated outcomes immediately following MHFA training. Studies with longer follow-up time frames allow us to draw stronger conclusions about sustained change. Shorter follow-ups may reflect temporary shifts or biases that are influenced by a motivation to provide socially desirable responses.

• Setting: Twelve studies were conducted in Australia, two in Sweden, and one in Canada; no studies were conducted in the United States.

• Participant Gender: Almost all studies had a majority of female participants.

• Outcomes Among Potential MHFA Recipients: The majority of studies in the meta-analysis focused on outcomes among individuals trained in MHFA (i.e., MHFAiders). Only two studies examined outcomes among the people MHFA is intended to assist, that is, individuals who may be at risk for or are experiencing mental health problems.

• Study Rigor: Hadlaczky et al. (2014) found that effect sizes were consistent and did not significantly vary depending on the rigor of the study design (i.e., inclusion of a control group). However, knowledge outcomes did vary based on the quality of the venue in which the study was published; effect sizes for knowledge were greater in non–peer reviewed outlets or in journals with a lower impact factor.

Overall, the evidence is reasonably strong that individuals trained in MHFA experience improvements in knowledge, attitudes, and help-provision behaviors. Perhaps the strongest limitation relevant to CalMHSA is the lack of data from the United States. Whether MHFA will be effective in the general California population will depend in part on the extent to which Australians and Californians are similar.

How Effective Is Mental Health First Aid for California Target Populations?

Building on this meta-analysis, we conducted a review to examine outcomes among MHFAiders who are part of specific target populations that are of interest to CalMHSA. Specifically, we looked at MHFAider outcomes for the following target populations: public-sector employees (e.g., law enforcement, social services), ethnic minority groups, school staff/youth organizational leaders (e.g., teachers, football coaches), and young adults.

We included nine of the 15 studies that were in the meta-analysis study. Of the six studies that were excluded, three did not report findings for specific target populations, two were in Swedish, and one was conducted with pharmacy students (not one of the target populations of interest to CalMHSA). In addition, we conducted a search for quantitative evaluations of MHFA starting in March 2014 up until August 2014 via PubMed, PsycInfo, and Web of Science to identify studies published after the meta-analysis. (The meta-analysis included studies published up until March 2014 [Hadlaczky et al., 2014].) We identified three additional studies (Lipson et al., 2014; Reavley et al., 2014; Svensson and Hansson, 2014). Two studies were conducted outside of Australia—one with U.S. college resident advisors (Lipson et al., 2014) and the other with Swedish government employees (Svensson and Hansson, 2014). With these added studies, the evidence base extends slightly into the United States, though only with a single study.

For each target population, we present findings in summary tables. We include information about the study sample; geographic location; whether a control group was included; the time frame for the follow-up assessment; and, for each type of outcome, whether there was a significant positive effect (“+”), no significant effect detected (“NS”), or the outcome was not assessed (“–”). Studies were coded as demonstrating a significant positive effect if a significant effect was found for any assessment within an outcome domain. For example, a study that found significant changes in treatment beliefs about the helpfulness of mental health counselors but not medications would be coded as having a significant positive effect.

In contrast to the Hadlaczky et al. (2014) meta-analysis, we broke out MHFA knowledge outcomes into two separate domains, namely mental health knowledge and treatment efficacy beliefs (i.e., perceived helpfulness of various approaches to mental illness). Further, we employed a stigma outcome measure that included not only social distance measures but also other measures of stigma, such as the degree to which participants viewed mental illness as a sign of personal weakness. Finally, in addition to examining outcomes related to participants' self-perceived confidence in providing help to individuals with mental health problems (confidence providing help), we also report on findings of whether or how often participants self-reported actually providing help to individuals with mental illness (help provided). In the next section, we present findings for these MHFA outcomes for each target population.

Public-Sector Employees

As seen in Table 1, four studies involved public-sector employees working in a variety of service settings. Two of the four studies involved rigorous RCT designs. All studies conducted longer-term follow-up assessments of at least six weeks. With respect to outcomes, there are clear effects for reducing stigma and for increasing confidence in help provision. The data also look fairly strong for self-reported help provision given that the two most rigorous (RCT) studies both found positive effects (Kitchener and Jorm, 2004; Svensson and Hansson, 2014). At the same time, these two studies tested—but did not find significant effects—for treatment efficacy beliefs, making the evidence base weak for this outcome. Although Kitchener and Jorm (2004) found no significant effects for treatment efficacy when examining beliefs about treatment for schizophrenia and depression separately, significant effects were found when treatment efficacy beliefs about both disorders were combined. Three of the four studies showed positive effects on knowledge; however, one of the RCT studies found small and positive effects that were non-significant (Kitchner and Jorm, 2004).

Table 1.

MHFA Outcomes Among Public-Sector Employees

Study	Location	Sample	Control Group	Follow Up	Knowledge	Stigma	Treatment Efficacy Beliefs	Confidence Providing Help	Help Provided (Self-Report)
Kitchener and Jorm, 2004	Australia	Departments of Health & Aging; Family & Community Services	Yes (RCT)	5 months	NS	+	NS*	+	+
Sartore et al., 2008	Australia	Rural/farming support workers	No	6–8 weeks	+	+	+	+	NS
Hossain, Gorman, and Eley, 2009	Australia	Rural/farming support workers	No	6 months	+	+	+	+	−
Svensson and Hansson, 2014	Sweden	Employment agencies, social services, police, corrections, rescue services, recreation centers	Yes (RCT)	6 months	+	+§	NS	+	+

^*Non-significant effects were found when beliefs about treatment for schizophrenia and depression were examined separately, but significant effects were found when items were combined.

^§Significant effect for willingness to be a neighbor of person with depression; non-significant effects for willingness to be in other social situations (i.e., be a friend, family, or colleague) with a person with depression; non-significant effects for being in all social situations with a person with schizophrenia.

Ethnic Minority Groups

We identified three studies that evaluated the effects of MHFA among ethnic minority groups (all residing in Australia; see Table 2) for the following populations: Vietnamese (Minas, Colucci, and Jorm, 2009), Chinese (Lam, Jorm, and Wong, 2010), and Aboriginal/Torres Strait (Morawska et al., 2013). All of the studies only assessed changes occurring immediately post-training. Although Morawska et al. (2013) conducted six-month follow-up assessments with a subset of participants, the domains assessed at six-month follow-up (e.g., help provision) were not assessed at baseline. This precluded an assessment of changes from baseline to six-month follow-up; thus, we do not report on the longer-term follow-up findings.

Table 2.

MHFA Outcomes Among Culturally Diverse Groups

Study	Location	Sample	Control Group	Follow Up	Knowledge	Stigma	Treatment Efficacy Beliefs	Confidence Providing Help	Help Provided (Self-Report)
Minas, Colucci, and Jorm, 2009	Australia	Vietnamese	No	Post	+	+	−	−	−
Lam, Jorm, and Wong, 2010	Australia	Chinese	No	Post	+	+	+	−	−
Morawska et al., 2013	Australia	Aboriginal/Torres Strait*	No	Post	−	+	+	−	−

^*Participants were recruited from community groups and organizations that work with multicultural communities. Approximately 20 percent of participants reported being bilingual and 10 percent were of Aboriginal or Torres Strait Islander ethnicity.

With respect to immediate post-follow-up outcomes, significant gains were found for mental health knowledge, stigma reduction, and treatment efficacy beliefs. However, none of these studies included a control group or assessed for confidence in help provision or actual help provision at baseline or immediate post-follow-up. Given the limited number of studies and the lack of controlled trials, the evidence for effects of MHFA training on outcomes among ethnic minority populations is quite limited. It must also be noted that the studies only involved particular ethnic minority groups (e.g., Vietnamese, Chinese) and for one study, only 10 percent of the sample participants were ethnic minorities (i.e., Morawska et al., 2013). Although these studies focused on a limited set of immediate outcomes, significant positive effects were found for all of the outcomes assessed, indicating some promise for the program. Additional studies are needed to examine whether MHFA training yields positive gains for other outcomes (e.g., help provision), whether longer-term gains can be demonstrated, and whether results would generalize to other ethnic minority groups (e.g., Latinos, Blacks).

School Staff/Youth Organizational Leaders

Three studies examined outcomes among school staff and youth organizational leaders trained in MHFA. As seen in Table 3, significant improvements in virtually all outcome domains were found, with the exception of actual help provision, which had mixed results, including a non-significant finding in the only RCT study (Jorm et al., 2010). In this study, 14 middle schools were randomly assigned to either a waitlist control or MHFA intervention condition. Compared with the waitlist control group, middle school teachers trained in MHFA exhibited no significant differences in help provision with respect to speaking with students about their mental health problems, discussing students' mental health problems with other teachers, or raising mental health issues in staff meetings at six-month follow-up (Jorm et al., 2010). Findings for this target group are quite promising across most outcomes, with the exception of the outcome that may be of most interest—whether actual help was provided. There, the evidence might be described as mixed or lacking.

Table 3.

MHFA Outcomes Among School Staff/Youth Organization Leaders

Study	Location	Sample	Control Group	Follow Up	Knowledge	Stigma	Treatment Efficacy Beliefs	Confidence Providing Help	Help Provided (Self-Report)
Jorm et al., 2010	Australia	Middle school teachers	Yes (cluster randomized)	6 months	+	+	+	+	NS
Massey et al., 2010	Canada	University Student Affairs staff	Yes	6 months	+	+	+	+	+
Pierce et al., 2010*	Australia	Football club coaches	No	6 months	+	−	+	+	−

^*In addition to MHFA training of football club coaches, formal and informal sources of mental health information were distributed to players (e.g., high-profile presentations, local media, and football league messages).

Young Adults

Only one study investigated outcomes among young adults trained as MHFAiders. Although this is the only study conducted in the United States, it is one of the most rigorous in the literature, involving a longer-term follow-up assessment and an RCT design. Lipson et al. (2014) conducted a peer-delivered intervention in which college resident advisors were trained in MHFA. Residence halls across 32 university and college campuses, located in 19 states, were randomized to the MHFA intervention or a control condition. Compared with resident advisors in the control condition, resident advisors trained as MHFAiders demonstrated positive outcomes with respect to knowledge, stigma, treatment efficacy beliefs, and confidence in help provision (see Table 4). However, no significant differences were found for actual help provided. Clearly, more studies are needed of this target group, particularly studies testing help provision.

Table 4.

MHFA Outcomes Among Young Adult Peers

Study	Location	Sample	Control Group	Follow Up	Knowledge	Stigma	Treatment Efficacy Beliefs	Confidence Providing Help	Help Provided (Self-Report)
Lipson et al., 2014	United States	Resident advisors	Yes (RCT)	2–3 months	+*	+	+	+	NS

^*Knowledge was measured by the following item: “Relative to the average person, how knowledgeable are you about mental illnesses and their treatments?”

What Is the Impact on Potential Recipients of Mental Health First Aid?

One key limitation of nearly all MHFA studies is the lack of data on potential recipients of MHFA—that is, individuals who are at risk for or experiencing mental illness and who may or may not receive help from a trained MHFAider. Only three of the studies we reviewed examine the impact of trainings on these individuals (see the first three studies listed in Table 5). All three studies had fairly strong designs, including control groups and longer-term follow-up assessments. In addition to assessing potential aid recipients' mental health knowledge, stigma, and treatment efficacy beliefs, these studies also measured their help-seeking intentions, help sought/received, or both.

Table 5.

Outcomes Among Potential Recipients of MHFA

Study	Setting	Sample	Control Group	Follow Up	Knowledge	Stigma	Treatment Efficacy Beliefs	Help-Seeking Intentions	Help Sought/Received
Jorm et al., 2010	Australia	Middle school students of trained teachers	Yes (cluster randomized)	6 months	NS	NS	NS	NS	+
Pierce et al., 2010	Australia	Football club players	Yes	6 months	−	NS	−	NS	−
Lipson et al., 2014	United States	Random subset of residents	Yes (RCT)	2–3 months	NS	NS	−	−	NS
Reavley et al., 2014	Australia	College students	Yes (cluster randomized)	6 months	NS	NS	NS	+	NS

As seen in Table 5, these studies demonstrate little support for MHFA's effectiveness with respect to potential aid recipients. Specifically, Jorm et al. (2010) found no significant differences between students at schools where staff were trained as MHFAiders versus students at waitlist control schools on any of the outcomes except help received. Although students at MHFA intervention schools were more likely to report having received help, this was only in the form of receiving information about mental health problems from a teacher via a class lesson, written materials, or reference to a website. No significant effects were found for other student outcomes, such as talking with school staff about a mental health problem or having staff members spend time listening to their problem, help them calm down, talk about suicidal thoughts, or recommend professional help—all specific parts of providing MHFA to an individual in distress. Similarly, Pierce et al. (2010) found no significant differences between football players in leagues where coaches received MHFA training and football players in another league not exposed to MHFA training. However, only stigma and help-seeking intentions were assessed. Finally, Lipson et al. (2014) also found no significant differences when comparing students in residence halls in which resident advisors were trained as MHFAiders versus the control condition. Moreover, analyses that controlled for demographics, treatment assignment, and baseline scores yielded counterintuitive findings. Residents in the MHFA condition were less likely to report receiving treatment or informal support. Further, significant increases in service use among residents in the MHFA condition were not observed even among those with a demonstrated baseline mental health need.

Reavley et al. (2014) tested the effects of a multi-component, campus-wide intervention in which MHFA training was made available to students, along with a mental health promotion campaign that involved social media messaging, online resources, email communications, special campus events, fact-sheets/booklets, posters on campus, and awareness-raising events. Nine university campuses were paired (based on staff and student numbers and courses offered), and each pair was randomly assigned to the multi-component intervention or control condition (no interventions). No significant differences were found between students at intervention and control campuses with the exception of help-seeking intentions (see Table 5). Students at intervention campuses were significantly more likely to report that they would obtain drug and alcohol services for an alcohol problem. The lack of MHFA effects may be a result of the very small proportion of students at intervention campuses who reported receiving MHFA training (6.6 percent). Evidently, offering MHFA to this population, even accompanied by promotional efforts, results in limited uptake, thereby also limiting the potential for program effectiveness.

Nonetheless, training young adults on how to effectively link peers to needed mental health treatment appears to be an important point of intervention. In a study conducted with a representative sample of California residents, young adults (ages 18–29) were more likely to report recent contact with someone who has a mental health problem compared with older adults (ages 30 and older) (Collins et al., 2014). Moreover, even though young adults had relatively positive attitudes toward people experiencing mental health problems, they were less likely than older adults to report providing assistance with obtaining professional help.

Overall, there are no data to suggest that MHFA results in increased support among those in distress. However, this is based on a small number of studies conducted with potential recipients of MHFA. Despite these findings, definitive conclusions about the lack of impact on individuals who may be at risk for or are experiencing mental illness cannot be made. Detecting shifts among potential recipients of MHFA can be challenging and may require tracking very large samples with substantial levels of mental health risk over a long period. For example, in a population where one in four may experience mental illness within a one-year period, only a quarter of the sample would be relevant for looking at outcomes such as whether MHFA assistance was received and referral to professional treatment was provided. Further, the ability to detect shifts may also be difficult if the number of people trained in MHFA relative to the number of people at risk is small—that is, if the penetration rate is low. For instance, evidence of effectiveness might be missed even if every person trained in MHFA helps at least one person over the course of a year, given that the overall effect on the at-risk population (the one in four who may experience a mental illness) could be very small.

The lack of significant effects among potential recipients of MHFA could also be related to other factors. The quality of help provision by MHFAiders has not been as rigorously evaluated as other outcomes. Many of the studies assess help provision by asking MHFAiders whether they provided help to individuals with mental health problems. Few studies extend beyond this to assess whether actual encouragement of professional treatment was provided. It is possible that effects are not detected among potential MHFA recipients because MHFAiders may not be administering all of the MHFA actions, which include encouraging appropriate professional treatment. For example, Jorm et al. (2010) assessed specific MHFA actions and found that teachers trained in MHFA were no more likely to encourage professional treatment than teachers who had not received training. Similarly, Lipson et al. (2014) found no significant differences in rates of referring students to professional counseling between resident advisors who had or had not received MHFA training. Thus, the nature and type of support provided by MHFAiders may play a role in whether there is an impact on not only potential recipients' help-seeking behaviors, but also their mental health knowledge, stigma, and treatment efficacy beliefs. Further study is needed to examine the quality of help provided by MHFAiders and whether more intense training or additional coaching is warranted.

Conclusions

Overall, MHFA appears to be useful in improving mental health knowledge and reducing stigma across most target populations of interest to CalMHSA. Although individuals trained in MHFA also reported more positive perceptions of mental health treatment, findings were not as consistent as those observed for knowledge and stigma. MHFA was also associated with self-reported increases in providing help to others with mental health needs, with the strongest evidence found for public-sector employees. Nonetheless, only one study was conducted in the United States. While it is unlikely that the effects of MHFA training would vary greatly between the general Australian and U.S. populations, the limited evidence base for ethnic minority groups is potentially a strong limitation. These groups make up a large segment of the California population, and some key groups in California, such as Latinos and African Americans, have not been studied. To the extent that there are differences in the cultural backgrounds and experiences of minorities in Australia versus the United States, particularly with respect to beliefs about mental illness, the appropriateness of various approaches to providing support for individuals with mental health needs, and the helpfulness of treatment, this would underscore the need for studies conducted with ethnic minority groups in the United States. Moreover, the evidence base for ethnic minority groups is the least robust of those examined, given the lack of rigorous studies and limited assessment of outcomes related to the level of confidence that individuals possess in providing support to those in need and actual rates of providing help to those in need.

We also found no evidence of effectiveness for potential recipients of aid. Among the handful of studies that examined outcomes among potential MHFA recipients, we observed only one significant effect across several outcomes and across three populations—middle school students, football players, and college residents—all of whom had gatekeepers in their contexts trained as MHFAiders. However, it is unclear whether the lack of significant findings is a result of the challenges associated with detecting such effects or a true reflection of the program's effectiveness. Further, it is unknown whether training gatekeepers to effectively administer MHFA is sufficiently adequate to overcome an individual's reluctance to seek treatment. Thus far, anti-stigma campaigns aimed at multiple levels (e.g., national social marketing, regional activities) and target groups (e.g., general public, young people) have yielded limited evidence for increasing utilization of mental health care (Henderson, Evans-Lacko, and Thornicroft, 2013). However, gains in mental health knowledge over the course of these campaigns were linked to increased help-seeking intentions, suggesting that mental health literacy may be a necessary and essential component to improving treatment utilization. This indirect evidence suggests that at least a portion of the process invoked by MHFA training (increasing mental health knowledge/literacy) may facilitate help-seeking, but does not provide a test of whether MHFA itself leads to such changes.

In spite of the limited evidence for the impact on potential recipients of aid and trainees in some key minority groups, there is evidence that MHFA training results in positive changes among trainees in target populations of interest to CalMHSA. The evidence base is stronger and more consistent for certain outcomes and certain target populations, but positive effects are observed with respect to increased mental health knowledge, reduced stigma, more favorable beliefs about treatment, and increased help provision among those trained in MHFA.