Skip to main content
PMC home page
International Journal of Methods in Psychiatric Research logoLink to International Journal of Methods in Psychiatric Research
. 2018 Nov 20;28(2):e1754. doi: 10.1002/mpr.1754

Increasing intentions to use mental health services among university students. Results of a pilot randomized controlled trial within the World Health Organization's World Mental Health International College Student Initiative

David Daniel Ebert 1,✉, Marvin Franke 1, Fanny Kählke 1, Ann‐Marie Küchler 2, Ronny Bruffaerts 3, Philippe Mortier 4, Eirini Karyotaki 5, Jordi Alonso 6, Pim Cuijpers 5, Matthias Berking 1, Randy P Auerbach 7, Ronald C Kessler 8, Harald Baumeister 2; On behalf of the WHO World Mental Health – International College Student collaborators
PMCID: PMC6877244  PMID: 30456814

Abstract

Background

The majority of university students with mental health problems are untreated. Only a small empirical literature exists on strategies to increase mental health service use.

Aims

To investigate the effects and moderators of a brief acceptance‐facilitating intervention on intention to use mental health services among university students.

Method

Within the German site of the World Health Organization's World Mental Health International College Student (WMH‐ICS) initiative, 1,374 university students were randomized to an intervention condition (IC; n = 664) or a control condition (CC; n = 710) that was implemented in the survey itself. Both conditions received the questions assessing mental disorders and suicidality that were included in other WMH‐ICS surveys. The IC group then additionally received: Internet‐based personalized feedback based on subject symptom severity in the domains of depression, anxiety, substance use, suicidal thoughts and behaviors, and nonsuicidal self‐injury; psychoeducation tailored to the personal symptom profile; and information about available university and community mental health services. The primary outcome was reported intention to use psychological interventions in the next semester, which was the last question in the survey. A broad range of potential moderating factors was explored.

Results

There was a significant main effect of the intervention with students randomized to IC, reporting significantly higher intentions to seek help in the next semester than students in the CC condition (d = 0.12, 95% CI: 0.02 to 0.23). Moderator analyses indicated that the intervention was more effective among students that fulfilled the criteria for lifetime (d = 0.34; 95% CI: −0.08 to 0.7) and 12‐month panic‐disorder (d = 0.32; 95% CI: −0.10 to 0.74) compared with those without lifetime (d = 0.11; 95% CI: 0.00 to 0.22) or 12‐month panic disorder (d = 0.11; 95% CI: 0.00 to 0.22), students with lower (d = 0.37; 95% CI: −0.77 to 1.51) than higher (d = −0.01; 95% CI: −0.36 to 0.34) self‐reported physical health, and students with nonheterosexual (d = 0.38; 95% CI: 0.08 to 0.67) compared with heterosexual (d = 0.06; 95% CI: −0.06 to 0.17) sexual orientation. The intervention had no effects among students who reported that they recognized that they had an emotional problem and “are already working actively to change it” (Stage 4 “stages of change”).

Conclusions

A simple acceptance‐facilitating intervention can increase intention to use mental health services, although effects, are on average, small. Future studies should investigate more personalized approaches with interventions tailored to barriers and clinical characteristics of students. In order to optimize intervention effects, the development and evaluation should be realized in designs that are powered to allow incremental value of different intervention components and tailoring strategies to be evaluated, such as in multiphase optimization designs.

Keywords: health behavior, psychoeducation, public mental health, risk factors

1. INTRODUCTION

Mental disorders are highly prevalent among university students, with 12‐month prevalence rates ranging from 20% to 45% (Auerbach et al., 2016; Blanco et al., 2008; Bruffaerts et al., 2018). Anxiety, mood, and substance use disorders are the most widespread mental disorders among these students (Auerbach et al., 2016). Despite the wide availability and well‐known efficacy of interventions (Cuijpers et al., 2013), the majority of university students with mental disorders remain untreated (Auerbach et al., 2016; Blanco et al., 2008; Demyttenaere et al., 2004; Eisenberg, Golberstein, & Hunt, 2009; Larisch et al., 2013). Utilization rates range from 5–42.4% among those suffering from alcohol and drug problems, 16–45% among those with anxiety disorder, and 24.5–57.5% among those with suicidal thoughts and behaviors (Bruffaerts et al., this issue, Blanco et al., 2008).

In addition to the obvious importance of structural barriers, reasons for low treatment rates include knowledge‐related and attitudinal barriers such as lacking knowledge about mental health services (Gulliver, Griffiths, & Christensen, 2010), limited mental health literacy (Jorm et al., 1997) fear of stigmatization (Clement et al., 2015; Gulliver et al., 2010), and the wish to deal with problems on their own (Gould et al., 2004; Gulliver et al., 2010; Mojtabai et al., 2011). Moreover, low risk‐perception (Brewer et al., 2007; Gulliver et al., 2010) expressed as either an inability to identify psychopathological symptoms (Biddle, Donovan, Sharp, & Gunnell, 2007) or an inability to identify the need of treatment despite acknowledging symptoms (Eisenberg, Golberstein, & Gollust, 2007; Mojtabai et al., 2011) has been found to predict low rates of seeking treatment for emotional problems.

Given that the impact of even the most effective interventions is low when utilization is low in the target population (Ebert et al., 2015), research on effective strategies to overcome such nonstructural barriers is of upmost importance. However, only limited empirical research exists on acceptance‐facilitating interventions (AFIs) designed to increase treatment uptake.

One of the most commonly used strategies in AFIs for mental disorders is giving information about relevant symptom clusters in the form of personalized risk feedback (Donker, Griffiths, Cuijpers, & Christensen, 2009; Lewis & Neighbors, 2006; Riper et al., 2009). This approach has shown promising effects in increasing intention to participate in preventive medical procedures (Albada, Ausems, Bensing, & van Dulmen, 2009; Brouwers et al., 2011; Curry, Taplin, Anderman, Barlow, & McBride, 1993; Hovick, Wilkinson, Ashida, de Heer, & Koehly, 2014; Sheridan et al., 2011) and in reducing the alcohol consumption of problem drinkers (Riper et al., 2009). However, other experimental evaluations of the effects of personalized risk feedback to increase treatment for mental health problems have yielded weak results (Chan et al., 2016; King et al., 2015; Quinlivan et al., 2014; Quinlivan et al., 2016), although one study reported an effect on utilization in some subgroups (Batterham, Calear, Sunderland, Carragher, & Brewer, 2016). Contrarily, more consistent evidence exists for positive effects of AFIs based on psychoeducation in changing attitudes toward help‐seeking behavior (Gonzales, Tinsley, & Kreuder, 2002; Gulliver, Griffiths, Christensen, & Brewer, 2012; Hadlaczky, Hökby, Mkrtchian, Carli, & Wasserman, 2014).

Despite these promising findings, AFI experiments are few in number (Cranen, Veld, in't, & Vollenbroek‐Hutten, 2011) and have never before been carried out among university students. More research is needed to increase the proportion of students who receive treatment given the high prevalence and burden of untreated mental disorders in this group (Bruefferts et al., this issue; Alonso et al., this issue). The current study was designed to evaluate an AFI administered to German university students. The AFI combined (a) personalized risk feedback with (b) brief tailored psychoeducation on personal symptoms and mental health treatments and (c) information about available on‐campus and community mental health facilities. We evaluated the effects of this AFI on reported intentions to seek help during the next semester among students regardless of the presence of a mental disorder. We also aimed to examine in an exploratory manner a broad range of potential moderators of AFI effects in order to help inform future intervention refinements that might include tailoring intervention messages to specific student characteristics.

2. METHODS

2.1. Design and procedures

Within the World Health Organization's (WHO) World Mental Health International College Student (WMH‐ICS) initiative, representative surveys examining the mental health of students were carried out in eight countries. StudiCare, the WMH‐ICS survey in Germany, was carried out at two German universities: the Friedrich‐Alexander‐University Erlangen‐Nuremberg (FAU) and the University of Ulm. The current experiment was embedded within these surveys.

All students of these universities received a personalized e‐mail invitation to participate in a student survey at the beginning of the semester. The e‐mail contained a short explanatory message as well as a personalized link to the questionnaire. If the student was over the age of 18 years and gave their consent to participate, they could proceed with completing the survey. Students received up to six reminder emails to encourage them to complete the survey. These recruitment procedures were the same for both experimental groups. The procedures were approved by the medical ethical committee of the FAU, all participants provided informed consent. The study was not preregistered in a clinical trial register.

In total, 11,169 students were invited to participate in the study, of which 2,895 started the WMH‐ICS survey (25.92%). Due to incremental drop‐out throughout the questionnaire, merely 1,374 first‐year (n = 1,036) and second‐year (n = 338) students finished the regular survey (drop‐out rate 52.54%) and were therefore eligible for the present study. All eligible students were randomized into either the intervention condition (IC; n = 664) or the control condition (CC; n = 710). Randomization was performed on an individual level by independent university administration staff and there was no way that researchers could foresee allocation of individual participants. Participants in the IC received the AFI whereas those in the CC did not receive any additional form of intervention. Intention to use mental health services in the next semester was subsequently assessed in both groups. Only students that completed the WMH‐ICS survey entered the study. The AFI was embedded in the survey WMH‐ICS survey, immediately following the regular survey questions. Intention to seek help (the primary outcome) was assessed as the last item of the WMH‐ICS survey in both study conditions.

2.2. Intervention

Embedded in the regular WMH‐ICS survey (for a description of procedures, see Auerbach et al., this issue), students randomized to the IC received (a) personalized feedback based on their symptom severity, (b) tailored psychoeducation, and (c) information about mental health services. Aiming to reduce the barrier of “low risk awareness” (Brewer et al., 2007; Eisenberg et al., 2007; Gulliver et al., 2010; Mojtabai et al., 2011), personalized feedback included information about their individual symptom severity in the symptom areas “depressed mood” (Kroenke & Spitzer, 2002); “worries, fears, and tensions” (Spitzer, Kroenke, Williams, & Löwe, 2006); “self‐injuring behavior” including suicidal thoughts, plans, attempts, and nonsuicidal self‐injury (adapted from SITBI; Nock, Holmberg, Photos, & Michel, 2007), and “substance use” (Kessler & Üstün, 2004; Saunders, Aasland, Babor, De la Fuente, & Grant, 1993). Based on whether students exceeded a specified cut‐off (yes/no) they received the information if their symptom severity was either low or above average in each of the assessed areas. This part of the AFI aimed to reduce the barrier of low‐risk awareness. Subsequently, psychoeducation on symptoms tailored on the individual symptom profile was provided to address the barrier of “low mental health literacy” (Biddle et al., 2007; Gulliver et al., 2010) and “limited outcome expectancies” (Eisenberg et al., 2007; Mitchell & Gordon, 2007) when addressing individual symptoms. A total of 14 different feedback profiles, based on type of symptom area as well as the number of symptom clusters in which the participants exceeded the cut‐off (0–4) were created. Psychoeducation included information about symptoms, frequency in the student population, the potential of reducing symptoms, and increasing well‐being using psychological interventions. Finally, information about mental health services were provided to address potential barriers of “insufficient knowledge about mental health services” (Gulliver et al., 2010) and “ease of access” (Gulliver et al., 2010). These information were not tailored to the symptom profile and included services available through the university, in community care (general practitioner, psychotherapist, psychiatrist or, inpatient clinic), as well as locally independent services such as mental health crisis hotlines and Internet‐based mental health interventions offered within the German WMH‐ICS initiative (http://www.studicare.com). Special emphasis was given to use positive, encouraging, and nonstigmatizing wording.

The AFI was presented in the form of a PDF file that the participants could download. The majority was in written form, but the personalized feedback was given by a customized graphic representation of the student's symptom severity. There was no technical way to control whether the participants had read the (entire) feedback before they answered the final question.

2.3. Measures/outcomes

2.3.1. Primary outcome

The primary outcome was the intention to use psychological interventions in the next semester, operationalized through the question: “How likely would you be to use any services regarding mental health (e.g., Internet‐based intervention, psychological counseling, family doctor, and psychotherapy) in the next semester?”

2.3.2. Moderators

The following variables were assessed as potential effects modifiers:

Lifetime and 12‐month mental disorders

Major depressive episode (MDE), generalized anxiety disorder (GAD), panic disorders (PD), broad mania, and drug abuse or dependence were assessed using the validated self‐report screening scales (Composite International Diagnostic Interview Screening Scales) of the widely used Composite International Diagnostic Interview (CIDI) scales (Kessler et al., 2013; Kessler & Üstün, 2004). These scales correlate highly with blinded clinical diagnoses based on the Structured Clinical Interview for DMS‐IV (SCID‐IV; First, Spitzer, Gibbon, & Williams, 1994), with a ranging area under the curve (AUC) from 0.70 to 0.78 (Kessler et al., 2013; Kessler, Calabrese, et al., 2013). Alcohol abuse or dependence were assessed using a version of the alcohol use disorders identification test (AUDIT; Saunders et al., 1993), with a total score of 16+ or a score 8–15 with 4+ on the AUDIT dependence questions as a definition for alcohol use disorder (Babor, Higgins‐Biddle, Saunders, & Monteiro, 2001). The concordance of the AUDIT with clinical diagnoses is in the range of AUC = 0.78 to 0.91 (Reinert & Allen, 2002).

Suicidal thoughts and behaviors

Lifetime and 12‐month suicidal thoughts and behaviors (STBs) were assessed using a modified version of the Columbia Suicidal Severity Rating Scale (Posner et al., 2011). The respective questions were: “Did you ever wish you were dead or would go to sleep and never wake up?” and “Did you ever in your life have thoughts of killing yourself?” for suicidal ideation, “Did you ever think about how you might kill yourself (e.g., taking pills or shooting yourself) or work out a plan of how to kill yourself?” for suicide plans, and “Have you ever made a suicide attempt (i.e., purposefully hurt yourself with at least some intent to die)?” for attempted suicide.

Nonsuicidal self‐injury

Nonsuicidal self‐injury (NSSI) was assessed using an adapted version of the Self Injurious Thoughts and Behaviors Interview (Nock et al., 2007) using the items “Did you ever do something to hurt yourself on purpose, without wanting to die (e.g., cutting yourself, hitting yourself, or burning yourself)?”, “About how old were you the very first time you did something to hurt yourself on purpose, without wanting to die?”, “About how many times in your life did you do something to hurt yourself on purpose, without wanting to die?”, and “How many times in the past year did you do something to hurt yourself on purpose, without wanting to die?”

Subjective health

Mental Health as well as physical health was rated from brilliant to bad (1 = brilliant, 2 = very good, 3 = good, 4 = acceptable, 5 = bad). Besides the average days per month (during last year) that were lost due to health, the subjective high energy level in the last month was assessed (1 = all or most of the time, 2 = sometimes, 3 = almost never or never).

Sociodemographics

Age (categorized into three categories [18 years/19 year/20 or more years old]), gender (male, female, transgender [male‐to‐female, female‐to‐male], other), family status (single, “in a relationship”, “married or relationship with common household”, “divorced without a new relationship”, and “divorced with a new relationship”), sexual orientation (categorized into heterosexual with no same‐sex attraction, heterosexual with some same‐sex attraction, nonheterosexual without same‐sex sexual intercourse, nonheterosexual with same‐sex sexual intercourse, and asexual), and religious preference (Christian, Jewish, Muslim, another, or none).

College‐related variables

Full‐time/part time student status, study subject, current semester (first‐year students vs. older students), expected to work alongside university (yes/no), self‐perceived academic performance in high school relative to peers from top 5% to bottom 10%, and categorized in upper and bottom half). Additionally, participants were asked about their reason to go to university, examining the most important one as well. Based on the results of a tetrachoric factor analysis, participants were categorized into those whose most important reason to go to university was intrinsic (i.e., achieving a degree, enjoying and studying, studying a subject that really interested them, improving job prospects generally, and training for specific type of job) or extrinsic (i.e., family wanted them to go, friends are going, teachers advised them, and did not want to get a job right away).

Treatment utilization

Lifetime and current utilization of psychological counseling, psychotherapy, and medication for any emotional or substance use problem was assessed by asking participants whether they ever received psychological counseling, psychotherapy, or medication for an emotional or substance problem, as well as the age of the first and last time they received medication or counseling (Kessler & Üstün, 2004; Ursano et al., 2014).

Intention to use mental health service

Intention to utilize mental health services in case of future emotional problems was assessed using items from the Army study to assess risk and resilience in servicemembers (Ursano et al., 2014), asking participants “If during this coming school year, you developed an emotional problem that caused you a lot of distress and interfered with your school work, how likely would you be to go to the student Counseling Center for help?”, “How likely would you be to go somewhere else for help, like to your doctor, a mental health professional, or religious advisor?” (definitely would go, probably would go, might or might not go, probably would not go, definitely would not go).

Barriers of treatment utilization

If participants indicated that they “definitely would not go” to seek help, they were asked about potential reasons: “If you decided not to seek help if you developed such a problem, how important do you think each of these would be as reasons for not seeking help?”. Those were: “You are not sure available treatments are very effective”; “You would want to handle the problem on your own”; “You would be too embarrassed”; “You would talk to friends or relatives instead”; ”You think it costs too much money”; “You are unsure of where to go or who to see”; “You anticipate problems with time, transportation, or scheduling”; “You are afraid it might harm your school or professional career”; “You are afraid of different treatment from others”; and “Other reasons” (1 = very important; 2 = important; 3 = moderately important; 4 = somewhat important; 5 = unimportant). Additionally, barriers for which students indicated they are “very important” to “important” were categorized as “relevant” (yes), all the others as nonrelevant (no). A total barriers sum score was calculated.

Readiness to change

The readiness to change potential emotional or substance‐related problems were assessed using five items related to the stages of change model (Aad, Miller, & Tonigan, 1996) “How would you rate your readiness or willingness to change any emotional or substance use problems you are experiencing at this time”; Stage 1 = I do not have a problem that I need to change; Stage 2 = I have a problem, but I am not yet sure I want to take action to change it; Stage 3: I have a problem and I intend to address it; Stage 4 = I have a problem and I already am working actively to change it; Stage 5 = I had a problem but I have addressed it and things are better now (yes/no).

2.4. Analyses

All analyses were conducted using SPSS Version 25 (IBM Corp., 2017). The sample size was not priory restricted, the achieved sample size allowed us to detect significant differences between the groups of d = 0.15, with a power of 80%. To account for nonresponse to the survey invitation, participants were weighted using propensity scores based on sociodemographic and college‐related variables made available by college officials (Groves & Couper, 1998). Because only participants who completed the WMH‐ICS survey items were randomized, no missing data occurred and thus no imputation method was used. Differences between the IC and the CC in intentions to utilize mental health services in university and community care in the next semester were examined using a t‐test. To identify potential intervention effect modifiers in an explorative manner, moderator analyses were performed using a series of linear multiple regression analyses with bootstrapped 95% CI (5,000 bootstrap samples), utilizing the SPSS macro PROCESS 3.0 (Hayes, 2013). Moderation variables were neither standardized nor mean‐centered before the analysis, because it doesn't impact the moderation effect and could possibly harm the interpretation if dichotomous variables are modified before the analysis. (Hayes, 2013). Although moderator analyses were intended to be exploratory, no alpha adjustment was made for multiple testing in order to avoid false‐negative findings (Kraemer, Wilson, Fairburn, & Agras, 2002). Cohen's d's and 95% CI were calculated comparing the means and standard deviations of the IC and CC on the primary outcome. According to Cohen, d = 0.2 can be considered a small effect, d = 0.5 a medium, and d = 0.8 a large effect (Cohen, 1992). To increase interpretability, the number needed to be treated (NNT) indicating the number of participants having received the AFI to generate one additional student to intend to use mental health services was calculated based on the formula of Kraemer and Kupfer (2006). Statistical significance in all analyses was set at α < 0.05 (two‐sided).

3. RESULTS

3.1. Sample description

In total and after weighting, 1,375 survey respondents (671 IC, 704 CC) were included in the analyses. Descriptive characteristics of the sample were similar across the IC and CC subgroups. (Table 1) In the total sample, 23.7% of participants stated a nonheterosexual sexual orientation, 64.8% answered that they were currently in a relationship, and the vast majority stated that they were of German nationality (91.2%). The majority of survey respondents had a Christian religious preference (70.6%), with another 6% having another religious preference, and 23.4% no religious preference. Nearly all participants studied in full‐time (98.7%), with science (16.2%), engineering (14.6%) and teaching (12.8%) being the most popular study subjects. 85.7% of students ranked themselves as being in the upper half of the class on their academic achievements and 76% expected to work in a student job at some point in the school year. There was much higher endorsement of intrinsic than extrinsic reasons for going to college, with an average of 3.6 intrinsic reasons and 1.2 extrinsic reasons.

Table 1.

Sociodemographic and college‐related variables. (n = 1,375)

Distribution in percent
Intervention condition Control condition Overall
Sociodemographic
Age 18 years 17.4 15.9 16.6
Age 19 years 23.8 26.2 25.0
Age 20 years 58.8 58.0 58.4
Being female 48.8 52.1 50.50
Being currently in a relationship 40.0 39.5 39.7
German nationality 89.9 92.5 91.2
experienced discrimination 9.6 12.1 10.9
being nonheterosexual 15.3 12.2 13.7
Religion
Christian 68.4 72.8 70.6
Another religion 6.3 5.6 6.0
No religion 25.3 21.6 23.4
College‐related information
Studying full‐time 98.7 98.6 98.7
First year student 60.6 65.1 62.9
Expected to work on a student job 74.8 78.4 76.0
Self‐reported ranking high school bottom 50% 15.9 12.8 14.3
Self‐reported ranking high school top 50% 84.1 87.2 85.7
Most important reason to go to college extrinsic 4.6 7.8 6.2
Open in a new tab

Note. The results are based on weighting using propensity scores based on sociodemographic and college‐related variables made available by college officials.

As can be seen in Table 2, the burden of mental disorders in the sample was quite high and, as with the background characteristics examined in Table 1, quite comparable among students in the IC and CC. Approximately, one‐third of all students experienced at least one lifetime (35.2%) and 12‐month (32.3%) mental disorder. The most often reported 12‐month disorder was MDE (20.7%) followed by GAD (13.3%) and PD (6.5%). Twelve‐month STBs and NSSI were common as well with one in five students of the sample reporting suicidal ideation (21.1%), one in 10 suicidal plans (10.6%), 0.4% a suicidal attempt, and 6.8% NSSI.

Table 2.

Clinical characteristics mental health and suicidal thoughts and behaviors (n = 1,375)

Distribution in percent or mean (SD)
Intervention condition Control condition Overall
12‐month lifetime 12‐month lifetime 12‐month lifetime
Type of mental disorder
Major depressive episode 22.4 24.4 19.9 21.1 20.7 22.7
Generalized anxiety disorder 14.3 15.4 12.2 13.5 13.3 14.4
Panic disorder 6.2 6.3 6.8 7.2 6.5 6.7
Broad mania 2.4 4.0 3.7 4.1 3.1 4.0
Alcohol abuse or dependence 2.8 3.0 2.9 2.9 2.8 2.9
Drug abuse or dependence 3.6 5.4 2.6 4.5 3.1 4.9
Number of mental disorders
None 67.6 64.5 67.8 65.0 67.7 64.8
Exactly one mental disorder 19.1 20.8 20.4 20.8 19.8 20.8
Exactly two mental disorders 8.6 8.8 9.1 10.8 8.9 9.8
Three or more mental disorders 4.7 5.8 2.1 3.4 3.7 5.6
Mean (SD) 0.52 (0.89) 0.58 (0.96) 0.47 (0.79) 0.54 (0.84) 0.49 (0.84) 0.56 (0.90)
Suicidal thoughts and behaviors
Never 64.9 31.0 70.8 37.1 67.9 34.2
Ideation only 22.3 42.0 20.1 39.9 21.1 40.9
Plan, no attempts 12.4 23.1 8.8 20.1 10.6 21.5
Planned or unplanned attempt 0.4 3.9 0.3 2.9 0.4 3.4
NSSI 7.8 23.5 5.8 18.4 6.8 20.9
Subjective Health a
Self‐rated mental health 2.83 (1.093) 2.70 (0.999) 2.77 (1.047)
Self‐rated physical health 2.63 (0.865) 2.65 (0.916) 2.64 (0.891)
Open in a new tab

Note. NSSI: nonsuicidal self injury SD: standard deviation.

a

Lower score equal better subjective health (1 = “brilliant”; 5 = “bad”)

We also examined lifetime treatment for emotional or substance‐related problem. (See online Table S1) and found that the distribution was quite comparable for students in the IC and CC conditions. Around 12.6% reported lifetime utilization of psychotherapy and 7.3% medication for emotional problems. Regarding intention to use future mental health services, 23.2% indicated that they would never use any mental health service in case of a future emotional or substance‐related problem. The barriers to utilization among students who said they would not definitely use mental health services in case of a future emotional problems were primarily attitudinal, including the wish to handle their problems on their own (64.7%), preference to talk to friends or relatives rather than a mental health professional (50.8%), fear of a different treatment from others (39.4%), embarrassment (33.6%), or not knowing where to go for help (28.7%).

Concerning readiness to change, 65.8% could be categorized in Stage 1 (I do not have a problem that I need to change), 10% in Stage 2 (I have a problem, but I am not yet sure I want to take action to change it), 6.9% in Stage 3 (I have a problem and I intend to address it), 9.5% in Stage 4 (I have a problem and I already am working actively to change it), and 7.8% in Stage 5 (I had a problem but I have addressed it and things are better now).

3.2. Primary outcome

There was a significant main effect of the AFI, with students randomized to receive the IC indicating significantly higher intentions to seek help in the next semester compared with participants of the CC (p = 0.024). With d = 0.12 (95% CI: 0.02 to 0.23) in the total sample, the magnitude of the effect size was small according to Cohen's criteria. The NNT to achieve one additional student intending to utilize mental health service in the next semester was 14.71 (95% CI: 83.33 to 7.69). For details see Table 3.

Table 3.

Primary outcome analysis, t‐Test Cohen's d, and NNT (n = 1,375)

Outcome Difference in means (95% CI) Cohen's d (95% CI) NNT (95% CI) two‐sample‐t (1374) IC mean (SD) CC mean (CC)
Intention to use psychological intervention in the next semester 0.1198 (0.02; 0.22) 0.12 (0.02; 0.23) 14.71 (83.33; 7.69) 2.246 (p = 0.024) 2.249 (0.9729) 2.130 (0.9930)
Open in a new tab

Note. CC: control condition; CI: confidence interval; IC: intervention condition; NNT: numbers needed to be treated; SD: standard deviation. The results are based on weighting using propensity scores based on sociodemographic and college‐related variables made available by college officials.

3.3. Moderator analysis

Lifetime (p = 0.0459) and 12‐month (p = 0.0493) DSM‐IV panic disorder, self‐reported physical health (p = 0.0067), being or not being in Stage 4 of the stages of change model (p = 0.0487), as well as sexual orientation (p = 0.01555) were found to significantly moderate the effect of the AFI on the intention to use psychological interventions in the next semester, when compared with the CC.

Significantly, larger effects were observed for those students that fulfilled the criteria for lifetime (d = 0.34; 95% CI: ‐0.08 to 0.7) and 12‐month panic disorder (d = 0.32; 95% CI: ‐0.10 to 0.74) compared with those without lifetime (d = 0.11; 95% CI: 0.00 to 0.22) or 12‐month panic disorder (d = 0.11; 95% CI: 0.00 to 0.22). NNT to achieve one additional student to intend to seek help was 5.26 for lifetime and 5.56 for 12‐month panic disorder, compared with the control group.

Students with lower self‐reported physical health (d = 0.37; 95% CI: ‐0.77 to 1.51; NNT = 4.85; for “bad”) showed significantly larger effects compared with those with higher self‐reported physical health (d = −0.01; 95% CI: ‐0.36 to 0.34; for “brilliant”).

Moreover, although no effects of the AFI were found for students that stated, “to have a problem and are already working actively to change it” (Stage 4 stages of change; d = −0.25; 95% CI: ‐0.59 to 0.10), significant findings were found for those students that indicated not to be in this stage (d = 0.18; 95% CI: 0.06 to 0.29; NNT = 9.8). Larger effects were also found for nonheterosexual oriented students (d = 0.38; 95% CI: 0.08 to 0.67; NNT = 4.72) compared with those with a heterosexual orientation (d = 0.06; 95% CI: ‐0.06 to 0.17; NNT = 29.41).

All other variables did not significantly moderate the AFI's effect on intention to utilize mental health services in the next semester (ranging from p = 0.08 for Stage 2 of the stages of change to p = 0.99 for 12‐month suicidal ideation). There was a trend (P < 0.10) for 12‐month MDE, lifetime alcohol abuse or dependence, being in stage two of the stages of change (I have a problem, but I am not yet sure I want to take action to change it; see Table S2).

4. DISCUSSION

This study found significant effects of an automated, simple AFI on students' intention to seek mental health service in the next semester compared with an untreated CC. Explorative moderator analyses indicated the AFI is more effective among students with lifetime or 12‐month PD compared with those without, and among students with lower compared with higher physical health, and those indicating not to already be working actively on the problem, or nonheterosexual students compared with heterosexual ones. These findings are comparable with previous studies in other target groups showing that it may be possible to increase the intention to seek help (Albada et al., 2009; Brouwers et al., 2011; Curry et al., 1993; Gonzales et al., 2002; Gulliver et al., 2012; Hadlaczky et al., 2014; Hovick et al., 2014; Riper et al., 2009; Sheridan et al., 2011) and that the effect may be determined by participants' characteristics (Batterham et al., 2016; Baumeister et al., 2014). A brief video‐based AFI focusing on reducing other attitudinal barriers, including low perceived effectiveness, perceived stigma, perceived ease of use and access to the intervention, for example, was found to be effective in increasing intention to use psychological interventions in case of future emotional problems in individuals already experiencing depressive symptoms (Baumeister et al., 2014, 2015; Ebert et al., 2015) but was proved to be ineffective in those not experiencing symptoms at the moment (Baumeister et al., 2014).

Despite being statistically significant, the magnitude of the effects found in this study was small. Potential reasons for the low effects might include it is generally not possible to increase the willingness to use mental health services over a certain extent, or only in very specific subgroups and in others, not. It might also be the case that not all students read the provided text, which was offered in a PDF to download. There was no technical way to control whether this was done. If this should be the case, the presented results might be an underestimation of the true effect. It is also noteworthy that the AFI was provided after an intense assessment of mental health and risk factors resulting possibly in some overload, which would be expected to reduce focus on the additional material. It is possible that the effects might be different if the AFI was combined with an assessment of only those questions relevant for the feedback. An alternative explanation for the low effects could be the insufficient tailoring to relevant characteristics given that it is likely that not all individuals respond to the same motivational strategy. Reviews indicate, for example, that men and women have specific barriers for the uptake of preventive interventions as well as different preferences concerning the foci of intervention approaches (Addis & Mahalik, 2003; Seidler, Dawes, Rice, Oliffe, & Dhillon, 2016; Spendelow, 2015a, 2015b; Yousaf, Grunfeld, & Hunter, 2015). Motivation psychology also indicates a distinction between individuals with a promotion‐oriented (“promoters”) or a prevention‐oriented (“preventers”) regulatory focus (Crowe & Higgins, 1997). Although “promoters” are considered to be motivated by advancement and accomplishment (e.g., they go running to feel good; Latimer et al., 2008; Spiegel, Grant‐Pillow, & Higgins, 2004), “preventers” are considered to be motivated by security needs to avoid adverse outcomes (e.g., they go running to avoid illness). Therefore, it might be beneficial for AFIs to consider such disparate health‐related motives as well as other potential relevant tailoring factors such as gender, cultural background, and treatment history. This is supported by metaanalytic findings showing that tailoring can increase the effects of printed health behavior change interventions, although the effects of tailoring are moderated by type of health behavior (Noar, Benac, & Harris, 2007). Moreover, because the AFI is automated, scalable, and easy to implement, it could be assumed that such an approach may nevertheless be a low‐cost method of having a meaningful impact at the population level.

In summary, the present findings indicate that a “one size fits all” is unlikely to be effective for everyone and that future approaches should target to important individual characteristics. This assumption is supported by the moderator analyses indicating that some students experience larger effects than others as well as findings that show that treatment utilization (Bruffaerts et al., this issue) and the intention to seek help in case of future emotional problems is associated with a range of factors, such as the preference to solve problems alone, feeling too embarrassed, and fulfilling the criteria for 12‐month MDD (Ebert et al., this issue). Subsequent analyses of the current data are needed to refine the preliminary moderator analysis to investigate the possible existence of complex interactions and to adjust for the likelihood of overfitting in our current exploratory analysis. The comparatively large sample size based on the experiment being embedded in a survey of all entering students is a great advantage in this regard in that complex moderator analyses to search for multivariate profiles associated with high intervention response require large samples (Luedtke, Sadikova, & Kessler, in press).

In addition, an expanded series of future cross‐national WMH‐ICS experiments are needed to examine more sophisticated AFI versions that tailor the intervention to participant characteristics that might plausibly be expected to influence intervention effects. These interventions might also take into consideration service‐related characteristics and include more interactive elements such as videos. It is also certain that the development of such individually‐tailored AFIs will require an iterative series of successive experiment that will take a number of years to perfect, making the ongoing WMH‐ICS initiative a perfect context in which to implement this program of research.

Results of the current study should be seen in the light of a range of limitations. First, the implementation of the study following an intensive questionnaire (up to 150 items) needed for other research purposes within the WMH‐ICS initiative contributed to a high dropout rate before entering the study. Moreover, the initial WMH‐ICS screener to which the students were invited was labeled as research, and the direct benefit for each student remained unclear. This might have led to selection bias, and therefore generalizability might be limited. Future studies should, therefore, investigate the reach and effectiveness of the investigated AFI when delivered solely with the items necessary for providing personal risk feedback and potential benefits for the students are clearly stated. Second, we only focused on intention to use mental health services instead of actual use. Although intention is the best available proxy for actual use (Eccles et al., 2006), it cannot directly being translated to actual utilization, and the so‐called “intention‐behavior gap” is widely known (Sniehotta, Scholz, & Schwarzer, 2005). Given the small effect size in the overall analysis, it is at least questionable whether the current AFI lead to an actual increase in mental health service utilization. As noted above, we plan to address both of these limitation in future WMH‐ICS analyses that focus on longitudinal associations and attempt to influence actual help‐seeking rather than hypothetical help‐seeking. Third, we conducted a large number of statistical tests in the moderator analyses. It may be the case that some of the significant findings occurred due to chance and future studies along the lines noted above are needed to confirm these results. Fourth, although a broad range of moderators was examined, the study was underpowered to explore moderators with small subsamples and low effect sizes. Limitations of sample size also resulted in the inability to examine multivariate profiles of moderators or potentially complex interactions among moderators. As noted above, we plan to address this problem in a future cross‐national expansion of AFI interventions in WMH‐ICS that investigate the possibility that different barriers are relevant for different students with disorders, different motivation statuses, different experience levels, different study situations, and more. The investigation of these possibilities will require more complex and contemporary study designs such as the multiphase optimization strategy (Collins, 2018) or sequential multiple assignment randomized trials (Collins, Murphy, & Strecher, 2007). In such designs, it is possible to determine the incremental value of specific intervention components and tailoring strategies. This can be done by example a series of fractional factorial designs, instead of only evaluating the average effect of packages of different intervention components as it is often being done in psychological outcome research. Building such studies on multinational collaborations such as the WMH‐ICS initiative that conducts similar studies across multiple countries allows researchers to achieve large enough sample sizes that are needed to realize complex evaluation designs.

Implications of the present study include that screening of mental health symptoms and providing feedback about symptom severity, psychoeducation on relevant symptoms, and information about available potentially effective treatment is not sufficient for many students. Effects of the AFI differ widely between students; and although some one‐way moderators such as 12‐month and lifetime prevalence of panic disorder, physical health, and sexual orientation could be identified, it is likely that more complex relationships affect the mechanism of action of acceptance facilitating interventions. Much more research is needed to better understand how individuals at risk for mental health disorders can be motivated to participate in mental health interventions.

WHO WMH‐ICS Collaborators

Australia: Penelope Hasking (PI), Mark Boyes (School of Psychology, Curtin University); Glenn Kiekens (School of Psychology, Curtin University, RG Adult Psychiatry KU Leuven, Belgium); Belgium: Ronny Bruffaerts (PI), Philippe Mortier, Koen Demyttenaere, Erik Bootsma (KU Leuven); France: Mathilde Husky (PI), Université de Bordeaux; Viviane Kovess‐Masfety, Ecole des Hautes Etudes en Santé Publique; Germany: David D. Ebert (PI), Matthias Berking, Marvin Franke, Fanny Kählke (Friedrich‐Alexander University Erlangen Nuremberg); Harald Baumeister, Ann‐Marie Küchler (University of Ulm); Hong Kong: Arthur Mak (PI), Chinese University of Hong Kong; Mexico: Corina Benjet (PI), Guilherme Borges, María Elena Medina‐Mora (Instituto Nacional de Psiquiatría Ramón de la Fuente); Adrián Abrego Ramírez, (Universidad Politécnica de Aguascalientes); Anabell Covarrubias Díaz, (Universidad La Salle Noroeste); Ma. Socorro Durán, Gustavo Pérez Tarango, María Alicia Zavala Berbena (Universidad De La Salle Bajío); Rogaciano González González, Raúl A. Gutiérrez‐García (Universidad De La Salle Bajío, campus Salamanca); Alicia Edith Hermosillo de la Torre, Kalina Isela Martínez Martínez (Universidad Autónoma de Aguascalientes); Sinead Martínez Ruiz (Universidad La Salle Pachuca); Netherlands: Pim Cuijpers (PI), Eirini Karyotaki (VU University Amsterdam); Northern Ireland: Siobhan O'Neill (PI),(Psychology Research Institute, Ulster University); Tony Bjourson, Elaine Murray, (School of Biomedical Sciences, Ulster University); South Africa: Dan J. Stein (PI), (Department of Psychiatry and Mental Health, MRC Unit on Risk & Resilience in Mental Disorders, University of Cape Town); Christine Lochner, Janine Roos, Lian Taljaard, (MRC Unit on Risk & Resilience in Mental Disorders, Department of Psychiatry, Stellenbosch University); Jason Bantjes, Wylene Saal, (Department of Psychology, Stellenbosch University); Spain: The UNIVERSAL study Group (Universidad y Salud Mental) includes: Jordi Alonso (PI), Gemma Vilagut, (IMIM‐Hospital del Mar Medical Research Institute/CIBERESP); Itxaso Alayo, Laura Ballester, Gabriela Barbaglia Maria Jesús Blasco, Pere Castellví, Ana Isabel Cebrià, Carlos García‐Forero, Andrea Miranda‐Mendizábal, Oleguer Parès‐Badell (Pompeu Fabra University); José Almenara, Carolina Lagares (Cadiz University), Enrique Echeburúa, Andrea Gabilondo, Álvaro Iruin (Basque Country University); María Teresa Pérez‐Vázquez, José Antonio Piqueras, Victoria Soto‐Sanz, Jesús Rodríguez‐Marín (Miguel Hernández University); and Miquel Roca, Margarida Gili, Margarida Vives (Illes Balears University); USA: Randy P. Auerbach (PI), (Columbia University); Ronald C. Kessler (PI), (Harvard Medical School); Jennifer G. Green, (Boston University); Matthew K. Nock, (Harvard University); Stephanie Pinder‐Amaker, (McLean Hospital and Harvard Medical School); Alan M. Zaslavsky (Harvard Medical School).

FUNDING INFORMATION

The funding sources had no role in the design and conduct of the study; collection, management, analysis, interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

CONFLICTS OF INTEREST

Dr. Ebert reports to have received consultancy fees/served in the scientific advisory board from several companies such as Sanofi, Novartis, Minddistrict, Lantern, Schoen Kliniken, and German health insurance companies (BARMER, Techniker Krankenkasse). He is also stakeholder of the Institute for health training online (GET.ON), which aims to implement scientific findings related to digital health interventions into routine care.

In the past 3 years, Dr. Kessler received support for his epidemiological studies from Sanofi Aventis; was a consultant for Johnson & Johnson Wellness and Prevention, Sage Pharmaceuticals, Shire, Takeda; and served on an advisory board for the Johnson & Johnson Services Inc. Lake Nona Life Project. Kessler is a co‐owner of DataStat, Inc., a market research firm that carries out health care research.

Supporting information

Table S1. Baseline motivational and acceptance‐related characteristics (n = 1375)

Table S2. Overview of all results from a series of linear multiple regression analyses with bootstrapped 95%‐confidence intervals (5000 bootstrap samples).

Click here for additional data file. (2.5MB, docx)

ACKNOWLEDGMENTS

Funding to support this initiative was received from the National Institute of Mental Health (NIMH) R56MH109566 (R. P. A.), and the content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Mental Health or NIMH; the Belgian Fund for Scientific Research (11N0514N/11N0516N/1114717N) (P. M.), the King Baudouin Foundation (2014‐J2140150‐102905; R. B.), and Eli Lilly (IIT‐H6U‐BX‐I002; R. B., P. M.); BARMER, a health care insurance company, for project StudiCare (D. D. E.); ZonMw (Netherlands Organisation for Health Research and Development; grant number 636110005) and the PFGV (PFGV; Protestants Fonds voor de Geestelijke Volksgezondheid) in support of the student survey project (P. C.); South African Medical Research Council and the Ithemba Foundation (D. J. S.); Fondo de Investigación Sanitaria, Instituto de Salud Carlos III ‐ FEDER (PI13/00343), ISCIII (Río Hortega, CM14/00125), ISCIII (Sara Borrell, CD12/00440), Ministerio de Sanidad, Servicios Sociales e Igualdad, PNSD (Exp. 2015I015); DIUE Generalitat de Catalunya (2017 SGR 452), FPU (FPU15/05728; J. A.); Fondo de Investigación Sanitaria, Instituto de Salud Carlos III‐FEDER (PI13/00506; G. V.); European Union Regional Development Fund (ERDF) EU Sustainable Competitiveness Programme for Northern Ireland, Northern Ireland Public Health Agency (HSC R&D), and Ulster University (T. B.); Consejo Nacional de Ciencia y Tecnología (CONACyT) grant CB‐2016‐01‐285548 (C. B.). The WMH‐ICS initiative is carried out as part of the WHO World Mental Health (WMH) Survey Initiative. The WMH survey is supported by the National Institute of Mental Health (NIMH) R01MH070884, the John D. and Catherine T. MacArthur Foundation, the Pfizer Foundation, the United States Public Health Service (R13‐MH066849, R01‐MH069864, and R01‐DA016558), the Fogarty International Center (FIRCA R03‐TW006481), the Pan American Health Organization, Eli Lilly and Company, Ortho‐McNeil Pharmaceutical, GlaxoSmithKline, and Bristol‐Myers Squibb (RCK). None of the funders had any role in the design, analysis, interpretation of results, or preparation of this paper.

We thank the staff of the WMH Data Collection and Data Analysis Coordination Centres for assistance with instrumentation, fieldwork, and consultation on data analysis. A complete list of all within‐country and cross‐national WMH publications can be found at http://www.hcp.med.harvard.edu/wmh/.

Ebert DD, Franke M, Kählke F, et al. Increasing intentions to use mental health services among university students. Results of a pilot randomized controlled trial within the World Health Organization's World Mental Health International College Student Initiative. Int J Methods Psychiatr Res. 2019;28:e1754 10.1002/mpr.1754

REFERENCES

  1. Aad, G. , Miller, W. R. , & Tonigan, J. S. (1996). Assessing drinkers' motivation for change: The Stages of Change Readiness and Treatment Eagerness Scale (SOCRATES). Psychology of Addictive Behaviors, 10(2), 81–89. 10.1037/0893-164X.10.2.81 [DOI] [Google Scholar]
  2. Addis, M. E. , & Mahalik, J. R. (2003). Men, masculinity, and the contexts of help seeking. The American Psychologist, 58(1), 5–14. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/12674814. 10.1037/0003-066X.58.1.5 [DOI] [PubMed] [Google Scholar]
  3. Albada, A. , Ausems, M. G. E. M. , Bensing, J. M. , & van Dulmen, S. (2009). Tailored information about cancer risk and screening: A systematic review. Patient Education and Counseling, 77(2), 155–171. 10.1016/j.pec.2009.03.005 [DOI] [PubMed] [Google Scholar]
  4. Auerbach, R. P. , Alonso, J. , Axinn, W. G. , Cuijpers, P. , Ebert, D. D. , Green, J. G. , … Bruffaerts, R. (2016). Mental disorders among college students in the World Health Organization World Mental Health Surveys. Psychological Medicine, 46(14), 2955–2970. 10.1017/S0033291716001665 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Babor, T. , Higgins‐Biddle, J. C. , Saunders, J. B. , & Monteiro, M. G. (2001). The alcohol use disorders identification test: Guidelines for use in primary care ( ed., Vol. 26) (pp. 324–329). Geneva: World Health Organization. doi: 10.1177/0269881110393051. [DOI] [Google Scholar]
  6. Batterham, P. J. , Calear, A. L. , Sunderland, M. , Carragher, N. , & Brewer, J. L. (2016). Online screening and feedback to increase help‐seeking for mental health problems: Population‐based randomised controlled trial. British Journal of Psychiatry Open, 2(1), 67–73. 10.1192/bjpo.bp.115.001552 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Baumeister, H. , Nowoczin, L. , Lin, J. , Seifferth, H. , Seufert, J. , Laubner, K. , & Ebert, D. D. (2014). Impact of an acceptance facilitating intervention on diabetes patients' acceptance of Internet‐based interventions for depression: A randomized controlled trial. Diabetes Research and Clinical Practice, 105(1), 30–39. 10.1016/j.diabres.2014.04.031 [DOI] [PubMed] [Google Scholar]
  8. Baumeister, H. , Seifferth, H. , Lin, J. , Nowoczin, L. , Lüking, M. , & Ebert, D. D. (2015). Impact of an acceptance facilitating intervention on patients' acceptance of Internet‐based pain interventions: A randomized controlled trial. The Clinical Journal of Pain, 31(6), 528–535. 10.1097/AJP.0000000000000118 [DOI] [PubMed] [Google Scholar]
  9. Biddle, L. , Donovan, J. , Sharp, D. , & Gunnell, D. (2007). Explaining non‐help‐seeking amongst young adults with mental distress: A dynamic interpretive model of illness behaviour. Sociology of Health & Illness, 29(7), 983–1002. 10.1111/j.1467-9566.2007.01030.x [DOI] [PubMed] [Google Scholar]
  10. Blanco, C. , Okuda, M. , Wright, C. , Hasin, D. S. , Grant, B. F. , Liu, S.‐M. , & Olfson, M. (2008). Mental health of college students and their non–college‐attending peers. Archives of General Psychiatry, 65(12), 1429 10.1001/archpsyc.65.12.1429–1437. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Brewer, N. T. , Chapman, G. B. , Gibbons, F. X. , Gerrard, M. , McCaul, K. D. , & Weinstein, N. D. (2007). Meta‐analysis of the relationship between risk perception and health behavior: The example of vaccination. Health Psychology. Brewer, Noel T.: UNC Schol of Public Health, 306 Rosenau Hall, CB#7440, Chapel Hill, NC, US, 27599, ntb1@unc.edu: American Psychological Association. doi: 10.1037/0278-6133.26.2.136, 136, 145 [DOI] [PubMed]
  12. Brouwers, M. C. , De Vito, C. , Bahirathan, L. , Carol, A. , Carroll, J. C. , Cotterchio, M. , … Wathen, N. (2011). What implementation interventions increase cancer screening rates? A Systematic Review. Implementation Science, 6(1), 111 10.1186/1748-5908-6-111 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Bruffaerts, R. , Mortier, P. , Kiekens, G. , Auerbach, R. P. , Cuijpers, P. , Demyttenaere, K. , … Kessler, R. C. (2018). Mental health problems in college freshmen: Prevalence and academic functioning. Journal of Affective Disorders, 225, 97–103. 10.1016/j.jad.2017.07.044 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Chan, M. K. Y. , Bhatti, H. , Meader, N. , Stockton, S. , Evans, J. , O'connor, R. C. , … Kendall, T. (2016). Predicting suicide following self‐harm: Systematic review of risk factors and risk scales. The British Journal of Psychiatry, Bjp‐Bp, 209, 277–283. 10.1192/bjp.bp.115.170050 [DOI] [PubMed] [Google Scholar]
  15. Clement, S. , Schauman, O. , Graham, T. , Maggioni, F. , Evans‐Lacko, S. , Bezborodovs, N. , … Thornicroft, G. (2015). What is the impact of mental health‐related stigma on help‐seeking? A systematic review of quantitative and qualitative studies. Psychological Medicine, 45(1), 11–27. 10.1017/S0033291714000129 [DOI] [PubMed] [Google Scholar]
  16. Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159. 10.1037/0033-2909.112.1.155 [DOI] [PubMed] [Google Scholar]
  17. Collins, L. M. (2018). Optimization of behavioral, biobehavioral, and biomedical interventions. Cham: Springer International Publishing. doi: 10.1007/978-3-319-72206-1 [DOI] [Google Scholar]
  18. Collins, L. M. , Murphy, S. A. , & Strecher, V. (2007). The multiphase optimization strategy (MOST) and the sequential multiple assignment randomized trial (SMART). American Journal of Preventive Medicine, 32(5), S112–S118. 10.1016/j.amepre.2007.01.022 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Cranen, K. , Veld, R. H. , in't Ijzerman, M. , & Vollenbroek‐Hutten, M. (2011). Change of patients' perceptions of telemedicine after brief use. Telemedicine Journal and E‐Health, 17(7), 530–535. 10.1089/tmj.2010.0208 [DOI] [PubMed] [Google Scholar]
  20. Crowe, E. , & Higgins, E. T. (1997). Regulatory focus and strategic inclinations: Promotion and prevention in decision‐making. Organizational Behavior and Human Decision Processes, 69(2), 117–132. 10.1006/obhd.1996.2675 [DOI] [Google Scholar]
  21. Cuijpers, P. , Sijbrandij, M. , Koole, S. L. , Andersson, G. , Beekman, A. T. , & Reynolds, C. F. 3rd. (2013). The efficacy of psychotherapy and pharmacotherapy in treating depressive and anxiety disorders: A meta‐analysis of direct comparisons. World Psychiatry: Official Journal of the World Psychiatric Association (WPA), 12(2), 137–148. 10.1002/wps.20038 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Curry, S. J. , Taplin, S. H. , Anderman, C. , Barlow, W. E. , & McBride, C. (1993). A randomized trial of the impact of risk assessment and feedback on participation in mammography screening. Preventive Medicine, 22(3), 350–360. 10.1006/pmed.1993.1029 [DOI] [PubMed] [Google Scholar]
  23. Demyttenaere, K. , Bruffaerts, R. , Posada‐Villa, J. , Gasquet, I. , Kovess, V. , Lepine, J. P. , … WHO World Mental Health Survey Consortium (2004). Prevalence, severity, and unmet need for treatment of mental disorders in the World Health Organization World Mental Health Surveys. JAMA, 291(21), 2581–2590. 10.1001/jama.291.21.2581 [DOI] [PubMed] [Google Scholar]
  24. Donker, T. , Griffiths, K. M. , Cuijpers, P. , & Christensen, H. (2009). Psychoeducation for depression, anxiety and psychological distress: A meta‐analysis. BMC Medicine, 7, 79 10.1186/1741-7015-7-79 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Ebert, D. D. , Berking, M. , Cuijpers, P. , Lehr, D. , Pörtner, M. , & Baumeister, H. (2015). Increasing the acceptance of internet‐based mental health interventions in primary care patients with depressive symptoms. A randomized controlled trial. Journal of Affective Disorders. 10.1016/j.jad.2015.01.056, 176, 9–17. [DOI] [PubMed] [Google Scholar]
  26. Eccles, M. P. , Hrisos, S. , Francis, J. , Kaner, E. F. , Dickinson, H. O. , Beyer, F. , & Johnston, M. (2006). Do self‐ reported intentions predict clinicians' behaviour: A systematic review. Implementation Science, 1(1), 28 10.1186/1748-5908-1-28 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Eisenberg, D. , Golberstein, E. , & Gollust, S. E. (2007). Help‐seeking and access to mental health care in a university student population. Medical Care, 45(7), 594–601. [DOI] [PubMed] [Google Scholar]
  28. Eisenberg, D. , Golberstein, E. , & Hunt, J. B. (2009). Mental health and academic success in college. The B.E. Journal of Economic Analysis & Policy, 9(1). 10.2202/1935-1682.2191 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. First, M. , Spitzer, R. , Gibbon, M. , & Williams, B. (1994). Structured clinical interview for Axis I DSM‐IV disorders. New York: New York State Psychiatric Instititute: Biometrics Research Department. [Google Scholar]
  30. Gonzales, J. M. , Tinsley, H. E. A. , & Kreuder, K. R. (2002). Effects of psychoeducational interventions on opinions of mental illness, attitudes toward help seeking, and expectations about psychotherapy in college students. Journal of College Student Development, 43(1), 51–63. [Google Scholar]
  31. Gould, M. S. , Veltin, D. , Kleinman, M. , Lucas, C. , Thomas, J. G. , & Chung, M. (2004). Teenagers' attitudes about coping strategies and help‐seeking behavior for suicidality. Journal of the American Academy of Child & Adolescent Psychiatry, 43(9), 1124–1133. 10.1097/01.chi.0000132811.06547.31 [DOI] [PubMed] [Google Scholar]
  32. Groves, R. M. , & Couper, M. P. (1998). Nonresponse in household interview surveys. Hoboken, NJ, USA: John Wiley & Sons, Inc. doi: 10.1002/9781118490082 [DOI] [Google Scholar]
  33. Gulliver, A. , Griffiths, K. M. , & Christensen, H. (2010). Perceived barriers and facilitators to mental health help‐seeking in young people: A systematic review. BMC Psychiatry, 10(1), 113 10.1186/1471-244X-10-113 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Gulliver, A. , Griffiths, K. M. , Christensen, H. , & Brewer, J. L. (2012). A systematic review of help‐seeking interventions for depression, anxiety and general psychological distress. BMC Psychiatry, 12(1), 81 10.1186/1471-244X-12-81 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Hadlaczky, G. , Hökby, S. , Mkrtchian, A. , Carli, V. , & Wasserman, D. (2014). Mental Health First Aid is an effective public health intervention for improving knowledge, attitudes, and behaviour: A meta‐analysis. International Review of Psychiatry, 26(4), 467–475. 10.3109/09540261.2014.924910 [DOI] [PubMed] [Google Scholar]
  36. Hayes, A. F. (2013). Introduction to mediation, moderation and conditional process analysis: A regression‐based approach. New York: The Guilford Press. [Google Scholar]
  37. Hovick, S. R. , Wilkinson, A. V. , Ashida, S. , de Heer, H. D. , & Koehly, L. M. (2014). The impact of personalized risk feedback on Mexican Americans' perceived risk for heart disease and diabetes. Health Education Research, 29(2), 222–234. 10.1093/her/cyt151 [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. IBM Corp (2017). IBM SPSS statistics for windows, version 25.0. Armonk, NY: IBM Corp. [Google Scholar]
  39. Jorm, A. F. , Korten, A. E. , Jacomb, P. A. , Christensen, H. , Rodgers, B. , & Pollitt, P. (1997). "Mental health literacy": A survey of the public's ability to recognise mental disorders and their beliefs about the effectiveness of treatment. The Medical Journal of Australia, 166(4), 182–186. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/9066546 [DOI] [PubMed] [Google Scholar]
  40. Kessler, R. C. , Calabrese, J. R. , Farley, P. A. , Gruber, M. J. , Jewell, M. A. , Katon, W. , … Wittchen, H.‐U. (2013). Composite international diagnostic interview screening scales for DSM‐IV anxiety and mood disorders. Psychological Medicine, 43(08), 1625–1637. 10.1017/S0033291712002334 [DOI] [PubMed] [Google Scholar]
  41. Kessler, R. C. , Santiago, P. N. , Colpe, L. J. , Dempsey, C. L. , First, M. B. , Heeringa, S. G. , … Ursano, R. J. (2013). Clinical reappraisal of the Composite International Diagnostic Interview Screening Scales (CIDI‐SC) in the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). International Journal of Methods in Psychiatric Research, 22(4), 303–321. 10.1002/mpr.1398 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Kessler, R. C. , & Üstün, T. B. (2004). The world mental health (WMH) survey initiative version of the world health organization (WHO) composite international diagnostic interview (CIDI). International Journal of Methods in Psychiatric Research, 13(2), 93–118. 10.1002/mpr.168 [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. King, C. A. , Eisenberg, D. , Zheng, K. , Czyz, E. , Kramer, A. , Horwitz, A. , & Chermack, S. (2015). Online suicide risk screening and intervention with college students: A pilot randomized controlled trial. Journal of Consulting and Clinical Psychology, 83(3), 630–636. 10.1037/a0038805 [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Kraemer, H. C. , & Kupfer, D. J. (2006). Size of Treatment Effects and Their Importance to Clinical Research and Practice. Biological Psychiatry, 59(11), 990–996. 10.1016/j.biopsych.2005.09.014 [DOI] [PubMed] [Google Scholar]
  45. Kraemer, H. C. , Wilson, G. T. , Fairburn, C. G. , & Agras, W. S. (2002). Mediators and moderators of treatment effects in randomized clinical trials. Archives of General Psychiatry, 59(10), 877–883. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/12365874. 10.1001/archpsyc.59.10.877 [DOI] [PubMed] [Google Scholar]
  46. Kroenke, K. , & Spitzer, R. L. (2002). The PHQ‐9:A new depression diagnostic and severity measure. Psychiatric Annals, 32(9), 1–7. [Google Scholar]
  47. Larisch, A. , Heuft, G. , Engbrink, S. , Brähler, E. , Herzog, W. , & Kruse, J. (2013). Behandlung psychischer und psychosomatischer Beschwerden–Inanspruchnahme, Erwartungen und Kenntnisse der Allgemeinbevölkerung in Deutschland. Zeitschrift für Psychosomatische Medizin und Psychotherapie, 59(2), 153–169. 10.13109/zptm.2013.59.2.153 [DOI] [PubMed] [Google Scholar]
  48. Latimer, A. E. , Rivers, S. E. , Rench, T. A. , Katulak, N. A. , Hicks, A. , Hodorowski, J. K. , … Salovey, P. (2008). A field experiment testing the utility of regulatory fit messages for promoting physical activity. Journal of Experimental Social Psychology, 44(3), 826–832. 10.1016/j.jesp.2007.07.013 [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Lewis, M. A. , & Neighbors, C. (2006). Social norms approaches using descriptive drinking norms education: A review of the research on personalized normative feedback. Journal of American College Health, 54(4), 213–218. 10.3200/JACH.54.4.213-218 [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Luedtke, A. , Sadikova, E. , & Kessler, R. C. (in press). Sample size requirements for multivariate models to predict between‐patient differences in best treatments of major depressive disorder (MDD). Clinical Psychological Science. [Google Scholar]
  51. Mitchell, N. , & Gordon, P. K. (2007). Attitudes towards computerized CBT for depression amongst a student population. Behavioural and Cognitive Psychotherapy, 35(04), 421 10.1017/S1352465807003700 [DOI] [Google Scholar]
  52. Mojtabai, R. , Olfson, M. , Sampson, N. A. , Jin, R. , Druss, B. , Wang, P. S. , … Kessler, R. C. (2011). Barriers to mental health treatment: Results from the National Comorbidity Survey Replication. Psychological Medicine, 41(08), 1751–1761. 10.1017/S0033291710002291 [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Noar, S. M. , Benac, C. N. , & Harris, M. S. (2007). Does tailoring matter? Meta‐analytic review of tailored print health behavior change interventions. Psychological Bulletin, 133(4), 673–693. 10.1037/0033-2909.133.4.673 [DOI] [PubMed] [Google Scholar]
  54. Nock, M. K. , Holmberg, E. B. , Photos, V. I. , & Michel, B. D. (2007). Self‐injurious thoughts and behaviors interview: Development, reliability, and validity in an adolescent sample. Psychological Assessment, 19(3), 309–317. 10.1037/1040-3590.19.3.309 [DOI] [PubMed] [Google Scholar]
  55. Posner, K. , Brown, G. K. , Stanley, B. , Brent, D. A. , Yershova, K. V. , Oquendo, M. A. , … Mann, J. J. (2011). The Columbia–Suicide Severity Rating Scale: Initial validity and internal consistency findings from three multisite studies with adolescents and adults. American Journal of Psychiatry, 168(12), 1266–1277. 10.1176/appi.ajp.2011.10111704 [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Quinlivan, L. , Cooper, J. , Davies, L. , Hawton, K. , Gunnell, D. , & Kapur, N. (2016). Which are the most useful scales for predicting repeat self‐harm? A systematic review evaluating risk scales using measures of diagnostic accuracy. BMJ Open, 6(2), e009297 10.1136/bmjopen-2015-009297 [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Quinlivan, L. , Cooper, J. , Steeg, S. , Davies, L. , Hawton, K. , Gunnell, D. , & Kapur, N. (2014). Scales for predicting risk following self‐harm: An observational study in 32 hospitals in England. BMJ Open, 4(5), e004732 10.1136/bmjopen-2013-004732 [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Reinert, D. F. , & Allen, J. P. (2002). The Alcohol Use Disorders Identification Test (AUDIT): A review of recent research. Alcoholism, Clinical and Experimental Research, 26(2), 272–279. 10.1111/j.1530-0277.2002.tb02534.x [DOI] [PubMed] [Google Scholar]
  59. Riper, H. , van Straten, A. , Keuken, M. , Smit, F. , Schippers, G. , & Cuijpers, P. (2009). Curbing problem drinking with personalized‐feedback interventions: A meta‐analysis. American Journal of Preventive Medicine, 36(3), 247–255. 10.1016/j.amepre.2008.10.016 [DOI] [PubMed] [Google Scholar]
  60. Saunders, J. B. , Aasland, O. G. , Babor, T. F. , De la Fuente, J. R. , & Grant, M. (1993). Development of the Alcohol Use Disorders Identification Test (AUDIT): World Health Organisation's Collaborative Project on early detection of persons with harmful alcohol consumption–II. Addiction, 88, 791–804. 10.1111/j.1360-0443.1993.tb02093.x [DOI] [PubMed] [Google Scholar]
  61. Seidler, Z. E. , Dawes, A. J. , Rice, S. M. , Oliffe, J. L. , & Dhillon, H. M. (2016). The role of masculinity in men's help‐seeking for depression: A systematic review. Clinical Psychology Review, 49, 106–118. 10.1016/j.cpr.2016.09.002 [DOI] [PubMed] [Google Scholar]
  62. Sheridan, S. L. , Draeger, L. B. , Pignone, M. P. , Keyserling, T. C. , Simpson, R. J. , Rimer, B. , … Gizlice, Z. (2011). A randomized trial of an intervention to improve use and adherence to effective coronary heart disease prevention strategies. BMC Health Services Research, 11(1), 331 10.1186/1472-6963-11-331 [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. Sniehotta, F. F. , Scholz, U. , & Schwarzer, R. (2005). Bridging the intention–behaviour gap: Planning, self‐efficacy, and action control in the adoption and maintenance of physical exercise. Psychology & Health, 20(2), 143–160. 10.1080/08870440512331317670 [DOI] [Google Scholar]
  64. Spendelow, J. S. (2015a). Cognitive–behavioral treatment of depression in men. American Journal of Men's Health, 9(2), 94–102. 10.1177/1557988314529790 [DOI] [PubMed] [Google Scholar]
  65. Spendelow, J. S. (2015b). Men's self‐reported coping strategies for depression: A systematic review of qualitative studies. Psychology of Men & Masculinity, 16(4), 439–447. 10.1037/a0038626 [DOI] [Google Scholar]
  66. Spiegel, S. , Grant‐Pillow, H. , & Higgins, E. T. (2004). How regulatory fit enhances motivational strength during goal pursuit. European Journal of Social Psychology, 34(1), 39–54. 10.1002/ejsp.180 [DOI] [Google Scholar]
  67. Spitzer, R. L. , Kroenke, K. , Williams, J. B. W. , & Löwe, B. (2006). A brief measure for assessing generalized anxiety disorder. Archives of Internal Medicine, 166(10), 1092 10.1001/archinte.166.10.1092–1097. [DOI] [PubMed] [Google Scholar]
  68. Ursano, R. J. , Colpe, L. J. , Heeringa, S. G. , Kessler, R. C. , Schoenbaum, M. , & Stein, M. B. (2014). The Army study to assess risk and resilience in servicemembers (Army STARRS). Psychiatry: Interpersonal and Biological Processes, 77(2), 107–119. 10.1521/psyc.2014.77.2.107 [DOI] [PMC free article] [PubMed] [Google Scholar]
  69. Yousaf, O. , Grunfeld, E. A. , & Hunter, M. S. (2015). A systematic review of the factors associated with delays in medical and psychological help‐seeking among men. Health Psychology Review, 9(2), 264–276. 10.1080/17437199.2013.840954 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Table S1. Baseline motivational and acceptance‐related characteristics (n = 1375)

Table S2. Overview of all results from a series of linear multiple regression analyses with bootstrapped 95%‐confidence intervals (5000 bootstrap samples).

Click here for additional data file. (2.5MB, docx)

Articles from International Journal of Methods in Psychiatric Research are provided here courtesy of Wiley

RESOURCES