Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Sep 1.
Published in final edited form as: Cogn Behav Ther. 2018 Dec 11;48(5):385–405. doi: 10.1080/16506073.2018.1533575

Web-Based CBT for the Prevention of Anxiety Symptoms among Medical and Health Science Graduate Students

Ashley N Howell 1, Alyssa A Rheingold 1, Thomas W Uhde 1, Constance Guille 1,*
PMCID: PMC6559871  NIHMSID: NIHMS1520311  PMID: 30526364

Abstract

Medical and health science graduate students experience more anxiety problems than the general population but are less likely to seek treatment. This incongruity may be due to concerns about stigma, anonymity, workload, and finances. One solution may be a preventative approach that overcomes these barriers to mental healthcare, such as web-based cognitive behavioral therapy (webCBT). It is unknown whether webCBT is effective for preventing anxiety escalation within this population. A randomized controlled trial was conducted, comparing the effects of webCBT versus a control group (CG). Medical university students (n=594; Mage=27; 67% female; 80% Caucasian) completed online baseline measures, randomized to webCBT or CG, and completed 4 assigned online activities. Measures were readministered at a three-month follow-up assessment. There was a small and significant interaction between time of assessment and treatment condition. Anxiety symptom severity was lower in the webCBT (M[SD]=2.88[3.36]) versus CG condition (M[SD]=3.69[3.35]) at follow-up. This effect was moderate for participants with mild, versus minimal, anxiety at baseline. The proportion of participants with a possible anxiety disorder was lower in the webCBT group (4.5%) compared to the CG (8.5%), RR = .55. In addition, the proportion of mildly anxious participants who had a clinically significant increase in symptoms was lower in the webCBT group (10%) compared to the CG (20%). WebCBT modules may aid in preventing anxiety escalation for some medical and health science graduate students, particularly for at-risk students who report mild anxiety symptoms.

Keywords: CBT, internet, web, anxiety, prevention, medical students, graduate students

Graduate students in health-related fields tend to experience elevated stress and anxiety compared to the general population and to non-students of similar ages (Dyrbye, Thomas, & Shanafelt, 2006, for a review). The (2013-2014) point prevalence rate of a positive screen for a possible anxiety disorder, per the 7-Item Generalized Anxiety Disorder Scale (GAD-7 ≥ 10), was over eight-fold higher in medical students, residents, and fellows (19% vs. 2%) than nationally representative age-matched controls (Mousa, Dhamoon, Lander, Dhamoon, 2016). This comparison data is generally consistent with a separate nationally representative sample not exclusive to age, which found that a positive screen occurred in about 4% of respondents.

At the trait level, individuals who pursue postbaccalaureate education are likely to report higher perfectionism and neuroticism than individuals who do not (Enns, Cox, Sareen, & Freeman, 2001; Tyssen, Dolatowski, & Røvik, 2007) —increasing risk and prevalence for anxiety-related problems. In addition, graduate students are a vulnerable population for anxiety escalation, because they are exposed to and report chronic stressors, such as long work hours, limited income, burnout, frequent evaluation, and sleep deprivation (Dyrbye, Thomas, Huntington et al., 2006; Mazzola, Walker, Shockley, & Spector, 2011; Myers et al., 2012; Oswalt & Riddock, 2007). Unfortunately, a “critical deficiency in resilience education and training” exists for medical school, residency, and graduate students (Beresin et al., 2016, p. 9), and experts have called for programs to increase psychological resiliency programs for this at-risk population (Rakesh, Pier, & Costales, 2017).

Graduate institutions, however, cite credible barriers to meeting these calls for action, including limited staff and funding (Novotney, 2014) and therefore long waiting lists if services are offered (Stecker, 2004). At the same time, the likelihood of these students seeking professional help for anxiety or other concerns is lower than that of the general population—creating a discrepancy between elevated anxiety symptoms and underutilized treatment options that are typically both on and off campus (Chew-Graham, Rogers, & Yassin, 2003; Dyrbye, Thomas, & Shanafelt, 2006). To illustrate, among people with an anxiety disorder in the USA (Wang, Lane, Olfson, Pincus, Wells, & Kessler, 2005) or Canada (Roberge, Fournier, Duhoux, Nguyen, & Smolders, 2011), 36.9% report receiving treatment within the past year. In one sample predominantly comprised of doctoral students, about 31% reported seeking mental health services for any problem at some point throughout their several years of graduate school enrollment (Hyun, Quinn, Madon, & Lustig, 2006). While service underutilization in the general population can be explained in some part by lack of knowledge about mental illness or treatment options for anxiety (Thompson, Hunt, & Issakidis, 2004; see also Gulliver, Griffiths, & Christensen [2010] and Sareen et al. [2007] for samples nonspecific to anxiety), many medical and health science graduate students learn about, promote, and even provide interventions for anxiety-related sequelae. Major reasons for explicit service underutilization for medical and health science graduate students include: stigma about emotional problems among health providers and/or academics; concerns about anonymity and confidentiality within local resources at which colleagues may work; fear of negative impacts on one’s career and/or academic record; time constraints; financial burden; and perceived inadequacy of mental health sessions available at the university (Brimstone, Thistlethwaite, Quirk, 2006; Dunn, Iglewicz, & Moutier, 2008; Guille et al., 2015; Tjia, Givens, & Shea, 2005).

Overall, medical and health science graduate students are not only uniquely vulnerable to anxiety due to particular traits and elevated chronic stress, but they also experience unique barriers to seeking professional help when anxiety and stress rise. Thus, while research is certainly needed to improve mental health treatment-seeking within this population, focusing resources on preventing maladaptive escalation of anxiety symptoms may help to obviate higher risk of onset, and unnecessarily prolonged distress and interference, for students. According to a review by Reavley and Jorm in 2010, there was limited to weak evidence that interventions (predominantly face-to-face CBT) were effective in preventing anxiety disorders in higher education students, at least in the longer term. More recently, Conley, Shapiro, Kirsch, and Durlak (2017) conducted a meta-analysis of mental health prevention programs for at-risk higher education students; studies that targeted anxiety were conducted primarily among undergraduates and for face-to-face interventions. They found that for students with subclinical signs of mental health symptoms, targeted anxiety prevention programs (e.g., CBT or relaxation skills training) resulted in a medium-to-large post-intervention effect size, g = .67, CI(95) = .50-.84. Barriers to implementing these prevention programs still exist, however, and the efficacy of anxiety prevention programs in medical and health science graduate students, who face additional unique stressors and barriers to prompt mental healthcare, is unclear.

One possible solution to preventing anxiety problems in this population is web-based cognitive behavioral therapy (webCBT). WebCBT (for the current study: MoodGYM) is anonymous, available at all hours to accommodate various schedules, and several webCBT programs are free or of low cost. WebCBT has been shown to reduce anxiety and other psychiatric problems in samples not exclusive to graduate students (for meta-analytic reviews see: Adelman, Panza, Bartley, Bontempo, & Bloch, 2014; Cuijpers, Marks, van Straten, Cavanagh, Gega, & Andersson, 2009; Griffiths, Farrer, & Christensen, 2010, review), with some evidence that webCBT is comparable in corrective treatment outcomes to face-to-face therapy (Andersson, Cuijpers, Carlbring, Riper, & Hedman, 2014, for a review; Adelman et al., 2014) with maintained effects over a six-month follow-up period (Kenardy, McCafferty, & Rosa, 2006). Less work has evaluated whether webCBT is effective for preventing anxiety symptoms, and these findings are mixed (e.g., see Sánchez-Gutiérrez, Barbeito, & Calvo, 2016; Deady, Choi, Glozier, Christensen, & Harvey, 2017, for reviews). Experts are calling for additional, randomized-controlled trials (RCTs) that select for individuals without mental health disorders at baseline, in order to (a) understand the preventative potential of internet-based CBT and to (b) identify the populations who may benefit most from webCBT prevention tools (see Ebert, Cuijpers, Muñoz, & Baumeister, 2017, for a review). It is currently unknown whether webCBT can serve as an effective early intervention to prevent anxiety symptom escalation for medical and health science graduate students, although there is some promising evidence that webCBT is efficacious for preventing suicidal ideation among medical interns (Guille et al., 2015).

The primary aim of this study was to address gaps in the literature regarding effective anxiety disorder prevention programs for medical and health science graduate students, who are at significant risk for developing anxiety-related problems and who experience barriers to receiving face-to-face interventions. We tested the efficacy of webCBT versus passive psychoeducation (i.e., web-based mental health symptom questionnaires with automated feedback and referrals as a control group [CG]) for anxiety symptom prevention among medical and health science graduate students, using an RCT design. Data were assessed at baseline (before the academic year) and reassessed after three-months (during the academic year). We hypothesized that (1) individuals assigned to the webCBT group would report lower anxiety symptoms than individuals in the CG at follow-up assessment. We also hypothesized that a smaller proportion of individuals in the webCBT group, versus CG group, would: (2) meet a cutoff score suggestive of clinically elevated anxiety symptoms and/or (3) would not demonstrate a clinically significant increase in symptoms during the school year (regardless of clinical status).

Methods

Participants

Please see Figure 1. 1,507 graduate students from the College of Medicine, College of Dental Medicine, College of Graduate Studies, College of Health Professions, College of Nursing, and College of Pharmacy at a U.S. medical university were sent an email two months prior to commencing the 2014-2015 academic year—inviting them to participate in the study. Participants were eligible for the study if they were a returning or new student beginning classes in August 2014 in one of the six colleges at the university. Email invitations were returned as undeliverable for 11.9% (179/1,507) of potential participants, and 71.0% (943) of individuals who received the email (1,328) agreed to participate in the study (webCBT: n = 468; CG: n = 475). Email addresses of consenting students were assigned to the CG or MoodGYM conditions using the method of 1:1 simple randomization using a random number generator. Of those who agreed, 12.2% (58) assigned to the CG condition never initiated participation in the study, and 24.1% (113) assigned to the webCBT condition never logged on to the MoodGYM website; non-initiators reported slightly higher anxiety symptoms at baseline (M = 6.18, SD = 5.26; per the GAD-7, see Measures, below) than initiators (M = 5.25, SD = 4.83), t = 2.17, p = .03, and did not significantly differ on demographic variables or depression, ps ≥ .10.The remaining 772 consenting individuals initiated their allocated interventions (webCBT: n = 355; CG: n = 417).

Figure 1.

Figure 1

Open in a new tab

Participant flow chart following Consolidated Standards of Reporting Trials (CONSORT) guidelines.

Participants were screened at baseline and included in the current analyses if they reported minimal to mild anxiety symptoms (see Measures, below), in order to test for the prevention of anxiety symptoms. As a result, 17.4% (134/772) of consenting and initiating participants were excluded from the current analyses due to moderate, clinically-elevated anxiety at baseline (i.e., GAD-7 score ≥ 10, see also Measures, below). The rate (17.4%) of clinically elevated anxiety in the current sample is consistent with prior work, which found that the point prevalence rate of a positive screen for clinically significant anxiety (GAD-7 ≥ 10) was over eight-fold higher in medical students, residents, and fellows (19%) than in age-matched controls (2%; Mousa et al., 2016) or in the general population (4%; Löwe et al., 2008). Also excluded from analyses were 5.7% of participants (44/772) who initiated their interventions but did not complete the baseline anxiety assessment. An intent-to-treat approach was used to control for treatment effect biases for missing follow-up assessment data, such that baseline measurements for non-completers were carried forward. No significant differences in baseline anxiety or depression emerged between eligible follow-up completers and participants who did not complete the follow-up assessments, ts ≤ │.79│, ps ≥ .43. There was a significantly greater percentage of follow-up assessment completers in the CG (86.9%) versus webCBT (75.2%) condition,χ2 = 13.43, p < .001. The final sample comprised 594 participants (webCBT: n = 266; CG: n = 328).

Assessment data were collected through a secure online website designed to maintain confidentiality, with participant data identified only by a non-decodable number. All participants were given information about symptoms and encouraged to seek treatment locally if necessary. Participants received $20 prior to the start of the academic year for completion of the baseline survey and $20 during the academic year for completion of the follow-up survey in online gift certificates for compensation. A waiver of written and oral consent, as well as study approval, was granted by the Institutional Review Board at the Medical University of South Carolina.

Measures

Baseline assessment.

Participants completed a secure baseline survey online, 6-8 weeks prior to starting the academic year. The survey included demographic characteristics (e.g., gender, race/ethnicity), type of academic program/college, and symptom measures.

The 7-Item Generalized Anxiety Disorder Scale (GAD-7; Spitzer, Kroenke, Williams, & Löwe, 2006) is a measure of generalized anxiety disorder symptoms. It is also moderately good at screening for panic, social anxiety, and posttraumatic stress disorders (Kroenke et al., 2007). Participants are asked how often they have been bothered by anxiety in the past two weeks (e.g., trouble relaxing). Participants respond on a 4-point scale, from 0 (not at all) to 3 (nearly every day). In general populations, confirmatory factor analyses substantiated a one-factor structure, with factor invariance for gender and age (Löwe et al., 2008). In addition, good internal consistency (α = .89) has been demonstrated, as well as criterion, factorial, procedural, and construct validity in both general and clinical samples (Kroenke et al., 2007; Löwe et al., 2008; Spitzer et al., 2006). Clinically relevant cut-off scores have been psychometrically supported (Kroenke et al., 2007; Löwe et al., 2008; Spitzer et al., 2006): (5 = Mild), (10 = Moderate and may warrant treatment), (15 = Severe), and participants who already expressed clinically elevated anxiety at baseline (≥ 10) were excluded from current analyses. This cut-off score of 10 was also used to operationalize whether participants did or did not experience clinically elevated anxiety symptoms at follow-up assessment. The percentage students who met criteria for statistically reliable change (Jacobson & Traux, 1991) on the GAD-7 was calculated using the GAD-7 baseline standard deviation and a test–retest reliability coefficient of 0.83, as reported by Spitzer et al. (2006). The reliable change index (RCI) in this sample = 3.03. Participants were categorized according to whether their anxiety symptoms clinically significantly (a) improved, (b) remained unchanged, or deteriorated from baseline to follow-up. In the current sample, the GAD-7 demonstrated good internal consistency at both baseline (α = .76) and follow-up (α = .88) assessments.

The Patient Health Questionnaire (PHQ-9; Spitzer et al., 1999) is a self-report questionnaire assessing the frequency of nine DSM-IV depression items (e.g., “feeling down, depressed, or hopeless”) in the past two weeks, with response options of 0 (not at all) to 3 (nearly every day). The PHQ-9 is commonly used for screening and informing diagnosis, as well as selecting and monitoring treatment. The PHQ-9 is highly correlated with diagnosis by mental health professionals and other depression assessment tools in a variety of populations (Henkel et al., 2004; Kroenke, Spitzer, & Williams, 2001; Löwe, Kroenke, Herzog, & Gräfe, 2004; Martin, Rief, Klaiberg, & Braehler, 2006). In addition, the PHQ-9 has demonstrated good to excellent internal consistency in both clinical and general/cigarette-smoking samples (αs ≥ .86; Löwe et al., 2004; Cannon et al., 2007). While the PHQ-9 has historically been moderately to strongly related to the GAD-7 (e.g., Spitzer, Kroenke, Williams, Löwe, 2006), factor analysis has confirmed two distinct dimensions between the PHQ-9 and the GAD-7 (Spitzer et al., 2006). In the current sample, the PHQ-9 demonstrated good internal consistency, α = .80, and was moderately related to the GAD-7 at both baseline and follow-up assessments (including both intent-to-treat and per-protocol data), rs = .43 - .51, ps < .001—suggesting that PHQ-9 scores were unsurprisingly related to, but not duplicative of, GAD-7 scores. In order to test multilevel chi square tests (see Data Analytic Plan, below), the PHQ-9 was categorically recoded as “above” or “below” a clinical cut-off score suggestive of depression pathology (i.e., scores ≥ 10; Kroenke et al., 2001).

Follow-up assessment.

Approximately three months after the start of the academic year, participants were asked to complete another secure web-based survey including current measures of anxiety (GAD-7). They were also asked to report face-to-face mental health treatment history since baseline assessment. Specifically, participants were asked to endorse or deny the following statements: “I have not had any mental health problems”; “I have had some mental health problems, but I have not sought help”; “I have consulted with [a mental health provider/a general practitioner]”; and “I have been admitted to a psychiatric hospital”. Responses were recoded into “No psychological problems”; “Problems: Did not seek help”; and “Problems: Sought help”.

Study Procedures

Approximately two months prior to starting the academic year, consenting participant email addresses were randomly assigned to either the webCBT or CG condition, using complete simple randomization with equal allocation, via an online random sequence generator operated by a person independent of the research. Participants in each group were directed to the relevant websites via email, including three weekly reminder emails. Each email provided information about the prevalence of mental health problems including depression, anxiety, substance use, and suicide among graduate students, as well as described symptoms of these problems. Students were encouraged to seek in-person mental health treatment, if necessary. Contact information for urgent and non-emergent local, confidential, and free mental health services was included in each email.

Control Group.

Following randomization, participants assigned to the CG were directed via email each week for four weeks to the institution’s Counseling and Psychological Services’ online resource center (http://screening.mentalhealthscreening.org/musc) to complete an anonymous mental health self-assessment related to mood, anxiety, and substance use. Participants gained access to the website using their student ID and password known only to the user. Upon completion of each set of symptom questionnaires, which took approximately 10 minutes to complete (totaling about 40 minutes across four weeks), participants were given automated clinical feedback about their scores. For example, for anxiety, participants were told whether or not their screening was suggestive of a specific anxiety disorder, although it was specified that screening was not a substitute for a clinical evaluation and could not provide an actual diagnosis. Regardless of the screening results, all students were encouraged to seek professional help if they had concerns about emotional health or substance use and were provided with resources.

webCBT intervention.

Participants assigned to the webCBT intervention were directed via email each week for four weeks to the intervention website (http://moodgym.anu.edu.au) to complete one of four webCBT modules. Participants gained access to the secure website via a username provided within the email. Once participants accessed the website, they created a unique password known only by them—allowing the content provided within the modules (e.g., individualized CBT exercises) to remain anonymous. Web-site developers were able to track date and time of program access as a proxy for verification of initiation in webCBT activities based on the assigned username. Due to study-related restrictions, frequency of log-ins, duration spent per module, and degree of engagement with interactive material could not be determined.

The webCBT program, MoodGYM, was developed by staff at the National Institute for Mental Health Research at The Australian National University, in order to obviate barriers to CBT and to treat and prevent depression (Christensen, Griffiths, & Korten, 2002). Since MoodGYM’s debut, several RCTs have been conducted to determine its efficacy and effectiveness at treating depression and other disorder symptoms, including anxiety (Twomey & O’Reilly, 2016, for a meta-analysis). There are also promising results that MoodGYM may help to prevent both depression and anxiety symptoms in adolescent and youth samples (e.g., Calear & Christensen, 2010), although less, and mixed yet promising, prevention work has been conducted in adult populations (e.g., Deady et al., 2017; Powell et al., 2013). The program consisted of 4 weekly web-based sessions lasting approximately 30 minutes each. The interactive program uses exercises, quizzes, and scenarios to facilitate an understanding of the interplay between thoughts, emotions, and behaviors (Module 1) and teaches cognitive restructuring techniques that promote the ability to identify and challenge inaccurate, unrealistic, or overly negative thinking (Module 2 and 3). The program also includes problem-solving strategies (Module 4).

Data Analytic Plan

Analyses were performed using SPSS, Version 24 (IBM corp., Armonk, NY). We first assessed the distribution of demographic and baseline symptom severity according to our randomization procedures. Specifically, we compared the webCBT and CG groups on demographic and baseline clinical characteristics, using chi-square tests for categorical variables and independent samples t-tests or analyses of variance (ANOVAs) for continuous variables. A Bonferroni correction (α = .05/5 = .01) was used to control for familywise error rate for these preliminary analyses.

To test treatment efficacy, we compared pre-post changes between treatment conditions on anxiety severity, clinical status, and reliable clinical change. A repeated measures (RM) ANCOVA was used to test for differences in anxiety severity from baseline to follow-up according to treatment condition, while covarying for variables with unequal allocation across groups. Effect size for baseline to follow-up score change was assessed via Cohen’s d test, using marginal means (i.e., after covariation). An a priori power analysis indicated that a total sample size of 200 (with n =100 per webCBT and CG group) was needed to detect a small (f = .10) significant within- and between-subjects interaction effect. We explored whether baseline anxiety level (i.e., minimal versus mild, per GAD-7 cut-off scores) moderated results, in order to inform who may benefit the most from preventative webCBT. The RM ANCOVA test was then repeated, but after adding covariates of gender and depression, which are known a priori to impact anxiety and treatment outcomes (DiMatteo, Lepper, & Croghan, 2000; McLean & Anderson, 2009; Pine, Cohen, Gurley, Brook, & Ma, 1998), in order to increase confidence about the unique effect of treatment condition on follow-up anxiety symptoms. To test clinical significance, clinical status and reliable clinical change were evaluated using the follow-up data that was carried forward. Two-way and three-way chi square tests were used to examine whether there were group differences in the proportion of individuals who met a cut-off score for likely anxiety pathology (i.e., GAD-7 ≥ 10), as well as for group differences in the proportion of individuals who had clinically significant symptom change (i.e., improved, deteriorated, or remained unchanged), and if these proportions differed across levels of our covariates or minimal/mild baseline anxiety status.

Results

Demographic and Clinical Characteristics

Please see Table 1. Demographic variables, academic college, and baseline depressive and anxiety symptoms were approximately equally distributed between conditions, with the exception of race/ethnicity; there was a higher percentage of minority students in the CG. Race/ethnicity groups did not significantly vary in follow-up anxiety symptoms, p = .68. Treatment-seeking during the study also differed by condition; about three times the number of participants in the webCBT group who experienced a psychological problem endorsed seeking face-to-face mental health treatment (n=28), versus not seeking treatment (n=9), at follow-up assessment. However, this proportion was equal (ns = 24) for participants with psychological problems in the CG. Treatment-seeking groups did not impact symptom change within each condition, ps > .70, but significantly varied in follow-up anxiety symptoms, p < .001. Therefore, race and mental health treatment-seeking were included as a covariates for all primary analyses. Both a priori variables, gender (t = 2.83, p = .005) and baseline depression (r = .37, p < .001), significantly related to follow-up anxiety.

Table 1.

Demographic and Clinical Characteristics of Medical and Health Science Graduate Students during the 2014-2015 Academic Year

webCBT Feedback Only
(CG)		 Group
Differences
N 266 328 ---
Agea 28.0(8.1) 27.0(6.1) t = 1.43
d = 0.14
Genderb 70% female 66% female χ2 = 1.05
φ = 0.04
Race/Ethnicityc 84% White 75% White χ2 =10.05*
6% Black 7% Black φ = 0.13
6% Asian 7% Asian
.4% Middle Eastern .6% Middle Eastern
.4% Multiracial 3% Multiracial
.4% Other 3% Other
3% Latino/a 4% Latino/a
Academic College 27% Health Professions 31% Health Professions χ2 = 6.32
23% M.D. 28% M.D. φ = 0.10
14% Nursing 13% Nursing
13% Pharmacy 9% Pharmacy
13% Graduate Studies 13% Graduate Studies
6% Dental 10% Dental
Mental Health 82% Denied Problems 84% Denied Problems χ2 = 6.30*
Treatment-Seekingd 4% Problems: No Tx 8% Problems: No Tx φ = 0.11
14% Problems: Tx 8% Problems: Tx
Baseline Anxiety 3.28(2.63) 3.41(2.71) t = 0.62
Range: 0-9 Range: 0-9 d = 0.05
Follow-Up Anxietye 2.97(3.59) 3.62(3.67) t = 2.31*
Range: 0-16 Range: 0-21 d = 0.18
Baseline Depression 3.96(3.28) 3.61(3.48) t = 1.31
Range: 0-17 Range: 0-27 d =0.10
Open in a new tab

Note:

*

p < .05; all other ps ≥ .10

Abbreviations: webCBT = web-based cognitive behavioral therapy; CG = control group; M.D. = medical resident; Tx = (professional mental health) treatment

a

n = 429 due to missing data

b

n = 587 due to missing data

c

Latino/a ethnicity is non-exclusive; to meet cell count assumptions for chi-square analysis, race/ethnicity was recoded into White, Black, Asian, and Other.

d

Treatment-seeking since baseline assessment; n = 501 due to missing data

e

Baseline data carried forward for missing follow-up data

Treatment Efficacy

Anxiety Severity.

See Table 2 and Figure 2. There was a small, significant multivariate interaction effect between time (Baseline; Follow-up) and treatment condition (webCBT; CG) when predicting change in anxiety symptom scores and while accounting for race and treatment-seeking.1, 2 There was also a small, significant between-subjects effect for treatment condition, collapsed across time and accounting for covariates. The covaried treatment effect (i.e., using marginal means) for anxiety symptoms at follow-up assessment was small, t = 2.65, p = .008, d = .24, CI(95)d = .06 - .42; students who engaged in webCBT prior to the academic year reported less anxiety at follow-up (marginal M[SD] = 2.88[3.36]; CI[95%] = 2.42 - 3.34) than students assigned to the CG (marginal M[SD] = 3.69[3.35]; CI[95%] = 3.30 - 4.07).3

Table 2.

Treatment Efficacy of webCBT versus a Control Group on Anxiety Symptoms for Medical and Health Science Graduate Students during the 2014-2015 Academic Year

F η2p p
Multivariate Effects
 Time (Baseline vs Follow-up) 5.74 .01 .02
 Time*Race/Ethnicity .76 .002 .39
 Time*Treatment-Seeking 17.29 .03 <.001
 Time*Treatment Condition 4.93 .01 .03
Between-Subjects Effects (averaged across assessment time-points)
 Race/Ethnicity 1.09 .002 .30
 Treatment-Seeking 77.95 .14 <.001
 Treatment Condition 4.17 .01 .04
Open in a new tab

Note: Bold font indicates significant probability results (p < .05)

Figure 2.

Significant time-by-assignment interaction effect, while controlling for race/ethnicity and mental health treatment-seeking. GAD-7 = Generalized Anxiety Disorder-7; webCBT = web-based cognitive behavioral therapy; CG = control group

Figure 2

Note: GAD-7 scores 5 – 9 = mildly elevated anxiety; error bars = standard error; marginal means t = 2.65, p = .008, d = .24.

Open in a new tab

Interaction and between-subjects effects were robust and small-to-moderate when additionally covarying for gender and baseline depression, F(472) = 4.44, η2p = .01, p = .04; d = .29, CI(95)d = .11 - .48. We then included baseline anxiety cutoff score status (i.e., minimal versus mild) as a between-subjects variable in the model. (Of note, there were no significant differences in condition assignment between the minimal and mild anxiety groups, χ2[1] = .17, p = .69.) There was a significant three-way interaction, F (1, 495) = 4.06, p = .04, for time, condition, and baseline anxiety status. The covaried treatment effect was significant only for mildly anxious students and was of moderate size, t = 3.08, p = .002, d = .48, CI(95)d = .17 - .79, Mdiff = 1.53 (minimally anxious: Mdiff = .39).

Clinical Status.

We then tested group differences in the proportion of individuals who met the clinical cutoff score for anxiety pathology at follow-up. See Table 3. Results indicated that 4.51% (n = 12) of webCBT participants developed clinically elevated anxiety, compared to 8.54% (n = 28) of CG participants, χ2(1) = 3.79, p = .05; LR = 3.92, p = .048; φ = .05.4 There was about a 47% reduction in the risk of developing clinically elevated anxiety symptoms for webCBT, versus CG participants, RR = .53, CI(95) = .27 - 1.02. The small effect for the differences in distribution was stronger among participants with mild (φ = .12) versus minimal (φ = .06) anxiety at baseline. The number needed to treat (NNT) to prevent one minimally or mildly anxious student from developing a clinically-elevated score during the school year (follow-up) was 25. The NNT for only mildly anxious students, however, was 12.

Table 3.

Rates of Possible Anxiety Disorder Status and Directions of Reliable Clinical Change per Treatment Condition and Anxiety Levels at Baseline

CG
(n = 266)		 webCBT
(n = 328)
Baseline Anxiety Status (GAD-7 scores) n % n %
 Clinical Status at Follow-Up (GAD-7 ≥ 10)
 Reliable Change at Follow-Up
Minimal (n = 397)
 GAD-7 ≥ 10 6 2.8 2 1.1
 Improved 15 6.9 9 5.0
 Unchanged 164 75.2 146 81.6
 Deteriorated 39 17.9 24 13.4
Mild (n = 197)
 GAD-7 ≥ 10 22 20.0 10 11.5
 Improved 39 35.5 41 47.1
 Unchanged 49 44.5 37 42.5
 Deteriorated 22 20.0 9 10.0
Total (n = 594)
 GAD-7 ≥ 10 28 8.5 12 4.5
 Improved 54 16.5 50 18.8
 Unchanged 213 64.9 183 68.8
 Deteriorated 61 18.6 33 12.4
Open in a new tab

Note: GAD-7 = Generalized Anxiety Disorder Scale- 7 Item; CG = Control Group; webCBT = web-based cognitive behavioral therapy (MoodGYM)

We then explored whether chi square results varied by our control variables of race/ethnicity and treatment-seeking, as well as gender and depression. Results did not vary by significance status according to race/ethnicity, baseline depression cutoff status (see Measures section), or help-seeking status, all ps > .09. However, there was a difference for gender.5 The results were significant for women (CG: n = 23, 10.8% [within CG]; webCBT: n = 10, 5.4% [within webCBT]), χ2(1) = 3.84, p = .05; LR = 3.96, p = .046; φ = .10, RR = .50, but not for men (CG: n = 5, 4.5%; webCBT: n = 2, 2.5%), χ2(1) = .55, p = .46; LR = .57, p = .45; φ = .05, RR = .55). However after switching the order of chi-square levels, for full perspective, results revealed gender gaps that trended toward significance for CG participants, χ2(1) = 3.23, p = .057; LR = 4.00, p = .045; φ = .11, RR = 2.39, but not for webCBT participants, χ2(1) = 1.09, p = .30; LR = 1.21, p = .27; φ = .06, RR = 2.16. A sensitivity power analysis indicated that a small (but larger) effect size (φ = .14) would have been required for power = .80, α = .05, and for the subsample size of men in the current study (n = 191), compared to φ = .10 for the subsample of women (n = 403).

Using a hierarchical binary logistic regression analysis, we further explored potential predictors of clinical status at follow-up, while covarying for proximity to clinical status at baseline (i.e., baseline GAD-7 scores). When controlling for baseline anxiety as a continuous variable (baseline anxiety: Wald = 34.18, df = 1, p < .001, B = .40), the odds of reporting clinically elevated anxiety at follow-up were 51% lower in the webCBT group compared to the CG, albeit to a near-significant degree, Wald = 3.33, df = 1, p = .068, exp[B] = .51, CI(95) = .25 – 1.05. The effect of treatment group was significant, however, Wald = 4.13, p = .04, Exp(B) = .46, when additionally covarying for gender (p = .21), baseline depression (p = .045), and race/ethnicity (composite and contrast ps ≥ .60), all Walds ≤ 4.02, but not when adding help-seeking (composite p = .01, contrast ps ≥ .15) to the model, p = .18, Exp[B] = .59.

Reliable Clinical Change.

See Table 3. Based on our criteria for statistically reliable change, 16.5% (n = 54) demonstrated reliable improvement in the CG, and 18.8% (n = 50) demonstrated reliable improvement in the webCBT group. Conversely, 18.6% (n = 61) demonstrated reliable deterioration in the CG, whereas 12.4% (n = 33) demonstrated reliable deterioration in the webCBT group. An approximately equal percentage of students did not have a clinically significant change in symptoms between the webCBT (70.0%; n = 140) and CG (72.6%; n = 207) conditions. Thus, when conducting chi-square analyses to determine proportional differences in change directionality (see Table 3), we only assessed students who demonstrated a clinically significant change; these results neared significance, both χ2s and LRs = 3.42, ps ≥ .07, φs = 13. Proportional differences were greater among participants with mild baseline anxiety, χ2(1) = 4.46, p = .04; LR = 3.60, φ = .20 (minimal: p = .96, φ = .01).6

Discussion

Experts have increasingly called for psychological resiliency-building programs for medical and health science graduate students (Beresin et al., 2016; Rakesh et al., 2017). In response to these calls, and to our knowledge, the current study is the first to test the efficacy of webCBT versus a control group (CG; i.e., automated symptom questionnaire feedback) for preventing anxiety symptoms within this population.

Anxiety Severity.

Anxiety symptom severity at follow-up was significantly lower (albeit by about one point) and to a small effect for students who were assigned to webCBT, compared to the CG, and this effect was robust to race/ethnicity, in-person treatment-seeking, gender, and baseline depression symptoms. This effect was significantly moderated by baseline anxiety severity status (i.e., minimal versus mild), suggesting that students who endorse mild anxiety (5 ≤ GAD-7 < 10) before the start of the school year may benefit the most from webCBT prevention efforts, to a moderate statistical effect (i.e., by about two points, consistent with prior MoodGYM anxiety treatment findings using the GAD-7 in an unselected general population; Powell et al., 2013).

Clinical Status and Degree of Reliable Clinical Change.

About 4.5% of the low-anxious students who engaged in webCBT developed clinically elevated anxiety during the academic year, and this was a significantly lower proportion than the 8.5% who developed clinically elevated anxiety in the CG. The webCBT rate for a positive GAD-7 screen (≥ 10) falls within the 95% CI (3.5 - 4.6) of the national rate (Mousa et al., 2016), whereas the CG rate falls above this CI. Furthermore, participants in the webCBT group were 46% less likely to report clinically elevated anxiety at follow-up than the CG, while covarying for gender, race/ethnicity, baseline depression, and baseline anxiety. For the subsample of students who met mild anxiety status before the academic year, clinical status rates at follow-up were 11.5% and 20% for the webCBT and CG conditions, respectively. This (subsample) rate for positive screens in the webCBT group was nearly half the rate of both the CG and a separate sample of medical students, residents, and fellows (19%; Mousa et al., 2016). The NNT was 25 (full sample) and 12 (mild anxiety subsample), consistent with a review and meta-analysis of internet-based interventions for the prevention of mental disorders (NNT ≈ 9 to 41; Sander, Rausch, & Baumeister, 2016).

Only within the subsample of mildly anxious, higher-risk students was there was a significant difference in the proportion of clinically significant improvement versus deterioration across the webCBT (47% versus 10%) and CG (36% versus 20%) conditions, regardless of clinical status. Thus, while webCBT did not fully prevent clinically significant anxiety symptoms from developing during the academic year, results suggest that webCBT may: (a) have a small to moderate preventative effect on anxiety escalation for a population that is at high risk for anxiety problems; (b) be more indicated for students who report mild, versus minimal, anxiety before the academic year; and (c) reduce the discrepancy of possible anxiety disorder rates between general and medical/graduate student populations.

Effect Sizes.

It is worth highlighting that our between-subjects and interaction effect sizes were small, overall, but were moderate for the subgroup of students with mild baseline anxiety (Cohen, 1988). These small-to-moderate effect sizes are comparable to other webCBT prevention/early intervention studies among undergraduates (Cukrowicz & Joiner, 2007; Musiat et al., 2014) and in the general population (standardized mean difference of .31 for both Deady et al., 2017, and Moreno-Peral et al., 2017, per meta-analytic reviews). Small corrective treatment effect sizes have similarly been found for MoodGYM-specific interventions for depression and anxiety in prior work (e.g., Twomey & O’Reilly, 2016). The lack of significant findings in the current study for students with minimal anxiety at baseline is consistent with nonsignificant post-study effects of webCBT on anxiety-related symptoms for healthy/low anxious participants from general, undergraduate, and injury patient populations (Christensen et al., 2014; Mouthaan, et al., 2013; Musiat et al., 2014; Deady et al., 2017, for a meta-analytic review).

It warrants consideration that the type of comparison group may have also had some impact on the size of our treatment effect. For example, a meta-analysis (Conley et al., 2017) of indicated mental health prevention programs for any targeted problem (e.g., anxiety, depression, social skills) for at-risk (i.e., subclinical) undergraduate and graduate students found that the effect of anxiety–specific interventions at follow-up assessment was larger when the intervention was compared to a no intervention or wait-list control group versus an attention CG. Moreno-Peral and colleagues (2017) also found larger meta-analytic effects of anxiety prevention interventions with wait-list control groups.

With regard to therapist involvement, the obtained effect sizes parallel meta-analytic results for webCBT treatment for clinically-elevated anxiety symptoms; web-based interventions with therapist support demonstrated a large effect size (d = 1.00) on anxiety symptom severity, whereas interventions without therapist support, such as in the current study, demonstrated a small effect size (d = .24; Palmqvist, Carlbring, & Andersson, 2007; Spek et al., 2007; Twomey & O’Reilly, 2016). Thus, it is possible that the effect sizes obtained in the current study would have been larger if we had compared outcomes from the self-guided, webCBT intervention group to a more typical scenario of no proactive psychoeducation or feedback (i.e., wait-list or no intervention); yet, it is important to capitalize on any opportunity to raise awareness about mental health problems and services in a high-risk population. Areas for effect size improvement may also lie in the addition of therapist support, when feasible.

Considerations for Gender and Depression.

In line with prior work (DiMatteo, Lepper, & Croghan, 2000; McLean & Anderson, 2009; Pine, Cohen, Gurley, Brook, & Ma, 1998), we found that gender and baseline depression were directly related to anxiety severity at follow-up. The treatment effect on clinical status was stronger for women than for men, and the gender gap in clinical status at follow-up was greater for the CG condition than for the webCBT group. The development of anxiety disorders, therefore, may especially be a concern for female students, compared to male students; it logically follows that any preventative effects of webCBT on clinical status may be greater for women than for men. Gender did not, however, significantly impact the interaction between time and treatment condition on anxiety severity—suggesting that gender differences in treatment outcomes are likely more apparent among clinically severe samples. It is also worth mentioning that men comprised about 33% of the total sample (consistent with our institution’s demographic characteristics). Given the lower base rate of anxiety disorders among men versus women (e.g., McClean & Anderson, 2009), replication of this study with a greater sampling of male students may detect small treatment effects for clinical status among men. Future studies expanding this work may determine whether there is a need for gender-specific assessments or interventions, or if there is a potential benefit of screening for depression prior to the academic year.

Secondary Outcomes: Treatment-Seeking.

Among participants who endorsed having mental health problems during the course of the study, significantly more participants in the webCBT group sought face-to-face mental health treatment than participants in the CG. Based on data that were collected, it is unclear whether the effect of treatment-seeking on follow-up anxiety is subsumed by a circumstantial allocation of treatment-seekers to treatment condition, or if exposure to webCBT increased rates of face-to-face treatment. Existing data suggest that webCBT (MoodGYM) may boost treatment-seeking for face-to-face CBT, compared to an attention CG (Christensen, Leach, Barney, Mackinnon, & Griffiths, 2006), but a higher percentage of consenting students did not initiate MoodGYM (24%) compared to the CG (12%), and non-initiators in general reported higher anxiety at baseline than initiators (i.e., by about 1 point, on average, on the GAD-7). The current findings are notable, because there is a well-documented issue of mental health treatment underutilization among medical and health science graduate students (Dunn et al., 2008; Guille et al., 2015). Additional research may assess factors that modulate webCBT adherence, such as motivation and willingness for intervention (Farrer, Griffiths, Christensen, Mackinnon, & Batterham, 2014), gender and setting (Neil, Batterham, Christensen, Bennett, & Griffiths, 2009), and perceived mental health stigma, as well as the mechanisms by which webCBT may enhance treatment-seeking. For example, it could be that webCBT improves attitudes and comfort about face-to-face treatment, especially for populations faced with treatment disparities and unique barriers.

Limitations and Future Directions.

In light of the present findings, there are limitations and additional future directions that warrant consideration. First, the study was conducted at a single medical university, and results may not generalize to other institutions. Results need to be replicated, within and across institutions, to demonstrate valid preventative effects. Second, anxiety symptoms were self-reported and could not be validated by standardized clinical assessments. Although clinician-administered measures may ostensibly improve the validity of symptom reporting, medical students are more likely to report more severe symptoms when mental health assessments are anonymous compared to when they are only confidential (Levine, Breitkopf, Sierles, & Camp, 2003). Third, it is important to highlight that while we could track whether or not participants initiated webCBT, the number of webCBT module completions, frequency of log-ins, duration of module use, and degree of interactive material engagement could not be verified. In studies with low adherence (i.e., below 50% of the modules), the meta-analytic treatment effect of MoodGYM on depression and anxiety has been small (Twomey & O’Reilly, 2016). Future studies should incorporate methods of confirming and/or controlling for module engagement, as well as for testing whether the degree of engagement is influenced by gender, depression, race/ethnicity, or treatment-seeking behavior. Fourth, tasks for the control (CG) versus webCBT groups differed in attentional demand; the webCBT materials were more engaging and time-demanding than the CG materials. The CG was designed to leverage a pre-existing assessment feature at the university that is available at no cost to students. Future work would benefit from including an attention control group into an RCT design, in order to rule out potentially confounding effects of engagement. Fifth, the present study did not assess for longer-term effects of webCBT on anxiety symptoms in this population. Future studies would benefit from longer and more frequent follow-up periods, such as into the following academic year, to determine if effects are maintained after one course of webCBT or if students need “booster” modules of webCBT before each academic year. Medical and graduate student stress levels are known to fluctuate with time (Adams, 2004); thus, future work may identify when the dispersion of webCBT would be optimally effective for students at risk for acute and/or chronic anxiety. Current data provide additional justification for developing studies with longer follow-up periods to address this question.

WebCBT may not only prevent the escalation of anxiety symptoms, but also impact areas of functioning, such as financial burden and withdrawal rates for students struggling with graduate school stress. Another limitation of the current study, therefore, is that it lacks a measure of functioning that describes the impact of anxiety symptoms (and/or symptom changes) on students’ lives. Future studies may determine whether webCBT improves academic performance and generalizes to other important areas associated with resiliency and quality of life (e.g., interpersonal relationships, emotion regulation skills) for medical and health science graduate students.

Acknowledgments:

The authors would like to acknowledge and thank the students for taking part in this study. The authors would also like to thank and acknowledge Helen Christensen, PhD., Professor at the Black Dog Institute, University of New South Wales, Sydney, Australia, who generously provided access to MoodGym to allow the conduct of this study. The authors would also like to acknowledge the assistance of Ms. Kylie Bennett, e hub manager, and Mr. Anthony Bennett, software engineer, at the Centre for Mental Health Research. Dr. Christensen, Ms. Bennett, and Mr. Bennett provided no financial support and were not involved in the design, analysis or interpretation of the study results.

Funding/Support: This work was supported by the Department of Health and Human Services (DHHS) and the Substance Abuse and Mental Health Services Administration (SAMHSA) Garrett Lee Smith Memorial Act, under Grant 1U79SM060490-01; and the National Institute on Drug Abuse (NIDA) under Grant 1K23DA039318-01; and the National Institute of Mental Health (NIMH) under Grant T32MH018869-30.

Footnotes

Disclosure Statement: Beyond funding, the authors report no conflicts of interest.

1

The pattern of results held when not including covariates, F (1, 592) = 3.91, η2p = .01, p = .048.

2

These results were consistent when analyses were conducted only among follow-up assessment completers (i.e., per-protocol analysis), multivariate F (1, 481) = 4.80, p = .03.

3

The pattern of results held when not including covariates, (M[SD] = 2.97[3.16]) versus CG (M[SD] = 3.62[3.67]) condition, t(598.83) = 2.31, p = .02, d = .19, CI(95)d = .03 - .35. Cohen’s d was corrected for dependence between the means, using Morris and DeShon’s (2002) equation 8.

4

Results neared significance and the pattern of results held among follow-up assessment completers (i.e., per-protocol analysis), χ2(1) = 2.72, p = .13; φ = .07.

5

Gender did not moderate the strength of the time-treatment condition interaction found in the RM ANCOVA results, F(1,471) = .33, η2p = .001, p =.57.

6

Results were consistent when analyses were conducted only among follow-up assessment completers (i.e., per-protocol analysis), χ2(1) = 6.21, p = .045; LR = 6.27, p = .04, φ = .19 (minimal: p = .78, φ = .78).

References

  1. Adams J (2004). Straining to describe and tackle stress in medical students. Medical Education, 38(5), 463–464. [DOI] [PubMed] [Google Scholar]
  2. Adelman CB, Panza KE, Bartley CA, Bontempo A, & Bloch MH (2014). A meta-analysis of computerized cognitive-behavioral therapy for the treatment of DSM-5 anxiety disorders. Journal of Clinical Psychiatry, 75, 695–704. [DOI] [PubMed] [Google Scholar]
  3. Andersson G, Cuijpers P, Carlbring P, Riper H, & Hedman E (2014). Guided Internet‐based vs. face‐to‐face cognitive behavior therapy for psychiatric and somatic disorders: A systematic review and meta‐analysis. World Psychiatry, 13(3), 288–295. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Beresin EV, Milligan TA, Balon R, Coverdale JH, Louie AK, & Roberts LW (2016). Physician wellbeing: A critical deficiency in resilience education and training. Academic Psychiatry, 40, 9–12. [DOI] [PubMed] [Google Scholar]
  5. Brimstone R, Thistlethwaite JE, & Quirk F (2007). Behaviour of medical students in seeking mental and physical health care: Exploration and comparison with psychology students. Medical Education, 41, 74–83. [DOI] [PubMed] [Google Scholar]
  6. Calear AL, & Christensen H (2010). Review of internet-based prevention and treatment programs for anxiety and depression in children and adolescents. Medical Journal of Australia, 192(11), S12–S14. [DOI] [PubMed] [Google Scholar]
  7. Cannon DS, Tiffany ST, Coon H, Scholand MB, McMahon WM, & Leppert MF (2007). The PHQ-9 as a brief assessment of lifetime major depression. Psychological Assessment, 19, 247–251. [DOI] [PubMed] [Google Scholar]
  8. Chew-Graham CA, Rogers A, & Yassin N (2003). ‘I wouldn’t want it on my CV or their records’: Medical students’ experiences of help-seeking for mental health problems. Medical Education, 37, 873–880. [DOI] [PubMed] [Google Scholar]
  9. Christensen H, Batterham P, Mackinnon A, Griffiths KM, Hehir KK, Kenardy J, … & Bennett K (2014). Prevention of generalized anxiety disorder using a web intervention, iChill: Randomized controlled trial. Journal of Medical Internet Research, 16(9), e199. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Christensen H, Griffiths KM, & Korten A (2002). Web-based cognitive behavior therapy: Analysis of site usage and changes in depression and anxiety scores. Journal of Medical Internet Research, 4(1): e3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Christensen H, Leach LS, Barney L, Mackinnon AJ, & Griffiths KM (2006). The effect of web based depression interventions on self-reported help seeking: Randomised controlled trial [ISRCTN77824516]. BMC Psychiatry, 6(13), 1–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Cohen J (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale: Erlbaum. [Google Scholar]
  13. Conley CS, Shapiro JB, Kirsch AC, & Durlak JA (2017). A meta-analysis of indicated mental health prevention programs for at-risk higher education students. Journal of Counseling Psychology, 64(2), 121–140. [DOI] [PubMed] [Google Scholar]
  14. Cuijpers P, Marks IM, van Straten A, Cavanagh K, Gega L, & Andersson G (2009). Computer‐aided psychotherapy for anxiety disorders: A meta‐analytic review. Cognitive Behaviour Therapy, 38(2), 66–82. [DOI] [PubMed] [Google Scholar]
  15. Cukrowicz KC, & Joiner TE (2007). Computer-based intervention for anxious and depressive symptoms in a non-clinical population. Cognitive Therapy and Research, 31(5), 677–693. [Google Scholar]
  16. Deady M, Choi I, Calvo RA, Glozier N, Christensen H, & Harvey SB (2017). eHealth interventions for the prevention of depression and anxiety in the general population: a systematic review and meta-analysis. BMC Psychiatry, 17: 310. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. DiMatteo MR, Lepper HS, & Croghan TW (2000). Depression is a risk factor for noncompliance with medical treatment: Meta-analysis of the effects of anxiety and depression on patient adherence. Archives of Internal Medicine, 160(14), 2101–2107. [DOI] [PubMed] [Google Scholar]
  18. Dunn LB, Iglewicz A, & Moutier C (2008). A conceptual model of medical student well-being: Promoting resilience and preventing burnout. Academic Psychiatry, 32, 44–53. [DOI] [PubMed] [Google Scholar]
  19. Dyrbye LN, Thomas MR, Huntington JL, Lawson KL, Novotny PJ, Sloan JA, & Shanafelt TD (2006). Personal life events and medical student burnout: A multicenter study. Academic Medicine, 81(4), 374–384. [DOI] [PubMed] [Google Scholar]
  20. Dyrbye LN, Thomas MR, & Shanafelt TD (2006). Systematic review of depression, anxiety, and other indicators of psychological distress among US and Canadian medical students. Academic Medicine, 81(4), 354–373. [DOI] [PubMed] [Google Scholar]
  21. Ebert DD, Cuijpers P, Muñoz RF, & Baumeister H (2017). Prevention of mental health disorders using Internet-and mobile-based interventions: A narrative review and recommendations for future research. Frontiers in Psychiatry, 8, 116. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Ellis LA, Campbell AJ, Sethi S, & O’Dea BM (2011). Comparative randomized trial of an online cognitive-behavioral therapy program and an online support group for depression and anxiety. Journal of CyberTherapy and Rehabilitation, 4(4), 461–467. [Google Scholar]
  23. Enns MW, Cox BJ, Sareen J, & Freeman P (2001). Adaptive and maladaptive perfectionism in medical students: A longitudinal investigation. Medical Education, 35(11), 1034–1042. [DOI] [PubMed] [Google Scholar]
  24. Farrer LM, Griffiths KM, Christensen H, Mackinnon AJ, & Batterham PJ (2014). Predictors of adherence and outcome in internet-based cognitive behavior therapy delivered in a telephone counseling setting. Cognitive Therapy and Research, 38(3), 358–367. [Google Scholar]
  25. Fisak BJ, Richard D, & Mann A (2011). The prevention of child and adolescent anxiety: A meta-analytic review. Prevention Science, 12(3), 255–268. [DOI] [PubMed] [Google Scholar]
  26. Griffiths KM, Farrer L, & Christensen H (2010). The efficacy of internet interventions for depression and anxiety disorders: A review of randomised controlled trials. Medical Journal of Australia, 192(11), S4–S11. [DOI] [PubMed] [Google Scholar]
  27. Guille C, Zhao Z, Krystal J, Nichols B, Brady K, & Sen S (2015). Web-based cognitive behavioral therapy intervention for the prevention of suicidal ideation in medical interns: A randomized clinical trial. JAMA Psychiatry, 72(12), 1192–1198. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Gulliver A, Griffiths KM, & Christensen H (2010). Perceived barriers and facilitators to mental health help-seeking in young people: A systematic review. BMC Psychiatry, 10: 113. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Henkel V, Mergl R, Kohnen R, Allgaier AK, Möller HJ, & Hegerl U (2004). Use of brief depression screening tools in primary care: Consideration of heterogeneity in performance in different patient groups. General Hospital Psychiatry, 26(3), 190–198. [DOI] [PubMed] [Google Scholar]
  30. Hyun JK, Quinn BC, Madon T, & Lustig S (2006). Graduate student mental health: Needs assessment and utilization of counseling services. Journal of College Student Development, 47(3), 247–266. [Google Scholar]
  31. Jacobson NS, & Truax P (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12–19. [DOI] [PubMed] [Google Scholar]
  32. Kenardy J, McCafferty K, & Rosa V (2003). Internet-delivered indicated prevention for anxiety disorders: A randomized controlled trial. Behavioural and Cognitive Psychotherapy, 31(3), 279–289. [Google Scholar]
  33. Kenardy J, McCafferty K, & Rosa V (2006). Internet‐delivered indicated prevention for anxiety disorders: Six‐month follow‐up. Clinical Psychologist, 10, 39–42. [Google Scholar]
  34. Kessler RC, Chiu WT, Demler O, & Walters EE (2005). Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry, 62(6), 617–627. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Kroenke K, Spitzer RL, Williams JB, Monahan PO, & Löwe B (2007). Anxiety disorders in primary care: Prevalence, impairment, comorbidity, and detection. Annals of Internal Medicine, 146(5), 317–325. [DOI] [PubMed] [Google Scholar]
  36. Kroenke K, Spitzer RL, & Williams JB (2001). The PHQ‐9. Journal of General Internal Medicine, 16(9), 606–613. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Levine RE, Breitkopf CR, Sierles FS, & Camp G (2003). Complications associated with surveying medical student depression. Academic Psychiatry, 27(1), 12–18. [DOI] [PubMed] [Google Scholar]
  38. Löwe B, Decker O, Müller S, Brähler E, Schellberg D, Herzog W, & Herzberg PY (2008). Validation and standardization of the Generalized Anxiety Disorder Screener (GAD-7) in the general population. Medical Care, 46(3), 266–274. [DOI] [PubMed] [Google Scholar]
  39. Löwe B, Kroenke K, Herzog W, & Gräfe K (2004). Measuring depression outcome with a brief self-report instrument: sensitivity to change of the Patient Health Questionnaire (PHQ-9). Journal of Affective Disorders, 81, 61–66. [DOI] [PubMed] [Google Scholar]
  40. Martin A, Rief W, Klaiberg A, & Braehler E (2006). Validity of the brief patient health questionnaire mood scale (PHQ-9) in the general population. General Hospital Psychiatry, 28, 71–77. [DOI] [PubMed] [Google Scholar]
  41. Mazzola JJ, Walker EJ, Shockley KM, & Spector PE (2011). Examining stress in graduate assistants: Combining qualitative and quantitative survey methods. Journal of Mixed Methods Research, 5(3), 198–211. [Google Scholar]
  42. McLean CP, & Anderson ER (2009). Brave men and timid women? A review of the gender differences in fear and anxiety. Clinical Psychology Review, 29(6), 496–505. [DOI] [PubMed] [Google Scholar]
  43. Moreno-Peral P, Conejo-Cerón S, Rubio-Valera M, Fernández A, Navas-Campaña D, Rodríguez-Morejón A, … & Rodríguez-Bayón A (2017). Effectiveness of psychological and/or educational interventions in the prevention of anxiety: A systematic review, meta-analysis, and meta-regression. JAMA Psychiatry, 74(10), 1021–1029. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Mousa OY, Dhamoon MS, Lander S, & Dhamoon AS (2016). The MD blues: under-recognized depression and anxiety in medical trainees. PloS one, 11(6), e0156554. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Mouthaan J, Sijbrandij M, De Vries GJ, Reitsma JB, Van de Schoot R, Goslings JC, … & Olff M (2013). Internet-based early intervention to prevent posttraumatic stress disorder in injury patients: randomized controlled trial. Journal of Medical Internet Research, 15(8). e:165. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Musiat P, Conrod P, Treasure J, Tylee A, Williams C, & Schmidt U (2014). Targeted prevention of common mental health disorders in university students: Randomised controlled trial of a transdiagnostic trait-focused web-based intervention. PLoS One, 9(4), e93621. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Myers SB, Sweeney AC, Popick V, Wesley K, Bordfeld A, & Fingerhut R (2012). Self-care practices and perceived stress levels among psychology graduate students. Training and Education in Professional Psychology, 6, 55–66. [Google Scholar]
  48. Neil AL, Batterham P, Christensen H, Bennett K, & Griffiths KM (2009). Predictors of Adherence by Adolescents to a Cognitive Behavior Therapy Website in School and Community-Based Settings. Journal of Medical Internet Research, 11(1), e6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Novotney A (2014). Students under pressure. Monitor on Psychology, 45 Retrieved from: http://www.apa.org/monitor/2014/09/cover-pressure.aspx). [Google Scholar]
  50. Oswalt SB, & Riddock CC (2007). What to do about being overwhelmed: Graduate students, stress and university services. College Student Affairs Journal, 27, 24–44. [Google Scholar]
  51. Palmqvist B, Carlbring P, & Andersson G (2007). Internet-delivered treatments with or without therapist input: Does the therapist factor have implications for efficacy and cost? Expert Review of Pharmacoeconomics & Outcomes Research, 7(3), 291–297. [DOI] [PubMed] [Google Scholar]
  52. Pine DS, Cohen P, Gurley D, Brook J, & Ma Y (1998). The risk for early-adulthood anxiety and depressive disorders in adolescents with anxiety and depressive disorders. Archives of General Psychiatry, 55, 56–64. [DOI] [PubMed] [Google Scholar]
  53. Powell J, Hamborg T, Stallard N, Burls A, McSorley J, Bennett K, … & Christensen H (2013). Effectiveness of a web-based cognitive-behavioral tool to improve mental well-being in the general population: randomized controlled trial. Journal of Medical Internet Research, 15: e2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Rakesh G, Pier K, & Costales TL (2017). A Call for Action: Cultivating Resilience in Healthcare Providers. American Journal of Psychiatry Residents’ Journal, 12(4), 3–5. [Google Scholar]
  55. Reavley N, & Jorm AF (2010). Prevention and early intervention to improve mental health in higher education students: A review. Early Intervention in Psychiatry, 4(2), 132–142. [DOI] [PubMed] [Google Scholar]
  56. Roberge P, Fournier L, Duhoux A, Nguyen CT, & Smolders M (2011). Mental health service use and treatment adequacy for anxiety disorders in Canada. Social Psychiatry and Psychiatric Epidemiology, 46(4), 321–330. [DOI] [PubMed] [Google Scholar]
  57. Rummell CM (2015). An exploratory study of psychology graduate student workload, health, and program satisfaction. Professional Psychology: Research and Practice, 46(6), 391–399. [Google Scholar]
  58. Sánchez-Gutiérrez T, Barbeito S, & Calvo A (2017). A revision of preventive web-based psychotherapies in subjects at risk of mental disorders. International Journal of Interactive Multimedia & Artificial Intelligence, 4(5), 50–54. [Google Scholar]
  59. Sander L, Rausch L, & Baumeister H (2016). Effectiveness of Internet-based interventions for the prevention of mental disorders: a systematic review and meta-analysis. JMIR mental health, 3(3): e38. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Sareen J, Jagdeo A, Cox BJ, Clara I, ten Have M, Belik SL, … & Stein MB (2007). Perceived barriers to mental health service utilization in the United States, Ontario, and the Netherlands. Psychiatric Services, 58(3), 357–364. [DOI] [PubMed] [Google Scholar]
  61. Sethi S, Campbell AJ, & Ellis LA (2010). The use of computerized self-help packages to treat adolescent depression and anxiety. Journal of Technology in Human Services, 28(3), 144–160. [Google Scholar]
  62. Spek V, Cuijpers PIM, Nyklíček I, Riper H, Keyzer J, & Pop V (2007). Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: A meta-analysis. Psychological Medicine, 37(3), 319–328. [DOI] [PubMed] [Google Scholar]
  63. Spitzer RL, Kroenke K, Williams JB, & Löwe B (2006). A brief measure for assessing generalized anxiety disorder: The GAD-7. Archives of Internal Medicine, 166(10), 1092–1097. [DOI] [PubMed] [Google Scholar]
  64. Spitzer RL, Kroenke K, Williams JB, & Patient Health Questionnaire Primary Care Study Group. (1999). Validation and utility of a self-report version of PRIME-MD: The PHQ primary care study. JAMA, 282(18), 1737–1744. [DOI] [PubMed] [Google Scholar]
  65. Thompson A, Hunt C, & Issakidis C (2004). Why wait? Reasons for delay and prompts to seek help for mental health problems in an Australian clinical sample. Social Psychiatry and Psychiatric Epidemiology, 39(10), 810–817. [DOI] [PubMed] [Google Scholar]
  66. Tjia J, Givens JL, & Shea JA (2005). Factors associated with undertreatment of medical student depression. Journal of American College Health, 53(5), 219–224. [DOI] [PubMed] [Google Scholar]
  67. Twomey C, & O’Reilly G (2017). Effectiveness of a freely available computerised cognitive behavioural therapy programme (MoodGYM) for depression: Meta-analysis. Australian & New Zealand Journal of Psychiatry, 51(3), 260–269. [DOI] [PubMed] [Google Scholar]
  68. Tyssen R, Dolatowski FC, Røvik JO, Thorkildsen RF, Ekeberg Ø, Hem E, … & Vaglum P (2007). Personality traits and types predict medical school stress: a six‐year longitudinal and nationwide study. Medical Education, 41(8), 781–787. [DOI] [PubMed] [Google Scholar]
  69. Wang PS, Lane M, Olfson M, Pincus HA, Wells KB, & Kessler RC (2005). Twelve-month use of mental health services in the United States: Results from the National Comorbidity Survey Replication. Archives of General Psychiatry, 62(6), 629–640. [DOI] [PubMed] [Google Scholar]

RESOURCES