Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Mar 1.
Published in final edited form as: Depress Anxiety. 2019 Nov 4;37(3):202–213. doi: 10.1002/da.22969

A Pragmatic Clinical Trial Examining the Impact of a Resilience Program on College Student Mental Health.

E Akeman 1, N Kirlic 1, AN Clausen 5,6, KT Cosgrove 1,3, TJ McDermott 1,3, LD Cromer 3, MP Paulus 1, HW Yeh 4, RL Aupperle 1,2
PMCID: PMC7054149  NIHMSID: NIHMS1054964  PMID: 31682327

Abstract

Background:

One in three college students experience significant depression or anxiety interfering with daily functioning. Resilience programs that can be administered to all students offer an opportunity for addressing this public health problem. The current study objective was to assess the benefit of a brief, universal resilience program for first-year college students.

Method:

First-year students at a private, mid-western university participated. This trial used a pragmatic design, delivering the intervention within university-identified orientation courses and was not randomized. The four-session resilience program included goal-building, mindfulness, and resilience skills. The comparison was orientation-as-usual. Primary outcomes included PROMIS® Depression and Anxiety and Connor-Davidson Resilience Scale. Secondary and exploratory outcomes included the Perceived Stress Scale, Emotion Regulation and Cognitive-Behavioral Therapy (CBT) Skills Questionnaires, and Freiburg Mindfulness Inventory. Time by treatment interactions at post-training and semester-end were examined using linear mixed models.

Results:

Analysis included 252 students, 126 who completed resilience programming and a matched comparison sample. Resilience programming did not relate to improvements in depression at post-training (CI: −2.53 to 1.02; p = 0.404, d = −0.08), but did at semester-end (95% CI: −4.27 to −0.72; p = 0.006, d = −0.25) and improvements in perceived stress were observed at post-training (CI: −3.31 to −0.44; p = 0.011, d = −0.24) and semester-end (CI −3.30 to −0.41; p = 0.013, d = −0.24). Emotion regulation, mindfulness, and CBT skills increased, with CBT skills mediating clinical improvements.

Conclusions:

Universal implementation of a brief, resilience intervention may be effective for improving college student mental health.

Trial Registration:

ClinicalTrials.gov

Keywords: depression, anxiety, stress, resilience, cognitive-behavioral therapy, mindfulness, college students, prevention

Introduction

Exceedingly high numbers of college students experience clinically significant anxiety and depression (Ibrahim, Shelly, Adams, & Glazebrook, 2013; Smith et al., 2008), with the one-year prevalence rate of anxiety and depressive disorders estimated between 15–30% (Hunt & Eisenberg, 2010; Ibrahim et al., 2013). This is compared to rates of 7–18% for the general adult population (Kessler et al., 2005). The number of students seeking campus mental health services reportedly increased by 29% from 2009 to 2014, compared to only a 6% increase in college enrollment (Xiao et al., 2017). While the specific reasons for this are unknown, the increased demand for mental health treatment underscores the crisis facing colleges and universities and the more than 20 million students they serve (Xiao et al., 2017).

There is convincing evidence that the observed elevations in anxiety, depression, and stress lead to serious consequences for students. One in six students report acute suicidal ideation (Mortier et al., 2018). Further, even moderate levels of anxiety and depression can lead to lower levels of academic engagement and performance (Regehr, Glancy, & Pitts, 2013; Vaez & Laflamme, 2008) and greater likelihood of dropping out of college (Eisenberg, Golberstein, & Hunt, 2009; Mowbray et al., 2006), together having deleterious outcomes on future income and accumulated debt.

The National Institute of Mental Health recognizes as one of its priorities the development of early interventions that prevent disorders from fully emerging (National Institute of Mental Health, 2015), leading to increased interest in scalable programs delivered broadly within settings outside of the clinic. College represents a significant stressor with a known time of onset for a population at high-risk for anxiety and depression, but who is also a “captive audience” for intervention delivery (Eisenberg et al., 2009). As such, university settings offer a unique opportunity for development, evaluation, and widespread dissemination of programs targeting anxiety and depression (Hunt & Eisenberg, 2010). Scalable, universal prevention programming could be particularly important for college populations, considering fewer than half of students experiencing significant mental health symptoms seek out professional help (King et al., 2015).

Meta-analyses indicate that preventative cognitive-behavioral or mindfulness-based interventions have a significant impact on anxiety and depression for college students, with an average effect size of Cohen’s d = .29 (Conley, Durlak, & Kirsch, 2015; Regehr et al., 2013). However, the mean duration of these interventions is 13.5 hours, similar to manualized treatments for anxiety or depression. In addition, previously reported effect sizes seem to be lower with larger samples (d =.23 for samples >100), and when targeting first year students (d = .11) (Conley et al., 2015). Further, many interventions are not tailored to address student-specific stressors and/or target specific disorders (i.e., social anxiety disorder, depression) (Aune & Stiles, 2009; Beardslee et al., 2013). Barlow and colleagues recently reported on the feasibility and acceptability of a college student workshop based on the Unified Protocol for Transdiagnostic Treatment of Emotional Disorders (Bentley et al., 2018). However, there remains a need for controlled outcomes trials assessing the effectiveness of brief, scalable interventions on clinical outcomes for students.

The current study aimed to identify whether a brief, scalable, universal resilience program targeting students early in their college career could beneficially impact depression and anxiety. The trial was designed to be pragmatic to maximize generalizability. Thus, the intervention was a brief, four-session program delivered within first-year orientation courses. The term “resilience” program is used due to delivery of the intervention during a stressor, to all individuals, regardless of current levels of distress (Chmitorz et al., 2018). The skills targeted overlap significantly with cognitive-behavioral therapies (CBT) for anxiety and depression, with the additional target of growth mindset processes highlighted in psychological resilience research (Steinhardt & Dolbier, 2008). Anxiety and depression represented primary symptom outcomes, with a secondary outcome of perceived stress, as a transdiagnostic contributor to student mental health (Barker, Howard, Villemaire-Krajden, & Galambos, 2018; Zahniser & Conley, 2018). We also aimed to identify potential mechanisms of change to inform future intervention optimization. Psychological resilience was a primary mechanistic outcome, with emotion regulation, mindfulness, and CBT skills as exploratory mechanisms. We hypothesized that a brief, four-session resilience program delivered in the classroom setting would lead to improvements in depression, anxiety, and stress, and that these effects would be mediated by changes in psychological resilience and cognitive-behavioral, mindfulness, and emotion regulation skills.

Methods

Participants

A total of 364 first-year undergraduate students (53.3% females) from a private, mid-Western university voluntarily enrolled in the study; 306 completed baseline measures; 101 were in Cohort 1 (enrolled 09/2016 – 11/2016) and 205 were in Cohort 2 (enrolled 09/2017 – 11/2017). Exclusion criteria were kept to a minimum to increase pragmatism and generalizability and included being under 18 years of age, unable to understand surveys presented in English, not in the first year of university enrollment, or reporting significant mental or physical health problems requiring immediate attention. International students were excluded due to federal and university regulations preventing receipt of research compensation. All subjects provided written informed consent prior to completing the study protocol, and were compensated $10 per survey time point. Students were not compensated for completing the intervention. Research was approved by the Western Institutional Review Board and conducted in accordance with the World Medical Association Declaration of Helsinki. The study was registered at the US National Institutes of Health (ClinicalTrials.gov) . The Consolidated Standards of Reporting Trials (CONSORT) diagram is provided in Figure 1.

Figure 1. CONSORT Diagram.

Figure 1.

Open in a new tab

Figure shows the flow of participants through the phases of the study and the number that withdrew at each time point. As indicated, linear mixed-effects models (LMMs) allowed inclusion of participants with data at any time point to be included in final analysis.

Trial Design

Refer to Figure S1 for investigator rating of the pragmatism of this trial using the PRagmatic-Explanatory Continuum Indicator Summary – PRECIS-2 wheel (Loudon, Zwarenstein, Sullivan, Donnan, & Treweek, 2013). Resilience programming was delivered within the real-world university setting and as such, condition assignment was not randomized. Resilience programming was provided to a subset of first-year orientation classes within each college (Health Sciences, Arts and Sciences, Business, Engineering and Natural Sciences), as selected by each college Dean. All students within these courses were provided training, but were not required to participate in the research study. Students in the comparison group were recruited from first-year orientation classes that did not receive the training. Of the 306 participants, 126 completed the resilience program while the remaining completed ‘orientation-as-usual’. Due to the nature of the intervention, neither participants nor those delivering the intervention were blind to group assignment. Participants were asked to complete self-report questionnaires pre-training, weekly during training, post-training, and at semester-end (12/2016 for Cohort 1; 12/2017 for Cohort 2). The time points for students in the orientation-as-usual condition matched that of students within the same college who were completing resilience programming.

Intervention

An overview of intervention content is displayed in Table 1; detailed information is provided in the supplement. All materials are provided at https://osf.io/u75tj/. The resilience program consisted of four weekly sessions, each lasting approximately 50 minutes. Class size ranged from approximately 10 to 40 students. The resilience program was developed and supervised by licensed clinical psychologists (RLA, LDC) and led by clinical psychology postdoctoral fellows or doctoral students. Trained undergraduate volunteers and research assistants served as small group facilitators (see supplement for supervision and training procedures). Intervention content was informed by previous behavioral activation (Martell, Dimidjian, & Herman-Dunn, 2013), cognitive-behavioral (Hofmann, Asnaani, Vonk, Sawyer, & Fang, 2012) and mindfulness interventions (Danitz & Orsillo, 2014; Hofmann & Gomez, 2017; Segal, Williams, & Teasdale, 2013), as well as resilience interventions focusing on the growth mindset, which involves viewing challenges and failure as opportunities to learn and improve (Steinhardt & Dolbier, 2008; Yeager & Dweck, 2012). Briefly, the four sessions included information and practice on the following resilience components: (session 1) value-driven and goal-oriented behavior, (session 2) mindfulness practice, and (sessions 3–4) cognitive restructuring strategies to implement growth mindset responses to college-related challenges and stressors, including the identification of potential campus/community resources. A feedback form was created to assess each participant’s understanding, compliance, and experience with the intervention (Table S1).

Table 1.

Brief overview of session content.

Session # Title and Content
1 Finding Meaning and Focus
Session content: Introduction to resilience program, identifying individual values and how these can be used to motivate goals, discussion and practice of “quiet” mindfulness (focusing on internal sensations and external environment).
Homework: Add to values list, identify one long-term goal, practice mindfulness daily.
2 Be Calm and Take Action!
Session content: Homework review, discussion and practice of “active” mindfulness (being mindful during active tasks), building process and outcome goals using SMART goals and goal ladders.
Homework: Add to goal ladders, complete process goals, practice mindfulness daily.
3 Rise Above
Session content: Homework review, discussion and practice of “quiet” mindfulness, discussion of growth versus fixed mindset.
Homework: Completion of process goals and daily mindfulness practice, and identification of growth mindset thoughts and actions for three challenges experienced.
4 Bringing it All Together
Session content: Homework review, discussion and practice of “active” mindfulness, discussion of seeking out resources as part of growth mindset and flexible use of skills learned.
Homework: Continue on own with completion of process goals and daily mindfulness practice, and identification of growth mindset responses and resources for challenges experienced.
Open in a new tab

Outcome Measures

The primary outcome measures included the National Institute of Health Patient Reported Outcome Measurement Information System (PROMIS) computer-adaptive Depression and Anxiety symptom measures(Cella et al., 2010; Gershon, Rothrock, Hanrahan, Bass, & Cella, 2010), and the Connor-Davidson Resilience Scale (CD-RISC 10) total score (Connor & Davidson, 2003) as a measure of a potential psychological mechanism. The secondary mental health outcome was the Perceived Stress Scale (PSS) (S. Cohen, Kamarak, & Mermelstein, 1983). Exploratory psychological mechanisms were assessed using the Emotion Regulation Questionnaire (ERQ) (Gross & O.P., 2003), mindfulness skills as measured by the Freiburg Mindfulness Inventory (FMI) (Walach, Buchheld, Buttenmüller, Kleinknecht, & Schmidt, 2006), and a modified version of the CBT Skills Questionnaire (Jacob, Christopher, & Neuhaus, 2011). Mental health symptom, resilience, and emotion regulation measures were administered at baseline, post-training, and at semester-end. The specific skills-based questionnaires (FMI, CBT skills) were administered at baseline, weekly during training, and at post-training. All measures were administered via survey links through Research Electronic Data Capture (REDCap) (Harris et al., 2009). For a survey to be submitted, participants had to complete all fields on that survey. Thus, there were no missing values for measures completed by participants. Further description of all measures are provided in the supplement.

Statistical Methods

Statistical analyses were conducted using R statistical package (R Core Team, 2013). Given identified baseline differences between resilience and orientation-as-usual groups on primary outcomes, orientation-as-usual cases were identified to match resilience program cases on PROMIS® Depression and Anxiety, CD-RISC, and sex at the individual level, using the nearest neighbor method based on Euclidean distance. Matching was implemented by R package e1071 (Meyer, Dimitriadou, Hornik, Weingessel, & Leisch, 2017), see supplement for further description). This resulted in a total sample of 126 × 2 = 252 for analysis. To focus on the most pragmatic, “intent-to-treat” effect, all matched participants were included in analysis. Linear mixed model analyses (LMM; conducted by lme4 package (Bates, Maechler, Bolker, & Walker, 2015)), with subject as random effects, were fit to measurements available at each time point to test whether within-subject changes differed by treatment condition, with post-training representing the primary endpoint and semester-end representing the secondary endpoint. These endpoints were assessed using their corresponding interaction terms (i.e. training-by-post-training and training-by-semester-end, using non-training and pre-intervention as reference categories for group and time, respectively). The overall time (pre, post-training, semester-end) by training interaction effects were also reported. The inclusion of potential covariates (sex; college; cohort) were determined by comparing models using the Bayesian Information Criterion (Bicanic, Hehenkamp, van de Putte, van Wijk, & de Jongh). For each time point, 95% confidence intervals, p-values, and effect size (Cohen’s d) (J. Cohen, 1988) were estimated by stats (R Core Team, 2013), lmerTest (Kuznetsova, Brockhoff, & Christensen, 2017), and EMAtools (Kleiman, 2017) packages. Results at each endpoint (post-training, semester-end) were Bonferroni corrected for multiple comparisons with three primary outcome variables (depression, anxiety, resilience), resulting in a p-value cutoff of 0.017. Changes in secondary and exploratory outcomes (stress, emotion regulation, mindfulness, CBT skills) were examined using the same LMM approach. Using Bonferroni correction for multiple comparisons would result in p < 0.013 for the secondary and exploratory outcomes. We conducted separate mediation models to determine whether change in ERQ reappraisal, CBT skills or FMI score (from baseline to post-training) mediated the relationship between training condition and change in depression symptoms and perceived stress (from baseline to semester-end). We used the R package MBESS (Kelley, 2018) to conduct mediation models, with a bias-corrected 95% bootstrap confidence interval for indirect effects based on 2,000 bootstrap samples. R package bda (Wang, 2018) was used to conduct a Sobel test for significance of the mediation.

Sample Size and Power

According to Conley et al. (Conley et al., 2015), the average effect size of intervention programs previously implemented in college populations is d = 0.29. We assessed (1) required sample size for 80% power and (2) achieved power for N = 126 per group, given the intra-cluster correlations (ICC) observed for PROMIS® Depression, Anxiety, and CD-RISC (0.62 – 0.70). Simulations indicated that to achieve 80% power for the interaction term at two-sided α = 0.017, we would need a sample size of 150 (for ICC = 0.7) to 200 per group (for ICC = 0.6). The current sample size at baseline was 306, with 252 (N = 126 per group) after matching procedures. This sample size was a product of researcher and university commitment of assessing intervention impact with two student cohorts (2016, 2017). The study was ended due to university requests for broader implementation of the intervention, thus preventing further examination of controlled outcomes. Simulation suggested the 252 matched sample size would provide 56% (for ICC = 0.6) to 72% power (ICC = 0.7) to detect d = .29 effect size at the adjusted p < .017 threshold. Please see supplement for further details concerning power analyses.

Data Sharing

The study protocol, intervention materials, and R code can be found on the open science framework at https://osf.io/u75tj/. The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.

Results

Descriptive Analysis

Demographic and baseline scores for the matched training groups are included in Table 2 (original sample demographics provided in Table S2). Using cutoff scores suggested for PROMIS® measures (Choi, Schalet, Cook, & Cella, 2014), approximately 19% and 18% of students reported moderate to severe levels of depression (T>60) and anxiety (T>62), respectively. An additional 26% and 31% reported mild symptoms of depression (T>56) or anxiety (T>55). On average, resilience training participants attended 3.47 (SD = 1.01) of four training sessions. Study attrition did not differ between training groups (based on survey completion rates; Depression: X2(2) = 0.09, p = 0.955; Anxiety: X2(2) = 0.06, p = 0.967; CD-RISC: X2(2) = 0.04, p = 0.978). No adverse events were reported by participants.

Table 2.

Baseline Demographics and Clinical Characteristics of Matched Sample

Characteristics Resiliency Training (N = 126) No Training (N = 126) Group Differences
Age, Mean (SD) 18.81 (1.22) 18.83 (1.13) t = −0.08 (p = 0.935)
Sex, N (%) χ2 = 0.00 (p = 1.00)
 Male 53 (42%) 53 (42%)
 Female 73 (58%) 73 (58%)
Ethnicity, N (%) χ2 = 0.05 (p = 0.830)
 Hispanic 11 (9%) 13 (10%)
 Non-Hispanic 115 (91%) 115 (90%)
Race, N (%) χ2 = 0.05 (p = 0.830)
 White 84 (67%) 93 (74%)
 American Indian or Alaska Native 3 (2%) 4 (3%)
 Black or African-American 4 (3%) 5 (4%)
 Asian or Pacific Islander 7 (6%) 6 (5%)
 Middle-Eastern 1 (1%) 1 (1%)
 More than one race 26 (21%) 15 (12%)
 ”Other”, unspecified 1 (1%) 2 (2%)
Psychotropic medication use, N (%) 4 (3%) 11 (9%) χ2 = 2.55 (p = 0.110)
Psychotherapy in past 3 months, N (%) 9 (7%) 14 (11%) χ2 = 0.766 (p = 0.382)
Annual parent or household income, N (%) χ2 = 14.30 (p = 0.217)
 $50,000 and less 36 (29%) 43 (34%)
 $50,000 – $100,000 32 (25%) 32 (25%)
 $100,000 – $150,000 21 (17%) 25 (20%)
 $150,000 and over 37 (29%) 26 (21%)
High school GPA, Mean (SD) 3.91 (0.44) 3.95 (0.48) t = −0.63 (p = 0.530)
Baseline Assessments, Mean (SD)
 PROMIS Depression 51.96 (7.46) 51.79 (7.60) t = −0.177 (p = 0.860)
 PROMIS Anxiety 54.56 (8.55) 54.96 (8.14) t = −0.376 (p = 0.707)
 Perceived Stress Scale 17.30 (6.71) 17.74 (7.26) t = 0.495 (p = 0.621)
 CD-RISC 29.0 (6.84) 29.01 (6.61) t = −0.009 (p = 0.993)
 ERQ reappraisal 4.71 (1.15) 4.74 (1.09) t = −0.213 (p = 0.832)
 ERQ suppression 3.98 (1.21) 3.97 (1.24) t = 0.059 (p = 0.953)
 FMI 39.25 (7.86) 39.05 (7.77) t = 0.201 (p = 0.841)
 CBT Skills 69.47 (13.26) 71.09 (13.97) t = −0.933 (p = 0.352)
College, N (%) χ2 = 1.21 (p = 0.270)
 A&S College 32 (25%) 42 (33%)
 HS College 39 (31%) 27 (22%)
 Business College 25 (20%) 18 (15%)
 Eng&NS College 30 (24%) 38 (29%)
Cohort, N (%) χ2 = 0.875 (p = 0.350)
 Year 1 46 (37%) 38 (30%)
 Year 2 80 (63%) 88 (70%)
Attendance at Training (available for N=108), N (%)
 all 4 sessions 77 (71%) -
 3/4 sessions 18 (16%) -
 2/4 sessions 6 (5%) -
 1/4 sessions 4 (4%) -
 0/4sessions 4 (4%) -
Open in a new tab

Abbreviations: PROMIS, Patient Reported Outcome Measurement Information System; FMI, Freiburg Mindfulness Inventory; CBT, Cognitive behavioral therapy; GPA, grade point average; A&S, Arts and Sciences; HS, Health Sciences; Eng&NS, Engineering and Natural Sciences.

Outcome Analysis

Improvements in depression were not observed at post-training (t(453) = −0.84, β = −0.76, 95% CI: −2.53 to 1.02, p = 0.404), but were observed at semester-end (t(453) = −2.76, β = −2.50, 95% CI: −4.27 to −0.72, p = 0.006) and a significant group (training versus no-training) by time (pre, post, semester-end) interaction was also observed (“interaction” hereafter: F(2, 451) = 3.94, p = 0.020). Anxiety symptoms did not significantly differ at post-training (t(445) = −1.18, β = −1.14, 95% CI: −3.04 to 0.76, p = 0.239) or semester-end (t(445) = −1.56, 95% CI: −3.46 to 0.34, p = 0.108; interaction: F(2, 444) = 1.41, p = 0.246). Significant improvements were identified for perceived stress at both post-training (t(451) = −1.55, β = −1.88, 95% CI: −3.32 to −0.43, p = 0.011) and semester-end (t(451) = −2.51, β = −1.86, p = 0.013; 95% CI: −3.31 to −0.41, interaction: F(2, 450 = 4.33, p = 0.014). Trajectory of symptoms are displayed in Figure 2.

Figure 2. Mental health symptom scores by training group over time.

Figure 2.

Open in a new tab

Shown here are interaction plots of estimated marginal means based on the fitted linear mixed-effects model for each outcome measure: (A) NIH PROMIS® Depression, (B) NIH PROMIS® Anxiety, (C) Perceived Stress Scale. Error bars represent 95% confidence intervals.

Concerning potential psychological mechanisms, no significant time by training effects were observed for CD-RISC score at either post-training (t(448) = 0.82, β = 0.58, 95% CI: −0.82 to 1.98, p = 0.417) or semester-end: t(449) = 0.92, β = −0.66, 95% CI: −0.75 to 2.07, p = 0.356; interaction: F(2, 447) = 0.92, p = 0.598). However, significant time by training effects were identified for ERQ reappraisal (post-training: t(446) = 2.07, β = −1.88, 95% CI: −3.32 to −0.43, p = 0.039; semester-end: t(448) = 2.77, p = 0.006; interaction: F(2, 445) = 4.19, p = 0.016), CBT skills (post-training: t(1143) = 4.19, p < 0.001; interaction: F(5, 1140) = 6.33, p < 0.001), and FMI score (post-training: t(1146) = 1.70, p = 0.089; interaction: F(5, 1143) = 8.04, p < 0.001). Trajectory of scores are displayed in Figure 3. Full statistical results are provided in Table 3.

Figure 3. Change in psychological skills by training group over time.

Figure 3.

Open in a new tab

Shown here are interaction plots of estimated marginal means based on the fitted linear mixed-effects model for each outcome measure: (A) Connor-Davidson Resilience Inventory (CD-RISC 10), (B) CBT Skills Questionnaire, (C) Emotion Regulation Questionnaire (ERQ) Reappraisal subscale, (D) Frieberg Mindfulness Inventory. Error bars represent 95% confidence intervals.

Table 3.

Unadjusted Means, Confidence Intervals, Effect Sizes and Main Analyses of Change from Baseline in Resilience Training Compared to No Training.

Outcome Variable Sample size Unadjusted Outcomes (Means, Confidence Intervals) Effect Sizes Main Analyses
Training No-training Training No-training Cohen’s d Mixed Model-Based Results: Effects of Change from Baseline for Training compared to No Training
Training Effect (SE) t (p-value)
PROMIS Depression Covariates: sex; random factors: subject
 Pre-training 126 126 51.96 (50.65 – 53.27) 51.79 (50.45 – 53.13)
 Post-training 113 112 51.56 (49.97 – 53.15) 52.08 (50.70 – 53.46) −0.08 −.76 (0.91) −0.84 (0.404)
 Semester End 110 115 49.66 (49.66 – 51.26) 52.47 (50.94 – 54.00) −0.26 −2.50 (0.91) −2.75 (0.006)
PROMIS Anxiety Covariates: sex; random factors: subject
 Pre-training 126 125 54.56 (53.05 – 56.08) 54.96 (53.53 – 56.40)
 Post-training 112 112 53.68 (51.90 – 55.46) 55.13 (53.60 – 56.66) −0.11 −1.14 (0.97) −1.18 (0.239)
 Semester End 115 110 52.64 (51.0 – 54.28) 54.89 (53.38 – 56.40) −0.15 −1.56 (.97) −1.61 (0.108)
Perceived Stress Covariates: sex; random factors: subject
 Pre-training 123 126 17.74 (16.44 – 19.04) 17.30 (16.12 – 18.48)
 Post-training 112 112 16.11 (14.74 – 17.47) 17.48 (16.13 – 18.84) −0.24 −1.88 (0.735) −2.55 (0.011)
 Semester End 108 112 15.79 (14.42 – 17.15) 17.25 (15.94 – 18.56) −0.24 −1.86 (0.740) −2.51 (0.013)
CD-RISC Covariates: sex; random factors: subject
 Pre-training 126 122 29.00 (27.77 – 30.23) 29.01 (27.84 – 30.17)
 Post-training 111 111 29.24 (27.94 – 30.55) 28.54 (27.31 – 29.79) 0.08 0.58 (0.72) 0.81 (0.417)
 Semester End 112 108 29.12 (27.76 – 30.48) 28.53 (27.09 – 29.96) 0.09 0.66 (0.72) 0.92 (0.231)
ERQ reappraisal Covariates: none; random factors: subject
 Pre-training 122 126 4.72 (4.51 – 4.92) 4.75 (4.55 – 4.94)
 Post-training 111 111 5.08 (4.88 – 5.27) 4.77 (4.53 – 5.00) 0.20 0.27 (0.13) 2.07 (0.039)
 Semester End 107 112 5.10 (4.89 – 5.31) 4.73 (4.50 – 4.95) 0.26 0.36 (0.13) 2.77 (0.006)
ERQ suppression Covariates: none; random factors: subject
 Pre-training 126 122 3.98 (3.76 – 4.19) 3.97 (3.75 – 4.19)
 Post-training 111 111 3.93 (3.72 – 4.15) 3.98 (3.75 – 4.20) −0.02 −0.03 (0.13) −0.24 (0.808)
 Semester End 112 107 4.02 (3.78 – 4.26) 4.04 (3.80 – 4.29) 0.01 0.02 (0.13) 0.13 (0.898)
FMI Covariates: none; random factors: subject
 Pre-training 121 126 39.25 (37.83 – 40.66) 39.05 (37.68 – 40.42)
 Week 1 117 120 38.51 (37.18 – 39.85) 40.33 (39.01 – 41.66) −0.17 −1.97 (0.68) −2.89 (0.004)
 Week 2 117 120 38.68 (37.43 – 39.94) 39.17 (37.85 – 40.49) −0.07 −0.80 (0.68) −1.16 (0.245)
 Week 3 112 116 40.37 (39.04 – 41.69) 38.88 (37.52 – 40.24) 0.08 0.95 (0.69) 1.37 (0.173)
 Week 4 114 117 41.46 (39.98 – 42.95) 39.40 (38.04 – 40.76) 0.15 1.77 (0.69) 2.56 (0.011)
 Post-training 111 110 41.64 (40.23 – 43.05) 40.06 (38.49 – 41.63) 0.10 1.19 (0.70) 1.70 (0.089)
CBT Skills Covariates: none; random factors: subject
 Pre-training 120 126 69.47 (67.07 – 71.86) 71.09 (68.62 – 73.55)
 Week 1 117 120 69.86 (67.40 – 72.32) 71.31 (69.12 – 73.50) 0.00 0.09 (1.19) 0.08 (0.937)
 Week 2 117 119 71.53 (69.07 – 73.99) 70.36 (67.98 – 72.75) 0.13 2.64 (1.19) 2.21 (0.027)
 Week 3 112 116 73.59 (70.93 – 76.25) 70.52 (68.02 – 73.02) 0.19 3.85 (1.21) 3.19 (0.002)
 Week 4 114 117 74.43 (71.66 – 77.20) 71.21 (68.61 – 73.82) 0.21 4.17 (1.20) 3.47 (<0.001)
 Post-training 111 109 76.08 (73.59 – 78.58) 71.92 (69.18 – 74.66) 0.25 5.11 (1.22) 4.19 (<0.001)
Open in a new tab

Note: Gender, college, cohort, and gender considered as potential covariates; the model with the lowest BIC is reported. Abbreviations: PROMIS= Patient-Reported Outcomes Measurement Information System; CD-RISC = Connor-Davidson Resilience Scale; ERQ = Emotion Regulation Questionnaire; FMI = Frieburg Mindfulness Inventory; CBT = Cognitive Behavioral Therapy; BIC = Bayesian Information Criterion; SE = standard error).

Mediation Analysis

Mediation analyses (Tables S3 and S4) indicated that the relationship between intervention condition and change in depression symptoms and perceived stress (from baseline to semester-end) were driven by the effect of change (baseline to post-training) in CBT skills (Sobel test; Depression: z = −2.52, p = .012; Stress: z = −2.65, p = .008) but not FMI (Depression: z = −1.40, p = 0.161; Stress: z = −1.88, p = .060) or ERQ reappraisal scores (Depression: z = −1.17, p = 0.242; Stress: z = −1.13, p = .257).

Discussion

The current study used a pragmatic clinical trial to identify whether a brief, scalable, universal resilience program could be beneficial for first-year college student mental health. Overall, the resilience program was feasible and acceptable to students and was effective at reducing levels of depression and stress, particularly as reported at the end of semester. The intervention had significant effects on CBT, mindfulness, and emotion regulation skills, with CBT skills serving as a mediator for intervention effects.

Effect sizes for the current intervention at semester-end were similar to that found for previous interventions with college populations (Conley et al., 2015). The group who received resilience programming had average depression scores approximately three T scores lower than the comparison group. This difference is similar to the minimally important differences (2.2 – 4.5) identified for this measure (Lee et al., 2017; Pilkonis et al., 2014; Yost, Eton, Garcia, & Cella, 2011), supporting the clinical relevance of effects. This was despite the current program being brief in duration, delivered in the classroom setting, and targeting first-year students regardless of baseline symptom severity (limiting potential for change in symptom scores) (Hunt & Eisenberg, 2010; Ibrahim et al., 2013).

The most robust findings were exhibited at the end of the first semester rather than post-training. It is possible effects were most evident during a defined stressor (e.g., finals), given training focused on increasing one’s ability to respond optimally to stressors. However, it is also possible that students in the training group may have continued to practice skills learned - thus leading to more significant gains over time. The impact on perceived stress was evident at both post-training and semester end, which is important given its predictive relationship to longitudinal outcomes (Barker et al., 2018; Zahniser & Conley, 2018). In comparison to prior studies (Conley et al., 2015), the current intervention more robustly impacted depression rather than anxiety. This may be due to differences in intervention content, outcome measures used, or that our comparison group exhibited patterns of increased depressive but not anxiety symptoms. Based on observed effect sizes, the “number needed to treat” to effect significant benefit ranges from 7 (depression measures) to 12 (anxiety measures) students (Preti, 2015). These potential benefits combined with the relative brevity and low cost (estimated $50–60/student) may lead universities to consider adopting such programs into first year orientation courses.

Significant changes were identified in CBT, emotional reappraisal, and mindfulness skills, indicating that the current intervention was effective at communicating and teaching the targeted skills. However, psychological resilience as measured by the CD-RISC was not impacted. This may be due to the CD-RISC assessing more stable beliefs rather than specific skills. Future research related to resilience training may consider using a variety of measures to assess modifiability of resilience factors (Cassidy, 2016; Julian et al., 2018; Smith et al., 2008; Windle, Bennett, & Noyes, 2011). Notably, CBT skills emerged as the only mediator for effects on depression symptoms and perceived stress. This serves to highlight the importance of cognitive-behavioral skills in preventing the rise of mental health symptoms and corroborates the plethora of evidence supporting CBT for depression (Cuijpers, van Straten, & Warmerdam, 2007; Hofmann et al., 2012).

Limitations and future research

Strengths of the current study include the focus on resilience programming that was brief, scalable, and delivered to the general student population, within the existing infrastructure of a university, thus increasing generalizability and potential impact. However, the present study was limited in that students were not randomized to interventions, diagnostic mental health status was not obtained, and the trial was not blinded. While randomization of participants to intervention condition would have been optimal, it was not feasible when conducted within university infrastructure. Lastly, the current analysis only included follow-up data through the end of the first semester. Further research is needed to assess for longer-term benefits of such an intervention.

Generalizability

This current sample consisted of first-year students at a private, midwestern university, which was of higher socioeconomic status and over-represented in regards to White and Native American populations, and under-represented in regards to Hispanic and Latino and Black and African American students, compared to the average student population (Chetty, Friedman, Saez, Turner, & Yagan, 2017; National Science Foundation, 2017). Future work is needed to establish whether findings would generalize to other universities or colleges, diverse student populations, or other contexts (e.g., high school, community colleges, employment settings).

Conclusion

The current study presented results from a brief, scalable, universal resilience program for first-year college students. This intervention had a positive impact on self-reported depression symptoms and perceived stress during the first semester of college as compared to students who did not receive training. Changes in CBT, mindfulness, and emotion regulation skills were observed as a result of the intervention, with CBT skills serving as a mediator for clinical improvements. Results demonstrate that widespread implementation of a brief intervention focused on CBT, mindfulness, and resilience-based skills is an effective way for universities to begin combating the notable rise in student mental health difficulties.

Supplementary Material

Supp info

Acknowledgements

We would like to acknowledge the support and work of administration and staff at The University of Tulsa that assisted in the planning stages and made it possible to incorporate training into first-year orientation courses. We also acknowledge the contribution of the graduate and undergraduate students who provided the intervention and the undergraduate students who volunteered their time as research participants. Without the dedication of students, staff, and administration at The University of Tulsa, this work would not have been possible.

Conflict of Interest

Dr. Cromer, Dr. Yeh, Elisabeth Akeman, Kelly Cosgrove, and Tim McDermott have no conflicts of interest to declare. Dr. Kirlic reports grants from National Institute for General Medical Sciences. Dr. Clausen is funded by the Department of Veterans Affairs Office of Academic Affiliations Advanced Fellowship Program in Mental Illness Research and Treatment, the Medical Research Service of the Durham VA Health Care System, and the Department of Veterans Affairs Mid-Atlantic Mental Illness Research, Education, and Clinical Center (MIRECC). Dr. Paulus has received royalties for an article about methamphetamine in UpToDate. Dr. Aupperle reports grants from National Institute of Mental Health, National Institute of General Medical Sciences, and the Oklahoma Science and Technology Research and Development. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States Government.

Financial Support

The current study was funded by the William K. Warren Foundation. The study sponsor had no direct role in data collection, analysis, or interpretation; trial design; patient recruitment; or any aspect pertinent to the study. The authors were not paid to write this article by a pharmaceutical company or other agency. The authors had full access to all the data in the study and had final responsibility for the decision to submit for publication.

Funding Source: William K. Warren Foundation

References

  1. Aune T, & Stiles TC (2009). Universal-based prevention of syndromal and subsyndromal social anxiety: A randomized controlled study. Journal of Consulting and Clinical Psychology, 77(5), 867–879. doi: 10.1037/a0015813 [DOI] [PubMed] [Google Scholar]
  2. Barker ET, Howard AL, Villemaire-Krajden R, & Galambos NL (2018). The Rise and Fall of Depressive Symptoms and Academic Stress in Two Samples of University Students. Journal of Youth and Adolescence, 47(6), 1252–1266. doi: 10.1007/s10964-018-0822-9 [DOI] [PubMed] [Google Scholar]
  3. Bates D, Maechler M, Bolker B, & Walker S (2015). Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software, 67(1), 1–48. doi: 10.18637/jss.v067.i01 [DOI] [Google Scholar]
  4. Beardslee WR, Brent DA, Weersing VR, Clarke GN, Porta G, Hollon SD, … Garber J (2013). Prevention of depression in at-risk adolescents: longer-term effects. JAMA Psychiatry, 70(11), 1161–1170. doi: 10.1001/jamapsychiatry.2013.295 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Bentley KH, Boettcher H, Bullis JR, Carl JR, Conklin LR, Sauer-Zavala S, … Barlow DH (2018). Development of a Single-Session, Transdiagnostic Preventive Intervention for Young Adults at Risk for Emotional Disorders. Behavior Modification, 42(5), 781–805. doi: 10.1177/0145445517734354 [DOI] [PubMed] [Google Scholar]
  6. Bicanic IAE, Hehenkamp LM, van de Putte EM, van Wijk AJ, & de Jongh A (2015). Predictors of delayed disclosure of rape in female adolescents and young adults. European Journal of Psychotraumatology, 6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Cassidy S (2016). The Academic Resilience Scale (ARS-30): A new multidimensional construct measure. Frontiers in Psychology, 7(1787). [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Cella D, Riley W, Stone A, Rothrock N, Reeve B, Yount S, … Group PC (2010). The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. Journal of Clinical Epidemiology, 63(11), 1179–1194. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Chetty R, Friedman JN, Saez E, Turner N, & Yagan D (2017). Mobility report cards: The role of colleges in intergenerational mobility. Retrieved from [Google Scholar]
  10. Chmitorz A, Kunzler A, Helmreich I, Tuscher O, Kalisch R, Kubiak T, … Lieb K (2018). Intervention studies to foster resilience - A systematic review and proposal for a resilience framework in future intervention studies. Clinical Psychology Review, 59, 78–100. [DOI] [PubMed] [Google Scholar]
  11. Choi SW, Schalet B, Cook KF, & Cella D (2014). Establishing a common metric for depressive symptoms: linking the BDI-II, CES-D, and PHQ-9 to PROMIS depression. Psychological Assessment, 26(2), 513–527. doi: 10.1037/a0035768 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Cohen J (1988). Statistical Power Analysis for the Behavioral Sciences. New York, NY: Routledge Academic. [Google Scholar]
  13. Cohen S, Kamarak T, & Mermelstein R (1983). A global measure of percieved stress. Journal of Health and Social Behavior, 24(4), 385–396. [PubMed] [Google Scholar]
  14. Conley CS, Durlak JA, & Kirsch AC (2015). A meta-analysis of universal mental health prevention programs for higher education students. Prevention Science, 16(4), 487–507. [DOI] [PubMed] [Google Scholar]
  15. Connor KM, & Davidson JR (2003). Development of a new resilience scale: The Connor-Davidson Resilience Scale (CD-RISC). Depression and Anxiety, 18(2), 76–82. [DOI] [PubMed] [Google Scholar]
  16. Cuijpers P, van Straten A, & Warmerdam L (2007). Behavioral activation treatments of depression: a meta-analysis. Clinical Psychology Review, 27(3), 318–326. doi: 10.1016/j.cpr.2006.11.001 [DOI] [PubMed] [Google Scholar]
  17. Danitz SB, & Orsillo SM (2014). The mindful way through the semester: An investigation of the effectiveness of an acceptance-based behavioral therapy program on psychological wellness in first-year students. Behavior Modification, 38(4), 549–566. [DOI] [PubMed] [Google Scholar]
  18. Eisenberg D, Golberstein E, & Hunt J (2009). Mental health and academic success in college. The B E Journal of Economic Analysis & Policy, 9(1), Article 40. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Gershon RC, Rothrock N, Hanrahan R, Bass M, & Cella D (2010). The use of PROMIS and assessment center to deliver patient-reported outcome measures in clinical research. Journal of Applied Measurement, 11(3), 304–314. [PMC free article] [PubMed] [Google Scholar]
  20. Gross JJ, & J. OP (2003). Individual differences in two emotion regulation processes: Implications for affect, relationships, and well-being. . Journal of Personality and Social Psychology, 85, 348–362. [DOI] [PubMed] [Google Scholar]
  21. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, & Conde JG (2009). Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics, 42(2), 377–381. doi: 10.1016/j.jbi.2008.08.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Hofmann SG, Asnaani A, Vonk IJ, Sawyer AT, & Fang A (2012). The Efficacy of Cognitive Behavioral Therapy: A Review of Meta-analyses. Cognitive Therapy and Research, 36(5), 427–440. doi: 10.1007/s10608-012-9476-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Hofmann SG, & Gomez AF (2017). Mindfulness-Based Interventions for Anxiety and Depression. Psychiatric Clinics of North America, 40(4), 739–749. doi: 10.1016/j.psc.2017.08.008 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Hunt J, & Eisenberg D (2010). Mental health problems and help-seeking behavior among college students. Journal of Adolescent Health, 46, 3–10. [DOI] [PubMed] [Google Scholar]
  25. Ibrahim AK, Shelly SJ, Adams CE, & Glazebrook C (2013). A systematic review of studies of depression prevalence in university students. Journal of Psychiatric Research, 47(3), 391–400. [DOI] [PubMed] [Google Scholar]
  26. Jacob KL, Christopher MS, & Neuhaus EC (2011). Development and validation of the cognitive-behavioral therapy skills questionnaire. Behavior Modification, 35(6), 595–618. [DOI] [PubMed] [Google Scholar]
  27. Julian M, Cheadle AC, Jurkiewicz O, Bilder RM, Knudsen KS, & Schetter CD (2018). Resilience Resources Scale (RRS): A new resilience scale for use in young adults. Paper presented at the Psychosomatic Medicine. [Google Scholar]
  28. Kelley K (2018). MBESS: The MBESS R Package. In. Retrieved from https://CRAN.R-project.org/package=MBESS [Google Scholar]
  29. Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, & Walters EE (2005). Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Co-morbidity Survey Replication. Archives of General Psychiatry, 62(6), 593–602. [DOI] [PubMed] [Google Scholar]
  30. King CA, Eisenberg D, Zheng K, Czyz E, Kramer A, Horwitz A, & Chermack S (2015). Online suicide risk screening and intervention with college students: A pilot randomized controlled trial. Journal of Consulting and Clinical Psychology, 83(3), 630–636. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Kleiman E (2017). EMAtools: Data Management Tools for Real-Time Monitoring/Ecological Momentary Assessment Data. In. Retrieved from https://CRAN.R-project.org/package=EMAtools [Google Scholar]
  32. Kuznetsova A, Brockhoff PB, & Christensen RHB (2017). lmerTest Package: Tests in Linear Mixed Effects Models. Journal of Statistical Software, 82(13), 1–26. doi: 10.18637/jss.v082.i13 [DOI] [Google Scholar]
  33. Lee AC, Driban JB, Price LL, Harvey WF, Rodday AM, & Wang C (2017). Responsiveness and Minimally Important Differences for 4 Patient-Reported Outcomes Measurement Information System Short Forms: Physical Function, Pain Interference, Depression, and Anxiety in Knee Osteoarthritis. Journal of Pain, 18(9), 1096–1110. doi: 10.1016/j.jpain.2017.05.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Loudon K, Zwarenstein M, Sullivan F, Donnan P, & Treweek S (2013). Making clinical trials more relevant: improving and validating the PRECIS tool for matching trial design decisions to trial purpose. Trials, 14, 115. doi: 10.1186/1745-6215-14-115 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Martell CR, Dimidjian S, & Herman-Dunn R (2013). Behavioral activation for depression: A clinician’s guide.: Guilford Press. [Google Scholar]
  36. Meyer D, Dimitriadou E, Hornik K, Weingessel A, & Leisch F (2017). e1071: Misc Functions of the Department of Statistics, Probability Theory Group (Formerly: E1071), TU Wien; In. Retrieved from https://CRAN.R-project.org/package=e1071 [Google Scholar]
  37. Mortier P, Cuijpers P, Kiekens G, Auerbach RP, Demyttenaere K, Green JG, … Bruffaerts R (2018). The prevalence of suicidal thoughts and behaviors among college students: a meta-analysis. Psychological Medicine, 48(554–565). [DOI] [PubMed] [Google Scholar]
  38. Mowbray CT, Mandiberg JM, Stein CH, Kopels S, Curlin C, Megivern D, … Lett R (2006). Campus mental health services: Recommendations for change. American Journal of Orthopsychiatry, 76(2), 226–237. [DOI] [PubMed] [Google Scholar]
  39. National Institute of Mental Health (Producer). (2015). NIMH strategic plan for research (NIH Publication No. 15–6368). Retrieved from http://www.nimh.nih.gov/about/strategic-planning-reports/index.shtml [Google Scholar]
  40. National Science Foundation, N. C. f. S. a. E. S. (2017). Women, Minorities, and Persons with Disabilities in Science and Engineering: 2017 Special Report NSF 17–310. Retrieved from Arlington, VA.: www.nsf.gov/statistics/wmpd/ [Google Scholar]
  41. Pilkonis PA, Yu L, Dodds NE, Johnston KL, Maihoefer CC, & Lawrence SM (2014). Validation of the depression item bank from the Patient-Reported Outcomes Measurement Information System (PROMIS) in a three-month observational study. Journal of Psychiatric Research, 56, 112–119. doi: 10.1016/j.jpsychires.2014.05.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Preti A (2015). How to calculate the Number Needed to Treat (NNT) from Cohen’s d or Hedges’g. Retrieved from: https://rpubs.com/RatherBit/78905 [Google Scholar]
  43. R Core Team. (2013). R: A language and environment for statistical computing. In. Retrieved from http://www.R-project.org/ [Google Scholar]
  44. Regehr C, Glancy D, & Pitts A (2013). Interventions to reduce stress in university students: A review and meta-analysis. Journal of Affective Disorders, 148(1), 1–11. [DOI] [PubMed] [Google Scholar]
  45. Segal ZV, Williams JMG, & Teasdale JD (2013). Mindfulness-based cognitive therapy for depression. New York: Guilford Press. [Google Scholar]
  46. Smith BW, Dalen J, Wiggins K, Tooley E, Christopher P, & Bernard J (2008). The brief resilience scale: assessing the ability to bounce back. International Journal of Behavioral Medicine, 15(3), 194–200. [DOI] [PubMed] [Google Scholar]
  47. Steinhardt M, & Dolbier C (2008). Evaluation of a resilience intervention to enhance coping strategies and protective factors and decrease symptomatology. Journal of American College Health, 56(4), 445–453. [DOI] [PubMed] [Google Scholar]
  48. Vaez M, & Laflamme L (2008). Experienced stress, psychological symptoms, self-rated health and academic achievement: A longitudinal study of Swedish university students. Social Behavior and Personality: an international journal, 36(2), 183–196. [Google Scholar]
  49. Walach H, Buchheld N, Buttenmüller V, Kleinknecht N, & Schmidt S (2006). Measuring mindfulness—the Freiburg mindfulness inventory (FMI). Personality and Individual Differences, 40(8), 1543–1555. [Google Scholar]
  50. Wang B (2018). bda: Density Estimation for Grouped Data. In. Retrieved from https://CRAN.R-project.org/package=bda
  51. Windle G, Bennett KM, & Noyes J (2011). A methodological review of resilience measurement scales. Health and Quality of Life Outcomes, 9, 8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Xiao H, Carney DM, Youn SJ, Janis RA, Castonguay LG, Hayes JA, & Locke BD (2017). Are we in crisis? National mental health and treatment trends in college counseling centers. Psychological Services, 14(4), 407–415. doi: 10.1037/ser0000130 [DOI] [PubMed] [Google Scholar]
  53. Yeager DS, & Dweck CS (2012). Mindsets that promote resilience: When students believe that personal characteristics can be developed. Educational Psychologist, 47(4), 302–314. [Google Scholar]
  54. Yost KJ, Eton DT, Garcia SF, & Cella D (2011). Minimally important differences were estimated for six Patient-Reported Outcomes Measurement Information System-Cancer scales in advanced-stage cancer patients. Journal of Clinical Epidemiology, 64(5), 507–516. doi: 10.1016/j.jclinepi.2010.11.018 [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Zahniser E, & Conley CS (2018). Interactions of emotion regulation and perceived stress in predicting emerging adults’ subsequent internalizing symptoms. Motivation and Emotion, 1–11. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supp info
NIHMS1054964-supplement-Supp_info.docx (2.3MB, docx)

RESOURCES