Skip to main content
PMC home page
Internet Interventions logoLink to Internet Interventions
. 2018 Mar 22;16:43–51. doi: 10.1016/j.invent.2018.03.005

An Internet based intervention for improving resilience and coping strategies in university students: Study protocol for a randomized controlled trial

Rocio Herrero a,b, Adriana Mira b,c, Giulia Cormo b,d, Ernestina Etchemendy b,c,e, Rosa Baños b,d,e, Azucena García-Palacios a,b,e, David D Ebert f, Marvin Franke f, Thomas Berger g, Michael P Schaub h, Dennis Görlich i, Corinna Jacobi j, Cristina Botella a,b,e,⁎
PMCID: PMC6364440  PMID: 30775264

Abstract

Background

The literature shows a high prevalence of depression and anxiety in young people. The university represents a change in the lives of students, and is considered a stress factor. Therefore, it is particularly relevant to develop interventions specifically addressed to students and foster supportive environments and resilient communities. As students are “digital natives”, online interventions offer several potential advantages in doing this. This study aims to develop and evaluate the efficacy of an Internet-based intervention (CORE: Cultivating our Resilience), based on the Ryff model of well-being, to promote resilience and coping skills, decrease symptoms of depression and anxiety, and increase overall wellbeing in young people confronting a crucial life event (the university). This paper summarizes the study protocol.

Method

The design of the planned study is a randomized controlled trial. A minimum of 464 participants will be randomly assigned to two conditions: 1) an unguided Internet-based intervention to enhance resilience (N = 232); 2) a care-as-usual condition (CAU) (N = 232). The primary outcome will be the Connor-Davidson resilience scale. Secondary outcomes will – among others - include other relevant clinical measures, such as anxiety and depressive symptoms, quality of life, and social support. Outcomes will be assessed 4 and 8 weeks, and 6 and 12-months follow-ups. Intention-to-treat and per-protocol analyses will be performed.

Discussion

The results of this study will contribute to the growing research on Internet-delivered interventions. The expected results may have a major impact on the prevention of mental disorders and possible negative consequences in at-risk populations, such as college students.

Keywords: Resilience, Internet intervention, University students, Well-being

Highlights

  • •

    University students are a risk population for mental disorders.

  • •

    Resilience is a key concept in the prevention field.

  • •

    Online intervention seems a suitable tool for a preventive intervention.

  • •

    CORE is an online intervention for enhancing resilience and coping skills.

1. Introduction

Mental health disorders diminish the quality of life of people who suffer from them, and they also increase vulnerability for the development of severe disabling diseases (Mihalopoulos et al., 2011). Early adulthood represent crucial periods in the development of mental disorders. The literature shows an alarming prevalence of depression and anxiety in young people as university students (Auerbach et al., 2016; Merikangas, et al., 2009). However, care of common mental health disorders (CMHD), such as depression, anxiety, adjustment disorders, or substance use, has primarily included treatment of rather than prevention in most European countries (WHO, 2013), and a shift to more prevention-oriented care has been urged by the European Commission (EC) health policy (WHO, 2013).

Resilience refers to an individual's ability to properly adapt to stress and adversity, to overcome the negative effects of risk exposure, or coping successfully with traumatic experiences. It is also important for maintaining quality of life, emotional well-being, and functional independence. Three resilience models have been identified to explain how promotive factors operate to change the effects of risk factors - compensatory, challenge and protective (Fergus and Zimmerman, 2005). Compensatory model refers to those models where the promotive factor counteracts a risk factor. The challenge model suggest that the risk exposure must be challenging enough to generate a coping response, a exposure to low and high levels of a risk factor are associated to negative effects, but moderate levels are associated to less negative effects. The protective model suggests that a assets or resources reduce the effects of a risk factor.

The transition from adolescence to adulthood represents a high-risk period during which young people are exposed to developing psychological problems (Giedd et al., 2008). The university represents a change in the lives of students, and it is considered a stress factor and a heightened risk for psychopathology (Cook, 2007) (e.g., new social relationships and contexts without the support of parents or long-time friends, academic pressure, stress during exams, social disconnection). Moreover, this stressful situation not only substantially affects students' normal day-to-day activities, but it also has negative consequences such as lower academic achievements, college dropout (Arria et al., 2009; Cook, 2007; Institute for Employment Studies, 2015). Therefore, a preventive intervention offered during this time could reduce the risk of developing a mental disorder and experiencing other negative consequences by enhancing protective factors. Although existing programs designed to promote resilience in young people have demonstrated a decrease in depressive and anxious symptomatology (e.g. Gillham et al., 2007; Stallard and Buck, 2013), and evidence for these preventive interventions is steadily growing (Das et al., 2016), while literature on this topic is still scarce (Huebner et al., 2009).

A key objective in the promotion of mental health is to propose and deliver interventions that are affordable and available to everyone, the Internet can be an effective medium to deliver mental health interventions (Kazdin, 2015). People can access the web contents of an intervention with little or no cost to the client, with or without minimal assistance from a therapist or other mental health professional (Ashford et al., 2016; Griffiths and Christensen, 2007; Hayward et al., 2007; Kaltenthaler et al., 2006; Mitchell et al., 2010; Muñoz, 2010). In the past decade, web-based interventions have been used to enhance the accessibility and effectiveness of traditional treatments. Global Internet penetration is constantly increasing. The statistic shows that in Europe, 79% of individuals aged 16 to 74 years use the Internet (Eurostat, 2016). Regarding its efficacy, several meta-analyses suggested that web-based programs can be as effective as traditional face-to-face programs, and significantly more effective than control conditions, for a variety of mental health disorders, as depression, anxiety, and adjustment disorders (e.g., Andrews et al., 2010; Botella et al., 2015; Montero-Marín et al., 2016), health behaviors (e.g., Baños et al., 2015), and the prevention of mental health disorders (Heber et al., 2017; Sander et al., 2016). Moreover, these interventions are acceptable and effective across different populations, as child, students and older adults (e.g. Calear and Christensen, 2010; Ebert et al., 2015; Farrer et al., 2013; Preschl et al., 2012).

Online interventions seem to offer several advantages for the promotion of mental health (Kazdin and Blase, 2011; Kazdin and Rabbitt, 2013; Christensen and Hickie, 2010). Through its use, existing barriers can be overcome, such as cost, service availability, wait time, transportation, and stigma, thus reducing health care-related disparities. Besides, as young people are immersed in the digital world, internet can be effective and efficient tools to implement preventive mental health interventions (Baños et al., 2017). Nevertheless, there are few studies that assessed the effectiveness of web-based preventive intervention to promote psychological well-being and resilience in young people (Baños et al., 2017; Lintvedt et al., 2013; Rose et al., 2013; Bolier et al., 2013; Abbott et al., 2008). Although the first results have shown promise, there is a need for more controlled studies based on well-grounded models with long-term follow-ups (Baños et al., 2017; Leppin et al., 2014). In this sense, the aim of the present paper is to describe the protocol for a randomized controlled trial that test the efficacy and effectiveness of an online intervention to promote resilience and care as usual in University students.

2. Materials and methods

2.1. Aim

The aim of the study is to explore the efficacy of a program developed for promoting resilience and coping skills in young people confronting a crucial life event such as the university. The present study is part of the European research project: Integrating Technology into Mental Health Care Delivery in Europe (ICare, No 634757; www.icare-online.eu). The main objective of the ICare project is to develop an integrated model of health promotion and risk prevention as well as detection of the most prevalent disorders in Europe. The present study is one of 7 clinical trials embedded in this European project. Specific hypothesis are that the program CORE will enhance resilience and coping skills, and increase wellbeing.

2.2. Study design

A multi-country (Spain, Germany, and Switzerland) two-armed single-blind randomized controlled trial (RCT) will be conducted with repeated measures (baseline, 8 weeks, 6 months, and 12 months). Participants that fulfill inclusion and exclusion criteria will be randomized to one of the two study arms in a 1:1 ratio. The randomization will be stratified by platforms (Spanish platform, Germany-Swiss-Austrian platform) and generated according to a permuted block randomization with fixed and concealed block length. Randomization lists will be prepared by ICare Partner WWU (Westfälische Wilhelms-Universität Münster) independently from the study personnel. The two study conditions are: 1) unguided Internet-based intervention for enhancing resilience; 2) care-as-usual condition (CAU). Participants in the intervention condition will receive a 8-week unguided Internet-based prevention program, CORE. Participants in the CAU condition will receive the usual attention at the university, and upon completion of the intervention period, they will receive access to the CORE intervention program.

The study was registered at the ISRCTN registry (ISRCTN13856522), and will be conducted in accordance with the CONSORT 2010 Statement (Moher, Schulz, Altman, and Group, 2001; Moher et al., 2010; Eysenbach and CONSORT-EHEALTH Group, 2011), the CONSORT-EHEALTH guidelines (Eysenbach and Consort E-Health Group, 2011) and the SPIRIT guidelines (Standard Protocol Items: Recommendations for Interventional Trials) (Chan et al., 2013). Fig. 1 shows the study design.

Fig. 1.

Fig. 1

Open in a new tab

Study design.

2.3. Sample size calculation

Power calculations were taken into consideration to estimate the necessary sample size. Because no information is available about the normality of the Connor-Davidson Resilience Scale (CD-RISC) scores within the targeted population, the CD-RISC scores at post-intervention between CORE and CAU will be compared using a two-sided Mann-Whitney U Test. So far, there are no data about the effect sizes that can be obtained in Internet-based interventions for the prevention of depression or other common mental health disorders mainly focusing on promoting resilience. However, previous studies on online interventions have shown effect sizes of between 0.36 and 0.50 for the reduction of symptoms of depression and between 0.32 and 0.42 for the reduction of symptoms of anxiety (e.g., van Straten et al., 2008; Lintvedt et al., 2013; Bolier et al., 2013). At least similar effects on improving resilience is expected. Adopting a conservative approach in relation to these findings, the sample size is based on the smallest effect size detected in previous studies (Cohen's d = 0.32), a significance level of 0.05, a power of 0.80, and a dropout rate of 30%, on the primary outcome measure (CD-RISC). The planned sample size of 464 participants is sufficient to detect a small effect size (Cohen's d = 0.32). Covariate adjustment for relevant baseline prognostic factors will further increase the power. To ensure recruitment feasibility, five universities (with an estimated number of 170,000 students) will take part in the trial.

2.4. Participants and recruitment

University students with low levels of resilience will be invited to participate.

2.4.1. Inclusion criteria

  • -

    University students with an score one standard deviation below the mean of the sample on the Connor-Davidson Resilience Scale (CD-RISC-25).

  • -

    Adequate knowledge to understand and read Spanish or German.

  • -

    Access to the Internet, and the ability to use a computer.

2.4.2. Exclusion criteria

  • -

    University students with a history of a CMHD in the past 12 months.

  • -

    University students who are on a waiting list for psychotherapy or are currently or have been in psychotherapeutic treatment within the past 12 months.

  • -

    Individuals with a current psychotic or bipolar disorder or a history of one.

  • -

    Individuals at risk for suicide.

Participants who do not meet the inclusion criteria will be encouraged to seek interventions alternatives better suited to their specific needs.

2.4.3. Recruitment

The study will be advertised online via universities websites: Universitat Jaume I, Castellón, Spain, https://www.uji.es/; Universitat of Valencia, Spain, http://www.uv.es/; Universität Bern, Switzerland, http://www.unibe.ch/; Universität Zürich, Switzerland, http://www.uzh.ch; Technische Universität Dresden, Germany, https://tu-dresden.de/. Furthermore, non-professional social-networks (i.e., Facebook and twitter) and advertisements in newspapers will be used. Moreover, posters will be placed in local universities, and an invitation email will be sent to all university students. Individuals interested in participating in the study will be directed to the research website, where they will find further information about the study and what participation entails, as well as an informed consent form. Individuals can request participation through the website and by signing the informed consent form.

2.5. Intervention

2.5.1. CORE condition

CORE is a 6-week Internet-based prevention program whose main objective is to teach skills and strategies to cope with daily life stressors in order to enhance resilience and coping skills, promote self-empowerment, and increase wellbeing.

The intervention consists of 6 interactive modules designed for weekly sessions (see Table 1). The therapeutic components of the program are evidence-based techniques selected following the Riff model of well-being (Ryff, 1989; Ryff, 2014), and organized in 6 dimensions: Autonomy, Self-Acceptance, Environmental Mastery, Purpose in Life, Positive Relations, and Personal Growth. In addition, each module includes exercises to practice the proposed skills. The program includes multimedia elements: videos, audios, vignettes, images (see Fig. 2). Furthermore, it allows the user to keep different records through the PC and Tablet.

Table 1.

CORE modules and their objectives.

Module Objective
(0) Welcome Introduction module to the program, with an explanation about the tools and how to use CORE.
(1) Psychoeducation Explanation of psychological wellbeing and the concept of resilience:
  • -

    Understand the concept of psychological well-being, its most important aspects, and their relevance in life.

  • -

    Understand the concept of resilience and the importance of training and cultivating it.
(2) Autonomy: building my way Enhancement of autonomy:
  • -

    Develop a healthy lifestyle (by pursuing balance in several areas: activity, food, sleep). This lifestyle will allow the person to focus on his/her goals in life.

  • -

    Increase psychological well-being by working on the abilities and potentially related to values and goals in life.
(3) Mindfulness and self-compassion Training in mindfulness, savoring, and an attitude of self-compassion:
  • -

    Learn the meaning of “mindfulness”, how to develop this ability, and the benefits that its practice can bring.

  • -

    Learn to distance ourselves from our thoughts and how to handle them.

  • -

    Understand the importance of, recognize, capture, and enjoy the good moments.

  • -

    Develop the skill of kindness and self-care, i.e., the capacity for self-compassion.
(4) Overcoming obstacles Development of coping strategies to deal with daily difficulties in life:
  • -

    Be aware of the importance of facing problems properly.

  • -

    Learn the Problem Solving Technique and how to apply it.

  • -

    Learn the role of our thoughts in the way we feel and how to be flexible in our way of interpreting situations.
(5) Connecting to others Acknowledge the relevance of relationships and how they can be helpful in the construction of well-being:
  • -

    Recognize the importance of our social relations.

  • -

    Learn to care for and improve our social relations.

  • -

    Learn to promote quality relationships, which can contribute to maintaining and strengthening resilience.
(6) Purpose in life and personal growth Encourage students to deal with the future with a positive attitude, taking into account what is important for each person and planning the future according to these objectives.
Open in a new tab
Fig. 2.

Fig. 2

Open in a new tab

CORE: example of design of a session module on Minddistrict platform.

CORE will be running on the Minddistrict platform (Minddistrict B.V., Amsterdam, The Netherlands), a web-based eHealth platform. Minddistrict is the technology partner within the ICare project. The Minddistrict platform allows researchers and therapists to produce intervention content and deliver this content to clients. Among the tools that the platform provides, this trial includes the possibility to record daily data in a mobile application, the use of a logical sequence that allows the therapist to deliver specific content related to the answer given by the client (conditional content). Moreover, it ensures secure and encrypted communication between clients and therapist.

2.5.2. Care as usual condition

Participants allocated to the control condition will be assessed Participants will be assessed at baseline, 4 weeks, 8 weeks, and at 6- and 12-month follow-ups. Furthermore, they will receive access to the prevention program by the end of the last follow-up assessment.

2.6. Data management

Data management and monitoring will be provided by the Institute of Biostatistics and Clinical Research (Münster, Germany) for the whole consortium in order to maintain comparable high quality in the conduct of the ICare research projects in trial planning, data management, online monitoring, and analysis.

2.7. Outcomes

Assessments will be conducted online using the Minddistrict platform. The measures used and assessment times in the study are described in Table 2.

Table 2.

Study measures, time of assessment, and source of measurement.

Construct Questionnaire S T0 T1 T2 T3 T4
Demographics Socio-demographic data x – – – – –
Resilience (primary outcome) CD-RISC x x – X x x
Depression severity PHQ-9 – x x X x x
Personality inventory BFI-10 – x – – – –
Anxiety symptoms GAD-7 – x x X x x
Global self-esteem RSE – x x X – –
Alcohol screening test AUDIT-C – x x X – –
Positive and negative affect PANAS – x x X x x
Psychological well-being PWBS-29 – x – X x x
Self-compassion SCS-SF – x – X x x
Client service receipt inventory CSRI – x – X x x
Perceived stress PSS-4 – x x X x x
Enjoyment orientation EOS – x – X x x
Credibility and expectancy of treatment CEQ – x – – – –
Therapeutic alliance WAI-TECH – – x X – –
Treatment satisfaction CSQ – – – X – –
Open in a new tab

*Assessment times: S – screening, T0 – baseline, T1–4 weeks after the allocation, T2–8 weeks after the allocation, T3–6 months follow-up, T4–12 month follow-up.

2.7.1. Primary outcome

The primary outcome is the Connor-Davidson Resilience Scale (CD-RISC; Connor and Davidson, 2003) is a 25-item self-report questionnaire that assesses the ability to cope with stress. Respondents are asked to indicate their level of agreement using a 5-point Likert scale from 0 to 4 (0 = strongly disagree, 4 = strongly agree). Scores range from 0 to 100, with higher scores reflecting greater resilience. Previous studies have shown that the CD-RISC has good internal consistency (Cronbach alpha above 0.70) (Yu and Zhang, 2007; Singh and Yu, 2010).

2.7.2. Secondary outcomes

The secondary outcomes are:

Well-being: The Ryff Scales of Psychological Well-Being - 29 items (PWBS-29; Ryff, 1989) is a theoretically based instrument that specifically focuses on measuring multiple facets of psychological well-being. These facets include six dimensions (i.e., autonomy, environmental mastery, personal growth, positive relations with others, purpose in life, and self-acceptance). The response scale ranges from 1 to 6 (1 = completely disagree, 6 = completely agree). This scale has been shown to have good psychometric properties (e.g., Van Dierendonck, 2004; Díaz et al., 2006).

Depression: The Patient Health Questionnaire (PHQ-9; Kroenke et al., 2001) is a 9-item instrument that can be used to screen and diagnose patients with depressive disorders. The 9 items are scored on a scale from 0 to 3 (0 = not at all, 3 = nearly every day). Total scores range from 0 to 27. Depression severity cut-off points of 5, 10, 15, and 20 represent the thresholds for mild, moderate, moderately severe, and severe depression, respectively. The PHQ-9 has been shown to have good psychometric properties (Wittkampf et al., 2007).

Positive and negative emotionality: Positive and Negative Affect Schedule (PANAS) (Watson et al., 1988). The PANAS consists of 20 items that evaluate two independent dimensions: positive affect (PANAS+) and negative affect (PANAS-). The range for each scale (10 items on each) is from 10 to 50 (Watson et al., 1988). It is a brief, reliable, and valid self-report measure. It has shown excellent convergent and divergent validity (Sandin et al., 1999; Watson et al., 1988).

Anxiety and stress: The Generalized Anxiety Disorder Questionnaire (GAD-7; Spitzer et al., 2006) is a one-dimensional self-administered scale used to detect the presence of the symptoms of Generalized Anxiety Disorder (GAD), as listed in the DSM-IV. The scale does not provide a definitive diagnosis of GAD, but rather it is a rapid, efficient, reliable, and valid instrument for detecting the presence of a common anxiety disorder. Scores range from 0 to 3 (0 = not at all, 3 = nearly every day), and the total score ranges from 0 to 21. Four severity cut-off points (minimal = 0 to 4, mild = 5 to 9, moderate = 10 to 14, serious = 14 to 20) represent the thresholds from minimal to serious general anxiety. The GAD-7 has shown good internal consistency and test–retest reliability, as well as convergent, construct, criterion, procedural, and factorial validity, for the diagnosis of GAD (Spitzer et al., 2006; Löwe et al., 2008).

The Perceived Stress Scale - 4 items (PSS-4; Cohen et al., 1983) is a 4-item self-report questionnaire that assesses the degree to which recent life situations are appraised as stressful (Cohen et al., 1983). The PSS-4 was introduced as a brief version for situations requiring a very short scale or telephone interviews (Cohen and Williamson, 1988). It is rated on a 5-step scale from 1 to 5 (1 = never, 5 = very often). It has been validated in different studies and shown good internal consistency reliability (e.g., Cohen and Williamson, 1988; Mitchell et al., 2008).

Self-esteem: Rosenberg's Self-Esteem Scale (RSE; Rosenberg, 1965) is a 10-item self-report questionnaire that measures global self-esteem by rating both positive and negative feelings about the self (Robins et al., 2001; Rosenberg, 1965). All the items are answered on a 4-point Likert scale format ranging from 1 to 4 (1 = totally disagree, 4 = totally agree). Previous studies have found good internal consistency and test–retest reliability (Gray-Little et al., 1997; Robins et al., 2001).

Self-compassion: The Self-Compassion Scale - Short Form (SCS-SF; Raes et al., 2011) is a shorter version of the original 26-item SCS (Neff, 2003). The original scale was designed to assess overall self-compassion (total score) and 3 facets of this construct: common humanity (SCSCH), mindfulness (SCS-M), and self-kindness (SCS-SK) (Raes et al., 2011). The factor analysis suggested 6 subscales representing positive and negative aspects of each facet, and this was also true for the SCS–SF (Raes et al., 2011). Respondents are asked to indicate how they perceive their actions toward themselves in difficult times using a 5-point Likert-type scale from 1 to 5 (1 = almost never, 5 = almost always). The SCS-SF has adequate reliability and validity (Raes et al., 2011).

Enjoyment: The Enjoyment Orientation Scale (EOS; Hervás and Vázquez, 2006) assesses the extent to which participants try to be receptive and make an effort to do pleasant things (anticipatory pleasure). It contains 6 items that represent different expressions of this construct (e.g. “I almost always try to enjoy new things every day, no matter how small”), rated on a Likert scale from 1 to 7 (1 = strongly disagree, 7 = strongly agree) (Hervás et al., 2009).

Substance abuse: The Alcohol Use Disorders Identification Test (AUDIT-C; Bush et al., 1998) is an alcohol screening test that can help to identify patients who are hazardous drinkers or have active alcohol use disorders (including alcohol abuse or dependence). The AUDIT-C is composed of 3 items scored from 0 to 4. Total score ranges from 0 to 12. In men, a score of 4 or more is considered positive alcohol misuse, whereas in women, a score of 3 or more is considered positive. The AUDIT-C has been shown to have good psychometric properties (e.g., Bush et al., 1998; Frank et al., 2008).

Personality: The 10-Item Big Five Inventory (BFI-10; Rammstedt and John, 2007) is a shorter version of the 44-item BFI (John and Srivastava, 1999; John et al., 1991). It is rated on a 5-step scale from 1 to 5 (1 = disagree strongly, 5 = agree strongly). It was developed to provide a personality inventory for research settings with extreme time constraints. Previous research has shown that the BFI-10 possesses psychometric properties that are comparable in size and structure to those of the full-scale BFI (Rammstedt and John, 2007).

Economic evaluation: The Client Service Receipt Inventory (CSRI) was developed to provide information about service utilization and ultimately inform service delivery by collecting retrospective information on service related issues (Chisholm et al., 2000). It is an established tool that has been used in a range of research studies, including in mental health outreach services, community nursing services, and community care of older people and people with challenging behavior. The questionnaire schedule is designed for interviewer/researcher administration with the person receiving the services, assisted by their main caregiver when necessary. The retrospective period is fixed according to the population being investigated and the specific aims of the research. The service receipt section is the core of the CSRI; for each service type, the number and average duration of contacts are recorded and used to summarize particular care packages, illustrate the variety of services used, and determine how resources are allocated.

Program evaluation: The Credibility and Expectancy Questionnaire (CEQ; Devilly and Borkovec, 2000) is a 6-item self-report instrument that assesses patients' expectations about treatment. Both factors (credibility and expectancy) have been shown to be stable across different populations, with high internal consistency within each factor (e.g., Devilly and Borkovec, 2000). The scale consists of 6 questions, with response options rated on a 10-point scale and on a 1–100% scale.

The Client Satisfaction Questionnaire (CSQ; Attkisson and Greenfield, 1996; Larsen et al., 1979) measures global patient satisfaction with the treatment. It consists of 8 items measured on a 4-point scale, with total scores ranging from 8 to 32. It has very good internal consistency reliability (e.g., Attkisson and Greenfield, 1996; Larsen et al., 1979; Roberts and Attkisson, 1983).

Working alliance: The Working Alliance Inventory for Technology-Based Interventions (WAI-TECH; Kiluk et al., 2014) is an 8-item questionnaire that assesses the therapeutic alliance between the technological tool and the patient. The questionnaire covers two dimensions of the working alliance: (1) therapeutic goals and (2) tasks. Respondents are asked to rate their answers on a 5-point Likert scale ranging from 1 to 5 (1 = never, 5 = always). The WAI-TECH has adequate reliability and validity (Kiluk et al., 2014).

Other measures: Other measurements include socio-demographic variables. The following sociodemographic variables will be collected: gender, household size and income, year of birth, marital status, employment status, total population of place of residence, nationality, level of education, and living situation. Other relevant health related variables measures will be assessed, specifically: presence of previous psychological disorders and if a treatment is currently undertaken.

2.8. Statistical methods

For the analysis of ICare study data we follow adopted guidelines, e.g. ICH E9 (http://www.ema.europa.eu/). The statistical analysis of the primary and secondary outcomes will be described in a statistical analysis plan that will be signed by the study committee and the responsible statistician.

For the primary analysis of the data, including the efficacy analysis, a statistical analysis plan (SAP) has been written. A blinded data review will be performed. Within the blinded data review decision on the multiple imputation strategy and the selection on multivariable models will be taken.

The overall analysis strategy consists of the following steps: (i) Data description, (ii) Analyses of the primary hypothesis (iii) secondary analyses (iv) further exploratory analyses.

Balancing of the randomization will be controlled by appropriate statistical tests of the baseline variables. The study collective will be characterized by descriptive statistical methods such as relative and absolute frequencies, mean, median, standard deviation, and inter-quartile-range (IQR), and appropriate graphics such as histograms, boxplots, and bar charts. Mean and median will be accompanied by 95%-confidence intervals. All measurement time points (S, T0, T1, T2, T3, T4) will be described separately. All descriptive statistics will be provided for both study arms. Assumptions for the appropriate statistical tests will be checked for normality by histograms, kurtosis, and the Kolmogorov-Smirnov test, sphericity will be checked using the Mauchly Test, (or any of the epsilon corrections in case it cannot be assumed), and the assumption of equality of variance-covariance matrices will be examined with the Box Test and the Levene Test.

2.8.1. Primary confirmatory analysis

The primary null-hypothesis tests whether CORE and CAU do not differ in their distribution of CD-RISC scores, indicated by shifted median CD-RISC scores. We will test this null-hypothesis non-parametrically using a two-sided Mann-Whitney U Test to answer the confirmatory question for the trial in the intention-to-treat (ITT) sample. The ITT sample comprises all randomized participants who provided the primary outcome measure within the initially assigned study arm. The primary hypothesis will be tested with a preset significance level of 5%. The primary confirmatory analysis will be performed in the SAS software (SAS Inc., Cary, NY, USA). All programming scripts will be validated by a second statistician at the University of Münster.

2.8.2. Sensitivity analyses of the primary analysis

A number of preplanned sensitivity analyses of the primary analysis will be performed. We will analyze the pre-post differences for the primary and all secondary outcomes using two-sided Mann-Whitney-U tests in case of non-normal differences or two-sided paired t-tests in case of normally distributed differences of the pre-post scores. Also, the primary analysis will be repeated using the per-protocol (PP) sample, i.e. participants without major protocol violations. The primary outcome will also be analyzed within a generalized linear mixed model including covariates as defined within the blinded data review and the study platform (i.e. center). Furthermore, the primary analysis will be repeated as stratified analysis using van-Elteren tests (stratified Mann-Whitney-U tests) and subgroup analyses. To assess the effect of missing data on the primary analysis the primary outcome will be reanalyzed after a multiple imputation strategy (developed within blinded data review) was applied to the data.

2.8.3. Analysis of secondary outcomes

Secondary outcomes will be compared between groups at individual time points using Student's-Test for unpaired data or the Mann-Whitney U Test, depending on the normal distribution of scores. Categorical variables will be tested using Fisher's Exact Test or Chi-Squared Tests. The collected longitudinal data will be analyzed using repeated-measures ANOVA or (generalized) linear mixed models (GLMM) with the appropriate link function.

The results of the primary and secondary analyses will be represented by appropriate effect estimates and 95% confidence intervals.

A Multilevel SEM analysis (MSEM) will be carried out, specifically to control for nested data and include potential confounders in the analysis. Time to onset of CMHD will be assessed by Log-rank tests and Cox proportional hazard models. The latter makes it possible to model additional covariates to control for confounding and obtain risk estimates (hazard ratios) for the onset of CMHD. For all multivariate analyses, covariates will be entered as necessary, e.g., baseline characteristics of participants or variables associated with dropout.

All secondary analyses have to be considered exploratory and hypothesis-generating. We will consistently use the nominal significance level of 0.05 (two-sided) also for exploratory analyses.

Furthermore, a Consensual Qualitative Research (CQR) approach, based on Grounded Theory but specifically designed for clinical content (Hill et al., 2005) will be applied to study the participants' opinions and attitudes about CORE (the online program).

CQR involves asking the participants specific open-ended questions about a topic following the principles specified in the CQR guidelines. Their responses are then coded into themes in a consensual manner by a group of researchers. In these methodology domains, categories and core ideas are derived from the interactive exchange of a group of coders. In the first place, domains are created by a thorough analysis of the whole content. Next, core ideas incorporate the explicit answers within the domains. Finally, the categories are built. For each instance, discrepancies are resolved within the consensual procedure that this approach entails. The categories are labeled in general if they apply to all cases, typical if they apply to at least half but not all of the participants, and variant if they apply to less than half. This CQR-based qualitative analysis will provide rich information about the participants' opinions and attitudes about the online program, in order to detect possible barriers and analyze possible strategies to overcome them.

3. Discussion

This paper describes CORE, a new Internet-based intervention based on the Ryff model of well-being designed for promoting resilience and coping skills in university students (Ryff, 1989; Ryff, 2014; Ryff and Singer, 1996; Ryff et al., 1998), and presents the study protocol to test its efficacy and effectiveness, compared to a CAU condition, using a two-armed RCT. Participants' acceptance of this program will also be studied in order to identify possible implementation barriers for the users.

Considering the high prevalence of emotional problems in young people worldwide (Copeland et al., 2011) prevention of psychological problems, promotion of resilience and well-being in this age is crucial (Stengård and Appelqvist-Schmidlechner, 2010). The literature shows that mental illness can be prevented in adult and youth populations (Calear and Christensen, 2010; Cuijpers et al., 2012; Muñoz, 2010; van Zoonen et al., 2014), and that intervening in young people maximizes the benefits of prevention labors (Gladstone et al., 2011; O'Connell et al., 2009).

A key objective of prevention-oriented health care is to propose and deliver interventions that are affordable and available to everyone, the Internet can be an effective medium to deliver mental health interventions. University students, are fully immersed in digital worlds, and their relationships, activities, tasks and concerns are being defined by ICTs. Therefore, implementing Internet-based interventions can potentially be advantageous, compared to face-to-face interventions (Christensen and Hickie, 2010), among this population.

The strengths of this study are: First, to our knowledge, this would be the first RCT to test an online intervention for enhancing resilience in university students. Second, CORE is a multi-country two-armed simple-blind RCT that will be conducted in three countries (Spain, Germany, and Switzerland) by four research institutions, which will allow good generalization of the results in Europe. Finally, some limitations should be mentioned. First, dropout rates are expected to be high, as reported by previous research (Spek et al., 2007; Van Ballegooijen et al., 2014). Indeed, dropout rates have been taken into account in the sample size calculation. Second, online mental health interventions also raise some particular ethical issues, such as confidentiality and anonymity and duty of care. For this reason, international guidelines will be followed to address these concerns (Barak et al., 2009; Ritterband et al., 2006).

4. Conclusion

We present the design of our study aimed to promote resilience among university students. The results of this study will contribute to the growing research on Internet-delivered treatments. If the expected results are obtained, this could be a first step in the implementation of these programs to enhance the resilience and coping skills in a risk populations, as university students, to prevent common mental disorders. Furthermore, we will analyze feasible strategies to implement the program and identify possible implementation barriers for final users, professionals of University counseling services, and University authorities.

Trial status

Recruitment started in September 2017 in Spain and will continue approximately through January 2018. Follow-up assessments for the remaining patients are expected to be completed by January 2019.

Competing interests

The authors declare that they have no competing interests.

Funding

This project has received funding from the European Union's Horizon 2020 Research and Innovation Programme under grant agreement No 634757.

Acknowledgments

Acknowledgment

This work was supported by Spanish Ministerio de Economía y Competitividad, PSI2014-54172-R. Conselleria de Educación, Cultura y Deporte Generalitat Valenciana, Programa Prometeo PROMETEOII/2013/003, and “Red PROMOSAM” (PSI2014-56303-REDT).

Author contributions

RH, AM, GC, EE, RB, AGP, and CB designed the study. DDE, MF, TB MPS, DG and CJ contributed significantly to the study design. AM, GC, RH, EE, RB, AGP, and CB wrote the first draft of the manuscript. DDE, MF, TB MPS, DG and CJ revised the manuscript and completed the final draft. All authors contributed feedback, read, and approved the final manuscript.

Ethical statement

This trial is conducted in compliance with the protocol, the Declaration of Helsinki and good clinical practice. The trial was registered at ISRCTN (ISRCTN13856522). The local ethical commissions in each country provided approval. Data security/confidentiality is guaranteed; all relevant EU legislation and international texts on privacy are observed and respected. Regarding regulation at international level, starting from the OECD guidelines including the “Guidelines on the protection of privacy and transborder flow of personal data” (1981) and “Guidelines for the security of information systems” (1991/92), the ICare consortium in particular acknowledges heterogeneity in international data protection jurisdiction.

Contributor Information

Adriana Mira, Email: miraa@uji.es.

Ernestina Etchemendy, Email: ernestina@labpsitec.es.

Rosa Baños, Email: banos@uv.es.

Azucena García-Palacios, Email: azucena@uji.es.

David D. Ebert, Email: David.Ebert@fau.de.

Marvin Franke, Email: marvin.franke@fau.de.

Thomas Berger, Email: thomas.berger@ptp.unibe.ch.

Michael P. Schaub, Email: michael.schaub@isgf.uzh.ch.

Dennis Görlich, Email: dennis.goerlich@ukmuenster.de.

Corinna Jacobi, Email: corinna.jacobi@tu-dresden.de.

Cristina Botella, Email: botella@uji.es.

References

  1. Abbott J.A.M., Klein B., Ciechomski L. Best practices in online therapy. J. Technol. Hum. Serv. 2008;26(2–4):360–375. [Google Scholar]
  2. Andrews G., Cuijpers P., Craske M.G., McEvoy P., Titov N. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS One. 2010;5(10) doi: 10.1371/journal.pone.0013196. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Arria A.M., O'Grady K.E., Caldeira K.M., Vincent K.B., Wilcox H.C., Wish E.D. Suicide ideation among college students: a multivariate analysis. Arch. Suicide Res. 2009;13:230–246. doi: 10.1080/13811110903044351. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Ashford M.T., Olander E.K., Ayers S. Computer-or web-based interventions for perinatal mental health: a systematic review. J. Affect. Disord. 2016;197:134–146. doi: 10.1016/j.jad.2016.02.057. [DOI] [PubMed] [Google Scholar]
  5. Attkisson C.C., Greenfield T.K. The client satisfaction questionnaire (CSQ) scales and the service satisfaction scale-30 (SSS-30) Outcomes Assess. Clin. Pract. 1996:120–127. [Google Scholar]
  6. Auerbach R.P., Alonso J., Axinn W.G., Cuijpers P., Ebert D.D., Green J.G.…Nock M.K. Mental disorders among college students in the World Health Organization world mental health surveys. Psychol. Med. 2016:1–16. doi: 10.1017/S0033291716001665. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. van Ballegooijen W., Cuijpers P., van Straten A., Karyotaki E., Andersson G., Smit J.H., Riper H. Adherence to internet-based and face-to-face cognitive behavioural therapy for depression: a meta-analysis. PLoS One. 2014;9(7):100674. doi: 10.1371/journal.pone.0100674. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Baños R.M., Mensorio M.S., Cebolla A., Rodilla E., Palomar G., Lisón J., Botella C. An internet-based self-administered intervention for promoting healthy habits and weight loss in hypertensive people who are overweight or obese: a randomized controlled trial. BMC Cardiovasc. Disord. 2015;15:83–92. doi: 10.1186/s12872-015-0078-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Baños R.M., Etchemendy E., Mira A., Riva G., Gaggioli A., Botella C. Online positive interventions to promote well-being and resilience in the adolescent population: a narrative review. Front. Psychol. 2017;8 doi: 10.3389/fpsyt.2017.00010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Barak A., Klein B., Proudfoot J.G. Defining internet-supported therapeutic interventions. Ann. Behav. Med. 2009;38(1):4–17. doi: 10.1007/s12160-009-9130-7. [DOI] [PubMed] [Google Scholar]
  11. Bolier L., Haverman M., Westerhof G.J., Riper H., Smit F., Bohlmeijer E. Positive psychology interventions: a meta-analysis of randomized controlled studies. BMC Public Health. 2013;13:119. doi: 10.1186/1471-2458-13-119. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Botella C., Mira A., Herrero R., García Palacios A., Baños R.M. Un programa de intervención auto-aplicado a través de Internet para el tratamiento de la depresión: “Sonreír es divertido.”. Aloma Rev. Psicol. 2015;33:39–48. http://dialnet.unirioja.es/servlet/articulo?codigo=5267058&info=resumen&idioma=ENG [Google Scholar]
  13. Bush K., Kivlahan D.R., McDonell M.B., Fihn S.D., Bradley K.A. The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Arch. Intern. Med. 1998;158(16):1789–1795. doi: 10.1001/archinte.158.16.1789. [DOI] [PubMed] [Google Scholar]
  14. Calear A.L., Christensen H. Review of internet-based prevention and treatment programs for anxiety and depression in children and adolescents. Med. J. Aust. 2010;192(11):12–14. doi: 10.5694/j.1326-5377.2010.tb03686.x. http://www.ncbi.nlm.nih.gov/pubmed/20528700 (Retrieved from) [DOI] [PubMed] [Google Scholar]
  15. Chan A.W., Tetzlaff J.M., Altman D.G., Laupacis A., Gøtzsche P.C., Krleža-Jerić K.…Moher D. Statement: defining standard protocol items for clinical trials. Ann. Intern. Med. 2013;158:200–207. doi: 10.7326/0003-4819-158-3-201302050-00583. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Chisholm D., Knapp M.R.J., Knudsen H.C., Amaddeo F., Gaite L., Van Wijngaarden B. Client socio-demographic and service receipt inventory – European version: development of an instrument for international research. Br. J. Psychiatry. 2000;177:28–33. doi: 10.1192/bjp.177.39.s28. [DOI] [PubMed] [Google Scholar]
  17. Christensen H., Hickie I.B. Using e-health applications to deliver new mental health services. Med. J. Aust. 2010;192(11):53–56. doi: 10.5694/j.1326-5377.2010.tb03695.x. http://www.ncbi.nlm.nih.gov/pubmed/20528711 (Retrieved from) [DOI] [PubMed] [Google Scholar]
  18. Cohen S., Williamson G. Perceived stress in a probability sample of the United States. In: Spacapan S., Oskamp S., editors. The Social Psychology of Health: Claremont Symposium on Applied Social Psychology. Sage; Newbury Park, CA: 1988. [Google Scholar]
  19. Cohen S., Kamarck T., Mermelstein R. A global measure of perceived stress. J. Health Soc. Behav. 1983;24(4):385–396. [PubMed] [Google Scholar]
  20. Connor K.M., Davidson J.R. Development of a new resilience scale: the Connor-Davidson resilience scale (CD-RISC) Depress. Anxiety. 2003;18(2):76–82. doi: 10.1002/da.10113. [DOI] [PubMed] [Google Scholar]
  21. Cook L.J. Striving to help college students with mental illness. J. Psychosoc. Nurs. 2007;45(4):40–44. doi: 10.3928/02793695-20070401-09. [DOI] [PubMed] [Google Scholar]
  22. Copeland W., Shanahan L., Costello E.J., Angold A. Cumulative prevalence of psychiatric disorders by young adulthood: a prospective cohort analysis from the Great Smoky Mountains Study. J. Am. Acad. Child Adolesc. Psychiatry. 2011;50:252–261. doi: 10.1016/j.jaac.2010.12.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Cuijpers P., Driessen E., Hollon S.D., van Oppen P., Barth J., Andersson G. The efficacy of non-directive supportive therapy for adult depression: a meta-analysis. Clin. Psychol. Rev. 2012;32(4):280–291. doi: 10.1016/j.cpr.2012.01.003. [DOI] [PubMed] [Google Scholar]
  24. Das J.K., Salam R.A., Lassi Z.S., Khan M.N., Mahmood W., Patel V., Bhutta Z.A. Interventions for adolescent mental health: an overview of systematic reviews. J. Adolesc. Health. 2016;59(4):49–60. doi: 10.1016/j.jadohealth.2016.06.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Devilly G.J., Borkovec T.D. Psychometric properties of the credibility/expectancy questionnaire. J. Behav. Ther. Exp. Psychiatry. 2000;31(2):73–86. doi: 10.1016/s0005-7916(00)00012-4. [DOI] [PubMed] [Google Scholar]
  26. Díaz D., Rodríguez-Carvajal R., Blanco A., Moreno-Jiménez B., Gallardo I., Valle C., Van Dierendonck D. Adaptación española de las escalas de bienestar psicológico de Ryff. Psicothema. 2006;18(3):572–577. [PubMed] [Google Scholar]
  27. Ebert D.D., Zarski A.C., Christensen H., Stikkelbroek Y., Cuijpers P., Berking M., Riper H. Internet and computer-based cognitive behavioral therapy for anxiety and depression in youth: a meta-analysis of randomized controlled outcome trials. PLoS One. 2015;10(3) doi: 10.1371/journal.pone.0119895. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Eurostat Internet access and use statistics - households and individuals. 2016. http://ec.europa.eu/eurostat/statistics-explained/index.php/Internet_access_and_use_statistics_-_households_and_individuals (Online, Available at)
  29. Eysenbach G., Consort E-Health Group CONSORT-EHEALTH: improving and standardizing evaluation reports of web-based and mobile health interventions. J. Med. Internet Res. 2011;13(4):126. doi: 10.2196/jmir.1923. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Farrer L., Gulliver A., Chan J.K., Batterham P.J., Reynolds J., Calear A.…Griffiths K.M. Technology-based interventions for mental health in tertiary students: systematic review. J. Med. Internet Res. 2013;15(5):101. doi: 10.2196/jmir.2639. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Fergus S., Zimmerman M.A. Adolescent resilience: a framework for understanding healthy development in the face of risk. Annu. Rev. Public Health. 2005;26:399–419. doi: 10.1146/annurev.publhealth.26.021304.144357. [DOI] [PubMed] [Google Scholar]
  32. Frank D., DeBenedetti A.F., Volk R.J., Williams E.C., Kivlahan D.R., Bradley K.A. Effectiveness of the AUDIT-C as a screening test for alcohol misuse in three race/ethnic groups. J. Gen. Intern. Med. 2008;23(6):781–787. doi: 10.1007/s11606-008-0594-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Giedd J.N., Keshavan M., Paus T. Why do many psychiatric disorders emerge during adolescence? Nat. Rev. Neurosci. 2008;9(12):947. doi: 10.1038/nrn2513. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Gillham J.E., Reivich K.J., Freres D.R., Chaplin T.M., Shatté A.J., Samuels B., Seligman M.E. School-based prevention of depressive symptoms: a randomized controlled study of the effectiveness and specificity of the Penn resiliency program. J. Consult. Clin. Psychol. 2007;75(1):9–…. doi: 10.1037/0022-006X.75.1.9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Gladstone T.R.G., Beardslee W.R., O'Connor E.E. The prevention of adolescent depression. Psychiatr. Clin. North Am. 2011;34(1):35–52. doi: 10.1016/j.psc.2010.11.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Gray-Little B., Williams V.S.L., Hancock T.D. An item response theory analysis of the Rosenberg self-esteem scale. Personal. Soc. Psychol. Bull. 1997;23:443–451. [Google Scholar]
  37. Griffiths K.M., Christensen H. Internet-based mental health programs: a powerful tool in the rural medical kit. Aust. J. Rural Health. 2007;15(2):81–87. doi: 10.1111/j.1440-1584.2007.00859.x. [DOI] [PubMed] [Google Scholar]
  38. Hayward L., MacGregor A.D., Peck D.F., Wilkes P. The feasibility and effectiveness of computer-guided CBT (FearFighter) in a rural area. Behav. Cogn. Psychother. 2007;35(4):409–419. [Google Scholar]
  39. Heber E., Ebert D.D., Lehr D., Cuijpers P., Berking M., Nobis S., Riper H. The benefit of web-and computer-based interventions for stress: a systematic review and meta-analysis. J. Med. Internet Res. 2017;19(2) doi: 10.2196/jmir.5774. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Hervás G., Vázquez C. Congress: 3th European Conference of Positive Psychology. Braga; Portugal: 2006. Enjoying every little thing: orientation to Daily Gratification as a mediator of the relationship between extraversion and subjective well-being. [Google Scholar]
  41. Hervás G., Ortiz A., Valiente C., Cantero D., Vázquez C. In Poster Presented at 43th Annual Convention of ABCT. Association for Behavioral and Cognitive Therapies; New York, USA: 2009. Anticipatory and consummatory pleasure in depression and paranoia: the key role of social activities. [Google Scholar]
  42. Hill C.E., Knox S., Thompson B.J., Williams E.N., Hess S.A., Ladany N. Consensual qualitative research: an update. J. Couns. Psychol. 2005;52:196–205. [Google Scholar]
  43. Huebner E.S., Gilman R.I.C.H., Furlong M.J. Handbook of Positive Psychology in Schools. 2009. A conceptual model for research in positive psychology in children and youth; pp. 3–8. [Google Scholar]
  44. Institute for Employment Studies . Report to HEFCE by the Institute for Employment Studies (IES) and Researching Equity, Access and Partnership (REAP) 2015. Understanding provision for students with mental health problems and intensive support needs. [Google Scholar]
  45. John O.P., Srivastava S. Handbook of Personality: Theory and Research. Vol. 2. 1999. The big five trait taxonomy: history, measurement, and theoretical perspectives; pp. 102–138. [Google Scholar]
  46. John O.P., Donahue E.M., Kentle R.L. 1991. The Big five Inventory—Versions 4a And 54. [Google Scholar]
  47. Kaltenthaler E., Brazier J., De Nigris E., Tumur I., Ferriter M., Beverley C.…Sutcliffe P. Computerised cognitive behaviour therapy for depression and anxiety update: a systematic review and economic evaluation. Health Technol. Assess. 2006;10(33):1–186. doi: 10.3310/hta10330. [DOI] [PubMed] [Google Scholar]
  48. Kazdin A.E. Technology-based interventions and reducing the burdens of mental illness: perspectives and comments on the special series. Cogn. Behav. Pract. 2015;22(3):359–366. [Google Scholar]
  49. Kazdin A.E., Blase S.L. Rebooting psychotherapy research and practice to reduce the burden of mental illness. Perspect. Psychol. Sci. 2011;6:21–37. doi: 10.1177/1745691610393527. [DOI] [PubMed] [Google Scholar]
  50. Kazdin A.E., Rabbitt S.M. Novel models for delivering mental health services and reducing the burdens of mental illness. Clin. Psychol. Sci. 2013;1(2):170–191. [Google Scholar]
  51. Kiluk B.D., Serafini K., Frankforter T., Nich C., Carroll K.M. Only connect: the working alliance in computer-based cognitive behavioral therapy. Behav. Res. Ther. 2014;63:139–146. doi: 10.1016/j.brat.2014.10.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Kroenke K., Spitzer R.L., Williams J.B. The Phq-9. J. Gen. Intern. Med. 2001;16(9):606–613. doi: 10.1046/j.1525-1497.2001.016009606.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Larsen D.L., Attkisson C.C., Hargreaves W.A., Nguyen T.D. Assessment of client/patient satisfaction: development of a general scale. Eval. Program Plann. 1979;2(3):197–207. doi: 10.1016/0149-7189(79)90094-6. [DOI] [PubMed] [Google Scholar]
  54. Leppin A.L., Bora P.R., Tilburt J.C., Gionfriddo M.R., Zeballos-Palacios C., Dulohery M.M.…Montori V.M. The efficacy of resiliency training programs: a systematic review and meta-analysis of randomized trials. PLoS One. 2014;9(10) doi: 10.1371/journal.pone.0111420. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Lintvedt O.K., Griffiths K.M., Sørensen K., Østvik A.R., Wang C.E.A., Eisemann M., Waterloo K. Evaluating the effectiveness and efficacy of unguided internet-based self-help intervention for the prevention of depression: a randomized controlled trial. Clin. Psychol. Psychother. 2013;20(1):10–27. doi: 10.1002/cpp.770. [DOI] [PubMed] [Google Scholar]
  56. Löwe B., Decker O., Müller S., Brähler E., Schellberg D., Herzog W., Herzberg P.Y. Validation and standardization of the generalized anxiety disorder screener (GAD-7) in the general population. Med. Care. 2008;46(3):266–274. doi: 10.1097/MLR.0b013e318160d093. [DOI] [PubMed] [Google Scholar]
  57. Merikangas K.R., Nakamura E.F., Kessler R.C. Epidemiology of mental disorders in children and adolescents. Dialogues Clin. Neurosci. 2009;11(1):7–20. doi: 10.31887/DCNS.2009.11.1/krmerikangas. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Mihalopoulos C., Vos T., Pirkis J., Carter R. The economic analysis of prevention in mental health programs. Annu. Rev. Clin. Psychol. 2011;7:169–201. doi: 10.1146/annurev-clinpsy-032210-104601. [DOI] [PubMed] [Google Scholar]
  59. Mitchell A.M., Crane P., Kim Y. Perceived stress in survivors of suicide: psychometric properties of the perceived stress scale. Res. Nurs. Health. 2008;31:576–585. doi: 10.1002/nur.20284. [DOI] [PubMed] [Google Scholar]
  60. Mitchell J., Vella-Brodrick D., Klein B. Positive psychology and the internet: a mental health opportunity. Sensoria. 2010;6(2):30–41. [Google Scholar]
  61. Moher D., Schulz K.F., Altman D.G., Group, C The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. J. Am. Podiatr. Med. Assoc. 2001;91(8):437–442. doi: 10.7547/87507315-91-8-437. [DOI] [PubMed] [Google Scholar]
  62. Moher D., Hopewell S., Schulz K.F., Montori V., Gøtzsche P.C., Devereaux P.J.…Altman D.G. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. J. Clin. Epidemiol. 2010;63(8):1–37. doi: 10.1016/j.jclinepi.2010.03.004. [DOI] [PubMed] [Google Scholar]
  63. Montero-Marín J., Araya R., Pérez-Yus M.C., Mayoral F., Gili M., Botella C.…Nogueira-Arjona R. An internet-based intervention for depression in primary Care in Spain: a randomized controlled trial. J. Med. Internet Res. 2016;18(8):231. doi: 10.2196/jmir.5695. [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Muñoz R.F. Using evidence-based internet interventions to reduce health disparities worldwide. J. Med. Internet Res. 2010;12(5) doi: 10.2196/jmir.1463. [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. Neff K.D. Development and validation of a scale to measure self-compassion. Self Identity. 2003;2:223–250. [Google Scholar]
  66. O'Connell M.E., Boat T., Warner K.E. National Academy Press; Washington, DC: 2009. Preventing Mental, Emotional, and Behavioral Disorders Among Young People: Progress and Possibilities. [PubMed] [Google Scholar]
  67. Preschl B., Maercker A., Wagner B., Forstmeier S.A., Baños R.M., Alcañiz M., Castilla D., Botella C. Life-review therapy with computer supplements for depression in the elderly: a randomized controlled trial. Aging Ment. Health. 2012;16(8):964–974. doi: 10.1080/13607863.2012.702726. [DOI] [PubMed] [Google Scholar]
  68. Raes F., Pommier E., Neff K.D., Van Gucht D. Construction and factorial validation of a short form of the self-compassion scale. Clin. Psychol. Psychother. 2011;18:250–255. doi: 10.1002/cpp.702. [DOI] [PubMed] [Google Scholar]
  69. Rammstedt B., John O.P. Measuring personality in one minute or less: a 10-item short version of the big five inventory in English and German. J. Res. Pers. 2007;41(1):203–212. [Google Scholar]
  70. Ritterband L.M., Andersson G., Christensen H.M., Carlbring P., Cuijpers P. Directions for the international society for research on internet interventions (ISRII) J. Med. Internet Res. 2006;8(3):23. doi: 10.2196/jmir.8.3.e23. [DOI] [PMC free article] [PubMed] [Google Scholar]
  71. Roberts R.E., Attkisson C.C. Assessing client satisfaction among Hispanics. Eval. Program Plann. 1983;6(3–4):401–413. doi: 10.1016/0149-7189(83)90019-8. [DOI] [PubMed] [Google Scholar]
  72. Robins R.W., Hendin H.M., Trzesniewski K.H. Measuring global self-esteem: construct validation of a single-item measure and the Rosenberg self-esteem scale. Personal. Soc. Psychol. Bull. 2001;27(2):151–161. [Google Scholar]
  73. Rose R.D., Buckey J.C., Zbozinek T.D., Motivala S.J., Glenn D.E., Cartreine J.A., Craske M.G. A randomized controlled trial of a self-guided, multimedia, stress management and resilience training program. Behav. Res. Ther. 2013;51(2):106–112. doi: 10.1016/j.brat.2012.11.003. [DOI] [PubMed] [Google Scholar]
  74. Rosenberg M. Vol. 61. 1965. Rosenberg Self-esteem Scale (RSE). Acceptance and Commitment Therapy. Measures Package. [Google Scholar]
  75. Ryff C. Happiness is everything, or is it? Explorations on the meaning of psychological well-being. J. Pers. Soc. Psychol. 1989;57(6):1069. [Google Scholar]
  76. Ryff C. Psychological well-being revisited: advances in the science and practice of eudaimonia. Psychother. Psychosom. 2014;83:10–28. doi: 10.1159/000353263. [DOI] [PMC free article] [PubMed] [Google Scholar]
  77. Ryff C.D., Singer B. Psychological well-being: meaning, measurement, and implications for psychotherapy research. Psychother. Psychosom. 1996;65:14–23. doi: 10.1159/000289026. [DOI] [PubMed] [Google Scholar]
  78. Ryff C.D., Singer B., Love G.D., Essex M.J. Resilience in adulthood and later life: defining features and dynamic processes. In: Lomranz J., editor. Handbook of Aging and Mental Health: An Integrative Approach. Plenum Press; New York: 1998. [Google Scholar]
  79. Sander L., Rausch L., Baumeister H. Effectiveness of internet-based interventions for the prevention of mental disorders: a systematic review and meta-analysis. JMIR Ment. Health. 2016;3(3) doi: 10.2196/mental.6061. [DOI] [PMC free article] [PubMed] [Google Scholar]
  80. Sandin B., Chorot P., Lostao L., Joiner T.E., Santed M.A., Valiente R.M. The PANAS scales of positive and negative affect: factor analytic validation and cross-cultural convergence. Psicothema. 1999;11(1):37–51. [Google Scholar]
  81. Singh K., Yu X.N. Psychometric evaluation of the Connor-Davidson Resilience Scale (CD-RISC) in a sample of Indian students. Aust. J. Psychol. 2010;1(1):23–30. [Google Scholar]
  82. Spek V., Cuijpers P.I.M., Nyklícek I., Riper H., Keyzer J., Pop V. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol. Med. 2007;37(03):319–328. doi: 10.1017/S0033291706008944. [DOI] [PubMed] [Google Scholar]
  83. Spitzer R.L., Kroenke K., Williams J.B., Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch. Intern. Med. 2006;166(10):1092–1097. doi: 10.1001/archinte.166.10.1092. [DOI] [PubMed] [Google Scholar]
  84. Stallard P., Buck R. Preventing depression and promoting resilience: feasibility study of a school-based cognitive–behavioural intervention. Br. J. Psychiatry. 2013;202(54):18–23. doi: 10.1192/bjp.bp.112.119172. [DOI] [PubMed] [Google Scholar]
  85. Stengård E., Appelqvist-Schmidlechner K. Mental Health Promotion in Young People - An Investment for the Future. Publications of WHO Regional Office for Europe. 2010. http://www.euro.who.int/__data/assets/pdf_file/0013/121135/E94270.pdf
  86. Van Dierendonck D. The construct validity of Ryff's scales of psychological well-being and its extension with spiritual well-being. Personal. Individ. Differ. 2004;36(3):629–643. [Google Scholar]
  87. van Straten A., Cuijpers P., Smits N. Effectiveness of a web-based self-help intervention for symptoms of depression, anxiety, and stress: randomized controlled trial. J. Med. Internet Res. 2008;10(1):7. doi: 10.2196/jmir.954. [DOI] [PMC free article] [PubMed] [Google Scholar]
  88. van Zoonen K., Buntrock C., Ebert D.D., Smit F., Reynolds C.F., Beekman A.T., Cuijpers P. Preventing the onset of major depressive disorder: a meta-analytic review of psychological interventions. Int. J. Epidemiol. 2014;43(2):318–329. doi: 10.1093/ije/dyt175. [DOI] [PMC free article] [PubMed] [Google Scholar]
  89. Watson D., Clark L.A., Tellegen A. Development and validation of brief measures of positive and negative affect: the PANAS scales. J. Pers. Soc. Psychol. 1988;54(6):1063. doi: 10.1037//0022-3514.54.6.1063. [DOI] [PubMed] [Google Scholar]
  90. Wittkampf K.A., Naeije L., Schene A.H., Huyser J., van Weert H.C. Diagnostic accuracy of the mood module of the patient health questionnaire: a systematic review. Gen. Hosp. Psychiatry. 2007;29(5):388–395. doi: 10.1016/j.genhosppsych.2007.06.004. [DOI] [PubMed] [Google Scholar]
  91. World Health Organization The European Mental Health Action Plan. Objectives. 2013. http://www.euro.who.int/__data/assets/pdf_file/0004/194107/63wd11e_MentalHealth-3.pdf (The Available from)
  92. Yu X., Zhang J. Factor analysis and psychometric evaluation of the Connor-Davidson resilience scale (CD-RISC) with Chinese people. Soc. Behav. Personal. Int. J. 2007;35(1):19–30. [Google Scholar]

Articles from Internet Interventions are provided here courtesy of Elsevier

RESOURCES