Skip to main content
PMC home page
Internet Interventions logoLink to Internet Interventions
. 2018 Mar 15;16:35–42. doi: 10.1016/j.invent.2018.02.004

Web-based indicated prevention of common mental disorders in university students in four European countries – Study protocol for a randomised controlled trial

Peter Musiat a,⁎,1, Rachel Potterton a,1, Gemma Gordon a, Lucy Spencer a, Michael Zeiler b, Karin Waldherr c, Stefanie Kuso b,c, Martina Nitsch c, Tanja Adamcik c, Gudrun Wagner b, Andreas Karwautz b, David Daniel Ebert d, Alyson Dodd e, Barbara Dooley f, Amy Harrison g, Emma Whitt h, Mark Haselgrove h, Helen Sharpe i, Jo Smith j, Rosie Tressler k, Nicholas Troop l, Chantal Vinyard j, Dennis Görlich m, Jenny Beecham n, Eva Bonin n, Corinna Jacobi o, Ulrike Schmidt a
PMCID: PMC6364328  PMID: 30775263

Abstract

Background

Mental disorders and their symptoms are highly prevalent in the university student population, and the transition from secondary to tertiary education is associated with a rise in mental health problems. Existing web-based interventions for the prevention of common mental disorders in student populations often focus on just one disorder and have not been designed specifically for students. There is thus a need for transdiagnostic, student-specific preventative interventions that can be widely disseminated. This two-arm, parallel group randomised controlled trial aims to evaluate the effectiveness and cost-effectiveness of a web-based transdiagnostic mental health problem prevention programme (PLUS) across several universities in four countries.

Method

Students (N = 5550) will be recruited through a variety of channels and asked to complete a personality assessment to determine whether they are at high risk for developing common mental disorders. Students at high risk will be randomly allocated to either PLUS or a control intervention, which provides practical support around issues commonly experienced at university. Students at low risk will be allocated to the control intervention. Both intervention groups will be assessed at baseline, 4 weeks, 3 months, 6 months and 12 months after randomisation. Depression and generalised anxiety, assessed using the Patient Health Questionnaire and the Generalised Anxiety Disorder scales, will form the primary outcomes in this study. Secondary outcome measures include alcohol and drug use, eating behaviour, self-esteem, and quality of life. The cost-effectiveness of the intervention will also be evaluated.

Conclusions

This study will contribute to understanding the role of transdiagnostic indicated web-based interventions for the prevention of common mental disorders in university students. It will also be one of the first studies to investigate the cost-effectiveness of such interventions.

Trial Registration

This trial was registered in the ISRCTN register (ISRCTN15570935) on 12th February 2016.

Keywords: Students, Internet, Mental health, Depression, Anxiety, Prevention, CBT

Highlights

  • •

    University students have high levels of unmet need in mental health prevention and treatment.

  • •

    Transdiagnostic interventions can be rolled out broadly and may have greater public health impact than disorder-specific alternatives.

  • •

    The PLUS intervention has previously demonstrated its effectiveness.

  • •

    This research builds on previous work by investigating effectiveness in a multi-country RCT.

1. Introduction

Emerging adulthood is increasingly recognised as a developmental period associated with distinct psychosocial challenges (e.g. feeling in-between, identity explorations, changes in relationships and work/study, enhanced self-focus and few obligations to others) (Arnett, Žukauskienė, & Sugimura, 2014). Many mental disorders have their first onset before the age of 24, and incidence of mental disorders peaks during emerging adulthood (Kessler et al., 2005). Emerging adults are thus a population group in manifest need of timely, appropriate and effective mental health interventions (McGorry, Goldstone, Parker, Rickwood & Hickie, 2014). Emerging adults studying at university are often exposed to additional stressors - such as leaving the familial home, building social networks and adapting to a new academic environment – which are likely to further impact mental health (Royal College of Psychiatrists, 2011). Indeed, university students experience more emotional problems than non-studying individuals of the same age (Jenkinson, Coulter & Wright, 1993).

Universities typically offer face-to-face advice, assessment and support for mental health problems. However, counselling services report being under significant pressure, with increased demand for services occurring in the context of decreasing available resources (British Association for Counselling & Psychotherapy, 2014). Long breaks between university terms can also make it difficult to provide continuity of face-to-face care. Moreover, research has indicated that many students delay or avoid seeking help from professional services (Eisenberg, Hunt, Speer & Zivin, 2011). Barriers to help-seeking include fear of stigmatisation, dislike of traditional healthcare, and lack of knowledge of what help is available (Czyz, Horwitz, Eisenberg, Kramer & King, 2013; Eisenberg, Hunt, Speer & Zivin, 2011; Mowbray et al., 2006; Vidourek, King, Nabors & Merianos, 2014). In light of such challenges, web-based interventions have a number of clear advantages over more traditional face-to-face services. Reduced need for face-to-face therapist input is likely to decrease costs of provision, and the potential for remote access may address issues regarding continuity of care. Further, web-based interventions may overcome many of the barriers to help-seeking associated with traditional services (Chan, Farrer, Gulliver, Bennett, & Griffiths, 2016; Lungu & Sun, 2016; Sánchez-Ortiz et al., 2011).

Several web-based mental health prevention programmes have been evaluated in students in tertiary education. Farrer et al. (2013) systematically reviewed such interventions and identified 27 studies. Just half of the studies reported one or more positive significant outcome, and approximately one third found no effect. Farrer and colleagues suggested that such findings may stem from the fact that many interventions were not specifically designed for students, with student samples used out of convenience. Many of the studies had methodological problems and were of relatively poor quality, and none of the studies included in the review had investigated the cost-effectiveness of their interventions. The authors concluded that, whilst technology based interventions show promise in university settings, there is a need for high quality trials in this field (Farrer et al., 2013).

Since the publication of Farrer and colleagues' systematic review, several additional trials have been published. Mazurek Melnyk et al. (2015) conducted a randomised controlled pilot study to evaluate the effectiveness of an online skills-building programme in first-year university students. They reported no significant differences in anxiety and depression symptoms between the control and intervention groups, although students with high anxiety at baseline reported a significant decline in symptoms following the intervention. Another trial by Räsänen, Lappalainen, Muotka, Tolvanen, and Lappalainen (2016) examined the effectiveness of an online guided acceptance and commitment therapy-based programme in university students. Rasanen and colleagues found that, following the intervention, participants reported increased wellbeing and life satisfaction and decreased depressive symptoms. Both of these studies were preliminary in nature, and further research is needed before firm conclusions can be drawn about the effectiveness of these programmes.

In a pilot randomised controlled trial, we previously investigated the efficacy of Personality and Living of University Students (PLUS), a transdiagnostic, personality trait-focused web-based prevention intervention (Musiat et al., 2014), compared to a control programme providing practical advice to issues commonly experienced by students. In line with Farrer and colleagues' recommendations, PLUS was developed specifically for university students. In contrast to previous disorder-specific interventions, PLUS targets personality-based risk factors underlying a range of common mental disorders. The rationale for this model arises from research indicating high levels of comorbid common mental disorders in students (Verger, Guagliardo, Gilbert, Rouillon, & Kovess-Masfety, 2010), as a result of overlapping of genetic and personality risk factors and associated information processing styles (e.g. Clark, Watson, & Mineka, 1994; Glindemann, Geller, & Fortney, 1999; Shafran & Mansell, 2001). Transdiagnostic, personality-based interventions can be offered and rolled out more broadly than disorder-specific alternatives, and thus may have greater public health impact (Brown & Barlow, 2009). Musiat and colleagues found that, after receiving the intervention, students at high risk for common mental disorders showed reduced symptoms of depression and anxiety and improved self-esteem.

This previous study assessed PLUS exclusively within the UK university context. Given between-country variability in educational systems and settings, as well as differing linguistic and cultural contexts, it is not clear to what extent the results of the study can be extrapolated to other countries. Further, the study did not explore the cost-effectiveness of PLUS. The current randomised controlled trial thus builds on our previous work by investigating the effectiveness and cost-effectiveness of PLUS in a pragmatic, multi-centre randomised controlled trial across four countries (UK, Republic of Ireland, Austria, and Germany).

2. Materials and methods

2.1. Objectives and hypotheses

The overall aim of the study is to investigate the effectiveness and cost-effectiveness of PLUS in a pragmatic, multi-centre randomised controlled trial across four European countries (UK, Republic of Ireland, Austria, and Germany).

The specific aims are as follows:

  • 1.

    To compare the effectiveness of PLUS versus an active control intervention on a range of outcomes, including depression, anxiety and eating disorder symptoms and self-esteem, in a university student population.

  • 2.

    To compare the cost-effectiveness of PLUS versus an active control intervention in a university student population.

2.2. Participants

A total of 5550 university students will be recruited from several universities in the UK, Republic of Ireland, Austria, and Germany. To be eligible for inclusion in the study, students must:

  • 1)

    be at least 18 years old;

  • 2)

    have not received any psychiatric diagnosis in the previous 12 months;

  • 3)

    have never been diagnosed with psychosis or bipolar disorder;

  • 4)

    not be currently receiving any psychological therapy.

Students will be recruited through a variety of channels, including online media, such as email circulars, social networks, websites, and offline approaches, such as posters, flyers or print advertisements. The recruitment material contains a web link to the Minddistrict online platform, on which students are provided with further information about the study. Informed consent is obtained electronically on this platform.

2.3. Study design

This study is a two-arm, prospective, parallel-group randomised controlled trial. Students will be recruited at universities in the UK, Republic of Ireland, Germany and Austria using a variety of recruitment channels. Students will access the project website and complete a baseline assessment (t0). Based on this assessment, and using a logistic regression model developed in a previous study (Musiat et al., 2014) students are categorised into low or high risk for developing mental disorders. Students at high risk will be randomly allocated (1:1) to the intervention group or the active control group. Simple randomisation with computer generated random numbers will be used. Both intervention groups will receive access to the online modules of the respective intervention and be asked to complete the modules over a period of 12 weeks. Due to the nature of the study, neither participants nor the research team will be blinded to participant condition assignment. Participants will not be permitted to be re-assigned to a different study condition at any time. However, participants will be informed that their participation in the study is entirely voluntary, and they are permitted to discontinue their involvement in the study at any time, without being required to give a reason. Post-intervention and follow-up assessments will be conducted at 3 months (T1), 6 months (T2), and 12 months (T3) after randomisation (Fig. 1). There will also be an intermediate assessment at 4 weeks.

Fig. 1.

Fig. 1

Open in a new tab

Study flow diagram.

This study will be conducted in compliance the Declaration of Helsinki (World Medical Association, 2001), the principles of good clinical practice (ICH-E6 guideline) and the ICH-E8 guideline. Results of this trial will be reported in accordance with the CONSORT 2010 Statement (Moher, Schulz, Altman, and Group, C., 2001) and the CONSORT-EHEALTH Statement (Eysenbach, 2011). This trial is registered in the ISRCTN register (ISRCTN15570935). Ethical approval has been obtained from the Psychiatry, Nursing and Midwifery Research Ethics Subcommittee of Kings College London (reference number: PNM-14/15–130, date of approval 23/04/2015) and the Research Ethics Committee of the Medical University Vienna (reference number 2208/2015, date of approval 22/01/2016). This protocol paper follows the guidelines of Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT). The research team do not expect there to be any significant modifications to the study protocol. Any unexpected changes will be reported to the relevant research authorities and trial registries by research teams in the UK and Germany. Communication with study participants of any unexpected modifications to the study would occur via email. Data security/confidentially will be guaranteed and all relevant EU legislation and international texts on privacy will be observed and respected. Regarding regulation at international level, starting from the OECD guidelines including the “Guidelines on the protection of privacy and transborder flow of personal data” (1981) and “Guidelines for the security of information systems” (1991/92), the I-CARE consortium in particular acknowledges heterogeneity in international data protection jurisdiction.

2.4. Interventions

Both PLUS and the active control intervention are provided via the Minddistrict online platform. All participants have a unique account on the platform, which can be logged into using a valid email address and a password of their choosing. Participants receive email notifications when new content (e.g. new modules; follow-up assessments) are available to them. Although the platform allows conversations between users, this functionality is disabled for the purposes of this trial. The platform can be accessed by several electronic devices (PC, tablet, smartphone). However, participants are advised that the programmes function optimally on PCs.

2.4.1. Transdiagnostic web-based prevention

PLUS (Personality and Living of University Students) is a web-based intervention for the prevention of depressive, anxiety, substance use and eating disorders in university students. The intervention in this trial is based on the intervention from our previous randomised controlled trial (Musiat et al., 2014), but has been modified to incorporate user feedback received in previous studies. For example, students in the previous trial could freely choose the order of access to each intervention module and noted that they would have liked to have a more linear progression through the content. As a result, the module order in the PLUS intervention is fixed, which also allows the modules to stronger build on each other's content. Other feedback included the perception that the intervention was perceived as very “text-heavy”. Thus the amount of text in each module was greatly reduced by shortening the content, splitting modules over two sessions, and including videos to illustrate concepts.

The intervention follows a cognitive-behavioural approach and consists of seven modules. Each module follows a similar structure and includes a brief summary of the previous module, take home messages for the current module, and exercises for students to practise between sessions. Case stories and examples relevant to emerging adults and focussing on student- specific vignettes and scenarios are included, showing how problems arise and new learning can be applied in University settings. The transdiagnostic intervention is designed to be completed by students independently and without personal support.

2.4.1.1. Module 1

In the first module, students are provided with some basic information about the purpose and structure of the intervention. After this information, they are asked to complete the baseline assessment, which consists of a range of personality and mental health questionnaires. The results of this assessment determine whether a student is considered at high or low risk for developing mental disorders.

2.4.1.2. Module 2

The second module starts with providing students with personalised feedback on their questionnaire results. Following the guidelines for providing feedback in e-mental health (Musiat, Hoffmann, & Schmidt, 2012), students are provided with information on what each scale assesses, as well as a summary of their responses. In the remainder of the module, a basic cognitive-behavioural model is introduced. The model highlights how thoughts, feelings, behaviour and bodily sensations are connected with each other and provides a foundation for the content of subsequent modules. In addition, through a range of examples related to student and young adult life (sharing accommodation, choosing student projects, relationships), participants learn about helpful and unhelpful cycles of behaviour and how unhelpful behaviours may appear beneficial initially, but have unintended negative consequences in the longer term, whereas the opposite is the case in helpful behaviours.

2.4.1.3. Module 3&4

Over two modules, students learn about the impact of low self-esteem on health, behaviour and well-being, and are provided with strategies for improving self-esteem. The modules start with a brief exercise helping students to conceptualise self-esteem and recognise their sources of self-worth. With the help of vignettes, students learn about the impact of low self-esteem on well-being, as well as patterns of thoughts and behaviours typically for individuals affected by low self-esteem. The modules provide a writing exercise to help students deal with excessive self-criticism, particularly with regard to self-criticism related to academic performance. Finally, the link between self-esteem and communication style is explored. Students learn about submissive, aggressive and assertive communication styles and are provided with techniques for communicating assertively.

2.4.1.4. Module 5

The focus of this module is on perfectionism and its impact on thoughts, behaviour, feelings, and bodily sensations. The module starts with defining perfectionism and exploring the impact of perfectionism on well-being by looking at the pros and cons of perfectionistic thinking and behaviour. Students then are encouraged to explore ten common unhelpful thinking styles associated with high levels of perfectionism, such as black and white thinking, catastrophising, or excessive self-criticism. The modules discussed how these thinking styles manifest themselves in typical perfectionistic behaviours, such as overcompensating or procrastination. The module concludes with a seven-step approach for challenging high levels of perfectionism. An example of a student with high perfectionism with regard to their academic performance is provided and the seven-step approach is illustrated using this example.

2.4.1.5. Module 6

In this module, students learn about anxious personality traits and worry, and how to overcome anxiety and worry. The module starts with exploring why anxiety and feelings of panic are important emotions and essential evolutionary mechanisms. This is done in the context of the model introduced in Module 2. Students learn about worry and rumination and are introduced to ten common behaviours that indicate excessive worry. In addition, the concepts of productive and unproductive worry are explored and how to identify unproductive worry. This module provides students with three techniques for dealing with excessive worry: gaining a more realistic perspective, dealing with uncertainty, and mindfulness. Students are encouraged to practice gaining a more realistic perspective using an example of a student in an unhappy relationship, who is anxious about breaking up with their partner.

2.4.1.6. Module 7

The final module of the intervention focuses on dealing with difficult emotions. In the context of the model introduced in Module 2, students learn about the importance of different emotions and their effect on thoughts, behaviour, and bodily sensation. Students explore the positive aspects of different emotions, both positive and negative. Students are encouraged to record their emotions over one week in order to identify unhelpful responses to emotions. Several unhelpful behaviours, such as emotional eating, or using drugs or alcohol, and their negative long-term impact are discussed. Finally, students are provided with a range of strategies to help them deal with negative emotions more effectively by being able to better recognise their emotions, recognising unhelpful behaviour and using more helpful behaviour.

2.4.2. Control intervention

The control intervention is designed to provide students with practical support around issues commonly experienced at university. The control intervention consists of four modules. The first module focuses on time management. Students learn about how to use goal setting and improve their time management through tracking time, planning time and being organised. The module also contains information about how to overcome procrastination and how to work effectively with academic texts. The second module provides information on how to find accommodation when studying away from home. It discusses different accommodation options for students, their pros and cons, and financial aspects. A moving home checklist is provided. The third module provides advice on safe alcohol use. In the fourth module, students are provided with a range of tips for saving money and managing their finances at university. The content for the control intervention was identified in a consultation process with undergraduate students. In our previous randomised controlled trial, students rated the control intervention as helpful and there were no differences in dropout rates between the control intervention and the transdiagnostic intervention (Musiat et al., 2014).

2.4.3. Intervention translation and adaptation

The transdiagnostic and control interventions were translated from English to German. The translation was checked by at least two further researchers fluent in English and German and changed if necessary. The German PLUS intervention corresponds to the English version with regard to the content and layout. Small adaptations were made regarding examples and vignettes to fit the Austrian and German context. In the control intervention cultural adaptations were made within the modules about accommodation and finances reflecting the Austrian situation. One first year student helped with the adaptation of the control intervention modules.

2.5. Assessment and data management

Data management and monitoring will be provided by DG (WWU) for the whole ICare consortium, in order to maintain comparable high quality in the conduct of the ICare research projects in trial planning, data management, online monitoring, and analysis. Within the ICare project a harmonised data management plan is implemented to provide high quality data with respect to accuracy, composition and organisation, completeness, transparency of processes, and timeliness. During the active phase of the trial data (i) completeness, (ii) timeliness and (iii) internal validity will be monitored. Internal validity will be check by plausibility rules. Data will be collected on the Minddistrict platform. After export from the platform data will be processed in a unified manner for all ICare studies, using programming scripts implemented in the SAS software (SAS Inc., Cary, NY, USA).

3. Data analysis

3.1. Outcomes

3.1.1. Primary outcomes

Severity of depression will be assessed using the Patient Health Questionnaire (PHQ-9, Kroenke, Spitzer, & Williams, 2001). This self-report measure includes nine items describing depressive symptoms. Participants have to indicate how often they have experienced each symptom within the last two weeks on a four-step rating scale, ranging from not at all (0) to nearly every day (3). The PHQ-9 has been extensively validated in both English and German and has good internal consistency (Cronbach's α = 0.88–0.89, Kroenke et al., 2001, Löwe et al., 2004) and correlates moderately (0.72) with the Beck Depression Inventory II (Titov et al., 2011).

Severity of anxiety will be assessed using the Generalised Anxiety Disorder scale (GAD-7, Spitzer, Kroenke, Williams, & Löwe, 2006). It consists of seven items and participants have to indicate how often they have experienced symptoms of generalised anxiety within the past two weeks on a four-step rating scale ranging from not at all (0) to nearly every day (3). In a validation study with a general population, the GAD-7 demonstrated good internal consistency (Cronbach's α = 0.89), a stable one-factor structure and was moderate correlated to depression as assessed with the PHQ-2 (0.64), as well as weakly negatively (−0.43) correlated with self-esteem as assessed with the Rosenberg Self-Esteem Scale (Löwe et al., 2008).

3.1.2. Secondary outcomes

To assess alcohol consumption, the Alcohol Use Disorders Identification Test (AUDIT, Saunders, Aasland, Babor, De La Fuente, & Grant, 1993) will be used. This self-report measure developed by the World Health Organization consists of 10 items with assessing hazardous drinking, harmful use and alcohol dependence. In student samples, the AUDIT demonstrated good internal consistency (Cronbach's α = 0.80, Fleming, Barry, & MacDonald, 1991) and appears superior for detecting high risk drinking in students to other measures (Kokotailo et al., 2004). In the German version of the AUDIT, reliability was found to be high (intraclass correlation coefficient) for the total score (ICC = 0.95)(Dybek et al., 2006).

Drug use will be assessed using the Drug Use Disorders Identification Test (DUDIT, Berman, Bergman, Palmstierna, & Schlyter, 2005). The self-report measure contains 11 items, which added together create a final score. Higher scores indicate more severe problems with drug use. A recent review of the psychometric properties of the DUDIT (Hildebrand, 2015) in different samples found internal consistencies (Cronbach's α) ranging from good (0.80) to excellent (0.97). However, results on the factor structure of the DUDIT in different studies were equivocal.

The Eating Disorders Diagnostics Scale (EDDS) will be used to assess the severity of eating disorder symptoms (Stice, Telch, & Rizvi, 2000). This commonly used screening tool for the identification of eating disorders according to the DSM-IV criteria also provides a symptom composite score as a measure of eating disorder severity. Internal consistency for this composite score is good (Cronbach's α = 0.89). With regard to the scale's convergent validity, the composite score has weak to strong correlations with scores from the Eating Disorders Examination (Stice, Fisher, & Martinez, 2004).

Quality of life will be assessed using the WHOQOL-BREF (The Whoqol Group, 1998). This self-report measure assesses quality of life using 26 items on the domains: physical health, psychological health, social relationships, and environment. A large scale cross-cultural validation study reported internal consistencies of at least acceptable level (>0.70) and demonstrated that WHOQOL scores from individuals with physical or mental illness were significantly different across domains that scores from individuals without (Skevington, Lotfy, & O'Connell, 2004). The subscales of the German version have internal consistencies ranging from r = 0.57 to r = 0.88 (Angermeyer, Kilian, & Matschinger, 2000).

The Rosenberg Self-Esteem Scale (RSES, Rosenberg, 1965) will be used to assess self-esteem. The measure consists of 10 items and participants have to indicate their agreement to each item on a four-step rating scale ranging from Strongly agree to Strongly disagree. The scale has been validated in university students and demonstrated good internal consistency (Cronbach's α = 0.85) and good test-retest reliability (r = 0.84) after four weeks (Martín-Albo, Núñez, Navarro, & Grijalvo, 2007). The German version of the RSES demonstrated good internal consistency (Cronbach's α = 0.81) and acceptable test-retest reliability (r = 0.73) after six months (Ferring & Filipp, 1996).

The 10-item Connor-Davidson Resilience Scale −10 (CD-RISC 10, Campbell-Sills & Stein, 2007; German version: unpublished translation by Ebert & Zarski, 2014) will be used to assess resilience. The CD-RISC-10 is a briefer version of the full 25-item version of the scale, assessing resilience through a 10-item self-report questionnaire. The 10-item version used within the current study has been found to display good levels of internal consistency and construct validity (Campbell-Sills & Stein, 2007).

An adapted version of the Working Alliance Inventory (WAIS-SR, Hatcher & Gillaspy, 2006; German version: Wilmers et al., 2008; adapted for online interventions) will also be used to assess therapeutic alliance.

The cost-effectiveness of the intervention will be assessed using the Client Service Receipt Inventory (CSRI, Beecham & Knapp, 2001). This self-report assesses the use of health services in a defined period. The measure has been extensively used in the cost-effectiveness evaluation of mental health interventions and demonstrated concurrent validity with regard to general practitioner visits (Patel et al., 2005).

3.1.3. Risk assessment measures

In this study, only students at high risk for the development of mental health problems are randomly allocated to one of the two conditions. This risk assessment is based on results from our previous research on student mental health (Musiat et al., 2014). To assess risk, the scales from three questionnaires assessing aspects of personality are used.

A range of personality factors will be assessed using the Big Five Inventory (BFI, John, Naumann, & Soto, 2008). This self-report measure assesses personality on the domains Neuroticism, Extraversion, Openness, Agreeableness, and Conscientiousness with 44 items. Each item is scored on a five-step Likert scale ranging from disagree strongly (1) to agree strongly (5). Schmitt, Allik, McCrae, and Benet-Martínez (2007) conducted a cross-cultural validation of the BFI and in a sample from Western Europe, all the subscales Neuroticism, Extraversion, Openness, and Conscientiousness had acceptable or good internal consistency (Cronbach's α, 0.79–0.84). However, the scale Agreeableness had questionable internal consistency (0.68). The authors also found the five factor structure to be stable across cultures (Schmitt et al., 2007). The BFI's subscales correlate moderately to highly with the same Five Factors as assessed by the NEO-FFI (Costa & MacCrae, 1992), thus demonstrating convergent validity (Benet-Martinez & John, 1998). Participants will be assessed on all dimensions, however, only the Neuroticism subscale in included in the model to determine risk.

Perfectionism will be assessed using the Frost Multidimensional Perfectionism Scale (FMPS, Frost, Marten, Lahart, & Rosenblate, 1990). This self-report measure assesses perfectionism on the domains Concern over Mistakes, Personal Standards, Parental Expectations, Parental Criticism, Doubts about Actions, and Organization. The questionnaire has 35 items to which participants have to respond on a five-point rating scale ranging from strongly disagree (1) to strongly agree (5). The original authors of this questionnaire demonstrated a good reliability of the FMPS. Internal consistencies (Cronbach's α) for the subscales ranged from 0.77 to 0.93 and an overall internal consistency of 0.90 was reported. The subscales of the FMPS were demonstrated to have moderate to high correlations with the Burns Perfectionism Scale (Burns, 1980) and the perfectionism subscale of the Eating Disorder Inventory (Garner, Olmstead, & Polivy, 1983). However, the factor structure of the FMPS has been subject to criticism. Particularly the Parental Expectations and Parental Criticism subscale seem to load on the same factor, and Organisation often loads on other factors (e.g. Stöber, 1998). For that reason and to reduce participant burden, only items for the subscales Concern over Mistakes, Personal Standards and Doubts about Action were included in this study.

The Substance Use Risk Profile (SURPS, Woicik, Stewart, Pihl, & Conrod, 2009) will be used to assess four personality risk factors associated with substance use disorders: introversion-hopelessness (H), anxiety sensitivity (AS), impulsivity (I), and sensation seeking (SS). This self-report measure contains a total of 23 statements and participants have to indicate their level of agreement with each item on a four-point rating scale ranging from strongly disagree (1) to strongly agree (4). The SURPS has been validated with undergraduate students and demonstrated questionable to good internal consistency (AS: 0.61, I: 0.64,SS: 0.70, H: 0.86) (Woicik et al., 2009). With regard to the measure's convergent validity, it has been shown that the subscales H, SS and I are significantly correlated with the frequency of drinking in students, whereas the AS subscale is correlated with the severity of alcohol related problems in students (Woicik et al., 2009).

The measures used in the study are listed in Table 1.

Table 1.

Measures included in study, and time-points at which assessed.

Measure Baseline
(T0)		 4 weeks after start of intervention
(Mediators Only)		 3 month follow up
(T1)		 6 month follow up
(T2)		 12 month follow up
(T3)
Socio-demographic variables X
AUDIT X X X X X
CD-RISC X X X X
CSRI X X X X
DUDIT X X X X
EDDS X X X
FMPS X
GAD X X X X X
PHQ X X X X X
RSES X X X X X
SURPS X
WHOQOL BREF X X X X
WAI-SR X X
Open in a new tab

3.2. Statistical methods

For the analysis of the PLUS study data we follow adopted guidelines, e.g. ICH E9 (http://www.ema.europa.eu/). The primary analysis of the data will be described in a statistical analysis plan (SAP). A blinded data review will be performed to take decisions on the multiple imputation strategy and the selection on multivariable models.

The overall analysis strategy consists of the following steps: (i) data description, (ii) analyses of the primary hypothesis (iii) secondary analyses (iv) further exploratory analyses.

Balancing of the randomisation will be controlled by appropriate statistical tests of the baseline variables. The study collective will be characterised by descriptive statistical methods such as relative and absolute frequencies, mean, median, standard deviation, and inter-quartile-range (IQR), and appropriate graphics such as histograms, boxplots, and bar charts. Mean and median will be accompanied by 95%-confidence intervals. All measurement time points (T0, T1, T2, and T3) will be described separately. Descriptive statistics will be provided for both study arms. Assumptions for the appropriate statistical tests will be checked (e.g. normality).

3.2.1. Primary confirmatory analysis

Two primary null-hypotheses will be tested to prove superiority of PLUS over the control intervention by comparison of the mean change in PHQ9 scores and mean change in GAD7 scores between baseline and 12-month follow-up. We will test both null-hypotheses with two-sided two-sample t-Test in the intention-to-treat (ITT) collective. The ITT sample comprises all randomised participants who provided the primary outcome measure within the initially assigned study arm. The primary hypotheses will be tested with a local significance level of 2.5%, applying a Bonferroni correction. By adjusting the local significance level we maintain a global significance level of 5% for the whole trial. The primary confirmatory analysis will be performed in the SAS software (SAS Inc., Cary, NY, USA). All programming scripts will be validated by a second statistician at the University of Münster.

3.2.2. Sensitivity analyses of the primary analysis

A number of preplanned sensitivity analyses of the primary analysis will be performed. We will analyse the pre-post differences for the primary and all secondary outcomes using two-sided Mann-Whitney-U tests in case of non-normal differences or two-sided paired t-Tests in case of normally distributed differences of the pre-post scores. Also, the primary analysis will be repeated using the per-protocol (PP) sample, i.e. participants without major protocol violations. The primary outcome will also be analyzed within a generalised linear mixed model including covariates as defined within the blinded data review and the study platform (i.e. center). Furthermore, the primary analysis will be repeated as stratified and subgroup analyses. To assess the effect of missing data on the primary analysis the primary outcome will be reanalysed after a multiple imputation strategy (developed within blinded data review) was applied to the data.

3.2.3. Analysis of secondary outcomes

Secondary outcomes will be compared between groups at individual time points using t-Test for unpaired data or the Mann-Whitney U Test, depending on the normal distribution of scores. Categorical variables will be tested using Fisher's exact test or Chi-squared tests. The collected longitudinal data will be analyzed using repeated-measures ANOVA or (generalised) linear mixed models (GLMM) with the appropriate link function.

The results of the primary and secondary analyses will be represented by appropriate effect estimates and 95% confidence intervals.

All secondary analyses have to be considered exploratory and hypothesis-generating. We will consistently use the nominal significance level of 0.05 (two-sided) also for exploratory analyses.

3.3. Sample size calculation

The primary endpoint in this study is the difference in student depression (PHQ9 scores) between baseline (t0) and the 12-month follow-up (t3). To detect an effect of, at least, d = 0.3 (see Musiat et al., 2014), with a probability of 90% (Type-II Error β = 10%) and a significance level of α = 1% using a two-sided two-sample t-test at least N = 666 students will need to be included in the analysis. At each assessment (t1, t2, t3), we assume a 15% drop out rate, leading to a total loss of trial participants of approximately 40% at t3. In addition, our previous research suggests that one in five students is at high risk for developing mental disorders (Musiat et al., 2014). Thus N = 5550 students will have to be recruited to yield a trial sample of 1110.

4. Discussion

The study described in this protocol aims to evaluate the effectiveness and cost-effectiveness of PLUS, a web-based transdiagnostic programme for the indicated prevention of common mental disorders in undergraduate university students. It is expected that students at high risk who receive PLUS will show a greater reduction in depression and generalised anxiety and in secondary outcomes than students in the control group.

It is expected that the present study will make an important contribution to the field of e-mental health and student mental health. This study investigates a transdiagnostic universal intervention targeting common mental disorders by addressing underlying risk factors and their impact on mental health. This approach may constitute a more efficient, pragmatic and economical approach to student mental health promotion than approaches focusing on single disorders. Thus, the results of this study will also shed light on whether this pragmatic approach is acceptable to students, leads to improved outcomes, and thus may present a viable public health strategy. Web-based cognitive behavioural interventions in other populations, generally have been shown to be cost-effective (Musiat & Tarrier, 2014). The present study will be one of the first studies to investigate the cost-effectiveness of such an intervention in a higher education context.

This study has a number of strengths. The large sample size will ensure that the study is sufficiently powered and this will be one of the largest trials of a web-based mental health intervention in tertiary education to date. By conducting the study across multiple centres in four countries, the findings will have greater generalisability than previous studies. The present study will also address some of the limitations of our previous study (Musiat et al., 2014). The follow-up period in the present study will be considerably longer, thus allowing us to investigate the stability of intervention effects.

5. Conclusion

This study is a multicentre randomised controlled trial investigating the efficacy of the transdiagnostic, web-based mental health prevention intervention (PLUS) for University students. University students are a group particularly at risk of mental health problems, yet are not adequately catered for by traditional mental health services. Web-based interventions hold much promise in student populations and, if efficacious and cost-effective, PLUS could constitute an innovative public health intervention to improve student mental health.

Trial status

The first participants were enrolled in the study on 23rd March 2017. Follow-up assessments for all participants are expected to be completed by December 2018.

Funding

This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 634757. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Author contributions

PM and US designed the study in the first place. DG and JB contributed significantly to the study design. PM and RP wrote the first draft of the manuscript. PM, RP, US & DG revised the manuscript and did the final draft. All authors read and approved the final manuscript.

Conflict of interest statement

The authors declare that they have no competing or conflicting interests.

Acknowledgements

We would like to thank all our participants, without whom this study would not be possible. US receives salary support from the National Institute for Health Research (NIHR) Biomedical Research Centre (BRC) at South London and Maudsley NHS Foundation Trust and King's College London. US is supported by an NIHR Senior Investigator Award. The views expressed herein are not necessarily those of the NHS or NIHR.

References

  1. Angermeyer M., Kilian R., Matschinger H. Hogrefe; Göttingen: 2000. Handbuch für die deutschsprachigen Versionen der WHO-Instrumente zur internationalen Erfassung von Lebensqualität (WHOQOL) [Google Scholar]
  2. Arnett J.J., Žukauskienė R., Sugimura K. The new life stage of emerging adulthood at ages 18–29 years: implications for mental health. Lancet Psychiatry. 2014;1(7):569–576. doi: 10.1016/S2215-0366(14)00080-7. [DOI] [PubMed] [Google Scholar]
  3. Beecham J., Knapp M. Costing psychiatric interventions. In: Thornicroft G., Brewin C., Wing J., editors. Measuring Mental Health Needs. Gaskell; London: 2001. pp. 179–190. [Google Scholar]
  4. Benet-Martinez V., John O.P. Los Cinco Grandes across cultures and ethnic groups: multitrait-multimethod analyses of the big five in Spanish and English. J. Pers. Soc. Psychol. 1998;75(3):729. doi: 10.1037//0022-3514.75.3.729. [DOI] [PubMed] [Google Scholar]
  5. Berman A.H., Bergman H., Palmstierna T., Schlyter F. Evaluation of the drug use disorders identification test (DUDIT) in criminal justice and detoxification settings and in a Swedish population sample. Eur. Addict. Res. 2005;11(1):22–31. doi: 10.1159/000081413. [DOI] [PubMed] [Google Scholar]
  6. British Association for Counselling & Psychotherapy Looking After Our Students' Mental Health. 2014. http://www.bacp.co.uk/docs/pdf/14844_student_mental_health_briefing.pd (Retrieved from)
  7. Brown T.A., Barlow D.H. A proposal for a dimensional classification system based on the shared features of the DSM-IV anxiety and mood disorders: implications for assessment and treatment. Psychol. Assess. 2009;21(3):256–271. doi: 10.1037/a0016608. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Burns D.D. The perfectionistís script for self-defeat. Psychol. Today. 1980;14(6):34–52. [Google Scholar]
  9. Campbell-Sills L., Stein M.B. Psychometric analysis and refinement of the Connor-Davidson resilience scale (CD-RISC): validation of a 10-item measure of resilience. J. Trauma. Stress. 2007;20(6):1019–1028. doi: 10.1002/jts.20271. [DOI] [PubMed] [Google Scholar]
  10. Chan J.K., Farrer L.M., Gulliver A., Bennett K., Griffiths K.M. University Students' views on the perceived benefits and drawbacks of seeking help for mental health problems on the internet: a qualitative study. JMIR Human Factors. 2016;3(1) doi: 10.2196/humanfactors.4765. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Clark L.A., Watson D., Mineka S. Temperament, personality, and the mood and anxiety disorders. J. Abnorm. Psychol. 1994;103(1):103–116. [PubMed] [Google Scholar]
  12. Costa P.T., MacCrae R.R. Professional manual: Psychological Assessment Resources. 1992. Revised NEO personality inventory (NEO PI-R) and NEO five-factor inventory (NEO FFI) [Google Scholar]
  13. Czyz E.K., Horwitz A.G., Eisenberg D., Kramer A., King C.A. Self-reported barriers to professional help seeking among college students at elevated risk for suicide. J. Am. Coll. Heal. 2013;61(7) doi: 10.1080/07448481.2013.820731. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Dybek I., Bischof G., Grothues J., Reinhardt S., Meyer C., Hapke U., Rumpf H.-J. The reliability and validity of the alcohol use disorders identification test (AUDIT) in a German general practice population sample. J. Stud. Alcohol. 2006;67(3):473–481. doi: 10.15288/jsa.2006.67.473. [DOI] [PubMed] [Google Scholar]
  15. Ebert D.D., Zarski A.C. Friedrich-Alexander Universität; Erlangen-Nürnberg: 2014. Deutsche Version Der Connor–Davidson Resilience Scale-10 (Unpublished) [Google Scholar]
  16. Eisenberg D., Hunt J., Speer N., Zivin K. Mental health service utilization among college students in the United States. J. Nerv. Ment. Dis. 2011;199(5):301–308. doi: 10.1097/NMD.0b013e3182175123. [DOI] [PubMed] [Google Scholar]
  17. Eysenbach G. CONSORT-EHEALTH: improving and standardizing evaluation reports of web-based and mobile health interventions. J. Med. Internet Res. 2011;13(4) doi: 10.2196/jmir.1923. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Farrer L., Gulliver A., Chan J.K., Batterham P.J., Reynolds J., Calear A.…Griffiths K.M. Technology-based interventions for mental health in tertiary students: systematic review. J. Med. Internet Res. 2013;15(5) doi: 10.2196/jmir.2639. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Ferring D., Filipp S.-H. Messung des Selbstwertgefühls: Befunde zu Reliabilität, Validität und Stabilität der Rosenberg-Skala. Measurement of self-esteem: findings on reliability, validity, and stability of the Rosenberg-scaleDiagnostica. 1996;42(3):284–292. [Google Scholar]
  20. Fleming M.F., Barry K.L., MacDonald R. The alcohol use disorders identification test (AUDIT) in a college sample. Int. J. Addict. 1991;26(11):1173–1185. doi: 10.3109/10826089109062153. [DOI] [PubMed] [Google Scholar]
  21. Frost R.O., Marten P., Lahart C., Rosenblate R. The dimensions of perfectionism. Cogn. Ther. Res. 1990;14:449–468. [Google Scholar]
  22. Garner D.M., Olmstead M.P., Polivy J. Development and validation of a multidimensional eating disorder inventory for anorexia nervosa and bulimia. Int. J. Eat. Disord. 1983;2(2):15–34. [Google Scholar]
  23. Glindemann K.E., Geller E.S., Fortney J.N. Self-esteem and alcohol consumption: a study of college drinking behavior in a naturalistic setting. J. Alcohol Drug Educ. 1999;45(1):60–71. [Google Scholar]
  24. Hatcher R.L., Gillaspy J.A. Development and validation of a revised short version of the working alliance inventory. Psychother. Res. 2006;16:12–25. [Google Scholar]
  25. Hildebrand M. The psychometric properties of the drug use disorders identification test (DUDIT): a review of recent research. J. Subst. Abus. Treat. 2015;53:52–59. doi: 10.1016/j.jsat.2015.01.008. [DOI] [PubMed] [Google Scholar]
  26. Jenkinson C., Coulter A., Wright L. Short form 36 (SF36) health survey questionnaire: normative data for adults of working age. Br. Med. J. 1993;306(6890):1437–1440. doi: 10.1136/bmj.306.6890.1437. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. John O.P., Naumann L.P., Soto C.J. Paradigm shift to the integrative big five trait taxonomy. In: John O.P., Robins R.W., Pervin L.A., editors. Handbook of Personality: Theory and Research. Vol. 3. 2008. pp. 114–158. [Google Scholar]
  28. Kessler R.C., Berglund P., Demler O., Jin R., Merikangas K.R., Walters E.E. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the national comorbidity survey replication. Arch. Gen. Psychiatry. 2005;62(6):593–602. doi: 10.1001/archpsyc.62.6.593. [DOI] [PubMed] [Google Scholar]
  29. Kokotailo P.K., Egan J., Gangnon R., Brown D., Mundt M., Fleming M. Validity of the alcohol use disorders identification test in college students. Alcohol. Clin. Exp. Res. 2004;28(6):914–920. doi: 10.1097/01.alc.0000128239.87611.f5. [DOI] [PubMed] [Google Scholar]
  30. Kroenke K., Spitzer R.L., Williams J.B. The PHQ-9: validity of a brief depression severity measure. J. Gen. Intern. Med. 2001;16:606–613. doi: 10.1046/j.1525-1497.2001.016009606.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Löwe B., Decker O., Müller S., Brähler E., Schellberg D., Herzog W., Herzberg P.Y. Validation and standardization of the generalized anxiety disorder screener (GAD-7) in the general population. Med. Care. 2008;46:266–274. doi: 10.1097/MLR.0b013e318160d093. [DOI] [PubMed] [Google Scholar]
  32. Löwe B., Spitzer R.L., Gräfe K., Kroenke K., Quenter A., Zipfel S., Buchholz C., Witte S., Herzog W. Comparative validity of three screening questionnaires for DSM-IV depressive disorders and physicians' diagnoses. J. Affect. Disord. 2004;78(2):131–140. doi: 10.1016/s0165-0327(02)00237-9. [DOI] [PubMed] [Google Scholar]
  33. Lungu A., Sun M. Time for a change: college students' preference for technology-mediated versus face-to-face help for emotional distress. Telemed. J. E Health. 2016 doi: 10.1089/tmj.2015.0214. [DOI] [PubMed] [Google Scholar]
  34. Martín-Albo J., Núñez J.L., Navarro J.G., Grijalvo F. The Rosenberg self-esteem scale: translation and validation in university students. Span. J. Psychol. 2007;10(2):458–467. doi: 10.1017/s1138741600006727. [DOI] [PubMed] [Google Scholar]
  35. McGorry P., Goldstone S.D., Parker A.G., Rickwood D.J., Hickie I.B. Cultures for mental health Care of Young People: an Australian blueprint for reform. Lancet Psychiatry. 2014;1(7):559–568. doi: 10.1016/S2215-0366(14)00082-0. [DOI] [PubMed] [Google Scholar]
  36. Melnyk B.M., Amaya M., Szalacha L.A., Hoying J., Taylor T., Bowersox K. Feasibility, acceptability, and preliminary effects of the COPE online cognitive-behavioral skill-building program on mental health outcomes and academic performance in freshmen college students: a randomized controlled pilot study. J. Child Adolesc. Psychiatr. Nurs. 2015;28:147–154. doi: 10.1111/jcap.12119. [DOI] [PubMed] [Google Scholar]
  37. Moher D., Schulz K.F., Altman D.G., Group, C The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet. 2001;357(9263):1191–1194. [PubMed] [Google Scholar]
  38. Mowbray C.T., Mandiberg J.M., Stein C.H., Kopels S., Curlin C., Megivern D.…Lett R. Campus mental health services: recommendations for change. Am. J. Orthop. 2006;76(2):226–237. doi: 10.1037/0002-9432.76.2.226. [DOI] [PubMed] [Google Scholar]
  39. Musiat P., Conrod P., Treasure J., Tylee A., Williams C., Schmidt U. Targeted prevention of common mental health disorders in university students: randomised controlled trial of a Transdiagnostic trait-focused web-based intervention. PLoS One. 2014;9(4) doi: 10.1371/journal.pone.0093621. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Musiat P., Hoffmann L., Schmidt U. Personalised computerised feedback in E-mental health. J. Ment. Health. 2012;21(4):346–354. doi: 10.3109/09638237.2011.648347. [DOI] [PubMed] [Google Scholar]
  41. Musiat P., Tarrier N. Collateral outcomes in e-mental health: a systematic review of the evidence for added benefits of computerized cognitive behavior therapy interventions for mental health. Psychol. Med. 2014;44(15):3137–3150. doi: 10.1017/S0033291714000245. [DOI] [PubMed] [Google Scholar]
  42. Patel A., Rendu A., Moran P., Leese M., Mann A., Knapp M. A comparison of two methods of collecting economic data in primary care. Fam. Pract. 2005;22(3):323–327. doi: 10.1093/fampra/cmi027. [DOI] [PubMed] [Google Scholar]
  43. Rosenberg M. Princeton University Press; Princeton, NJ: 1965. Society and the Adolescent Self-Image. [Google Scholar]
  44. Räsänen P., Lappalainen P., Muotka J., Tolvanen A., Lappalainen R. An online guided ACT intervention for enhancing the psychological wellbeing of university students: a randomized controlled clinical trial. Behav. Res. Ther. 2016;78:30–42. doi: 10.1016/j.brat.2016.01.001. [DOI] [PubMed] [Google Scholar]
  45. Royal College of Psychiatrists . Royal College of Psychiatrists; London: 2011. The Mental Health of Students in Higher Education. [Google Scholar]
  46. Sánchez-Ortiz V., Munro C., Stahl D., House J., Startup H., Treasure J., Schmidt U. A randomized controlled trial of internet-based cognitive-behavioural therapy for bulimia nervosa or related disorders in a student population. Psychol. Med. 2011;41(02):407–417. doi: 10.1017/S0033291710000711. [DOI] [PubMed] [Google Scholar]
  47. Saunders J.B., Aasland O.G., Babor T.F., De La Fuente J.R., Grant M. Development of the alcohol use disorders identification test (AUDIT): WHO collaborative project on early detection of persons with harmful alcohol consumption-II. Addiction. 1993;88(6):791–804. doi: 10.1111/j.1360-0443.1993.tb02093.x. [DOI] [PubMed] [Google Scholar]
  48. Schmitt D.P., Allik J., McCrae R.R., Benet-Martínez V. The geographic distribution of big five personality traits patterns and profiles of human self-description across 56 nations. J. Cross-Cult. Psychol. 2007;38(2):173–212. [Google Scholar]
  49. Shafran R., Mansell W. Perfectionism and psychopathology: a review of research and treatment. Clin. Psychol. Rev. 2001;21(6):879–906. doi: 10.1016/s0272-7358(00)00072-6. [DOI] [PubMed] [Google Scholar]
  50. Skevington S.M., Lotfy M., O'Connell K.a. The World Health Organization's WHOQOL-BREF quality of life assessment: psychometric properties and results of the international field trial. A report from the WHOQOL group. Qual. Life Res. 2004;13:299–310. doi: 10.1023/B:QURE.0000018486.91360.00. [DOI] [PubMed] [Google Scholar]
  51. Spitzer R.L., Kroenke K., Williams J.B.W., Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch. Intern. Med. 2006;166:1092–1097. doi: 10.1001/archinte.166.10.1092. [DOI] [PubMed] [Google Scholar]
  52. Stice E., Fisher M., Martinez E. Eating disorder diagnostic scale: additional evidence of reliability and validity. Psychol. Assess. 2004;16(1):60–71. doi: 10.1037/1040-3590.16.1.60. [DOI] [PubMed] [Google Scholar]
  53. Stice E., Telch C.F., Rizvi S.L. Development and validation of the eating disorder diagnostic scale: a brief self-report measure of anorexia, bulimia, and binge-eating disorder. Psychol. Assess. 2000;12(2):123–131. doi: 10.1037//1040-3590.12.2.123. [DOI] [PubMed] [Google Scholar]
  54. Stöber J. The frost multidimensional perfectionism scale: more perfect with four (instead of six) dimensions. Personal. Individ. Differ. 1998;24:481–491. [Google Scholar]
  55. The Whoqol Group Development of the World Health Organization WHOQOL-BREF quality of life assessment. The WHOQOL Group. Psychiatr. Med. 1998;28:551–558. doi: 10.1017/s0033291798006667. [DOI] [PubMed] [Google Scholar]
  56. Titov N., Dear B.F., McMillan D., Anderson T., Zou J., Sunderland M. Psychometric comparison of the PHQ-9 and BDI-II for measuring response during treatment of depression. Cogn. Behav. Ther. 2011;40(2):126–136. doi: 10.1080/16506073.2010.550059. [DOI] [PubMed] [Google Scholar]
  57. Verger P., Guagliardo V., Gilbert F., Rouillon F., Kovess-Masfety V. Psychiatric disorders in students in six French universities: 12-month prevalence, comorbidity, impairment and help-seeking. Soc. Psychiatry Psychiatr. Epidemiol. 2010;45(2):189–199. doi: 10.1007/s00127-009-0055-z. [DOI] [PubMed] [Google Scholar]
  58. Vidourek R.A., King K.A., Nabors L.A., Merianos A.L. Students' benefits and barriers to mental health help-seeking. Health Psychol. Behav. Med. 2014;2(1):1009–1022. doi: 10.1080/21642850.2014.963586. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Wilmers F., Munder T., Leonhart R., Herzog T., Plassmann R., Barth J. Die deutschsprachige Version des Working Alliance Inventory - short revised (WAI-SR) - Ein schulenübergreifendes, ökonomisches und empirisch validiertes Instrument zur Erfassung der therapeutischen Allianz. Klinische Diagnostik und Evaluation. 2008;1(3):343–358. [Google Scholar]
  60. Woicik P.A., Stewart S.H., Pihl R.O., Conrod P.J. The substance use risk profile scale: a scale measuring traits linked to reinforcement-specific substance use profiles. Addict. Behav. 2009;34(12):1042–1055. doi: 10.1016/j.addbeh.2009.07.001. [DOI] [PubMed] [Google Scholar]
  61. World Medical Association World Medical Association Declaration of Helsinki. Ethical principles for medical research involving human subjects. Bull. World Health Organ. 2001;79(4):373. [PMC free article] [PubMed] [Google Scholar]

Articles from Internet Interventions are provided here courtesy of Elsevier

RESOURCES