Skip to main content
NCBI home page
Internet Interventions logoLink to Internet Interventions
. 2018 Nov 23;15:28–34. doi: 10.1016/j.invent.2018.11.002

Guided internet-based transdiagnostic intervention for Indonesian university students with symptoms of anxiety and depression: A pilot study protocol

Metta Rahmadiana a,b,⁎, Eirini Karyotaki a,b, Jan Passchier a,b, Pim Cuijpers a,b, Wouter van Ballegooijen a,b,c, Supra Wimbarti d, Heleen Riper a,b,c
PMCID: PMC6290259  PMID: 30568878

Abstract

Background

There is an increasing prevalence of mental health problems among university student populations. In Indonesia, treatment rates are low among university students because of the high cost of treatment, stigma and the limited availability of mental care services. Internet-based interventions have been found to be effective in treating anxiety and depression in Western countries. However, little is known about the effectiveness of Internet-based interventions in low- and middle-income countries (LMICs) such as Indonesia. In this paper, we describe how we culturally adapted an Internet-based intervention to meet the needs of Indonesian students with depression and anxiety based on the theoretical framework of Barrera et al. (2013).

Method

We culturally adapted a Western Internet-based student intervention for depression and anxiety. This intervention consists of 8 guided online sessions. 50 students from Universitas Gadjah Mada Yogyakarta who have mild to moderate depression or anxiety as assessed with the Patient Health Questionnaire-9 (PHQ-9) and Generalized Anxiety Disorder-7 (GAD-7), will participate in a feasibility study in which we will test participants' satisfaction, system usability and uptake. Secondary outcomes include assessment of participants' depression, anxiety and quality of life. Outcomes will be measured pre and post intervention.

Discussion

The present paper presents the protocol of a pilot study aimed at assessing the feasibility and acceptability of a culturally adapted intervention for Indonesian university students with depression and anxiety. The results from the feasibility study will further guide the development of the intervention and may inform the protocol of a future randomized controlled trial (RCT) examining the effectiveness of the Internet-based intervention.

Keywords: Anxiety, Cultural adaptation, Depression, Internet-based intervention, Low-and middle- income countries, University students

Highlights

  • •

    The first Internet-based intervention to target university student population in low-and middle-income countries with depression and anxiety.

  • •

    A systematic pilot study protocol to assess the feasibility and acceptability of a culturally adapted Internet-based intervention.

  • •

    This study has involved end users in the cultural adaptation of the Internet-based intervention.

1. Introduction

There is a growing awareness of university students' mental health as indicated by various studies worldwide (Raveau, 2013; Ran et al., 2016; Kraft, 2011). Starting university life represents a significant developmental milestone for students. This transitional period may be accompanied by intense psychological pressure as students seek to cope with various changes, such as study commitments, financial shortage and living independently for the first time in their lives (Lin, 2010). These challenges can be overwhelming for many college students and can trigger the onset of mental disorders such as depression and anxiety (Bland et al., 2012; Mahmoud et al., 2012), or lead to unhealthy behaviour such as substance abuse (Kitzrow, 2003), eating disorder (Stoliker and Lafreniere, 2015; Bland et al., 2012) and feelings of loneliness (Stoliker and Lafreniere, 2015). If early-onset of a common mental disorder such as anxiety and depression is not identified or adequately managed, it may have important negative implications for college students, such as poor academic achievement and study dropout (Bojuwoye, 2002).

Depression and anxiety have higher prevalence rates in high-income countries than low- and middle-income countries (World Health Organization, 2017), including Indonesia. This may be a result of under-reporting due to differences in perceived stigma, sociodemographic status, and in the way of expressing psychological distress between Western and non-Western countries (Heim et al., 2017). Results from the World Health Organization (WHO) have shown that 3.7% (approximately 9,200,000 persons) and 3.3% (approximately 8,000,000 persons) of the Indonesian general population (approximately 260,000,000) suffer from depressive disorder and anxiety disorders respectively (WHO, 2017). Epidemiological research on upper-middle income countries showed that roughly 22% of university students experience a mental health disorder (Auerbach et al., 2016). Although Indonesia is an upper-middle income country, the prevalence rates of mental disorders might differ from other upper-middle income countries. To the best of our knowledge, there are no studies reporting prevalence rates of typical problems of Indonesian university students such as family and financial pressure that may trigger anxiety and depression.

Despite the high prevalence and adverse consequences, depression and anxiety often go untreated in university students globally (Blanco et al., 2008; Eisenberg et al., 2012; Vogel and Wade, 2009). According to a large-scale cross-national epidemiological study (Auerbach et al., 2016), only a small minority (16.4%) of university students receive minimal adequate treatment. This treatment rate is even lower in low- and middle-income countries (Becker and Kleinman, 2013; Patel et al., 2010). More specifically in Indonesia, university students have access to counseling services provided mainly by the Faculty of Psychology. However, in many cases students do not utilize these services. Treatment underutilization might also be a result of potential fear of stigmatization (Vogel and Wade, 2009) as well as the cost factor for university students in Indonesia.

Internet-based interventions may be an effective way to address the aforementioned reasons for treatment underutilization (Arjadi et al., 2016; Escoffery et al., 2005). In Indonesia, most university students have access to the Internet and are familiar with its usage (Asosiasi Penyedia Jasa Internet Indonesia, 2014). Therefore, the use of Internet may be promising in delivering evidence-based interventions for university students in Indonesia. The most important benefit of this intervention is the potential to reduce substantially the cost of treatment, which is an important barrier for the Indonesian university students (Donker et al., 2018). In this way, treatment availability will be increased as currently there are limited psychological services in Indonesia. Moreover, Internet-based interventions can ensure confidentiality to circumvent stigmatization (Wright, 2018).

Several studies have supported the effectiveness of Internet-based interventions in treating depression and anxiety in Western high-income countries (HICs) (Kaltenthaler et al., 2008; Richards et al., 2016; Jakobsen et al., 2017), also among students (Davies et al., 2014). However, in low-and middle-income countries (LIMCs) research on the effectiveness of internet-based interventions is scarce (Arjadi et al., 2015). As far as we know, there is only one ongoing study examining the effect of a web-based intervention for treating depression in the general population in Indonesia, with results that are not yet known (Arjadi et al., 2016). Based on previous literature, psychological interventions developed in western countries are at least as effective in non-western countries for adults with depression (Cuijpers et al., 2018). Moreover, with at least as effective results were found from a meta-analysis examining whether ethnicity moderates psychotherapy outcomes (Ünlϋ Ince, 2014). We thus, decided to explore whether Internet-based interventions may be an effective way to provide mental health support for Indonesian university students. To this end, we culturally adapted a Western Internet-based intervention to incorporate and fit the needs and preferences for an existing student population in Indonesia based on the theoretical framework of Barrera and colleagues (2013). Results from a recent meta-analysis indicated that culturally adapted CBT interventions are more efficacious in reducing depressive and anxiety symptoms when compared with non-culturally adapted interventions (Shehadeh et al., 2016; Vally and Maggott, 2015; Hall et al., 2016).

Barrera and colleagues theoretical framework resides in five phase: 1) information gathering, 2) preliminary adaptation design, 3) preliminary testing, 4) adaptation refinement and 5) cultural adaptation trial. We have already performed the first two phase (information gathering and preliminary adaptation design) based on the theoretical framework. Therefore, the aim of the paper is to present a protocol of the current third phase which entails a pilot study that will examine the feasibility and acceptability of a culturally adapted Internet-based intervention, named I-AiMentalWELLness (Saya menuju mental sehat) for Indonesian university students with depression and anxiety.

2. Methods

2.1. The first two phase of Barrera's theoretical framework

In phase 1, for the purpose this study, we used an Internet-based intervention, namely I-Care Prevent, which was originally developed in Germany and Switzerland for the general population. We chose this intervention due to pragmatic reasons. First, there are no Indonesian Internet-based Interventions. Second, the development of a new intervention would require more time and financial investments. Third, this intervention was suitable for college students as it was translated into English and Dutch and adapted to the needs of Dutch and international university students in the Netherlands (Bolinski et al., 2018).

We subsequently translated the English version of the intervention into Bahasa Indonesia and culturally adapted it to fit the needs of university students in Indonesia. A team of translators, local pilot study supervisors, intervention designers and mass communication experts was set up in Indonesia (as adjunct to the team based in the Netherlands). The Indonesian translation team checked the translation (based on a forward-backward translation procedure) and licensed psychologists checked the translations to ensure that the translated psychological terms conveyed the appropriate meaning. The Indonesian design team was responsible for the development of the intervention. The mass communication team prepared advertisements for the purpose of the study campaign. The pilot study supervisors are clinical psychologists and university lecturers and their primary role is to moderate the progress of the study in Indonesia (e.g., resolving technical issues, coordinating research tasks and providing feedback about the clinical content of study).

In phase 2, we gathered information related to the cultural appropriateness of the intervention's content, and we conducted two series of online focus groups with 2–3 students in each group with a total of twenty students from Gadjah Mada University in Yogyakarta, Indonesia. Focus group participants suggested to change the Western examples and testimonials of the intervention to examples of local university students. Furthermore, they suggested omitting or changing all parts related to unmarried cohabitation, sexual activity, alcohol, anti-depressants, sleep medication and inpatient care to meet the needs, values and current living circumstances of the Indonesian population. We decided to follow the suggestion of the focus group because the vast majority of the Indonesian university students are Muslim. Thus, discussing about such topics is not appropriate in the Islamic religion. We have made all necessary changes to the intervention accordingly. We ensured that these changes did not influence the main therapeutic components of the internet-based CBT intervention (e.g., cognitive restructuring) but they solely reflected the local context. See also Fig. 1 for the flow chart of the study process.

Fig. 1.

Fig. 1

Open in a new tab

Three-phase structure of the study based on the theoretical framework of Barrera et al. (2013).

2.2. Phase 3-ongoing

The project is based on a collaboration between the Universitas Gadjah Mada Yogyakarta, Indonesia and the Vrije Universiteit Amsterdam in The Netherlands.

2.3. Study design

This study is a single-group pre-test (t0) and post-test (t1) design where t1 will be administered at the end of the intervention (7 weeks after t0).

2.4. Participants: recruitment and procedure

We aim to recruit 50 students from Universitas Gadjah Mada, Yogyakarta Indonesia. A study website https://i-aimentalwellness.com containing useful information about the study and registration form has been created and advertised through posters, chat groups and social media. Participants who register to express their interest in this study will be assessed on their depression and anxiety level for screening purposes. Subsequently, participants will also be asked to fill in the EuroQoL 5D (EQ-5D-5 L) to assess their level of quality of life and this will be used as their baseline. Participants who meet the inclusion criteria and eligible to join the study will receive an online information sheet to ensure they understand all aspects (the purpose, duration, risk and benefits) of their participation in this feasibility study and an online informed consent which has to be filled in within maximum 2 weeks of time.

2.5. Inclusion criteria

Participants will be included if they meet these following criteria: 1) students at Universitas Gadjah Mada Yogyakarta, Indonesia having access to a computer and internet, 2) being 18 years of age or older, 3) speak and read fluently Bahasa Indonesia, 4) experience mild to moderate depression (as defining by scoring above the cut-off score of 4 on the Patient Health Questionnaire (PHQ-9) and anxiety symptoms (as defining by scoring above the cut off score of 4 on the Generalized Anxiety Disorder Scale – 7 items (GAD-7).

2.6. Exclusion criteria

Participants who fulfil any of the following criteria will be excluded from participation in this study: (1) students with severe depression (as defining by scoring above the cut-off score of 14 on the PHQ-9) and anxiety symptoms (as defining by scoring above the cut-off score of 14 on the GAD-7 scale), (2) Currently receiving psychological treatment for depression and/or anxiety in the past 12 months, (3) having no internet connection (e.g. no broadband Internet or something comparable).

2.7. The intervention: I-AiMentalWELLness

The first session focuses on goal setting and behavioral activation in the context of basic psychological needs and important values. The second session aims at solving problems. The third session contains psychoeducation on depression and anxiety and the fourth session focuses on cognitive restructuring. Moreover, the intervention has two disorder specific sessions (5th & 6th online sessions) that can be chosen to either solve problems concerning depressive symptoms or exposure to anxiety provoking situations. The seventh session of the intervention focuses on planning for the future by reflecting on how to attain goals and how to implement intentions for behavioral changes. Finally, an eight-booster session is given four weeks after the completion of the 7th session and aims at refreshing the content learned in the previous sessions. All online sessions include text, exercises, testimonials, audio, and illustrated pictures, which help participants to integrate the content of the sessions into their daily lives. Each of the sessions lasts approximately 60 min.

The I-AiMentalWELLness intervention is accompanied by limited feedback provided by e-Coaches (Clinical Psychology master students and licensed psychologists). An estimated 20 min will be spent per feedback and ECoaches are encouraged not to spend more than 30 min per participant and feedback. Over the course of the whole intervention, ECoaches are expected to spend 2, 5 h per participant. The principal investigator (MR) will provide training for the e-Coaches consisting of the theoretical background of the intervention and practical exercises such as simulations. Moreover, E-Coaches will receive a manual written in the Indonesian language containing technical and content-related explanations per session.

2.8. Outcome assessments

2.8.1. Primary outcomes

2.8.1.1. Feasibility

The primary outcome of the feasibility is based on the participants' acceptability and satisfaction, usability and uptake.

2.8.1.2. Acceptability

The acceptability outcome is based on satisfaction regarding the Internet intervention and is defined as a Client Satisfaction Questionnaire (CSQ-8) average score of 20 or more (Lubis et al., 2013; Attkisson and Zwick, 1982). The CSQ-8 has 8 items that can be scored on a 4-point Likert scale with a total score ranging from 8 to 32, where higher scores indicate a greater level of satisfaction. The CSQ-8 has good internal consistency (Lubis et al., 2013; Attkisson and Zwick, 1982) and will be administered at post-test assessment (t1).

Moreover, we will select 10 participants (taking into account the heterogeneity in age, gender, social and ethnic background) and invite them to partake in semi-structured interviews to gain a better understanding of participants' satisfaction with the use of I-AiMentalWELLness. Qualitative interviews permit the gathering of in-depth data that cannot be easily obtained through standardized questionnaires (Edwards and Holland, 2013) and allow the interviewer to better understand the perspective of the participants. For the semi-structured interviews we will apply the set of generic questions as developed by Reena et al. (2014) to capture user experiences in a new web-based intervention. The original questions have been modified to fit the purpose of our study. Questions included are: 1) How did you experience using the intervention through the Internet? 2) Were there any enjoyable or less enjoyable parts of the program? 3) Was the Internet-based intervention easy to use? Why or why not?

2.8.1.3. Usability

We will use the Indonesian translated version of the System Usability Scale (SUS) by Sharfina and Santoso (2016) to evaluate the usability of the current intervention. The SUS is a 10-item Likert scale from 1 (strongly disagree) to 5 (strongly agree) (Lewis et al., 2015). Total scores of the SUS range from 0 to 100 where higher scores indicate better usability. Acceptability usability can be defined as an average score of 70 or more (Bangor et al., 2008). Final scores from the SUS measurement range from 0 to 100. The SUS will be administered at post-treatment (t1).

2.8.1.4. Uptake and log data

Uptake refers to the degree to which the participant experiences the content of the intervention by using or not using (Lillevoll et al., 2014) the I-AiMentalWELLness intervention. We will collect and report adherence with respect to logons, time spent on site, and number of modules attempted (Christensen et al., 2009). Based on previous literature, approximately 65% of participants complete all intervention modules when the online intervention is provided with some form of therapeutic guidance (Van Ballegooijen et al., 2014). We would consider a more conservative completion rate of 35% as we anticipate a lower adherence rate based on the novelty of the Internet-based interventions in Indonesia compared to those observed in similar trials on transdiagnostic Internet-based intervention for college students in Western countries (61% observed in the study of Day et al. (2013) and 43% observed in the study of Mullin et al. (2015).

2.9. Secondary outcomes

Secondary outcomes are quality of life based on the EuroQoL-5 Dimension-5 Level scale (EQ-5D-5 L) (Purba et al., 2017), depression measured by the PHQ-9 (Kroenke et al., 2001; Smarr and Keefer, 2011) and anxiety measured by the GAD-7 (Löwe et al., 2008; Garcia-Campanyo et al., 2010). We will also assess the following additional outcome measures: 1) ECoach evaluation and 2) Feedback per session and cultural appropriateness of the adapted Internet-based intervention. The participants' will be assessed based on the evaluation of their assigned ECoach with questions derived from the study of Knaevelsrud and Maercker (2007) as follow: 1) How did you experience being treated through the Internet instead of face-to-face interaction? 2) To what extent was the ECoach supportive? 3) How long did it take the ECoach to give feedback? 4) How communicative was the ECoach? 5) How comprehensive was the feedback from the ECoach? Moreover, participants will be assessed for their feedback of each session with the following open-ended question: 1) Were the goals of this Internet session clearly defined? 2) What did you learn from this session? 3) Was the content from this online session clear and easy to understand? 4) Was this online session easy to navigate? 5) Was the length of this session appropriate to the topic? 6) How were the illustrated picture in this online session? 7) Do you understand the language, idiom, and words used in this module? and 8) Do you think the case examples given in this module are appropriate for Indonesian university students?

2.9.1. EuroQoL-5 dimension-5 level scale (EQ-5D-5 L)

The Indonesian version of the EQ-5D-5 L will be used. The EQ-5D-5 L examines health related quality of life based on 5 dimensions: mobility, self-care, usual activities, pain/discomfort and anxiety or depression. This instrument has been adjusted according to the Indonesian terms related to severity: no problems, slight problems, moderate problems, severe problems, and incapacity/extreme problems (Purba et al., 2017; Herdman et al., 2011; Devlin and Krabbe, 2013). The EQ-5D-5 L shows perfect agreement between the two tests (Gwet's AC: 0,85–0,99 and percentage agreement: 90–99%) which indicates representation of the Indonesian general population. The EQ-5D-5 L will be administered at (t0) and post-test assessment (t1) (Purba et al., 2018).

2.9.2. Patient health questionnaire-9 (PHQ-9)

We will use the Indonesian version of the Patient Health Questionnaire-9 (PHQ-9), which is a self-rated scale measuring depressive symptoms. The PHQ-9 consists of 9 items based on the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV). It is used to assess depression severity, and scores are interpreted as: 1–4 = no depression, 5–9 = mild depression, 10–14 = moderate depression, 15–19 = moderately severe depression and 20–27 = severe depression. The PHQ-9 is considered as a reliable outcome measure for depression (Kroenke et al., 2001; Smarr and Keefer, 2011). However, to the extent of our knowledge, the PHD-9 has not been validated in Indonesia. The PHQ-9 will be administered at the t0 and t1.

2.9.3. Generalized anxiety disorder -7 (GAD-7)

We will use the Indonesian version of the Generalized anxiety disorder GAD-7 which is a 7-item self-rated scale designed to assess symptoms of anxiety. The items are scored as: 0 = not at all, 1 = several days, 2 = more than half the days, 3 = nearly every day. The total scores of GAD-7 are interpreted as: minimal (0–4), mild (5–9), moderate (10–14) and severe (14–20). The GAD-7 is reported to be reliable to assess anxiety (Löwe et al., 2008; Garcia-Campanyo et al., 2010). However, to the extent of our knowledge, the PHD-9 has not been validated in Indonesia. The GAD-7 will be administered at the pre- t0 and t1.

2.9.4. Demographic variables

During baseline assessment, participants will provide the following demographic information: 1) age, 2) gender, 3) socioeconomic status, 4) marital status, 5) education and 6) ethnicity.

2.9.5. Study drop-out rates

Drop-out from study rates are defined as the number of participants who fail to complete the post-treatment assessment.

3. Sample power calculation

The purpose of this pilot study includes assessment of feasibility and acceptability of an intervention (Hertzog, 2008). There is no gold standard for calculating the sample size of a pilot study. Some studies recommend at least 12 participants (Julious, 2005), 30 or greater (Lancaster et al., 2004) or 10–40 participants (Hertzog, 2008 and Johanson and Brooks, 2010). After careful consideration of the existing literature of this topic, we decided to follow the suggestion by Arnold et al., 2009 which reported that approximately 50 participants will be enough for testing the feasibility of an intervention. We have performed a post hoc power calculation to describe statistically the power of our chosen sample. Given α = 0.05 and the slope of a linier bivariate regression set at 0.43, we found that if we include 50 participants we can achieve a power of 0.95 in one tailed hypothesis. We therefore estimate that a sample size of 50 participants would be sufficient in order to assess the feasibility and acceptability of the I-AiMentalWELLness intervention.

3.1. Analysis

3.1.1. Feasibility parameters

Quantitative and qualitative analysis will be used to assess the participants' use of the intervention, their satisfaction with the intervention (CSQ-8), system usability (SUS), adherence, e-Coach evaluation, feedback related to the sessions and the cultural appropriateness of the adapted Internet-based intervention.

Descriptive statistics will be used to summarise the participants' satisfaction, system usability and level of adherence. All responses given to qualitative measures (e.g., qualitative interview) will be recorded and analyzed using thematic analysis (Clarke and Braun, 2017). This method identifies and categorizes themes captured in the study. Thematic analysis in the present study starts by making a transcription of verbal data. It is then coded and organized into meaningful themes. In the next phase, refinements are made by identifying whether each theme consists of sub-themes. In the final stage of this analysis, interpretative analyses are made.

3.1.2. Other study parameters

Changes in symptom severity as measured by the PHQ-9, GAD-7 and EQ-5D will be analyzed using two-tailed paired t-tests using a significance level α = 0.05.

4. Discussion

In the present paper, we described the procedures of a cultural adaptation process based on the theoretical framework of Barrera et al. (2013), which includes: 1) information gathering 2) preliminary adaptation design and 3) preliminary adaptation test. The main aim of this study is to examine the feasibility of an Internet-based culturally adapted intervention for treating anxiety and depression in Indonesian university students. To the extent of our knowledge, there are no studies examining the effectiveness of Internet-based interventions in treating anxiety and depression in university students in Indonesia.

The strength of this study is systematic cultural adaptation based on the theoretical framework of Barrera et al. (2013), which includes immediate involvement of end users. The results of the online focus groups are encouraging for exploring whether Internet-based interventions can be an acceptable way to provide mental health support for university students with depression and anxiety in Indonesia.

Several challenges could be expected in this study. Many people in Indonesia do not have access to a fast Internet. In the original version of the I-Care intervention, videos are provided, which takes a large amount of bandwidth. Therefore, we tried to overcome this challenge by changing videos into illustrated pictures that will not require as much bandwidth. Illustrated pictures did not constitute a contentious issue among university students who participated in a series of online focus groups. However, given that videos can be more vivid and compelling, the adherence of participants in this study might be affected. Further, the study may have low uptake as a result of academic demands, such as study workload, assignment deadlines, exams, and other academic duties. Such duties and demands may affect the amount of time students have for participating in the online intervention. Finally, students will be self-referred to this trial. It is possible that only highly motivated students will be included in our study and thus, the sample will not be representative of all university students with depression and anxiety.

The next steps are further adjustment of the Intervention based on the feasibility study results and subsequently, a randomized controlled trial testing the effectiveness. This Internet-based intervention has the potential to increase accessibility of effective treatment for students with mental health problems around the globe. This will supplement existing services in low resourced countries, such as Indonesia.

Competing interests

The authors declare that they have no competing interests.

Funding

This work was supported by the Indonesia Endowment Fund for Education (Lembaga Pengelola Dana Pendidikan), Ministry of Finance, Republic of Indonesia which provided a PhD scholarship and research funding for the first author.

Author contributions

HR contributed in the design of the study. MR drafted the manuscript under the supervision of EK. All authors contributed in revising the manuscript. All authors read and approved the final manuscript.

Trial status

Medical and Health Research Ethics Committee (MHREC) Faculty of Medicine Gadjah Mada University – DR Sardjito General Hospital Ref: KE/FK/0098/EC/2018, registered on February 9, 2018.

Acknowledgements

The author would like to thank all the people involved in the first two phase of this study. The university students for their feedback on the cultural appropriateness of the intervention's content. Angelina, Dzul, Felix, Galuh, Kris, Lastri, Mahar, Nopi, Nurmey, Olive, Satya, Siska and Wulan for the development of the intervention.

References

  1. Arjadi R., Nauta M.H., Chowdhary N., Bockting C.L.H. Global Mental Health. Vol. 2. 2015. A systematic review of online interventions for mental health in low and middle income countries: a neglected field; p. e12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Arjadi R., Nauta M.H., Scholte W.F., Hollon S.D., Chowdhary N., Suryani A.O., Bockting C.L.H. Guided act and feel Indonesia (GAF-ID) - internet-based behavioral activation intervention for depression in Indonesia: study protocol for a randomized controlled trial. Trials. 2016;17(1):455. doi: 10.1186/s13063-016-1577-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Arnold D.M., Burns K.E.A., Adhikari N.K.J., Kho M.E., Meade M.O., Cook D.J. The design and interpretation of pilot trials in clinical research in critical care. Crit. Care Med. 2009;37(Suppl. 1):69–74. doi: 10.1097/CCM.0b013e3181920e33. [DOI] [PubMed] [Google Scholar]
  4. Asosiasi Penyelenggara Jasa Internet Indonesia - APJII . Apjii; 2014. Profil Pengguna Internet Indonesia 2014.https://www.apjii.or.id/survei2016 Retrieved from: [Google Scholar]
  5. Attkisson C.C., Zwick R. The client satisfaction questionnaire. Eval. Program Plann. 1982;5(3):233–237. doi: 10.1016/0149-7189(82)90074-x. [DOI] [PubMed] [Google Scholar]
  6. Auerbach R.P., Alonso J., Axinn W.G., Cuijpers P., Ebert D.D., Green J.G.…Bruffaerts R. Mental disorders among college students in the World Health Organization world mental health surveys. Psychol. Med. 2016;46(14):2955–2970. doi: 10.1017/S0033291716001665. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bangor A., Kortum P.T., Miller J.T. An empirical evaluation of the system usability scale. Int. J. Hum. Comput. Interact. 2008;24(6):574–594. [Google Scholar]
  8. Barrera M., Gastro F.G., Strycker L.A., Toobert D. Cultural adaptations of behavioral health interventions: a progress report. J. Consult. Clin. Psychol. 2013;81(2):196–205. doi: 10.1037/a0027085. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Becker A.E., Kleinman A. Mental health and the global agenda. N. Engl. J. Med. 2013;369(1):66–73. doi: 10.1056/NEJMra1110827. [DOI] [PubMed] [Google Scholar]
  10. Blanco C., Okuda M., Wright C., Hasin D.S., Grant B.F., Liu S., Olfson M. Mental health of college students and their non–college-attending peers. Arch. Gen. Psychiatry. 2008;65(12):1429–1437. doi: 10.1001/archpsyc.65.12.1429. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Bland H., Melton B., Welle P., Bigham L. Stress tolerance: new challenges for millennial college students. Coll. Stud. J. 2012;46(Issue 2):362–376. http://www.freepatentsonline.com/article/College-Student-Journal/297135954.html Jun2012, p362 14p. Retrieved from: [Google Scholar]
  12. Bojuwoye O. Stressful experiences of first year students of selected universities in South Africa. Couns. Psychol. Q. 2002;15(3):277–290. [Google Scholar]
  13. Bolinski F., Kleiboer A., Karyotaki E., Bosmans J.E., Zarski A.C., Weisel K.K.…Riper H. Effectiveness of a transdiagnostic individually tailored internet-based and mobile-supported intervention for the indicated prevention of depression and anxiety (ICare prevent) in Dutch college students: study protocol for a randomised controlled trial. Trials. 2018;19(1):1–13. doi: 10.1186/s13063-018-2477-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Christensen H., Griffiths K.M., Farrer L. Adherence in internet interventions for anxiety and depression. J. Med. Internet Res. 2009;11(2):1–16. doi: 10.2196/jmir.1194. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Clarke V., Braun V. Thematic analysis. J. Posit. Psychol. 2017;12(3):297–298. [Google Scholar]
  16. Cuijpers P., Karyotaki E., Reijnders M., Purgato M., Barbui C. Psychotherapies for depression in low- and middle-income countries: a meta-analysis. World Psychiatry. 2018;17(1):90–101. doi: 10.1002/wps.20493. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Davies E.B., Morriss R., Glazebrook C. Computer-delivered and web-based interventions to improve depression, anxiety, and psychological well-being of university students: a systematic review and meta-analysis. J. Med. Internet Res. 2014;16(5):1–22. doi: 10.2196/jmir.3142. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Day V., McGrath P.J., Wojtowicz M. Internet-based guided self-help for university students with anxiety, depression and stress: a randomized controlled clinical trial. Behav. Res. Ther. 2013;51(7):344–351. doi: 10.1016/j.brat.2013.03.003. [DOI] [PubMed] [Google Scholar]
  19. Devlin N.J., Krabbe P.F.M. The development of new research methods for the valuation of EQ-5D-5L. Eur. J. Health Econ. 2013;14(S1):1–3. doi: 10.1007/s10198-013-0502-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Donker T., Blankers M., Hedman E., Ljótsson B., Petrie K., Christensen H. 2018. Economic Evaluations of Internet Interventions for Mental Health: A Systematic Review, (2015) pp. 3357–3376. [DOI] [PubMed] [Google Scholar]
  21. Edwards R., Holland J. What is qualitative interviewing? “What is?”. Res. Methods Ser. 2013;7 [Google Scholar]
  22. Eisenberg D., Hunt J., Speer N. Help seeking for mental health on college campuses: review of evidence and next steps for research and practice. Harv. Rev. Psychiatry. 2012;20(4):222–232. doi: 10.3109/10673229.2012.712839. [DOI] [PubMed] [Google Scholar]
  23. Escoffery C., Miner K.R., Adame D.D., Butler S., McCormick L., Mendell E. Internet use for health information among college students. J. Am. Coll. Heal. 2005;53(4):183–188. doi: 10.3200/JACH.53.4.183-188. [DOI] [PubMed] [Google Scholar]
  24. Garcia-Campanyo J., E. Z., R. M.A., P. A., F. O., P-P. M., L.-G. V. Cultural adaptation into spanish of the generalized anxiety disorder scale-7 (GAD-7) scale. Health Qual. Life Outcomes. 2010;8(8):S538. doi: 10.1186/1477-7525-8-8. http://0-ovidsp.ovid.com.lib.exeter.ac.uk/ovidweb.cgi?T=JS&PAGE=reference&D=emed12&NEWS=N&AN=70256382 Retrieved from: [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Hall G.C.N., Huang E.R., Marti C.N. ScienceDirect a meta-analysis of cultural adaptations of psychological interventions. Behav. Ther. 2016 doi: 10.1016/j.beth.2016.09.005. [DOI] [PubMed] [Google Scholar]
  26. Heim E., Wegmann I., Maercker A. Cultural values and the prevalence of mental disorders in 25 countries: a secondary data analysis. Soc. Sci. Med. 2017;189:96–104. doi: 10.1016/j.socscimed.2017.07.024. [DOI] [PubMed] [Google Scholar]
  27. Herdman M., Gudex C., Lloyd A., Janssen M., Kind P., Parkin D.…Badia X. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L) Qual. Life Res. 2011;20(10):1727–1736. doi: 10.1007/s11136-011-9903-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Hertzog Considerations in determining sample size for pilot studies. Research in Nursing & Health. 2008;31(4):341–354. doi: 10.1002/nur.20247. [DOI] [PubMed] [Google Scholar]
  29. Ince B.U. 2014. Recruiting and Treating Depression in Ethnic Minorities: The Effects of Online and Offline Psychotherapy. [Google Scholar]
  30. Jakobsen H., Andersson G., Havik O.E., Nordgreen T. Guided internet-based cognitive behavioral therapy for mild and moderate depression: a benchmarking study. Internet Interv. 2017;7:1–8. doi: 10.1016/j.invent.2016.11.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Johanson G.A., Brooks G.P. Initial scale development: Sample size for pilot studies. Educ. Psychol. Meas. 2010;70(3):394–400. [Google Scholar]
  32. Julious S.A. Sample size of 12 per group rule of thumb for a pilot study. Pharm. Stat. 2005;4(4):287–291. [Google Scholar]
  33. Kaltenthaler E., Sutcliffe P., Parry G., Beverley C., Rees A., Ferriter M. The acceptability to patients of computerized cognitive behaviour therapy for depression: a systematic review. Psychol. Med. 2008;38(11):1521–1530. doi: 10.1017/S0033291707002607. [DOI] [PubMed] [Google Scholar]
  34. Kitzrow M.A. The mental health needs of today ’ s college students: challenges and recommendations. J. Stud. Aff. Res. Pract. 2003;41(1):165–180. [Google Scholar]
  35. Knaevelsrud C., Maercker A. Internet-based treatment for PTSD reduces distress and facilitates the development of a strong therapeutic alliance: a randomized controlled clinical trial. BMC Psychiatry. 2007;7:1–10. doi: 10.1186/1471-244X-7-13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Kraft D.P. One hundred years of college mental health. J. Am. Coll. Heal. 2011;59(6):477–481. doi: 10.1080/07448481.2011.569964. [DOI] [PubMed] [Google Scholar]
  37. Kroenke K., Spitzer R.L., Williams J.B.W. The phq-9. J. Gen. Intern. Med. 2001;16(9):606–613. doi: 10.1046/j.1525-1497.2001.016009606.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Lancaster G.A., Dodd S., Williamson P.R. Design and analysis of pilot studies: recommendations for good practice. J. Eval. Clin. Pract. 2004:307–312. doi: 10.1111/j..2002.384.doc.x. [DOI] [PubMed] [Google Scholar]
  39. Lewis J.R., Utesch B.S., Maher D.E. Measuring perceived usability: the SUS, UMUX-LITE, and AltUsability. Int. J. Hum. Comput. Interact. 2015;31(8):496–505. [Google Scholar]
  40. Lillevoll K.R., Vangberg H.C.B., Griffiths K.M., Waterloo K., Eisemann M.R. Uptake and adherence of a self-directed internet-based mental health intervention with tailored e-mail reminders in senior high schools in Norway. BMC Psychiatry. 2014;14:14. doi: 10.1186/1471-244X-14-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Lin Y.-N. Taiwanese university students' perceptions of university life. Couns. Psychol. Q. 2010;23(2):189–202. [Google Scholar]
  42. Löwe B., Decker O., Müller S., Brähler E., Schellberg D., Herzog W., Herzberg P.Y. Validation and standardization of the generalized anxiety disorder screener (GAD-7) in the general population. Med. Care. 2008;46(3):266–274. doi: 10.1097/MLR.0b013e318160d093. [DOI] [PubMed] [Google Scholar]
  43. Lubis D.U., Jaya E.S., Arjadi R., Hanum L., Astri K., Putri M.D.D. Preliminary study on the effectiveness of short group cognitive behavioral therapy (GCBT) on Indonesian older adults. PLoS One. 2013;8(2) doi: 10.1371/journal.pone.0057198. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Mahmoud J.S.R., Staten R.T., Hall L.A., Lennie T.A. The relationship among young adult college students' depression, anxiety, stress, demographics, life satisfaction, and coping styles. Issues Ment. Health Nurs. 2012;33(3):149–156. doi: 10.3109/01612840.2011.632708. [DOI] [PubMed] [Google Scholar]
  45. Mullin A., Dear B.F., Karin E., Wootton B.M., Staples L.G., Johnston L., Titov N. The UniWellbeing course: a randomised controlled trial of a transdiagnostic internet-delivered cognitive behavioural therapy (CBT) programme for university students with symptoms of anxiety and depression. Internet Interv. 2015;2(2):128–136. [Google Scholar]
  46. Patel V., Maj M., Flisher A.J., De Silva M.J., Koschorke M., Prince M.…Richardson G. Reducing the treatment gap for mental disorders: a WPA survey. World Psychiatry. 2010;9(3):169–176. doi: 10.1002/j.2051-5545.2010.tb00305.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Purba F.D., Hunfeld J.A.M., Iskandarsyah A., Fitriana T.S., Sadarjoen S.S., Ramos-Goñi J.M.…Busschbach J.J.V. The Indonesian EQ-5D-5L value set. PharmacoEconomics. 2017;35(11):1153–1165. doi: 10.1007/s40273-017-0538-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Purba F.D., Hunfeld J.A.M., Iskandarsyah A., Fitriana T.S., Sadarjoen S.S., Passchier J., Busschbach J.J.V. Quality of life of the Indonesian general population: test-retest reliability and population norms of the EQ-5D-5L and WHOQOL-BREF. PLoS One. 2018;13(5):1–20. doi: 10.1371/journal.pone.0197098. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Ran M.S., Mendez A.J., Leng L.L., Bansil B., Reyes N., Cordero G.…Tang M. Predictors of mental health among college students in Guam: implications for counseling. J. Couns. Dev. 2016;94(3):344–355. [Google Scholar]
  50. Raveau H. Wayne State University; 2013. Ethnicity, Culture, and Mental Health among College Students of Middle Eastern Heritage. [Google Scholar]
  51. Reena D., Carpenter Christine, P. John, Sallysingh Exploring the experience of using a web-based cardiac rehabilitation programme in a primary care angina population: a qualitative study. Int. J. Ther. Rehabil. 2014;21(9):434–440. [Google Scholar]
  52. Richards D., Murphy T., Viganó N., Timulak L., Doherty G., Sharry J., Hayes C. Acceptability, satisfaction and perceived efficacy of “space from depression” an internet-delivered treatment for depression. Internet Interv. 2016;5:12–22. doi: 10.1016/j.invent.2016.06.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Sharfina Z., Santoso H.B. 2016 International Conference on Advanced Computer Science and Information Systems, ICACSIS 2016. 2016. An Indonesian adaptation of the System Usability Scale (SUS) pp. 145–148. [Google Scholar]
  54. Shehadeh M.H., Heim E., Chowdhary N., Maercker A., Shehadeh M.H. Vol. 3. 2016. Cultural Adaptation of Minimally Guided Interventions for Common Mental Disorders?: A Systematic Review and Meta-Analysis Corresponding Author; pp. 1–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Smarr K.L., Keefer A.L. Measures of depression and depressive symptoms: Beck depression inventory-II (BDI-II), Center for Epidemiologic Studies Depression Scale (CES-D), geriatric depression scale (GDS), hospital anxiety and depression scale (HADS), and patient health Questionna. Arthritis Care Res. 2011;63(S11):S454–S466. doi: 10.1002/acr.20556. [DOI] [PubMed] [Google Scholar]
  56. Stoliker B., Lafreniere K. The influence of perceived stress, loneliness, and learning burnout on university Students' educational experience. Coll. Stud. J. 2015;49(1):146–160. [Google Scholar]
  57. Vally Z., Maggott C. 2015. Evaluating the Outcome of Cultural Adaptations of Cognitive-Behavioural Therapy for Adult Depression: a Meta-Analysis of Treatment Studies in Developing Countries; pp. 293–304. [Google Scholar]
  58. Van Ballegooijen W., Cuijpers P., Van Straten A., Karyotaki E., Andersson G., Smit J.H., Riper H. Adherence to internet-based and face-to-face cognitive behavioural therapy for depression: a meta-analysis. PLoS One. 2014;9(7) doi: 10.1371/journal.pone.0100674. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Vogel D.L., Wade N. Stigma and help seeking. Psychol. 2009;22(1):20–23. [Google Scholar]
  60. World Health Organization . World Health Organization; Geneva: 2017. Depression and Other Common Mental Disorders: Global Health Estimates; pp. 1–24. (https://doi.org/CCBY-NC-SA3.0IGO) [Google Scholar]
  61. Wright K.B. 2018. Researching Internet-Based Populations: Advantages and Disadvantages of Online Survey Research, Online Questionnaire Authoring Software Packages, and Web Survey Services Advantages of Online Survey Research; pp. 1–26. [Google Scholar]

Articles from Internet Interventions are provided here courtesy of Elsevier

RESOURCES