Abstract
Background
There have been no previous studies of the variables that predict adherence to online depression and anxiety intervention programs among adolescents. However, research of traditionally delivered intervention programs for a variety of health conditions in adolescence suggests that health knowledge, type and level of symptomatology, race, socioeconomic status, treatment setting, and support may predict adherence.
Objective
The aim was to compare adherence rates and identify the predictors of adherence to a cognitive behavior therapy website in two adolescent samples that were offered the program in different settings and under different conditions of support.
Methods
The first adolescent sample consisted of 1000 school students who completed the MoodGYM program in a classroom setting over five weeks as part of a randomized controlled trial. The second sample consisted of 7207 adolescents who accessed the MoodGYM program spontaneously and directly through the open access URL. All users completed a brief survey before the start of the program that measured background characteristics, depression history, symptoms of depression and anxiety, and dysfunctional thinking.
Results
Adolescents in the school-based sample completed significantly more online exercises (mean = 9.38, SD = 6.84) than adolescents in the open access community sample (mean = 3.10, SD = 3.85; t 1088.62 = −28.39, P < .001). A multiple linear regression revealed that school-based setting (P < .001) and female gender (P < .001) were predictive of greater adherence, as were living in a rural area (P < .001) and lower pre-test anxiety (P = .04) scores for the school-based sample and higher pre-test depression scores (P = .01) for the community sample. A history of depression (P = .33) and pre-test warpy thoughts scores (P = .35) were not predictive of adherence in the school-based or community sample.
Conclusion
Adherence is greater in monitored settings, and the predictors of adherence differ between settings. Understanding these differences may improve program effectiveness and efficiency.
Keywords: Adolescent, prevention, anxiety disorders, mood disorders, patient non-adherence, Internet
Introduction
A number of early intervention and prevention programs have been developed to address the prevalence of anxiety and depressive disorders in children and adolescents [1,2]. However, despite the recognized importance of adherence as a factor in determining the success of a range of mental health programs [3,4], very little research has been undertaken to measure levels of adherence or to determine the factors that increase it. This paper aims to investigate factors and contexts that increase adherence to an evidence-based website offering cognitive behavior therapy (CBT) for depression and anxiety in adolescents.
Knowledge about the predictors of adherence to intervention programs in adolescent populations is largely based on research targeting physical health or smoking behaviors. Kyngäs [5], for instance, found that the adolescent’s motivation, perceived threat to social well-being, and fear of the development or expression of acute problems predicted compliance to rheumatoid arthritis treatment. Studies of smoking cessation programs, on the other hand, have identified high baseline smoking rate, increased knowledge of the effects of tobacco, and white race as factors associated with improved adherence. Gender and age were not associated with outcomes [6,7]. Poor attendance for child and adolescent psychotherapy sessions and premature termination of treatment have also been found to be associated with low socioeconomic status, ethnic minority status, parental psychopathology, and severity of child psychopathology [8].
Program adherence for medical conditions among adults and children has also been found to depend on the type of treatment and program factors such as reminders, rewards, and monitoring [4]. System or program factors are also recognized as important in fostering adherence. Although not investigated in adolescents, evidence from adult studies indicates the importance of context—the setting, the amount of personal contact, and the support provided—as potentially important in maintaining adherence in both Web and non-Web environments [9]. Other predictors of adult adherence to Web-based programs have included disease severity, treatment length, chronicity, younger age, and female gender [10].
Previous research on program adherence has been hampered by imperfect data collection methods, using retrospective self-reports (which rely on good memory), and absence of social desirability bias [11,12]. Behavioral data (comprising records of use during actual exposure to a program) are a preferred method to measure adherence. Internet-based interventions programmed to record user activity are a useful platform to investigate completion rates.
The current study was designed to investigate adherence, with the level of activity on the website serving as the primary measure of this construct. A specific objective of the study was to compare the module and exercise completion rates of adolescents for a mental health website (MoodGYM) in two distinct settings. The first sample constitutes a group of users who completed MoodGYM within a classroom, where teachers provided encouragement, support, and guidance in the use of the program as part of the curriculum. Teachers did not receive specific reports on student progress through the program. The second sample consisted of community adolescent users who spontaneously logged on to the open access site. In both settings, the website program was fully automated and participant progress within the program was self-directed, with the exception that the school-based participants were directed to a different module each week. We predicted that adherence rates would be greater for the monitored classroom setting compared to the community setting. The current study also aimed to identify the demographic, setting, and symptom-related factors that contribute to adherence in the two samples.
Methods
Sample
School Sample
The school sample comprised 1000 students who completed the MoodGYM program as part of the YouthMood Project during 2006 and 2007. The YouthMood Project is a randomized controlled trial evaluating the effectiveness of the MoodGYM program among adolescents aged 13 to 17 years. Thirty schools from across Australia took part in the project. During a designated class period of 50 minutes, once a week for five weeks, students were instructed to log on to the program and individually undertake the self-directed program. Using the teacher manual, the classroom teacher provided information, support, and referral advice, if required, and maintained classroom order. No psychological intervention was provided. Approximately 60% (597/1000) of the students participating in the YouthMood Project were female, 19% (193/1000) reported living in a rural area, and 29% (287/1000) had previously been depressed.
Community Sample
The community sample consisted of 7207 adolescents from Australia who registered on the MoodGYM public site between January 2006 and November 2007. Adolescent users were identified as users who selected the age category “19 or under” when registering on the website. Approximately 72% (5223/7207) of the adolescent community users were female, with 66% (4734/7207) reporting a previous history of depression and 19% (1396/7207) indicating that they lived in a rural area.
Intervention Details
The MoodGYM program is an interactive, Internet-based CBT program designed to prevent and decrease symptoms of anxiety and depression. MoodGYM aims to modify dysfunctional thoughts and beliefs, improve self-esteem, and teach important life skills such as problem solving and relaxation. The MoodGYM program consists of five interactive modules, each of which includes information, animated demonstrations, quizzes, and “homework” exercises. A user’s answers to the quizzes and exercises in the MoodGYM program are recorded in his or her own personal MoodGYM Workbook, which he or she can access any time throughout the program. Upon registering, all users consent to their data being collected and stored in a secure database for analysis. The anonymity of users is protected through the use of a pseudonym.
Measures
Users completed a number of measures prior to beginning the MoodGYM program, including the Goldberg Depression and Anxiety Scales [13] and the Warpy Thoughts Scale [14]. Demographic information was also collected, including the user’s age, gender, whether they lived in an urban or rural/remote region, and whether they had a history of marked depression (yes/no).
Anxiety and depressive symptoms were measured using the Goldberg Depression and Anxiety Scales [13]. Each of these scales consists of nine items that are rated with a yes/no response. Total scale scores are calculated by summing the number of “yes” responses. Scores for each scale range from 0 to 9, with higher scores indicative of greater anxiety or depression.
The 42-item Warpy Thoughts Scale [14] was used to measure dysfunctional thinking. Users responded to the items on a 5-point Likert scale ranging from “strongly agree” to “strongly disagree.” A total scale score is calculated by obtaining the mean of the 42 items. Scores can range from 1 to 5, with higher scores reflecting more dysfunctional thinking.
Adherence Measures
The number of modules and exercises completed on the MoodGYM website were the primary outcome measures in the current study. The MoodGYM program automatically tracks and records a user’s activity on the website, including the number of log-ons and the number of modules and exercises started and completed. While the school-based users were encouraged to complete all modules of the program, the completion of exercises within the modules was optional, making this variable a good measure of adherence to, and level of engagement with, the treatment program. Since there were 28 exercises, the number of exercises completed could range from 0 to 28, while the number of modules completed could range from 0 to 5.
Analysis
Exploratory descriptive analyses and an independent samples t test were conducted to compare the adherence rates of the two adolescent samples. A hierarchical multiple linear regression was conducted on the combined sample to explore the predictors of adherence. In this model, the setting was entered first, followed by gender and location in the second step. Depression history was added in the third step, followed by the pre-test symptom variables of anxiety, depression, and warpy thoughts in the fourth step. In the final step, the interaction between setting and the other predictor variables was added to explore differences in adherence predictors between settings. All analyses were performed using SPSS v15 (SPSS Inc, Chicago, IL, USA).
Results
Descriptive Statistics
The mean depression pre-test score for school-based participants was 2.62 (SD = 2.42), while mean pre-test anxiety and warpy thoughts scores were 2.51 (SD = 2.44) and 2.58 (SD = .65), respectively. The mean depression pre-test score for community users was 5.46 (SD = 2.42), while mean pre-test anxiety and warpy thoughts scores were 5.50 (SD = 2.59) and 3.16 (SD = .71), respectively.
Adherence Rates
Figure 1 and Figure 2 present the module and exercise completion rates of participants in the community and school-based samples. Both module and exercise completion rates were significantly higher in the monitored school-based sample than in the community sample. In the community sample, a high proportion of adolescents completed no or only one module (6413/7206 participants, 89%) and less than five exercises (6160/7207 participants, 85%), while in the school-based sample over half of the participants completed three or more modules (483/874 participants, 55%) and nine or more exercises (526/1000 participants, 53%).
Figure 1.
Number of modules completed
Figure 2.
Number of exercises completed
Adolescents in the school-based sample completed significantly more exercises (mean = 9.38, SD = 6.84) than adolescents in the community sample (mean = 3.10, SD = 3.85; t 1088.62 = −28.39, P < .001). Mean exercise completion rates also varied between schools in the school-based sample (mean = 10.87, SD = 4.71, range = 2.47-17.00).
Figure 3 presents the average number of exercises completed by participants as a function of the number of modules completed. This graph demonstrates that the number of exercises completed by participants in each sample is approximately equivalent once two or more modules were completed. More exercises are completed by the school-based sample when no or only one module is completed.
Figure 3.
Average number of exercises completed, by number of modules completed
Predictors of Adherence
A hierarchical multiple linear regression was conducted on the combined adolescent sample to explore the predictors of adherence to the MoodGYM program and to investigate interactions of these predictors with setting. Table 1 presents the results of this analysis.
Table 1.
Predictors of adherence for the combined adolescent sample
| Step | Variable | Coefficient | SE | P |
| 1 | Setting (school vs community) | 6.31 | .16 | < .001 |
| 2 | Setting (school vs community) | 6.39 | .16 | < .001 |
| Gender (female vs male) | .69 | .12 | < .001 | |
| Location (rural vs urban) | .43 | .14 | .01 | |
| 3 | Setting (school vs community) | 6.39 | .16 | < .001 |
| Gender (female vs male) | .69 | .12 | < .001 | |
| Location (rural vs urban) | .43 | .14 | .01 | |
| Depression history | −.01 | .01 | .67 | |
| 4 | Setting (school vs community) | 6.55 | .17 | < .001 |
| Gender (female vs male) | .61 | .12 | < .001 | |
| Location (rural vs urban) | .42 | .14 | .01 | |
| Depression history | −.01 | .01 | .62 | |
| Depression pre-test score | −.01 | .03 | .72 | |
| Anxiety pre-test score | .07 | .03 | .01 | |
| Warpy thoughts pre-test score | −.03 | .09 | .71 | |
| 5 | Setting (school vs community) | 6.99 | .71 | < .001 |
| Gender (female vs male) | .65 | .13 | < .001 | |
| Location (rural vs urban) | .15 | .15 | .31 | |
| Depression history | −.01 | .01 | .56 | |
| Depression pre-test score | .02 | .03 | .58 | |
| Anxiety pre-test score | .05 | .03 | .10 | |
| Warpy thoughts pre-test score | −.02 | .10 | .85 | |
| Setting × Gender | −.16 | .34 | .63 | |
| Setting × Location | 1.82 | .39 | < .001 | |
| Setting × Depression history | .34 | .35 | .33 | |
| Setting × Depression pre-test score | −.23 | .09 | .01 | |
| Setting × Anxiety pre-test score | .20 | .09 | .04 | |
| Setting × Warpy thoughts pre-test score | −.26 | .28 | .35 |
In the final model, setting and gender were significant predictors of adherence, as were the interactions between setting and urban verus rural location, pre-test depression, and pre-test anxiety. A school-based setting and being female predicted better adherence. For the school-based sample, living in a rural area or having lower pre-test anxiety scores predicted better adherence, while higher pre-test depression scores predicted better adherence for the community sample.
Discussion
The study first aimed to compare the adherence rates of a monitored and non-monitored sample of adolescent users of the MoodGYM program. As predicted, adolescents in the school-based sample completed significantly more exercises than did community users. In terms of program completion, there was an almost 10-fold difference between the two samples, highlighting the success of the monitored setting in increasing compliance. These findings support previous research that has also found that the setting and monitoring can improve adherence by increasing motivation and interest [4,9].
An unexpected finding from the current study was that the exercise completion rates of the two samples were equivalent once two or more modules were completed. This suggests that if community users are sufficiently self-motivated to complete more than one module, they continue to engage with the MoodGYM program as strongly as participants in the monitored school-based setting.
The second aim of the current study was to identify the predictors of adherence to the MoodGYM program for the two samples of adolescent users. Participating in the intervention in a school-based setting was found to be a significant predictor of adherence, in line with previous research that has found monitored settings to yield greater adherence [4,9]. The positive relationship between female gender and adherence is also consistent with past research that has found adolescent females to be more likely to seek mental health assistance [15].
The two other predictors of adherence among the adolescent school-based sample were living in a rural area and having a lower level of anxiety at pre-test. These findings may reflect the gap in adolescent mental health services in rural areas or greater compliance and motivation among rural students and teachers. The basis for the lower rate of exercise completion among adolescents with higher anxiety scores is not clear. However, anxiety may reduce school attendance and hence reduce the opportunity to engage in the program, or it may have more direct effects on program engagement. For example, individuals may effectively avoid the program by surfing other Internet sites during the class time, they may engage in other avoidance activities, or they may fail to complete all exercises within the time specified because of rumination or worry [16]. Future research could explicitly measure these variables as a means of exploring and explaining these relationships further.
A higher pre-test depression score was identified as an additional predictor of adherence for adolescents in the community sample. Adolescents with lower pre-test depression scores may have a lower motivation to complete the program as it has less relevance or benefit to them than it does for a highly symptomatic person. Another potential reason for this finding could be that people with fewer depressive symptoms at pre-test may not require as much of the program to feel “well” and thus drop out after attaining what they need [17].
There are some limitations to the current study that should be highlighted. First, the program reviewed is Internet based and therefore the specific predictors of adherence in this study may not generalize to face-to-face interventions. The predictive ability of the setting, demographic, and symptom-related factors explored in the current study was relatively small (accounting for only 21% of the variance), suggesting that other factors may also be predictive of adherence to the MoodGYM program. Future research, therefore, should focus on measuring other potential predictors of adherence such as motivation to change, stage of change, computer savviness, perceived efficacy of computer-based programs, level of support, and personal expectations.
The variability in the exercise completion rates between schools in the school-based sample should also be investigated further. School-level predictors might include the gender distribution of students (co-educational vs single-sex), type of school (private vs public), and the socioeconomic status of the school. In addition to this, future studies should also explicitly measure the extent to which users implement the techniques taught in the MoodGYM program as a means of identifying if people actively engage with the material on the website or are simply passive browsers. This would enable a user’s engagement with the program to be more closely assessed.
We also acknowledge that there are many measured and unmeasured differences between our direct open access users and school students. It is clear that the open access adolescents had higher levels of anxiety and depression than the sample of adolescents in school classrooms. The motivation and context in which the program is likely to be undertaken is also very different. Open access users are likely to be seeking immediate help for mental health problems, while the average attendee at school may have no need for the website. These factors are likely to impact adherence rates and to confound differences between the adherence rates of the two samples described here. Nevertheless, our analyses, while identifying the importance of the difference between settings, control for this factor and provide additional information about the impact of other variables on adherence in both settings.
The clarification and identification of further predictors of adherence would enable appropriate strategies to be developed that could help improve adherence to automated Internet applications [18]. Such strategies could include tailoring the program to better address the individual needs of the adolescent (effectively reducing the amount of irrelevant material) and therapy preparation sessions aimed at a user’s stage of change or knowledge of computer-based programs. For instance, if a user is unsure of the likely effectiveness of an Internet intervention, then an education program providing details and results of the program’s success may help to increase the user’s motivation and drive to complete the program. In addition to this, automated emails to prompt log-on to the program may improve adherence. Such hypothesized strategies may enable more effective programs to be developed and implemented in the health and mental health domains.
Acknowledgments
The YouthMood Project was funded by the Vincent Fairfax Family Foundation. Helen Christensen is a recipient of an NHMRC Research Fellowship 366781.
Abbreviations
- CBT
cognitive behavior therapy
Footnotes
Helen Christensen and Kathleen Griffiths are authors of the MoodGYM program.
References
- 1.Merry S, McDowell H, Hetrick S, Bir J, Muller N. Psychological and/or educational interventions for the prevention of depression in children and adolescents. Cochrane Database Syst Rev. 2004;(1):CD003380. doi: 10.1002/14651858.CD003380.pub2. [DOI] [PubMed] [Google Scholar]
- 2.Neil Alison L, Christensen Helen. Australian school-based prevention and early intervention programs for anxiety and depression: a systematic review. Med J Aust. 2007 Mar 19;186(6):305–8. doi: 10.5694/j.1326-5377.2007.tb00906.x. http://www.mja.com.au/public/issues/186_06_190307/nei10799_fm.html.nei10799_fm [DOI] [PubMed] [Google Scholar]
- 3.Davis M J, Addis M E. Predictors of attrition from behavioral medicine treatments. Ann Behav Med. 1999;21(4):339–49. doi: 10.1007/BF02895967. [DOI] [PubMed] [Google Scholar]
- 4.van Dulmen Sandra, Sluijs Emmy, van Dijk Liset, de Ridder Denise, Heerdink Rob, Bensing Jozien. Patient adherence to medical treatment: a review of reviews. BMC Health Serv Res. 2007;7(1):55. doi: 10.1186/1472-6963-7-55. http://www.biomedcentral.com/1472-6963/7/55.1472-6963-7-55 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Kyngäs Helvi. Motivation as a crucial predictor of good compliance in adolescents with rheumatoid arthritis. Int J Nurs Pract. 2002 Dec;8(6):336–41. doi: 10.1046/j.1440-172X.2002.00389.x.389 [DOI] [PubMed] [Google Scholar]
- 6.Siddiqui O, Flay B R, Hu F B. Factors affecting attrition in a longitudinal smoking prevention study. Prev Med. 1996;25(5):554–60. doi: 10.1006/pmed.1996.0089.S0091-7435(96)90089-4 [DOI] [PubMed] [Google Scholar]
- 7.Turner Lindsey R, Mermelstein Robin, Berbaum Michael L, Veldhuis Cindy B. School-based smoking cessation programs for adolescents: what predicts attendance? Nicotine Tob Res. 2004 Jun;6(3):559–68. doi: 10.1080/14622200410001696475.712JXNFAJL21WB93 [DOI] [PubMed] [Google Scholar]
- 8.Nock Matthew K, Ferriter Caitlin. Parent management of attendance and adherence in child and adolescent therapy: a conceptual and empirical review. Clin Child Fam Psychol Rev. 2005 Jun;8(2):149–66. doi: 10.1007/s10567-005-4753-0. [DOI] [PubMed] [Google Scholar]
- 9.Meichenbaum D, Turk DC. Facilitating Treatment Adherence: A Practitioner's Guidebook. New York: Plenum Press; 1987. [Google Scholar]
- 10.Batterham PJ, Neil AL, Bennett K, Griffiths KM, Christensen H. Predictors of adherence among community users of a cognitive behaviour therapy website. Patient Prefer Adher. 2008;2:97–105. [PMC free article] [PubMed] [Google Scholar]
- 11.Celio Angela A, Winzelberg Andrew J, Dev Parvati, Taylor C Barr. Improving compliance in on-line, structured self-help programs: evaluation of an eating disorder prevention program. J Psychiatr Pract. 2002 Jan;8(1):14–20. doi: 10.1097/00131746-200201000-00003.00131746-200201000-00003 [DOI] [PubMed] [Google Scholar]
- 12.Greening L, Stoppelbein L, Reeves CB. A model for promoting adolescents’ adherence to treatment for type 1 diabetes mellitus. Child Health Care. 2006;35(3):247–267. doi: 10.1207/s15326888chc3503_4. [DOI] [Google Scholar]
- 13.Goldberg D, Bridges K, Duncan-Jones P, Grayson D. Detecting anxiety and depression in general medical settings. BMJ. 1988 Oct 8;297(6653):897–9. doi: 10.1136/bmj.297.6653.897. http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pubmed&pubmedid=3140969. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Parslow Ruth A, Christensen Helen, Griffiths Kathy M, Groves Chloe. The warpy thoughts scale: a new 20-item instrument to measure dysfunctional attitudes. Cogn Behav Ther. 2006;35(2):106–16. doi: 10.1080/16506070500372279.U84662T071438506 [DOI] [PubMed] [Google Scholar]
- 15.Sheffield JK, Fiorenza E, Sofronoff K. Adolescents’ willingness to seek psychological help: Promoting and preventing factors. J Youth Adolesc. 2004;33(6):495–507. doi: 10.1023/B:JOYO.0000048064.31128.c6.. [DOI] [Google Scholar]
- 16.Barrett PM, Pahl KM. School-based intervention: Examining a universal approach to anxiety management. Aust J Guid Counc. 2006;16(1):55–75. doi: 10.1375/ajgc.16.1.55. [DOI] [Google Scholar]
- 17.Christensen Helen, Mackinnon Andrew. The law of attrition revisited. J Med Internet Res. 2006;8(3):e20; author reply e21. doi: 10.2196/jmir.8.3.e20. http://www.jmir.org/2006/3/e20/v8i3e20 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Eysenbach Gunther. The law of attrition. J Med Internet Res. 2005;7(1):e11. doi: 10.2196/jmir.7.1.e11. http://www.jmir.org/2005/1/e11/v7e11 [DOI] [PMC free article] [PubMed] [Google Scholar]