Abstract
In recent years, the necessity of providing online interventions for adolescents, as an alternative to face-to-face interventions, has become apparent due to several barriers some adolescents face in accessing treatment. This need has become more critical with the coronavirus disease 2019 (COVID-19) global pandemic impacting the delivery of psychotherapy and limiting accessibility of face-to-face therapy. Whilst it has been established that face-to-face psychotherapy for adolescents with personality disorder can be effective in reducing the impact these complex mental illnesses have on functioning, online interventions for adolescents are rare, and to our knowledge there are no empirically validated online interventions for personality disorder. The development of novel online interventions are therefore necessary. To inform the development of online interventions for adolescents with personality disorder or symptoms of emerging personality disorder, a two-phase rapid review was conducted. Phase one consisted of a search and examination of existing online mental health programs for adolescents with symptoms of personality disorder, to understand how to best use online platforms. Phase two consisted of a rapid review of empirical literature examining online interventions for adolescents experiencing symptoms of personality disorder to identify characteristics that promote efficacy. There were no online programs specific to personality disorder in adolescence. However, 32 online mental health programs and 41 published empirical studies were included for analysis. Common intervention characteristics included timeframes of one to two months, regular confidential therapist contact, simple interactive online components and modules, and the inclusion of homework or workbook activities to practice new skills. There is an urgent need for online interventions targeting personality dysfunction in adolescence. Several characteristics of effective online interventions for adolescents were identified. These characteristics can help inform the development and implementation of novel online treatments to prevent and reduce the burden and impact of personality disorder, or symptoms of emerging personality disorder, in adolescents. This has implications for the COVID-19 pandemic when access to effective online interventions has become more urgent.
Key words: Adolescents, personality disorder, online intervention, COVID-19
Introduction
Personality disorders are complex mental illnesses and, as such, have a pervasive impact on functioning particularly if they are not treated early (Chanen et al., 2009; Grenyer, Ng, Townsend, & Rao, 2017). Studies have recently shown that of all mental health presentations to hospital, 20.5% of emergency department and 26.6% of inpatients had a primary diagnosis of personality disorder (Lewis, Fanaian, Kotze, & Grenyer 2019). Data supports a prevalence of personality disorders in the community at 6.5% (Jackson & Burgess, 2000). Further, an estimated 40-50% of psychiatric patients have a personality disorder with 22% of psychiatric outpatients having the diagnosis of Borderline Personality Disorder (BPD; Korzekwa, Dell, Links, Thabane, & Webb, 2008). Presentation in emergency wards is common for those with personality disorder, particularly BPD, which often involves selfharm and suicidality.
Psychotherapy is the first line of treatment for personality disorders, with no recognised pharmacological interventions (National Health and Medical Research Centre [NHMRC], 2012). Intervention using targeted, evidencebased psychotherapy may prevent symptom escalation and reduce cost of hospitalisations. Empirically-based psychotherapy has been shown to significantly reduce direct and indirect health care costs and burden of disease, particularly when intervention occurs early in the symptom trajectory (Chanen et al., 2009). One study found a mean cost saving for treating BPD with evidence-based therapy of USD $2,987.82 per patient per year (Meuldijk, McCarthy, Bourke, & Grenyer, 2017).
As personality disorders tend to emerge in adolescence, early intervention efforts seek to target this developmental stage to prevent symptoms, such as suicidality, self-harm and emotional dysregulation, from worsening (Grenyer et al., 2017). While there is ongoing debate regarding the diagnosis of personality disorder, in particular BPD, during adolescence (Miller, Muehlenkamp, & Jacobson, 2008), current evidence suggests personality disorders can be diagnosed during these years and individuals respond to prevention and intervention for personality disorder (Chanen & McCutcheon, 2013; Johnson et al., 2000; Kaess, Brunner, & Chanen, 2014). Further, access to treatment early in the symptom trajectory also has potential to decrease incidents of deliberate selfharm and suicidal behaviour that may otherwise impact emergency services (Grenyer et al., 2017). As such, adolescence is a key period to target (Chanen et al., 2009), but interventions developed specifically with adolescents in mind are rare (Chanen et al., 2009).
Evidence supports efficacy of face-to-face psychotherapy for personality disorder (Levy, McMain, Bateman, & Clouthier, 2018), however results are subject to publication bias with therapy not being effective for all individuals (Cristea et al., 2017). Therefore, there is a need to address how therapy can be accessible and helpful for more individuals with personality disorder. For some individuals, location and resources limits accessibility and given the increasing use of digital technology in young people’s lives, online therapy could be used to overcome these barriers (Clark, Kuosmanen, & Barry, 2015) The international coronavirus disease 2019 (COVID-19) pandemic has led to an even more urgent need to deliver online psychotherapy as it has placed strain on the ability to provide face-to-face therapy to individuals with personality disorder (Álvaro et al., 2020). People with existing mental health conditions are particularly vulnerable to anxiety, feelings of uncertainty and social isolation that COVID-19 mitigation measures have entailed (Sher, 2020; Yao, Chen, & Xu, 2020), motivating a push for greater access to telehealth and internet therapy (Zhou et al., 2020). High treatment seeking and severity of illness means that improving access particularly within this period is crucial. Problems may be further exacerbated for young people with mental health issues, with social isolation, loss of structured work/school environments and changes to service delivery thought to increase risk (Power, Hughes, Cotter, & Cannon, 2020).
Regardless of COVID-19 related restrictions, access problems often preclude young people from beginning their treatment journey, particularly if they reside in rural or remote locations where face-to-face therapy is not available (Grenyer et al., 2017). Therapist-assisted online therapy has been proposed as a means of bridging access gaps and providing treatment when face-to-face contact is not feasible (Johansson et al., 2012). Internet delivery means therapy can be accessed regardless of restrictions, including those imposed by COVID-19 mitigation measures. Online therapy is also not restricted by geographic location or office hours and is less time-consuming, making it more cost effective, and allowing therapists to treat more clients (Hedman et al., 2011; Vigerland et al., 2016). Furthermore, it has been proposed that this type of therapy is well suited to adolescents, due to their willingness to engage with online platforms (Pretorius et al., 2009). Research on internet-delivered psychotherapy for other mental health conditions (e.g., depression, anxiety, bulimia nervosa, suicidal ideation) has demonstrated that adolescents find online therapies acceptable and appealing, and appreciate the confidentiality they afford (Christensen, Batterham, & O’Dea, 2014; Pretorius et al., 2009; Richardson, Stallard, & Velleman, 2010). Further, adolescents may be more competent and confident using online platforms than older adults (Richardson et al., 2010) and online therapy may address concerns around stigma associated with seeking help (Vigerland et al., 2016).
Internet-delivered therapies often take a guided-selfhelp approach, with clients gradually completing activities and modules and the therapist providing continuous support (e.g., Johansson et al., 2012). In addition, this approach commonly includes therapist contact, usually via webcam or secure online messaging system (e.g., Johansson et al., 2012). A review on the acceptability of online mental health programs for adolescents and young people found that completion rates were higher when there was concurrent therapist contact alongside the online component and young people reported therapist assistance allowed an opportunity to share experiences and receive support, and made the programs feel more personable (Struthers et al., 2015). However, high quality studies evaluating the effectiveness of online interventions, including the addition of therapist involvement, is limited and warrants further investigation (Ye et al., 2014).
Previous research has demonstrated the equivalence of online therapies to face-to-face treatments of anxiety and depression for both adults and adolescents (e.g., Andrews, Cuijpers, Craske, McEvoy, & Titov, 2010; Hedman et al., 2011; Richardson et al., 2010; Vigerland et al., 2016). Internet-based therapies have also shown promise for reducing self-harm (Lewis, Heath, Michal, & Duggan, 2012) and suicidal ideation (Christensen et al., 2014), two symptoms closely associated with personality disorder, or emerging personality disorder. What remains unknown is whether these online interventions can be applied to adolescent personality dysfunction. As such, this study aims to identify characteristics that maximise effectiveness of online psychotherapy for adolescents with personality disorder or showing signs and symptoms of emerging personality disorder. We aim to identify relevant existing online interventions and empirical literature on online interventions prior to the declaration of the COVID-19 pandemic, to inform development of novel interventions. More specifically, the review will be presented in two parts: an examination of the characteristics of existing online mental health programs that target personality disorder or symptoms of emerging personality disorder for adolescents (a scoping review); and a review of empirical literature evaluating online interventions that target personality disorder or symptoms of emerging personality disorder for adolescents (rapid review). Taken together, it is hoped that results and recommendations from these reviews may inform development of feasible, acceptable and effective online interventions for personality disorder and emerging personality disorder in adolescents, with implications for the COVID-19 global pandemic.
Materials and Methods
This research will use scoping review and rapid review methodology conducted prior to the declaration of the COVID-19 pandemic to answer the question: What characteristics are common to online intervention approaches for adolescents with personality disorder, or adolescents showing signs and symptoms of emerging personality disorder? This will involve identification and examination of existing online mental health programs that target personality disorder or emerging personality disorder symptoms for adolescents, and a rapid review of the empirical literature to inform how these may be adapted to inform online interventions.
Phase one: Scoping review
During January and February 2020, a search of existing publicly available online apps and websites dealing with adolescent interventions with an interactive online component was conducted. A grey literature scoping review search was conducted using the search engines Google and Google scholar. In addition, well-known mental health organisations were searched, and online interventions known to the researchers were also included.
Scoping review methodology was selected to identify what online interventions targeting personality disorder or symptoms of emerging personality disorder were currently available, and the features and content used to address adolescent mental health (Munn et al., 2018). Grey literature searches can identify content that is accessible to the public (Adams et al., 2016; Benzies, Premji, Hayden, & Serrett, 2006; Godin, Stapleton, Kirkpatrick, Hanning, & Leatherdale, 2015). Google and Google Scholar were chosen as they yield large results and speed up the required searching time (Haddaway, Collins, Coughlin, & Kirk, 2015).
Selection process
Due to time and resource demands, search engines were limited to the top 100 results. An initial search indicated the minimal number of any interventions targeted towards personality disorder in adolescents. The following search terms were therefore chosen to broaden the scope of the searches in order to yield a larger variety of interventions. This included programs which targeted symptoms that are commonly recognised in adolescents with personality disorder or emerging personality disorder (i.e. depression, suicide, and self-harm; Chanen, Jovev, & Jackson, 2007; Kaess et al., 2013) and which were recognised to already have a large evidence-base for online therapy (i.e. depression and anxiety; Calear & Christensen, 2010; Clarke et al., 2015; Struthers et al., 2015) to help identify what would be effective for online programs targeting personality disorder or emerging personality disorder. The search terms used were: online therapy, or e-therapy, or app, or self-help or online intervention, or online treatment; and adolescent, or teen, or child; and personality disorder, or depression or suicide or self-harm or self-injury or cutting or mental health or borderline. Despite the aim of the research targeting adolescents specifically, child was included as a search term due to differences in definition and age-range of adolescents. We used the world health organisation (WHO) age range of 10 to 19 years old (World Health Organisation [WHO], 2020a). Inclusion criteria included online apps or websites dealing with adolescents or young people (from age 10 to age 25) mental health interventions with an interactive component. Programs targeting young adults, classified as ages 20 to 25 were included in analysis due to many of programs targeting both adolescents and young adults and the acknowledgment that effective components may overlap.
We aimed to identify mental health programs that addressed self-harm, suicidality, interpersonal difficulties, emotion regulation difficulties, and other features of personality disorder (American Psychiatric Association, 2013; World Health Organization, 2018). However, the results were not limited to this due to the lack of interventions that target this, and no programs specific to personality disorder were identified. We therefore included depression, generalised anxiety and general mental health programs in our analysis due to the symptom overlap and comorbidity with personality disorder symptoms (i.e. BPD in adolescents comorbidity and symptomology: Chanen et al., 2007; Ha, Balderas, Zanarini, Oldham, & Sharp, 2014). Exclusion criteria included programs targeted towards individuals out of the 10 to 25 year age range, programs not designed to target mental health, programs designed to target mental health aspects not applicable to emerging personality disorder, [including chronic pain, chronic health conditions, eating disorders, parents and carer interventions, alcohol and drug use and addiction, obsessive compulsive disorder (OCD) and post-traumatic stress disorder (PTSD)], programs with no online app or website component, programs with no interactive component, online helpline or therapist communication platform only, websites and apps that were not interventions, and programs where the authors could not access any information without a reference or payment required. In addition to this, mental health organisations were searched, any references to other interventions during the search were identified, and any known interventions not already identified were also included. The study flow is presented in Figure 1.
Data extraction
Screening was conducted by one author (EM) and a discussion took place between authors to review screening and decide the final programs to include for review. Once screening was completed, the authors discussed elements to review. Data was extracted by one author (EM) and these included structure of the programs, interactive elements of the program and information about how the content of the program was presented to participants. A second author (SR) then reviewed the data extraction. Once initial review was complete, authors discussed data and considered most essential information.
Phase two: Rapid review of empirical literature
Phase two involved a rapid review of the literature to explore characteristics that may enhance efficacy of online mental health interventions targeting personality disorder or emerging personality disorder symptoms for adolescents, conducted in January 2020. We were interested in data around efficacy of online interventions and characteristics those interventions share. Rapid reviews streamline traditional systematic review methods to achieve a synthesis of evidence within a short timeframe (Haby et al., 2016), and have been shown to generate similar conclusions to those from more intensive systematic reviews when undertaken in a structured and transparent manner (Abou-Setta et al., 2016; Varker et al. 2015).
Selection process
We aimed to keep our methodological decisions as close as possible to those used in the scoping review. As in the scoping review, search terms were chosen to yield a broad variety of interventions due to the lack of specific interventions which target personality disorder. This included programs which targeted symptoms that are commonly recognised in adolescents with personality disorder or emerging personality disorder (i.e., depression, suicide, and self-harm; Chanen et al., 2007; Kaess et al., 2013) and which were recognised to already have a large evidencebase for online therapy (i.e., depression and anxiety; Calear & Christensen, 2010; Clarke et al., 2015; Struthers et al., 2015). Three databases were searched (PSYCInfo, Scopus, and ProQuest central) using a combination of the search terms: online therapy, or online treatment, or online intervention, or online self-help, or e-therapy, or internet therapy or internet-based CBT, or internet-based cognitive behaviour therapy, or internet-based therapy, or internetbased psychotherapy, or internet-based treatment, or internet- based intervention; and adolescent, or teen, or young people, or young person, or youth; and personality disorder, or depression, or suicide, or self-harm, or self-injury, or cutting, or mental health, or borderline; and efficac*, or effectiv*, or outcome*, or impact*, or improv*. In addition, any efficacy papers identified during phase one based on the apps and website interventions were included in screening. Any additional research papers known to the researchers were also included in screening. After an initial search of PSYCInfo database, authors discussed inclusion and exclusion criteria. As with the scoping review, inclusion criteria included any empirical studies that evaluated the effectiveness of adolescents (age 10 to 19) and young people (age 20-25) online mental health interventions. We were particularly interested in interventions targeted towards emotion regulation, interpersonal difficulties, selfharm, suicidality, and other emerging personality disorder symptoms. However, due to the lack of interventions developed specifically for personality disorder, interventions targeting depression, generalised anxiety, and general mental health and wellbeing were included because of the recognised symptom overlap and comorbidity with personality disorder (i.e., BPD in adolescents comorbidity and symptomology: Chanen et al., 2007; Ha et al., 2014). Exclusion criteria included interventions that were not psychological, interventions that did not target mental health, interventions that targeted non-relevant, and specific mental health concerns (including mental health impact of chronic or other health conditions, neurobiological disorders, sleep-related disorders, grief and loss, children of parents with mental illness, brain tumours and injuries, eating disorders, alcohol and drug use and addiction, pregnancy and postnatal depression, PTSD, OCD, Tourette’s, domestic violence, and chronic pain), review articles, articles with no evaluation of the intervention (for example, protocol papers), interventions targeted for individuals outside a 10-25 age range, and interventions where the primary mode of delivery was not online. The study flow is presented in Figure 2.
Figure 1.
Study flow diagram for phase one: scoping review.
Data extraction
Screening of all articles and data extraction was conducted by one author (EM). All screening and data extraction were then reviewed by a second author (SR) and any discrepancies were discussed.
Results
Phase 1: Scoping review
Results from phase one are presented in Table 1. Of the 32 programs reviewed, eight (5%) specifically targeted adolescents, three (9%) targeted both adolescents and adults, three (9%) targeted both children and adolescents, three (9%) were directed towards older adolescents or young people (18-25 years old) or tertiary students, eight (25%) were targeted to adults, including young people aged 18 to 25 years, and seven (22%) had no age target. The primary mode of delivery for 17 (53%) of the programs was online, 11 (34%) of the programs were delivered via a phone application (App), and four (13%) had both online and app delivery.
Structure and practical components of the programs
The structure of the programs varied depending on whether they were delivered online or via app and whether they were modelled on evidence-based therapy models (for example, cognitive behaviour therapy). Apps were generally less structured, and 14 (44%) of the programs did not include specific sessions or modules. Eighteen programs (56%) had specific sessions or modules, the terms used to describe the sessions included modules, lessons, challenges, units, or activities. On average, programs had 7.2 sessions (SD=2.2, range= 4-14) with some programs tailoring content and sessions based on personal responses to initial assessment or personal goals.
Programs were recommended to be completed either daily, weekly, or fortnightly, ranging in the time required per session from 10-15 minutes to four hours. Majority of programs required sessions to be completed in a sequential order, or recommended order of sessions, however some were less structured with self-paced completion of sessions, within either long time frames (for example, 90 days) or no time frames. Two (6%) of the programs were specifically designed as a game and involved completing levels or challenges.
Five (16%) of the programs included therapist contact, either via email, online platform, or telephone. Of the 27 (84%) programs that did not provide access to a therapist, eight had the option to share or link data to a personal health professional, and three were specifically designed for this purpose, with one program only being available after a referral by a health professional. Four (13%) of the programs also provided resources or sessions for parents or carers, with one of these programs being specifically designed for the parent or carer to mentor the young person through the program.
Program elements
All programs included information and content related to the mental health difficulties being addressed. How this information was communicated varied and included text, video files, audio files, images, and comics. Some of the programs also included case studies or real-life examples both in text and video format to communicate information. All programs also provided some information on skills or tips to overcome or manage the mental health difficulties discussed. Some programs provided interactive activities or games to practice skills, such as simple breathing or focus exercises, and others provided homework and downloadable/printable content to practice skills away from the program.
Other common features of the programs included an initial assessment in the form of a telephone call or online questionnaire, with feedback; surveys or quizzes to monitor symptoms or check knowledge; opportunities to make personal goals and review progress made towards goals; feelings, mood, or anxiety trackers often in the form of ratings out of ten when logging-on to the program; sections for journal or diary entries to track thoughts and experiences; additional resources and relaxation exercises; forums or chat rooms to connect with peers; and email reminders, online calendars or encouragements to help keep participants on track.
Phase 2: Rapid review
Results of the rapid review of empirical literature are presented in Table 2. No studies directly targeting personality disorder in adolescence were identified. Those reviewed represent interventions targeting conditions and symptoms related to personality disorder in young people or emerging personality disorder. Of the 41 studies reviewed, the most common target problem was depression 44%, 18 studies); followed by anxiety (20%, 8 studies), depression and anxiety together (17%, 7 studies), and suicidal behaviours/ideation (10%, 4 studies). Other studies targeted non-suicidal self-injury (2%, 1 study), mental wellbeing generally (2%, 1 study), and one intervention was multi-diagnostic and tested with adolescents in inpatient care (2%). One study (2%; Alvarez-Jimenez et al., 2018) evaluated an intervention targeting young people at high risk of developing psychosis. After a full-text screening of this study, it was reviewed due to the high similarities in symptomology for young people considered at high risk for psychosis and those with personality disorder symptoms.
Figure 2.
Study flow diagram for phase 2: Rapid review of empirical literature.
Approximately half of the studies (51%, 21 studies) reported results of randomised control trials, with pilot studies (17%, 7 studies) and feasibility and/or acceptability trials also common (10%, 4 studies). It should be noted that summary statistics reported reflect the number of articles identified, rather than number of interventions. As such the same intervention may be examined more than once within Table 2.
Whilst all studies included adolescent participants (aged 10-19 years), some studies also included either young adults (20-25 years; 34%, 14 studies) or children (0-9 years; 2%, 1 study) in their sample. One additional study examined psychiatric nurse data for the purposes of evaluating the adolescent online intervention. As such, studies were heterogeneous in terms of both participants and methods. Because of inconsistent reporting of effect sizes across the studies, and wide variations in study design (e.g., RCT versus pre-post design), method, target of the interventions, and outcome measures used to assess change in symptoms, it was deemed inappropriate to calculate an average effect size via meta-analysis (Cochrane Effective Practice and Organisation of Care, 2017; Haidich, 2010). Effect sizes reported in Table 2 ranged from very small to large for between- and within-subjects effects.
Characteristics of interventions
In cases where the intervention was described in detail, summary data was collected around the characteristics of the interventions in terms of the frequency of module delivery, duration of the intervention, the content/activities/ format that comprised the intervention and any therapist contact that occurred as part of the intervention.
Stages of the intervention were most frequently described to participants as modules (12 studies), sessions (9 studies), or steps (2 studies). As such, for the purposes of Table 2 these have been termed modules. The average number of modules was 8.0 (SD=2.7), but these generally ranged from 4 to 14 modules. Additionally, two studies of the same intervention (Rice et al., 2018; Santeseban- Echarri et al., 2017) reported a large number (56) of optional steps for participants that were shorter in duration (approximately 20 minutes), which diverged significantly from the other studies identified as was not included in the average calculation. Frequency of module delivery also varied widely, from 4 times per week to fortnightly, with the majority requiring one module per week to be completed by participants. Duration of modules ranged from 10 to 90 minutes, excluding in-between module homework, and the average length of intervention was 7.6 weeks (SD=2.6).
Table 3 provides a descriptive summary of the various components and features included in the interventions. It should be noted that these represent a summary of the information available, and interventions varied in how many of these components they included.
Role of the therapist
Information was also extracted regarding therapist/moderator contact with 68% (28 out of the 41) of studies reporting on interventions including at least some contact with a therapist/moderator/facilitator. Three studies did not provide this information and one intervention was described as self-help but participants were concurrently receiving regular face-to-face treatment alongside the intervention.
The role of the therapists/moderator/contact person varied widely and included: psychologists, clinical psychologists, moderators, facilitators, school wellbeing staff, nurses, psychology students in final year of placement, research assistants, mental health clinicians and lived experience peers.
Contact with the therapist usually occurred via email/messaging, phone or in-person and varied in frequency. Some interventions had initial consultations with an option for contact if needed. Others had weekly contact following the completion of the module (generally around 15 minutes in length). More intensive interventions included completion of module along with the therapist but this was rare. The most common procedure was for the client to complete the module individually with the option to ask the therapist a question. The client would then receive a follow-up call/email with feedback from therapist once module was complete. For the most part, this comprised weekly therapist/moderator contact.
The role of the therapist/moderator also varied widely among interventions. Roles and tasks included: providing feedback on homework and session activities; answering questions; providing encouragement and motivation (e.g., congratulating client for completing module); reminders and follow-up for non-completion of tasks/activities/modules; checking in at different time points (sometimes after conclusion of intervention) to see if/how skills were applied; assistance with applying skills; summarising content for client, encourage homework and practicing skills; resolve/ work through any challenges.
Other aspects of interventions
Other common and notable aspects of interventions included: Therapists/moderators generally had access to the client workbooks and activities in order to provide feedback; client often had option whether to contact via phone or email; in some cases (with permission of client) therapist contact details were provided to parents; some interventions required specific training of therapists in the online intervention; some interventions provided therapists with templates, set criteria, or protocols to guide protocols to guide communication with the client.
Discussion
There is an urgent need to develop and test empirically based online interventions for personality dysfunction in adolescence. This research used scoping and rapid review methodology to identify characteristics that may maximise the effectiveness of online intervention for adolescents, but no studies or interventions could be identified targeting personality disorder specifically. A scoping review identified 32 online mental health programs relevant to adolescents and young people with personality disorder or emerging personality disorder symptoms, including anxiety and depression (see: Chanen et al., 2007; Ha et al., 2014). A rapid review of the empirical literature uncovered 41 studies that targeted symptoms that may present alongside personality disorder or emerging personality disorder (e.g., depression, suicidal ideation, self-harm) - but again none of these directly targeted personality disorder. The research provides an indication of the necessity of developing online interventions targeted to the prevention and treatment of personality disorder or emerging personality disorder in adolescence. While there were no personality disorder-specific interventions, our results have implications for the development of these interventions and should be used to aid adolescents in accessing treatment which may be otherwise unavailable. This is particularly relevant to the current global COVID-19 pandemic due to the emergence of a situation where access to face-to-face therapy has become increasingly limited.
Despite no specific programs or interventions targeting personality disorder, integration of information from the empirical and scoping reviews made it possible to isolate common characteristics of the interventions which may be beneficial to integrate in future interventions that seek to target personality disorder or emerging personality disorder for adolescents. These common characteristics cover the type, delivery and components that are thought to increase success and improve adherence to interventions. Common characteristics include:
Interventions comprise of a number of modules, steps, sessions or lessons - approximately 7 or 8 - delivered weekly (over approximately 10 weeks);
Interventions utilise sequential presentation of modules - making new modules available to the client only when they have completed the previous one (e.g., Dear et al., 2018; “Mindspot Mood Mechanic Course”, 2020);
Interventions include homework, workbooks or diaries between modules in order to promote understanding, record practicing of skills, and prompt subsequent discussions with their therapist around issues they found challenging (e.g., Bjureberg et al., 2018; “Chilled Out Online,” 2020);
Interventions provide an option to connect with a therapist either via phone, email/messaging or face to face. Therapists monitor and provide feedback on activities/ homework, answer questions, assist the client to overcome challenges, provide reminders to complete homework/activities, and motivate adherence and completion of the intervention. Many successful interventions emphasised the importance of regular (albeit brief) therapist contact (e.g., Bjureberg et al., 2018; Dear et al., 2018; Gladstone et al., 2015; Hetrick et al., 2017; Lindqvist et al., 2020; Santesteban- Echarri et al., 2017; Spence et al., 2011);
Inclusion of screening/assessment of clients prior to them being given access to modules (via phone or face to face), so the client has initial information about the intervention and so that the appropriateness of the intervention for that client can be gauged (e.g., Alvarez- Jimenez et al., 2013; “myCompass,” 2020; “Mindspot Mood Mechanic Course”, 2020);
The use of characters, cartoons or avatars with different personalities/difficulties to illustrate symptoms and demonstrate skills (e.g., Bobier, Stasiak, Mountford, Merry, & Moor, 2013; “Super Better”, 2020; “Sparx”, 2009; “This Way Up TeenStrong”, 2020);
A section for resources/factsheets/skills that can be accessed at any time and are collected as the client progresses through modules (e.g., March, Spence, Donovan, & Kenardy, 2018; “The BRAVE program”, 2020; “Mindshift CBT”, 2019);
Modules may include short videos providing information and details around skills and then interactive activities to assist the client in practicing the skills (e.g., Hetrick et al., 2017; Lindqvist et al., 2020; Richards, Cogan, Hacker, Amin, & Beoing, 2004; “Beating the blues,” n.d.);
Sections where clients can create a personalised safety plan for times of crisis (e.g., selects and lists support people; records preferred relaxation activities that work for them in times of distress). It is important that this safety plan can be accessed by the client at any time (e.g., Bjureberg et al., 2018);
A method to allow the client to track their progress (e.g., via assessment of symptoms, quizzes, therapist feedback) and a certificate of completion once the intervention has concluded (e.g., Stjerneklar, Hougaard, Nielsen, Gaardsvig, & Thastum, 2018; “moodgym”, 2020; “E couch”, 2018);
Including parent workbooks or parent interventions (often with fewer modules than those for clients). This allows parents to learn skills and understand their child’s difficulties, as well as assist them in motivating their child to continue with the intervention (e.g., Waite, Marshall, & Creswell, 2019; “This Way Up Teenstrong,” 2020; “Chilled Out Online,” 2020)
A specific component reviewed within interventions and programs was therapist contact. In phase one, only a limited number of programs had the addition of therapist contact. It should be noted that phase one was a scoping review designed to gain understanding on the scope of available programs, and no evaluation on the efficacy of these programs was conducted. It is of note that there were limited options for therapist contact as previous research has indicated that programs with concurrent therapist contact have highest completion rates, with self-guided phone applications having the lowest completion rates (Struthers et al., 2015). Therefore, there is an imperative for more evidence-based online programs to be developed. However, phase two supported evidence on therapist involvement, with well over half the studies including at least some therapist contact. One study that evaluated therapist contact found that young people were satisfied with therapist support, finding it motivational and helpful and when provided with the option for additional contact, all participants accepted it (Silfvernagel, Gren-Landell, Emanuelsson, Carlbring, & Andersson, 2015). Another study compared an intervention with either a self-guided or therapist-guided component and found that while both had significant improvements, satisfaction with the intervention among young people was higher when it was clinician-guided (Dear et al., 2018). Previous reviews indicate the benefits of therapist or supervisor support either via phone, email, teleconferencing or in-person when adolescents and young people are engaging in online therapy and has been linked to better engagement and adherence, as well as intervention outcomes (Clark et al., 2015; Struthers et al., 2015). Specific to the COVID-19 pandemic, a recent review found that although there are limitations, telepsychotherapy used when face-to-face therapy is not available is found to be effective for adults (Poletti et al., 2020) and given the increasing engagement of adolescents in digital technologies, it is likely telepsychotherapy will be similarly effective (Montague et al., 2015). Our study specifically excluded online platforms which replaced face-to-face therapy through an online platform with no interactive intervention component in order to meet the aim of identifying specific online interventions, however the future research could aim to identify and evaluate telepsychotherapy.
This review was conducted prior to COVID-19 being declared a pandemic and the isolation and health restrictions that followed reducing access to face-to-face interventions. It is likely that following this declaration, new online interventions for adolescents were developed to address the urgency of accessibility which this study was not able to review. However, it was important to identify the pre-existing evidence to understand what may be necessary and effective to include in the development of new programs and interventions. This research has implications extending to provision of more cost-effective, accessible prevention and treatment of personality disorder or emerging personality disorder symptoms in adolescence, particularly to those who experience added barriers such as living in remote locations, and when face-to-face access is limited, including during the time of COVID-19.
Personality disorders are complex mental illnesses that are costly for both the health sector and the individual. Enriching understanding of components of successful prevention and treatment, and developing more accessible treatments for personality disorder that can be delivered early in the symptom trajectory has been recognised as a national mental health priority area in Australia (NHMRC, 2012). Given the effect that the COVID-19 global pandemic has had on mental illness risk in the general population (e.g., Serafini et al., 2020) and individuals with pre-existing mental health conditions and personality disorders specifically (Álvaro et al., 2020; Yao et al., 2020) providing effective online therapy has clear implications for mitigating symptom progression and improving access for those who are geographically isolated. As such, flow on research generated by this review has potential to impact health and wellbeing for adolescents with personality disorder or emerging personality disorder symptoms, and their carers. Future research will integrate these results to develop and test a brief online intervention targeting personality disorder and emerging personality disorder symptoms in adolescence.
The common characteristics identified is supported by previous research which evaluates the effectiveness of online therapy for adolescent mental health. Skillsbased online interventions with specific sessions have been found to be effective for adolescent mental health (Clark et al., 2015) and human support elements and therapist support has been found to be valuable in the implementation of online therapies (Clark et al., 2015; Struthers et al., 2015). Further, research indicates that young people are increasingly engaged with technology and willing to engage in online psychotherapy however care still needs to be taken to ensure a collaboration between client and therapist and foster the therapeutic relationship (Montague et al., 2015).
While there is previous research on adolescent mental health interventions in online format, the research specific to personality disorders in adolescence is scarce. This makes it difficult to conclusively identify what may or may not be effective. However, research does indicate that developing evidence-based psychotherapy into online formats is effective for adolescents and young people (Clark et al., 2015; O’Dea et al., 2015). Therefore, integrating common elements of adolescent online interventions identified in this review, as well as evidence for effective psychotherapy for personality disorder may be beneficial in the development of new online interventions which target personality disorder, or emerging personality disorder symptoms in adolescents. Evidence suggests that therapeutic approaches traditionally applied to personality disorder populations, like dialectical behaviour therapy (DBT), can be effectively delivered using online formats (Wilks et al., 2018). Whilst online therapies for conditions related to personality disorder may inform new approaches, it is recommended that empirically supported treatment components specific to personality disorder treatment are incorporated into new online therapies. Since personality disorders can be understood as a function of relationships between self and other, it is recommended that a relational model of care is incorporated into approach (Grenyer, 2014; Townsend, Haselton, Marceau, Gray, & Grenyer, 2018). This model emphasises the need to understand symptoms of personality disorder as stemming from problematic and dysfunctional relationship styles that have developed over time. The Relational Model is closely linked to attachment theory (Bowlby, 1979) which posits that models of the self and others are formed via early interactions with significant others, and these impact personality organisation, self-worth, and interpersonal relationships throughout the lifespan. Attachment is highly relevant to both development and treatment of personality disorders (e.g., Fonagy, Target, & Gergely, 2000), as it has been shown to affect in therapy variables (e.g., the therapeutic alliance; Diener & Monroe, 2011). Furthermore, changes towards a more secure attachment representations have been demonstrated to be predictive of recovery across many mental disorders (Levy, Kivity, Johnson, & Gooch, 2018). The Relational Model expands on attachment theory, incorporating multiple relationships relevant to adolescents: their relationship to self, the clinician, family, peers and the school and community (Townsend et al., 2018). Psychological education and connection with family members, carers, schools and health services should therefore be considered within new approaches for personality disorder. Other intervention components may address factors specific to personality disorder, like emotional dysregulation, rejection sensitivity and mentalisation problems, to provide a more targeted treatment approach (Grenyer, 2014).
Recommendations emerging from the literature and scoping review have clear implications for future research and development of interventions. Despite this, there were some limitations to this methodology, particularly in terms of the restricted time frame of the scoping and rapid review. Whilst rapid review methodology is acknowledged to be a robust method of synthesising data (Abou-Setta et al., 2016; Haby et al., 2016) a more thorough meta-analysis including effect sizes for each intervention, would provide useful information. This was outside of the scope of the present research but would be an important next step in demonstrating the efficacy of different types of online interventions for adolescents with personality disorder or emerging personality disorder symptoms. Currently the limited amount of literature around this specific topic precludes the ability to conduct meta-analyses. Future research should utilise randomised controlled trial methodology to evaluate new online therapies and interventions and should also seek to provide quantitative data synthesis from these studies. Limitations of the scoping review include the time sensitive nature, and location-restrictions of search algorithms (on Google, for example). It is possible that a greater number of interventions for adolescents exist that were not included, but efforts were made to identify each relevant source of data (e.g., stringent inclusion/ exclusion criteria).
Since the reviews were conducted in early 2020 and prior to the declaration of COVID-19 as a pandemic by WHO on the 11th March 2020 (for a full timeline of COVID-19 see: WHO, 2020b), the full impact of COVID-19 had not yet been felt in many parts of the world, so it is possible that a greater number of relevant studies have been published since then. Due to the currently unfolding and unpredictable nature of the pandemic, and the evident urgency of the need for online interventions, our study is valuable in providing an indication of what existed prior to the pandemic and has implications for the development of emerging and effective online interventions. However, future research should endeavour to identify programs and interventions that may have been developed during, or due to, the COVID-19 pandemic, and identify the characteristics and evaluate the effectiveness of these. Some studies also included participants below the age of 10, and over the age of 19, which is outside the WHO (2020a) definition of adolescence (10 to 19 years). Given the scarcity of studies, and the value of the data they provided, these were not excluded, but this limits the specificity of findings to adolescents. Finally, the time limited nature of rapid reviews means that a more thorough examination of each article included and the risk of bias inherent within those articles was not possible. There were a multitude of factors and components of each intervention that were beyond the scope of this review, but may provide additional useful information, so future research may examine more intricate aspects of the online interventions.
Overall, results of this study indicated that while many online interventions for adolescents targeting depression and anxiety exist, no studies have directly evaluated an intervention for adolescents with personality disorder or showing signs and symptoms of emerging personality disorder. This research highlights the need to develop new evidence-based interventions that can improve access to treatment for adolescents, particularly relevant during the time of the COVID-19 pandemic when face-to-face therapies are not easily accessible. This study and recommendations generated highlight characteristics of other online treatments that may be generalised to prevention and treatment of personality disorder, and thus represent an initial step towards the development of new interventions for adolescents experiencing these problems.
Table 1.
Summary of online interventions included in phase one scoping review.
| Intervention name | Publisher | Format | Country of development | Participants | Target |
|---|---|---|---|---|---|
| AIMhi Stay strong app | Menzies School of Health research | App | Australia | Adults: 18 + yearsc | Mental health and wellbeing for Indigenous Australians |
| Anxiety Coach | Mayo Clinic | App | United States of America | Children and Adolescents | Anxiety |
| Anxiety Reliever | Anxiety Reliever LLC | App | United States of America | No age | Anxiety |
| Beating the blues | 365 Health Solutions Limited | Online program | United Kingdom | Adults: 18 + yearsc | Anxiety and depression |
| BetterBET | University of Carlifornia, Palo Alto University | Online program | United States of America | Adults: 18 + yearsc | Depression |
| Bite Back - mental fitness challenge | Black Dog Institute | Online program | Australia | Adolescents: 12-18 years | Resilience and wellbeing |
| The BRAVE program | Beyond Blue | Online program | Australia | Adolescents: 12-17 yearsb | Anxiety |
| Calm Harm app | Stem 4 | App | United Kingdom | Adolescents and adults: 13 + years | Self-harm |
| Chilled Out Online | Macquarie University | Online program | Australia | Adolescents: 13-17 years | Anxiety |
| Clear Fear app | Stem 4 | App | United Kingdom | Adolescents: 11-19 years | Anxiety |
| Dealing with Depression (DWD) | Centre for Applied Research in Mental Health and Addiction | Online program | Canada | Adolescents: 13-19 years | Depression |
| E-Coucha | e-hub Health | Online program | Australia | Adolescents and adults: 12 + years | Depression, anxiety, mental health |
| Kooth | Xenzone | App | United Kingdom | Adolescents: 11-18 years | Emotional Wellbeing |
| Living life to the full | Five areas Limited | Online program | United Kingdom | Adults: 18 + yearsc | Anxiety, Stress and depression |
| Mentalhealthonlinea | Swinburne University of Technology | Online program | Australia | Adults: 18 + yearsc | Generalised anxiety, depression |
| Mindshift CBT | Anxiety Canada | App | Canada | No age | Anxiety |
| MindSpot Indigenous Wellbeing course | MindSpot | Online program | Australia | Adults: 18 + yearsc | Indigenous Australians with symptoms of depression and anxiety |
| MindSpot Clinic Mood Mechanic Course | MindSpot | Online program | Australia | Young people: 18-25 years | Young adults with symptoms of depression and anxiety |
| Moodgym | ehub Health | Online program | Australia | Adolescents and adults: 16 + years | Anxiety and depression |
| MoodKit | Thrive Port | App | United States of America | Adults: 18 + yearsc | Mood improvement |
| MoodMission | MoodMission | App | Australia | No age | Low mood and anxiety |
| My Anxiety Plan for children and teens (MAP) | Anxiety Canada | Online program | Canada | Children and adolescents | Anxiety |
| My Compass | Black Dog Institute | Online program | Australia | No age | Mental Health |
| My Digital Health mental health and wellbeing coursesa | My Digital Health | App and Online program | Australia | Adults: 18 + yearsc | Depression, anxiety and mental health |
| SPARX | The University of Auckland | Online program | New Zealand | Young people | Mild to Moderate depression |
| Stress and anxiety in teenagers | National Health Service (NHS), Richards et al. | Online program | United Kingdom | Adolescents: 12-19 years | Stress and anxiety |
| SuperBetter | SuperBetter | App and Online program | United States of America | No age | Resilience and wellbeing |
| Thedesk | The University of Queensland | Online program | Australia | Tertiary students | Mental health and wellbeing |
| This Way Up adult mental health coursesa | St Vincent’s Hospital | App and Online program | Australia | Adolescents and adults: 16 + years | Depression, anxiety, stress |
| This Way Up TeenSTRONG | St Vincent’s Hospital | App and Online program | Australia | Adolescents: 12-17 years | Worry and sadness |
| WellMind | National Health Service (NHS) | App | United Kingdom | No age | Anxiety and depression |
| What’s Up | What’s Up, Tempra | App | Australia | No age | Depression, anxiety, anger, stress |
aPrograms offer several courses targeted towards different mental health concerns, or aspects of wellbeing; bThis program had several courses for different ages, the course designed for this age range was reviewed; cPrograms age target included young people aged 18-25 and were therefore included in review.
Table 2.
Summary of findings from rapid empirical literature review.
| Study | Country | Design | Participants | N | Type of intervention | Frequency/ duration | Therapist assistance | Outcomes/effect size if available |
|---|---|---|---|---|---|---|---|---|
| Alvarez-Jimenez et at (2018) | Australia | Pilot study | Adolescents and young adults: 15-25 years | 14 | Strength based psychotherapy model based on Social Determination Theory for adolescents at high risk of developing psychosis | Unrestricted access | Therapist assisted: Phone, online messaging | Results indicated medium to large increases in personal strengths use (d = 0.7), mindfulness skills (d = 0.66), attachment (d = 0.7), guidance (d = 0.75), subjective wellbeing (d = 0.75). Participants displayed reliable improvement in depressive symptoms. |
| Anderson et al. (2012) | Australia | Randomised trial | Adolescents: 12-18 years | Face to face therapy condition n =38, internet condition n = 35 (Study 1); n=132 (Study 2) | CBT targeting anxiety | 10 sessions over 10 weeks | Therapist assisted (minimal email and phone contact) | Study 1 found no significant differences between groups on working alliance scores for adolescents (d = 0.15), but parent working alliance was stronger for face to face condition (d = 0.64) than internet condition. Study 2 demonstrated significant improvement in anxiety and global functioning for participants exposed to the online therapy (at 6 months follow up). |
| Bannink et al. (2014) | Netherlands | RCT | Adolescents: 15-16 years | Online intervention only n = 392; Online intervention with consultation n = 430; Control group n = 434 | Messages and information focussed on mental wellbeing generally | Single session (online intervention and consultation group only) | Therapist assisted: 45 minute consultation with option for further consultation if needed | Online intervention and online intervention with consultation groups reported better mental health status compared to control (d = 0.12). Adolescents at risk who were part of the online intervention with consultation group reported significantly better mental health status (d = 0.34) and quality of life (d = 0.37) at follow-up compared to adolescents as risk in the control group. |
| Bjureberg et al. (2018) | Sweden | Pilot Study | Adolescents: 13-17 years | 25 | Acceptance-based behavioural intervention therapy - targeting emotional regulation for non-suicidal self injury (NSSI) | 11 modules over 11 weeks | Weekly therapist feedback and assistance with activities; telephone contact | There was a large reduction in NSSI frequency pre- to post-treatment (d = 0.88); a medium reduction post-treatment to 3-month follow-up (d = .57); and no significant change from 3-month to 6-month follow up. Overall change in NSSI frequency from baseline to 6 month follow up was large (d = 1.36). Emotional dysregulation also decreased pre- to post- treatment (d = 0.75). |
| Bobier et al. (2013) | New Zealand | Feasibility and acceptability study | Adolescents: 16-18 years | 20 | Third-person fantasy based game designed around CBT principles targeting variety of mental health issues | 7 modules (30 minutes each) | Therapist assisted (inpatient facility) | Adherence and uptake of intervention was satisfactory. Facilitators of satisfaction for patients included the localised look and feel of program, self-paced nature, game design and ability to undertake intervention while in hospital. Barriers to satisfaction included the quality of graphics. One participant felt talking therapy was superior. |
| Calear et al. (2009) | Australia | Cluster RCT | Adolescents: 12-17 years | Intervention condition n = 563; waitlist control n = 914 | CBT targeting symptoms of anxiety/depression | 5 modules over 5 weeks | No therapist contact (teachers supervised at school) | Greater reduction in anxiety for intervention group compared to waitlist group post-intervention (d = 0.15); with similar pattern of results at 6 month follow up (d = 0.25). Depression scores were lower for male participants in intervention group compared to waitlist post-intervention (d = 0.43), but there was no significant difference for female participants. |
| Chapman et al. (2016) | UK | Feasibility study | Adolescents: 13-16 years | 11 | CBT intervention, mindfulness and self-regulation skills targeting anxiety and low mood | 7 levels over 7 weeks | Therapist assisted | Reductions in symptoms of anxiety and depression were shown for 4 of the 11 participants and one participant demonstrated clinically significant change. Parent ratings of depression and anxiety indicated that six participants showed reductions on these indices. Benefits reported by participants included: improving skills around negative thought recognition and relaxation; computer delivered intervention; game was relaxed and fun; promoted feelings of being understood. Areas for improvement noted by participants included: too much information/too complex; not age-appropriate; and the difficulty of short-term work. |
| de Voogd et al. (2018) | Netherlands | RCT | Adolescents: 11-19 years | Intervention n=134; Placebo n = 39 | Interpretation bias modification training targeting anxiety and depression | 8 sessions over approximately 4 weeks | No therapist assistance | Interpretation bias improved for the intervention group compared to the placebo group. There were no differences in depression, anxiety or emotional scores between intervention and placebo groups. |
| de Voogd, et al. (2016) | Netherlands | RCT | Adolescents: 11-18 years | Emotion working memory condition n = 129; Placebo n = 39 | Emotional working memory training targeting anxiety and depression | 12 sessions over 3 weeks | No therapist assistance | No significant differences in working memory capacity, anxiety or depression were evident between intervention and placebo groups. |
| Dear et al. (2018) | Australia | RCT | Adolescents and young adults: 18-24 years | Clinician guided group n= 110; self-guided group n= 107 | Transdiagnostic psychological principles and cognitive and behavioural skills to target anxiety and depression | 5 lessons over 5 weeks | Therapist assisted (telephone contact and feedback for clinician guided group only) | Both groups reported significant improvements (self-guided and therapist-guided work) with satisfaction slightly higher for clinician guided group. |
| Gladstone et al. (2014) | USA | No control study | Adolescents and young adults: 14-21 years | 83 | Cognitive-behavioural, humanistic, interpersonal approaches for depression | 14 modules | Therapist telephone contact | Participants demonstrated improvement in symptoms of depression and reductions in automatic negative thoughts from baseline to follow up. |
| Hetrick et al. (2017) | Australia | RCT | Adolescents: 13-19 years | Intervention group n = 26; Control group n = 24 | CBT for suicide-related behaviours | 8 modules over 10 weeks | Therapist contact via online message board | Suicide ideation scores decreased in the intervention group from baseline to 10 week follow up and from baseline to 22 week follow up but differences between groups were not statistically significant. Fewer suicide attempts were reported for the intervention group at follow up. |
| Hetrick et al. (2014) | Australia | Pilot study | Adolescents: 14-18 years | Pre-treatment n = 32; Post-treatment n = 21 | CBT for suicide-related behaviours | 8 modules over 8 weeks | Therapist contact via online message board | Problem solving improved from baseline to post-treatment with a large effect size (eta-squared = 0.96). Emotion focussed coping improved from baseline to post-treatment with a large effect size (eta-squared = 1.2) |
| Ip et al. (2016) | China | RCT | Adolescents: 13-17 years | Intervention group n = 123; control group n=127 | CBT targeting depression | 10 modules over approximately 8 months | No therapist assistance (except crisis intervention) | At 12 month follow-up the intervention group had greater reductions in depression symptoms with a medium effect size (d = 0.36). |
| Karbasi and Haratian (2018) | Iran | RCT | Adolescent females: 10-18 years | Intervention group n = 15; Control group n = 15 | Internet based CBT (iCBT) targeting anxiety | 7 stages over 3 months | Therapist email contact | Both control and ICBT groups had significantly lower anxiety scores at post-test. The ICBT group had significantly lower anxiety scores from pre-test to post-test compared to the control group. |
| Kramer et al. (2014) | Netherlands | RCT | Adolescents and young adults: 12-22 years | Intervention group n = 131; Waitlist group n = 132 | Solution-focussed brief therapy targeting depression | 5 sessions (1 hour) once a week | Therapist contact via online chat | Greater reduction in symptoms of depression for the intervention group compared to waitlist from baseline to 9 weeks with a small effect size (d = 0.18) and from baseline to 4.5 months with a large effect size (d = 0.79). For the intervention group, there was a large effect size for reduction in depression symptoms between baseline and 7.5 month follow up (d = 1.6). |
| Kruger et al. (2017) | USA | Phase II Clinical trial | Adolescents and young adults: 14-21 years | Motivational interviewing with intervention group n = 40; Brief advice with intervention group n = 43 | CBT, interpersonal psychotherapy, behavioural action, community resiliency approaches targeting depression | 14 modules | Therapist contact in person and via phone | All participants had improved levels of automatic negative thoughts and emotional impairment over time but no differences in perceived social support from family and friends. Change was not evident in first 6 weeks but there was a marked decrease in automatic negative thoughts from 6 weeks at 2.5 year follow up. |
| Kuosmanen et al. (2018) | Ireland | RCT - but reporting on implementation study findings | Adolescents and young adults: 15-20 years | 28 | CBT-based self-help intervention for mild to moderate depression | 7 levels (approximately 30 minutes each) | No therapist assistance (staff monitored at point of delivery) | Participant feedback indicated that less than half liked the look/ thought it was fun and less than a third would recommend it to a friend. This was attributed to lengthy modules, technical problems and a perceived lack of positive focus. However the majority of participants found the program easy to use and understand. Individuals at high risk of depression rated it more relevant and useful than those at low risk. |
| Kurkl et al. (2018) | Finland | Qualitative descriptive study | Psychiatric nurses treating adolescents | 9 | Internet-based support system for outpatient psychiatric care targeting depression | 6 topics over 6 weeks | No therapist assistance (nurse assisted) | Psychiatric nurses perceived intervention positively but had difficulties encouraging adolescents to participate. They perceived technology to be useful in interventions, but felt that a lack of support made it difficult to implement the intervention, which was not easily integrated into regular care. |
| Lillevoll et al. (2014) | Norway | RCT | Adolescents and young adults: 15-20 years | Control group n = 180; Intervention with no reminder n = 176; Intervention with standard reminders n = 176; Intervention with tailored reminders n = 175 | CBT for symptoms of depression | 5 modules over 6 weeks | No therapist contact except email reminders delivered according to condition | No significant changes in self-esteem and depression symptoms were found (possibly attributable to drop out after Module 2). Tailored or standard emails did not differ in effectively prompting use of the intervention. |
| Lindqvist et al. (2020) | UK | RCT | Adolescents: 15-18 years | Intervention group n = 34, Control group n = 38 | Affect-focussed psychodynamic therapy targeting awareness, experience and expression of emotions | 8 modules over 8 weeks | Therapist online feedback, instant messaging and 30 minute chat each week | Large between groups effect size (intervention to control) for primary depression symptoms measures (d = 0.82), anxiety measures (d = 0.78), emotion regulation (d = 0.97), and self-compassion (d = 0.65) |
| March et al. (2018) | Australia | Feasibility and acceptability study | Children and adolescents: 7-17 years | 4425 | Interactive web-based CBT targeting anxiety | 10 sessions over 10 weeks | No therapist support | Anxiety scores for adolescent version of the intervention improved significantly between baseline and final session (d = 0.65). Greater improvements in anxiety were associated with a greater number of completed sessions but participants still showed improvement irrespective of the amount of sessions taken. |
| Neil et al. (2009) | Australia | Comparison study | Adolescents: 13-19 years | School sample n = 1000; Community sample n = 7207 | CBT targeting anxiety and depression | 5 modules over 5 weeks | No therapist support | Adolescents in the school-based sample completed significantly more exercises than those in community sample. Setting (school), gender (female) predicted greater adherence. |
| O'Kearney et al. (2009) | Australia | Controlled trial | Adolescent females: 15-16 years | Intervention group n = 67; Control n = 90 | CBT targeting depression and coping skills | 5 modules over 6 weeks (self-paced) | No therapist support | Immediately following intervention there were small, non-significant effects for depression scores for the total sample (d = 0.19) and for the group with high initial depression scores (d = 0.12). At follow-up there was a significant moderate effect on depression scores for the total sample (d = 0.46) and a moderate to large significant effect for the group with high depression scores at intake (d = 0.92). |
| Perry et al. (2017) | Australia | Cluster RCT | Adolescents: 16-17 years | Intervention group n = 242; Control n = 298 | Interactive program using format of a fantasy game providing CBT skills to target depression, anxiety and stress | 7 modules over 5-7 weeks | No therapist support | Participants in the intervention condition experienced small, significantly greater reductions in depression scores (compared to control condition) at post intervention (d = 0.29); and 6-month follow-up (d = 0.21). There was a small, non-significant effect (intervention to control) at 18-month follow up (d = 0.33). |
| Redzic et al. (2014) | USA | Development and evaluation study | Adolescents: 14-15 years | Sample 1 n = 158; Sample 2n=132 | Positive psychology and CBT approaches aimed at prevention of depression | 8 sessions over 8 weeks | Therapist monitoring via discussion board | After program was improved (i.e., when comparing sample 1 to sample 2), sample 2 found the intervention significantly more helpful (d = 1.2), interesting (d = 1.1) and more fun (d = 1.1) than sample 1. |
| Rice et al. (2018) | Australia | Pilot study | Adolescents and young adults: 15-25 years | 42 | Positive psychology, mindfulness and strength-based intervention targeting symptoms of depression and improved social functioning | 56 (optional) therapy steps, each 20 minutes long and selected according to clients' needs | Therapist moderators (and also peer support) | Change between baseline and 12 week follow up indicated significant improvement, with small-moderate effect, in depression scores (d = 0.45). 5 participants met criteria for MDD at baseline and were all in remission at 12 weeks follow up. |
| Richards et al. (2016) | USA | RCT | Adolescents and young adults: 14-21 years | 44 | Behavioural activation CBT, interpersonal psychotherapy and community resilience approaches targeting depression | 14 modules | Brief therapist contact at intake and follow up and social worker contact via phone | Participants reported significant decreases in depression symptoms using from baseline to 2.5 years post intervention. There were no significant changes in self-harm risk assessment or serious thoughts of suicide at 2.5 year follow up (possibly due to small number having any at baseline) |
| Rickhi et al. (2015) | Canada | Parallel group, randomized controlled clinical pilot trial | Adolescents and young adults: 13-24 years | Intervention group n = 31; Control n = 31 | Exploration of spiritually informed principles (e.g. forgiveness, gratitude, compassion) targeting depression | 8 modules over 8 weeks | No therapist contact (but participants could continue their regular treatment) | Average depression scores significantly decreased from baseline to 8 weeks and there was also a significant decrease from week 16 to week 24 for the intervention group. The difference between baseline and week 8 was not significant for the waitlist group, but after intervention (week 16) score significantly decreased and this trend continued week 16 to week 24. |
| Robinson et al. (2016) | Australia | Pilot study (pre-post design) | Adolescents: 14-18 years | 21 | Internet-based CBT targeted at young people experiencing suicidal ideation | 8 modules over 8 weeks | Therapist assisted (in person and online) | Participants experienced significant decreases in suicide ideation, with a moderate effect size (d = 0.66), depressive symptoms, with small-moderate-effect sizes (d = 0.60; d = 0.48), and hopelessness, with a small effect size (d = 0.40). |
| Robinson et al. (2014) | Australia | Pilot study (pre-post design) and acceptability study | Adolescents: 14-18 years | 21 | Internet-based CBT targeted at young people experiencing suicidal ideation | 8 modules over 8 weeks | Therapist assisted (in person and online) | Study purpose was to examine acceptability. Suicide ideation and distress remained the same or decreased over the course of each module. One participant found programme moderately distressing, four found it mildly distressing and 16 reported they did not find it anyway distressing. |
| Santesteban-Echarri et al. (2017) | Australia | Qualitative post-intervention evaluation (semi-structured interviews and focus groups) | Adolescents and young adults: 15-25 years | 38 | Positive psychology, mindfulness, CBT and strength-based approach targeting depression | 56 (optional) therapy steps, each 20 minutes long and selected according to clients' needs | Therapist moderators (and also peer support) | At 12 week follow up, all participants found the site helpful and said they would recommend it; 26% of participants reported therapy content as most liked component; 47% of participants reported social networking as most liked component. |
| Silfvernagel et al. (2015) | Sweden | Pilot effectiveness study | Adolescents: 15-19 years | 11 | CBT targeting anxiety and possible comorbid depression | 6-9 modules over 6-18 weeks (depending on client) | Therapist assisted (face to face, phone, email) | Scores pre-post treatment indicated large effect sizes and significant decreases in anxiety scores (d = 2.51); fewer negative thoughts (d = 1.15); increases in psychological wellbeing (d = 1.29) and decreases in depressive symptoms (d= 1.51). |
| Smith et al. (2015) | UK | RCT (intervention and waitlist control) | Adolescents: 12-16 years | Intervention condition n = 55; Waitlist control condition n = 57 | Computerised CBT (cCBT) program targeting mild to moderate depression | 8 sessions over 8 weeks | No therapist assistance (school setting) | From baseline to post-intervention self-reported depression scores were lower for intervention (compared to waitlist) group, with a larger effect size (d = 0.82) and anxiety scores were also lower, with a small effect size (d = 0.41). |
| Spence et al. (2011) | Australia | RCT | Adolescents: 12-18 years | Internet intervention condition n = 44; Clinic intervention condition n = 44; Waitlist condition n = 27 | CBT targeting anxiety | 10 sessions over 10 weeks | Therapist assisted | Rate of session completion was significantly slower in the online intervention compared to clinic intervention, (eta-squared = 0.15). The online and clinical groups evidenced significantly lower clinical severity (anxiety) ratings from baseline to 12 weeks (d = 2.12 and d = 3.42 respectively), and improved global assessment scores (d = 1.56 and d = 2.11 respectively). The waitlist condition showed no significant change on these two measures. Overall, there was no significant difference between online and clinic groups in mean number of sessions completed but the online group had a significantly lower proportion of participants complete all sessions. |
| Stjerneklar et al. (2018) | Denmark | Feasibility study | Adolescents: 13-17 years | 6 | CBT targeting anxiety | 8 modules over 12 weeks | Therapist assisted (phone and feedback on worksheets) | Participants evidenced significant decreases in clinical severity ratings of anxiety pre to post intervention (d = 1.36) and in anxiety symptoms pre to post intervention (d = 0.45). Worsening symptoms, with a small effect size were reported at post to follow-up time point (d = 0.28). |
| Tillfors et al. (2011) | Sweden | RCT (intervention and waitlist control) | Adolescent and young adults: 15-21 years | Intervention condition n =10; Waitlist condition n = 9 | CBT self-help manual targeting social anxiety | 9 modules over 9 weeks | Therapist assisted (email and phone) | Pre-post analyses indicated that the intervention group reported significantly lower anxiety (d = 1.47) and depression scores (d = 1.39) compared to the waitlist condition but there was no significant difference in quality of life scores. |
| Topooco et al. (2018) | Sweden | RCT | Adolescents: 15-19 years. | Intervention condition n = 33, Control condition n = 37 | Internet-based CBT targeting depression | 8 modules over 8 weeks | Therapist assisted | Pre-post analyses indicated a significant improvement for intervention condition participants compared to controls on depression (d = 0.71) and self-efficacy (d = 1.33) but no significant differences in satisfaction with life and other measures of anxiety/depression. |
| Topper et al. (2017) | The Netherlands | RCT | Adolescents and young adults:15-22 years | Group intervention n = 82; Internet intervention n = 84; Waitlist control n = 85 | Modified rumination focussed CBT to target excessive worry and rumination | 6 sessions over 6 weeks | Therapist assisted | Pre-post analyses indicated that the internet and group intervention conditions evidenced significant reductions in repetitive negative thinking (d = 0.79 and d = 0.98 respectively) and depressive symptoms (d = 0.51 and d = 0.80 respectively). The waitlist control group showed no significant differences on these measures. |
| van der Zanden et al. (2012) | The Netherlands | RCT | Adolescents and young adults: 16-25 years. | Intervention condition n = 121; Control condition n = 123 | Structured CBT targeting depression | 6 sessions over 6 weeks | Therapist assisted | Significantly greater improvement was demonstrated for the intervention group compared to the control group: in depression symptoms (d = 0.94), anxiety symptoms (d = 0.49) and mastery of control (d = 0.44) during the baseline to 12 week time period. |
| Waite et al. (2019) | UK | RCT | Adolescents: 13-18 years | Intervention condition n = 30; Waitlist condition n = 30 | CBT targeting anxiety | 10 sessions over 10 weeks with 2 booster sessions | Therapist assisted | There was no significant difference between groups in post-treatment anxiety diagnostic status. There was a significant difference in global clinical improvement, with the intervention group showing greater improvement than the waitlist group. |
Note: Adolescents were classified as individuals aged 10-19 years (WHO, 2020a), children were those under 10 years, and young adults were aged 20-25 years. RCT, randomised controlled trial; CBT, cognitive behaviour therapy; MDD, major depressive disorder.
Table 3.
Summary of intervention components and features.
| Component/Feature | Description |
|---|---|
| Information | • Presented using text, audio or video |
| Real life case examples (age appropriate) | • Presented via text, videos (either interviews or actors), vignettes, cartoons |
| • Typically used to depict/normalise common symptoms or experiences | |
| Homework | • Set at the end of the module and based on the information given in the module |
| • Usually to be completed before following module is accessed | |
| • At the beginning of subsequent module participants were often asked to either reflect on homework or input what they did | |
| • Often required the client to personalise homework (e.g., apply skill in a certain situation) | |
| • Sometimes included a measure of mood or other assessment | |
| Demonstration videos | • Demonstrations around how to use a skill/strategy |
| • Video diaries | |
| Worksheets and resources | • Either completed online or downloadable and sent via email to therapist |
| • Downloadable factsheets | |
| Interactive exercises | • Quizzes (to check understanding of content) |
| • Fill in the blanks | |
| • Online worksheets | |
| • Interactive skills activities (i.e. step ladders, support network webs) | |
| Progress tracking | • Tracking change in symptoms using mood/ anxiety measures (often taken weekly) |
| • Track progress of intervention (e.g., “You have completed 7 out of 10 modules”) | |
| Workbooks | • Answers to exercises recorded |
| • Could be accessed at any time after completion | |
| Online Journal | • To track progress, record notes, record thoughts, record experiences of trying skills and homework, record challenges and hurdles |
| Characters | • Use of characters/ personal or therapist avatars |
| Automatic alerts | • In the form of emails/ text messages to inform when a new module is ready, congratulate on completing module, remind participant to complete module |
| • Can be personalised (i.e. addressed to the person) | |
| Parent information/sessions | • Hard-copy workbook |
| • Intervention/sessions for parents | |
| • Information brochures and factsheets | |
| • Access to therapist to ask questions | |
| Crisis plan | • Individualised crisis plan that can be accessed at any time |
| Music and relaxation | • Dedicated mindfulness and relaxation sections |
| • Audio and music clips for mindfulness that can be downloaded by participants | |
| Chat rooms and social networking | • Some interventions encouraged peer-to-peer discussion (requires therapist monitoring/moderation) |
References
- Abou-Setta A. M., Jeyaraman M., Attia A., Al-Inany H. G., Ferri M., Ansari M. T., Norris S. L. (2016). Methods for developing evidence reviews in short periods of time: a scoping review. PLoS One, 11(12), e0165903. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Adams J., Hillier-Brown F. C., Moore H. J., Lake A. A., Araujo-Soares V., White M., Summerbell C., (2016). Searching and synthesising ‘grey literature’and ‘grey information’in public health: critical reflections on three case studies. Systematic Reviews, 5(1), 164. [DOI] [PMC free article] [PubMed] [Google Scholar]
- AIMhi Stay Strong App. (2019). Australia: Menzies school of health research. Retrieved from: https://www.menzies.edu.au/page/Resources/Stay_Strong_iPad_App/ [Google Scholar]
- Alvarez-Jimenez M., Bendall S., Lederman R., Wadley G., Chinnery G., Vargas S., Gleeson J. F. (2013). On the HORYZON: Moderated online social therapy for long-term recovery in first episode psychosis. Schizophrenia Research, 143(1), 143-149. doi:10.1016/j.schres.2012.10.009 [DOI] [PubMed] [Google Scholar]
- Alvarez-Jimenez M., Gleeson J. F., Bendall S., Penn D. L., Yung A. R., Ryan R. M., Nelson B. (2018). Enhancing social functioning in young people at Ultra High Risk (UHR) for psychosis: A pilot study of a novel strengths and mindfulness-based online social therapy. Schizophrenia Research, 202, 369-377. doi:10.1016/j.schres.2018.07.022 [DOI] [PubMed] [Google Scholar]
- Álvaro F., Navarro S., Palma C., Farriols N., Aliaga F., Solves L., Riera A. (2020). Clinical course and predictors in patients with borderline personality disorder during the COVID-19 outbreak: A 2.5-month naturalistic exploratory study in Spain. Psychiatry research, 292. [DOI] [PMC free article] [PubMed] [Google Scholar]
- American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington: American Psychiatric Association. [Google Scholar]
- Anderson R. E., Spence S. H., Donovan C. L., March S., Prosser S., Kenardy J., (2012). Working alliance in online cognitive behavior therapy for anxiety disorders in youth: comparison with clinic delivery and its role in predicting outcome. Journal of Medical Internet Research, 14(3). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Andrews G., Cuijpers P., Craske M. G., McEvoy P., Titov N., (2010). Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS One, 5(10). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Anxiety Reliever. (2015). Anxiety Reliever, LLC. Retrieved from: https://www.anxietyrelieverapp.com/ [Google Scholar]
- Bannink R., Broeren S., Joosten-van Zwanenburg E., van As E., van de Looij-Jansen P., Raat H., (2014). Effectiveness of a web-based tailored intervention (E-health4Uth) and consultation to promote adolescents’ health: Randomized controlled trial. Journal of Medical Internet Research, 16(5), 51-66. doi:10.2196/jmir.3163 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Beating the blues. (n.d.). 365 Health Solutions Limited. Retrieved from: https://www.beatingtheblues.co.uk/ [Google Scholar]
- Benzies K. M., Premji S., Hayden K. A., Serrett K., (2006). State-of-the-evidence reviews: advantages and challenges of including grey literature. Worldviews on Evidence-Based Nursing, 3(2), 55-61. [DOI] [PubMed] [Google Scholar]
- BetterBET. (n.d.). University of Carlifornia, Palo Alto University Retrieved from https://ucsf.co1.qualtrics.com/jfe/form/SV_0vc9VjaJ9Q0FvXS?gclid=CjwKCAiAmNbwBRBOEiwAqcwwpWD7bJRD3nvqJ9TNmSC3MefegLTD2oxDSnEOii74ujx-F4QrBW6mxoCnIUQAvD_BwE [Google Scholar]
- Bite Back. (2020). Black Dog Institute. Retrieved from: https://www.biteback.org.au/ [Google Scholar]
- Bjureberg J., Sahlin H., Hedman-Lagerlöf E., Gratz K. L., Tull M. T., Jokinen J., Ljótsson B. (2018). Extending research on Emotion Regulation Individual Therapy for Adolescents (ERITA) with nonsuicidal self-injury disorder: Open pilot trial and mediation analysis of a novel online version. BMC Psychiatry, 18 Retrieved from: http://ezproxy.uow.edu.au/login?url=https://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2018-51505-001&site= ehost-live [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bobier C., Stasiak K., Mountford H., Merry S., Moor S., (2013). When ‘e’ therapy enters the hospital: Examination of the feasibility and acceptability of SPARX (a cCBT programme) in an adolescent inpatient unit. Advances in Mental Health, 11(3), 286-292. doi:10.5172/jamh.2013.11.3.286 [Google Scholar]
- Bowlby J. (1979). The Bowlby-Ainsworth attachment theory. Behavioral and Brain Sciences, 2(4), 637-638. [Google Scholar]
- (The) BRAVE program. (2010). St Lucia, QLD: Beyond Blue. Retrieved from: https://brave4you.psy.uq.edu.au/teen-program [Google Scholar]
- Calear A. L., Christensen H., Mackinnon A., Griffiths K. M., O’Kearney R., (2009). The YouthMood Project: a cluster randomized controlled trial of an online cognitive behavioral program with adolescents. Journal of Consulting and Clinical Psychology, 77(6), 1021. [DOI] [PubMed] [Google Scholar]
- Calear A. L., Christensen H., (2010). Review of internetbased prevention and treatment programs for anxiety and depression in children and adolescents. Medical Journal of Australia, 192, S12-S14. 10.5694/j.1326-5377.2010.tb03686.x [DOI] [PubMed] [Google Scholar]
- Calm Harm app. (2019). Stem4. Retrieved from: https://calmharm.co.uk/ [Google Scholar]
- Chanen A. M., Jovev M., Jackson H. J., (2007). Adaptive functioning and psychiatric symptoms in adolescents with borderline personality disorder. Journal of Clinical Psychiatry, 68(2), 297. [DOI] [PubMed] [Google Scholar]
- Chanen A. M., Mccutcheon L. K., Germano D., Nistico H., Jackson H. J., Mcgorry P. D. (2009). The HYPE Clinic: an early intervention service for borderline personality disorder. Journal of Psychiatric Practice, 15(3), 163-172. [DOI] [PubMed] [Google Scholar]
- Chanen A. M., McCutcheon L. (2013). Prevention and early intervention for borderline personality disorder: current status and recent evidence. British Journal of Psychiatry, 202(s54), s24-s29. 10.1192/bjp.bp.112.119180 [DOI] [PubMed] [Google Scholar]
- Chapman R., Loades M., O’Reilly G., Coyle D., Patterson M., Salkovskis P., (2016). ‘Pesky gNATs’: Investigating the feasibility of a novel computerized CBT intervention for adolescents with anxiety and/or depression in a Tier 3 CAMHS setting. The Cognitive Behaviour Therapist, 9. [Google Scholar]
- Chilled Out Online. (2020). Macquarie University. Retrieved from https://www.mq.edu.au/about/campus-services-and-facilities/hospital-and-clinics/centre-for-emotional-healthclinic/programs-for-children-and-teenagers/online-treatment-accordions/chilled-out-online [Google Scholar]
- Christensen H., Batterham P. J., O’Dea B., (2014). E-health interventions for suicide prevention. International Journal of Environmental Research and Public Health, 11(8), 8193-8212. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cochrane Effective Practice and Organisation of Care (EPOC). EPOC Resources for review authors, 2017. Available from: epoc.cochrane.org/resources/epoc-resources-review-authors [Google Scholar]
- Cristea I. A., Gentili C., Cotet C. D., Palomba D., Barbui C., Cuijpers P., (2017). Efficacy of psychotherapies for borderline personality disorder: a systematic review and metaanalysis. JAMA Psychiatry, 74(4), 319-328. [DOI] [PubMed] [Google Scholar]
- Clarke A., Kuosmanen T., Barry M., (2015). A systematic review of online youth mental health promotion and prevention interventions. Journal of Youth & Adolescence, 44(1), 90-113. 10.1007/s10964-014-0165-0 [DOI] [PubMed] [Google Scholar]
- Clear Fear app. (2019). Stem4. Retrieved from: https://www.clearfear.co.uk/ [Google Scholar]
- de Voogd E. L., Wiers R. W., Zwitser R. J., Salemink E., (2016). Emotional working memory training as an online intervention for adolescent anxiety and depression: A randomised controlled trial. Australian Journal of Psychology, 68(3), 228-238. doi:10.1111/ajpy.12134 [DOI] [PMC free article] [PubMed] [Google Scholar]
- de Voogd L., Wiers R. W., de Jong P. J., Zwitser R. J., Salemink E., (2018). A randomized controlled trial of multisession online interpretation bias modification training: Short- and long-term effects on anxiety and depression in unselected adolescents. PLoS One, 13(3). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dear B. F., Fogliati V. J., Fogliati R., Johnson B., Boyle O., Karin E., Titov N. (2018). Treating anxiety and depression in young adults: A randomised controlled trial comparing clinician-guided versus self-guided Internet-delivered cognitive behavioural therapy. Australian & New Zealand Journal of Psychiatry, 52(7), 668-679. [DOI] [PubMed] [Google Scholar]
- Dealing with Depression (DWD). (n.d.). Canada: Centre for Applied Research in Mental Health and Addiction. Retrieved from: https://dwdonline.ca/ (The)Desk. (2020). Australia: University of Queensland; Retrieved from: https://www.thedesk.org.au/login?login [Google Scholar]
- Diener M. J., Monroe J. M., (2011). The relationship between adult attachment style and therapeutic alliance in individual psychotherapy: A meta-analytic review. Psychotherapy, 48(3), 237. [DOI] [PubMed] [Google Scholar]
- Fonagy P., Target M., Gergely G., (2000). Attachment and borderline personality disorder: A theory and some evidence. Psychiatric Clinics, 23(1), 103-122. [DOI] [PubMed] [Google Scholar]
- E couch. (2018). eHub Health Pty Limited. Retrieved from: https://ecouch.anu.edu.au/new_users/welcome01 [Google Scholar]
- Gladstone G., Marko-Holguin M., Rothberg P., Nidetz J., Diehl A., DeFrino D. T., Van Voorhees B. W. (2015). An internet-based adolescent depression preventive intervention: Study protocol for a randomized control trial. Trials, 16(1). doi:10.1186/s13063-015-0705-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gladstone T., Marko-Holguin M., Henry J., Fogel J., Diehl A., Van Voorhees B. W., (2014). Understanding adolescent response to a technology-based depression prevention program. Journal of Clinical Child and Adolescent Psychology, 43(1), 102-114. doi:10.1080/15374416.2013. 850697 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Godin K., Stapleton J., Kirkpatrick S. I., Hanning R. M., Leatherdale S. T., (2015). Applying systematic review search methods to the grey literature: a case study examining guidelines for school-based breakfast programs in Canada. Systematic Reviews, 4(1), 138. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Grenyer B. F. (2014). An integrative relational step-down model of care: The Project Air Strategy for Personality Disorders. The ACPARIAN, 9(8-13). [Google Scholar]
- Grenyer B. F. S., Ng F. Y., Townsend M. L., Rao S., (2017). Personality disorder: A mental health priority area. Australian & New Zealand Journal of Psychiatry, 51(9), 872-875. [DOI] [PubMed] [Google Scholar]
- Ha C., Balderas J. C., Zanarini M. C., Oldham J., Sharp C., (2014). Psychiatric comorbidity in hospitalized adolescents with borderline personality disorder. J Clin Psychiatry, 75(5), e457-464. [DOI] [PubMed] [Google Scholar]
- Haby M. M., Chapman E., Clark R., Barreto J., Reveiz L., Lavis J. N., (2016). What are the best methodologies for rapid reviews of the research evidence for evidence-informed decision making in health policy and practice: a rapid review. Health Research Policy and Systems, 14(1), 83. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Haddaway N. R., Collins A. M., Coughlin D., Kirk S., (2015). The role of Google Scholar in evidence reviews and its applicability to grey literature searching. PLoS One, 10(9), e0138237. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Haidich A. B. (2010). Meta-analysis in medical research. Hippokratia, 14(Suppl 1), 29. [PMC free article] [PubMed] [Google Scholar]
- Hedman E., Andersson G., Ljótsson B., Andersson E., Rück C., Mörtberg E., Lindefors N., (2011). Internet-based cognitive behavior therapy vs. cognitive behavioral group therapy for social anxiety disorder: a randomized controlled non-inferiority trial. PLoS One, 6(3), e18001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hetrick S., Yuen H. P., Cox G., Bendall S., Yung A., Pirkis J., Robinson J., (2014). Does cognitive behavioural therapy have a role in improving problem solving and coping in adolescents with suicidal ideation? The Cognitive Behaviour Therapist, 7. doi:10.1017/S1754470X14000129 [Google Scholar]
- Hetrick S. E., Yuen H. P., Bailey E., Cox G. R., Templer K., Rice S. M., Robinson J. (2017). Internet-based cognitive behavioural therapy for young people with suiciderelated behaviour (Reframe-IT): A randomised controlled trial. Evidence-Based Mental Health, 20(3), 76-85. doi:10.1136/eb-2017-102719 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ip P., Chim D., Chan K. L., Li T. M. H., Ho F. K. W., Voorhees B. W., Wong W. H. S. (2016). Effectiveness of a culturally attuned Internet-based depression prevention program for Chinese adolescents: A randomized controlled trial. Depression and Anxiety, 33(12), 1123-1131. doi: 10.1002/da.22554 [DOI] [PubMed] [Google Scholar]
- Jackson H. J., Burgess P. M. (2000). Personality disorders in the community: a report from the Australian National Survey of Mental Health and Wellbeing. Social Psychiatry and Psychiatric Epidemiology, 35, 531-538. [DOI] [PubMed] [Google Scholar]
- Johansson R., Ekbladh S., Hebert A., Lindström M., Möller S., Petitt E., Cuijpers P. (2012). Psychodynamic guided self-help for adult depression through the internet: a randomised controlled trial. PLoS One, 7(5), e38021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Johnson J. G., Cohen P., Kasen S., Skodol A. E., Hamagami F., Brook J. S., (2000). Age-related change in personality disorder trait levels between early adolescence and adulthood: a community-based longitudinal investigation. Acta Psychiatrica Scandinavica, 102(4), 265-275. 10.1034/j.1600-0447.2000.102004265.x [DOI] [PubMed] [Google Scholar]
- Kaess M., Brunner R., Chanen A., (2014). Borderline personality disorder in adolescence. Pediatrics, 134(4), 782-793. [DOI] [PubMed] [Google Scholar]
- Karbasi A., Haratian A., (2018). The efficacy of internetbased cognitive behavioral therapy on the anxiety disorders among adolescent girls. Advanced Biomedical Research, 7(1), 13. doi: 10.4103/abr.abr_203_16 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Korzekwa M. I., Dell P. F., Links P. S., Thabane L., Webb S. P., (2008). Estimating the prevalence of borderline personality disorder in psychiatric outpatients using a two-phase procedure. Comprehensive Psychiatry, 49(4), 380-386. [DOI] [PubMed] [Google Scholar]
- Kooth. (2020). USA: XenZone. Retrieved from https://www.kooth.com/ [Google Scholar]
- Kramer J., Conijn B., Oijevaar P., Riper H., (2014). Effectiveness of a web-based solution-focused brief chat treatment for depressed adolescents and young adults: Randomized controlled trial. Journal of Medical Internet Research, 16(5), 40-50. doi:10.2196/jmir.3261 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kruger J. R., Kim P., Iyer V., Marko-Holguin M., Fogel J., De-Frino D., Van Voorhees B. W. (2017). Evaluation of protective and vulnerability factors for depression following an internet-based intervention to prevent depression in at-risk adolescents. International Journal of Mental Health Promotion, 19(2), 69-84. doi:10.1080/14623730.2017.1308264 [Google Scholar]
- Kuosmanen T., Fleming T. M., Barry M. M., (2018). The implementation of SPARX-R computerized mental health program in alternative education: Exploring the factors contributing to engagement and dropout. Children and Youth Services Review, 84, 176-184. doi:10.1016/j.childyouth.2017.11.032 [Google Scholar]
- Kurki M., Anttila M., Koivunen M., Marttunen M., Välimäki M., (2018). Nurses’ experiences of the use of an Internet-based support system for adolescents with depressive disorders. Informatics for Health & Social Care, 43(3), 234-247. doi:10.1080/17538157.2016.1269110 [DOI] [PubMed] [Google Scholar]
- Levy K. N., Kivity Y., Johnson B. N., Gooch C. V., (2018). Adult attachment as a predictor and moderator of psychotherapy outcome: A meta-analysis. Journal of Clinical Psychology, 74(11), 1996-2013 [DOI] [PubMed] [Google Scholar]
- Levy K. N., McMain S., Bateman A., Clouthier T. (2018). Treatment of borderline personality disorder. Psychiatric Clinics of North America, 41(4), 711-728. 10.1016/j.psc.2018.07.011 [DOI] [PubMed] [Google Scholar]
- Lewis K. L., Fanaian M., Kotze B., Grenyer B. F. S. (2019). Mental health presentations to acute psychiatric services: 3- year study of prevalence and readmission risk for personality disorders compared with psychotic, affective, substance or other disorders. British Journal of Psychiatry Open, 5, 1-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lewis S. P., Heath N. L., Michal N. J., Duggan J. M., (2012). Non-suicidal self-injury, youth, and the Internet: What mental health professionals need to know. Child and Adolescent Psychiatry and Mental Health, 6(1), 13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lillevoll K. R. Vangberg H. C. B., Griffiths K. M. Waterloo K. Eisemann M. R., (2014). Uptake and adherence of a self-directed internet-based mental health intervention with tailored e-mail reminders in senior high schools in Norway. BMC Psychiatry, 14. doi:10.1186/1471-244X-14-14 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lindqvist K., Mechler J., Carlbring P., Lilliengren P., Falkenström F., Andersson G., Bergsten K. L. (2020). Affect- focused psychodynamic internet-based therapy for adolescent depression: randomized controlled trial. Journal of Medical Internet Research, 22(3), e18047. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Living Life to the Full. (2020). Five Areas Limited. Retrieved from: https://llttf.com/ [Google Scholar]
- March S., Spence S. H., Donovan C. L., Kenardy J. A., (2018). Large-scale dissemination of internet-based cognitive behavioral therapy for youth anxiety: Feasibility and acceptability study. Journal of Medical Internet Research, 20(7). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mayo Clinic Anxiety Coach. (2020). Mayo Foundation for Medical Education and Research. Retrieved from: https://anxietycoach.mayoclinic.org/anxiety/mentalhealthonline (2020). Aus: Swinburne University of Technology; Retrieved from: https://www.mentalhealthonline.org.au/Default.aspx [Google Scholar]
- Meuldijk D., McCarthy A., Bourke M. E., Grenyer B. F. S. (2017). The value of psychological treatment for borderline personality disorder: Systematic review and cost offset analysis of economic evaluations. PLoS One, 12, e0171592. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Miller A. L., Muehlenkamp J. J., Jacobson C. M., (2008). Fact or fiction: diagnosing borderline personality disorder in adolescents. Clinical Psychology Reviews, 28(6), 969-981. 10.1016/j.cpr.2008.02.004 [DOI] [PubMed] [Google Scholar]
- Mindshift CBT. (2019). Canada: Anxiety Canada. Retrieved from: https://www.anxietycanada.com/resources/mindshift-cbt/ [Google Scholar]
- MindSpot Indigenous Wellbeing Course. (2020). MindSpot, Macquarie University. Retrieved from: https://mindspot.org.au/indigenous-wellbeing [Google Scholar]
- Mindspot Mood Mechanic Course. (2020). Australia: Minspot, Macquarie University. Retrieved from: https://mindspot.org.au/mood-mechanic-coursemoodgym (2020). Australia: ehub Health; Retrieved from: https://moodgym.com.au/ [Google Scholar]
- Montague A. E., Varcin K. J., Simmons M. B., Parker A. G., (2015). Putting technology into youth mental health practice: Young people’s perspectives. SAGE Open, 5(2), 2158244015581019. doi:10.1177/2158244015581019 [Google Scholar]
- MoodKit. (2020). USA: Thriveport, LLC. Retrieved from: http://www.thriveport.com/products/moodkit/ [Google Scholar]
- MoodMission. (2019). USA: MoodMission Ptd Ltd. Retrieved from: http://moodmission.com/ [Google Scholar]
- Munn Z., Peters M. D. J., Stern C., Tufanaru C., McArthur A., Aromataris E., (2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology, 18(1), 143. doi:10.1186/s12874-018-0611-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- My Anxiety Plan (MAP) for Children and Teens. (2019). Canada: Anxiety Canada; Retrieved from: https://maps.anxietycanada.com/courses/child-map/ [Google Scholar]
- My Digital Health. (2020). Australia: My Digital Health. Retrieved from: https://www.mydigitalhealth.org.au/myCompass (2020). Australia: Black Dog Institute; Retrieved from: https://www.mycompass.org.au/ [Google Scholar]
- NHMRC (2012). Clinical practice guidelines for the management of Borderline Personality Disorder. Melbourne, VIC, Australia: National Health and Medical Research Council. [Google Scholar]
- Neil A. L., Batterham P., Christensen H., Bennett K., Griffiths K. M., (2009). Predictors of adherence by adolescents to a cognitive behavior therapy website in school and community- based settings. Journal of Medical Internet Research, 11(1), 42-49. doi:10.2196/jmir.1050 [DOI] [PMC free article] [PubMed] [Google Scholar]
- O’Kearney R., Kang K., Christensen H., Griffiths K., (2009). A controlled trial of a school-based Internet program for reducing depressive symptoms in adolescent girls. Depression and Anxiety, 26(1), 65-72. doi:10.1002/da.20507 [DOI] [PubMed] [Google Scholar]
- Perry Y. P., Werner-Seidler A. P., Calear A. P., Mackinnon A. P., King C., Clin M. P., Batterham P. J. P. (2017). Preventing depression in final year secondary students: schoolbased randomized controlled trial. Journal of Medical Internet Research, 19(11). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Poletti B., Tagini S., Brugnera A., Parolin L., Pievani L., Ferrucci R., Silani V. (2020). Telepsychotherapy: a leaflet for psychotherapists in the age of COVID-19. A review of the evidence. Counselling Psychology Quarterly, 1-16. doi:10.1080/09515070.2020.1769557 [Google Scholar]
- Power E., Hughes S., Cotter D., Cannon M., (2020). Youth mental health in the time of COVID-19. Irish Journal of Psychological Medicine, 1-15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pretorius N., Arcelus J., Beecham J., Dawson H., Doherty F., Eisler I., Jones A. (2009). Cognitive-behavioural therapy for adolescents with bulimic symptomatology: the acceptability and effectiveness of internet-based delivery. Behaviour Research and Therapy, 47(9), 729-736. [DOI] [PubMed] [Google Scholar]
- Redzic N. M., Taylor K., Chang V., Trockel M., Shorter A., Taylor C. B., (2014). An internet-based positive psychology program: Strategies to improve effectiveness and engagement. The Journal of Positive Psychology, 9(6), 494-501. doi:10.1080/17439760.2014.936966 [Google Scholar]
- Rice S., Gleeson J., Davey C., Hetrick S., Parker A., Lederman R., Mario A. J. (2018). Moderated online social therapy for depression relapse prevention in young people: pilot study of a ‘next generation’ online intervention. Early Intervention in Psychiatry, 12(4), 613-625. doi: 10.1111/eip.12354 [DOI] [PubMed] [Google Scholar]
- Richards C., Cogan N., Hacker T., Amin M., Beoing L., (2004). Stress and anxiety in teenagers. UK: National Health Service (NHS) Retrieved from http://stressandanxietyinteenagers.com/ [Google Scholar]
- Richards K., Marko-Holguin M., Fogel J., Anker L., Ronayne J., Van Voorhees B. W., (2016). Randomized clinical trial of an Internet-based intervention to prevent adolescent depression in a primary care setting (CATCH-IT): 25-year outcomes. Journal of Evidence-Based Psychotherapies, 16(2), 113-134. [PMC free article] [PubMed] [Google Scholar]
- Richardson T., Stallard P., Velleman S., (2010). Computerised cognitive behavioural therapy for the prevention and treatment of depression and anxiety in children and adolescents: A systematic review. Clinical Child and Family Psychology Review, 13(3), 275-290. [DOI] [PubMed] [Google Scholar]
- Rickhi B., Kania-Richmond A., Moritz S., Cohen J., Paccagnan P., Dennis C., Toews J. (2015). Evaluation of a spirituality informed e-mental health tool as an intervention for major depressive disorder in adolescents and young adults - a randomized controlled pilot trial. BMC Complementary and Alternative Medicine, 15(1). doi:10.1186/s12906-015-0968-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- Robinson J., Hetrick S., Cox G., Bendall S., Yuen H. P., Yung A., Pirkis J., (2016). Can an internet-based intervention reduce suicidal ideation, depression and hopelessness among secondary school students: Results from a pilot study. Early Intervention in Psychiatry, 10(1), 28-35. doi:10.1111/ eip.12137 [DOI] [PubMed] [Google Scholar]
- Robinson J., Hetrick S., Cox G., Bendall S., Yung A., Yuen H. P., Pirkis J. (2014). The development of a randomised controlled trial testing the effects of an online intervention among school students at risk of suicide. BMC Psychiatry, 14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Santesteban-Echarri O., Rice S., Wadley G., Lederman R., D’Alfonso S., Russon P., Álvarez-Jiménez M. (2017). A next-generation social media-based relapse prevention intervention for youth depression: Qualitative data on user experience outcomes for social networking, safety, and clinical benefit. Internet Interventions, 9, 65-73. doi:10.1016/j.invent.2017.06.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Serafini G., Parmigiani B., Amerio A., Aguglia A., Sher L., Amore M. The psychological impact of COVID-19 on the mental health in the general population. QJM: An International Journal of Medicine, 1, 7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sher L. (2020). Psychiatric disorders and suicide in the COVID- 19 era. QJM: An International Journal of Medicine, 113(8), 527-528. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Silfvernagel K., Gren-Landell M., Emanuelsson M., Carlbring P., Andersson G., (2015). Individually tailored internetbased cognitive behavior therapy for adolescents with anxiety disorders: A pilot effectiveness study. Internet Interventions, 2(3), 297-302. doi:10.1016/j.invent.2015.07.002 [Google Scholar]
- Smith P., Scott R., Eshkevari E., Jatta F., Leigh E., Harris V., Yule W. (2015). Computerised CBT for depressed adolescents: Randomised controlled trial. Behaviour Research and Therapy, 73, 104-110. doi:10.1016/j.brat.2015.07.009 [DOI] [PubMed] [Google Scholar]
- Sparx. (2009). New Zealand: The University of Auckland. Retrieved from: https://www.sparx.org.nz/home [Google Scholar]
- Spence S. H., Donovan C. L., March S., Gamble A., Anderson R. E., Prosser S., Kenardy J., (2011). A randomized controlled trial of online versus clinic-based CBT for adolescent anxiety. Journal of Consulting and Clinical Psychology, 79(5), 629. [DOI] [PubMed] [Google Scholar]
- Stjerneklar S., Hougaard E., Nielsen A. D., Gaardsvig M. M., Thastum M., (2018). Internet-based cognitive behavioral therapy for adolescents with anxiety disorders: A feasibility study. Internet Interventions, 11, 30-40. doi:10.1016/j.invent.2018.01.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Struthers A., Charette C., Bapuji S. B., Winters S., Ye X., Metge C., Sutherland K. (2015). The acceptability of emental health services for children, adolescents, and young adults: a systematic search and review. Canadian Journal of Community Mental Health, 34(2), 1-21. doi:10.7870/cjcmh-2015-006 [Google Scholar]
- Super Better. (2020). USA: SuperBetter, LLC. Retrieved from: https://www.superbetter.com/ [Google Scholar]
- Tempra J. (2019). What’s Up - Mental Health App. Australia: Retrieved from: https://play.google.com/store/apps/details?id=com.jacksontempra.apps.whatsup&hl=en [Google Scholar]
- This Way Up. (2020). Australia: This Way Up St Vincent’s Hospital. Retrieved from: https://thiswayup.org.au/ [Google Scholar]
- This Way Up TeenStrong. (2020). Australia: This Way Up, St Vincent’s hospital. Retrieved from: https://thiswayup.org.au/howwe-can-help/courses/teenstrong/ [Google Scholar]
- Tillfors M., Andersson G., Ekselius L., Furmark T., Lewenhaupt S., Karlsson A., Carlbring P., (2011). A randomized trial of internet-delivered treatment for social anxiety disorder in high school students. Cognitive Behaviour Therapy, 40(2), 147-157. doi:10.1080/16506073.2011.555486 [DOI] [PubMed] [Google Scholar]
- Topooco N., Berg M., Johansson S., Liljethörn L., Radvogin E., Vlaescu G., Andersson G. (2018). Chat- and internet- based cognitive-behavioural therapy in treatment of adolescent depression: randomised controlled trial. BJPsych open, 4(4), 199-207. doi: 10.1192/bjo.2018.18 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Topper M., Emmelkamp P. M. G., Watkins E., Ehring T., (2017). Prevention of anxiety disorders and depression by targeting excessive worry and rumination in adolescents and young adults: A randomized controlled trial. Behaviour Research and Therapy, 90, 123-136. doi:10.1016/j.brat.2016. 12.015 [DOI] [PubMed] [Google Scholar]
- Townsend M. T., Haselton S.J., Marceau E.M., Gray A. S., Grenyer B.F.S. (2018). Adolescent Intervention: Guide for Clinicians. Wollongong: University of Wollongong, Illawarra Health and Medical Research Institute. [Google Scholar]
- van der Zanden R., Kramer J., Gerrits R., Cuijpers P., (2012). Effectiveness of an online group course for depression in adolescents and young adults: A randomized trial. Journal of Medical Internet Research, 14(3), 296-309. doi:10.2196/jmir.2033 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Varker T., Forbes D., Dell L., Weston A., Merlin T., Hodson S., O’Donnell M., (2015). Rapid evidence assessment: increasing the transparency of an emerging methodology. Journal of Evaluation in Clinical Practice, 21(6), 1199-1204. [DOI] [PubMed] [Google Scholar]
- Vigerland S., Lenhard F., Bonnert M., Lalouni M., Hedman E., Ahlen J., Ljotsson B. (2016). Internet-delivered cognitive behavior therapy for children and adolescents: a systematic review and meta-analysis. Clinical Psychology Review, 50, 1-10 [DOI] [PubMed] [Google Scholar]
- Waite P., Marshall T., Creswell C., (2019). A randomized controlled trial of internet-delivered cognitive behaviour therapy for adolescent anxiety disorders in a routine clinical care setting with and without parent sessions. Child and Adolescent Mental Health, 24(3), 242-250. doi:10.1111/camh.12311 [DOI] [PubMed] [Google Scholar]
- Well Mind. (2008). UK: National Health Service. Retrieved from: http://www.dwmh.nhs.uk/wellmind/ [Google Scholar]
- WHO (World Health Organization). (2020a). Adolescent Health. Geneva: World Health Organisation; Retrieved from: https://www.who.int/health-topics/adolescent-health#tab=tab_1 [Google Scholar]
- WHO (World Health Organization). (2020b). Timeline: WHO’s COVID-19 response. Geneva: World Health Organisation; Retrieved from: https://www.who.int/emergencies/diseases/novel-coronavirus-2019/interactive-timeline?gclid=Cj0KCQjw8fr7BRDSARIsAK0Qqr4tj8g_FI8nJ5jUxmcjoXAlXGlDR8WgYnxBm-MAE92bjj9vX3pkov UaAoC5EALw_wcB#! [Google Scholar]
- WHO (World Health Organization). (2018). International classification of diseases for mortality and morbidity statistics. Geneva: World Health Organisation; Retrieved from: https://icd.who.int/en [Google Scholar]
- Wilks C. R., Lungu A., Ang S. Y., Matsumiya B., Yin Q., Linehan M. M., (2018). A randomized controlled trial of an Internet delivered dialectical behavior therapy skills training for suicidal and heavy episodic drinkers. Journal of Affective Disorders, 232, 219-228. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yao H., Chen J. H., Xu Y. F., (2020). Patients with mental health disorders in the COVID-19 epidemic. The Lancet Psychiatry, 7(4), e21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ye X., Bapuji S. B., Winters S. E., Struthers A., Raynard M., Metge C., Sutherland K. (2014). Effectiveness of internet- based interventions for children, youth, and young adults with anxiety and/or depression: A systematic review and meta-analysis. BMC Health Services Research, 14(1). doi:10.1186/1472-6963-14-313 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zhou X., Snoswell C. L., Harding L. E., Bambling M., Edirippulige S., Bai X., Smith A. C., (2020). The role of telehealth in reducing the mental health burden from COVID-19. Telemedicine and e-Health, 26(4), 377-379. [DOI] [PubMed] [Google Scholar]