A randomised controlled trial of a relationship‐focussed mobile phone application for improving adolescents’ mental health

Abstract

Background

This study evaluated the acceptability and effectiveness of a relationship‐focussed mobile phone application (WeClick) for improving depressive symptoms and other mental health outcomes in adolescents.

Methods

A randomised controlled trial involving 193 youth (M age: 14.82, SD: 0.94, 86.5% female) from Australia was conducted. Youth were recruited via the Internet and randomly allocated to the intervention or a 4‐week wait list control condition, stratified for age and gender. The primary outcome was change in depressive symptom scores measured using the Patient Health Questionnaire for Adolescents (PHQ‐A) at baseline, 4‐week post‐test and 12‐week follow‐up. Secondary outcomes included anxiety, psychological distress, wellbeing, help‐seeking intentions for mental health, social self‐efficacy and social support. Participants in the intervention condition received access to the intervention for four weeks. Thematic analysis was utilised to identify and examine acceptability.

Results

The change in PHQ‐A scores from baseline to 4‐week post‐test did not differ significantly (d = 0.26, p = .138) between the intervention (Mchange = −2.9, SD = 5.3) and wait list control conditions (Mchange = −1.7, SD = 4.3). However, significant between‐group improvements were observed in wellbeing (d = 0.37, p = .023), help‐seeking intentions (d = 0.36, p = .016) and professional help‐seeking intentions for mental health problems (d = 0.36, p = .008). Increases in help‐seeking intentions were sustained at follow‐up in the intervention condition. No differential effects were found for generalised anxiety, separation anxiety, social self‐efficacy or for any social support outcomes. Over 90% of participants indicated the app was enjoyable, interesting and easy to use. The app provided ‘advice and direction’ (n = 42; 46.15%), an ‘opportunity for self‐reflection’ (n = 33; 36.3%) and ‘normalised experiences’ (n = 21; 23.1%).

Conclusions

The WeClick app was found to be effective for improving wellbeing and help‐seeking intentions for mental health in adolescents. A larger, adequately powered trial is now required to establish differential effects on depressive symptoms. This trial was registered with the Australian New Zealand Clinical Trials Registry (ANZCTR): ACTRN12618001982202.

Introduction

Relationship problems have a bi‐directional association with the development and exacerbation of mental illness (Thapar, Collishaw, Pine, & Thapar, 2012). Relationship problems are more common among young people with depression and anxiety (Rickwood, Telford, Parker, Tanti, & McGorry, 2014), co‐occurring alongside feelings of hopelessness, suicidal ideation and self‐harm (Price, Hides, Cockshaw, Staneva, & Stoyanov, 2016). The early signs of mental illness often manifest through changes in social behaviour and increased interpersonal issues such as conflict, peer victimisation and shifts in social networks (Thapar et al., 2012). Moreover, many of the characteristics and behaviours symptomatic of mental illness in youth, such as increased social withdrawal, low self‐esteem, heightened worry or fear of others, separation anxiety, and excessive reassurance seeking, can negatively impact young people’s capacity to further develop and maintain healthy relationships with friends and family (Abela & Hankin, 2008). Relationship problems cause significant emotional distress among youth, a loss of belonging, anxiety (Vannucci, Ohannessian, Flannery, De Los Reyes, & Liu, 2018), as well as jealousy and aggression (Kraft & Mayeux, 2018). Despite this, fewer than half of all adolescents seek help for their relationship difficulties (Boldero & Fallon, 1995). Given that 50% of mental illnesses first emerge in adolescence (Patel, Flisher, Hetrick, & McGorry, 2007), relationships may be an ideal target for initiating and sustaining help‐seeking behaviour while also reducing the onset and exacerbation of depressive and anxiety symptoms.

While there are components of evidence‐based treatments that can assist young people with depression or anxiety arising from interpersonal conflict (e.g. Cognitive Behavioural Therapy, Interpersonal Therapy), such treatments were developed for youth with clinical disorders (Klein, Jacobs, & Reinecke, 2007; Oud et al., 2019; Pu et al., 2017). Many of these treatments are administered face‐to‐face by trained clinicians, limiting uptake and access among youth. The Internet and mobile phone applications are becoming increasingly popular tools for young people to seek information and support due to the convenience, privacy, ease of access and favourable mode of delivery (Carlisle et al., 2018). Despite the prevalence and impact of relationship problems, two recent systematic reviews of mobile apps and technology‐delivered interventions for youth indicated that there are currently no evidence‐based programs that primarily target relationship issues (Grist, Croker, Denne, & Stallard, 2019; Grist, Porter, & Stallard, 2017). To address this gap, the Black Dog Institute has developed a mobile phone application (app) called WeClick to reduce mental health problems in youth by targeting relationships.

Based on the principles of Cognitive Behavioural Therapy (CBT) (Richardson, Stallard, & Velleman, 2010) and Social Learning Theory (Bandura & Walters, 1977), the WeClick app is designed to reduce depression and anxiety symptoms by influencing cognitive and behaviour change in relationships. This CBT app focuses exclusively on adolescent‐specific relationship issues and involves cognitive challenging and restructuring to alter negative thoughts about one’s self and relationships. The app also includes practical problem‐solving strategies for relationship issues using exposure‐based techniques (Thorndike, Ritterband, Cox, Gonder‐Frederick, & Kovatchev, 2009) and social learning whereby youth observe the positive outcomes of the behaviour of others who seek help for and overcome relationship challenges (Bandura & Walters, 1977; Telzer, van Hoorn, Rogers, & Do, 2018). As outlined in Figure 1A, WeClick uses a range of activities and techniques to help young people to think differently about their relationship challenges and provide them with the skills to overcome these in more constructive ways (Erozkan, 2013). Figure 1B provides an example of one of the characters. The app content was reviewed by young people who deemed it to be helpful and relatable (O'Dea et al., 2018).

Figure 1.

(A) Therapeutic components within the WeClick app. (B) Example of a character’s profile within the WeClick app

Taking approximately one hour to complete, the WeClick app was designed as a brief ‘single session’ intervention. Single Session Interventions (SSIs) are a newer type of brief (i.e. delivered in sessions ranging from 5 to 90 min) emotional and behavioural intervention that specifically targets maladaptive beliefs or behaviours to the developmental or population‐specific needs of a specific sample (Schleider & Weisz, 2017a). Past SSIs for youth have targeted anxiety (Cartwright‐Hatton et al., 2018) and depression (Cardamone‐Breen et al., 2018) using simple strategies, including developing growth mindset (Schleider, Mullarkey, & Weisz, 2019; Schleider & Weisz, 2016) cultivating adaptive meaning‐making (Schleider, Mullarkey, & Chacko, 2019), behavioural activation (Wasil et al., 2020) and cognitive bias modification (Fu, Du, & xAu, 2013). In a meta‐analysis of 50 RCTs, SSIs were found to have a significant positive effect on adolescent psychological problems (Schleider & Weisz, 2017b) with many of these self‐administered to youth via technology. Technology‐delivered SSIs broaden access and maximise scalability by reducing costs and participant burden. Further, the brief nature of SSIs may lead to greater uptake and completion among youth, with this enhanced efficiency magnified by young people’s reluctance to use professional face‐to‐face treatments. Like other brief interventions, the WeClick app adopts an empathetic approach to increase a young person’s self‐efficacy and optimism for overcoming their relationship difficulties. The WeClick app may offer a new and novel way to reduce mental health problems in adolescents, although the appeal and effectiveness of this type of approach is yet to be evaluated.

Objectives and hypotheses

This randomised controlled trial was conducted to evaluate the acceptability and effectiveness of a relationship‐focussed mobile app (WeClick) for improving a range of mental health outcomes among adolescents. This study aimed to determine the appeal of the WeClick app among youth participants and measure their uptake and openness towards receiving mental health support in this way. This trial also aimed to determine the effectiveness of the WeClick app for reducing depressive symptoms (primary outcome), as well as anxiety symptoms and psychological distress, and increasing wellbeing, help‐seeking intentions for mental health, social self‐efficacy, social support and belongingness among adolescents, in comparison to a wait list control condition. It was hypothesised that youth who received the WeClick app would report lower levels of depressive symptoms at 4‐week post‐test than those in the control condition. It was also hypothesised that youth who received the WeClick app would report greater reductions in anxiety and distress, and greater improvements in wellbeing and help‐seeking intentions, social self‐efficacy, social support and belongingness at 4‐week post‐test, when compared to those in the control condition. The current trial also aimed to measure young people’s satisfaction with the WeClick app, the barriers to app use and potential app improvements as past studies on self‐paced digital programs have been negatively affected by low rates of adherence and engagement (Beatty & Binnion, 2016; Donkin et al., 2011). This will determine whether a mobile phone app that targets relationships offers an engaging, accessible and effective mental health intervention for youth.

Method

Study design

A two‐arm randomised controlled trial was conducted with data collected at baseline (i.e. Day 0), 4‐week post‐test (i.e. Day 29 from baseline completion/randomisation) and 12‐week follow‐up (i.e. Day 120 from baseline completion/randomisation). The primary outcome was assessed at 4‐week post‐test. Assessments commenced in October 2018 and were completed in April 2019. This trial was conducted entirely online with limited contact from the research team. Ethics approval was obtained from the University of New South Wales Human Research Ethics Committee (180460). The reporting of this study is compliant with CONSORT guidelines.

Sample size and participants

The target sample size was set at 166 participants, with 83 in each condition. This target was based on having 80% power (and α = .05) to detect an initial medium effect size of 0.50, similar to that obtained in prior wellbeing, distress and help‐seeking interventions for youth (Calear, Christensen, Mackinnon, Griffiths, & O'Kearney, 2009), and assuming a 30% attrition rate. Participants were aged between 12 and 16 years, living in Australia, fluent in English, could provide parental consent, had access to the Internet, an active email address and access to a mobile phone (iOS or Android). There were no exclusion criteria.

Randomisation and masking

Randomisation was carried out according to the International Council for Harmonisation (ICH) guidelines (Lewis, 1999) and performed immediately after participants completed baseline using a computerised adaptive randomisation procedure hosted by the Black Dog Institute’s online research platform. A stratification approach with a block size of 4 (1: 1 ratio) was used to ensure balance across the two conditions for age (12–14 years vs. 15–16 years) and gender (male vs. female). Although participants and researchers were not blinded to the allocation assignment, the allocation was fully automatic with no interference from researchers.

Recruitment and consent

Recruitment took place from October to December 2018 and utilised a paid Facebook campaign targeting young people aged between 12 and 16 years. Young people viewed the advertisement on Facebook and were directed to the study website which provided information and a brief online screener to assess the inclusion criteria. Eligible participants were invited to create a study account (name, email, mobile phone number, date of birth). Participants were then asked to download and complete the consent form, including parental signature. Signed forms were returned to the research team via email.

Procedure

The research team reviewed the consent forms upon receipt. Using the online research platform, the research team approved participants which triggered an automatic email/SMS invitation to complete the baseline survey. Participants also received an SMS and email invitation for the post‐test and follow‐up surveys. All surveys remained active for 5 days, with two reminders. Participants received a 15AUD voucher for each survey completed, with a maximum study reimbursement of 45AUD. Participants who reported experiencing recent thoughts of death or of harming oneself (i.e. score > 0 on item 9 of the PHQ‐9) were immediately displayed crisis contact details and encouraged to seek help from a trusted adult.

Intervention

WeClick is an interactive story‐telling app that consists of four characters, each facing different relationship difficulties including family conflict, peer conflict, intimate relationships, as well as other adolescent issues including substance use, self‐esteem, low mood and anxiety. The user selects a character and works through a series of activities that aim to develop the skills to overcome negative thinking and problem solving by observing what occurs in others’ relationships. By completing each of the stories, the user builds their own character profile that outlines who they can turn to for help, what they can do to keep on top of things, how they would seek help and what they can do to stay calm when faced with challenging situations (see Figure 1A,B). The WeClick app is entirely self‐paced with in‐built weekly reminders. The app was downloaded through the Apple and Google Play stores and participants were given four weeks of access using their study code. The app was only accessible to participants in the current study.

Control

Participants allocated to the wait list control condition were provided access to the app after completing the 4‐week post‐test survey. There were no restrictions on the use of other relationship or mental health apps or programs during this time.

Measures

Primary outcome

Depressive symptoms

The Patient Health Questionnaire for Adolescents (PHQ‐A) (Johnson, Harris, Spitzer, & Williams, 2002), a nine‐item questionnaire, assessed the presence of depressive symptoms in the past two weeks. Participants were asked to rate the frequency of depressive symptoms on a scale of 0 (not at all) to 3 (nearly every day). Summed items create a total score (range: 0 to 27) with a cut‐off of 10 indicating a likely case of depression. The scale showed good reliability in this trial (Cronbach’s α = .89).

Secondary outcomes

Anxiety symptoms

These were measured using the six‐item subscales for generalised anxiety and separation anxiety from the Spence Children’s Anxiety Scale (Spence, Barrett, & Turner, 2003). Participants were asked to rate the frequency of recent generalised anxiety symptoms on a scale of 0 (not at all) to 3 (nearly every day) and the frequency of separation anxiety symptoms on a scale ranging from 0 (never) to 3 (always). Total scores were calculated by summing the six items (range: 0–18) with higher scores indicating higher anxiety. In this study, the Cronbach’s alpha was .85 for generalised anxiety and .74 for separation anxiety.

Psychological distress

The Distress Questionnaire‐5 (DQ5), a brief five‐item screener, measured psychological distress (Batterham et al., 2016). The items were rated from 1 (never) to 5 (always) with total scores ranging from 5 to 25. Higher scores indicate greater psychological distress with the cut point of 11 indicating the likelihood of mental disorder (Batterham et al., 2016). In this study, the Cronbach’s alpha was .86.

Mental wellbeing

The Short Warwick Edinburg Mental Wellbeing Scale (SWEMWBS), a seven‐item scale, assessed wellbeing over the past two weeks (Tennant et al., 2007) The items were rated on a five‐point Likert scale from 1 (none of the time) to 5 (all of the time), with total scores ranging from 7 to 35. Higher scores indicate a greater mental wellbeing. In this study, the Cronbach's alpha was .85.

Help‐seeking intentions for mental health

Using the General Help‐Seeking Questionnaire (GHSQ), participants were asked to rate how likely they were to seek help from 13 different sources if faced with a mental health problem (Wilson, Deane, Ciarrochi, & Rickwood, 2005). Each item was answered on a five‐point scale ranging from 1 (extremely unlikely) to 5 (extremely likely) Participants were also able to indicate if they would not seek help from anyone. In the current study, a total help‐seeking intentions score was calculated by summing the 13 responses, with total scores ranging from 13‐65. A professional help‐seeking intentions score was also calculated by combining responses to three adult sources: school counsellor, doctor and other mental health professional.

Social self‐efficacy

The Self‐Efficacy Questionnaire for Children (SEQ‐C), an eight‐item scale, measured social self‐efficacy (Muris, 2001). Items were rated on a five‐point Likert scale from 1 (not very well) to 5 (very well) with total scores ranging from 8 to 40. Higher scores indicate greater social self‐efficacy. In this study, the Cronbach's alpha was .76.

Social support

The Shuster Social Support scale, a 10‐item questionnaire, assessed the frequency of social support received from family and friends (Schuster, Kessler, & Aseltine, 1990). Participants were asked how often their friends and family made them feel cared for, expressed interest, made demands, criticised, created tensions or arguments, answered on a scale from 1 (never) to 4 (often). Supportive and negative interaction items were then averaged separately, with total scores ranging from 1 to 4. Positive and negative support from different sources were examined separately as research suggests that each concept has differential effects on psychological health (Lincoln, 2000; Schuster et al., 1990). In this study, the Cronbach’s alpha for the supportive interaction subscale was .72 for family and .81 for friends. The Cronbach’s alpha for the negative interaction subscales was .75 for family and .62 for friends.

Belongingness

The Thwarted Belongingness Scale (TBS), an 8‐item scale, measured the impairment level of belongingness (Ma, Batterham, Calear, & Sunderland, 2019). Participants were asked to rate their agreement with statements about loneliness (e.g. ‘I feel isolated’) and the absence of caring relationships (e.g. ‘Nobody cares about me’). Each item was answered on a 7‐point scale ranging from 1 (not at all true for me) to 7 (true for me) with total scores ranging from 8 to 56. Higher scores indicate greater thwarted belongingness. In this study, the Cronbach’s alpha was .92.

Other measures

Demographics

Participants provided their gender and age at study registration. At baseline, participants reported whether they were Aboriginal and/or Torres Strait Islander or Lesbian, Gay, Bisexual, Trans, Queer, Intersex (LGBTQI) answered yes, no or I’d rather not say.

Mental health history and relationship difficulties

At baseline, participants were asked if they had ever experienced a mental health problem or been diagnosed with a mental illness, and if they had ever had any relationship difficulties (answered yes, no, or I’m not sure). If yes, participants were asked to select with whom (answered parents, friends, peers, classmates, boyfriend/girlfriend, workmates). Participants were also asked if they had ever used a mobile app for information or help for relationship problems (answered yes, no, or rather not say) and if so, had they found it helpful (answered yes, no, or can’t remember).

Expectations of success

Using four items answered on a five‐point Likert scale, participants were asked to rate how confident they were in learning skills to improve their relationships from a mobile app (answered strongly disagree to strongly agree), the importance of improving their relationships (answered strongly disagree to strongly agree), the importance of participating in relationship research (answered not important to very important) and their readiness to improve their relationships by using an app (answered not ready to completely ready). A higher score indicated a more positive expectation.

App use and satisfaction

App use was measured by the number of stories completed (maximum of four) and the time taken to complete these. The satisfaction questionnaire was delivered at 4‐week post‐test to the intervention condition and at 12‐week follow‐up to the control condition. Participants were asked to rate their agreeance with eight statements of ease of use and app usefulness. Participants were also asked to report their experience of 11 personal, intervention‐specific and technical barriers (e.g. ‘I didn’t have time to use WeClick’, answered yes or no). Participants rated the overall helpfulness of the app using a five‐point Likert scale from 1 (extremely unhelpful) to 5 (extremely helpful). Participants who rated the app as helpful were then asked the free‐response item ‘In what ways did WeClick help you?’. Participants who rated the app as unhelpful or neither helpful/unhelpful were then asked the free‐response item ‘What could we do to make WeClick better?’

Data storage and analysis

The data were collected and stored securely via the Black Dog Institute online research platform. During registration, participants created a password‐protected account and were automatically allocated a unique identification number. Participants’ names, emails, phone numbers and IP addresses were removed for analysis. Data were then downloaded into Microsoft Excel and exported to SPSS Version 22.0 (SPSS Inc., Chicago, IL, USA) for analysis.

Primary analyses were conducted to determine the effect of the intervention on depressive symptoms and were undertaken on an intention to treat basis, including all participants randomised, regardless of treatment received. As the control condition received the intervention at 4‐week post‐test, the effectiveness of the WeClick app was established by a change on the PHQ‐9‐A between baseline and 4‐week post‐test for the intervention condition relative to the control, based on the interaction between time and condition, using a mixed‐effects model repeated measures (MMRM) ANOVA with an unadjusted p value of .05. Effect sizes were calculated based on differences in change scores between baseline and 4‐week post‐test for the intervention condition versus control condition, using standard deviations of the change scores pooled across both conditions. MMRM models account for all available data (from baseline and post‐test), under the missing‐at‐random assumption, and are less prone to bias than completer analyses (e.g. ANCOVA) or legacy methods such as last observation carried forward (Verbeke & Molenberghs, 2000). Differences in the rates of missingness between the conditions at post‐test and follow‐up were examined and reported. These analyses were repeated for the secondary outcomes. Due to a lack of trials on universal relationship interventions for youth, the analyses also evaluated whether mental health outcomes changed within the intervention and control conditions over time. No adjustments were made for multiple testing.

The free‐response questions from the app satisfaction questionnaire were analysed by three researchers using thematic analysis (Braun & Clarke, 2006) to identify the patterned meaning across responses. This method is considered robust (Braun & Clarke, 2014), while also being accessible and flexible (Nowell, Norris, White, & Moules, 2017). An inductive approach, independent of a theoretical confirmative method, was used to identify and group themes. The first stage of analysis commenced with familiarisation of the dataset by one researcher (MA). Open coding was used to summarise the data and create preliminary labels. Next, initial codes were generated, and a coding framework was created. A second researcher (BP) then collaborated to refine the codes and determine the final coding framework. Both researchers recoded the data with the final coding framework. An independent third researcher (BOD) reviewed the data and themes and consolidated the findings. To support the validity of the analysis, analyst triangulation was used whereby higher‐order codes and final themes were determined by consensus among the researchers (Patton, 1999; Tracy, 2010). Furthermore, the researchers coding the data regularly reflected on their personal reactions and were considerate not to contaminate the data. The themes are outlined in the results.

Results

Figure 2 outlines the participant flow. In total, 1,310 adolescents completed the eligibility screening. Of these, 193 (M age: 14.82 years, SD: 0.93, age range: 12–16, 86.53% female) completed baseline and were randomised. There were no differences in the rates of missingness in the data between conditions at post‐test (p = .44) or follow‐up (p = .38). There were also no associations between age, gender or experience of previous mental illness and completion of the post‐test or follow‐up assessments (all p > .05).

Figure 2.

CONSORT diagram showing participant enrolment, allocation and analysis

Table 1 outlines the baseline characteristics of the sample. Overall, 58.0% (n = 112/193) had experienced or been diagnosed with a mental illness. Two‐thirds (69.4%, n = 134) had experienced relationships difficulties, and the majority of these were with friends (82.1%, n = 110/134) and parents (76.9%, n = 103/134). A total of 63.7% (n = 123) reported that they were confident that an app could help them learn the skills needed for improving their relationships and 73.9% (n = 142) felt they were ready to improve their relationships using a mobile app. A total of 21.8% (n = 42) had previously used a mobile app for relationships and 54.8% (n = 23/42) had found it helpful.

Table 1.

Baseline sample characteristics (N = 193)

	Intervention (n = 98)		Control (n = 95)
	n	%	n	%
12–14 years	32	32.7	29	30.5
Female	85	86.7	82	86.3
Aboriginal and/or Torres Strait Islander	3	3.1	4	4.2
Lesbian, Gay, Bisexual, Trans, Queer, or Intersex	34	34.7	24	25.3
Experienced relationship difficulties	71	72.4	63	66.3
Parents	56	78.9	47	74.6
Friends	56	78.9	54	85.7
Peers/classmates	36	50.7	32	50.8
Partner	27	38.0	26	41.3
Workmates	6	8.5	6	9.5
Experienced or diagnosed with a mental illness	59	60.2	53	55.8
Used an app in the past for relationship help	21	21.4	21	22.1
Considered past app use to be helpful	12	57.1	11	52.4

	M	SD	M	SD
Age	14.79	0.98	14.86	0.89
Confidence in ability to learn relationships skills from a mobile app	3.65	0.66	3.65	0.78
Importance of relationship research	4.30	0.56	4.26	0.57
Importance of improving relationships	3.93	1.05	4.09	0.90
Readiness to improve relationships via a mobile app	3.98	0.89	4.04	0.88

Table 2 outlines the observed group means and standard deviations of the primary and secondary outcomes for each timepoint.

Table 2.

Observed group means and standard deviations of the primary and secondary outcomes

	Intervention		Control
	n	M (SD)	n	M (SD)
Depressive symptoms (PHQ‐9)
Baseline	98	14.59 (6.88)	95	12.73 (6.68)
Post‐test	81	12.33 (7.11)	82	11.13 (7.20)
Follow‐up	55	12.05 (7.10)	60	11.27 (7.12)
Psychological distress (DQ‐5)
Baseline	98	16.35 (4.71)	95	15.79 (4.77)
Post‐test	80	15.04 (4.86)	82	14.99 (4.78)
Follow‐up	55	15.07 (4.71)	60	14.53 (5.19)
Generalised anxiety (SCASGAD)
Baseline	98	10.64 (3.93)	95	10.09 (4.21)
Post‐test	80	10.35 (4.17)	82	9.85 (3.88)
Follow‐up	55	9.78 (4.20)	60	9.12 (4.15)
Separation anxiety (SCASSAD)
Baseline	98	4.62 (3.36)	95	4.19 (3.27)
Post‐test	80	4.25 (3.35)	82	4.12 (3.20)
Follow‐up	55	3.31 (2.74)	60	3.07 (2.72)
Mental Wellbeing (WEMWS)
Baseline	98	19.71 (4.81)	95	20.71 (4.92)
Post‐test	80	21.20 (4.80)	82	20.85 (5.31)
Follow‐up	55	20.93 (4.36)	60	21.70 (5.61)
Belongingness (TBS)
Baseline	98	31.39 (11.69)	95	30.13 (11.76)
Post‐test	81	29.54 (11.52)	82	28.87 (12.47)
Follow‐up	58	28.76 (11.39)	60	28.35 (11.22)
Social self‐efficacy (SSES)
Baseline	98	24.53 (5.64)	95	24.81 (5.46)
Post‐test	81	24.86 (5.40)	82	24.37 (6.14)
Follow‐up	57	26.23 (5.74)	60	25.58 (5.23)
Help‐seeking intentions (GHSQ)
Baseline	98	33.91 (7.84)	95	34.99 (7.67)
Post‐test	80	35.29 (8.44)	82	33.43 (8.31)
Follow‐up	55	35.25 (7.34)	60	36.13 (7.72)
Professional help‐seeking intentions (GHSQ‐P)
Baseline	98	7.33 (3.13)	95	7.69 (3.23)
Post‐test	80	7.75 (2.82)	82	6.87 (3.11)
Follow‐up	55	7.91 (2.80)	60	7.43 (3.10)
Social Support (SS)
Friends – positive
Baseline	98	3.15 (0.77)	95	3.07 (0.72)
Post‐test	82	3.12 (0.68)	83	3.00 (0.70)
Follow‐up	89	3.21 (0.55)	62	3.17 (0.67)
Friends – negative
Baseline	98	2.43 (0.56)	95	2.37 (0.66)
Post‐test	82	2.38 (0.59)	83	2.43 (0.70)
Follow‐up	89	2.26 (0.51)	62	2.37 (0.70)
Family – positive
Baseline	98	3.20 (0.74)	95	3.29 (0.70)
Post‐test	82	3.18 (0.71)	83	3.34 (0.70)
Follow‐up	89	3.19 (0.72)	62	3.33 (0.66)
Family – negative
Baseline	98	2.59 (0.70)	95	2.55 (0.74)
Post‐test	82	2.55 (0.72)	83	2.60 (0.72)
Follow‐up	89	2.62 (0.67)	62	2.57 (0.68)

Table 3 displays the mixed‐effects models for the primary and secondary outcomes, based on change in these outcomes from baseline to 4‐week post‐test. There was a significant main effect for time on depressive symptoms, psychological distress, mental wellbeing and belongingness, with both conditions reporting improved scores throughout the study. However, there was no significant interaction between condition and time for depressive symptoms, psychological distress or belongingness, indicating that the decreases in these three outcomes over time were not significantly different between the intervention and control conditions. There were significant between‐group differences in mental wellbeing scores and help‐seeking intentions, also reflected in professional help‐seeking intentions. These differences indicated that the intervention condition had greater increases in wellbeing (d = 0.37) and intentions to seek help (d = 0.36) than the control condition, from baseline to 4‐week post‐test. Furthermore, no significant differences were found for generalised anxiety, separation anxiety, social self‐efficacy or for any social support outcomes.

Table 3.

Mixed‐effects model repeated measures estimates

	df		F	p	Cohen’s d
Depressive symptoms (PHQ‐A)
Condition (intervention vs. control)	1	191.59	1.90	.170
Time (post‐test vs. baseline)	1	165.18	34.35	<.001
Condition × Time	2	165.18	2.22	.138	0.26
Generalised anxiety (SCASGAD)
Condition (intervention vs. control)	1	190.50	0.67	.415
Time (post‐test vs. baseline)	1	165.91	1.29	.257
Condition × Time	2	165.91	0.20	.659	0.09
Separation anxiety (SCASSAD)
Condition (intervention vs. control)	1	192.12	0.26	.610
Time (post‐test vs. baseline)	1	167.70	1.07	.301
Condition × Time	2	167.70	1.28	.259	0.02
Psychological distress (DQ5)
Condition (intervention vs. control)	1	188.19	0.06	.808
Time (post‐test vs. baseline)	1	163.43	18.28	<.001
Condition × Time	2	163.43	2.28	.133	0.26
Mental wellbeing (WEMWS)
Condition (intervention vs. control)	1	191.04	0.15	.697
Time (post‐test vs. baseline)	1	167.06	8.38	.004
Condition × Time	2	167.06	5.30	.023	0.37
Help‐seeking intentions (GHSQ)
Condition (intervention vs. control)	1	188.97	0.07	.797
Time (post‐test vs. baseline)	1	166.02	0.22	.640
Condition × Time	2	166.02	5.97	.016	0.36
Professional help‐seeking intentions (GHSQ‐P)
Condition (intervention vs. control)	1	189.87	0.15	.700
Time (post‐test vs. baseline)	1	170.24	0.56	.457
Condition × Time	2	170.24	7.10	.008	0.36
Belongingness (TBS)
Condition (intervention vs. control)	1	188.57	0.17	.678
Time (post‐test vs. baseline)	1	162.92	11.21	.001
Condition × Time	2	162.92	1.00	.319	0.17
Social self‐efficacy (SSES)
Condition (intervention vs. control)	1	185.75	0.06	.800
Time (post‐test vs. baseline)	1	163.69	0.25	.620
Condition × Time	2	163.69	1.91	.169	0.23
Social Support: Friends – positive
Condition (intervention vs. control)	1	192.61	0.82	.365
Time (post‐test vs. baseline)	1	176.58	1.13	.289
Condition × Time	2	176.58	0.02	.900	0.01
Social Support: Friends – negative
Condition (intervention vs. control)	1	185.79	0.01	.916
Time (post‐test vs. baseline)	1	168.19	0.08	.778
Condition × Time	2	168.19	1.30	.255	0.17
Social support: Family – positive
Condition (intervention vs. control)	1	191.39	1.48	.225
Time (post‐test vs. baseline)	1	170.83	0.35	.556
Condition × Time	2	170.83	0.56	.456	0.11
Social support: Family – negative
Condition (intervention vs. control)	1	188.44	0.00	.951
Time (post‐test vs. baseline)	1	167.05	0.44	.508
Condition × Time	2	167.05	0.46	.496	0.08

Bold values indicate p < .05

Participants in the intervention condition reported significantly greater improvements in wellbeing scores at 4‐week post‐test (M: 21.36, SE: 0.54, 95% CI: 20.30–22.42) relative to the control condition (M: 20.87, SE: 0.54, 95% CI: 19.81–21.93; p = .023). Figure 3 displays the mean (SE) wellbeing scores at each timepoint and condition, based on estimated scores from the mixed‐effects model. Participants in the intervention condition had no significant change in wellbeing from 4‐week post‐test to 12‐week follow‐up (p = .709), indicating effects were sustained.

Figure 3.

Estimated marginal means (SE) from mixed model repeated measures for wellbeing (WEMWBS) scores over time in the intervention and control conditions

Intervention participants also reported significantly greater intentions to seek help for mental health problems at 4‐week post‐test (M: 35.49, SE: 0.91, 95% CI: 33.71–37.27) relative to control [(M: 33.83, SE: 0.90, 95% CI: 32.06–35.61; F(2, 157.29) = 5.34, p = .006)]. Figure 4 displays the mean (SE) help‐seeking intentions scores at each timepoint and condition, based on estimated scores from the mixed‐effects model. Intervention participants (M: 7.70, SE: 0.32, 95% CI: 7.07–8.32) also reported significantly greater intentions to seek help from professionals at 4‐week post‐test relative to control [(M: 7.03, SE: 0.32, 95% CI: 6.40–7.65; F(2,156.67) = 3.64, p = .028)]. Figure 5 similarly displays the mean (SE) professional help‐seeking intentions scores at each timepoint and condition. These effects were sustained at follow‐up, with no significant decline in help‐seeking intentions (p = .415) or professional help‐seeking intentions in the intervention condition (p = .319).

Figure 4.

Estimated marginal means (SE) from mixed model repeated measures for total help‐seeking (GHSQ) scores over time in the intervention and control conditions

Figure 5.

Estimated marginal means (SE) from mixed model repeated measures for GHSQ professional help‐seeking scores over time in the intervention and control conditions

Figure 2 displays the app uptake across the conditions. A total of 95.9% of the intervention participants (n = 94/98) used the app. Of these, 68.1% (n = 64/94) completed all four stories and 4.3% (n = 4/94) completed none. In total, 170 study participants used the app: 60.0% (n = 102) completed all four stories and 7.1% (n = 12) completed none. The mean time taken to complete each story was 00:05:55 (SD: 00:03:03; range: 00:01:45 to 00:26:30), with total time on the app averaging 00:19:00 (SD: 00:12:00; range: 00:02:00 to 01:46:00). The mean number of app logins was 4.29 (n = 170, SD: 2:25, range: 1–11).

Table 4 outlines participants’ satisfaction with the WeClick app. More than 90% reported that the app was enjoyable, easy to use and understand, and that they would recommend it to a friend. The barriers most commonly experienced were mismatched need, forgetfulness and being time poor. The mean helpfulness rating was 3.65 (SD: 0.70, n = 138, range: 1–5) and 65.9% (n = 91/138) rated the app as helpful or extremely helpful. Table 5 outlines the ways in which the app helped participants alongside their suggestions for improvement.

Table 4.

Satisfaction with app and barriers to use

Satisfaction with app (n = 140)	Agreed (n)	%
Ease of use
I enjoyed using WeClick	135	96.4
WeClick was easy to understand	134	95.7
I found WeClick easy to use	133	95.0
I thought WeClick was interesting	127	90.7
Usefulness
I would tell a friend to use WeClick if I thought they needed to	125	89.3
I would use WeClick again in the future	106	75.7
WeClick helped me to feel in control of my feelings	87	62.1
The things I learned from WeClick helped me a lot in everyday life	74	52.9

Barriers to use (n = 138)	Agreed (n)	%
Personal
I felt WeClick wasn't what I needed	60	43.5
I forgot to use WeClick	55	39.9
I didn't have time to use WeClick	40	29.0
I couldn't be bothered to use WeClick	28	20.3
I felt too worried or too down to use WeClick	23	16.7
Intervention
I thought that the content took too long to read	30	21.7
I thought the text was too small and too hard to read on my phone	9	6.5
I didn't trust WeClick	7	5.1
I thought WeClick used up too much of my phone data	6	4.4
WeClick made me feel worse	3	2.2
Technical
I didn’t have a mobile phone to use WeClick	4	2.9

Table 5.

Thematic analysis of participants’ responses

Theme	Description	n (%)	Example
In what ways did the WeClick app help you? (n = 91)
Provided advice and direction	Helped participants learn practical skills, provided information and direction. It also served to highlight alternative actions to utilise in future situations	42 (46.2)	‘gave me a few suggestions on how to approach friends’ ‘useful information on how to deal with my emotional problems in different circumstances’.
Opportunity for reflection on self and relationships	Helped participants understand themselves better by reflecting on how they have managed their own relationships and consider alternative approaches.	33 (36.3)	‘to understand a few of my problems more’ ‘to realise what actions and thoughts were unhelpful’
Normalised own experiences through social comparison	Helped participants realise that other people have similar experiences and feelings as they do by relating to the characters and scenarios	21 (23.1)	‘to realise I am not alone with the issues I have’. ‘to realise I’m not the only person who’s dealing with this’
Provided emotional support	Helped participants feel better and positive, calming and comforting support	8 (8.8)	‘provided comfort’ ‘made me feel slightly calmer’
Promoted sense of agency	Helped participants to feel in control of their own problems and as though they could overcome future problems	5 (5.5)	‘made me feel a bit more in control of my feelings’ ‘next time I will know how to react and respond’.
Positive distraction	Helped by redirecting participants’ attentions and enabling them to think about something other than their worries or concerns	2 (2.2)	‘it was a good distraction’
Unclear response	Responses that were unclear and unable to be coded	6 (6.6)	N/A
What could we do to make WeClick better? (n = 47)
Increase content	More resources, suggestions and stories covering cover a wider range of scenarios	20 (42.6)	‘just expand on what is already on WeClick’. ‘wasn’t much content and could easily be completed in an hour… Honestly there just wasn’t enough stuff to do on it’
Increase interactivity	More interactivity and visual elements such as images, journal, and a social component	9 (19.2)	‘make it a social platform so other teens could talk to each other about what they are going through’ ‘put more interactive story stuff’
Improve usability	Repairing technical issues and making the app more user friendly	5 (10.6)	‘make it easier to navigate the app’ ‘it hadn’t saved or something cause I had to do the whole thing over again’.
Include reminders/notifications	Increasing push notifications to remind them to use the app	4 (8.5)	‘daily messages or reminders to make people feel better and also remind them to use it’. ‘notifications so people don't forget to use it’
Decrease time to complete	Reducing some of the content and changing the type of activities	4 (8.5)	‘less text responses, I rarely had the time to respond in full sentences’ ‘less content’
Target to specific audience	The app may be more appropriate for younger teens or those who have not sought help before.	3 (6.4)	‘could be very helpful for someone who's never reached out before’. ‘it should be directed more at younger kids’
Increase personalisation	Making the app more personalised	3 (6.4)	‘different advice according to the user’s choices in the app’
Unclear responses	Responses that were unable to be coded	7 (12.8)	N/A

For the ‘In what ways did the WeClick app help you?’ question, n = 91 provided 117 responses which indicated that some participants reported more than one reason the app was helpful. For the ‘What can we do to make WeClick better?’ question, n = 47 provided 55 responses which indicated that some participants reported more than one suggestion for improvement. The percentages therefore reflect the prevalence of the theme among these participants.

Discussion

This study aimed to investigate the acceptability and effectiveness of a relationship‐focussed mobile phone app for improving mental health outcomes in adolescents. The results confirmed that some aspects of mental health can be improved in as little as four weeks, with less than one hour of exposure to therapy, with sustained effects at 12‐week follow‐up. The WeClick app was effective for improving wellbeing alongside general and professional help‐seeking intentions for mental health problems. The app may have driven these effects by increasing the awareness of the need for help (i.e. knowledge), observing the positive help‐seeking encounters of others (i.e. attitudes), providing information on coping strategies and identifying of new sources (i.e. availability) and normalising the help‐seeking process. This is supported by participants’ qualitative responses. Despite the nonsignificant findings for depression, the results appear promising with a larger sample likely to detect small effects. Based on the observed between‐group effect sizes, the overall effects of the WeClick app are likely to be small, although consistent with other universal mental health prevention studies (Werner‐Seidler, Perry, Calear, Newby & Christensen, 2017). Consequently, WeClick may have meaningful public health impacts when delivered in a universal setting to large numbers of adolescents. This trial provides further support the benefits of SSIs (Schleider & Weisz, 2017b) and highlights the potential of a brief mobile therapy app for improving youth mental health (Rickwood et al., 2014).

Consistent with other youth studies (Park & Kwon, 2018), our sample was open to using a mobile app for relationship help, a finding that may indicate a previously missed opportunity for engaging youth with care. Apps have the advantage of providing immediate access, support and resources to youth when they are most in need. In this way, a brief, accessible, relationship‐focussed intervention like WeClick may provide a highly acceptable and effective approach to improving the mental health of adolescents. Uptake and completion of the WeClick app was high and two‐thirds found it helpful. These rates are within the upper range and higher than other youth e‐mental health programs (Killikelly, He, Reeder, & Wykes, 2017; Simco, McCusker, & Sewitch, 2014), further suggesting young people are enthusiastic about relationship‐focussed interventions. The brief nature of WeClick may have contributed to the high adherence, providing additional support for SSIs. Alternatively, the study incentives may have driven engagement, although this is unlikely as the majority of participants used the app repeatedly, reported positive experiences and requested more content. Notably, however, one in five found app use to be negatively impacted by forgetfulness, lack of perceived need and low motivation. These barriers may be addressed by increasing push notifications, adding a progress tracker and more detailed onboarding or symptom checkers to contextualise the need for help and support. These strategies may be particularly important for increasing uptake and engagement among depressed youth due to the impact of depression on motivation. Further, as one‐third of participants reported low levels of confidence in an app’s ability to improve their relationships, some youth may not yet perceive mobile apps to be appropriate sources of help or support for relationship difficulties. Taken together, these findings confirm the need for an evidence‐based relationship app for young people that is trusted, credible, easy to use and widely promoted. Appropriate marketing and onboarding strategies will help to increase the appeal, awareness and perceptions of this type of support.

Given the speed at which technology develops, a major challenge to the evaluation of mobile apps is ensuring that studies are rigorous and completed swiftly (Nicholas, Boydell, & Christensen, 2016). A major strength of the current study was that it was completed in a short time frame (six months). The use of Facebook recruitment enabled a sample of Australian youth to be recruited in less than 10 weeks. While there was an over‐representation of females, this may be due to the greater likelihood of interpersonal distress and emotional difficulties experienced by this gender (Liu, 2006; Tillfors, Persson, Willén, & Burk, 2012). Females also tend to engage more with mental health research and treatment (Batterham et al., 2020). Future studies would be strengthened by increasing sample sizes and the number of male participants, lengthening the duration of follow‐up and adjusting for multiple statistical testing. This will allow for any small effects on depressive and anxious symptoms to be firmly established, and for investigation of mediation or moderation effects on the basis of demographic, mental health and relationship characteristics and app use patterns. The current study had a higher representation of LGBTQI youth than population norms (Fisher et al., 2019; Richters et al., 2014), which may suggest that this demographic has a greater need or interest in relationship‐focussed interventions. Furthermore, the severity of mental health symptoms in this sample was well above population averages, suggesting self‐selection of youth with mental health problems. Other recruitment strategies may be needed to obtain a more representative sample of youth. Future studies may benefit from incorporating measures of behaviour change to establish the effects on active help‐seeking, alongside optimisation strategies and factorial experiments to evaluate the components of the app, to better understand the mechanism by which it influences help‐seeking and other mental health outcomes (Tombor et al., 2018).

Conclusion

To our knowledge, this is the first study to demonstrate that a brief, relationship‐focussed, mobile app is effective for improving wellbeing and help‐seeking for mental health among adolescents, although it did not have a significant effect on depressive symptoms. The high rates of adherence, engagement and positive feedback indicate that this type of intervention is acceptable for improving mental health outcomes in youth. Future improvements such as expanding the content to include more scenarios and interactivity will help improve the app and may lead to stronger effects on other mental health outcomes. Once complete, a larger, adequately powered trial will be used to determine the effects of this novel approach for reducing depressive symptoms in youth.