Skip to main content
NCBI home page
Internet Interventions logoLink to Internet Interventions
. 2019 May 3;18:100249. doi: 10.1016/j.invent.2019.100249

‘E’ therapy in the community: Examination of the uptake and effectiveness of BRAVE (a self-help computer programme for anxiety in children and adolescents) in primary care

Stephanie Moor a,⁎, Jonathan Williman a, Sarah Drummond a, Caroline Fulton a, Wendy Mayes a, Nicola Ward a, Elizabeth Dovenberg a, Charis Whitaker a, Karolina Stasiak b
PMCID: PMC6926167  PMID: 31890607

Abstract

Objective

Recognizing and treating anxiety early is an important public mental health objective. There is clinical trial evidence that ‘e’ therapies are appealing, engaging and effective and have the potential to improve access to treatment. However, their implementation and effectiveness in real world settings is still emerging. Following a major natural disaster in NZ, an online therapist assisted CBT programme for children with anxiety, (BRAVE_TA) was made available in primary care.

Methods

Uptake and feasibility of BRAVE_TA delivery was assessed by examining referral patterns, non-engagement rates and programme progression within a geographic region of NZ (Canterbury population of around 500,000). Effectiveness of BRAVE_TA was measured by the extent of change in the primary outcome measure of anxiety, the CAS8, between baseline and last completed session.

Results

There were 1361 referrals to BRAVE_TA over 2014–2018, mostly from primary care doctors and nurses in primary schools. After attrition due to triage and family withdrawal, 75% (N = 1026) were enrolled. Around half of children/adolescents completed 4 sessions with moderate effect sizes achieved Children/adolescents who completed more sessions, had lower anxiety after their last session, with most of the improvement occurring within the first three to four sessions.

Conclusion

BRAVE_TA has shown to be an acceptable and effective ‘e’ therapy tool in a ‘real world’ primary care setting for children/adolescents with anxiety. This study supports the role of ‘e’ therapy as part of a stepped-care model within primary care in improving access to treatment and thus improving public mental health in children.

Trial registry: ACTRN12612000063819.

Keywords: Anxiety, Cognitive behaviour therapy, Computerised cognitive behaviour therapy, Natural disaster, Technology in psychotherapy research & training

Highlights

  • •

    Childhood anxiety is common, persistent and adversely impacts lifespan health.

  • •

    Early treatment is an important public mental health aim.

  • •

    BRAVE, an ‘e’ CBT therapy for childhood anxiety was evaluated in primary care.

  • •

    Uptake and feasibility of delivery in a ‘real world’ setting was confirmed.

  • •

    This translational study demonstrates effectiveness and ‘reach’ of an ‘e’ therapy.

1. Background

Anxiety disorders in childhood are common, with a prevalence of around 5% in young children (Ford et al., 2003), increasing up to 15% in adolescence (Costello et al., 2004). For many, anxiety persists across the life span (Kessler et al., 2010), impacting both mental health (Bittner et al., 2007) and physical health (Jokela et al., 2009). Therefore, recognizing and treating anxiety disorders early are important public mental health objectives (Bayer et al., 2011). Effective treatments for children and adolescents are available, with psychological treatments such as Cognitive Behavioural Therapy (CBT) having the greatest body of evidence (James et al., 2015). However, the availability of effective treatments within primary care settings is limited as there are not enough suitably trained therapists to meet demand. The recent development of ‘e’ therapy tools is seen as one part of the answer to the current limited therapeutic reach of effective interventions (Andersson and Titov, 2014) and for paediatric anxiety in particular (Jolstedt et al., 2018). Against this background, CBT for child/adolescent anxiety was adapted into an online programme called BRAVE–ONLINE and shown to be superior to wait-list with children (March et al., 2009) and as effective as face-to-face therapy in adolescents (Spence et al., 2011) by the programme developers Spence, Donovan and Marsh. However, there are few evaluations of child mental health treatments by researchers independent from the programme developers, and robust evaluations of effective therapies in ‘real world’ settings are scarce (Hill et al., 2017; Fleming et al., 2018).

The therapist assisted version of BRAVE-ONLINE, BRAVE_TA, was made available in Canterbury, New Zealand in 2012 after New Zealand's largest ever natural disaster. A small pilot study found that BRAVE_TA was feasible to deliver in primary care and in a qualitative analysis, participants found it helpful and acceptable to use (Stasiak et al., 2016). Based on the developers' efficacy trials and local positive experience with the pilot study, a community rollout and evaluation of effectiveness commenced in 2014.

In this ‘real world’ open effectiveness trial we aimed to:

  • (1)

    Assess the uptake and feasibility of delivery of BRAVE_TA in primary care by examining referral patterns, non-engagement rates and participant programme progression; and.

  • (2)

    Evaluate the effectiveness of BRAVE_TA in primary care by measuring the extent of change in our primary outcome measure of anxiety.

2. Method

2.1. Participants

Children/adolescents aged 7–17 years who lived within the Canterbury District Health Board (CDHB) catchment area (population of 500,000) identified by their family doctor, school nurse or other child mental health NGO with anxiety symptoms impacting on their functioning, were eligible for BRAVE_TA. Referring clinicians offered BRAVE_TA as part of their treatment plan. Referrers were required to check that families had a computer with internet access and that the child's reading age was at or above 7 year old level. The study period was from 1 May 2014 to 1 April 2018.

2.2. Measures

  • (1)

    Assessment of uptake and feasibility of delivery

Uptake and feasibility of delivery of BRAVE_TA in primary care was assessed by examining referral patterns, appropriateness of referrals, non-engagement rates and programme progression using nonstarter and starter rates. These were calculated as follows:

  • (i)

    Referral patterns included the source of the referral to BRAVE _TA and the numbers of referrals per month over the 4 year study period. These were examined to assess whether the community referrers found BRAVE_TA to be a potentially acceptable and accessible treatment option for children/adolescents with anxiety. In addition, information from the referrer on participant age and gender were collected at baseline.

  • (ii)

    To assess the appropriateness of referrals to BRAVE_TA in this primary care setting, the extent of anxiety symptoms and their impact on functioning were more fully assessed at programme entry using both the child self-report and parent report versions of the 38 item Spence Children's Anxiety Scale (SCAS) (Spence, 1998). Baseline anxiety scores were grouped according to Australian norms (Spence, 2018) into ‘not elevated’ (for those children/adolescents referred with anxiety, but whose scores were within one standard deviation of age-normed community population scores) and ‘elevated anxiety’ (greater than one standard deviation above the norm). We used the child self-report version for children aged 11 years and over; however as symptom reporting may be less reliable in younger children, we used the parent report SCAS data for participants under 11 years of age.

  • (iii)

    Non-engagement was measured by examining the number of referred children/adolescents who did not start the BRAVE_TA programme. These were designated as ‘did not engage’.

  • (iv)

    A further group of children were enrolled on the programme, but did not complete one session. These were designated as ‘non-starters’.

  • (v)

    Programme Progression on BRAVE_TA was classified by the number of sessions completed by the child/adolescent. Participants who completed one or more sessions of BRAVE_TA were designated as ‘starters’.

  • (2)
    Evaluation of BRAVE_TA programme effectiveness
    • (i)
      Primary outcome: Effectiveness of BRAVE_TA in primary care was measured by the extent of change from baseline to last completed session in our primary outcome measure, the 8 item Children's Anxiety Scale (CAS8) (Reardon et al., 2018). The CAS8, has been shown to be reliable and valid in community samples and was completed at the start of each session. The child/adolescent participants were asked to indicate how often they experienced each anxiety symptom using a 4-point scale (“Never”, “Sometimes”, “Often” or “Always”), with total scores ranged from 0 to 24 and with higher scores reflecting higher levels of anxiety. Baseline CAS8 scores were derived from the baseline SCAS score. Throughout the programme, the CAS8 questionnaire ‘popped up’ before sessions 2 and 10. Therefore, a session 2 CAS8 score reflected having completed session 1 of BRAVE_TA. The CAS8 ‘pop up’ was added part way through the study to improve outcome evaluation data collection and was only available for the 438 child/adolescent participants who started the programme after 2016.
    • (ii)
      Secondary outcome: At the start of each session of their accompanying parent programme, parents completed the 8 question parent-report CAS8 on their child's anxiety symptoms. Baseline parent CAS8 scores were derived from the baseline SCAS score. Grouped parent data assessing the change in total scores from baseline to last parent programme completed session was used to corroborate the grouped child/adolescent change in anxiety. Data from parents with more than one child in the programme was not included as we were unable to ascertain which child they were rating.

The participant CAS8 data was collected on a Qualtrics data base with a unique identifier ID number. This was downloaded to an ACCESS data base on a secure University of Otago server.

2.3. Procedure

All referrals to BRAVE_TA were screened by the supervising child psychiatrist (SM) and referrers contacted if the referral was deemed inappropriate. Exclusion criteria included children/adolescents with depression and self-harming behaviour and/or severe anxiety which would be more appropriately treated by local specialist mental health services. These were ‘triaged out’ with accompanying advice to the referrer (by phone or email) about other local specialist resources. Referrals for children/adolescents out of age range or not domiciled within the local health board (CDHB) were declined (designated ‘ineligible’) and the referrer notified.

For all ‘eligible’ referrals, the child and parent were sent an email with a link to the Qualtrics survey containing the baseline anxiety measure, the 38 item SCAS (both child and parent versions). If no questionnaire was returned after encouraging reminder emails and two months had passed, the referrer was informed that their patient ‘did not engage’. When baseline anxiety questionnaires were returned, participants were allocated unique identifiers, the start date was recorded, a therapist was allocated and these families were designated as ‘enrolled’. Some older adolescents accessed the programme without their parents participating.

When ‘starter’ participants had completed the programme or clearly failed to progress to the next session (allowing 4–6 weeks since last session completed), the BRAVE therapists sent brief treatment summaries to the referrers. These described the extent of treatment delivered with comments about content or participation style that might be helpful for treating clinicians in the future.

2.4. Data management and statistical analysis

All data were collated and stored in a custom Microsoft ACCESS database located on a password protected secure server hosted by the University of Otago, and only available to the BRAVE_TA administrator (WM), the team psychiatrist (SM) and the biostatistician (JW). Data was extracted into the R statistical software package for manipulation and analysis.

Participant characteristics were summarized by enrolment status using simple descriptive statistics, and differences between enrolment groups were tested using Wilcoxon or Pearson χ2 tests for continuous data or categorical data respectively. The maximum number of sessions completed by participants was described using means and standard deviations, and counts and percentages. A multivariable Poisson regression model was used to investigate differences in sessions completed by female gender, age (as a non-linear continuous variable), and baseline anxiety (categorized as ‘not elevated’ versus ‘elevated’). Programme progression was evaluated by calculating the absolute number and proportion of participants (with 95% ‘Wilson’ binomial CI) exiting the programming after each session. Attrition rates by session were plotted and observed trends summarized descriptively.

Programme effectiveness was assessed via a paired pre post analysis where participants were limited to those with a child report CAS8 at baseline and at their last completed brave session, and these paired data were used to calculate change in CAS8 scores over the intervention. Multivariable linear regression models were used to assess the associations between age and gender with CAS8 scores at baseline, last completed session, and change between baseline and last completed session. Cohen's d effect sizes for change in CAS8 scores were calculated by dividing the mean change in scores by the model residual standard deviation and were stratified by sessions completed and baseline anxiety.

2.5. Intervention

BRAVE_TA is an online CBT programme for children and adolescents and is therapist supported. It has developmentally tailored versions; ‘child’ version for 7–12 year olds and ‘teen’ version for 13 to 18 year olds. The programme is described in detail elsewhere (Spence et al., 2008) and will only be described briefly here.

2.5.1. Structure

The programme consists of ten 30–40 min sessions for the children/adolescents and five (teen version) or six (child version) sessions for their parents. The two versions of the programme are age-appropriate and designed to meet the developmental and cognitive level of younger children or adolescents. Information is presented through interactive exercises, text, colourful graphics, animation, quizzes, and games. To avoid activities being skipped, most exercises require a response before the next screen can be accessed. Homework exercises are set at the end of each session. If a participant misses a session, the system sends out automatic reminders. The sessions are designed to be completed sequentially at weekly intervals (the next available session becomes accessible seven days after the previous one is completed). Access to the programme is password protected and all content created by the child/adolescent and parent is stored on a secure server accessible only by the designated BRAVE therapist (see below).

2.5.2. Content

The programme targets four types of anxiety: social anxiety, generalised anxiety, separation anxiety and specific phobias. The content is based on standard CBT anxiety management techniques including psychoeducation, relaxation training, recognition of physiological symptoms of anxiety, cognitive strategies of coping self-talk and cognitive restructuring, graded exposure, problem solving and self-reinforcement. Parent sessions include acquisition of the above-mentioned CBT strategies and in addition, include parenting strategies to empower parents to help their child to implement anxiety management skills and deal with situations in which their child gets anxious.

Clinical psychologists (BRAVE therapists) were trained to deliver BRAVE_TA using a manual designed by the programme developers. They reviewed the completed exercises and homework tasks and sent weekly emails (based on a template) to the participants and their parents, with encouragement and feedback. Clinical supervision was by fortnightly videoconferencing with the supervising child psychiatrist along with support by phone or email between supervision sessions.

The BRAVE_TA system was hosted on the University of Queensland server and supported by their IT team. A site licence was purchased for the duration of this study.

3. Results

3.1. Assessment of uptake and feasibility of delivery

3.1.1. Referral patterns

Fig. 1 describes the recruitment and retention numbers. Between 1 May 2014 and 1 April 2018 there were 1361 referrals to the BRAVE_TA programme of which 48 were ineligible. Referral letters were scrutinised by the team child psychiatrist and after telephone discussion with the referrer, 52 referrals were triaged out. From the clinical history given, these referrals were for children with probable PTSD, severe anxiety or OCD or adolescents with other comorbid diagnoses such as significant mood disorder and self-harming behaviour for which BRAVE was not designed.

Fig. 1.

Fig. 1

Open in a new tab

Participant flow.

Around two thirds of the referrals (65%) were from Family Doctors, with 19% referred by from Public Health nurses who work in the primary schools, with the remainder from other community based NGO organisations working in child mental health. As can be seen in Fig. 2, referral rates increased throughout the study to around 30 referrals per month over 2015–2017. Referral rates remained steady as the utility of this tool in the community treatment of child and adolescent anxiety became known and better understood.

Fig. 2.

Fig. 2

Open in a new tab

Referrals received over the study period (1 May 2014–1 April 2018).

3.1.2. Appropriateness of referrals

There were 48 ‘ineligible’ referrals for children/adolescents out of age range or not domiciled within the local health board and along with the 52 ‘triaged out’ referrals, <10% of referred children/adolescents were not eligible for BRAVE _TA. Seventy-seven percent of enrolled participants were in the elevated anxiety range (+1 SD above normal range of the SCAS) at baseline indicating that the majority of participants had at least mild anxiety at baseline.

3.1.3. Non-engagement

One thousand two hundred and sixty-one (1261) were eligible for BRAVE_TA and were contacted by the BRAVE administrator. To enroll, participants were required to fill in a baseline anxiety questionnaire and 235 families (19% of eligible families) either notified the administrator that they did not want to try BRAVE_TA or did not return (despite reminders) the baseline measure.

Table 1 reports the demographic characteristics of eligible participants by enrolment and commencement status. The 235 participants who did not engage were on average one year older (12.2 vs 11.2 years), but did not differ in gender or referral source from the 1026 participants who went on to enroll on the programme. The gender distribution of the enrolled participants differed by age, with the proportion of girls being approximately half (314 girls versus 306 boys) for age < 12 years, two-thirds (158 vs 77) for ages 12 and 13 years, and three quarters (132 vs 39) for ages 14 years or more (χ2 = 47.82, p < 0.001).

Table 1.

Participant characteristics by enrollment status.

Eligible for BRAVE (1261)
 Enrolled (N = 1026)
Did not engage Enrolled F/χ2a, p Non starters Starters F/χ2a, p
N 235 1026 163 863
Age in years, median (IQR) 12 (10, 15) 11 (9,13) 12 (9, 14) 11 (9, 13)
Mean (SD) 12.2 (3.1) 11.2 (2.9) F = 24,p < 0.001 11.9 (3.2) 11.0 (2.8) F = 11, p < 0.001
<11 years old, N (%) 155 (66) 540 (53) X2 = 14,p < 0.001 60 (37) 437 (51) χ2 = 8.7,P = 0.003
Female gender, N (%) 146 (62) 604 (59) X2 = 0.84,p = 0.36 101 (62) 503 (58) χ2 = 0.77, p = 0.38
Referral source N (%) GP referrers 164 (70) 647 (63) X2 = 3.5,p = 0.061 111 (68) 536 (62) χ2 = 1.9,P = 0.17
Elevated baseline anxiety N (%) – 776 (77) 113 (72) 663 (77) χ2 = 1.9, p = 0.17
Open in a new tab
a

Wilcoxon or Pearson test used.

3.1.4. Programme progression

Following programme enrolment, the 1026 participants were grouped into those who did not complete one session of BRAVE_TA (N = 163, 12% of those eligible) and those who completed one or more sessions (N = 863). Non-starters were older but did not differ significantly in gender or baseline anxiety than those who complete 1 or more sessions.

Fig. 3 shows the proportion of children/adolescents who completed each session on the child or teen programme along with their parent's progression through their own programme. Of the 863 ‘starter’ child participants, 54% completed 4 sessions, 24% completed 7 sessions and 17% completed 9 (or more) sessions.

Fig. 3.

Fig. 3

Open in a new tab

Number of sessions completed by participants according to subject (child versus parent) and programme type (child versus teen).

The mean number of sessions completed by children/adolescents was 4.4 (sd = 2.8). Using multivariable Poisson regression models, older children completed fewer sessions than younger children (predicted sessions completed by age: 5.1 sessions at 7 years, 4.4 sessions at 12 years, 3.0 sessions at 17 years). Boys completed slightly fewer sessions on average than girls (7% fewer sessions; 95% CI 1–13%) and children with ‘elevated anxiety’ at baseline also completed slightly fewer sessions on average (8% fewer sessions; 95% CI 0–14%). The dropout rate after each session was on average one in five participants, with a peak after session 6 of around one in four (27%).

3.2. Evaluation of BRAVE _TA programme effectiveness

Evidence of effectiveness was calculated using the change in the primary outcome CAS8 anxiety score from baseline to last session for the 438 children from whom we had this paired data.

‘Starters’ with paired data were on average slightly younger than ‘starters’ without paired data (10.7 years vs 11.3 years, Wilcoxon F = 12, p < 0.001), but there were no group differences in gender (χ2 = 3.1, p = 0.079) or level of baseline anxiety (χ2 = 0.28, p = 0.59) across groups.

As shown in Table 2, for these 438 ‘starters’, there was an association between age and gender (with interaction) and CAS8 at baseline (p < 0.001), with older females having the highest anxiety scores. However baseline anxiety did not predict the number of sessions completed (p = 0.83) and neither age nor gender was associated with the mean change in anxiety (p = 0.33). Children/adolescents who completed more sessions reported greater reductions in anxiety (p < 0.001), with most of the improvement occurring within the first three to four sessions.

Table 2.

Change in the primary outcome measure (CAS8 scores from baseline to last session) and Cohen's d effect sizes.

CAS8 at baseline
 CAS8 change
 CAS8 change effect sizea
Mean 95% CI p value Mean 95% CI p value Mean 95% CI
Age by gender interaction p < 0.001 p = 0.33
 <11 years
 Male 10.7 (9.9 to 11.5) −3.2 (−4.0 to −2.4) 0.83 (0.6 to 1.0)
 Female 10.4 (9.7 to 11.1) −2.4 (−3.0 to −1.7) 0.62 (0.4 to 0.8)
 >11 years
 Male 10.3 (9.3 to 11.2) −2.2 (−3.1 to −1.3) 0.58 (0.3 to 0.8)
 Female 13.4 (12.7 to 14.1) −2.5 (−3.1 to −1.8) 0.64 (0.5 to 0.8)
Number of sessions completed p = 0.83 p < 0.001
 1 to 2 11.0 (10.4 to 11.7) −0.7 (−1.4 to −0.1) 0.19 (0.0 to 0.4)
 3 to 4 11.1 (10.3 to 11.8) −2.7 (−3.4 to −1.9) 0.70 (0.5 to 0.9)
 5 to 7 11.5 (10.6 to 12.4) −3.2 (−4.1 to −2.4) 0.84 (0.6 to 1.1)
 8 or more 11.1 (10.3 to 12.0) −3.6 (−4.4 to −2.9) 0.94 (0.7 to 1.1)
Open in a new tab
a

Cohen's d effect size (change in CAS 8) calculated using standard deviation of 3·85.

As shown in Fig. 4, programme effect sizes (Cohen's d) for children/adolescents in both baseline ‘not elevated’ and ‘elevated’ anxiety groups were in the moderate range (0.41–0.82 by child self-report). Completing more sessions led to greater reductions in anxiety, especially for those who had elevated anxiety at baseline. These improvements were confirmed by parent report from those parents who had completed three or more sessions of their parent programme. Parents of children within the ‘not elevated’ range at baseline (N = 104) reported small effects (Cohen's d = 0.26, 95%CI, 0.14 0.66), and parents of children/adolescents within the ‘elevated anxiety’ group at baseline (N = 295) reported effects in the moderate range (Cohen's d = 0.57, 95%CI, 0.33 0.81).

Fig. 4.

Fig. 4

Open in a new tab

Programme effects sizes (Cohen's d) for participants by number of sessions of BRAVE_TA completed and baseline anxiety levels.

3.2.1. Evidence of potential harm

For the 69 Starter/Completers who had paired CAS8 data and had completed 9 or more sessions, 84% had decreased their anxiety scores from baseline, but the other 16% had the same or worse (higher) self-reported CAS8 anxiety scores. We were unable to identify predictors of poor response using our data set of baseline information (age, gender, extent of anxiety at programme entry), and programme progression (CAS8 scores). The referrers of all these participants were contacted by the team psychiatrist of these poor outcomes and alternative treatment options available locally were discussed.

4. Discussion

BRAVE_TA, an online therapist assisted CBT programme for child/adolescent anxiety has been shown to be efficacious in university trial settings. This is the first evaluation of uptake and effectiveness of this ‘e’ therapy in a primary care setting. BRAVE_TA was made freely available to anxious children and adolescents in a district health board in New Zealand. These children had been assessed as having mild to moderate anxiety needing treatment by family doctors, school public health nurses and community mental health care providers, the health sector group most in contact with this age group. More than ninety percent of the referrals were deemed eligible to participate in the programme. The appropriateness of referrals was further confirmed by the participants, with three quarters reporting at least mild anxiety at programme commencement.

Uptake of this ‘e’ therapy remained steady over the period of study demonstrating that it was acceptable to referrers and families, was accessible and that delivery was feasible.

There was significant attrition between referral and starting the programme with 32% choosing not to start, despite having already sought help from the referrer, or having a diagnosis of anxiety being made by the referrer and receiving information on an evidence based treatment. Uptake is a common problem in online therapies. The freely available CBT programme for depression, Mood GYM, had <7% of users progressing beyond the first two sessions of the programme (Christensen et al., 2006). Similarly, MindSpot, a website providing online treatment services found that only a quarter of those who completed an assessment went on to use the treatment tool despite having similar symptom profiles and clinical levels of distress to those who accessed treatment options (Titov et al., 2017).

In our study we do not know what prompted a family's decision to not take up treatment. We followed up a subset (N = 65) families who were enrolled on the programme but did not start. The most common reason given by adolescents and parents at telephone interview was either that the treatment offered was no longer relevant or that the effort required for treatment was greater than the burden of symptoms/behaviours. Clearly if the child/adolescent does not want treatment then offering even a free effective treatment, delivered at home and with minimal treatment related disruptions to routines, is still not acceptable to some families For others, an ‘e’ therapy was not an acceptable treatment option, so ensuring that referrers offer technology-based treatments within a range of other options seems important.

Having started the BRAVE_TA programme, half of the participants completed four sessions, then stopped. Furthermore, over half of participants were in the population ‘normal’ anxiety range having completed their last session. The decision of participants to stop the programme may have been because they no longer needed it rather than because they ‘dropped out’. Calculating the ‘dose’ of treatment needed to treat anxiety in community settings is an uncertain art and perhaps participants ‘voting with their feet’ is a helpful gauge. However, for some participants with elevated anxiety, BRAVE_TA was only partially helpful, underscoring the importance of monitoring outcomes to ensure an efficient stepped care approach in the community setting.

Interestingly, anxiety levels at baseline did not predict the number of sessions completed by the child or their parent. So although children/adolescents were classified as in the non-elevated range at the start of the programme, there were clearly symptoms or behaviours that had prompted their parents to seek assessment, and referral and programme effectiveness was of a similar magnitude across both groups.

Having been referred to BRAVE_TA, younger children were more likely to engage and complete more sessions. This likely reflects the impact of parents in initiating referral and encouraging and supervising children to complete their sessions. However the existing evidence is mixed on whether parental involvement enhances outcomes (James et al., 2015) and a further Cochrane review is currently underway to examine this issue and other aspects of treatment delivery for child anxiety.

The results of our study fit well with the current growing literature on the effectiveness of computerised CBT. A meta-analysis of 13 RCTs (including 796 children and adolescents) of computerised/online programs targeting anxiety and/or depression found moderate post-test effect (Hedges g = 0.72) and, in the case of anxiety-specific programs the effect size was g = 0.68 (Ebert et al., 2015). This also compares well with the effectiveness of face-to-face psychological therapies of childhood anxiety (g = 0.66) (James et al., 2015).

That such a low therapist input (plus the programme content), could be effective for some children with high levels of anxiety is noteworthy. It implies that for some families with children with significant anxiety, home based ‘e’ therapy is a cost effective pathway.

Primary care practitioners clearly found BRAVE_TA helpful as exemplified by steady referral numbers of appropriate referrals. However, anxiety is still being under-recognised and treated in primary care. The child/adolescent population of Canterbury is around 100,000 (Statistics New Zealand, 2018) and with referral rates to BRAVE_TA of around 450 per annum, this represents <10% of the expected number of children with anxiety in a community. Despite the known effects of natural disasters on increasing child anxiety (Wang et al., 2013), these low referral rates further demonstrate the under recognition of this treatable disorder with life-long sequelae (Andrews and Erskine, 2003).

The major strength of this study is that it includes a large number of participants in a primary care setting, who received their intervention with minimal resource. There are, however, several limitations of the study. The assessment of programme effectiveness by self-report information from participants receiving the intervention is an inherent source of bias. Similarly, self-evaluation and reporting of anxiety, especially in younger children may be unreliable. As children and parents progress through the BRAVE_TA programme at different rates, it was not possible to align individual child and parent session completion data to corroborate changes in anxiety. However, grouped child self-reported improvements were confirmed by grouped parent report. We do not have ethnicity or socioeconomic data to judge whether the referrals were representative of the population of the catchment area. However, both primary care assessments for children under 13 years old and public health nurse school assessments are free in NZ, so the barrier of assessment and treatment costs should be minimal. A further limitation was the missing self-report anxiety data as the participants were able to bypass the anxiety ‘pop up’ evaluation and still access the session. There is a fine line between providing as few barriers as possible to accessing mental health treatment and ensuring outcome evaluation data is collected, making this a common limitation in real world setting evaluations. Finally, this was an open trial with no control group. However, both CBT and this version of CBT (BRAVE_TA) have already been shown to be efficacious for child/adolescent anxiety and as the principal aim of this study was to evaluate the feasibility of delivery in a real world setting, further evaluation against a control arm was not undertaken.

In summary, BRAVE_TA has proved to be an acceptable and effective ‘e' therapy tool in a ‘real world’ primary care setting for children with mild to moderate anxiety. This adds to the importance of having a menu of tools in the armamentarium of primary care workers who are the first port of call for families with concerns about their children's emotional and behavioural problems. ‘E’ therapies are proving to be an important part of the stepped care model for improving public mental health by treating anxiety in children and preventing later physical and psychological morbidity.

Contributors

KS- responsible for study conceptualisation, protocol development and data analysis strategy and manuscript writing.

SD, CF, WM, NW, ED, CW- responsible for protocol development, study implementation, modification of programme delivery and manuscript writing.

JW- responsible for data analysis strategy and implementation.

SM- responsible for study conceptualisation, protocol development, study implementation, data analysis and manuscript writing.

Declaration of interests

We declare no competing interests.

Acknowledgments

Acknowledgements

We wish to acknowledge the support of the Australian BRAVE _TA programme developers Prof Sue Spence, Dr. Caroline Donovan and Dr. Sonja March and the collegial support of Professor Sally Merry, University of Auckland.

Funding

This study was funded by the Canterbury District Health Board (CDHB), the public health provider for the Canterbury, NZ district.

References

  1. Andersson G., Titov N. Advantages and limitations of internet-based interventions for common mental disorders. World Psychiatry. 2014;13:4–11. doi: 10.1002/wps.20083. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Andrews G., Erskine A. Reducing the burden of anxiety and depressive disorders: the role of computerized clinician assistance. Current Opinion in Psychiatry. 2003;16:41–44. [Google Scholar]
  3. Bayer J.K., Rapee R.M., Hiscock H., Ukoumunne O.C., Mihalopoulos C., Clifford S., Wake M. The cool little kids randomised controlled trial: population-level early prevention for anxiety disorders. BMC Public Health. 2011;11:11. doi: 10.1186/1471-2458-11-11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bittner A., Egger H.L., Erkanli A., Costello E.J., Foley D.L., Angold A. What do childhood anxiety disorders predict? J. Child Psychol. Psychiatry. 2007;48:1174–1183. doi: 10.1111/j.1469-7610.2007.01812.x. [DOI] [PubMed] [Google Scholar]
  5. Christensen H., Griffiths K., Groves C., Korten A. Free range users and one hit wonders: community users of an internet-based cognitive behaviour therapy program. Aust N Z J Psychiatry. 2006;40:59–62. doi: 10.1080/j.1440-1614.2006.01743.x. [DOI] [PubMed] [Google Scholar]
  6. Costello E., Egger H., Angold A. Developmental epidemiology of anxiety disorders. In: Ollendick T.H., March J.S., editors. Phobic and Anxiety Disorders in Children and Adolescents: A Clinician's Guide to Effective Psychosocial and Pharmacological Interventions. Oxford University Press; New York: 2004. [Google Scholar]
  7. Ebert D.D., Zarski A.C., Christensen H., Stikkelbroek Y., Cuijpers P., Berking M., Riper H. Internet and computer-based cognitive behavioral therapy for anxiety and depression in youth: a meta-analysis of randomized controlled outcome trials. PLoS One. 2015;10 doi: 10.1371/journal.pone.0119895. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Fleming T., Bavin L., Lucassen M., Stasiak K., Hopkins S., Merry S. Beyond the trial: systematic review of real-world uptake and engagement with digital self-help interventions for depression, low mood, or anxiety. J. Med. Internet Res. 2018;20 doi: 10.2196/jmir.9275. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Ford T., Goodman R., Meltzer H. The British child and adolescent mental health survey 1999: the prevalence of DSM-IV disorders. J. Am. Acad. Child Adolesc. Psychiatry. 2003;42:1203–1211. doi: 10.1097/00004583-200310000-00011. [DOI] [PubMed] [Google Scholar]
  10. Hill C., Martin J.L., Thomson S., Scott-Ram N., Penfold H., Creswell C. Navigating the challenges of digital health innovation: considerations and solutions in developing online and smartphone-application-based interventions for mental health disorders. Br. J. Psychiatry. 2017;211:65–69. doi: 10.1192/bjp.bp.115.180372. [DOI] [PubMed] [Google Scholar]
  11. James A.C., James G., Cowdrey F.A., Soler A., Choke A. Cognitive behavioural therapy for anxiety disorders in children and adolescents. Cochrane Database Syst Rev. 2015:CD004690. doi: 10.1002/14651858.CD004690.pub4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Jokela M., Ferrie J., Kivimaki M. Childhood problem behaviors and death by midlife: the British National Child Development Study. J. Am. Acad. Child Adolesc. Psychiatry. 2009;48:19–24. doi: 10.1097/CHI.0b013e31818b1c76. [DOI] [PubMed] [Google Scholar]
  13. Jolstedt M., Wahlund T., Lenhard F., Ljotsson B., Mataix-Cols D., Nord M., Ost L.G., Hogstrom J., Serlachius E., Vigerland S. Efficacy and cost-effectiveness of therapist-guided internet cognitive behavioural therapy for paediatric anxiety disorders: a single-centre, single-blind, randomised controlled trial. Lancet Child Adolesc Health. 2018 doi: 10.1016/S2352-4642(18)30275-X. [DOI] [PubMed] [Google Scholar]
  14. Kessler R.C., Ruscio A.M., Shear K., Wittchen H.U. Epidemiology of anxiety disorders. Curr. Top. Behav. Neurosci. 2010;2:21–35. [PubMed] [Google Scholar]
  15. March S., Spence S.H., Donovan C.L. The efficacy of an internet-based cognitive-behavioral therapy intervention for child anxiety disorders. J. Pediatr. Psychol. 2009;34:474–487. doi: 10.1093/jpepsy/jsn099. [DOI] [PubMed] [Google Scholar]
  16. Reardon T., Spence S.H., Hesse J., Shakir A., Creswell C. Identifying children with anxiety disorders using brief versions of the Spence Children's Anxiety Scale for children, parents, and teachers. Psychol Assess. 2018;30(10):1342–1355. doi: 10.1037/pas0000570. Advance online publication. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Spence S.H. A measure of anxiety symptoms among children. Behav. Res. Ther. 1998;36:545–566. doi: 10.1016/s0005-7967(98)00034-5. [DOI] [PubMed] [Google Scholar]
  18. Spence, S. H. 2018. Spence Children's Anxiety Scale. [Online]. Available: http://www.scaswebsite.com/ [Accessed].
  19. Spence S.H., Donovan C.L., March S., Gamble A., Anderson R., Prosser S., Kercher A., Kenardy J. Online CBT in the treatment of child and adolescent anxiety disorders: issues in the development of BRAVE-ONLINE and two case illustrations. Behav. Cogn. Psychother. 2008;36:411–430. [Google Scholar]
  20. Spence S.H., Donovan C.L., March S., Gamble A., Anderson R.E., Prosser S., Kenardy J. A randomized controlled trial of online versus clinic-based CBT for adolescent anxiety. J. Consult. Clin. Psychol. 2011;79:629–642. doi: 10.1037/a0024512. [DOI] [PubMed] [Google Scholar]
  21. Stasiak K., Merry S.N., Frampton C., Moor S. Delivering solid treatments on shaky ground: feasibility study of an online therapy for child anxiety in the aftermath of a natural disaster. Psychother. Res. 2016:1–11. doi: 10.1080/10503307.2016.1244617. [DOI] [PubMed] [Google Scholar]
  22. Statistics New Zealand National Population Census. 2018 https://www.stats.govt.nz/2018-census/ [Google Scholar]
  23. Titov N., Dear B.F., Staples L.G., Bennett-Levy J., Klein B., Rapee R.M., Andersson G., Purtell C., Bezuidenhout G., Nielssen O.B. The first 30 months of the MindSpot clinic: evaluation of a national e-mental health service against project objectives. Aust N Z J Psychiatry. 2017;51:1227–1239. doi: 10.1177/0004867416671598. [DOI] [PubMed] [Google Scholar]
  24. Wang C.W., Chan C.L., Ho R.T. Prevalence and trajectory of psychopathology among child and adolescent survivors of disasters: a systematic review of epidemiological studies across 1987–2011. Soc. Psychiatry Psychiatr. Epidemiol. 2013;48:1697–1720. doi: 10.1007/s00127-013-0731-x. [DOI] [PubMed] [Google Scholar]

Articles from Internet Interventions are provided here courtesy of Elsevier

RESOURCES