The Brain Games study: protocol for a randomised controlled trial of computerised cognitive training for preventing mental illness in adolescents with high-risk personality styles

Abstract

Introduction

A broad range of mental disorders are now understood as aberrations of normal adolescent brain development. In both adolescents and adults, executive dysfunction has been implicated across a range of mental illnesses, and enhancing executive functioning may prove to be a useful prevention strategy for adolescents at risk for a range of psychopathology.

Methods and analysis

This study will consist of a double-blind, randomised controlled trial with a 12-month follow-up period. Participants will consist of 200 people aged 16–24 years who are at risk for a range of mental disorders based on personality risk factors, but have not experienced a lifetime mental illness as determined by a structured diagnostic interview. Participants will be randomly allocated to either an intervention group who complete an online cognitive training programme specifically targeting executive functioning ability or a control group who complete an online cognitive training programme that has limited executive functioning training potential. Superiority of the executive functioning training programme compared with the control training programme will be assessed at baseline, post-training and at 3-month, 6-month and 12-month follow-up. All assessments will be conducted online. The primary outcome of the study will be general psychopathology as measured by the Strengths and Difficulties Questionnaire. Secondary outcomes will include executive functioning ability, day-to-day functioning and alcohol consumption. All analyses will be undertaken using mixed-model repeated measures analysis of variance with planned contrasts.

Ethics and dissemination

Ethics approval has been obtained from the University of New South Wales Human Research Ethics Committee (HC15094). Results of the trial immediately post-treatment and at 12 months follow-up will be submitted for publication in peer-reviewed journals.

Trial registration number

ACTRN12616000127404; Pre-results.

Strengths and limitations of this study.

• This study has been designed to demonstrate genuine transfer of skills beyond task-specific learning, a limitation often identified within the cognitive training literature.

• This study includes a rigorous active control condition meaning any differences between conditions can be confidently ascribed to improvements in executive functioning rather than other confounding factors.

• This study will assess whether any effects are maintained over an extended 12-month follow-up period.

• Participant contact will be limited and many processes will be automated, meaning that significant attrition is expected and has been accounted for in sample size calculations.

• It may also be that the executive functioning intervention tasks are more engaging than the control tasks by virtue of their complexity and the mental process required for their completion, with subsequent differential attrition rates across the conditions.

Introduction

Anxiety, depressive and substance use disorders frequently have the same risk factors, co-occur and interact in adolescence and young adulthood. Among young people (aged 15–24 years), the top 10 causes of burden of disease are dominated by mental illness and substance use.¹ The US National Comorbidity Survey Adolescent Supplement indicated that approximately one in four adolescents in the general population met criteria for an anxiety or affective disorder and one in ten adolescents in the general population met criteria for a substance use disorder.² The same data indicated that almost one-third (29%) of adolescents with one disorder in the general population also met criteria for another disorder, with a mean of three psychiatric disorders among those with comorbidity.³ Mental illness is highly prevalent and comorbid in young people, and prevention programmes need to be initiated early to reduce the occurrence, cost and significant functional impairment associated with these problems.

Given their typical age of onset, a broad range of mental disorders are increasingly being understood as the result of aberrations of developmental processes that normally occur in the adolescent brain.^4–6 Executive functioning, and its neurobiological substrate, the prefrontal cortex, matures during adolescence.⁵ The relatively late maturation of executive functioning is adaptive in most cases, underpinning characteristic adolescent behaviours such as social interaction, risk taking and sensation seeking which promote successful adult development and independence.⁶ However, in some cases it appears that the delayed maturation of prefrontal regulatory regions leads to the development of mental illness, with neurobiological studies indicating a broad deficit in executive functioning which precedes and underpins a range of psychopathology.⁷ A recent meta-analysis of neuroimaging studies focusing on a range of psychotic and non-psychotic mental illnesses found that grey matter loss in the dorsal anterior cingulate, and left and right insula, was common across diagnoses.⁸ In a healthy sample, this study also demonstrated that lower grey matter in these regions was found to be associated with deficits in executive functioning performance. Similarly, another recent functional imaging study focused on 1129 community youths (mean age 15.5 years) and investigated the relationship between psychopathology and activation of the executive system during a working memory task.⁹ Overall psychopathology was associated with hypoactivation in the frontal pole, anterior cingulate, anterior insula and precuneus, implicating a network of executive regions across a range of psychiatric diagnoses. In both adolescents and adults, executive dysfunction has therefore been implicated across a range of mental illnesses.

While the functional and structural organisation of the human brain was once considered static after critical developmental periods, it is now evident that neural changes are possible throughout the lifespan.¹⁰ Experience-dependent neural reorganisation, or neuroplasticity, is apparent in response to repetitive and adaptive task engagement, disease, neglect and injury.¹¹ Cognitive training is one of many interventions (including brain stimulation, neuropharmacology, physical exercise and neurofeedback) that harnesses experience-dependent neuroplasticity with the aim of achieving durable behavioural and functional change.¹¹ Cognitive training generally consists of a programme of exercises targeting specific cognitive skills, such as executive functioning, with the aim of achieving adaptive neural changes which underpin improvements in cognition and/or behaviour.¹² Cognitive training therapies, targeting a variety of cognitive processes, have demonstrated success in a range of clinical populations, including preliminary evidence of improvement in symptoms and cognition in those with affective, anxiety and substance use disorders.¹² The usefulness of cognitive training for schizophrenia has been studied extensively, with the research suggesting that such interventions can lead to enduring benefits for cognition and contribute to gains in overall functioning in individuals with both early^{13 14} and established^{15 16} symptoms.

Evidence therefore suggests that cognitive training may be a useful intervention strategy for those already showing symptoms of psychopathology, whether delivered during the early stages of mental illness or later in the course of disease progression. However, the effectiveness of cognitive training as a preventative strategy has not been thoroughly investigated. This is despite the evidence suggesting that cognitive impairments,⁴ and associated loss of day-to-day functioning,^{17 18} frequently precede the onset of mental illness and may represent a non-specific prodromal phase for a broad range of mental illnesses. As such, for individuals identified as at high risk for developing a mental illness, cognitive training may therefore delay and, in some cases, prevent the onset of a clinical disorder. Given the evidence to suggest that deficits in executive functioning are associated with psychopathology across a range of mental illnesses,^{8 9} this may be particularly the case for cognitive training using exercises focusing on this domain.

A small pilot study has demonstrated the feasibility of using cognitive training as a targeted prevention strategy during the critical adolescent period.¹⁹ In this study, 15 adolescents (mean age ~13 years) experiencing social, emotional and behavioural difficulties, but without a clinical diagnosis, were randomised to a cognitive training intervention group (n=7) or a passive control group (n=8). The training group completed a 30–40 min battery of visuospatial and verbal working memory tasks, 5 days a week, for 5 weeks. Compared with the passive control group, those in the intervention group showed significantly better post-training scores on measures of IQ, inhibition, test anxiety and teacher-reported behaviour, attention and emotional symptoms. These findings provide encouraging support for the notion that cognitive training delivered to at-risk adolescents may potentially prevent the onset of more serious social, emotional and behavioural problems.

Another study piloted a preventative cognitive training programme, this time for those at clinical high risk for psychosis.²⁰ While there were encouraging results, particularly in respect to improvements in processing speed and prodromal symptoms, there was no comparison group used. However, the findings provide further evidence of the feasibility of an intensive cognitive training programme for adolescents at risk for the development of mental illness. On the other hand, in another pilot study targeting individuals at clinical high risk for psychosis, there were no significant differences in cognition between the intervention and control groups immediately post-training.²¹ Furthermore, high attrition rates led the authors to suggest that more engaging interventions should be trialled, especially for younger age groups.

Despite these promising findings in terms of both the treatment and prevention of mental illness, several limitations of the cognitive training literature have been noted.²² The ultimate goal of cognitive training is to demonstrate that improvements in performing a particular task transfer to improvements in an underlying cognitive ability more generally (near transfer, ie, training on a specific executive functioning task leads to improvements in overall executive functioning ability). This improvement in cognition generally would then be the basis for any improvements in behaviour (far transfer, ie, improvements in overall executive functioning ability lead to decreases in psychopathology). However, the ways in which near transfer has been investigated in many previous studies have been flawed. Often only a single untrained task is used to measure near transfer to the underlying cognitive ability.²² The capacity for a single task to accurately reflect multifaceted and complex cognitive abilities is questionable. Multiple tasks which measure different dimensions of the underlying cognitive ability are therefore required. Many previous studies have also used no control group or a ‘no contact’ control group, which means that it is often not possible to rule out placebo effects.²² Finally, the assessment of training effects rarely extends beyond immediately post-training,²² meaning that the durability of any training effects has not been established.

The primary objective of the Brain Games study is to therefore investigate whether cognitive training delivered online within a ‘smart gaming’ platform is a viable targeted prevention strategy by examining its ability to reduce psychopathology in young people at high risk for developing a mental illness. In order to identify those at risk for developing a mental illness, this study will target personality risk factors, including hopelessness, anxiety sensitivity, impulsivity and sensation seeking, which have been shown to reliably predict substance misuse, anxiety, emotional and behavioural disorders in young people.^{23 24} It is hypothesised that the intervention cognitive training programme (focusing on executive functioning) will be more effective than the active control cognitive training programme (focusing on cognitive abilities other than executive functioning) in reducing psychopathology. Secondary aims include assessing the comparative effects of the intervention and control training programme on executive functioning ability, day-to-day functioning and alcohol consumption.

Methods and analysis

Study design

This study will consist of a parallel group randomised controlled trial with a 12-month follow-up period. The conduct and reporting of the trial will be in accordance with the CONSORT Consolidated Standards of Reporting Trials statement for non-pharmacological interventions. Participants will be randomly allocated to one of two cognitive training conditions: (1) the intervention group (cognitive training specifically focused on executive functioning) or (2) the control group (cognitive training focusing on other cognitive abilities). The allocation ratio will be 1:1. Superiority of the executive functioning training programme compared with the control training programme will be assessed at baseline, post-training and at 3-month, 6-month and 12-month follow-up. All assessments will be conducted online. The primary outcome of the study will be general psychopathology as measured by the Strengths and Difficulties Questionnaire (SDQ). Secondary outcomes will include executive functioning ability as measured by online neuropsychiatric tasks, as well as day-to-day functioning and alcohol consumption as measured by standardised measures listed below. See figure 1 for enrolment, assigned intervention, intervention and follow-up scheme. 

Figure 1.

Flowchart of the randomised control trial: enrolment, assigned intervention, intervention and follow-up scheme. SDQ, Strengths and Difficulties Questionnaire.

Sample size calculations

The primary outcome measure will be scores on the self-report version of the SDQ, a measure of psychopathology in young people with excellent psychometric properties²⁵ and test–retest reliability.²⁶ One previous pilot study has investigated the effect of cognitive training on psychopathology in at-risk young people using the SDQ.¹⁹ According to this study, there was a between-group effect size of 0.36 for the SDQ. A similar effect size difference on the SDQ would therefore be expected in the current study. A total sample size of n=140 (n=70 per group) is powered to have an 80% chance of detecting 0.36 effect size differences at p<0.5. We will therefore aim to recruit n=220 (n=110 per group) in order to account for potential attrition.

Procedure

Participants and recruitment

Participants will be aged 16–24 years and at risk for a range of mental disorders. Participants will be included if they meet the following criteria: (1) at high risk for development of a mental illness based on elevated levels of personality risk factors, including hopelessness, anxiety sensitivity, impulsivity and sensation seeking (as measured by the Substance Use Risk Profile Scale (SURPS), described below); (2) ability to access the internet via a computer; (3) residing within Australia; and (4) willingness to provide contact details. Exclusion criteria will be (1) insufficient English comprehension; (2) a lifetime mental illness diagnosis; (3) current involvement in psychotherapy or taking medication for mental illness, pain, thyroid problems and/or epilepsy; (4) intellectual disability; (5) a history of neurological or cardiovascular disorders, brain surgery, electroconvulsive or radiation treatment, thyroid disorders, brain haemorrhage or tumour, stroke, diabetes, seizures or epilepsy; and (6) significant previous experience with online cognitive training programme.

Community participants will be recruited through advertisements on social media platforms. Recruitment through social media has been shown to be effective and cost-efficient, with obtained samples of similar representativeness as those recruited via traditional methods²⁷ Prospective participants will indicate their interest by clicking on the advertisement, after which they will be taken to the study website (www.braingames.org) that provides more detailed information about the study. After reading this information, the participant will continue onto eligibility screening which will be conducted through the study website. Online eligibility screening will assess for all exclusion and inclusion criteria, except for a prior diagnosis of a mental illness. If eligible, participants will then be contacted by a trained researcher from the University of New South Wales, Australia, who will conduct a telephone-administered diagnostic interview to assess for a lifetime diagnosis of major depressive disorder, panic disorder, social anxiety disorder, generalised anxiety disorder, obsessive-compulsive disorder, post-traumatic stress disorder, alcohol dependence, other substance dependence, attention deficit hyperactivity disorder, conduct disorder and oppositional defiant disorder. Those who do not meet eligibility criteria after completion of the online or telephone-administered interviews will be informed of their ineligibility, and given referrals and/or information for crisis care where necessary. After telephone screening, eligible participants will be sent a link to complete the baseline assessment through the study website. Following baseline assessment, participants will be given a unique code which will be randomly associated with one of the two cognitive training conditions. Randomisation of these codes to the intervention and control conditions will be conducted by an offsite technician who will allocate all unique codes to one of the two conditions through the website random.org. Both the research team and participant will be blind to this allocation. Participants will be free to withdraw from the study at any time, and will not be prohibited from seeking concomitant care. Those participants who do withdraw will not complete any further assessments. Enrolment into the study will be rolling, with each participant starting their training the day after they have completed their baseline tasks.

Participants will provide online consent for the internet-administered eligibility screening, as well as formal written consent for the diagnostic interview and participation in the study itself. Online and written consent forms have been included in online supplementary files 1 and 2.

Intervention and control conditions

Both the intervention and control conditions will complete cognitive training consisting of computerised tasks provided by Lumosity (http://www.lumosity.com/), and administered on a remote internet-connected computer owned by the participant. All tasks within the commercially available Lumosity package have ‘game-like’ features making them visually engaging and motivating. The tasks are designed to be dynamic and adaptive, such that the difficulty increases as the participants’ performance improves. Training for both conditions will consist of an intensive programme of 10 games per day (~30–40 min), 5 days per week, over 5 weeks. Training compliance will be assessed during the active training phase, with reminder emails sent after one missed session, and a further email sent after two missed sessions. Non-compliance to the training protocol will be recorded if a participant (1) misses four or more consecutive days of training; (2) completes less than 20 full sessions of training; or (3) is unable to complete their programme within 6 weeks (42 days). However, post-training and follow-up data will be collected for non-compliant participants unless they formally withdraw from the study or are unable to be contacted. Given the non-invasive nature of the intervention, adverse events are not anticipated. Spontaneous reporting of adverse events will be discussed with the study clinicians and responded to according to their recommendations. Training games for the intervention group will specifically focus on executive functioning, and will be based on classic paradigms used in cognitive neuroscience and executive functioning batteries. The control group will be administered cognitive training games that do not target executive functioning, and instead focus on field of view, verbal fluency and quantitative reasoning. The features of the control tasks will be matched to those of the intervention tasks in all other respects. Tasks included in both the intervention and control training programme were selected on the basis of consensus ratings with experienced clinicians.

Assessment occasions

Both those assigned to the intervention and to the control group will be assessed at baseline, post-trial and at 3-month, 6-month and 12-month follow-up intervals. All assessments at each occasion will be fully automated and conducted online using the participants’ own computer.

Measures

Eligibility screening

Demographic data on gender, age, country of birth and rurality will be collected online before eligibility is assessed. Eligibility screening will consist of a 23-item self-report online questionnaire assessing personality risk factors (the SURPS),²⁸ with four subscales representing hopelessness, anxiety sensitivity, impulsivity and sensation seeking. Participants scoring one or more SD above the mean on any of the four SURPS subscales will be deemed eligible for the current study. Threshold means and SD will be based on a previous validity study of the SURPS in Australian adolescents.²³ Scores on these subscales have been shown to reliably predict future substance use, as well as anxiety, emotional and behavioural disorders. The SURPS has been shown to have good concurrent and predictive validity, and the separate subscales display good specificity.²⁹

For those aged less than 18 years, the telephone-administered diagnostic interview for lifetime mental illnesses will be conducted using a lifetime version of the Mini International Neuropsychiatric Interview for Children and Adolescents (MINI-KID). The MINI-KID is a structured, lay-administered self-report diagnostic interview for children of ages 6 to 17 years old which is designed to assess DSM-IV Diagnostic and Statistical Manual of Mental Disorders, 4th Edition (DSM-IV) psychiatric disorders. The MINI-KID has been shown to have excellent reliability and excellent concordance with clinician diagnoses.³⁰ For those aged 18 years and older, a lifetime version of the MINI will be administered. The MINI is also a structured, lay-administered self-report diagnostic interview designed to assess DSM-IV psychiatric disorders with excellent reliability and good concordance with a clinician-administered diagnostic interview.^{31 32}

Baseline assessment

Psychopathology will be assessed using the SDQ, a 25-item self-report behavioural screening inventory for children and adolescents which measures positive and negative attributes of participants.³³ The SDQ has been used extensively and has strong psychometric properties.²⁵ The SDQ is also brief and multidimensional, providing information across a wide range of psychopathology as required for the current study. An online version of this questionnaire will be developed for the current study in consultation with the original SDQ developers. Recently, a young adult version of the SDQ has also been developed, with minor changes to the wording and scoring of the adolescent instrument. According to the developers of this instrument, it has been used extensively in practice but validation studies of this instrument have only been published in special populations at present.^{34 35} The young adult version will also be used to assess psychopathology in those aged 18 years and over in the current study to allow continuity of measurement across the full sample.

An online version of the World Halth Organisation Disability Assessment Schedule (WHODAS) 2.0 12-item version (World Health Organisation, 2010) will be used to assess overall functioning and disability experienced as a result of health conditions (including mental illness or substance use) over the past 30 days. The WHODAS has been used extensively as a measure of functioning and disability and has been validated for both online use and in individuals aged 16 years and over in the Australian population.^{36 37}

Participants’ recent and historical alcohol consumption will be assessed online by 14 questions adapted from the School Health and Alcohol Harm Reduction Project ‘Patterns of Alcohol’ index.³⁸ These questions ask responders about the frequency and quantity of different drinking behaviours, and age of first participation in these behaviours.

Online executive functioning assessment will be included as a measure of executive functioning improvement on untrained tasks. This assessment will consist of the N-Back Task³⁹ to evaluate working memory, the Trail Making Test⁴⁰ to assess task shifting and the Stroop Task⁴¹ to evaluate inhibitory control. These tasks will be administered using Inquisit software (http://www.millisecond.com/download/library/).

Participants will also be asked to answer an online questionnaire about their satisfaction with the training programme.

Follow-up assessments

Each of the assessments at post-training, 3-month, 6-month and 12-months follow-up will consist of the online versions of the SDQ, SURPS, WHODAS and alcohol consumption questionnaires, as well as the online executive functioning assessment. At the 12-month assessment, data will also be collected on service use and any use of the Lumosity programme over the follow-up period.

All data collected in computerised format will be stored on password protected computers in databases using predefined codes. Rigorous data encryption will restrict external access to participant online content. The website uses industry leading encryption for all transmissions. The SSL certificate includes domain authentication and 256-bit SSL encryption. Hyper Text Transfer Protocol Secure (HTTPS) (SSL; Secure Sockets Layer) will be used for complete website including all subdomains. A data access layer is used for all database functionality requirements within the site with the majority of database connections set up with stored procedures.

Statistical analysis

Data will be analysed on an intention-to-treat basis. All analyses will be undertaken using mixed-model repeated measures analysis of variance, with measurement occasion as a within-groups factor, and experimental group as a between-groups factor. These models are robust to missing data under missing-at-random assumptions. Relationships between observations at different occasions will be modelled with an unstructured covariance matrix. For each experimental group, planned contrasts will be used to compare changes from baseline to post-test, 3-month, 6-month and 12-month follow-up intervals. Given the high rates of attrition expected, secondary analyses will also be conducted for all outcome variables on a per-protocol basis.

Ethics and dissemination

Ethical approval for the Brain Games study has been granted from the UNSW Human Research Ethics Committee (HC15094). Any protocol amendments will first be discussed by the team of investigators after which an ethics modification will be submitted.

Results of the trial immediately post-treatment and at 12-month follow-up will be submitted for publication in peer-reviewed journals. After all study data have been collected, all participants will receive a document outlining the main results of the study. All lead investigators will be listed as authors on all publications unless they opt out of authorship.

The de-identified dataset generated and analysed during the current study is available from the corresponding author on reasonable request.

Discussion

This study protocol presents the design of a randomised controlled trial that assesses whether online executive functioning training is a viable strategy for the targeted prevention of mental illness and substance use in high-risk youth. The primary aim of the Brain Games trial is to investigate whether an executive functioning training programme is more effective at preventing symptoms of psychopathology than cognitive training that has limited executive functioning training potential. By investigating a novel prevention strategy targeting executive functioning, the Brain Games study will be the first full-scale trial conducted internationally that investigates the utility of cognitive training in the prevention of mental illness in high-risk adolescents. This study targets underlying executive functioning deficits that have been identified across the boundaries of traditional categorical psychiatric diagnoses.

Novel universal prevention programmesthat cross diagnostic boundaries are currently being trialled in Australian schools.⁴² For high-risk adolescents, however, universal prevention programme has been shown to have limited benefits.⁴³ In terms of practical applications, it is envisaged that transdiagnostic prevention strategies that target high-risk adolescents, such as the present one, would eventually complement and augment a transdiagnostic approach to school-based universal prevention. This stepped-care sequential prevention model has the potential to maximise outcomes for both high-risk and low-risk students across a range of mental illnesses and reduce the considerable burden of disease associated with these illnesses in adolescence.