Outcomes and outcome measurement instruments reported in randomised controlled trials of anxiety disorder treatments in children and adolescents: a scoping review protocol

Abstract

Introduction

Paediatric anxiety disorders (AD) are prevalent and persistent mental health conditions worldwide affecting between 10% and 20% of children and adolescents. Despite the high prevalence of paediatric AD, there is limited understanding of which treatments work best. Outcome heterogeneity across paediatric mental health trials has been a significant factor in hindering the ability to compare results and assess the efficacy of such trials. This scoping review will help to identify and synthesise the outcomes reported in paediatric AD trials to date.

Methods and analysis

Following the Joanna Briggs Institute scoping review methodology, a comprehensive electronic bibliographic database search (MEDLINE, APA PsycINFO, Embase, CINAHL) strategy will be applied to identify articles examining interventions for children diagnosed with an AD. Articles will be eligible for inclusion if they assess at least one AD intervention (eg, psychological), in children 4–18 years of age inclusive. Initial title and abstract screening will be completed by two trained reviewers independently and in duplicate. Full-text screening of each included article will be completed independently and in duplicate by two of three trained reviewers. Identified outcomes will be mapped to a standard outcome taxonomy developed for core outcome sets. Trial and outcome characteristics will be synthesised using quantitative metrics (counts and frequencies).

Ethics and dissemination

As this is a scoping review of the literature and patient information or records were not accessed, institutional ethics approval was not required. Results of this scoping review will be disseminated to clinicians, researchers inclusive of trialists and other stakeholders invested in outcome selection, measurement and reporting in paediatric AD trials. In addition, scoping review results will inform the development of a Core Outcome Set for paediatric AD trials—a minimum set of outcomes that should be measured across trials in an area of health, without precluding the inclusion of other outcomes.

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• This review follows systematic methods based on the Joanna Briggs Institute scoping review methods manual and the guidelines provided by the Core Outcome Measures in Effectiveness Trials Initiative.

• This review will employ a rigorous search strategy using validated search filters developed with research librarians and will include studies published in English within the last 10 years, to prioritise the most recently conducted and reported paediatric anxiety disorder trials.

• Only English language trials will be included.

• The proposed scoping review will allow for comprehensive identification of reported outcomes and outcome measurement instruments.

Background

Anxiety disorders (ADs) are among the most common mental health conditions found in the paediatric population.¹ Among the various age groups, the prevalence of AD has been found to be between 10% and 20%.^2–4 ADs are known to impact important aspects of a child’s life, including their family life, social functioning (eg, interactions with peers) and education (eg, academic achievement).^{5 6} Children and adolescents with AD are also at increased risk of developing other psychiatric illnesses (eg, depression), as well as engaging in suicidal behaviours (eg, suicidal ideation and attempts) and substance use.^5–8 Child and adolescent AD often continue into adulthood,^{9 10} resulting in impaired functioning in areas of life such as interpersonal relationships, finances and personal health.^{5 11} The burdens of paediatric AD highlight the need for targeted interventions.

Despite the high prevalence and impact of paediatric AD, there is a lack of understanding of what treatments are optimal. The most commonly used treatments currently include cognitive–behavioural therapy (CBT) and pharmacological treatments with a serotonin selective reuptake inhibitor.¹ However, the ability to compare treatments and assess their efficacy and safety profile is hampered due to significant heterogeneity in outcome data reported. To date, research on these treatments have typically used many different symptom rating scales to assess changes that occur during treatment. Due to a lack of standardisation across commonly used paediatric mental health instruments, there may be little symptom overlap between measurement scales resulting in individual trials, producing results unique to the specific scale or questionnaire used.¹² For example, a Cochrane systematic review of CBT trials in paediatric AD found 27 different outcome measurement instruments (OMIs) used to measure anxiety symptoms across 87 randomised controlled trials (RCTs).¹³ Similarly, the International Consortium for Health Outcomes Measurement reviewed 247 treatment studies for child and youth internalising disorders (anxiety, depression, obsessive–compulsive disorder, post-traumatic stress disorder and identified 30 different OMIs used to measure anxiety.¹⁴ In a recently conducted scoping review of adolescent major depressive disorder RCTs, 86 unique outcomes were found across 32 RCTs, assessed using 118 different OMIs, highlighting the heterogeneity of outcomes and OMIs in paediatric mental health trials.¹⁵ Some evidence to date suggests that similar heterogeneity of outcomes and OMIs is seen in paediatric AD, although a scoping review to evaluate this has not yet been done.¹⁶

In other areas of medicine, the development of a Core Outcome Set (COS) has led to increased consistency across trials, maximised potential for a trial to contribute to systematic reviews of key outcomes, increased relevance to stakeholders, and a reduction in selective outcome reporting, which can lead to a biased estimation of treatment effects.^17–19 A COS as defined by the Core Outcome Measures in Effectiveness Trials Initiative (COMET), is a standardised set of outcomes that should be measured and reported in specific areas of health or healthcare.¹⁷ Development of a COS warrants consensus on what is to be measured, followed by determination on how the outcomes will be defined and measured.¹⁷ Development of a COS would allow for future paediatric AD RCTs to improve outcome selection and measurement, which would increase the comparability between trials and foster meaningful comparisons of trials and meta-analysis.¹⁷

Across the field of paediatric mental health worldwide, there is increased recognition for the need for standard sets of outcomes and the International Consortium for Health Outcomes Measurement to date has developed standard sets of outcomes for use in routine clinical treatment of various mental health conditions including paediatric anxiety and depression.¹⁴ Importantly, the context and requirements of clinical trials are quite distinct from those of routine care and a COS developed specifically for use in paediatric AD clinical trials is lacking, as evidenced by a search of the COMET database. Similar searches through The International Prospective Register of Systematic Reviews, COnsensus-based Standards for the selection of health Measurement INstruments, and Open Science Framework, show no preregistrations for a review of this nature. Separate work is ongoing to develop a COS for use in adolescent depression RCTs.²⁰

This protocol outlines the methods for a scoping review that will define the outcomes measured in paediatric AD trials to date, an important first step that will inform the development of a COS for this health condition. Plans to develop a COS in paediatric AD were registered with the COMET initiative in October 2020.²¹ The aim of this scoping review is to identify and synthesise the outcomes and OMIs reported in paediatric AD RCTs between January 2010 and May 2021. The results will be used to evaluate the extent of existing outcome heterogeneity in RCTs conducted in children and adolescents with AD and which will provide us with knowledge of the outcomes and OMIs used to date in paediatric AD trials. The results will be used to evaluate the extent of existing outcome heterogeneity in RCTs conducted in children and adolescents with AD, which will provide an initial list of outcomes for consideration in the development of a COS.

Methods

Study design

A scoping review is a form of knowledge synthesis that maps the key concepts underlining an area of research. This process helps to identify the range, sources and types of available evidence^{19 22 23} and is the most suitable approach to address the primary objectives of this study. This protocol follows the recommendations outlined in the Joanna Briggs Institute scoping review methods manual¹⁹ and follows appropriate systematic methods.²⁴

Protocol

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Protocols (PRISMA) reporting guideline was followed to draft this review protocol.²⁵ The reporting of the scoping review will follow the PRISMA extension for Scoping Review (online supplemental marerial 1).²⁶ The development of the search strategy for this scoping review commenced in May 2021, before the submission of this protocol. Initial article screening is ongoing. Data collection began in December 2021, extraction and analysis is anticipated to be completed by October 2022.

Eligibility criteria

Inclusion criteria for eligible studies are based on the Population, Intervention, Comparators, Outcomes (PICO) approach.²⁷ Studies will be eligible if published between January 2010 and May 2021 (inclusive), as we aim to capture the most recently conducted and reported trials. There will be no restrictions on when outcomes were measured or duration of follow-up after administration of the intervention. Trials from any country or setting will be eligible for inclusion however only RCTs published in English will be included for feasibility of review. Pilot and feasibility trials and interim reports will be eligible for inclusion only when a final trial report is not available for inclusion to avoid double counting of any outcomes.¹⁵

Study population

Studies which include children and adolescents aged 4–18 years with a diagnosis made using a validated diagnostic interview and/or through a clinician diagnosis of an AD as defined by the diagnostic criteria in the Diagnostic and Statistical Manual of Mental Disorders-5 or as per the International Statistical Classification of Diseases criteria will be eligible (eg, generalised anxiety disorder, separation anxiety disorder, social anxiety disorder, selective mutism, panic disorder, agoraphobia, neurotic disorders and phobic disorders).^{28 29}

Studies involving children and adolescents with comorbid psychiatric conditions will be included, provided that the AD is the primary diagnosis. RCTs that include participants with ages outside the selected range will be included if (A) The reported mean or median participant age falls within the range of 4–18 years or (B) There is a subgroup analysis that contains children and adolescents aged between 4 and 18 years inclusive with AD (eg, trials with a subgroup analysis of ages 8–12 years would be eligible, but a subgroup analysis of ages 16–22 years would not be eligible).

Study interventions

All treatment interventions for paediatric AD (ie, pharmacological and non-pharmacological) will be eligible. Prevention studies for paediatric AD will not be eligible.

Comparison

Studies must have a control group and randomisation; however, there will be no comparator restrictions.

Outcomes

All outcomes will be eligible, including all outcomes specified in the published trial methods to be collected for randomised group comparisons. Treatment emergent adverse events will not be included as these are not planned outcomes of interest and are specific to the intervention of interest.¹⁵

Information sources and search strategy

Studies for inclusion will be found using an electronic bibliographic database search applied to MEDLINE (MEDical Literature Analysis and Retrieval System), the APA Psychological Information database (APA PsycINFO), Excerpta Medica database (Embase), and CINAHL (Cumulative Index to Nursing and Allied Health Literature).

The search strategy was collaboratively developed by review team authors experienced in electronic bibliographic database search strategies (NJB and RD), including a child and adolescent psychiatrist (SM), in consultation with an experienced research librarian (QM). Our search strategy was supported by an analysis of the Medical Subject Headings title terms and text words, abstracts, and keyword headings from a sample of relevant trial articles compiled via informal literature searching. The proposed search strategy was then reviewed by a second expert research librarian (SV) using the Peer Review of Electronic Search Strategies checklist, which is a formal peer-review process for librarians and information specialists, allowing for a second look at the work once a draft search strategy has been made.¹⁹ MEDLINE and PsycINFO search strategies use validated search filters to identify RCTs,³⁰ such as Modified Wolters Kluwer Expert Search Children Broad Medline (OVID) Search Filter³¹ and The Cochrane Highly Sensitive Search Strategy for identifying randomised trials in MEDLINE.³² The searches are reported here in accordance with PRISMA search³³ (online supplemental materials 2, 3). Deduplication will be completed using the systematic review software Covidence.³⁴

Source selection

Initial screening

The titles and abstracts of all studies located will first be screened to assess eligibility. Two trained reviewers (MCP and MP) will screen the studies independently and in duplicate. Any discrepancies identified will be reviewed by a third reviewer (RD), so that study eligibility criteria can be clarified as needed and decisions on studies that are best included or excluded can be made at this stage. The two reviewers will complete training sets of 100 randomly chosen articles until an adequate inter-rater reliability is reached (ie, ≥80% agreement), prior to independent duplicate screening. Studies agreed on by both reviewers, as well as studies with unresolved discrepancy will move on to full-text screening.

Full-text screening

Three trained reviewers (MCP, MP and RD) will screen the full text of studies meeting eligibility independently, with all full-text articles reviewed in duplicate by random pairs of the three reviewers. Any discrepancies that arise will be resolved through discussion among the three reviewers and any outstanding discrepancies will be resolved through discussion with a child and adolescent psychiatrist (SM and NJB). Reviewer pairs will complete a training set with a randomly chosen sample of articles until an adequate inter-rater reliability is reached (eg, ≥80% agreement). Reasons for study exclusion will be recorded using Covidence, a web-based systematic review software.³³ When necessary, study authors will be contacted to clarify eligibility criteria. Included studies will move to the next stage in the process: data charting. The final list of included articles will be reviewed by a child and adolescent psychiatrist (SM) and any additional RCTs identified meeting study eligibility criteria will also be included.

Data extraction and charting

Studies that are included from full-text screening will go through data charting by two trained reviewers using a standardised charting form developed using Covidence.³⁴ Any disagreements will be first discussed between the two reviewers and any unresolved disagreements will be resolved with the third trained reviewer, as required. The full-text review and data charting forms will be piloted on a sample of 10 relevant documents before full-text review begins. A preliminary analysis will also be conducted to pilot the data summary process.

The following data will be extracted and charted: publication details (eg, title, first author, year of publication), study population (eg, participant age range, inclusion and exclusion criteria), study characteristics (eg, sample size of study, type of intervention, follow-up period, study location and setting), reported candidate outcomes, and OMIs used. Quantitative measures (eg, counts, frequencies) will be used to identify and quantify all outcomes and OMIs. For each outcome, the following data will be charted: definition of the outcome, definition of meaningful change, outcome type (eg, single vs composite), and OMIs used to measure outcome. In the context of paediatric AD, an example of an outcome would be ‘severity of anxiety symptoms’, and an example of an OMI would be the ‘Paediatric Anxiety Rating Scale’.³⁵ Whether outcomes are categorised as primary, secondary or not specified as either will also be charted. An outcome will be classified as primary when studies explicitly report at least one of the following: (1) a study outcome is explicitly referred to as a ‘primary outcome’; (2) outcome data were used to calculate sample size or (3) the study objective explicitly included examining an intervention effect on that outcome.^{15 36}

All outcome terms will then be grouped or ‘mapped’ to an outcome framework, the Dodd framework.³⁷ The Dodd framework³⁸ allows for categorisation of outcomes across five core areas, which are further subdivided into outcome domains including health status outcomes (eg, severity of anxiety symptoms), resource-use outcomes (eg, service use by primary caregiver, cost-effectiveness of study interventions), and delivery of care outcomes (eg, intervention satisfaction, treatment group attrition). Mapping of outcomes to the Dodd Framework will allow for an understanding of the core areas and outcome domains that outcomes selected and measured in the literature to date fall under. In addition, conceptually similar outcomes will be combined where deemed appropriate, and grouped under unique ‘outcome terms’—for example, outcomes reported in individual RCTs as ‘anxiety disorder symptoms’, ‘anxiety disorder severity’ and/or ‘level of anxiety disorder symptomology’ could be grouped under the same outcome term, ‘anxiety disorder symptom severity,’ reflecting their similarity in meaning. Outcome grouping and mapping will be done in consultation with youth and family representatives, child and adolescent psychiatrists, and methodological experts.³⁶

Synthesis of results

Data analysis will be comprised of quantitative measures (counts and frequencies) of both study and outcome characteristics, such as the total number of outcomes and total number of OMIs. Characteristics of the studies, their outcomes, and any variation in outcome definitions will be organised and shown in tables. The results of mapping outcome terms may be displayed using a modified outcome matrix model.^{36 39}

Patient and public involvement

Members of our research team have lived experience with mental health challenges, including youth anxiety, and will contribute to all stages of the scoping review, including data collection and analysis. In addition, our youth and caregiver partners will consult on outcome grouping and mapping of outcomes from this scoping review and provide their input on what outcomes from the literature they feel are important.

Implications

This scoping review will identify the extent of outcome heterogeneity in published paediatric AD RCTs between January 2010 and May 2021. The resulting comprehensive list of identified outcomes from the literature will help inform the development of a COS for paediatric AD. Additional candidate outcomes for the COS, such as those that are most important to youth and their caregivers, new outcomes being measured in upcoming or ongoing RCTs, or those that may not have been identified in this review due to search limitations, may be identified during later stages of the COS development process by engaging with relevant stakeholders. Methods outlining the development of this COS will be published separately. The results of this scoping review will provide a repertoire of the outcomes selected and measured in the literature to date for key stakeholders, for example, clinicians, researchers inclusive of trialists, methodologists etc. In addition, it will provide the basis for the development of a planned harmonised COS for paediatric AD, which will lead to improved research quality and efficiency, allowing for a deeper understanding of what treatments work best for which patient, ultimately leading to better child mental health outcomes.

Ethics and dissemination

Institutional ethics approval was not required as this review did not collect personal, sensitive or confidential information and did not involve any participants; only publicly accessible documents were used as evidence. The results of this scoping review will be published in a peer-reviewed journal. We will circulate the publication to the COMET Initiative as well as other relevant stakeholders and social media platforms.

Ethics statements

Patient consent for publication

Not applicable.