A systematic review with meta-analysis of comprehensive interventions for preschool children with autism spectrum disorder (ASD): study protocol

Abstract

Introduction

The aims of this study are to (1) conduct a systematic review of the intervention literature in preschool children with autism spectrum disorder (ASD), including types of interventions that are tested and the classification of outcome measures used and (2) to undertake a meta-analysis of the studies, allowing for the first time the comparison of different approaches to intervention using comparative outcomes. There are a number of alternative modalities of intervention for preschool children with ASD in use with different theoretical background and orientation, each of which tend to use different trial designs and outcome measures. There is at this time an urgent need for comprehensive systematic review and meta-analyses of intervention studies for preschool children with ASD, covering studies of adequate quality across different intervention types and measurement methods, with a view to identifying the best current evidence for preschool interventions in the disorder.

Methods and analysis

The authors will perform a systematic review of randomised controlled trials for preschool children with ASD aged 0–6 years, along with a meta-analysis of qualifying studies across intervention modality. The authors will classify the interventions for preschool children with ASD under three models: behaviour, multimodal developmental and communication focused. First, the authors will perform a systematic review. Then, the authors will conduct a meta-analysis by comparing the three models with various outcomes using an inverse variance method in a random effect model. The authors will synthesise each outcome of the studies for the three models using standardised mean differences.

Dissemination and ethics

This study will identify each intervention's strengths and weaknesses. This study may also suggest what kinds of elements future intervention programmes for children with ASD should have. The authors strongly believe those findings will be able to translated into the clinical practices and patients and their family benefits. Review registration: PROSPERO CRD42011001349.

Article summary

Article focus

• This is a protocol of a systematic review and meta-analysis of comprehensive interventions for preschool children with autism spectrum disorder (ASD).

• We will conduct a systematic review of the intervention literature in preschool ASD, including type of intervention that is tested and classification of outcome measures used.

• We will undertake a meta-analysis of the studies, allowing for the first time the comparison of different approaches to intervention using comparative outcomes.

Key messages

• We will classify the interventions for preschool ASD under three models: behaviour, multimodal developmental and communication focused.

• We will perform a systematic review and conduct a meta-analysis by comparing the three models with various outcomes.

Strengths and limitations of this study

• This study will identify each intervention's strengths and weaknesses.

• This study may also suggest what kinds of elements future intervention programmes for children with ASD should have.

• The measures used for outcome are varied between studies, and the standardised data will be heterogeneous. We do not assume that each study is estimating exactly the same quantity. However, those heterogeneous data can be synthesised in the analyses using random effects models.

Introduction

Recent epidemiological studies estimate a prevalence of 1:100 for autism spectrum disorder (ASD),¹ an increase over reported rates in the past.² There has been increasing interest in developing effective interventions for young children with ASD since the evidence suggests that early intervention programmes are indeed beneficial for children with ASD, often improving developmental functioning and decreasing maladaptive behaviours and symptom severity³ and also can improve outcomes in later years for many individuals.⁴

An increasing volume of published trials of psychosocial intervention programmes for preschool children with ASD have been seen in recent years. These programmes tend to fall into three models: (1) those based on behaviour change which use applied behavioural analysis (eg, Smith et al⁵); (2) those focused on therapies targeted at improving the social communication impairment, the core symptom of autism (eg, Green et al⁶) and (3) multimodal interventions targeted across areas of autistic children's development (eg, Dawson et al⁷). In addition, an increasing number of these studies have followed CONSORT guidelines,⁸ and some meta-analyses and systematic reviews about intervention programmes for preschool children with ASD have been published.^9–11 These meta-analyses and systematic reviews focused exclusively on one or the others of these groups of intervention styles; there has been no systematic review or meta-analysis of studies comparing results from different types of intervention approach from the viewpoint of the three models. For clinicians and commissioners, this poses a difficulty in making general choices in a field containing often strong and partisan claims of effect from different traditions of intervention. Related to this, there has been great variation in end point measures used in these reported studies, making comparison of effects between studies difficult. Specifically, there has been variation in whether end points have been framed in terms of specific autism symptom outcomes, non-autism-specific outcomes that are not specific to autism (such as for instance IQ) or ‘intermediate’ end points relating to aspects of development that may have some relationship to later autism symptoms—examples would be changes in joint attention or parent–child interaction. These latter two kinds of outcome are often reported, without necessarily strong justification, as if they were the equivalent of change in autism symptoms (ie, as ‘surrogate’ end points), and this can cause real confusion. We think that these considerations indicate the need for a more comprehensive review of intervention studies for preschool children with ASD, covering studies of adequate quality across different intervention types and measurement methods, with a view to identifying the best current evidence for preschool interventions in the disorder. In this study, we will investigate it by comparing three major types of interventions with various outcomes.

We will undertake a systematic review and a meta-analysis of randomised controlled trials (RCTs) for preschool children with ASD. Recently, many RCTs for children with ASD have emerged sufficient enough to perform meta-analyses. RCT methodology has been identified as the gold standard in efficacy research.¹² In addition, meta-analyses of RCTs are at the top of the evidence-based medicine hierarchy.¹³ Thus, the findings of this study will provide strong evidence about interventions for children with ASD. Howlin et al are asserting that there are three main strands of early interventions for children with ASD: programmes with a particular emphasis on the use of behavioural principle to improve learning and behaviour, those that have a specific focus on communication and those in which developmental/educational strategies have been employed.¹⁴ In this study, we named those strands as behavioural, communication-focused and multimodal developmental interventions, respectively. Understanding the mechanisms that underlie this attenuation of treatment effects and how these can be overcome is one current challenge.¹⁵ This study may reveal each type of the intervention's strong and weak points to various kinds of treatment factors, respectively. Its findings will guide us to develop new types of interventions to overcome the attenuation of treatment effects in the core symptoms of autism. It will contribute to the appropriate choices of the interventions for children with ASD for their families, clinicians and the policymakers.

The objectives of our study are to (1) conduct a systematic review of all the preschool intervention literature in ASD, including the type of intervention that is being tested and classification of outcome measures used and (2) to undertake a meta-analysis of methodologically adequate studies using the Cochrane tool, which will allow for the first time comparison of different approaches to intervention on comparative outcome measures.

Methods

Types of studies

We will include RCTs and subject these to a rating on the Cochrane Collaboration tool for assessing risk of bias.

Types of participants

Participants comprise preschool children aged 0–6 years with a diagnosis of ASD as below.

Diagnostic and Statistical Manual of Mental Disorders-IV-Text Revision (DSM-IV-TR)¹⁶

• Autistic disorder

• Asperger disorder

• Pervasive developmental disorder not otherwise specified (PDD-NOS)

International Classification of Diseases-10 (ICD-10)¹⁷

• Childhood autism

• Asperger syndrome, atypical autism

• Other pervasive developmental disorders

• Pervasive developmental disorders, unspecified.

Types of interventions

We classify interventions for preschool children with ASD in three groups: (1) behavioural interventions—based essentially on learning theory and on applied behaviour analysis; (2) communication-focused interventions, targeting social communication impairment, as the core symptom of autism and (3) multimodal developmental interventions targeting a range of aspects of children's development.

Types of outcome measures

A feature of this review is that we will systematically classify the various outcome measures used within recent intervention trials into the following categories:

Primary outcomes

Autism behavioural symptoms are qualitative impairment in social interaction; qualitative impairment in communication and restricted repetitive and stereotyped patterns of behaviour, interests and activities. These are the triad of diagnostic criteria for autism in DSM-IV-TR and the definitional symptoms of the disorder and key indicators of psychopathology (eg, the autism Diagnostic Observation Schedule-Generic¹⁸ will be used for these outcomes).

Secondary outcomes

Non-specific developmental outcomes. These are not directly related by definition to autism diagnosis but are used in some studies as substitute outcomes—examples are adaptive behaviour (eg, the Vineland Adaptive Behaviour Scale¹⁹ will be used for this outcome) and IQ and cognitive development (eg, the Wechsler Preschool and Primary Scale of Intelligence third edition²⁰ will be used for these outcomes).

Intermediate outcomes relevant to the known development of autism—which might be candidates for surrogate end points. These outcomes are often defined as the proximal targets of intervention approaches from a developmental perspective. Examples (along with appropriate measures) are as follows: measures of joint attention (the Early Social Communication Scales²¹), imitation ability (the Imitation Battery²²), symbolic play (the Communication and Symbolic Behaviour Scales Developmental Profile²³), parent–child interaction (the Dyadic Communication Measure for Autism²⁴), receptive language (the MacArthur–Bates Communicative Development Inventory²⁵) and expressive language (MacArthur–Bates Communicative Development Inventory²⁵).

Electronic searches

We will do a systematic review of the published work according to the PRISMA statement (Figure 1).²⁶ Relevant studies will be identified by searching the following data sources: PsycINFO (from 1956 to January 2011), Medline via Ovid (from 1950 to January 2011), ERIC (from 1950 to January 2011) and the Cochrane Database.

Figure 1.

PRISMA 2009 Flow diagram. Interventions based on behavioural, developmental or communication-focused models for ASD in pre-school children 0–6 years. From Moher et al.²⁶

We will use the following search terms to search all trials registers and databases: ‘autism’, ‘autism spectrum disorder’, ‘ASD’, ‘high function autism’, ‘high function ASD’, ‘Asperger syndrome’, ‘pervasive developmental disorder’, ‘PDD-NOS’, ‘intervention’, ‘treatment’, ‘therapy’, ‘communication’, ‘interpersonal’, ‘speech’, ‘interaction’, ‘synchrony’, ‘relationship’, ‘language’, ‘social’, ‘development’, ‘behaviour’, ‘intensive behavioural intervention’, ‘trial’, and ‘outcome’. Their search will be limited by age group from 0 to 6 years old and ‘RCT’. This search strategy has been peer reviewed by a librarian of University of Manchester.

Validity assessment

Two of the authors, YT and YH, will independently review the abstracts of the potentially relevant studies. This will be followed by a consensus discussion with JG. The quality of the RCTs will be coded independently by YT and YH and disagreements will be resolved by consensus discussions.

Searching other resources

Reference lists from identified trials and review articles will be manually scanned to identify any other relevant studies. The clinicaltrials.gov and the Cochrane Library website will be also searched for randomised trials that were registered as completed but not yet published.

Data collection and analysis

Selection of studies

Inclusion

1. Participants comprise preschool children with a diagnosis of ASD or pervasive developmental disorder.

2. RCTs.

3. Interventions delivered to the parents/guardians and/or directly to the child, by special educators, teachers, speech pathologists, psychologists or other allied health professional students will be included.

4. Studies carried out while the children were at a preschool aged between 0 and 6 years.

Exclusion

1. The study was not primary research on preschool children with ASD.

2. The study did not assess a cognitive/behavioural intervention for preschool children with ASD.

3. The study design was not a RCT.

4. Alternative or complementary medicine was used as the main intervention of the study.

5. The intervention was a pharmacological one.

6. The intervention was not classified into behavioural, multimodal developmental or communication-focused model.

7. The control group received a specific early intervention programme for children with autism which was not a usual treatment provided by their local services.

8. The study was judged to be in high risk of bias by the Cochrane Collaboration tool for assessing risk of bias.

All citations sourced from the search strategy will be transferred to EndNote, a reference management database software. Initial screening of titles and abstracts by an experienced research fellow (YT) will eliminate all those citations obviously irrelevant to the topic, for example, prevalence studies, studies not relating to ASDs and single-case studies. Thereafter, two review authors (YT and YH) will assess and select studies for inclusion from the group of superficially relevant studies. In the event of a disagreement, resolution will be reached in discussion with the third author (JG), if necessary following inspection of the full paper.

Data extraction and management

YT and YH will independently extract data from selected trials using a specially designed data extraction form. Extracted data will consist of methods (dose and frequency of intervention), diagnostic description of participants and type of intervention, including target, intensity, duration and method of application (parent mediated, therapist, school based, etc). Data will be extracted independently by two review authors (YT and YH) and disagreements will be resolved by negotiation with a third author (JG).

Assessment of risk of bias in the studies

Risk of bias will be assessed by two independent review authors (YT and YH) and disagreements will be resolved by negotiation with a third review author (JG). We will use the Cochrane Collaboration tool for assessing risk of bias in these areas.²⁷ The assessed risk of bias in studies will include in the following domains: sequence generation, allocation concealment, blinding, incomplete outcome data, selective outcome reporting and other sources of bias. The process will involve recording the appropriate information for each study (eg, describing the method used to conceal allocation in detail) and evaluating whether there is risk of bias in that area (eg, was allocation adequately concealed). We will allocate studies to the three categories according to our judgement of each area or potential risk of bias: (1) low risk of bias, (2) moderate (or unclear) risk of bias and (3) high risk of bias. Whether the studies should be included for the analyses or not will be judged individually based on the results of the risk of bias assessments.

Measures of treatment effect

Continuous data

Continuous data will be analysed on the basis that the means and SDs are available and that there is no clear evidence of skew in the distribution.

Dealing with missing data

Missing data will be assessed for each individual study. Where a loss of significant quantities of participant data is reported such that the review authors agree that the conclusions of the study are compromised, trial authors will be contacted. If no reply is forthcoming or full data are not made available, these studies will not be included in the final analysis. For included studies reporting dropout, we will report the number of participants included in the final analysis as a proportion of those participants who began the intervention. Reasons for missing data will be reported. The extent to which the results of the review could be altered by the missing data will be assessed and discussed. If summary data are missing, trial authors will be contacted. If no reply is forthcoming or the required summaries are not made available, the authors will include the study in the review and assess and discuss the extent to which its absence from meta-analysis affects the review results.

Assessment of heterogeneity

Consistency of results will be assessed visually and by χ² tests.²⁸ In addition, since χ² can have low power when only few studies or studies of a small sample size are available,²⁹ we will use the I² statistic to calculate the degree to which heterogeneity is having an impact on the analysis.²⁷

Assessment of reporting biases

If sufficient studies are found, funnel plots will be drawn to investigate any relationship between effect size and sample size. Such a relationship could be due to publication or related biases or due to systematic differences between small and large studies. If a relationship is identified, clinical diversity of the studies will be further examined as a possible explanation. Every attempt will be made to obtain unpublished data and data from conference proceedings.

Data synthesis

Data synthesis will be performed using Review Manager V.5.1 (Cochrane Collaboration software). We will assess continuous and binary data. Assuming that two or more studies that are suitable for inclusion are found and that the studies are considered to be homogeneous, a meta-analysis will be performed on the results. The categories of outcome measure mentioned above differ conceptually in important ways and have been used in a systematic different way across trials of the different intervention types identified above. Our review aims to make comparison across these different types of intervention study, thus we will standardise and synthesise the various categories of outcome measure using an inverse variance method in a random effect model.²⁷ We will compare the types of intervention model effectiveness for each outcome using a standardised mean difference.

Subgroup analysis and investigation of heterogeneity

We will undertake subgroup analyses and meta-regression. These will be pooled to calculate a final effect size. While these analyses may enable us to hypothesise as to possible causes of differences between studies' findings, some heterogeneity is likely to remain, and any statistical analysis will be accompanied by a narrative synthesis.

Subgroup analysis will be undertaken if clinically different interventions are identified or there are clinically relevant differences between participant groups. Anticipated clinically relevant differences are as follows:

1. intervention delivery type (eg, therapist, parent mediated, school based) and length

2. intervention target skill (eg, Theory of Mind as a whole, joint attention, emotion recognition, false-belief understanding)

3. participant age (eg, preschool, young children), IQ (low vs normal or high), specific diagnosis and verbal ability.

Relevant subgroup analyses will also include:

• Severity of autism at baseline

• Social economic status and other demographic variables

• Age of child

• Type of intervention (our three groups as above)

• Parent-mediated (directing parents to train their children, not training the children directly) versus child-mediated (training the children directly) intervention delivery

• Cognitive ability at baseline

Sensitivity analysis

Sensitivity analysis will be conducted to assess the impact of study quality on the results of the meta-analyses. For example, we will test to see if studies with high rates of loss to follow-up or inadequate blinding are more likely to show positive outcomes and also to assess the impact of imputing missing data.

Discussion

Meta-analysis of RCTs across types of intervention for preschool children with ASD is an important step in providing a reliable basis for implementation decisions. Since previous analyses have been essentially restricted to specific intervention types and often with different outcome criteria, a study across three representative models: behavioural, multimodal developmental or communication-focused models will guide future clinical practice and research trials for children with ASD. This study will provide information about which kind of intervention has strong points and weak points and what are those strong points and weak points are. This study may also suggest what kinds of elements future intervention programmes for children with ASD should have. We strongly believe those findings will be able to translated into the clinical practices and patients and their family benefits.

Anticipated challenges in synthesise the literature exist. The measures used for outcome are varied between studies, and the standardised data will be heterogeneous. We do not assume that each study is estimating exactly the same quantity. Thus, we will use random effect models for the analyses.²⁷ In addition, the durations of the interventions will be different among the studies included in this study. We will synthesise the data regardless of the durations of the interventions and will discuss the diversity of the durations in our paper.