Pivotal Response Treatment for autism spectrum disorder (ASD)

Notes

Editorial note

This protocol will not be progressed to the review stage due to a lack of progress.

Objectives

This is a protocol for a Cochrane Review (intervention). The objectives are as follows:

To systematically review evidence on the effectiveness of PRT in individuals with ASD. Our specific objectives are:

1. to assess the effect of PRT compared with treatment as usual or wait‐list control on social communication skills in individuals with ASD based on: 1. direct observation; and 2. parent/caregiver or teacher report (or both);

2. to assess the effect of PRT compared to treatment as usual or wait‐list control on: 1. a. intelligence; b. restricted and repetitive behaviour; c. internalising and externalising behaviour; d. global clinical improvement; e. the dose‐response of PRT; and f. possible reported adverse effects; and 2. parenting stress;

3. to examine differential effects of age, gender, IQ measure before treatment, and severity of ASD symptoms on treatment outcomes; and

4. to examine differential effects of fidelity of PRT implementation, duration of intervention, intensity of intervention, implementation setting and therapist involvement on treatment outcomes.

Background

Description of the condition

Autism spectrum disorder (ASD) is characterised by persistent deficits in social communication and interaction, and by restricted, repetitive patterns of behaviour, interests, or activities (APA 2013). As a neurodevelopmental disorder, ASD emerges at a young age (Ibanez 2014), and the diagnosis is stable throughout life in most individuals (Woolfenden 2012). Both heritability and environmental factors contribute to the aetiology of ASD (Sandin 2014; Sealy 2016). ASD affects boys more often than girls, with a ratio of 5:1 among individuals with average intellectual functioning and a ratio of 2:1 among those with an intellectual disability (Fombonne 2005). A meta‐analysis that focused on ASD symptoms in boys versus girls found that girls showed fewer restricted interests and behaviours and stereotypes than did boys, but only in children older than six years, which suggests that the ASD phenotype differs by age and gender (Van Wijngaarden‐Cremers 2014).

ASD is typically diagnosed using criteria from the Diagnostic and Statistical Manual of Mental Disorders, currently in its fifth edition (DSM‐5; APA 2013). In the previous version of the DSM (DSM‐Fourth Edition‐Text Revision; APA 2000), ASD was separated into four distinct disorders: autistic disorder, Asperger's disorder, childhood disintegrative disorder and pervasive developmental disorder not otherwise specified (PDD‐NOS). Although these subtypes still exist in the current version of the International Statistical Classification of Diseases and Related Health Problems (ICD‐10) (WHO 1992), they have been subsumed into a single diagnosis of ASD in the DSM‐5 (APA 2013), since no valid differences as regards clinical and demographic characteristics, neuropsychological profiles, comorbidity and prognosis have been found (Volkmar 2014; Witwer 2008). In addition, three severity classifications have been included in the DSM‐5, to indicate the level of support that an individual needs in the areas of both social communication and restricted and repetitive behaviors (Weitlauf 2014).

The prevalence of ASD in the general population is estimated to be about 1% (Baird 2006; Elsabbagh 2012). There is heterogeneity in the severity of ASD symptoms, developmental trajectories, and cognitive, language and adaptive functioning (Georgiades 2013). Although ASD can affect individuals with any level of intelligence, a significant proportion of affected individuals have an intellectual disability (varying from around 40% to 70%) (CDC 2012; Fombonne 2006; Yeargin‐Allsopp 2003). Some studies have found the ASD symptom profile to be different in individuals with a lower (non‐verbal) intelligence quotient (IQ) (Bishop 2006; Kjellmer 2012), but other studies have not found evidence in support of this (Charman 2011). Most individuals with ASD (70%) have a comorbid psychiatric disorder, such as anxiety or attention‐deficit hyperactivity disorder (ADHD), with 41% having two or more comorbid conditions (Simonoff 2008).

Although the long‐term outcomes for most children with ASD are poor (Billstedt 2005; Van Heijst 2015), behavioural and social outcomes can vary highly between individuals with ASD. Some individuals improve markedly in social and behavioural functioning, while others experience deterioration in functioning or show a stable course (Levy 2011). Individuals with ASD who have a higher IQ and fewer comorbid conditions are more likely to experience better outcomes in adolescence or early adulthood (Levy 2011). A minority of people with ASD are able to find suitable employment, to develop meaningful relationships and to live independently (Howlin 2004), but the majority rely on family, caregivers and professional services throughout life. Estimated costs due to productivity loss for an individual with ASD can range from about USD 27,000 to USD 32,000 per year in the UK and can be about USD 11,000 in the USA (Buescher 2014). Additionally, costs due to productivity loss for parents of children with ASD can range from about USD 800 to USD 7000 per year in the UK and can be about USD 24,000 per year in the USA (Buescher 2014). Total societal costs for caring for and treating a person with ASD in the USA are estimated to be USD 3.2 million per year (Ganz 2007).

Treatments

Pharmacological treatments target symptoms that may co‐occur with ASD, such as symptoms of ADHD, aggression, irritability, self‐injurious behaviour and anxiety (Myers 2007), because there are currently no treatments for the core symptoms of ASD (Ghosh 2013). Interventions designed to address the core symptoms of ASD are non‐pharmacological interventions, such as communication‐focused interventions, social skill development interventions, sensory motor interventions and milieu therapy (Ospina 2008). These interventions vary considerably in their underlying theoretical frameworks, intensity, modes of delivery and cost‐effectiveness (Seida 2009). Most of the 30 treatment models available for ASD have clearly defined procedures and a detailed curriculum, or both (Odom 2010). The evidence for most of these interventions is weak (Odom 2010), but promising treatment methods are characterised by a focus on the core symptoms of ASD, a manualised approach, systematic measurement of effect (after treatment and follow‐up), monitoring of fidelity of implementation, and being embedded in daily activities in the individual's natural environment (Ospina 2008).

Interventions for ASD based on the principles of applied behaviour analysis (ABA) cover a number of these elements (Kodak 2011; Odom 2010), and a meta‐analysis of the effectiveness of ABA for young children with ASD has reported positive medium‐to‐large effects on intellectual functioning, language development and adaptive behaviour, regardless of the specific ABA method used (Virués‐Ortega 2010). Traditional ABA‐based interventions involve discrete trial training in a structured one‐to‐one situation (Ghezzi 2007). This is highly labour intensive (Smith 2001), and so attention has been paid to how to limit the intensity of ABA‐based treatments for ASD and to implement ABA procedures in the individual's natural environment (Foran 2015).

Description of the intervention

Pivotal Response Treatment (PRT) is an ABA‐based intervention that focuses on the generalisation of learned skills in the child's natural environment (Koegel 2016), and is used for the core symptoms of ASD (Koegel 1999; Koegel 2001; Koegel 2006). It was developed on the basis of the natural language paradigm (NLP) and represents a shift towards naturalistic procedures for language interventions in contrast to imitation and drill procedures outside the natural environment of the individual (Camarata 1996). PRT can be distinguished from other traditional ABA‐based treatments by focusing on 'pivotal' skills in contrast to one individual target skill, using child‐selected material versus adult‐selected material, being embedded in the child's natural environment versus therapy in a structured, intensive one‐to‐one setting, and using different people (e.g. parents, teachers and peers) to provide the intervention in contrast to therapist(s) only (Koegel 1999). PRT focuses on four pivotal (or 'core') areas, namely, motivation for social communication, self‐initiation, responding to multiple cues, and self‐management (Koegel 2001; National Autism Center 2009; National Autism Center 2015). Empathy has been proposed recently as a fifth pivotal area although scientific evidence to support this is limited (Koegel 2013; Koegel 2016). The hypothesis is that the acquisition of skills in these pivotal areas may promote widespread, collateral improvements in other skills.

In PRT, 'learning opportunities' are created according to the principles of ABC (i.e. antecedents, behaviour, consequences), with emphasis on techniques related to the antecedents and consequences of behaviour. For the 'antecedent' domain, three techniques are used to elicit target behaviour, namely: 1. following the child's interests (e.g. by child choice); 2. catching the child's attention (e.g. dividing material); and 3. providing a clear learning opportunity with appropriate help (e.g. the use of 'prompts'). The techniques used for the 'consequences' domain include: 1. consistent reinforcement; 2. use of natural reinforcements (i.e. that is directly and functionally related to a task); and 3. rewarding attempts (Koegel 1999; Koegel 2001; Koegel 2006; Koegel 2014a). As an additional technique, interspersing maintenance (i.e. previously learned) tasks with new tasks provides an appropriate and reliable sequence of learning opportunities. In Appendix 1, we provide two examples of PRT learning opportunities.

Specific goals for PRT vary depending on the developmental level and verbal abilities of the individual, ranging from teaching communicative intent and first words in very young children, to self‐management of social behaviour in adolescents with ASD (Koegel 1999; Koegel 2006). The education of parents, siblings, peers, and teachers is a crucial component of PRT because it improves learning opportunities in the individual’s natural environment (Koegel 2006; Koegel 2014a). Furthermore, training parents will improve family interactions and diminish parenting stress (Koegel 2012). For these reasons, PRT can be considered a comprehensive intervention that is implemented across people, settings and environments, with a view to achieving optimal outcomes in individuals with ASD.

PRT can be implemented at home, in a clinic, in classrooms or in play gardens, and by various implementers (Bozkus Genc 2013). Implementation in the natural environment by parents and other caregivers is a critical component of the intervention (Koegel 2006; Renshaw 2011). Clinicians that are familiar with basic theoretical as well as practical knowledge of ABA are trained in PRT techniques to become a certified PRT therapist. During intervention sessions, PRT therapists adapt the techniques to caregivers, give them opportunities to practice implementing the techniques, provide feedback and, at the same time, monitor the fidelity of treatment implementation (Koegel 2006; Koegel 2014a). Monitoring of the fidelity of treatment implementation, that is, the correct use of PRT techniques by clinicians, caregivers and teachers, is an important aspect in optimising the quality of PRT (Bryson 2007; Koegel 2012). This is often assessed over a 10‐minute period, with the criterion that PRT motivational techniques should be used for 80% of the time (i.e. child attending, clear opportunity to respond, interspersing maintenance and new tasks, using multiple cues, using child choice, contingent reinforcement, natural reinforcement and reinforcing attempts) (Koegel 2006).

How the intervention might work

PRT targets 'pivotal' or core areas of functioning that are impaired in individuals with ASD. For instance, young children with ASD take the initiative less often in social situations because they are not motivated to do so (Koegel 2012). This may be because they have limited skills for focusing on others, such as joint attention and emotion recognition, which might be due to a deficit in the development of theory of mind (Baron‐Cohen 2000; Fletcher‐Watson 2014). Because they are less motivated, young children with ASD have fewer opportunities for using language, making eye contact, taking turns, developing conversational skills, recognising emotions and learning social rules, etc. (Koegel 2012). They also have fewer opportunities for interacting with others, for developing language, for increasing their vocabulary and for learning about the outside world (Koegel 2014b). Increasing motivation to learn in children by using desired objects may benefit the acquisition of academic skills such as reading and calculating (Koegel 2006). Also, the lack of self‐management abilities in older children with ASD may result in their using requests and protests as almost the sole means of social interaction (Wetherby 1984). Thus, targeting pivotal areas impaired in individuals with ASD may lead to collateral improvement in other social and communicational skills.

PRT enables an individual with ASD to learn that social communication can be functional and rewarding (Koegel 2014a; Koegel 2014b). Prompts (i.e. levels of help) are used to help the individual make an appropriate response (Koegel 2012). These are often gradually phased out, going from a high level of help (e.g. tell prompt) to a low level of help (e.g. waiting prompt), as the new skill is acquired (Koegel 2012).

Considering the promising ingredients of PRT, the intervention has the potential to facilitate improvements in different areas of functioning of the individual with ASD and his/her relatives. However, qualitative and quantitative comparisons of the effects of PRT treatment with the effects of other treatments for individuals with ASD are needed to draw valid conclusions.

Why it is important to do this review

Lang 2009 reviewed studies of parents’ ability to implement communication interventions and the effects of PRT on children’s communication, but did not specifically compare PRT with other communication interventions for ASD. Bozkus Genc 2013 evaluated methods of PRT implementation, investigating participant characteristics, settings, which social skills were taught, characteristics of implementers, research model, and social validity; however, they only included information from single‐subject research. Bozkus‐Genc 2016 quantitatively evaluated these single‐subject studies but did not report on the quality of the studies, especially the risk of bias. One previous Cochrane Review focused on a parent‐mediated intervention for ASD but investigated young children only and did not specifically study the effects of PRT (Oono 2013).

A systematic review and meta‐analysis of the efficacy of PRT in children with ASD that adheres to Cochrane guidelines is needed for several reasons. First, the use of different methods in PRT studies makes it impossible to compare the effects of PRT with the effects of other ASD interventions. However, the use of the GRADE system to evaluate intervention studies in ASD can improve transparency and comparability across studies (Guyatt 2008). Second, it is important to present the results of randomised trials of PRT separately from those of non‐randomised studies, as the latter potentially have a higher risk of bias (Higgins 2017). In earlier reviews, evidence from randomised controlled trials (RCTs) was not evaluated (Cadogan 2015), or effects were only reviewed descriptively (Verschuur 2014). Third, no quantitative systematic review has compared studies using group designs to evaluate the effect of PRT and usual care on outcomes in individuals with ASD and their relatives. In addition to addressing these gaps in the evidence base, it is important to explore the role of individual characteristics and environmental factors as potential moderators of the impact of PRT, and the impact of 'dosage' factors such as frequency, intensity and implementation setting.

Objectives

To systematically review evidence on the effectiveness of PRT in individuals with ASD. Our specific objectives are:

1. to assess the effect of PRT compared with treatment as usual or wait‐list control on social communication skills in individuals with ASD based on: 1. direct observation; and 2. parent/caregiver or teacher report (or both);

2. to assess the effect of PRT compared to treatment as usual or wait‐list control on: 1. a. intelligence; b. restricted and repetitive behaviour; c. internalising and externalising behaviour; d. global clinical improvement; e. the dose‐response of PRT; and f. possible reported adverse effects; and 2. parenting stress;

3. to examine differential effects of age, gender, IQ measure before treatment, and severity of ASD symptoms on treatment outcomes; and

4. to examine differential effects of fidelity of PRT implementation, duration of intervention, intensity of intervention, implementation setting and therapist involvement on treatment outcomes.

Methods

Criteria for considering studies for this review

Types of studies

Randomised controlled trials (RCTs), quasi‐RCTs (trials using a quasi‐random method of allocation; for example, based on date of admission), non‐randomised studies (non‐RCTs) and controlled before‐and‐after studies (CBA) in which a PRT treatment group is compared with a control group.

Types of participants

Any person with a primary clinical diagnosis of ASD based on any version of the DSM (APA 1987; APA 1994; APA 2000; APA 2013) or the ICD‐10 (WHO 1992), with or without comorbid medical or psychiatric disorders, defined according to diagnostic terms relevant to ASD, such as autistic disorder, Asperger's disorder, Asperger's syndrome, PDD‐NOS, childhood autism, infantile autism, and atypical autism.

Types of interventions

Studies that explicitly state the use of PRT techniques in the treatment group compared with treatment as usual or wait‐list control (waiting for treatment).

Types of outcome measures

We will not exclude studies on the basis of the outcomes measured. The primary and secondary outcomes listed below are accompanied by examples of measures that are often used to evaluate treatment effects in individuals with ASD.

Primary outcomes

1. Social communication skills based on direct observations, including observational measures such as the Social and Communication domains of the Autism Diagnostic Observation Scale (ADOS; Lord 2000), or observed expressive communication such as the Production Scale of the Reynell Developmental Language Scale (RDLS; Reynell 1985), the observed mean length of utterance (MLU), and observed functional verbalisations

2. Social communication skills as measured by parent/caregiver or teacher report, or both, using scales such as the Social Responsiveness Scale (SRS; Constantino 2005), the Social Communication Questionnaire (SCQ; Rutter 2003), the Children's Communication Checklist (CCC; Bishop 1998) or the Socialization and Communication domains of the Vineland Adaptive Behaviour Scales (VABS; Sparrow 2005)

Secondary outcomes

1. Intelligence (IQ), as measured with IQ scales such as the Wechsler Intelligence Scale for Children (WISC; Wechsler 1991), or the Mullen Scales of Early Learning (MSEL; Mullen 1995)

2. Restricted and repetitive behaviour, as measured with scales such as the Restricted and Repetitive domain of the ADOS (Lord 2000) or the Repetitive Behaviour Scale (RBS; Bodfish 1999)

3. Internalising and externalising behaviour, as measured with scales such as the Child Behaviour Checklist (CBCL; Achenbach 1991) or the Strengths and Difficulties Questionnaire (SDQ; Goodman 1997)

4. Global improvement, as measured with scales such as the Clinical Global Impression ‐ Improvement (CGI‐I) scale (Guy 1976)

5. Possible reported adverse effects, including reports of exhaustion

6. Parenting stress, as measured with parent‐report scales such as the Parenting Stress Index (Abidin 1995), or the Parental Stress Scale (Berry 1995)

We will analyse patient‐, parent‐, and teacher‐reported information and data separately.

Search methods for identification of studies

We will not exclude studies on the basis of language, publication status or publication date.

Electronic searches

We will search the following electronic databases and trials registers.

1. Cochrane Central Register of Controlled Trials (CENTRAL, current issue) in the Cochrane Library, which includes the Cochrane Developmental, Psychosocial and Learning Problems Specialised Register.

2. MEDLINE Ovid (1946 onwards).

3. MEDLINE In‐Process and Other Non‐Indexed Citations Ovid (current issue).

4. MEDLINE Epub Ahead of Print Ovid (current issue).

5. Embase Ovid (1974 onwards).

6. CINAHL EBSCOhost (Cumulative Index to Nursing and Allied Health Literature; 1937 onwards).

7. PsycINFO Ovid (1806 onwards).

8. ERIC Ovid (Education Resources Information Center; 1965 onwards).

9. Linguistics and Language Behaviour Abstracts ProQuest (1973 onwards).

10. Science Citation Index ‐ EXPANDED Web of Science (SCI‐EXPANDED; 1945 onwards).

11. Social Sciences Citation Index Web of Science (SSCI; 1956 onwards).

12. Arts & Humanities Citation Index Web of Science (A&HCI; 1975 onwards).

13. Emerging Sources Citation Index Web of Science (ESCI; 2015 onwards).

14. Sociological Abstracts ProQuest (1952 onwards).

15. Epistemonikos (www.epistemonikos.org/nl/advanced_search).

16. National Autistic Society Library Catalogue (library.autism.org.uk/Portal/Default/en-GB/Search/AdvancedSearch).

17. Networked Digital Libary of Theses and Dissertations (NDLTD; search.ndltd.org; 1990 onwards).

18. ClinicalTrials.gov (www.clinicaltrials.gov).

19. WorldCat OCLC (www.worldcat.org/default.jsp).

20. World Health Organization International Clinical Trials Registry Platform (ICTRP; apps.who.int/trialsearch).

The search strategy for Ovid MEDLINE is reported in Appendix 2. We will adapt this search strategy for searches in other databases as appropriate.

Searching other resources

We will handsearch full‐text reports published in the scientific journals listed below, since PRT or NLP may be investigated in a study but not be mentioned in the title, abstract, or keywords.

1. Autism (July 1997 onwards).

2. Autism Research (2008 onwards).

3. Developmental Disabilities Research Reviews (1995 onwards).

4. Focus on Autism and Other Developmental Disabilities (June 1986 onwards).

5. Good Autism Practice (May 2007 onwards).

6. Journal of Applied Research in Intellectual Disabilities (JARID; 1988 onwards).

7. Journal of Autism and Developmental Disorders (1971 onwards).

8. Journal of Applied Behavior Analysis (1968 onwards).

9. Conference abstracts of the Association for Behavior Analysis International (ABAI), including annual conventions, international conferences, autism conferences, other conferences (2001 onwards).

10. European Journal of Behavior Analysis (2000 onwards).

11. Conference abstracts of the European Association for Behavior Analysis (EABA; 2003 onwards).

For identifying other, possibly relevant studies, we will handsearch the following two websites.

1. Research Autism (researchautism.net/interventions/88/pivotal-response-training-and-autism/Studies%20and%20Trials).

2. Koegel Autism Centre (education.ucsb.edu/autism/pivotal-response-treatment).

In addition, we will ask experts in the field if they know of other studies not identified by the search. We will also search the reference lists of included studies, as well as the reference lists of earlier systematic reviews.

Data collection and analysis

Selection of studies

Two review authors (IS and IO) will independently assess the titles and abstracts yielded by the searches for relevance against the inclusion criteria (see Criteria for considering studies for this review). They will then retrieve and examine the full texts of potentially relevant reports, grouping together multiple reports related to the same study. If additional information on methodology is needed to determine whether the study should be included, the review authors will contact the corresponding authors of the original article. We will resolve any possible disagreements about eligibility by discussion and, if necessary, by consulting another review author (MVD‐B). The final decision about study inclusion will be made by all review authors.

Studies will be blinded before assessment of their eligibility to lessen the possibility of selection bias. We will assess studies for inclusion without information regarding authors, affiliations, year and journal of publication; we will only export this information from Endnote 2014. Before examining the full text of potentially relevant reports, an independent research assistant (Manon de Korte, MSc.), who will not be involved in study selection, will remove information regarding authors, affiliations, year and journal of publication.

Data extraction and management

Using a pre‐designed form, two review authors (IS and IO) will independently extract study data (see Appendix 3 for a description of the data to be extracted) and compare both sets of extracted data for accuracy, resolving any discrepancies by discussion. One review author (IS) will enter the extracted data into Review Manager 5 (RevMan 5) (RevMan 2014), and a second review author (IO) will check it.

Assessment of risk of bias in included studies

Two review authors (IS and IO) will independently assess the risk of bias of the included studies, using Cochrane's tool for assessing risk of bias for randomised trials (Higgins 2017). Specifically, for each included study, the review authors will assess the following sources of bias and assign judgements of low, high, or unclear risk of bias: random sequence generation; allocation concealment; blinding of participants and personnel; blinding of outcome assessment; incomplete outcome data; selective reporting; and other possible sources of bias (Higgins 2017). See Appendix 4 for a description of each possible source. For cluster‐randomised designs, we will assess the risk of bias in the following categories: 1. recruitment bias; 2. baseline imbalance; 3. loss of clusters; 4. incorrect analysis; and 5. comparability with individually randomised trials (Higgins 2011).

For assessing the risk of bias in non‐RCTs and CBAs we will use the 'Risk of Bias in Non‐Randomised Studies ‐ of Interventions' (ROBINS‐I) assessment tool (Sterne 2016). Potentially important confounding domains for these studies are differences between groups in severity of ASD symptoms, (verbal) IQ, presence of comorbid psychiatric disorders, and presence of one or more pharmacological or non‐pharmacological co‐intervention(s) besides PRT, treatment as usual or wait‐list control. Bias domains included in ROBINS‐I are: 1. bias due to confounding; 2. bias in selection of participants into the study; 3. bias in classification of interventions; 4. bias due to deviations from intended interventions; 5. bias due to missing data; 6. bias in measurement of outcomes; and 7. bias in selection of the reported result. Two review authors will assign a judgement of 'low risk of bias', 'moderate risk of bias', 'serious risk of bias', 'critical risk of bias', or 'no information' to each of the seven domains. See Appendix 5 for a detailed description of risk of bias judgements for each domain.

The two review authors will resolve disagreements by discussion and, if necessary, by consulting another review author (MVD‐B). We will present the 'Risk of bias' assessment and its justification in a table.

Measures of treatment effect

Dichotomous data

When dichotomous outcomes are reported (e.g. clinically significant response to a treatment versus no clinically significant response), we will use the risk ratio (RR) with a 95% confidence interval (CI). If study outcomes are not reported within a 2 x 2 table, we will contact the investigators for this information. To calculate the RR, we will use the algorithm described in the Cochrane Handbook for Systematic Reviews of Interventions (Deeks 2017). We have chosen the RR rather than the odds ratio (OR) because the OR can be misleading when event rates (i.e. rates of clinically significant improvement) are high, which is common in clinical trials and systematic reviews (Deeks 1998).

Continuous data

For those studies that report continuous outcomes assessed with the same instruments, we will use the mean difference (MD) as the summary statistic in the meta‐analyses and present these with a 95% CI. If continuous outcomes have been assessed with different instruments across studies, we will use the standardised mean difference (SMD) with a 95% CI for between‐subject designs, using the algorithm described in the Cochrane Handbook for Systematic Reviews of Interventions (Deeks 2017).

Since the SMD method does not correct for differences in the direction of the scale, we will multiply mean values by –1 if outcomes are presented such that a higher score means less severe problem severity. If there is insufficient information to enable calculation of the SMD, we will describe the study in the systematic review but will not include it in the meta‐analysis.

Unit of analysis issues

Cluster‐randomised designs

Studies with a cluster‐randomised design should report an intracluster correlation coefficient (ICC). If the ICC is not reported, we will obtain external estimates from similar studies (Higgins 2011), and then correct the analysis by reducing the size of each trial to its 'effective sample size'. We will enter the data into RevMan 5 (RevMan 2014), and use the generic inverse‐variance method for analysis. When studies report outcomes from individuals of the same family (e.g. in multilevel models), we will include effect estimates from analyses that properly account for the cluster design, if this information is available. However, since we expect that this information will not be available for each study, we can only account for the impact of including siblings on the effect estimates for studies that report this information properly.

Cross‐over designs

We will screen studies with a cross‐over design for information about the evaluation of the carry‐over or period effects. If neither carry‐over nor period effects are thought to be a problem, we will calculate the effect estimate of the difference measure in the meta‐analysis, using the generic inverse‐variance method in RevMan 5 (RevMan 2014). However, if carry‐over or period effects are present or insufficient information about these effects is available, we will base the analysis on the data from the first period only.

Studies with multiple intervention groups

If a study describes multiple treatment groups, we will combine all relevant treatment groups (i.e. using PRT techniques) into a single treatment group and combine all relevant control groups into a single control group, as recommended by Higgins 2011. The effect estimates that are used for both dichotomous and continuous data from between‐subject designs are described in earlier sections (Measures of treatment effect).

Controlled before‐and‐after studies (CBA)

We will quantitatively combine the data of CBA studies provided that there is little bias in the studies and that the studies are relatively homogeneous (Reeves 2011). We will perform a meta‐analysis of adjusted effect estimates as an inverse‐variance weighted average, using the generic inverse‐variance outcome type in RevMan 5 (RevMan 2014). For studies that report multiple estimates of effect, we will use the estimate from the model that adjusted for most covariates.

Dealing with missing data

If study details or numerical data are missing, we will contact the corresponding author to ask them to supply the missing information. If this fails and we do not obtain sufficient summary data, we will use only the data reported in the original article. We will include the study in the review and report that study data are missing (Higgins 2011). We will present these data in a 'Risk of bias' table. We will only analyse available data because using imputation to deal with missing data artificially inflates the precision of the effect estimate (Higgins 2011).

Assessment of heterogeneity

We will assess clinical heterogeneity by comparing participant, intervention and outcome characteristics across studies, and methodological heterogeneity by comparing differences in methods across studies, such as allocation procedures, blinding and measurement of outcomes. To assess statistical heterogeneity, we will perform the Chi² test using RevMan 5 (RevMan 2014). A low P value (P < 0.10) provides evidence of heterogeneity of intervention effects. Additionally, we will estimate the amount of variation (tau²), and we will calculate the I² statistic (Higgins 2003) to determine the percentage of variability in the effect estimates due to heterogeneity rather than sampling error: an I² of 0% to 40% indicates that heterogeneity is not present or is not important, 30% to 60% that there is moderate heterogeneity, 50% to 90% that there is substantial heterogeneity, and 75% to 100% that there is considerable heterogeneity (Deeks 2017). If heterogeneity is present, we will investigate potential causes of heterogeneity by means of a subgroup analysis, provided that there are enough studies (see Subgroup analysis and investigation of heterogeneity).

Assessment of reporting biases

If at least 10 studies are included in the meta‐analysis, we will draw funnel plots (i.e. estimated differences in treatment effects against their standard error) and test for funnel plot asymmetry, to detect whether reporting bias is present. However, asymmetry in funnel plots can be caused by other sources, such as differences in methodological quality and true heterogeneity in intervention effects (Sterne 2017). We will also perform the Egger 1997 test for funnel plot asymmetry, as proposed by Sterne 2017.

Data synthesis

We will carry out a pair‐wise meta‐analysis if at least two relevant studies are available and if there is no substantial clinical or methodological heterogeneity (see Assessment of heterogeneity). As we expect heterogeneity in the clinical features of people with ASD, we will use the random‐effects model (DerSimonian 1986) for meta‐analysis (inverse variance approach in RevMan 5 (RevMan 2014)). By means of subgroup analysis, we will conduct separate meta‐analyses for the randomised and non‐randomised study designs. The results will be displayed in a forest plot (Reeves 2011).

If meta‐analysis is not appropriate (e.g. because of too few studies or considerable heterogeneity across studies), we will present a narrative description of the results.

We will prepare a 'Summary of findings' table in RevMan 5 (RevMan 2014), for the main comparison of the following outcomes:

1. social communication skills based on direct observation; and

2. social communication skills based on parent/caregiver or teacher report, or both.

We will provide information about the population, setting, intervention, and comparison in the table headings. For each outcome, we will describe (per column) the information listed below.

1. Illustrative comparative risks.

2. Number of participants and studies.

3. Quality of evidence, using the GRADE approach for assessment (high, moderate, low, very low) (Guyatt 2008). Factors that may decrease the quality of evidence are: 1. limitations in the design and implementation (e.g. lack of allocation concealment, lack of blinding, loss to follow‐up); 2. indirectness of evidence (e.g. to what extent PRT and treatment as usual or wait‐list control are compared indirectly); 3. unexplained heterogeneity or inconsistency of results (e.g. widely differing effect estimates); 4. imprecision of results (e.g. few participants and wide CIs); and 5. high probability of publication bias (e.g. failing to report studies that show no effect or even worsening). Two independent review authors (IS and IO) will assess the quality of evidence. If there is disagreement, they will discuss their findings to try to reach consensus.

4. Additional comments or differences from the standard methods.

Subgroup analysis and investigation of heterogeneity

We will explore possible causes of heterogeneity in study results by subgroup analysis, using meta‐regression, if there are at least 10 studies per outcome. Exploratory subgroup analysis will involve the following factors.

1. Participant characteristics:

   1. age (pre‐school (under 6 years of age) versus school‐age (6 to 12 years of age) versus adolescents (over 12 years of age));

   2. gender (male versus female);

   3. IQ (below average IQ (less than 85) versus average IQ (85 to 115) versus above average IQ (more than 115)); and

   4. severity of ASD symptoms based on scores on standardised tests such as the ADOS (Lord 2000) and ADI‐R (Lord 1994) (low versus moderate versus high).

2. Treatment characteristics:

   1. reported fidelity of implementation of PRT techniques (insufficient (less than 80%) versus sufficient (80% or more));

   2. duration of intervention;

   3. intensity or hours per week of intervention;

   4. implementation setting (at a clinic, at home, at school or at a community setting);

   5. therapist involvement (direct training versus indirect training (via parents, teachers, or peers) versus both direct and indirect training of individuals with ASD); and

   6. type of treatment used in the control group (ABA‐based versus non‐ABA‐based versus not known).

Sensitivity analysis

We will carry out sensitivity analyses to evaluate the robustness of the pooled effect sizes across the following components of methodological quality.

1. Lack of blinding. We will re‐analyse data, excluding data from studies with a high risk of blinding bias (see Assessment of risk of bias in included studies). High risk of blinding bias is defined as a high or unclear risk of bias in both the blinding of participants and personnel, and blinding of outcome assessments, given that evidence supporting psychological interventions can be strongly influenced by these study characteristics (Sonuga‐Barke 2013).

2. Incomplete outcome reporting. We will re‐analyse data, excluding data from studies with a high risk of bias due to incomplete outcome reporting.

3. Dropout rate. We will re‐analyse data, excluding data from studies with a high dropout rate (30% or higher) or different distribution of dropout between groups within the study, or both.

Additionally, we will perform sensitivity analyses to evaluate the robustness of the pooled effect sizes across the following two components related to medication use.

1. Lack of reporting on medication status. We will re‐analyse data, excluding data from studies that do not report on patients' medication status.

2. Use of medication. We will re‐analyse data, excluding data from studies in which patients use medication.

What's new

Date	Event	Description
10 August 2022	Amended	Correcting typo

History

Protocol first published: Issue 12, 2017

Date	Event	Description
29 July 2022	Amended	This protocol will not be progressed to the review stage. Please see Published notes for further information.

Appendices

Appendix 1. Examples of Pivotal Response Treatment learning opportunities

In Pivotal Response Treatment (PRT), 'learning opportunities' are created with PRT techniques that are based on the principles of ABC (i.e. antecedents, behaviour, consequences).

Example of a learning opportunity for the target behaviour 'three‐word sentence' in a young child with ASD

1. The child chooses to play with a ball (PRT technique: following the child's interests)

2. The therapist holds the ball after the child rolls the ball (PRT technique: catching the child's attention)

3. The therapist prompts the child by saying a three‐word sentence: 'roll the ball' (PRT technique: providing a clear learning opportunity with appropriate help)

4. If the child responds with 'roll ball', the therapist rolls the ball (PRT techniques: providing a natural reward and rewarding attempts)

Example of a learning opportunity for the target behaviour 'asking an informative question' in an adolescent with ASD

1. The adolescent enjoys playing different video games and the therapist brings a new video game (PRT technique: following the adolescent's interests)

2. The video game is on the table, but is still wrapped in paper (PRT technique: catching the adolescent's attention)

3. The therapist prompts the adolescent first by waiting on a response. If the adolescent does not respond, the therapist prompts the adolescent again by saying: "you can ask now: which game did you bring?'' (PRT technique: providing a clear learning opportunity with appropriate help)

4. If the adolescent says, "which game is it?", the therapist answers the question and unwraps the paper (PRT techniques: providing a natural reward and rewarding attempts)

Appendix 2. Ovid MEDLINE search strategy

1 exp child development disorders, pervasive/ or exp asperger syndrome/ or exp autism spectrum disorder/ or exp autistic disorder/
2 pervasive development$ disorder$.tw,kf.
3 (pervasive adj3 child$).tw,kf.
4 (PDD or PDDs or PDD‐NOS or ASD or ASDs).tw,kf.
5 autis$.tw,kf.
6 asperger$.tw,kf.
7 kanner$.tw,kf.
8 or/1‐7
9 pivotal$.tw,kf.
10 PRT.tw,kf.
11 Natural Language Paradigm.tw,kf.
12 NLP.tw,kf.
13 or/9‐12
14 8 and 13

Appendix 3. Study data to be extracted

Source

1. Study ID (created by review author)

2. Report ID (created by review author)

3. Review author ID

Methods

1. Study design

2. Total study duration

3. Random sequence generation

4. Allocation concealment

5. Blinding

6. Other concerns about bias

Participants

1. Total number

2. Setting

3. Diagnostic criteria

4. Diagnostic subtypes and number of participants per subtype

5. Age

6. Intelligence

7. Sex

8. Country

9. Co‐morbidity

Interventions

1. Total number of intervention groups

2. Intervention details for treatment group and control group

3. Fidelity of implementation for treatment group

Outcomes

1. Outcomes and time points

   1. Collected

   2. Reported

2. For each outcome of interest

   1. Outcome definition

   2. Measurement unit

   3. For scales: upper and lower limits, and whether high or low score is good

Analysis

1. Method of analysis (e.g. intention‐to‐treat or per protocol)

Results

1. Number of participants allocated to each intervention group

2. For each outcome of interest

   1. Sample size

   2. Missing participants

   3. Summary of data for each intervention group (e.g. 2 x 2 table for dichotomous data; means and SDs for continuous data)

Miscellaneous

1. Miscellaneous comments from study authors

2. Miscellaneous comments by review authors

Footnotes

ID: identifier; SD: standard deviation.

Appendix 4. Assessment of risk of bias for randomised studies

Random sequence generation

1. Low risk of bias: the investigators describe a random component in the sequence generation (e.g. referring to a random number table, using an electronic random number generator)

2. High risk of bias: the investigators describe a component in the sequence generation process that is not strictly random or is non‐random (e.g. by date of birth, by judgement of the clinician or by preference of the participant)

3. Unclear risk of bias: there is insufficient information available regarding the sequence generation process to make a judgement of high or low risk of bias

Allocation concealment

1. Low risk of bias: both participants and investigators could not foresee assignment (e.g. due to central allocation or sealed envelopes)

2. High risk of bias: participants or investigators could possibly foresee the assignments (e.g. using an open random allocation schedule, alternation on rotation)

3. Unclear risk of bias: there is insufficient information available regarding allocation concealment to make a judgement of high or low risk of bias

Blinding of participants and personnel

1. Low risk of bias: blinding of participants and key study personnel is ensured; or the outcomes are not likely to be influenced by the lack of blinding

2. High risk of bias: the lack of blinding is likely to have influenced the outcome; or the blinding could have been broken

3. Unclear risk of bias: there is insufficient information available regarding this issue to make a judgement of high or low risk of bias

Blinding of outcome assessment

1. Low risk of bias: blinding of outcome assessment is ensured; or it is judged unlikely that outcome measurement could have been influenced by the lack of blinding

2. High risk of bias: a lack of blinding is likely to have influenced the outcome measurement; or the blinding could have been broken

3. Unclear risk of bias: insufficient information is available to make a judgement of high or low risk of bias

Incomplete outcome data

1. Low risk of bias: there are no missing outcome data; it is unlikely that missing outcome data are related to the true outcomes; and/or missing data are imputed using appropriate methods

2. High risk of bias: it is likely that missing outcome data are related to the true outcome because of, for example, imbalance in numbers, when an 'as‐treated' analysis is done with substantial differences between the intervention received and the intervention assigned in the randomisation, or when there is a potentially inappropriate application of imputation

3. Unclear risk of bias: there is insufficient information reported to make a judgement of high or low risk of bias

Selective reporting

1. Low risk of bias: the study protocol is available and all of the pre‐specified outcomes have been reported in the pre‐specified way; or the study protocol is not available but the published reports include all expected or pre‐specified outcomes

2. High risk of bias: there are discrepancies between reporting of outcomes that were pre‐specified and the actual reported outcomes; or outcomes are reported incompletely

3. Unclear risk of bias: there is insufficient information available to make a judgement of high or low risk of bias

Other sources of bias

1. Low risk of bias: no additional sources of bias can be identified

2. High risk of bias: a potential source of bias is identified (e.g. related to study design)

3. Unclear risk of bias: there is insufficient information available or there is insufficient rationale or evidence that an identified problem will introduce bias

Appendix 5. Assessment of risk of bias for non‐randomised studies

Pre‐intervention

Bias due to confounding

1. Low risk of bias: no baseline or time‐varying confounding expected

2. Moderate risk of bias:

   1. confounding expected, all known important confounding domains appropriately measured and controlled for; and

   2. reliability and validity of measurement of important domains were sufficient, such that we do not expect serious residual confounding

3. Serious risk of bias:

   1. at least one known important domain was not appropriately measured, or not controlled for; or

   2. reliability or validity of measurement of an important domain was low enough that we expect serious residual confounding

4. Critical risk of bias:

   1. confounding inherently not controllable; or

   2. the use of negative controls strongly suggest unmeasured confounding

5. No information: no information on whether confounding might be present

Bias in selection of participants into the study

1. Low risk of bias:

   1. all participants who would have been eligible for the target trial were included in the study; and

   2. for each participant, start of follow‐up and start of intervention coincided

2. Moderate risk of bias:

   1. selection into the study may have been related to intervention outcome and the authors used appropriate methods to adjust for the selection bias; or

   2. start of follow‐up and start of intervention do not coincide for all participants; and

      1. the proportion of participants for which this was the case was too low to induce important bias; or

      2. the authors used appropriate methods to adjust for the selection bias; or

      3. the review authors are confident that the rate (hazard) ratio for the effect of intervention remains constant over time

3. Serious risk of bias:

   1. selection into the study was related (but not very strongly) to intervention and outcome, and this could not be adjusted for in analyses; or

   2. start of follow‐up and start of intervention do not coincide, and a potentially important amount of follow‐up time is missing from analyses, and the rate ratio is not constant over time

4. Critical risk of bias:

   1. selection into the study was very strongly related to intervention and outcome and this could not be adjusted for in analyses; or

   2. a substantial amount of follow‐up time is likely to be missing from analyses, and the rate ratio is not constant over time

5. No information: no information is reported about selection of participants into the study or whether start of follow‐up and start of intervention coincide

At intervention

Bias in classification of interventions

1. Low risk of bias:

   1. intervention status is well defined; and

   2. intervention definition is based solely on information collected at the time of intervention

2. Moderate risk of bias:

   1. intervention status is well defined; and

   2. some aspects of the assignments of intervention status were determined retrospectively

3. Serious risk of bias:

   1. intervention status is not well defined; or

   2. major aspects of the assignments of intervention status were determined in a way that could have been affected by knowledge of the outcome

4. Critical risk of bias: an extremely high amount of misclassification of intervention status, e.g. because of unusually strong recall biases

5. No information: no definition of the intervention or no explanation of the source of information about intervention status is reported

Post‐intervention

Bias due to deviations from intended intervention

Effect of assignment to intervention

1. Low risk of bias:

   1. any deviations from intended intervention reflected usual practice; or

   2. any deviations from usual practice were unlikely to impact on the outcome

2. Moderate risk of bias: there were deviations from usual practice, but their impact on the outcome is expected to be slight

3. Serious risk of bias: there were deviations from usual practice that were unbalanced between the intervention groups and likely to have affected the outcome

4. Critical risk of bias: there were substantial deviations from usual practice that were unbalanced between the intervention groups and likely to have affected the outcome

5. No information: no information is reported on whether there is deviation from the intended intervention

Effect of starting and adhering to intervention

1. Low risk of bias: the important co‐interventions were balanced across intervention groups, and there were no deviations from the intended interventions (in terms of implementation or adherence) that were likely to impact on the outcome

2. Moderate risk of bias:

   1. there were deviations from intended intervention, but their impact on the outcome is expected to be slight; or

   2. the important co‐interventions were not balanced across intervention groups, or there were deviations from the intended interventions (in terms of implementation or adherence, or both) that were likely to impact on the outcome, and the analysis was appropriate to estimate the effect of starting and adhering to intervention, allowing for deviations (in terms of implementation, adherence and co‐intervention) that were likely to impact on the outcome

3. Serious risk of bias:

   1. the important co‐interventions were not balanced across intervention groups, or there were deviations from the intended interventions (in terms of implementation or adherence, or both) that were likely to impact on the outcome; and

   2. the analysis was not appropriate to estimate the effect of starting and adhering to intervention, allowing for deviations (in terms of implementation, adherence and co‐intervention) that were likely to impact on the outcome

4. Critical risk of bias:

   1. there were substantial imbalances in important co‐interventions across intervention groups, or there were substantial deviations from the intended interventions (in terms of implementation or adherence, or both) that were likely to impact on the outcome; and

   2. the analysis was not appropriate to estimate the effect of starting and adhering to intervention, allowing for deviations (in terms of implementation, adherence and co‐intervention) that were likely to impact on the outcome

5. No information: no information is reported on whether there is deviation from the intended intervention

Bias due to missing data

1. Low risk of bias:

   1. data were reasonably complete; or

   2. proportions of and reasons for missing participants were similar across interventions groups; or

   3. the analysis addressed missing data and is likely to have removed any risk of bias

2. Moderate risk of bias:

   1. proportions of and reasons for missing participants differ slightly across intervention groups; and

   2. the analysis is unlikely to have removed the risk of bias arising from the missing data

3. Serious risk of bias:

   1. proportions of missing participants differ substantially across interventions, or reasons for missingness differ substantially across interventions; and

   2. the analysis is unlikely to have removed the risk of bias arising from the missing data, or missing data were addressed inappropriately in the analysis, or the nature of the missing data means that the risk of bias cannot be removed through appropriate analysis

4. Critical risk of bias:

   1. there were critical differences between interventions in participants with missing data; and

   2. missing data were not, or could not, be addressed through appropriate analysis

5. No information: no information is reported about missing data or the potential for data to be missing

Bias in measurement of outcomes

1. Low risk of bias:

   1. the methods of outcome assessment were comparable across intervention groups; and

   2. the outcome measure was unlikely to be influenced by knowledge of the intervention received by study participants (i.e. is objective), or the outcome assessors were unaware of the intervention received by study participants; and

   3. any error in measuring the outcome is unrelated to intervention status

2. Moderate risk of bias:

   1. the methods of outcome assessment were comparable across intervention groups; and

   2. the outcome measure is only minimally influenced by knowledge of the intervention received by study participants; and

   3. any error in measuring the outcome is only minimally related to intervention status

3. Serious risk of bias:

   1. the methods of outcome assessment were not comparable across intervention groups; or

   2. the outcome measure was subjective (i.e. vulnerable to influence by knowledge of the intervention received by study participants), and the outcome was assessed by assessors aware of the intervention received by study participants; or

   3. error in measuring the outcome was related to intervention status

4. Critical risk of bias: the methods of outcome assessment were so different that they cannot reasonably be compared across intervention groups

5. No information: no information is reported about the methods of outcome assessment

Bias in selection of the reported result

1. Low risk of bias: there is clear evidence (usually through examination of a pre‐registered protocol or statistical analysis plan) that all reported results correspond to all intended outcomes, analyses and sub‐cohorts

2. Moderate risk of bias:

   1. the outcome measurements and analyses are consistent with an a priori plan, or are clearly defined and both internally and externally consistent; and

   2. there is no indication of selection of the reported analysis from among multiple analyses; and

   3. there is no indication of selection of the cohort or subgroups for analysis and reporting on the basis of the results

3. Serious risk of bias:

   1. outcomes are defined in different ways in the methods and results sections, or in different publications of the study; or

   2. there is a high risk of selective reporting from among multiple analyses; or

   3. the cohort or subgroup is selected from a larger study for analysis and appears to be reported on the basis of the results

4. Critical risk of bias:

   1. there is evidence or strong suspicion of selective reporting of results; and

   2. the unreported results are likely to be substantially different from the reported results

5. No information: there is too little information to make a judgement (e.g. only an abstract is available for the study)

Contributions of authors

Iris Smeekens (IS) has overall responsibility for the review. IS revised the protocol and completed the search strategy. Martine van Dongen‐Boomsma (MvDB) and Wouter Staal (WS) had an equal contribution in supervising IS in writing the protocol and thus share last authorship. Iris Oosterling, Jenny den Boer, and Jan Buitelaar also provided feedback on the protocol, which was incorporated by IS in the final version.

Sources of support

Internal sources

• Karakter Child and Adolescent Psychiatry University Centre, Nijmegen, Netherlands

  Employer for all six authors

External sources

• None, Other

  N/A

Declarations of interest

Iris Smeekens' institution received funding from a grant from ZonMW (Netherlands Organisation for Health Research and Development) for her work on this review.
Iris Oosterling is employed at Karakter Child and Adolescent Psychiatry University Centre, Nijmegen, Netherlands, and her institution has received grants for clinical research activities at the centre.
Jenny den Boer ‐ none known.
Jan Buitelaar is employed at Karakter Child and Adolescent Psychiatry University Centre, Nijmegen, Netherlands, and his institution has received grants for clinical research activities at the centre.
Wouter Staal is employed as a Child and Adolescent Psychiatrist at Karakter Child and Adolescent Psychiatry University Centre, Nijmegen, Netherlands, and his institution has received grants for clinical research activities at the centre.
Martine van Dongen‐Boomsma is employed as a Child and Adolescent Psychiatrist at Karakter Child and Adolescent Psychiatry University Centre, Nijmegen, Netherlands, and her institution has received grants for clinical research activities at the centre.

Wouter G Staal shares last authorship with Martine van Dongen‐Boomsma.

Edited (no change to conclusions)