Abstract
There has been increasing interest in parent-mediated interventions (PMIs) for children with autism spectrum disorders (ASDs). The objective of this systematic review and meta-analysis was to examine the effect of PMIs compared to no PMI for children with ASD aged 2–17 years. The primary outcome was adaptive functioning rated by a parent or clinician. The secondary outcomes were long-term adaptive functioning rated by the parents, adverse events, core symptoms of ASD, disruptive behavior, parental well-being, quality of life of the child rated by the parents and anxiety. The MEDLINE, PsycInfo, Embase, and CINAHL databases were searched in March 2020. The Cochrane Risk of Bias Tool was used to rate the individual studies, and the certainty in the evidence was evaluated using GRADE. We identified 30 relevant randomized controlled trials (RCTs), including 1,934 participants. A clinically relevant effect of PMIs on parent-rated adaptive functioning was found with a low certainty of evidence [Standard mean difference (SMD): 0.28 (95% CI: −0.01, 0.57)] on Vineland Adaptive Behavior Scales (VABS), whereas no clinically relevant effect was seen for clinician-rated functional level, with a very low certainty of evidence [SMD on Clinical Global Impressions (CGI)-severity scale: SMD −0.45 [95% CI: −0.87, −0.03)]. PMIs may slightly improve clinician-rated autism core symptoms [SMD: −0.35 (95% CI: −0.71, 0.02)]. Additionally, no effect of PMIs on parent-rated core symptoms of ASD, parental well-being or adverse effects was identified, all with a low certainty of evidence. There was a moderate certainty of evidence for a clinically relevant effect on disruptive behavior [SMD: 0.55 (95% Cl: 0.36, 0.74)]. The certainty in the evidence was downgraded due to serious risk of bias, lack of blinding, and serious risk of imprecision due to few participants included in meta-analyses. The present findings suggest that clinicians may consider introducing PMIs to children with ASD, but more high-quality RCTs are needed because the effects are not well-established, and the results are likely to change with future studies. The protocol for the systematic review is registered at the Danish Health Authority website (www.sst.dk).
Keywords: autistic disorder, autism spectrum disorder, parent-mediated intervention, caregiver-mediated intervention, early intervention, treatment outcome
Introduction
Autism spectrum disorder (ASD) is a neurodevelopmental disorder characterized by early-onset difficulties in social interaction, communication and stereotyped repetitive behaviors and interests (1, 2), and it affects ~1% of the population in high-income countries (3). The disorder is expected to be lifelong, and people on the spectrum are reported to have an elevated mortality risk, lower educational level, reduced quality of life and higher frequency of comorbid disorders, e.g., depression and anxiety (2, 4–6).
Various behavioral and structural interventions have been developed to improve adaptive functioning, behavioral problems and quality of life for people with ASD, and some interventions reduce ASD symptomatology (2, 7, 8). Several early behavioral interventions have also been developed (9). In recent decades, increasing interest in early parent-child interactions has been observed, the involvement of parents in therapy has increased, and various parent-mediated interventions (PMIs) have been developed.
PMIs are advantageous because they reduce the demands on children with ASD compared to behavioral approaches and bring treatment into a home and community setting, enabling the transfer of skills to real-life settings. PMIs engage parents in the role of a therapist to implement interventions in an individualized and sensitive way (10). Parents already spend much time with their children, which provides an obvious opportunity for a cost-effective intervention, with extensive implementation and generalization opportunities in everyday life and through different contexts (11).
The PMIs vary, including teaching comprehensive skills and others targeting specific impairments, e.g., joint attention, communication, or language (2). The various PMIs cover education, training and coaching from clinicians to parents, with the overarching aim of improving opportunities for children to learn through different contexts. This is essential to ensure generalization and maintain treatment gains (11, 12). Initially, PMIs targeted younger children with ASD; however, simultaneously with the broadening of treatment targets, the age group has broadened to include subjects from early childhood into young adulthood (11).
Several reviews with or without meta-analysis have been conducted in the research field (10, 13–16), but only a few have used Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) to assess the certainty of the evidence (10, 13), and none have addressed both beneficial and adverse aspects of PMIs. The outcome focus of the reviews varies considerably, differing from child outcomes, such as language, adaptive functioning and social communication, to parent outcomes, such as parental stress and quality of life; only a few reviews included a meta-analysis (10, 13, 15–17). Some reviews expanded the parent concept to parent-focused interventions and included all interventions that included parents (17).
A more recent systematic review and meta-analysis of outcomes of PMIs for younger children by Nevill et al. (13) found moderate positive outcomes of PMIs regarding language communication, autism symptom severity, and cognition, but the evidence of positive changes in socialization was very low. The review concluded that the overall quality of more recent randomized controlled trials (RCTs) is improving (13). Several RCTs have been published since the Nevill et al. (13) review and meta-analysis. Thus, to provide clinicians and guideline panels as well as caregivers with an updated overview of the current evidence from RCTs, the objective of this systematic review and meta-analysis was to synthesize the effect of PMIs for children and adolescents with autism aged 18 months−17 years on both beneficial and adverse outcomes.
In particular the following questions were addressed:
- What is the overall effect of the PMIs for children and adolescents with autism?
- What is the effect of the PMIs on parental stress, parental well-being and quality of life?
- Which adverse effects of the PMIs are seen in current research?
- What is the quality of the identified research in this field?
- Will current evidence of the PMIs be sufficient to recommend PMIs for children with autism?
Methods
This systematic review and meta-analysis was conducted according to the recommendations of the Cochrane Handbook (18) and is reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement (19) (Prisma checklist is provided in Supplementary Table 1). The systematic review also follows the structure described by the population, intervention, comparison and outcome (PICO) characterization (20). The certainty in the evidence was rated by the GRADE approach (21). The study is part of the national Clinical Practice Guidelines on the treatment of ASD among children and adolescents published by the Danish Health Authority in 2021 (22). The content of the study protocol, including review question, search strategy, inclusion and exclusion criteria, and risk of bias assessment, was prespecified, registered and approved by the management at the Danish Health Authority in November 2019 (and the protocol is available at the Danish Health Authority website: www.sst.dk).
Search Strategy
The systematic search was conducted in March 2020 by a search specialist (B.H.P.). The databases searched were MEDLINE, PsycInfo, Embase, and CINAHL. The search was performed in two steps: (a) search for systematic reviews and meta-analyses with a filter and (b) search for primary literature, both with a combination of medical subject heading (MeSH)/index terms and free-text searches. The search was limited to articles in English and Scandinavian language referring to language skills of the review authors. The detailed search strategy is presented in the Supplementary Table 2.
A cross reference search and screening of reference lists of included articles and previous reviews was performed, and the guideline working group members (content experts) conferred whether any studies were missing from the search. Study authors were not contacted to identify additional studies.
Study Selection
Articles generated from the defined search strategy of individual RCTs were deduplicated and imported from RefWorks into Covidence software for literature screening and data management (www.covidence.org). According to prespecified PICO criteria, one reviewer (M.L.R.) evaluated the titles and abstracts of eligible articles (see below). The identified full-text articles were independently screened by two reviewers (review authors: M.L.R., C.C. and M.B.L.). Any disagreements were resolved through discussion. The eligible studies had to match the following criteria:
Population
Children and adolescents from 18 months to 17 years of age diagnosed with ASD according to diagnostic criteria with or without comorbidities.
Intervention
Parent-mediated interventions for children and adolescents with ASD with 8 or more sessions.
Comparator
No parent-mediated intervention.
Outcomes
Primary Outcome
Adaptive functioning was rated at a minimum of 8 weeks by the parent or clinician.
Secondary Outcomes
- Parent-rated adaptive functioning after at least 6 months of follow-up
- Adverse effects at minimum 8 weeks
- Core symptoms of ASD, parent- or clinician-rated at a minimum of 8 weeks
- Disruptive behavior, parent-rated at a minimum of 8 weeks
- Parental well-being at a minimum of 8 weeks
- Quality of life, parent-rated at a minimum of 8 weeks
- Anxiety at a minimum of 8 weeks
Study Design
Only RCTs were included in this review. The aim of random assignment used in RCTs is to prevent selection bias by distributing the characteristics of patients who may influence the outcome randomly between the groups; therefore, RCTs are considered to minimize the risk of confounding factors influencing the results, thus providing the most reliable evidence on the effectiveness of interventions.
Data Extraction of Individual Randomized Trials
A predefined template in Covidence software was used to conduct data extraction independently by two out of three reviewers (J.F.R., C.B.K., M.N.H.). Extraction of the following descriptive and quantitative characteristics was performed:
Characteristics of the study: authorship, year, country, setting, sample size, design, methods, duration of follow-up, source of funding, conflict of interest.
Characteristics of the population: age, race/ethnicity, socioeconomic status, cointerventions, information regarding respondent bias, or representativeness of the included population.
Description of the intervention.
Description of the comparator group.
Outcomes and timepoints for outcomes, as mentioned above.
Risk of Bias and Certainty of Evidence
We used GRADE to assess the certainty of evidence, which was categorized as very low, low, moderate, and high (21). Each RCT study started at a high certainty level and was assessed for possibly being rated down based on five domains: overall risk of bias, inconsistency, indirectness, imprecision, and publication bias. Following GRADE, whenever sample size in the analysis were <100 participants, the study was downgraded for imprecision.
The criterion provided by the Cochrane Collaboration's tool for assessing the risk of bias of RCTs (18) was used. The Cochrane Collaboration tool consists of seven quality domains, and each domain is classified into three levels of risk of bias (low, high, or unclear). The seven domains are: (a) sequence generation, (b) allocation concealment, (c) blinding of participants and personnel, (d) blinding of outcome assessment, (e) incomplete outcome data, (f) selective outcome reporting, and (g) other sources of bias. The quality of the included studies was assessed independently by two out of three reviewers (J.F.R., C.B.K., and M.N.H.).
Data Synthesis
The effect size was calculated using a standardized mean difference (SMD) [95% confidence interval (CI)] if data were reported as a continuous variable, and different between-study measurement methods were applied. The SMD was translated back to a mean difference for the primary outcome, parent- or clinician rated adaptive functioning using the SD from the control group from the median largest study with the lowest risk of bias.
Adverse effects were expressed as relative risk (RR) with 95% CI, and in the analysis, there were zero adverse effects in both the intervention and control groups, a risk difference meta-analysis (RD; 95% CI) was calculated.
The following subgroups of intervention targets and/or content were applied in the analysis: (a) language, (b) aggression management, (c) training in social skills, and (d) other interventions. For adverse events, the included studies had zero events in both the intervention and control groups; thus, a risk difference meta-analysis (RD; 95% CI) was calculated. A random-effect model was applied for all models. Statistical heterogeneity was quantified using I2 statistics (23). Since only a few studies were included in each outcome, we did not perform funnel plots to address the potential risk of publication bias.
Review Manager Software (version 5.3) (The Nordic Cochrane Collaboration, Copenhagen, Denmark) (24) was used to perform the analysis and forest plots.
Results
In the initial search, we identified five systematic reviews (10, 13–16) (Figure 1). From these, 22 RCTs were identified (25–46). Through a search for primary literature, an additional six RCTs were identified (47–52) (Figure 2). An additional two studies were included from reference lists in the included articles (53, 54). In total, 30 studies were included in this review and meta-analysis (25–52).
A list of the 14 excluded studies from full-text screening, including reasons for exclusion (55–68), is provided in the Supplementary Table 3.
Description of the Primary Studies
Characteristics of the included studies are presented in the Supplementary Table 4. The interventions of the included RCTs consisted of PMIs with various target groups and contents. The interventions' commonalities were the parents being direct recipients of the intervention and the parents' intentions to implement and train applied skills with the child. In most studies, the intervention targeted specific areas of ASD (such as joint attention and social-communicative skills). However, in five studies (26, 27, 34, 42, 45), the intervention primarily focused on improving parent-child social interaction and reducing behavioral difficulties and demand-avoidant behavior. In total, parents of 1,934 children and adolescents participated in the studies, and the age range was 16 months to 17 years. Most of the trials were developed for younger children with ASD, with 23 of 30 studies including children 7 years of age and younger, and only seven studies focusing on children below 4 years of age (28–30, 36, 48, 53, 54). No studies investigated adolescents older than 14 years of age, and only four articles included children older than 11 (26, 37, 38, 42). These four interventions either targeted disruptive behavior or supported positive behavior. The intervention period in the included studies were 8 weeks-24 months.
The control groups consisted of waitlist or other passive control conditions in 11 of the studies (31, 36, 37, 42, 45–47, 49, 51, 52, 54); treatment/management as usual in 10 of the studies (25, 28, 30, 32, 39, 40, 43, 44, 48, 53); an active control with a less extensive educational program, psycho-education, placebo parent-intervention in eight of the studies; and anti-psychotic medicine (risperidone) alone in a single study (26, 27, 29, 33–35, 38, 41, 50). Four studies (36, 41, 47, 54) either did not report data predefined as primary or secondary outcomes in the present context or did not report data in a manner for them to be included in the meta-analysis. A conclusion as to whether the individual studies were rated with high, low or unclear risk of bias within each domain was reached (Figure 3).
Synthesis of the Results on Primary Outcome—Adaptive Functioning (Parent- and Clinician-Reported)
With respect to the primary outcome defined in this review and meta-analysis eight studies reported the effects of PMIs on parent-reported adaptive functioning (25, 28, 29, 40, 44, 48, 52, 53). There was a small but clinically relevant effect [SMD: 0.28 (95% CI:−0.01, 0.57)] (Figure 4), corresponding to an MD on the Vineland scale of 3.5 [95% CI: 0.92, 6.02] calculated from the endpoint SD from the control group in Vernon (52). There was a low degree of heterogeneity (I2 = 10). All studies included in the meta-analysis used the Vineland Adaptive Behavior Scales 1st or 2nd Edition to assess adaptive functioning; however, they did not all report scores on the same sub-domains. For the meta-analysis, the adaptive composite score was preferred; when not reported, the daily living subdomain was secondarily prioritized. However, three of the included studies (25, 48, 53) did not report any of these scores; for these, the Communications subdomain outcome was used for the meta-analysis. The outcomes were considered parent-rated when based on a parent-rated questionnaire or the clinicians rating a semi-structured interview with parents. One study Tonge et al. (44) combined the parent interview with a simultaneous observation of the child but was categorized as parent-rated since the main information for the scoring was still derived from the parents.
The results showed no subgroup differences in parent-rated adaptive functioning between the different targets of parent-mediated interventions (p = 0.58).
For the primary outcome parent-rated adaptive functioning there was low certainty in the effect estimates due to rating down for serious risk of bias due to lack of blinding of participants and outcome assessors, as well as serious risk of imprecision due to few participating children. In conclusion, PMIs may slightly improve parent-rated adaptive functioning.
Only 2 studies of PMIs focusing on language reported clinician-rated adaptive functioning (33, 49), and the studies found no effect of PMIs [SMD −0.45 (95% CI: −0.87, −0.03) (Figure 5) corresponding to a MD on CGI severity scale: −0.36 (95% CI: −0.70, −0.02)]. The degree of heterogeneity was low (I2 = 0%). There was very low certainty in the evidence on clinician-rated adaptive functioning due to rating down for serious risk of bias due to problems with lack of blinding of participants and outcome assessors and very serious risk of imprecision due to few participating children and wide confidence intervals. Thus, it is uncertain if the PMIs increase the clinician-rated adaptive functioning.
Synthesis of the Results on Secondary Outcomes
A single study (44) reported on parent-rated adaptive functioning after 6 months of follow-up on Vineland Adaptive Behavior Scales Daily Living subscale. This study did not report an end of treatment effect, thus was included in the meta-analysis of the primary outcome (Figure 4). The study showed significant improvement in the daily living subdomain of VABS [effect size −0.62 (95% CI: −1.10, −0.13)] and the socialization subdomain [effect size−0.60 (95% CI: −1.09, −0.12)] in the PMI group compared to a psychoeducative group (44). Adverse effects were only reported in 2 of 30 studies (29, 33) and reported no adverse effects in any of the two groups (Figure 6). One of the studies compared the Early Start Denver Model to treatment as usual (29), and the other compared Pivotal Response Training to 12 sessions of psychoeducation (33). There was low certainty in the evidence due to rating down for very serious risk of imprecision because of few participating children and few studies reporting adverse effects. Thus, PMIs may not cause substantial adverse effects.
Seven studies reported the effect of PMIs on parent-rated core symptoms of ASD (30, 31, 33, 35, 43, 44, 49). No relevant effect was found [SMD: 0.06 (95% CI: −0.18, 0.30)] (Figure 7). The degree of heterogeneity was low for the parent-rated outcome (I2 = 20%). The results showed no subgroup differences between the different targets of PMIs, e.g., language and disruptive behavior (p = 0.91). There was low certainty in the evidence due to rating down for serious risk of bias because of lack of blinding of participants and outcome assessors, as well as serious risk of imprecision due to few participating children. Thus, PMIs may result in little or no clinical relevant change in parent-rated autism core symptoms.
Nine studies reported a small effect of PMIs on clinician-rated core symptoms of ASD [SMD: −0.35 (95% CI: −0.71, 0.02)] (Figure 8) (25, 28, 29, 32, 35, 39, 48, 50, 52). The degree of heterogeneity was considerable for the clinician-rated outcome (I2 = 69%). The results showed no subgroup differences between the different targets of PMIs, e.g., language and disruptive behavior (p = 0.67). There was low certainty in the evidence due to rating down for serious risk of bias because of lack of blinding of participants and outcome assessors, as well as serious risk of imprecision due to few participating children. Thus, PMIs may slightly improve clinician-rated autism core symptoms.
Regarding parent-rated disruptive behavior, there was a moderate and clinically relevant effect of PMIs [SMD: −0.55 (95% CI: −0.74, −0.36)] (Figure 9). None of the language interventions reported on this outcome. There was a low degree of heterogeneity (I2 = 26%), and the results showed no subgroup differences between the different targets of PMIs (p = 0.53). There was moderate certainty in the evidence due to rating down for serious risk of bias because of lack of blinding of participants and outcome assessors. Thus, PMIs probably improves parent-rated disruptive behavior considerably.
Parental well-being showed a small and not clinically relevant effect of PMIs [SMD: −0.16 (95% CI: −0.32, −0.01)] (Figure 10). There was a low degree of heterogeneity (I2 = 6%), and the results showed no subgroup differences between the different targets of PMIs (p = 0.64). There was low certainty in the evidence due to rating down for very serious risk of imprecision because of few participating children and wide confidence intervals (Supplementary Table 5). Thus, PMIs may result in little or no clinically relevant change in parental well-being.
None of the included studies reported on child anxiety or parent-rated quality of life of the child.
Discussion
The current systematic review identified 30 RCTs of PMIs for children and adolescents with ASD. The results showed a clinically relevant effect on parent-rated adaptive functioning but no effect when the outcome was rated by a clinician. Moreover, PMIs may be a valuable treatment for disruptive behavior in children and adolescents with ASD. There was no effect on core symptoms of ASD (both parent- and clinician-rated), parent-rated adaptive functioning after 6 months or adverse effects. No studies reported on anxiety or parent-rated quality of life of the child. The certainty of the effect estimates reported in the studies was moderate to very low due to serious risk of bias and very serious risk of imprecision; thus, the effect of PMIs on children and adolescents with ASD is still uncertain. A qualitative synthesis of the studies revealed significant variability across studies.
Meanwhile, previous reviews (10, 13–16) presented evidence in favor of PMIs and emphasized the positive effect of PMIs on joint attention and social interaction in the parent-child dyad, which was not among the prespecified outcomes of this review (10, 15, 16).
This meta-analysis suggests a clinically relevant effect of PMIs for children and adolescents with ASD on parent-rated adaptive functioning, which was not found in the review and meta-analysis by Oono et al. (10). This may be explained by the inclusion of more studies in our review compared to Oono et al. (10) and thus an increase in power. This is a strong indicator of the efficacy of PMIs, since none of the interventions directly affects adaptive functioning, and the positive outcome on adaptive functioning could be a cascading effect of the PMIs.
The results from the meta-analysis align with results from previous reviews regarding favorable results of child disruptive behavior (14, 15). Interestingly, interventions targeting behavior as well as social communication and joint attention for younger children demonstrated positive effects of PMIs on disruptive behavior (50).
Previous reviews have also identified minor improvements in core symptoms of ASD in children participating in PMIs (10, 13), which was not confirmed in this meta-analysis. However, the clinician-rated core symptom measures favored of the PMIs, with a non-significant but clinically relevant effect.
As early interventions have been developed for children with ASD over the past several years (69), it is expected that most studies target younger children when searching for PMIs. This was confirmed in the current review, where only two of 30 included studies targeted children 5 years of age and older (38, 42), although the age range of the study population in this review 18 months to 17 years. The age range was suggested by the scientific committee, and interest associations which were both appointed by the Danish Health Authority to develop national Clinical Practice Guidelines in Denmark. The inclusion of both children and adolescents was relevant for most of the frameworks in the guideline development and was therefore applied to all of the PICOs.
Typically, interventions targeting social communication are recommended to younger children. This underlines the necessity of using interventions appropriate for the child's age and developmental stage. With respect to the remaining secondary outcomes, a clinically relevant effect regarding clinician rated core symptoms of ASD was seen in favor of PMIs. Moreover, the interventions targeting conduct problems were provided to younger children with ASD aged 2–9 years of age (27, 34, 45); interestingly, some of the studies extended the intervention to be applied to children 4 and 5 years of age to 12 and 13-year-old children (26, 42), suggesting that these kinds of interventions are effective at improving positive behavior in both younger and older children.
Strengths and Limitations of the Included Studies
A critical limitation of the included studies is that different outcome measures were used to assess the same effect; and when they used the same assessment tools, they were administered differently. For example, the Vineland Adaptive Behavior Scale (VABS) was used to assess adaptive functioning, and it was administered either as an interview or a parent-reported questionnaire (70). Another limitation is a generally small sample size in several studies.
Strengths and Limitations of the Methodology of the Present Systematic Review and Meta-Analysis
A major strength of this systematic review and meta-analysis is that it was performed according to principles described in GRADE and PRISMA as well as the PICO framework. However, from a resource saving perspective, we chose a stepwise literature search by initially searching for existing reviews to identify eligible studies, followed by a search for primary studies based on the latest search date of an existing high-quality systematic review, and finally screening reference lists of included studies and conferring content experts (working group at the Danish Health Authority). Post-hoc, we identified three articles with a total of 307 participants (71–73) (Supplementary Table 6) that were not identified in any of the included reviews by consulting content experts or in the reference list of the included studies. Since the ability to identify relevant studies is mainly dependent on the scope and search quality of the existing reviews, we acknowledge that our search may have been limited regarding both search specificity (recall) and sensitivity (precision). However, the results did not change substantially when performing post-hoc sensitivity analysis and including results from the unidentified studies. However, the increased power strengthened the positive results of PMIs on both parent-rated adaptive functioning [SMD: 0.27 (95% CI: 0.02, 0.52)] and clinician-rated autism core symptoms [SMD: −0.34 (95% CI: −0.64, −0.03)] (Supplementary Table 7).
Future Research
To estimate the effect of PMIs, further research with large high-quality RCTs investigating manualized interventions and following standardized principles for trial design, content and reporting is needed. Interventions previously investigated in RCTs need replication studies to build on the evidence of the intervention. In future research, it could be interesting to investigate the association between contextual factors, such as age and effect size, to address which children are most likely to benefit from PMIs. A need for measurement consistency still remains and is recommended in future research, to improve comparison between studies. Furthermore, there is a need to assess a core outcome set to investigate the importance of anxiety, parent-rated adaptive functioning after a minimum of 6 months and parent-rated child's quality of life.
Conclusion
When PMIs are delivered to children with ASD, it is recommended to use manualized interventions targeting autism spectrum disorders, and the characteristics of the included PMIs are that the child should be directly or indirectly involved in the intervention. However, the parents are the primary participants in the treatment and must actively train the different skills included in the intervention, both during the sessions and/or at home between sessions. The intervention must be adapted to the age and development of the child.
Based on the current evidence, there appears to be a benefit of providing PMIs to parents of children and adolescents with ASD concerning adaptive functioning and disruptive behavior reported by the parents. Perhaps by enhancing the parents' understanding and management of their child's pervasive disorder, it seems that the parents are empowered, which is supported by the trend toward improving parental well-being. As expected, there were minor differences between the intervention and control groups in changing core symptoms of ASD. PMIs may slightly improve clinician-rated autism core symptoms. Even though there were few reports on adverse effects, any adverse effects of the PMIs were considered insignificant and few. Adverse effects should be included in future studies. However, since the evidence base's certainty is low, the limitations of the current literature hinder the possibility of drawing any solid conclusions, and more well-designed, high-quality clinical trials of sufficient duration are required.
Author Contributions
MR, JR, BP, CK, ST, and MH: methodology. MR, JR, BP, CK, and MH: data curation. MH: formal analysis, visualization, supervision, and project administration. All authors investigation, resources, writing—original draft preparation, writing—review and editing, funding acquisition, and read and agreed to the published version of the manuscript.
Funding
This research was funded by the Danish Health Authority. The Parker Institute was supported by a core grant by the OAK foundation (OCAY-18-774-OFIL).
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Acknowledgments
The authors would like to thank the work and reference group, as well as the secretary of the National Clinical Guideline for the Treatment of Autism Spectrum Disorders in Children and Adolescents, The Danish Health Authority. The sources of support had no influence on the content of the manuscript.
Supplementary Material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2021.773604/full#supplementary-material
References
- 1.Organization WH . ICD-10: International Statistical Classification of Diseases and Related Health Problems. tenth revision. 2nd ed. (2004). Available online at: https://apps.who.int/iris/handle/10665/42980
- 2.Lai M-C, Lombardo MV, Baron-Cohen S. Autism. Lancet. (2014) 383:896–910. 10.1016/S0140-6736(13)61539-1 [DOI] [PubMed] [Google Scholar]
- 3.Fombonne E, MacFarlane H, Salem AC. Epidemiological surveys of ASD: advances and remaining challenges. J Autism Dev Disord. (2021) 51:4271–90. 10.1007/s10803-021-05005-9 [DOI] [PubMed] [Google Scholar]
- 4.Knüppel A, Telléus GK, Jakobsen H, Lauritsen MB. Quality of life in adolescents and adults with autism spectrum disorder: results from a nationwide Danish survey using self-reports and parental proxy-reports. Res Dev Disabil. (2018) 83:247–59. 10.1016/j.ridd.2018.09.004 [DOI] [PubMed] [Google Scholar]
- 5.Knüppel A, Telléus GK, Jakobsen H, Lauritsen MB. Characteristics of young adults with autism spectrum disorder performing different daytime activities. J Autism Dev Disord. (2019) 49:542–55. 10.1007/s10803-018-3730-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Cusack J, Shaw S, Spiers J, Sterry R. Personal tragedies, public crisis: the urgent need for a national response to early death in autism. Autistica March. (2016). [Google Scholar]
- 7.Maglione MA, Gans D, Das L, Timbie J, Kasari C. Nonmedical interventions for children with ASD: recommended guidelines and further research needs. Pediatrics. (2012) 130(Suppl.2):S169–78. 10.1542/peds.2012-0900O [DOI] [PubMed] [Google Scholar]
- 8.French L, Kennedy EMM. Annual Research Review: early intervention for infants and young children with, or at-risk of, autism spectrum disorder: a systematic review. J Child Psychol Psychiatry. (2018) 59:444–56. 10.1111/jcpp.12828 [DOI] [PubMed] [Google Scholar]
- 9.Reichow B, Hume K, Barton EE, Boyd BA. Early intensive behavioral intervention (EIBI) for young children with autism spectrum disorders (ASD). Cochrane Database Syst Rev. (2018) 5:Cd009260. 10.1002/14651858.CD009260.pub3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Oono IP, Honey EJ, McConachie H. Parent-mediated early intervention for young children with autism spectrum disorders (ASD). Cochrane Database Syst Rev. (2013) 2013:CD009774. 10.1002/14651858.CD009774.pub2 [DOI] [PubMed] [Google Scholar]
- 11.Shalev RA, Lavine C, Di Martino A. A systematic review of the role of parent characteristics in parent-mediated interventions for children with autism spectrum disorder. J Dev Phys Disabil. (2020) 32:1–21. 10.1007/s10882-018-9641-x [DOI] [Google Scholar]
- 12.Brookman-Frazee L, Vismara L, Drahota A, Stahmer A, Openden D. Parent training interventions for children with autism spectrum disorders. In: Matson JL, editor, Applied Behavior Analysis for Children With Autism Spectrum Disorders. 1 ed. New York, NY: Springer-Verlag; (2009). p. 237–57. 10.1007/978-1-4419-0088-3_14 [DOI] [Google Scholar]
- 13.Nevill RE, Lecavalier L, Stratis EA. Meta-analysis of parent-mediated interventions for young children with autism spectrum disorder. Autism. (2018) 22:84–98. 10.1177/1362361316677838 [DOI] [PubMed] [Google Scholar]
- 14.Postorino V, Sharp WG, McCracken CE, Bearss K, Burrell TL, Evans AN, et al. A systematic review and meta-analysis of parent training for disruptive behavior in children with autism spectrum disorder. Clin Child Fam Psychol Rev. (2017) 20:391–402. 10.1007/s10567-017-0237-2 [DOI] [PubMed] [Google Scholar]
- 15.Tarver J, Palmer M, Webb S, Scott S, Slonims V, Simonoff E, et al. Child and parent outcomes following parent interventions for child emotional and behavioral problems in autism spectrum disorders: a systematic review and meta-analysis. Autism. (2019) 23:1630–44. 10.1177/1362361319830042 [DOI] [PubMed] [Google Scholar]
- 16.Althoff CE, Dammann CP, Hope SJ, Ausderau KK. Parent-mediated interventions for children with autism spectrum disorder: a systematic review. Am J Occup Ther. (2019) 73:7303205010p1–p13. 10.5014/ajot.2019.030015 [DOI] [PubMed] [Google Scholar]
- 17.Rutherford M, Singh-Roy A, Rush R, McCartney D, O'Hare A, Forsyth K. Parent focused interventions for older children or adults with ASD and parent wellbeing outcomes: a systematic review with meta-analysis. Res Autism Spectr Disord. (2019) 68:101450. 10.1016/j.rasd.2019.101450 [DOI] [Google Scholar]
- 18.Higgins JP, Altman DG, Gotzsche PC, Juni P, Moher D, Oxman AD, et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ. (2011) 343:d5928. 10.1136/bmj.d5928 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. (2021) 372:n71. 10.1136/bmj.n71 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Schardt C, Adams MB, Owens T, Keitz S, Fontelo P. Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Med Inform Decis Mak. (2007) 7:16. 10.1186/1472-6947-7-16 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Guyatt GH, Oxman AD, Schunemann HJ, Tugwell P, Knottnerus A. GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology. J Clin Epidemiol. (2011) 64:380–2. 10.1016/j.jclinepi.2010.09.011 [DOI] [PubMed] [Google Scholar]
- 22.Sundhedsstyrelsen . Behandling af autismespektrumforstyrrelser hos børn og unge, National klinisk retningslinje (2021). Available online at: https://www.sst.dk/da/Udgivelser/2021/NKR-behandling-af-autismespektrumforstyrrelser (accessed October 29, 2021).
- 23.Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. (2002) 21:1539–58. 10.1002/sim.1186 [DOI] [PubMed] [Google Scholar]
- 24.Cochrane . Review Manager 5. (2020). Available online at: https://training.cochrane.org/online-learning/core-software-cochrane-reviews/revman [Google Scholar]
- 25.Aldred C, Green J, Adams C. A new social communication intervention for children with autism: pilot randomised controlled treatment study suggesting effectiveness. J Child Psychol Psychiatr. (2004) 45:1420–30. 10.1111/j.1469-7610.2004.00338.x [DOI] [PubMed] [Google Scholar]
- 26.Aman MG, McDougle CJ, Scahill L, Handen B, Arnold LE, Johnson C, et al. Medication and parent training in children with pervasive developmental disorders and serious behavior problems: results from a randomized clinical trial. J Am Acad Child Adolesc Psychiatry. (2009) 48:1143–54. 10.1097/CHI.0b013e3181bfd669 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Bearss K, Johnson C, Smith T, Lecavalier L, Swiezy N, Aman M, et al. Effect of parent training vs. parent education on behavioral problems in children with autism spectrum disorder: a randomized clinical trial. J Am Med Assoc. (2015) 313:1524–33. 10.1001/jama.2015.3150 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Carter AS, Messinger DS, Stone WL, Celimli S, Nahmias AS, Yoder P, et al. randomized controlled trial of Hanen's 'More Than Words' in toddlers with early autism symptoms. J Child Psychol Psychiatry. (2011) 52:741–52. 10.1111/j.1469-7610.2011.02395.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Dawson G, Rogers S, Munson J, Smith M, Winter J, Greenson J, et al. Randomized, controlled trial of an intervention for toddlers with autism: the early start denver model. Pediatrics. (2009) 125:e17–23. 10.1542/peds.2009-0958 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Drew A, Baird G, Baron-Cohen S, Cox A, Slonims V, Wheelwright S, et al. A pilot randomised control trial of a parent training intervention for pre-school children with autism. Preliminary findings and methodological challenges. Eur Child Adolesc Psychiatry. (2002) 11:266–72. 10.1007/s00787-002-0299-6 [DOI] [PubMed] [Google Scholar]
- 31.Ginn NC, Clionsky LN, Eyberg SM, Warner-Metzger C, Abner JP. Child-directed interaction training for young children with autism spectrum disorders: parent and child outcomes. J Clin Child Adolesc Psychol. (2017) 46:101–9. 10.1080/15374416.2015.1015135 [DOI] [PubMed] [Google Scholar]
- 32.Green J, Charman T, McConachie H, Aldred C, Slonims V, Howlin P, et al. Parent-mediated communication-focused treatment in children with autism (PACT): a randomised controlled trial. Lancet. (2010) 375:2152–60. 10.1016/S0140-6736(10)60587-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Hardan AY, Gengoux GW, Berquist KL, Libove RA, Ardel CM, Phillips J, et al. A randomized controlled trial of Pivotal Response Treatment Group for parents of children with autism. J Child Psychol Psychiatry. (2015) 56:884–92. 10.1111/jcpp.12354 [DOI] [PubMed] [Google Scholar]
- 34.Iadarola S, Levato L, Harrison B, Smith T, Lecavalier L, Johnson C, et al. Teaching Parents Behavioral Strategies for Autism Spectrum Disorder (ASD): effects on Stress, Strain, and Competence. J Autism Dev Disord. (2018) 48:1031–40. 10.1007/s10803-017-3339-2 [DOI] [PubMed] [Google Scholar]
- 35.Jocelyn LJ, Casiro OG, Beattie D, Bow J, Kneisz J. Treatment of children with autism: a randomized controlled trial to evaluate a caregiver-based intervention program in community day-care centers. J Dev Behav Pediatr. (1998) 19:326–34. 10.1097/00004703-199810000-00002 [DOI] [PubMed] [Google Scholar]
- 36.Kasari C, Gulsrud AC, Wong C, Kwon S, Locke J. Randomized controlled caregiver mediated joint engagement intervention for toddlers with autism. J Autism Dev Disord. (2010) 40:1045–56. 10.1007/s10803-010-0955-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Kuravackel GM, Ruble LA, Reese RJ, Ables AP, Rodgers AD, Toland MD, et al. Hope: evaluating the effectiveness of a parent training and support program for children with ASD. J Autism Dev Disord. (2018) 48:404–16. 10.1007/s10803-017-3333-8 [DOI] [PubMed] [Google Scholar]
- 38.Lecavalier L, Pan X, Smith T, Handen BL, Arnold LE, Silverman L, et al. Parent stress in a randomized clinical trial of atomoxetine and parent training for children with autism spectrum disorder. J Autism Dev Disord. (2018) 48:980–7. 10.1007/s10803-017-3345-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Pajareya K, Nopmaneejumruslers K. A pilot randomized controlled trial of DIR/Floortime parent training intervention for pre-school children with autistic spectrum disorders. Autism. (2011) 15:563–77. 10.1177/1362361310386502 [DOI] [PubMed] [Google Scholar]
- 40.Reitzel J, Summers J, Lorv B, Szatmari P, Zwaigenbaum L, Georgiades S, et al. Pilot randomized controlled trial of a Functional Behavior Skills Training program for young children with Autism Spectrum Disorder who have significant early learning skill impairments and their families. Res Autism Spectr Disord. (2013) 7:1418–32. 10.1016/j.rasd.2013.07.025 [DOI] [Google Scholar]
- 41.Siller M, Hutman T, Sigman M. A parent-mediated intervention to increase responsive parental behaviors and child communication in children with ASD: a randomized clinical trial. J Autism Dev Disord. (2013) 43:540–55. 10.1007/s10803-012-1584-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Solomon M, Ono M, Timmer S, Goodlin-Jones B. The effectiveness of parent-child interaction therapy for families of children on the autism spectrum. J Autism Dev Disord. (2008) 38:1767–76. 10.1007/s10803-008-0567-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Solomon R, Van Egeren LA, Mahoney G, Quon Huber MS, Zimmerman P. PLAY project home consultation intervention program for young children with autism spectrum disorders: a randomized controlled trial. J Dev Behav Pediatr. (2014) 35:475–85. 10.1097/DBP.0000000000000096 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Tonge B, Brereton A, Kiomall M, Mackinnon A, Rinehart NJ. A randomised group comparison controlled trial of 'preschoolers with autism': a parent education and skills training intervention for young children with autistic disorder. Autism. (2014) 18:166–77. 10.1177/1362361312458186 [DOI] [PubMed] [Google Scholar]
- 45.Whittingham K, Sofronoff K, Sheffield J, Sanders MR. Stepping Stones Triple P: an RCT of a parenting program with parents of a child diagnosed with an autism spectrum disorder. J Abnorm Child Psychol. (2009) 37:469–80. 10.1007/s10802-008-9285-x [DOI] [PubMed] [Google Scholar]
- 46.Wong VC, Kwan QK. Randomized controlled trial for early intervention for autism: a pilot study of the Autism 1-2-3 Project. J Autism Dev Disord. (2010) 40:677–88. 10.1007/s10803-009-0916-z [DOI] [PubMed] [Google Scholar]
- 47.Barrett AC, Vernon TW, McGarry ES, Holden AN, Bradshaw J, Ko JA, et al. Social responsiveness and language use associated with an enhanced PRT approach for young children with ASD: Results from a pilot RCT of the PRISM model. Res Autism Spectr Disord. (2020) 71:101497. 10.1016/j.rasd.2019.101497 [DOI] [Google Scholar]
- 48.Brian JA, Smith IM, Zwaigenbaum L, Bryson SE. Cross-site randomized control trial of the Social ABCs caregiver-mediated intervention for toddlers with autism spectrum disorder. Autism Res. (2017) 10:1700–11. 10.1002/aur.1818 [DOI] [PubMed] [Google Scholar]
- 49.Gengoux GW, Abrams DA, Schuck R, Millan ME, Libove R, Ardel CM, et al. A pivotal response treatment package for children with autism spectrum disorder: an RCT. Pediatrics. (2019) 144:178. 10.1542/peds.2019-0178 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Valeri G, Casula L, Menghini D, Amendola FA, Napoli E, Pasqualetti P, et al. Cooperative parent-mediated therapy for Italian preschool children with autism spectrum disorder: a randomized controlled trial. Eur Child Adolesc Psychiatry. (2020) 29:935–46. 10.1007/s00787-019-01395-5 [DOI] [PubMed] [Google Scholar]
- 51.Williams ME, Hastings RP, Hutchings J. The incredible years autism spectrum and language delays parent program: a pragmatic, feasibility randomized controlled trial. Autism Res. (2020) 13:1011–22. 10.1002/aur.2265 [DOI] [PubMed] [Google Scholar]
- 52.Vernon TW, Holden AN, Barrett AC, Bradshaw J, Ko JA, McGarry ES, et al. A pilot randomized clinical trial of an enhanced pivotal response treatment approach for young children with autism: the PRISM model. J Autism Dev Disord. (2019) 49:2358–73. 10.1007/s10803-019-03909-1 [DOI] [PubMed] [Google Scholar]
- 53.Schertz HH, Odom SL, Baggett KM, Sideris JH. Effects of joint attention mediated learning for toddlers with autism spectrum disorders: an initial randomized controlled study. Early Child Res Q. (2013) 28:249–58. 10.1016/j.ecresq.2012.06.006 [DOI] [Google Scholar]
- 54.Schertz HH, Odom SL, Baggett KM, Sideris JH. Mediating parent learning to promote social communication for toddlers with autism: effects from a randomized controlled trial. J Autism Dev Disord. (2018) 48:853–67. 10.1007/s10803-017-3386-8 [DOI] [PubMed] [Google Scholar]
- 55.Beaudoin AJ, Sebire G, Couture M. Parent-mediated intervention tends to improve parent-child engagement, and behavioral outcomes of toddlers with ASD-positive screening: a randomized crossover trial. Res Autism Spectr Disord. (2019) 66:101416. 10.1016/j.rasd.2019.101416 [DOI] [Google Scholar]
- 56.Bradshaw J, Bearss K, McCracken C, Smith T, Johnson C, Lecavalier L, et al. Parent education for young children with autism and disruptive behavior: response to active control treatment. J Clin Child Adolesc Psychol. (2018) 47:S445–S55. 10.1080/15374416.2017.1381913 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Brignell A, Chenausky KV, Song H, Zhu J, Suo C, Morgan AT. Communication interventions for autism spectrum disorder in minimally verbal children. Cochrane Database Syst Rev. (2018) 11:CD012324. 10.1002/14651858.CD012324.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Corti C, Pergolizzi F, Vanzin L, Cargasacchi G, Villa L, Pozzi M, et al. Acceptance and commitment therapy-oriented parent-training for parents of children with autism. J Child Fam Stud. (2018) 27:2887–900. 10.1007/s10826-018-1123-3 [DOI] [Google Scholar]
- 59.Dababnah S, Olson EM, Nichols HM. Feasibility of the incredible years parent program for preschool children on the autism spectrum in two US sites. Res Autism Spectr Disord. (2019) 57:120–31. 10.1016/j.rasd.2018.10.010 [DOI] [Google Scholar]
- 60.Freitag CM, Jensen K, Teufel K, Luh M, Todorova A, Lalk C, et al. Empirically based developmental and behavioral intervention programs targeting the core symptoms and language development in toddlers and preschool children with autism spectrum disorder. Z Kinder Jugendpsychiatr Psychother. (2020) 48:224–43. 10.1024/1422-4917/a000714 [DOI] [PubMed] [Google Scholar]
- 61.Gengoux GW, Schapp S, Burton S, Ardel CM, Libove RA, Baldi G, et al. Effects of a parent-implemented Developmental Reciprocity Treatment Program for children with autism spectrum disorder. Autism. (2019) 23:713–25. 10.1177/1362361318775538 [DOI] [PubMed] [Google Scholar]
- 62.Green J, Pickles A, Pasco G, Bedford R, Wan MW, Elsabbagh M, et al. Randomised trial of a parent-mediated intervention for infants at high risk for autism: longitudinal outcomes to age 3 years. J Child Psychol Psychiatry. (2017) 58:1330–40. 10.1111/jcpp.12728 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Heidlage JK, Cunningham JE, Kaiser AP, Trivette CM, Barton EE, Frey JR, et al. The effects of parent-implemented language interventions on child linguistic outcomes: a meta-analysis. Early Child Res Q. (2020) 50:6–23. 10.1016/j.ecresq.2018.12.006 [DOI] [Google Scholar]
- 64.Ho MH, Lin LY. Efficacy of parent-training programs for preschool children with autism spectrum disorder: a randomized controlled trial. Res Autism Spectr Disord. (2020) 71:101495. 10.1016/j.rasd.2019.101495 [DOI] [Google Scholar]
- 65.Lecavalier L, Smith T, Johnson C, Bearss K, Swiezy N, Aman MG, et al. Moderators of parent training for disruptive behaviors in young children with autism spectrum disorder. J Abnorm Child Psychol. (2017) 45:1235–45. 10.1007/s10802-016-0233-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66.Palmer M, Tarver J, Paris Perez J, Cawthorne T, Romeo R, Stringer D, et al. A novel group parenting intervention to reduce emotional and behavioural difficulties in young autistic children: protocol for the Autism Spectrum Treatment and Resilience pilot randomised controlled trial. BMJ Open. (2019) 9:e029959. 10.1136/bmjopen-2019-029959 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 67.Ruane A, Carr A, Moffat V, Finn T, Murphy A, O'Brien O, et al. A randomised controlled trial of the Group Stepping Stones Triple P training programme for parents of children with developmental disabilities. Clin Child Psychol Psychiatry. (2019) 24:728–53. 10.1177/1359104519827622 [DOI] [PubMed] [Google Scholar]
- 68.Whitehouse AJO, Varcin KJ, Alvares GA, Barbaro J, Bent C, Boutrus M, et al. Pre-emptive intervention versus treatment as usual for infants showing early behavioural risk signs of autism spectrum disorder: a single-blind, randomised controlled trial. Lancet Child Adolesc. (2019) 3:605–15. 10.1016/S2352-4642(19)30184-1 [DOI] [PubMed] [Google Scholar]
- 69.Zwaigenbaum L, Bauman ML, Choueiri R, Kasari C, Carter A, Granpeesheh D, et al. Early intervention for children with autism spectrum disorder under 3 years of age: recommendations for practice and research. Pediatrics. (2015) 136(Suppl.1):S60–81. 10.1542/peds.2014-3667E [DOI] [PMC free article] [PubMed] [Google Scholar]
- 70.Sparrow SS, Cicchetti DV, Balla D. Vineland Adaptive Behavior Scales: (Vineland II), Survey Interview Form/Caregiver Rating Form. Livonia, MN: Pearson Assessments; (2005). 10.1037/t15164-000 [DOI] [Google Scholar]
- 71.Rahman A, Divan G, Hamdani SU, Vajaratkar V, Taylor C, Leadbitter K, et al. Effectiveness of the parent-mediated intervention for children with autism spectrum disorder in south Asia in India and Pakistan (PASS): a randomised controlled trial. Lancet Psychiatry. (2016) 3:128–36. 10.1016/S2215-0366(15)00388-0 [DOI] [PubMed] [Google Scholar]
- 72.Mahoney G, Solomon R. Mechanism of developmental change in the PLAY project home consultation program: evidence from a randomized control trial. J Autism Dev Disord. (2016) 46:1860–71. 10.1007/s10803-016-2720-x [DOI] [PubMed] [Google Scholar]
- 73.Scahill L, McDougle CJ, Aman MG, Johnson C, Handen B, Bearss K, et al. Effects of risperidone and parent training on adaptive functioning in children with pervasive developmental disorders and serious behavioral problems. J Am Acad Child Adolesc Psychiatry. (2012) 51:136–46. 10.1016/j.jaac.2011.11.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.