The devil is in the details: Advancing our collective understanding of naturalistic developmental behavioral interventions

Abstract

Given the growing body of randomized trials examining various Naturalistic Developmental Behavioral Intervention (NDBI) approaches, a dialog has emerged exploring the overlap in strategies across NDBIs to create single measures that propose to capture core strategies across the interventions. This commentary will ask readers to consider the current state of the science, the potential value of looking not only for similarities but also for differences across approaches, and present five scientific next steps to advance our collective understanding of the NDBIs including: (a) operationalizing intervention strategies and outcomes, (b) expansion of the effectiveness evidence base and begin testing implementation strategies for individual NDBIs, (c) rigorous testing of core intervention components and the mechanism of each intervention, (d) personalization, and (e) supporting transparency with a priori trial registration.

Lay Summary

A group of interventions called Naturalistic Developmental Behavioral Interventions (NDBIs) that combine principles of learning and what is understood about children’s development are showing up more both in scientific research and in community practice. A number of approaches that vary in philosophy and strategies fall under the umbrella of “NDBI”. There is a growing discussion about what is the same about these interventions, yet less discussion about what might be different. This commentary will present five next steps to help us understand the details of the individual interventions so that then, we can better understand what is similar and what is different among the approaches to guide individual selection.

INTRODUCTION

Emerging from a workgroup of intervention scientists (Schreibman et al., 2015), a group of interventions that represent the merger of applied behavior analysis and developmental sciences were defined as Naturalistic Developmental Behavioral Interventions (NDBIs: Schreibman et al., 2015). The NDBIs can be appealing to those looking for interventions informed by applied behavior analysis while occurring within naturalistic settings, including child choice, and building learning opportunities into social, play, and/or daily routines, through change agents in the child’s life. Broadly, NDBIs support skill growth using contingent natural reinforcement (including imitation), scaffolding learning through environmental supports, modeling as well as additional prompting, and providing activities/materials that are interesting for the child to create a context for interaction (Schreibman et al., 2015). A wide variety of approaches can fit under this umbrella, each with their own interpretation and application of the naturalistic, behavioral, and developmental components, and potentially targeting different developmental outcomes. Yet together, the NDBIs have amassed a base of efficacy trial data, demonstrating outcomes primarily for children’s social communication skills (e.g., meta-analysis: Tiede & Walton, 2019). Given the growing body of randomized trials examining various NDBI approaches, a dialog has emerged exploring the overlap in strategies across the NDBIs to create single measures to capture the proposed core strategies across the interventions (e.g., Edmunds et al., 2022; Frost et al., 2020; Vibert et al., 2020) as well as their outcomes (e.g., Klein et al., 2021 use of BOSCC). Although parallels in the delivery context (e.g., families’ homes, interactions including toys and home routines), broad approach (e.g., contingent responding), and overlap in strategy terminology create superficial similarities across the NDBIs (and arguably all early intervention approaches), notable differences exist in the primary child skills they aim to support and their implementation with children and families. Further, the evidence base among the approaches varies greatly where some approaches demonstrate gains for children across multiple randomized trials while others have failed to replicate or have been tested only in small pilot studies that have not been subjected to fully powered rigorous RCTs. Yet, such differences are rarely included in this dialog. While precision public health and precision medicine embrace heterogeneity to explore the personalization of interventions (Kosorok & Laber, 2019), we risk missing this opportunity to advance the personalization of behavioral interventions to the strengths and needs of children with a lack of attention to these differences.

To engage in efficient and accurate examination of the similarities and differences across NDBI approaches, it is necessary to empirically and rigorously study their clinical strategies, mechanisms of action, and the conditions under which efficacy and effectiveness have been demonstrated. Clinical hypotheses are necessary to inform this work but not sufficient. Therefore, within this commentary, I will comment on the current state of intervention science specific to the NDBIs and present five scientific next steps to advance understanding of these approaches.

FIVE SCIENTIFIC STEPS TO ADVANCE OUR UNDERSTANDING OF NDBIS

Clearly operationalize intervention strategies and target outcomes

Terms have emerged in the NDBI literature to describe intervention strategies that can be found across approaches. However, what is less clear is when a single term may capture different strategies across approaches or when the strategies captured by the same term are implemented differently across approaches. For example, the term “environmental arrangement” can refer to an opportunity to elicit requesting communication (e.g., holding up two items out of the child’s reach) or it can refer to the selection of developmentally appropriate materials and how they are placed in the play environment, among many other strategies. As another example, when we consider a strategy, such as “following the child’s lead”, this strategy can be implemented on a spectrum from following everything the child does without bounds to imitation of specific actions or noticing a child’s choice of items. If we assume similarities at face value based on terminology alone, we risk misunderstanding where there is true overlap across approaches and where there may be meaningful differences. A shared measure of NDBI strategies is not possible without first communicating how strategies are defined and applied within each approach.

Clearly defined, operationalized intervention and implementation strategies are necessary to also understand if and what adaptations are being made when interventions are moved into new contexts (e.g., from clinic to classroom) or transferred to other implementers (e.g., from clinician to caregiver, clinician to educator). While some adaptations may not fundamentally alter the intervention, others may. If expectations or strategies are changing in these contexts, it is critical to be able to describe the adaptations and then test the modified intervention for child outcomes. Although word limits in peer-reviewed journals can restrict the amount of detail that can be provided about intervention strategies, article appendices, intervention manuals, and free web materials could be methods to provide this information.

By clearly defining the component strategies in each intervention, the field can move toward systematic exploration of similarities and differences in approaches. For example, strategies, such as the Distillation and Matching Model (DMM: Chorpita et al., 2005) have emerged in mental health. The DMM includes methodology with the level of analysis at the component parts of manualized interventions. This allows for exploration of frequency patterns in strategies to inform the empirical construction of a distillation tree that organizes the components in accordance with a priori selected variables. The DMM allows for the mapping of practices with positive outcome data that can help to understand relations among practices and context variables and facilitate hypothesis generation.

Expand the effectiveness evidence base and test implementation strategies

The path of translational research has been framed as a subway line (Lane-Fall et al., 2019) where the first stop for a practice of interest (POI) is the test of initial efficacy (whether an intervention works under optimal, highly controlled conditions: Marchand et al., 2011). Once efficacy is demonstrated, the POI continues down the path to effectiveness testing, where it is tested under real world conditions with participants and practitioners in authentic settings (Marchand et al., 2011). Several NDBIs have passed the first stop by demonstrating efficacy via one or more randomized controlled trials where research/university-based staff deliver the intervention directly to the child or coaching to the caregiver (e.g., Pivotal Response Teaching (PRT): Hardan et al., 2015; Reciprocal Imitation Training: Ingersoll, 2010, Joint Attention, Symbolic Play, Engagement and Regulation (JASPER): Kasari et al., 2006, Early Start Denver Model: Rogers et al., 2012; Social ABCs: Brian et al., 2017). Although not addressed explicitly by Lane-Fall et al. (2019), replication of a single efficacy study could also be considered a critical aspect of this first stop and if so, few practices have passed this point.

Even so, several approaches have moved to the second station focused on interventions delivered in real world settings. Single group pre-post designs continue to be presented in this space. However, this design does not account for the effects of natural development and therefore, does not provide the experimental control needed to assess the influence of the intervention on the outcome. Randomized effectiveness trials can be designed to examine whether there are significant gains for children receiving the POI versus a comparison in authentic educational settings. Such trials have included community educators who deliver approaches adapted for school classrooms (e.g., JASPER: Panganiban et al., 2022; Social Communication Emotion Regulation Transactional Supports: Morgan et al., 2018), practitioners in childcare settings (e.g., Early Achievements: Feuerstein & Landa, 2020) and community practitioners who coach caregivers to deliver the intervention (e.g., JASPER: Shire et al., 2022). Not all attempts to transition a practice or adapt a practice to transition into the community have demonstrated effects for children. Although implementers (e.g., caregivers, educators) may show some gains in their strategy use, children may not show significant gains compared to children in active control conditions. For example, in the transition of the Early Start Denver Model (ESDM) to a caregiver mediated model, significant effects for children found in the clinician mediated model (Dawson et al., 2010) were not found in the caregiver-mediated form of the intervention (Rogers et al., 2012). Further, in the classroom adaptation of Pivotal Response Teaching (C-PRT: Stahmer et al., 2022), few significant differences between students in the C-PRT classrooms and those in the control classrooms were found. Teachers in C-PRT classrooms reported fewer approach/ withdrawal problems while student engagement was not observed to change at exit, but small significant differences (~3% difference) were observed at follow up in favor of C-PRT classrooms. Understanding the determinants of the community implementation context may help to inform the identification of strategies to support the implementers that could be tested in future studies.

Although there is a call for wider implementation of NDBIs (D’Agostino et al., 2023) it is critical to understand the conditions under which the interventions can be delivered with high fidelity and lead to significant change for the children before limited human and financial resources in community settings are expended on intervention deployment. Testing of implementation outcomes (Proctor et al., 2011) can help us understand the degree to which a POI functions in real world settings. For example, is the POI accepted, delivered with fidelity, and sustained over time? If not, this may inform implementation strategies that could be tested to support implementation outcomes. Continuing to the third stop on the subway line, hybrid effectiveness-implementation trials (Curran et al., 2012) are one promising approach to engage in rigorous randomized trials with control/comparison groups that explicitly have the dual focus (a prior aims and outcomes) to examine impact of intervention on child and family outcomes (effectiveness) while also examining implementation strategies and implementation outcomes in real-world systems. Curran et al. (2012) describe three types of hybrid trials where the primary focus can be primarily on implementation (Type III), both implementation and effectiveness (Type II) or focus primarily on effectiveness (Type I). Hybrid trials are only beginning to emerge specific to autism and intervention. Recent hybrid examples include a randomized waitlist control examining the effectiveness of classroom PRT while monitoring teachers’ implementation of the intervention (Stahmer et al., 2022). Although between group student effectiveness, outcomes were limited to teacher reported approach and withdrawal problems, greater CPRT implementation fidelity was associated with student learning outcomes. Hybrid designs have also been used to examine the implementation and effectiveness of JASPER in center-based toddler classrooms demonstrating significant gains over classroom as usual for children’s joint engagement and social communication as well as significant gains for teaching assistants’ implementation fidelity (Shire et al., 2017). The few trials of hybrid implementation models highlight that there is still much to learn about the transfer of NDBIs to authentic educational settings as well as the design and testing of implementation strategies to support the efficient and effective deployment of NDBIs within real-world systems.

One intervention approach outside of the NDBIs that has moved down the subway line from initial implementation through to implementation in low resource community settings is the Pediatric Autism and Communication Therapy (PACT) approach. PACT began with demonstration of initial efficacy in a large, randomized trial (Green et al., 2010) followed by testing of candidate mediators including parental synchrony and child initiations on children’s Autism Diagnostic Observation Schedule scores (Pickles et al., 2015) and long-term effects of the intervention (Pickles et al., 2016). PACT has also been adapted for South Asian communities in Pakistan and India and tested in a randomized trial demonstrating significant gains favoring PASS over control in parental synchrony and child’s communicative initiations (Rahman et al., 2016). Further, by exploring facilitators and barriers to caregiver implementation (Carruthers et al., 2023), this work sets the foundation to test implementation strategies to support caregivers’ learning. This progression of the development and understanding of the mechanism and effects for children after adaptation of the approach is an example of the path that can provide necessary empirical information about each of the NDBI approaches.

Broaden empirical understanding of active ingredients and mechanisms within each NDBI approach

Expert reviews have long called for attention to intervention mechanism as well as examination of intervention components (e.g., Green & Garg, 2018; Kasari, 2002; Kasari & Smith, 2016; Smith & Iadarola, 2015). These are questions that can be empirically tested using mediation analyses, yet recent literature proposes measures and models that work primarily on information gathering and synthesis (e.g., literature review, interviews) to generate hypotheses on which components are central to the model. Clinical and developmental hypotheses will inform the conceptual models needed to identify candidate mediators. Candidate mediators may include “active ingredients” or components of the intervention approach that are delivered by the adult implementer and children’s skills that act as developmental mechanisms that build the foundation for further developmental gains. Candidate mediators must be empirically tested to understand if our clinical hypotheses are supported by data from diverse participant samples from the communities in which the interventions will be used. Yet, to date candidate mediators have been tested in only two approaches. First, in a caregiver-mediated intervention model, Yoder et al. (2021) tested a hypothesis that caregivers’ use of ImPACT intervention strategies would mediate children’s proximal outcomes including motor imitation and intentional communication. Caregivers’ strategy use was associated with gains in a composite variable designed to capture children’s intentional communication. Second, in a clinician-mediated model, Shih et al. (2021) tested time children were jointly engaged (child coordinates a partner and a shared activity: Adamson et al., 2009) as the mediator of the intervention effect on the proximal outcome initiations of joint attention (IJA: spontaneous use of gaze, gestures, and/or words to socially share). Time jointly engaged significantly mediated the effect of intervention on IJA and IJA then predicted improvements in standardized language scores. These two mediation models test different mechanisms (e.g., sequence of parents’ implementation to children’s motor imitation/intentional communication versus social engagement) in two different intervention approaches and it is possible that additional mechanisms will be tested for other NDBIs. When the proposed drivers of the effects of the interventions differ, it is logical that the corresponding intervention strategies as well as the anticipated proximal and distal outcomes may also differ.

Naturalistic developmental behavioral intervention approaches often include a complex set of strategies that require considerable resources to deliver with high quality. A component of the rationale to identify active ingredients of complex intervention approaches is to explore whether there are strategies that are driving the impact of the intervention on children’s outcomes. This has implications for community implementation if the number of strategies and complexity can be reduced while still demonstrating gains for children. Given that special education and community autism services report national shortages with high rates of staff turnover, the goal to reduce training time and ease implementation is important. Yet, given those same barriers, compounded by pandemic learning losses that are yet to be fully understood, we cannot afford to ask for these limited and critical public resources without empirical data to support our recommendations. To date, published empirical examples that test for active ingredients of NDBI approaches are still too few. For example, strategies applied by parents in the caregiver-mediated JASPER model have been examined by Gulsrud and colleagues (Gulsrud et al., 2016). Composite variables addressing environmental strategies, prompting, communication and one referred to as “mirrored pacing” were tested as candidate mediators of intervention on children’s joint engagement. Only mirrored pacing (including imitation of children’s play acts, the contingent and immediate timing of parents’ action, and its placement in the child’s line of sight), significantly mediated the relationship between intervention and children’s joint engagement. Second, Pellecchia et al. (2015) examined components of the Strategies for Teaching based on Autism Research (STAR) intervention, which includes the NDBI, PRT as well as Discrete Trial Teaching (DTT) and Functional Routines. In this analysis, these three interventions that each includes numerous strategies are considered the three “active ingredients” and only PRT fidelity was associated with children’s cognitive gains in STAR. This is a higher-level view of ingredients as full manualized approaches versus individual or groups of strategies within one of these approaches. However, as noted by Chorpita et al (Chorpita & Daleiden, 2009), by making the manualized intervention the unit of analysis, we do not get information about the contributions or assembly of the specific practices within an intervention, a process necessary to better understand intervention ingredients.

Candidate mediators do not function in isolation, rather, intervention strategies may be layered or contingent upon one another and children’s skills take place within dyadic interaction as well as within a sequence of development. An unanswered question is how interactions among mediators may contribute to intervention effects. Are there instances where it is the combination of strategies that produces the effect? Are additive effects found when strategies are used in combination or sequence? Does the interplay among adult strategies and children’s behavior (e.g., gaining time engaged) lead to further effects? Greater understanding of what is driving change within each approach will support evaluation of similarities and differences across approaches.

Personalization

Understanding the differences among NDBIs in their mechanism, strategies, and treatment effects can support the personalization of intervention programs. No single intervention will fit all children’s strengths and needs or even a single child’s needs across all stages of development as needs shift over time. Although baseline moderators of intervention outcome have been examined including age and developmental level (e.g., review of intervention moderators: Klinger et al., 2021), it is not yet well understood what characteristics may help one select an intervention approach. Adaptive Interventions (AIs) allow for the sequencing of interventions through a prespecified set of decision rules to guide if, how, when, and based on what measures, intervention options are offered over a period of time (Almirall & Chronis-Tuscano, 2016). Experimental designs (e.g., Sequential Multiple Assignment Randomized Trials) can support the development of AIs to better understand when and for whom intervention augmentations may be required (Almirall & Chronis-Tuscano, 2016) and are beginning to be applied (e.g., Kasari et al., 2014). Educators face critical questions daily about when and how to make changes to a child’s program of intervention, yet there is little empirically derived guidance to aid practitioners in making these decisions. Such guidance has the potential to make efficient use of limited human and financial resources while providing tailored high-quality supports to those who need them, when they need them. Systematic and empirical a priori examination of intervention augmentations and intervention sequences is a necessary step to advance autism intervention science.

Documentation of target outcomes: A priori trial registration

A priori registration of clinical trials as well as publication of design papers allow for greater transparency in science (Benning et al., 2019) including documentation of both primary and secondary outcomes as well as the stage of trial (e.g., pilot or feasibility, full scale randomized trial). Clearly defining when a trial is designed as a pilot will support grant reviewers, manuscript reviewers, and broader readership to understand the goals and scope of the project. Pilot studies can include small scale versions of a larger study that are focused on understanding foundational components including acceptability and feasibility to prepare for a full-scale trial (Almirall et al., 2012). Pilot studies can support the identification of facilitators and barriers as well as potential adaptations to support the fit of the intervention to the local context and plan for sustainability in real-world settings. However, to date, many pilot studies do not progress to fully powered studies or fail to demonstrate reproducible effects. Once piloted, testing the approach in a fully power efficacy or effectiveness trial will advance both the internal and external validity of the approach.

CONCLUSIONS

Generalized assumptions of similarities without empirical testing can generate practice recommendations that are too broad and nonspecific, taking clinicians further away from the ability to personalize intervention programming by matching specific child targets to models that when implemented in full, have demonstrated efficacy. Before we can effectively come together as intervention scientists to examine the similarities and differences among approaches, there is a critical need for rigorous scientific efforts to understand within each approach as well as for significant adaptations of a given approach: if and when intervention effects are present for which child outcomes, the mechanism of those effects, the core components (aka active ingredients), and for whom and in what combination or sequence the approach is beneficial. This work has only begun to emerge in the literature and only for some of the NDBI approaches. Given the vast range of strengths and needs of children with ASD, we know the answer is not this simple. Therefore, attention to what works for whom and when is critical. To move toward the systematic personalization of developmentally appropriate early intervention programming that respects and matches the strengths and needs of an individual child, it is imperative that we understand if, when, and how specific interventions are leading to change. Without this information, as a field we run the risk of making premature recommendations for clinical practice that fail to attend to the similarities, but perhaps more importantly, the differences between interventions.

DATA AVAILABILITY STATEMENT

Data sharing is not applicable to this article as no new data were created or analyzed in this study.