Training behavioural therapists in presession pairing skills to evaluate the impact on children’s life skill acquisition rates

Abstract

Presession pairing is a well-documented behaviour analytic practice that supports the development of a positive instructional environment and is associated with reduced rates of challenging behaviour among children with developmental disabilities. However, there is limited research evaluating the impact of presession pairing on children’s skill acquisition rates. Therefore, in the current study, four behavioural therapists were systematically trained in an established presession pairing protocol and using a multiple baseline across participants design, the impact of this training on life skill acquisition rates among a group of four children with Autism Spectrum Disorder was evaluated. A robust improvement rate difference was calculated for each child and a robust omnibus improvement rate difference showed that the impact of training therapists in the presession pairing protocol had a very small effect on the children’s life skill acquisition rates. These findings were discussed in the context of the importance of reporting null findings and designing replication studies that provide an understanding of why an effect was not demonstrated, in order to develop and refine clinical practice.

Quality of life is defined as an individual’s “perception of their position in life in the context of the culture and value systems in which they live and in relation to their goals, expectations, standards and concerns” (The World Health Organization Quality of Life Group 1995, p.1405). Level of adaptive functioning has been shown to be a significant predictor of quality of life, with individuals who display lower levels of adaptive skills achieving less personal outcomes than their peers with higher functioning levels (Maes et al. 2007, Schalock et al. 1994). However, when personalised supports, which foster growth, education, interests and well-being, are in place for an extended period, adaptive skills and quality of life can subsequently improve (Luckasson et al. 2003).

Autism Spectrum Disorder (ASD) is a complex neurobiological disorder that produces persistent deficits in social communication and interaction that can lead to clinically significant impairment in occupational function across multiple contexts (American Psychiatric Association 2013). In addition, individuals with ASD are frequently diagnosed with an intellectual disability (American Association on Intellectual and Developmental Disabilities, n.d.; Matson and Shoemaker 2009), further increasing the risk that they will experience limitations in adaptive behaviour, including everyday practical life skills (American Association on Intellectual and Developmental Disabilities, n.d.).

However, ASD can now be detected as early as 14 months (Landa et al. 2013), with high levels of diagnostic stability by 18 months of age (Ozonoff et al. 2015) and recent systematic reviews and meta-analyses have shown that early intervention can have a positive impact on the developmental trajectory of a young child with ASD (Landa 2018). Early intervention aims to counteract the atypical attention, engagement and sensory/motor processing patterns that can compromise the quality and quantity of learning opportunities experienced by children with ASD (Landa 2018, Lane et al. 2010, Sacrey et al. 2013, Vivanti et al. 2013, Vivanti et al. 2017). Early intervention seeks to take advantage of the brain’s experience-dependent neuroplasticity to promote learning through a child’s interactions with their environment (Kolb and Gibb 2011, Landa 2018).

Early Intensive Behavioural Intervention (EIBI) is an evidence-based approach to early intervention that relies on the delivery of principles of applied behaviour analysis (ABA) (National Autism Center 2015). EIBI is based on the assumption that the quality and quantity of learning opportunities made available to children with ASD can be significantly enhanced through operant conditioning strategies (i.e. associating responses with particular consequences) (Smith 2011). As such, this early intervention approach typically involves presenting structured trials during individual or small group sessions. Instructions are delivered using adult-selected materials and tasks. Trials are characterised by instruction-response-consequence chains, and learning is supported through well-defined prompt hierarchies, consistent delivery of precisely defined consequences and other applied behaviour analytic strategies (Eldevik et al. 2009, National Autism Center 2015, Smith 2011). In a systematic review of the literature, Smith and Iadarola (2015) classified EIBI as a well-established intervention approach for promoting adaptive behaviour gains in young children with ASD.

While EIBI has been shown to be effective in positively supporting the development of young children with ASD, several characteristics of effective EIBI programmes, including intervention intensity, can inadvertently establish aversive intervention conditions (Lugo et al. 2017). A common feature across EIBI programmes is that they are delivered year round, with 20–30 h of structured sessions per week. In addition, these structured sessions are often supplemented with informal instruction during a child’s remaining waking hours (Eldevik et al. 2009). The high demand requirements associated with this level of programmatic input can lead to off-task behaviour among some children (Carbone et al. 2007), which has the potential to undermine the effectiveness of EIBI at a crucial stage of development for a young child with ASD (Lugo et al. 2017).

Therefore, when providing EIBI at the intensity that is required for effectiveness (Eldevik et al. 2009), the delivery of an instruction, the presence of the teacher or work materials, or the request to transition to the instructional setting may signal worsening conditions in the child’s environment and consequently lower levels of task engagement (Carbone et al. 2007). Michael (2000) proposed that the nature of instruction could be modified to improve the instructional session “to the point at which it may not function as a demand but rather as an opportunity” (p.409) to access positive experiences. Subsequent research has discussed the benefits of associating or pairing the teaching environment, teaching personnel and task materials with positive reinforcement and an improving set of conditions. These benefits include lower levels of engagement in disruptive and off-task behaviour by children during instructional sessions (e.g. Carbone et al. 2007. Kelly et al. 2015, Smith 2001). Presession pairing (PSP) is a well-documented practice used to achieve this type of positive association and typically involves engaging a child in preferred activities, imitating their actions and providing preferred items before delivering an instruction or demand (Kelly et al. 2015, Lugo et al. 2017, Smith 2001).

Although evidence supports the use of PSP to optimise conditions in the instructional context, many teaching personnel supporting children with ASD may not have the skills necessary to effectively and efficiently engage in PSP strategies (Lugo et al. 2017). As a result, Lugo et al. (2017) operationally defined behaviours that they proposed were essential to successful pairing with children with ASD. Seven skills, in total, were defined and included commenting on (describe) and praising appropriate play (praise), imitating the child’s play (imitate) and vocalisations (reflect), offering tangible items to the child (initiate), changing the function of a toy to create a new activity (create), and staying in close proximity to the child during the pairing period (proximity).

In a subsequent study, Lugo et al. (2019) tested the social validity of their PSP protocol. They compared the differential impact of PSP, free play or immediate onset of instruction on the challenging behaviour displayed by a 4-year old with ASD during intensive instruction. They also assessed preference for these three antecedent conditions. Although negative vocalisations were observed to decrease across all conditions, results indicated a relative preference for PSP across multiple therapists. However, caution must be exercised as the generalisability of the latter finding is limited; there was only one participant in the study. Shillingsburg et al. (2019) also employed the Lugo et al. (2017) PSP protocol to reduce the occurrence of problem behaviour among four children with ASD during intensive instruction. Results showed increased acceptance of therapist prompts without resistance following the introduction of the PSP procedures. However, the study did not evaluate the impact of PSP on the acquisition of the skills targeted during the intensive instruction sessions.

Previous studies have clearly established the beneficial impact of PSP, in terms of reducing levels of off-task behaviour associated with instruction delivery. However, the subsequent impact on skill development and skill acquisition rates has received relatively little attention in the literature. It is possible that the reduced engagement in off-task behaviour, which is linked to the delivery of PSP protocols, could result in improved skill acquisition rates for children with ASD, who are participating in EIBI. Therefore, the current study set out to train behavioural therapists in the Lugo et al. (2017) PSP protocol and directly evaluate the resulting impact on life skill acquisition rates among a group of children with ASD.

Method

Participants and setting

Four children (three boys, one girl), who were diagnosed with autism by an independent, qualified professional, participated in the study. All children attended an early intervention preschool setting and participant selection was based on formal observations that demonstrated limitations in the children’s life skills repertoires. Table 1 provides demographic information for each child, including name, ethnicity, age, diagnoses, skill strengths (Assessment of Basic Language and Learning Skills - ABLLS-R), areas for focused intervention (ABLLS-R) and target life skill for the current study.

Table 1.

Demographic information for child participants

Name	Gender	Ethnicity	Age	Diagnosis	Mode of Communication	Relative Strengths (ABLLS-R)	Areas for Focus (ABLLS-R)	Target Life Skill
Child 1	Male	Caucasian	4.1	Autism Spectrum Disorder (ASD) Global Developmental Disorder (GDD) - not specified	Vocal	Receptive understanding; Play skills	Expressive communication; Motor imitation	Washing hands
Child 2	Male	Caucasian	4.1	ASD Borderline Intellectual Delay	Picture Exchange	Receptive understanding; Play skills	Fine motor skills; Motor imitation	Putting on socks
Child 3	Female	Caucasian	3.1	ASD GDD - mild	Picture Exchange	Play skills; Expressive communication	Motor imitation; Visual performance	Washing hands
Child 4	Male	Caucasian	3.11	ASD	Picture Exchange	Play skills; Visual performance	Fine motor skills; Receptive understanding; Expressive communication	Putting on trousers

Four behavioural therapists employed in the early intervention setting from which the four children were recruited, also participated. Immediately prior to study commencement, all four therapists scored below 80% on the Lugo et al. (2017) PSP protocol (Table 2). At the time of the study, none of the therapists were Board Certified Behaviour Analysts (BCBA)®, Board Certified Assistant Behaviour Analysts (BCaBA)® or Registered Behaviour Technicians (RBT)®. One of the therapists had begun graduate level coursework in ABA as the study commenced.

Table 2.

Presession pairing checklist - Lugo et al. (2017)

Target Behaviours
1. Staff selects toys/games/materials that the child is interested in
2. Staff invites the child in play by sitting with them and engaging with the toys and games
3. Proximity – Stay within an arm's length of the child throughout the session
4. Praise appropriate play- Staff intermittently praises the specific action the child is engaged in/performing *
5. Staff member uses an enthusiastic tone when engaging with child
6. Staff member reflects vocalisations made by the child by repeating the word, sentence or vocalisations into the play (Ex. Child says “Oh no!” Staff repeats “Oh no!” and adds “Barbie fell down!” *
7. Staff member intermittently imitates the child’s play *
8. Staff intermittently describes the appropriate play skills of the child in sentences *
9. Initiate – Staff provides the child with new/additional toys/items/materials throughout the session that are utilized in the staff/child joint play
10. Staff members use materials available to create new forms/types of play throughout the pairing session (Ex. Staff uses a box to create a barn for the toy farm animals and encourages the child to put the animals in the barn) *

^*Must be displayed at least 4 times during 5-minute therapist-child PSP session.

All therapists had received standard induction training provided by the preschool prior to study recruitment. This training was compulsory for all employees in the host setting and was completed immediately following the commencement of employment. All participating therapists had received this training in the 3-month period prior to the study starting. The induction training included modules on reinforcement, systematic prompting, verbal behaviour and functional communication training, discrete triaI instruction (DTI), and task analysis and chaining procedures. All modules covered during induction were competency-based and delivered using a behaviour skills training (BST) format. Table 3 provides demographic information for each therapist, including name, age, years of ABA experience, type of ABA experience, highest level of educational attainment, whether they had engaged in graduate level ABA coursework, and their assigned child.

Table 3.

Demographic information for therapist participants

Name	Age	ABA experience	Type of ABA experience	Educational Attainment	ABA Coursework	Assigned Child
Therapist 1	24	1 year	Early Intervention	MSc in Psychology	ABA module during postgraduate studies	Child 1
Therapist 2	31	2 years	Early Intervention	BA in Sociology	ABA module during postgraduate studies	Child 2
Therapist 3	26	1 year	Early Intervention; Adult services	BA in Education	Currently undertaking a postgraduate course in ABA	Child 3
Therapist 4	24	2 years	Early intervention	BA in Education	None	Child 4

During all phases of the study, therapists conducted PSP sessions with their designated child in a room located in the preschool. The room contained at least one table and two chairs, as well as educational and play materials typically found in a preschool setting (e.g. puzzles, cause-and-effect toys, crayons, blocks). Only preferred and novel play items were placed on the table during training sessions. All life skill teaching sessions between the therapist and child took place in areas of the preschool, where the target life skills would naturally occur (e.g. bathroom, exit area from the classroom). These sessions were conducted individually with each child to minimise incidental learning. The experimenter conducted PSP training with each therapist in a room located in the preschool. Environmental arrangements during these training sessions were consistent with those present during therapist-child PSP sessions.

We obtained informed consent from all children and therapists in the study, and all procedures conducted in the study were in accordance with the Research and Ethics Policy of the host setting, university and the 1964 Helsinki Declaration and its later amendments.

Dependent variables and data collection

Each child received instruction on one target life skill for the duration of the study. Target life skills were task analysed, breaking these complex behaviours into chains of discrete responses (see Supplementary Material). Across all phases, data were collected on the number of independent steps completed and data were analysed based on the number of steps completed independently relative to the total number of steps. For a step to be scored as independent, it had to be completed, without prompts, within 5 s of the previous step being completed. The first step in the task analysis chain was scored as independent, if it was completed, without prompts, within 5 s of the initial instruction (e.g. ‘It’s time to wash your hands’).

Experimental design and data analysis

A concurrent multiple baseline across participants design was the primary method used to evaluate the impact of training behavioural therapists in the Lugo et al. (2017) PSP protocol on life skill acquisition rates for a group of four children with autism. In accordance with a concurrent multiple baseline design, phase 1 data was collected contemporaneously across all four participants. While visual analysis is currently the most recognised method of analysing single-case data, research has frequently shown that this approach is subject to inadequate interrater agreement (Lanovaz et al. 2020, Parker et al. 2009). In addition, the objective of this study was to assess changes to the children’s life skill acquisition rates, as a result of staff being trained in the Lugo et al. (2017) PSP protocol and this would be difficult to fully assess using visual analysis alone. Therefore, to address any potential concerns about the reliability, objectivity and interpretability of results, we decided to supplement visual analysis with an effect size statistic. We selected improvement rate difference (IRD) because it offered (a) accessible interpretation of the difference in improvement rates between phase 1 and phase 2; (b) straightforward calculations; (c) known sampling distribution, which allows for confidence intervals to be established; and (d) few data distribution assumptions (Parker et al. 2009).

Robust IRD, which has been used extensively in evidence-based medicine, is defined as the improvement rate (IR) of the treatment phase (i.e. phase 2) minus the improvement rate of the baseline phase (phase 1) (Parker et al. 2009, Parker et al. 2011). When calculating the robust form of IRD, it is recommended that the number of overlapping data points are split evenly across phases (Parker et al. 2011). In the current study, each of the four children acted as the individual units of analysis and a robust omnibus IRD was calculated to assess the difference in rate of improvement for the entire design. Robust omnibus IRDs reflect an aggregation of data points across participants rather than a mean, thereby providing an overall robust IRD as if data for each individual were part of one larger experiment. Hand calculations, based on visual analysis and verified using the ‘SingleCaseES’ package available in ‘r’ (https://www.r-project.org), were used to calculate a robust omnibus IRD for the overall study, as per Parker et al. (2009) and Parker et al. (2011).

Procedure

Pre-training (phase 1)

Each session started with the therapist pairing with their assigned child for five minutes. During phase 1, therapists were told to pair with their designated child but were not given any instructions or guidance on how to do this. Immediately following the 5-minute pairing session, the therapist and child walked to the area where the life skills teaching session would take place. The therapist positioned themselves within 1 metre of the child and instructed the child to complete the life skills task (e.g. ‘It’s time to wash your hands’). In accordance with backward chaining procedures, the child was fully prompted through all discrete responses in the chain until the target step was reached. At this point, the child was given the opportunity to independently complete this step and any subsequent steps (i.e. previously acquired steps).

If the child did not complete the target step and/or any previously acquired steps independently, these steps were taught using a least-to-most prompting protocol with the following hierarchy: (a) gesture; (b) light touch; (c) manual guidance at elbow; and (d) hand-over-hand. At the target step and any previously acquired steps, the child was given 5 s to respond independently. If there was no response, the therapist delivered the next most intrusive prompt at 3 s intervals until the step was completed. The target step at the start of each session was based on the child’s performance in the previous session. Any errors made on the target step or on previously acquired steps were immediately corrected using hand-over-hand guidance. Reinforcement, consisting of descriptive praise, was delivered immediately following prompted completion of the target step and any previously acquired steps. Reinforcement, consisting of descriptive praise and access to a preferred item, was delivered immediately following independent completion of the target step, along with any previously acquired steps.

We selected backward chaining as the method of teaching the target life skills because research has demonstrated that for learners with disabilities, it has a reduced probability of error compared to total task presentation (Walls et al. 1981). This is particularly important when instructing children with ASD, given the association between error-making and elevated levels of frustration (Smith et al. 1995, Smith 2001). In line with recommendations from Libby et al. (2008), we combined the backward chaining procedure with a least-to-most prompting protocol because all children in the study had previously demonstrated a positive skill acquisition history when least-to-most prompting was used in the context of DTI and communication training. In addition, all errors were immediately corrected using hand-over-hand guidance, as per Libby et al. (2008), to minimise the potential for frustration.

Presession pairing (PSP) training

Therapists were trained individually. The experimenter delivered PSP training, via a BST format, across three stages: 1) written and verbal instruction outlining the seven skills in the Lugo et al. (2017) protocol; 2) live modelling of the seven skills by the experimenter, rehearsal of these skills by the therapist with a confederate, and corrective feedback at the end of each rehearsal session; and 3) rehearsal of the seven skills by the therapist with their assigned child, and corrective feedback at the end of each rehearsal session. Each rehearsal session lasted 5 min, which was consistent with the duration of the PSP sessions in phase 1 and phase 2. Mastery criterion for stage 2 and stage 3 was 100% accuracy, on all seven skills outlined in the Lugo et al. (2017) protocol, for one session.

Post-training (phase 2)

Conditions were identical to phase 1; each life skills teaching session was preceded by a 5-minute pairing session, during which therapists could apply the skills learned during PSP training.

Procedural fidelity

Two types of procedural fidelity data were collected during this study. Firstly, we collected fidelity data on therapists’ accurate implementation of the Lugo et al. (2017) PSP protocol for 41% and 30% of phase 1 and phase 2 sessions, respectively. Average PSP implementation fidelity across all therapists was 66% (61%-75%) in phase 1 and 95% in phase 2 (86%-100%). Fidelity data was also collected on therapists’ accurate implementation of the life skills teaching procedures for 63% and 80% of phase 1 and phase 2 sessions, respectively. Average fidelity across all therapists was 99% in phase 1 (range 97%-100%) and 98% in phase 2 (range 94%-100%).

Interobserver agreement (IOA)

An independent observer collected IOA for the number of steps completed independently by each child on their target life skill for 75% of sessions during phase 1 and 87% of sessions during phase 2. A point-by-point method was used and IOA was calculated by dividing the total number of agreements by the total number of agreements plus disagreements and multiplying by 100. Mean IOA across all children was 100% for phase 1 and 98% for phase 2. IOA was collected for 68% of PSP fidelity checks in phase 1 and 85% in phase 2. The mean percentage of agreement for the PSP fidelity checks was 100% in phase 1 and 99% in phase 2. IOA was also collected for 78% of the life skills teaching fidelity checks in phase 1 and 62% in phase 2. A point-by-point method was used. The mean percentage of agreement for the life skills teaching fidelity checks was 96% in phase 1 and 98% in phase 2.

Social validity

On completion of the study, the therapists filled out an 11-item respondent-based measure that examined the PSP training intervention, in terms of relevance, potential for improvement and overall usefulness.

Results

All four therapists demonstrated an improvement in their accurate implementation of the Lugo et al. (2017) PSP protocol following training. Figure 1 shows the average percentage of steps on the Lugo et al. (2017) PSP protocol that were accurately implemented by each therapist during phase 1 and phase 2. On average from phase 1 to phase 2, therapist 1’s PSP implementation fidelity improved by 25% (phase 1: 75%; phase 2: 100%); therapist 2’s fidelity improved by 37% (phase 1: 63%; phase 2: 100%); therapist 3’s fidelity improved by 23% (phase 1: 63%; phase 2: 86%); and therapist 4’s fidelity improved by 32% (phase 1: 61%; phase 2: 93%). Training sessions lasted an average of 2 h and therapists mastered the implementation of the Lugo et al. (2017) PSP protocol after 1–2 trials with a confederate and 1–3 trials with their assigned child.

Figure 1.

Average percentage of steps accurately implemented on the Lugo et al. (2017) PSP protocol during phase 1 and phase 2.

Life skill acquisition

Acquisition graphs presented in Figure 2 show the number of independent steps performed on task analyses of target life skills, for each child across all sessions. Relative to phase 1 there was a gradual overall increase in the number of steps completed independently by child 1, child 2 and child 4 in phase 2. Child 3 did not demonstrate any increase in the number of steps completed independently from phase 1 to phase 2. In addition, there was evidence of substantial overlap between data in phase 1 and phase 2 across all four children. Therefore, based on visual analysis, it cannot be concluded that PSP improved skill acquisition for any of the children in the study. The life skills teaching procedures, which were in place for phase 1 and phase 2, may have accounted for the gradual improvements observed across 3 of the 4 children. We also examined the difference in improvement rates between phase 1 and phase 2 by calculating a robust omnibus IRD for the entire study; IRD = 0.34 CI ₉₅ (0.16, 0.52). This effect would be considered “very small” by conventional standards (Parker et al. 2009).

Figure 2.

Number of steps on the life skill task analyses completed independently during phase 1 and phase 2.

Social validity

All four therapists rated the PSP training very favourably, in terms of relevance, planning, opportunities for participation, providing motivation to learn more on the topic and overall impact in their current role.

Discussion

The overall purpose of this study was to train four behavioural therapists in the Lugo et al. (2017) PSP protocol, and examine the impact on life skill acquisition rates among a group of four young children with ASD. All therapists demonstrated substantial improvements in their accurate implementation of the PSP protocol, following training that was delivered through a BST format. Average implementation fidelity was 66% across phase 1, rising to 95% across phase 2. Furthermore, this improvement in implementation fidelity was achieved with relatively minimal time investment. Although effective therapist training in the Lugo et al. (2017) PSP protocol was critical to achieving the objectives of this study, it was secondary to the main aim, which was the evaluation of the impact of therapist training in this protocol, on the life skill acquisition rates of children with ASD.

According to the visual analysis, 3 of the 4 children demonstrated a gradual overall increase in the number of steps completed independently on their respective life skill task analyses. However, despite this positive trend, we observed a minimal increase in responding across all skills immediately following staff training in the Lugo et al. (2017) PSP protocol. In addition, the robust omnibus IRD for the entire study showed that the independent variable was minimally effective. As such, the life skills teaching procedures, which entailed a combination of backward chaining and least-to-most prompting, most likely produced the gradual increase in the number of steps completed independently by 3 of the 4 children, on their respective life skill task analyses. Therefore, we concluded that the specific PSP protocol, employed by therapists in phase 2 of this study, did not have any functional control over the children’s rate of life skill acquisition. In this instance, there was no evidence to indicate that the Lugo et al. (2017) PSP protocol was an effective intervention to enhance life skill acquisition for young children with ASD.

Null results, like the one reported in the current study, are essential to upholding the principle of self-correction, on which science is built (Ioannidis 2012, Munafo and Neill 2016). However, because there are numerous reasons why a study may fail to show an effect of the independent variable, studies reporting null findings must demonstrate specific characteristics of quality. These characteristics include investigating socially valid issues, employing robust methodological and analytical strategies, and presenting clear, conclusive results; readers must be convinced that the finding is valid and not the result of a technical flaw (Baxter and Burwell 2017).

All of the features of quality, outlined above, are evident in the current study. Staff training that results in observable, clinically effective gains for service users is a socially significant, essential area to research (Gormley, Healy, Doherty, et al. 2020) and this was the focus of the current study. Training in the Lugo et al. (2017) PSP protocol was delivered via BST because it is a proven method for the effective dissemination of skills to staff supporting individuals with intellectual and developmental disabilities (Gormley, Healy, O'Sullivan, et al. 2019, Maffei-Almodovar and Sturmey 2018). In addition, we selected the life skills teaching procedures based on two large-scale meta-analyses of evidence based practices for children with ASD (National Autism Center 2015, Wong et al. 2015).

In order to ensure the validity of our analyses, we proactively included treatment fidelity checks on a substantial portion of sessions during phase 1 and phase 2. These fidelity checks assessed the accurate implementation of the Lugo et al. (2017) PSP protocol, as well as the accurate implementation of the life skills teaching procedures. A significant percentage of these fidelity checks were verified by the observations of independent observers. To enhance the robustness, objectivity and interpretability of our analysis, we employed a combination of visual analysis, which was based on a multiple baseline design, with a measure of effect size (IRD). We had also decided to include a measure of effect size because emerging standards within social science and single case designs, in particular, are focusing on the need to include effect sizes to allow for ease of summary and meta-analysis across studies examining a common intervention (Hedges et al. 2012).

However, while findings from the current study did not show that the application of the Lugo et al. (2017) PSP protocol can augment life skill acquisition rates for young children with ASD, from a clinical perspective caution must be exercised in overextending the significance of our findings. This is the first study, to the best of our knowledge, which has evaluated the effects of this PSP protocol on life skill acquisition rates. Furthermore, the generalisability of our findings is naturally limited, as we have only examined these effects in the context of embedding the Lugo et al. (2017) PSP protocol within a teaching procedure that employs backward chaining and least-to-most prompting strategies. Therefore, before dismissing the clinical potential for the Lugo et al. (2017) PSP protocol to augment life skill acquisition rates, we would recommend that the current study be replicated with therapists and children, with similar characteristics to the current cohort. In addition, it would be interesting to investigate the impact of this PSP protocol on the life skill acquisition rates of older children with ASD, as well as evaluating the effects when it is combined with other life skill teaching procedures, such as total task presentation and most-to-least prompting.

The key limitation of the current study was that it was not designed to answer why the Lugo et al. (2017) PSP protocol did not impact life skill acquisition rates. For example, an analysis of levels of off-task behaviour across phase 1 and phase 2 may have helped answer this question. However, any hypothesising after the fact would be considered poor research practice, given that the failed findings would be analysed as if they were a priori hypotheses posited before the study was conducted (Baxter and Burwell 2017). Therefore, from a clinical perspective, it is important that future studies on this topic are designed in a manner that provides an understanding of why the independent variable did not have an effect, should the null finding from the current study be replicated. As researchers and practitioners, an openness to null findings, as well as a proactive approach to planning for and interpreting null findings, is critical to effective development and refinement of evidence based practice in real-world, applied settings. An understanding of the reasons why an intervention failed to have a positive impact, will be important in helping practitioners decide if the intervention is worth pursuing with their clients, as well as guiding context-specific adaptations to the intervention.

Disclosure statement

Three of the authors, including the first-named author, were employed by the setting, in which the research took place.