A Pilot Feasibility Randomized Trial of the RUBI in Educational Settings Intervention With Paraeducators Supporting Autistic Students in Public Elementary Schools

Abstract

Purpose

This pilot randomized study evaluates the feasibility, acceptability, and preliminary effectiveness of RUBI in Educational Settings (RUBIES), a school-based adaptation of the evidence-based Research Unit in Behavioral Intervention (RUBI) parent training program, when implemented by elementary school paraeducators supporting autistic students with externalizing behaviors that impact safety and well-being.

Methods

A 24-week pilot randomized controlled trial was conducted with 67 paraeducators and 65 autistic students across 39 public schools in the United States. Paraeducators were randomly assigned to RUBIES or an active comparator, Psychoeducation on Autistic Students in Schools (PASS).

Results

Feasibility outcomes, including adherence to intervention strategy implementation, were high for paraeducators receiving RUBIES. Acceptability measures demonstrated strong engagement and satisfaction among paraeducators, with RUBIES paraeducators reporting significantly greater confidence in managing student behaviors compared to those who received PASS. While both groups showed reductions in externalizing behaviors, no significant differences were found between RUBIES and PASS on standardized outcome measures of student externalizing behaviors.

Conclusion

Findings suggest that RUBIES is a feasible and acceptable intervention for paraeducators, though further research is needed to assess its effectiveness in promoting behavioral change in autistic students at school.

About 13% of students with disabilities being served in U.S. public schools under the Individuals with Disabilities Education Act are identified as autistic, a number that has grown significantly over the past two decades (Cardinal et al., 2021; Individuals with Disabilities Education Act [IDEA], 2004; National Center for Education Statistics, 2024). As part of IDEA, autistic students should be included in their least restrictive environment alongside neurotypical peers, which has shown to contribute to positive academic and social outcomes, as well as improved quality of life (De Bruin, 2020; Larcombe et al., 2019). However, across the United States, only approximately 30% of autistic students spend the majority of their day in general education settings (US Department of Education, 2020). Many autistic students face significant challenges that can hinder their full participation in general education settings (Barry et al., 2020; Segall & Campbell, 2012). Among the most common reasons for exclusion from the least restrictive environment is the presence of externalizing behaviors, including meltdowns, aggression, or property destruction, which can impact student safety and well-being (Ahlers et al., 2023).

The Research Unit on Behavioral Interventions (RUBI) Network developed a manualized, behavioral, parent training intervention for families of autistic youth who engage in externalizing behaviors. RUBI has been found to be acceptable to caregivers, reliably delivered by trained therapists, and effective in reducing externalizing behaviors and increasing adaptive skills in children ages 3–14 (Aman et al., 2009; Bearss et al., 2015; Scahill et al., 2016). Despite its efficacy, RUBI as originally designed may encounter implementation barriers in school settings due to a mismatch between the intervention content and school-based implementation requirements and contextual constraints (e.g., data collection requirements, time needed to make visual supports; Locke et al., 2015, Locke et al., 2017).

In response to these limitations, RUBI content was iteratively redesigned (and renamed RUBI in Educational Settings or RUBIES), in close collaboration with 40 elementary school partners (i.e., general education teachers, special education teachers, paraeducators, and school administrators, among others) from 28 schools. The goal of the redesign effort was to tailor RUBI content to be more practical and accessible within school environments (Bearss et al., 2022; Locke et al., Under Review). Feedback resulted in several major changes to RUBI session content: (a) inclusion of school-specific examples, (b) addition of an “Autism 101” psychoeducation module, (c) provision of a Visual Toolkit to address logistical challenges with creating highly-regarded visual supports, (d) collapsing of the two reinforcement sessions into one session; (e) redesign of compliance training procedures to focus on proactive support with instructional follow through, (f) integration of planned ignoring skills with functional communication, and (g) elimination of the two teaching skills sessions (Bearss et al., 2022). The redesigned RUBIES intervention now includes eight modules plus a booster session (see Table 1 for content modifications from RUBI to RUBIES).

Table 1.

Content modifications from RUBI to RUBIES

RUBI	Modification	RUBIES
	New Module	Autism 101
Behavioral Principles		Behavioral Principles
Prevention Strategies		Prevention Strategies
Daily Schedules		Daily Schedules
Reinforcement 1	Combined Modules	Reinforcement
Reinforcement 2
Planned Ignoring	Combined Modules	Mindful Attention [formerly Planned Ignoring] + FCT
Functional Communication Training (FCT)
Compliance Training	Removed Physical Prompting	Following Instructions [formerly Compliance Training]
Teaching Skills 1	Removed
Teaching Skills 2	Modules
Generalization and Maintenance		Generalization and Maintenance
Booster		Booster

RUBI Research Units on Behavioral Interventions, RUBIES Research Units on Behavioral Interventions in Educational Settings

In addition to content modifications, identification of ideal end users of RUBIES was prioritized as part of the redesign process. Special education teachers described confidence in employing behavioral strategies but the inability to follow students into their respective general education placements throughout the day (Bearss et al., 2022; Schuermann et al., 2003). General education teachers reported a need for training in the implementation of evidence-based practices (Tagavi et al. 2024); however, they also reported numerous barriers to implementing strategies due to time constraints and the high number of other students in their care (Harbin et al., 2024). Educator feedback also highlighted that the presence of externalizing behaviors during inclusion time increases the risk of autistic students being returned to more restrictive classroom placements. Lastly, paraeducators described receiving minimal training on autism and behavioral support strategies despite spending the most amount of direct time with autistic students throughout the school day, including during inclusion time (National Resource Center on Paraeducators, 2023). Thus, end-users of RUBIES implementation were identified as paraeducators supporting autistic students with co-occurring externalizing behaviors that were engaging in inclusion time with general education peers. Paraeducators’ continuous interaction with students provides a unique opportunity for consistent behavioral support, making them the ideal end-users of RUBIES (Bearss et al., 2022). However, there is limited evidence on the feasibility of training paraeducators to utilize complex behavioral management techniques and the effectiveness of such training in improving student outcomes (Walker et al., 2021; Yates et al., 2020).

The primary aim of this pilot randomized-controlled trial was to evaluate the feasibility and acceptability of RUBIES when implemented by paraeducators serving autistic students in elementary schools. Feasibility was assessed via (a) paraeducator adherence to RUBIES strategy implementation (practice plan); (b) paraeducator behavior support plan implementation; (c) session duration, (d) total length of treatment, (e) paraeducator ratings of feasibility, and (f) trainer fidelity to RUBIES. Acceptability was assessed via paraeducator (a) session attendance; (b) engagement during RUBIES sessions; (c) attrition; (d) ratings of satisfaction with RUBIES; and (e) ratings of RUBIES acceptability and appropriateness. The secondary aim of this pilot study was to examine the preliminary effectiveness of RUBIES when compared to an active control condition (Psychoeducation on Autistic Students in Schools; PASS) in order to obtain initial signals of improvement in student target behaviors. Establishing the feasibility of RUBIES is necessary to justify a large-scale randomized trial.

Methods

Study Design

This was a 24-week pilot randomized clinical trial involving paraeducator-student dyads from 39 public elementary schools in 12 states who were enrolled over a three-year period. The Seattle Children’s Research Institute Institutional Review Board approved the trial. Study approval was also received from school districts with research approval requirements. Recruitment efforts initiated through contact with school district officials who then identified eligible elementary schools with autistic students. Principals from those identified schools received study materials describing procedures, benefits, and risks. Principals facilitated introductions between school personnel (teachers, paraeducators) and research staff who then obtained educator written informed consent. School personnel then distributed plain-language, visually engaging materials to families of autistic students, and interested parents or legal guardians provided contact information for follow-up. Research staff reviewed study details with the student’s parent or legal guardian and written informed consent and child assent, when appropriate, was obtained.

At baseline, eligible paraeducator-student dyads were randomly assigned via REDCap to receive either RUBIES or PASS in a 1:1 ratio and stratified based on student grade level (Kindergarten to 2nd grade versus 3rd to 5th grade) and percentage of time in inclusive classroom settings (>/=50% vs. < 50%) using permuted blocks and allowing for concealment of allocation prior to randomization. To avoid intervention cross-contamination, only one paraeducator-student dyad per grade band (K-2nd, 3rd −5th) per school was eligible for enrollment. Outcome measures were administered at Baseline, Week 12, and Week 24. Paraeducators, teachers, and caregivers received compensation for completion of study measures at each assessment timepoint.

Statement of Community Involvement

The core research team included an autistic individual and a parent of an autistic child, both of whom contributed substantially to the study (e.g., design, data collection, analyses). In addition, two autistic young adults independently reviewed the RUBIES manual and workbook and provided feedback around language, content, and approaches to managing externalizing behaviors. We incorporated their revisions to ensure the RUBIES intervention content was appropriate for and meaningful to autistic elementary-aged students.

Participants

Eligible paraeducators were employed at a public elementary school in the United States and actively working with at least one autistic child with co-occurring externalizing behaviors. Eligible students had a documented autism diagnosis (per educational classification or clinical documentation), a clinically elevated Social Responsiveness Scale-2 (SRS-2) total raw score (> 70 for boys; >65 for girls), were enrolled in a public Kindergarten-5th grade with an established education plan (stable for a minimum of 6 weeks) and were engaged in inclusion time in a general education setting a minimum of 4 h/week. Students also needed to exhibit mild to moderate externalizing behaviors as rated by a teacher or paraeducator at baseline on the Aberrant Behavior Checklist-Irritability (ABC-I) subscale (raw score ≥ 10) and/or the ABC-Hyperactivity (ABC-H) subscale (raw ≥ 20). To help ensure broad representation in the sample, students were not excluded from enrollment if they had co-occurring mental health and/or medical conditions as long as they otherwise met the inclusionary criteria. Students were excluded if another student in their grade band was already enrolled in the study, if their paraeducator declined participation, or if their primary legal guardian did not provide consent for study enrollment.

RUBIES Intervention

RUBIES is a manualized intervention, with 8 Core Modules and 1 Booster Session. Sixteen weeks were allotted to complete the program, to account for cancellations, shortened meetings (e.g., 30 min), holidays, sick days, and conflicting school meetings (in-service days, field trips) (Owens et al., 2014). Paraeducators received continuing education credits, a RUBIES workbook, and a visual support resource kit.

RUBIES is based on behavior-analytic principles and is designed to help paraeducators learn how to understand behavior as communication (i.e., the function of a behavior) and as informed by autistic characteristics, such as communication differences and sensory sensitivities. This approach helps to directly counter common misattributions of behaviors as “oppositional.” RUBIES then builds the paraeducators’ behavioral skills “toolbox,” to allow for application of function-informed and tailored positive behavioral supports (PBS) (i.e., matching strategy to function; Conroy et al., 2005; Reichow & Barton, 2014). Finally, learning the RUBIES “toolbox” helps promote a paraeducator’s ability to generalize approaches when new behaviors emerge, and as new students are assigned to paraeducators each year.

Table 1 outlines the content of the 8 Core and 1 Booster modules. RUBIES begins with foundational education on autism (Module 1), followed by instruction on the Antecedent-Behavior-Consequence model to inform hypotheses on the function(s) of behavior (Module 2). Training is then provided on strategies to set students up for success (Module 3 Prevention; e.g., do things in small doses; give warnings; provide choices). As autistic children often have relative strengths in visual processing, visual cues and schedules are highlighted as effective ways to communicate expectancies and promote predictability (Module 4). Module 5 reviews the use of reinforcers to promote positive behaviors while mindful attention and functional communication training (Module 6) target the promotion of alternative, functional means of communicating while minimizing responses to inappropriate communication attempts. Module 7 focuses on effective ways for the paraeducator to set the student up for success with following instructions, while Module 8 reviews ways to generalize positive behavior gains across time and with other staff and students. Finally, paraeducators participate in a Booster Session approximately one month following RUBIES completion in order to review progress with behaviors and strategy implementation as well as problem-solve remaining needs.

While RUBIES follows a standardized manual, module implementation is tailored to the autistic student, informed by behaviors and their hypothesized function, identified in the first two sessions, as well as child- and paraeducator-specific classroom needs. This approach allows for bridging of skill practice to mastery that can then be generalized to other autistic children (i.e. the paraeducator learns how to create and implement visual schedules and practices with the autistic child they are currently paired with; this provides opportunities to build the skill while also implementing direct behavioral supports for their current student).

RUBIES sessions were conducted by a trainer who had a master’s degree or higher and completed training to fidelity in RUBIES (i.e. manual review, viewing of RUBIES sessions, co-leading sessions with RUBIES-trainer) prior to independently conducting sessions with randomized cases. RUBIES sessions were delivered 1-to-1 via Zoom at a time convenient for the paraeducator (e.g., before/after school, during preparatory period). Sessions followed a set structure: (a) weekly practice plan review (10 min); (b) didactic instruction on targeted behavioral strategies (40 min); and (c) practice planning to tailor the use of the newly learned strategies during the upcoming week (10 min). Sessions used interactive training strategies including didactics, written vignettes, in-session activities, and implementation discussion with feedback in order to assess paraeducator comprehension of materials, illustrate proper strategy use, and support mastery of techniques. Strategies developed and implemented over the course of RUBIES were recorded in a Behavior Support Plan (BSP), which is a document that is separate from the student’s Individual Educational Plan (IEP) and specific to RUBIES. The BSP was reviewed at the beginning of each session, which allowed the trainer to check in on paraeducator maintenance of RUBIES strategy implementation.

Psychoeducation on Autistic Students in Schools (PASS)

To support enrollment and provide an active clinical comparator to RUBIES, participating paraeducators randomized to PASS were given 16 weeks to complete an 8-module online, self-paced webinar series as well as two optional live check-in meetings with a trainer to discuss webinar content. PASS, which was hosted on a web-based learning management system, was focused broadly on supporting autistic children in schools and included modules on (a) Introduction to Autism, (b) Autism in Schools, (c) Interventions for Supporting Communication, (d) Executive Functioning, (e) Inclusion in Schools, (f) Autism Evidence-Based Practices, (g) Social Functioning in Autism, and (h) Recess Engagement Strategies. Each module was 20–35 minutes and intentionally did not discuss specific behavioral management strategies. The program format was similar to continuing education that paraeducators must complete each year. Autistic students whose paraeducators were assigned to PASS received standard of care school-based interventions at the professional discretion and direction of school staff (i.e., no research intervention).

Subject Characterization

A Paraeducator Demographic Form was developed for the trial to collect information such as age, gender, race, ethnicity, education, and years in position. A Student Demographic Form (completed by the caregiver) was developed to collect information such as age, gender, grade, race, ethnicity, and school placement.

The Social Responsiveness Scale-School Age Form, 2nd Edition (SRS-2; Constantino & Gruber, 2012) is a 65-item parent and teacher-reported questionnaire designed to measure social abilities associated with autism spectrum disorder in individuals from 4 to 18 years of age. Each item is rated on a 4-point Likert scale, ranging from not true = 1 to always true = 4. Higher overall Total Scores are associated with more severe social impairments. Raw scores are calculated separately for males and females. A clinically elevated total raw score (> 70 for boys; >65 for girls) was used for study inclusion.

Feasibility Outcome Measures

After each RUBIES session, the paraeducator’s adherence to their Practice Plan Implementation was rated by their trainer on three domains, each evaluated on a 4-point scale: (a) practice plan data collection (1 = no practices documented; 4 = 5 + practices documented); (b) correct implementation (1 = all practices implemented with errors; 4 = all practices implemented correctly; and (c) practice frequency (1 = no practices; 4 = practiced every day).

After each RUBIES session, the paraeducator’s Behavior Support Plan (BSP) Implementation was rated by their trainer on a scale of 1–8: 1 = none of the components of the BSP are being implemented; 4 = BSP is being partially implemented and with some implementation errors; 8 = The BSP is being implemented consistently and effectively in its entirety.

At baseline, Week 12, and Week 24, paraeducators completed the Feasibility of Intervention Measure (FIM; Weiner et al., 2017), which is a 4-item measure completed by intervention end-users in order to assess intervention feasibility (in this case, RUBIES). Raters score each item on a 5-point scale ranging from “Completely Disagree” to “Completely Agree”. The FIM has good internal consistency (α = 0.89) and test-retest reliability (α = 0.83).

The RUBIES Trainer Treatment Fidelity Checklist (TFC) tracks trainer implementation of each session goal for the 8 core RUBIES sessions (range of 9–18 goals per session). After completion of each session, trainers rated themselves on each treatment goal for a session as: 0 = Goal was not achieved; 1 = Goal was partially achieved; 2 = Goal was fully achieved.

Acceptability Outcome Measures

For each of the eight core RUBIES sessions, the trainer rated paraeducator engagement in session discussions, queries, and in-session activities via a Paraeducator In-Session Engagement Checklist. Paraeducator performance related to each goal was rated on a 3-point scale: 0 = goal not achieved; 1 = goal partially achieved; 2 = goal fully achieved. For example, in the Behavioral Principles session, the trainer rated the paraeducator’s performance on goals such as “…identifying the antecedents on Activity Sheet #2”, “…providing behavioral definitions of meltdowns on Activity Sheet #3”, and “…identifying the functions of behavior on Activity Sheet #4.” RUBIES core sessions have a range of 3–7 goals per session.

The 16-item RUBIES Satisfaction Questionnaire was rated by paraeducators upon completion of the RUBIES intervention to evaluate: (a) satisfaction with program components (e.g., helpfulness of various session content); (b) usefulness of the teaching tools (e.g., activity sheets); (c) the acceptability of program elements (e.g., number and length of sessions); and (d) confidence in handling current and future behavioral challenges. Items were scored on a 3- or 4-point Likert scale, with higher scores reflecting greater levels of satisfaction.

At baseline, Week 12, and Week 24, paraeducators completed the Acceptability of Intervention Measure (AIM; Weiner et al., 2017) and the Intervention Appropriateness Measure (IAM; Weiner et al., 2017). The AIM and IAM each include 4-items completed by intervention end-users to assess intervention acceptability and appropriateness (in this case, RUBIES). Raters score each item on a 5-point scale ranging from “Completely Disagree” to “Completely Agree.” Both the AIM and IAM have good internal consistency (α = 0.89 and 0.89, respectively) and test-retest reliability (α = 0.83 and 0.83, respectively).

Effectiveness Outcome Measures

The Aberrant Behavior Checklist (ABC; Aman & Singh, 2017) is a reliable and valid 58-item, informant-based measure that includes five subscales: Irritability (meltdowns, aggression and self-injury, 15 items); Social Withdrawal (16 items); Stereotypies (7 items); Hyperactivity (16 items); and Inappropriate Speech (4 items). Each item is rated on a 0 to 3 scale with higher scores indicating greater behavioral severity. Internal consistency for the five ABC subscales ranges from 0.72 to 0.89. The ABC also has normative data with autistic children (Kaat et al., 2014). The ABC-Irritability (ABC-I) and ABC-Hyperactivity (ABC-H) subscales were completed by paraeducators at Baseline, Week 12, and Week 24, and served as primary behavioral outcome measures for this pilot study. These subscales have been used as outcome measures in prior large-scale clinical trials on RUBI (Aman et al., 2009; Bearss et al., 2015; Handen et al., 2015).

Analytic Plan

Feasibility and Acceptability Outcomes

For feasibility outcomes, we calculated: (a) paraeducator adherence to RUBIES practice plan domains (data collection, correct implementation, practice frequency), expressed as a mean score for each domain, and averaged across sessions; (b) Behavior Support Plan implementation, expressed as a mean score averaged across sessions; (c) mean length of RUBIES sessions (in minutes); (d) mean length of treatment (in days); (e) pre-post change scores on the paraeducator-rated FIM; and (f) trainer fidelity to RUBIES manual goals (expressed as a percentage, averaged across sessions).

For acceptability outcomes, we calculated: (a) paraeducator session attendance (actual number of sessions attended by all paraeducators divided by the expected number of sessions, multiplied by 100 and mean number of sessions attended); (b) paraeducator engagement during RUBIES sessions (expressed as a percentage, averaged across sessions and based on ratings by their trainer); (c) attrition rates (expressed as percentage); (d) paraeducator satisfaction ratings of the RUBIES intervention (expressed as percentages), and (e) pre-post change scores on the paraeducator-rated AIM and IAM.

Effectiveness Outcomes

For the preliminary effectiveness outcomes, baseline comparisons between treatment groups (RUBIES vs. PASS) were conducted using t tests, χ² tests, and Fisher’s exact test (to assess the success of randomization). Generalized linear mixed models with main effects of intervention groups (RUBIES and PASS) and time (baseline, Week 12, and Week 24), group by time interactions (intervention group difference across time), and subject-level random intercepts were used to model the longitudinal trajectories of ABC-I and ABC-H subscale ratings. At each end point, the Least Square (LS) mean difference in measurement outcomes was estimated together with two-sided 95% confidence intervals (CI). Concurrent with the mixed model framework, missing data were assumed to be at random. Within-group Effect sizes (ES) were calculated by dividing the absolute LS mean difference from Baseline by the standard deviation at Baseline. ES were considered as small (0.2), medium (0.5), or large (0.8).

Results

Between September 2020 and February 2023, 84 paraeducators and 146 students were screened for study eligibility (see Study Consort Chart in Fig. 1). Seventeen paraeducators were excluded (8 did not meet inclusion criteria, 3 declined to participate, 4 unresponsive, 2 other) and 79 students were deemed ineligible (e.g., did not meet cutoff scores on the ABC, school district IRB declined participation, student’s paraeducator or teacher declined participation, there was another student in their school/grade band already enrolled in the study). Sixty-seven paraeducators and 65 autistic students with co-occurring externalizing behaviors were deemed eligible and randomly assigned to RUBIES (N = 34) or PASS (N = 33). Note, two paraeducators dropped out after randomization but before initiating their assigned intervention so their yoked student was assigned a different paraeducator and re-randomized, resulting in 65 paraeducator-autistic student dyads.

Fig. 1.

Study consort chart

*Note: Specific breakdown of student ineligibility reasons were not systematically documented.

Aside from paraeducator gender, the study groups were similar at baseline (see Table 2 for demographic information for paraeducators and students). Paraeducators were primarily female (58/67; 86.5%), white (47/67; 70.1%), with a college or advanced degree (33/67; 49.3%). Enrolled students ranged in age from 5 to 11 (M = 7.94 years, SD = 1.85). Most students were male (78.5%; 51/65) and predominantly white (40/65; 61.5%). At baseline, 65% (42/65) of students spent a majority of their time in general education classrooms. All students had an educational and/or community diagnosis of autism spectrum disorder, and all scored above the SCQ cutoff, supporting the diagnosis of autism.

Table 2.

Paraeducator and student characteristics

Paraeducators	RUBIES (n = 34)	PASS (n = 33)	p-value
Age in years [M(SD)]	42.3 (11.4)	40.2 (12.0)	0.45
Gender
Female	34 (100%)	24 (73%)	< 0.01
Male	-	5 (15%)
Non-binary	-	3 (9%)
Prefer not to answer	-	1 (3%)
Hispanic	6 (18%)	3 (9%)	0.50
Race			0.78
White	22 (65%)	25 (76%)
Unknown	5 (15%)	4 (12%)
Asian	3 (9%)	1 (3%)
Multiracial	1 (3%)	2 (6%)
Native American	2 (6%)	1 (3%)
Black	1 (3%)	-
Grade Support			0.99
Kindergarten to 2nd	17 (50%)	17 (52%)
3rd to 5th	15 (44%)	14 (42%)
Multi-Grade	2 (6%)	2 (6%)
Years in Position [M(SD)]	3.50 (3.84)	3.88 (3.99)	0.69
Education			0.87
Elementary/Middle School	1 (3%)	-
High School/GED	5 (15%)	6 (18%)
Some College	8 (24%)	11 (33%)
Professional/Vocational/Tech	1 (3%)	-
College or Advanced Degree	18 (53%)	15 (45%)
Other	1 (3%)	1 (3%)
Students	RUBIES (n=33)	PASS (n=32)	p-value
Age in years [M(SD)}	8.0 (1.9)	7.7 (1.9)	0.53
% Male	24 (73%)	27 (90%)	0.40
Grade			0.99
Kindergarten to 2nd Grade	19 (58%)	18 (56%)
3rd to 5th Grade	14 (42%)	14 (44%)
Hispanic	5 (15%)	5 (16%)	0.99
Race
White	21 (64%)	19 (59%)	0.54
Multiracial	5 (15%)	7 (22%)
Native American	2 (6%)	1 (3%)
Black	0 (0%)	2 (6%)
Asian	2 (6%)	0 (0%)
Middle Eastern	1 (3%)	0 (0%)
Unknown	2 (6%)	3 (9%)
Years since Autism Diagnosis [M(SD)]	4.53 (1.85)	4.26 (2.32)	0.61
Time in Special Education Classroom			0.93
< 50%	11 (33.3%)	12 (37.5%)
50%+	22 (66.7%)	20 (62.5%)
SRS Total Score [M(SD)]	120.56 (28.01)	112.09 (22.40)	0.19

GED General educational development, SRS Social Responsiveness Scale

Feasibility and Acceptability Outcomes

Table 3 delineates RUBIES feasibility and acceptability outcomes.

Table 3.

Feasibility and acceptability benchmarks

Implementation Outcomes	RUBIES Mean (SD)	PASS Mean (SD)	p-value
Feasibility
RUBIES Adherence to Practice Plan
Data Collection	2.90 (1.22)
Correct Implementation	3.29 (0.80)
Practice Frequency	3.28 (0.91)
BSP Implementation	6.78 (1.25)
Session Length (in minutes)	59.1 (13.5)
Length of Treatment (in days)	55.5 (21.7)
RUBIES Trainer Fidelity	97.3% (2.3%)
Feasibility (FIM)
Week 12	4.51 (0.60)	4.24 (0.55)
Week 24	4.72 (0.48)	4.25 (0.59)	0.057
Aility
RUBIES Sessions Attended	7.7 (1.3)
RUBIES In-Session Engagement	78.1% (20.5%)
Attrition	11.8%	0.0%	0.114
Satisfaction
Appropriate # of RUBIES sessions	83%
Recommend RUBIES	100%
Current Confidence	91%	46%	0.002
Future Confidence	96%	54%	0.003
Acceptability (AIM)
Week 12	4.63 (0.53)	4.31 (0.60)
Week 24	4.72 (0.47)	4.24 (0.65)	0.629
Intervention Appropriateness (IAM)
Week 12	4.50 (0.64)	4.16 (0.62)
Week 24	4.70 (0.47)	4.21 (0.61)	0.071

PASS Psychoeducation on Autistic Students in Schools, RUBIES Research Units on Behavioral Interventions in Educational Settings, FIM Feasibility of Intervention Measure, AIM Acceptability of Intervention Measure, IAM Intervention Appropriateness Measure, BSP Behavior support plan

Feasibility

Trainer mean ratings of RUBIES paraeducators’ adherence to weekly practice plan implementation was variable across the three domains that were rated: (a) practice plan data collection across the 8 Core Sessions averaged 2.90 (SD = 1.22; scale of 1–4); (b) correct implementation of assigned strategy averaged 3.29 (SD = 0.80; scale of 1–4); and (c) frequency of practice averaged 3.28 (SD = 0.91; scale of 1–4). Overall Behavior Support Plan implementation averaged 6.78 (SD = 1.25; scale of 1–8).

RUBIES sessions averaged 59 min (SD = 13.5). Among the 30 paraeducators that completed all 8 core RUBIES sessions, treatment duration averaged 55.5 days (SD = 21.7; range 28–109 days). RUBIES was delivered by 9 trainers with high fidelity to the RUBIES manual (mean fidelity = 97.3%, SD = 2.3%).

Using the FIM on a 5-point scale, paraeducators rated the feasibility of both RUBIES and PASS as high, with no significant differences between the interventions or over time (Week 12 and 24) (F(1,22) = 4.02, p = 0.057).

Acceptability

Of the 272 expected core sessions to be attended (8 core sessions × 34 paraeducators in RUBIES), actual attendance was 246 sessions (90.4%). The average number of RUBIES sessions completed by paraeducators was 7.7 (SD = 1.3). Paraeducator in-session engagement was moderate with notable variability (mean % engagement = 78.1%, SD = 20.5%, range 20–97.3%). Attrition for RUBIES was 11.8% (4 of 34) and 3.0% for PASS (1 of 33). Reasons for RUBIES drop-out included personal reasons (N = 1; medical leave), scheduling challenges (N = 2), and leaving the school (N = 1).

At Week 24, 83% of RUBIES paraeducators reported that the number of sessions was appropriate. 100% of paraeducators indicated that they would recommend RUBIES to other educators supporting autistic students in the classroom. Ninety-one (91%) percent of paraeducators who received RUBIES reported having greater confidence in managing current behaviors while 96% rated as being more confident in managing future behaviors. These confidence rates were significantly higher than PASS paraeducators, where 46% rated as having more confidence in managing current behaviors and 54% reported being more confident in managing future behaviors (Present confidence ${\chi }_1^2=9.18$,p = 0.002; Future confidence ${\chi }_1^2=8.56$,p = 0.003).

Paraeducators rated the acceptability (AIM) and appropriateness (IAM) of both RUBIES and PASS as high, with no significant differences between the interventions or over time (Week 12 and 24) (AIM F(1,22) = 0.24, p = 0.629; IAM F(1,22) = 3.58, p = 0.071).

Primary Effectiveness Outcome

On paraeducator ratings of the ABC-I subscale, no significant differences were observed in the rate of change between RUBIES and PASS from Baseline to Week 24 [F(1,103) = 0.001, p = 0.965] (See Table 4). Paraeducators in both RUBIES and PASS reported decreases on the ABC-I subscale from Baseline to Week 24 [F(1,103) = 13.77, p = 0.001]. Specifically, at Week 24, RUBIES showed an average 24.3% decline on the ABC-I subscale [16.88 (9.33) to 12.78 (7.93); Effect Size (ES) = 0.44] compared with a 20.6% decrease for PASS ([18.12 (9.07) to 14.38 (7.77); ES = 0.41].

Table 4.

Aberrant behavior checklist outcomes

Outcomes	RUBIES Mean (SD)	PASS Mean (SD)
ABC Irritability
Baseline	16.88 (9.33)	18.12 (9.07)
Week 12	14.18 (8.86)	15.9 (7.84)
Week 24	12.78 (7.93)	14.38 (7.77)
Within Group Effect Size	0.44	0.41
ABC Hyperactivity
Baseline	22.85 (9.73)	28.12 (10.93)
Week 12	16.18 (8.44)	23.37 (9.1)
Week 24	14.83 (6.91)	22.04 (9.04)
Within Group Effect Size	0.82	0.56

PASS Psychoeducation on Autistic Students in Schools, RUBIES Research Units on Behavioral Interventions in Educational Settings, ABC Aberrant Behavior Checklist

Similarly, students in both RUBIES and PASS also improved over time on the paraeducator-rated ABC-H subscale [F(1,103) = 30.62, p < 0.001] with no significant difference in the rate of change from Baseline to Week 24 [F(1,103) = 0.03, p = 0.852]. However, ABC-H was significantly lower (5.5 points) in the RUBIES group compared to the PASS at baseline [F(1,103) = 6.09, p = 0.0153]. At week 24, RUBIES showed an average 35.1% decline on the ABC-H subscale [22.85 (9.73) to 14.83 (6.91), ES = 0.82] compared with a 21.6% decrease for PASS [28.12 (10.93) to 22.04 (9.04); ES = 0.56].Table 4 presents baseline, week 12, and week 24 scores and effect sizes for the ABC-I and ABC-H subscales.

Discussion

In this pilot study of the redesigned RUBI in Educational Settings (RUBIES) intervention when implemented by paraeducators supporting autistic students in public elementary schools, RUBIES was found to be feasible and acceptable for paraeducators to implement, as noted by high rates of between-session practices that were implemented with fidelity and high sustainment of RUBIES strategy use. These findings were noted in spite of lower rates of practice documentation. RUBIES appears to be structured in a manner that is achievable for paraeducators to engage in, noting sessions lasted an average of 59 min and treatment lasted an average of 56 days. Attrition was low for RUBIES (11.8%) and paraeducators reported high satisfaction with the intervention. Finally, RUBIES uniquely increased paraeducator current and future confidence in addressing students’ externalizing behaviors (RUBIES at 91% and 96% versus PASS at 46% and 54%, respectively).

Qualitative indictors of paraeducator satisfaction with RUBIES were also found during exit interviews with educators (Tagavi et al., Under Review). To illustrate, one paraeducator who received RUBIES training expressed:

Oh, I think it works great…I’ll use these strategies, not just with kids on the autism spectrum. I use these same strategies with kids who just have behavior issues. Because there might be some differences, but behavior is behavior. And it has a function, and it has a purpose, and I think it is just really good to bring it back to not looking – not so much looking at the behavior, but what is the function of this behavior? What is the child trying to communicate? And how can we shape this to a healthier way of getting to what they want or need or replacing or fixing it?

Nine trainers also were able to maintain high fidelity to the RUBIES manual (97.3%). Despite the practical complexities of implementing interventions in school settings (holiday/summer breaks, student and educator illness, finding time to attend sessions), these positive indicators for RUBIES’ feasibility and acceptability are noteworthy, particularly considering two of three years of study recruitment occurred during school years significantly impacted by COVID-19 (2020–2021 and 2021–2022).h

While feasibility and acceptability benchmarks suggest RUBIES may be a promising intervention for paraeducators supporting autistic students with co-occurring externalizing behaviors, these findings did not translate to robust differences in change scores between groups (RUBIES vs. PASS) on standardized outcome measures of student behavior, with both groups reporting comparable positive behavioral change. Aside from being an underpowered sample size, there may be several considerations around this lack of finding. First, RUBIES was designed to support an identified paraeducator who is actively supporting an autistic student during the school day. While targeted intervention approaches in RUBIES are designed to facilitate the ability of the paraeducator to set the student up for success, there is the expectation that the larger context (in this case, other students and paraeducators, overall classroom design) is oriented in a manner that allows the strategies to be effective, or at a minimum, to not actively interfere with effectiveness. In practice, it is possible, if not certain, that the opposite is likely, such that individual paraeducator success with RUBIES is a necessary but not sufficient condition to promote robust behavioral change.

To this end, in addition to the “what” (i.e., RUBIES) we must consider the “how” or the implementation strategies used to train paraeducators to learn and use RUBIES with fidelity in this unique context of care. In US public elementary schools, autistic children are often not educated in a vacuum with one sole educator. Rather, autistic children generally have a team of supports that comprise general and special education educators, paraeducators, and specialists. Throughout the day, different team members support autistic children in various settings (e.g., general vs. special education classroom, cafeteria, playground, library, gym). The one-to-one RUBIES training used in this trial did not systematically include the broader system of care that supports autistic children in schools. An integrated and collaborative care team planning process, which should include integration of relevant student IEP goals and strategies into the RUBIES-focused Behavior Support Plan and ideally vice versa, may be needed to more effectively ensure generalization and maintenance of strategies across different daily activities, school settings, and educators in order to produce positive behavioral change that is significantly different from an active comparator condition.

Another consideration involves measurement issues. The study used the ABC-I and ABC-H subscales as the primary outcome measures, replicating all previous trials of the RUBI intervention (Aman et al., 2009; Bearss et al., 2015; Handen et al., 2015). It is possible that the ABC is not the best fit for the range of behaviors being targeted via RUBIES in the school context. For example, the ABC-I subscale includes items on self-injury, which may be less likely with students in our sample who are engaging in inclusion efforts. Future studies of RUBIES may benefit from including behavioral measures specifically designed for school-based contexts.

Another measurement-related factor involves the inclusionary criteria for this trial; specifically scoring 10 or higher on the ABC-I subscale or 20 or higher on the ABC-H subscale by either the paraeducator or teacher. While this approach was designed to enhance study enrollment, noting Year 1 and Year 2 recruitment occurred during the height of COVID-19, this criterion was lower than the traditional ABC-I minimum raw score of 15 in other RUBI clinical trials (Aman et al., 2009; Bearss et al., 2015; Handen et al., 2015). It also resulted in the enrollment of eligible students where the teacher endorsed externalizing behaviors, whereas the paraeducator did not. In fact, 8 of 33 (24%) students who were enrolled in RUBIES and 5 of 32 (16%) students who were enrolled in PASS scored < 10 on the ABC-I by their paraeducator. Inclusion of students with low ratings at baseline creates a natural restriction in the amount of behavioral change possible over the course of the study. Ultimately, while paraeducator ABC-I ratings were selected as the primary effectiveness outcome due to their engagement in RUBIES, it will be important to examine ratings concordance between paraeducators and teachers to further inform generalization of behaviors within the school context as well as potential reach of change in student outcomes.

We also found that both RUBIES and PASS resulted in improvements in paraeducator ratings of student challenging behavior. A different hypothesis for this lack of differences in outcomes may be that access to any training, whether focused specifically on strategies for externalizing behavior or focused more generally on increasing understanding of autism may result in better perceptions of students. Being able to contextualize student behavior may create an indirect pathway to change in behavioral ratings. This is important when we consider paraeducators typically receive limited training on either autism or positive behavioral supports prior to starting their positions, and they also are not provided with access to continual professional development (Giangreco, 2010). To this end, the acceptability, flexibility, low cost, and low demand effort of PASS may suggest its potential as a useful front line professional development training for educators supporting neurodiverse students. Lastly, there is the possibility that, despite strong indicators of intervention feasibility and acceptability, including paraeducator high satisfaction and testimonials suggesting positive impacts, RUBIES, which is a low intensity and targeted intervention, may not be sufficiently powerful to result in robust changes in student behavior.

Limitations

Several limitations are noted. First, since the study occurred during two school years that were significantly impacted by the COVID-19 pandemic, we experienced a number of recruitment challenges (e.g., increased student and paraeducator absences due to illnesses, staffing shortages) that resulted in a smaller than anticipated sample size (recruitment goal was 80 paraeducator-student dyads). Second, our primary outcome measure relies on paraeducator report, noting they are not masked to intervention assignment and instead were actively participating in the interventions (RUBIES and PASS), which increases vulnerability to reporting bias. A common protection from rater bias is to include behavioral observation data (Matson & Nebel-Schwalm, 2007). In this trial, direct student observation could have potentially captured differences in behavioral change between the two interventions where paraeducator-reported measures did not. While this trial originally proposed virtual, recorded behavior observation as a secondary outcome measure for paraeducator implementation fidelity and student externalizing behavior, due to consent challenges (e.g. requirements that all students in the classroom receive notification of incidental recording) and school policies (e.g. blanket policies that did not allow classroom recording), this proved to be difficult in practice. A future large scale clinical trial would also expect high costs associated with conducting and coding behavioral observations on a large sample in schools (Blom-Hoffman et al., 2009). A feasible alternative involves the inclusion of a masked independent evaluator who can conduct standardized ongoing assessments of change. A final limitation relates to concerns around scalability, which is a recognized challenge if schools or districts are not fully on board to initiate training and/or sustain implementation of an evidence-based intervention, such as RUBIES.

To address these limitations, future studies on RUBIES should ensure inclusionary criteria minimize floor effects in outcomes (i.e., minimum criteria for baseline scores are above the mean and based on ratings of the educator who will be participating in RUBIES). Second, the protocol should utilize standardized measures specifically designed to evaluate school-based externalizing behaviors and that account for rating bias through the inclusion of a masked Independent Evaluator ratings of change via the Clinical Global Impressions-Improvement scale (Busner & Targum, 2007). Future studies also could examine whether a fidelity benchmark for weekly practice or behavior support plan implementation predicts positive behavioral change and also examine whether there are targeted systems engagement approaches that could potentially support scalability in school settings.

To this end, we are currently conducting a follow-up pilot trial of RUBIES where the primary focus is on the impact of modified implementation strategies on educator fidelity and student behavior, including some of the hypothesized contributors in this trial (i.e., inclusion of the broader student team in intervention planning, integration of the student’s IEP into the Behavior Support Plan, enhanced focus on generalization and cross-team communication). This study also includes a cost analysis related to the implementation of RUBIES which may inform scalability, as well as methodology adaptations, such as changes in inclusion criteria and outcome measures (e.g., adding the Sutter-Eyberg Student Behavior Inventory [SESBI; Eyberg & Pincus, 1999], Emotion Dysregulation Inventory [EDI; Mazefsky et al., 2018], and Independent Evaluator ratings). Generating evidence for the effectiveness of RUBIES addresses a significant gap in educator-delivered interventions to minimize highly prevalent externalizing behaviors among autistic children in schools. Because RUBIES was developed and tested in partnership with local education and autistic partners, it is designed to be highly feasible and usable in public schools, which has significant implications for widespread use to support autistic children in their naturalistic environment.

Data Availability

De-identified baseline data from this trial are available through the National Database for Autism Research (NDAR).