A Large-Scale Naturalistic Evaluation of the AIM Curriculum in a Public-School Setting

Abstract

This study evaluated the effectiveness of the AIM curriculum when implemented in a public-school setting by schoolteachers and direct care staff. Three hundred eighteen students took part in this quasiexperimental design where all received the AIM curriculum every day for an entire school year. The participants completed a series of self-assessments (the Avoidance and Fusion Questionnaire for Youth, the Child and Adolescent Mindfulness Measure, and the Strength and Difficulties Questionnaire) at the onset of the study and at the end of the school year to assess psychological flexibility, mindfulness, and emotional behavioral skills. Results suggest that at the end of the school year, participants increased psychological flexibility and mindfulness. State standardized testing scores also showed increases school-wide as compared to the previous 2 years. These results suggest that the AIM curriculum may be effective in large school settings, appeared easy to implement by school staff to address the needs of both the individual student and the entire student body, and likely participated in improving school-wide academic success.

Children today enter the school classroom with a wide variety of adverse conditions that impact learning. Many students face chronic stress, academic demands, and social pressures that compete with contingencies designed to educate (Leonard et al., 2015) and thus hinder their overall well-being at school. These challenges appear to be getting worse, as the last decade has revealed an increasing number of students struggling with mental health conditions and accelerated rates of risky behaviors such as substance abuse and violence (Conner et al., 2009; Suldo et al., 2008). There are many statistics to consume that reveal a concerning trend away from progress in the current U.S. educational system. For example, a recent survey of 3,645 students in high-performing high schools discovered that 70% felt often or always stressed by schoolwork, 60.9% felt that schoolwork had kept them from doing other things such as spending time with family and friends, 24% indicated that they frequently felt depressed and used over-the-counter or legal stimulants, and 8% reported use of illegal stimulants or drugs to boost their performance (Conner et al., 2009). Additional concerns have been raised that considerable variation exists in teacher reports of the at-risk levels of students who struggle emotionally (Splett et al., 2020), which has serious implications for which children actually receive treatment. Data such as these indicate that perhaps the keys to solving the educational challenge in this country will need to be broader than pure academic interventions. Rather, a comprehensive approach to overall student improvement will most likely require concurrent attention to social and emotional well-being.

A variety of behavior-analytic approaches have been used within school contexts to improve overall student social and emotional success. Furthermore, these treatments have been attempted at both small- and large-scale intervention levels. The Good Behavior Game (Barrish et al., 1969) and its slight variations have been used to create group contingencies that promote positive social interactions among groups of students. This intervention often consists of a friendly competition game across teams of students in which points are given to the team when they display positive behaviors and removed when they display negative behaviors. Positive behavioral interventions and support (PBIS) arose from traditional behavioral interventions with a slight rebranding and packaging that emphasized ease of use, a focus on positive consequences, and a tiered system of intensity for intervention. PBIS has been well documented to enhance classroom management procedures (Parsonson, 2012), reduce individual child behavior problems (Carr and Curand, 1985), and promote skill acquisition (Vladescu & Kodak, 2010). Although the evidence for PBIS interventions appears robust along a number of dimensions, some researchers have posited that integrating this approach with treatments designed to more directly target social-emotional learning may be the best means of addressing the mental health needs of students (e.g., Bradshaw et al., 2014 and Cook et al., 2015). As noted by Wilson (2015), mindfulness- and acceptance-based interventions could provide an additional avenue by which to address students’ needs in a manner that addresses some of the potential shortcomings within the PBIS model. Additionally, when programs like PBIS are implemented with variability across schools, the active treatment components may be missing, and thus student outcomes may be hindered (Molloy et al., 2013). Although natural variation will occur when treatments are spread widely across settings, optimal interventions will need consistency, ease of administration, and active elements universally delivered by all stakeholders.

Recently, there has been growth in both the peer-reviewed research and mainstream popularity of what have been termed mindfulness-based interventions for school-age children. The specific names of such treatments vary and are often coined by authors or implementers, yet they remain rather consistent in theme. In such programs, an emphasis is placed on attending more closely to the current environment, noticing thoughts that may recall past events or hypothesized futures, and not allowing such thoughts that thus pull the child out of the present moment to dominate their focus. Behavior-analytically, the process of attaining mindfulness might be conceptualized as an increase in stimulus control toward the objects and events within the present environmental space, and a reduction in the abstraction such stimuli may evoke, which could include transfers of stimulus functions. In other words, the purpose of mindfulness may be to reduce the level of covert referential verbal behavior and transfers of stimulus functions that tend to occur in verbally complex humans. For example, when seeing a sunrise, one can attend to the actual stimuli of the sun, the horizon, the temperature, and the sounds around. These actual stimulus objects and one’s orientating toward them at great intensity as natural objects may be thought of as being mindful. The person is fully experiencing the present moment of the context surrounding them. However, when that same person is exposed to that same stimulus event of a sunrise, and the image of the sun “reminds” this person of sunscreen and then about not being burnt, and then causes them to worry about getting skin cancer and finally how their mother died early from stomach cancer, this same event of a sunrise may be experienced quite differently at a covert, private level of interaction. The latter would be an example of a nonmindful, or mindless, moment in which the person quickly abstracts and relates functions from the stimulus objects and most likely ends up experiencing private events that might be verbally described or labeled as lonely, painful, or anxious.

Applications of mindfulness for school-age children have documented stress-reduction and readiness-to-learn gains, along with improvements in social and emotional functioning (Center for Wellness, & Achievement in Education, & SFUSD Research, Planning,, & Accountability Department, 2015), improvements in self-reported well-being (Huppert & Johnson, 2010), and reductions in aggressive and noncompliant behaviors (Singh et al., 2007). Additional support for mindfulness implementation in schools is that it appears to produce improvements in attention and social skills and decreases in anxiety (Liehr & Diaz, 2010; Schonert-Reichl & Lawlor, 2010). As mindfulness-based interventions continue to emerge, there are also discussions on the utility of expansion beyond just remaining present. Rather, it may be of added utility to attempt treatments that also provoke active behavior of the child directed toward achieving personal values. Doing so may just more align with behavioral approaches that tend to see mindfulness as a means to an end, rather than an end in itself. In the context of school, mindfulness appears to have potential benefits, some of which might be of clear value to the student being asked to become mindful. Perhaps the value-directed behavior of the student could serve as a motivating operation to enhance the probability of their engagement in mindful tasks. At a school-system level, desired values that might drive initiation of mindfulness interventions might include improving test scores, reducing bullying, improving the social and emotional culture, or creating an improved positive learning environment.

Behavior-analytic mindfulness interventions are often prescribed under the vernacular of acceptance and commitment therapy or training (ACT; Hayes et al., 1999). The treatment approach of ACT tends to focus on teaching the client to embrace the present moments of life, accept the full array of thoughts and experiences, and engage in behavior toward personally identified values. Thus, a little greater in scope than pure mindfulness, ACT tends to include techniques to promote behavioral activation and goal setting. The nontechnical language of ACT may result in difficulty for some behavior analysts to comprehend ACT’s place within a behavioral framework of interventions, yet recent tutorials have focused on attempting to clarify the matter (Dixon et al., 2020; Tarbox et al., 2020). Topical applications of ACT by behavior analysts have included increasing novel food consumption of young children (Kennedy et al., 2014), improving rock-climbing skills of persons with autism (Szabo et al., 2019), increasing test performance in graduate school (Paliliunas et al., 2018), and extending attention to various tasks in children with attention-deficit/hyperactivity disorder (Enoch & Dixon, 2017). As ACT treatments move from controlled studies to large-scale implementations such as within school systems, there may be value in constructing circular materials that can be easily implemented by staff without extensive behavioral expertise.

The AIM curriculum (Dixon & Paliliunas, 2018) was developed as a means to introduce the concepts of mindfulness, ACT, and behavior-analytic principles to non-behavior analysts. Although the approach of AIM is themed toward frontline implementers and school personnel, the tools found within remain of utility to even skilled behavior analysts. Composed of 75 mindfulness exercises, 175 ACT lessons, and a function-based contingency management system, AIM attempts to make the technical approach of social and emotional intervention an easy-to-understand system designed for school-age children. Furthermore, AIM’s lessons are presented in a tiered system allowing for brief preventative doses for general education students, as well as longer duration, intensive reactionary lessons for those students in need of immediate intervention for social and emotional support. AIM couples these therapeutic lessons with exemplars of classroom-wide behavior management systems that rely on point-based contingencies that yield opportunities to gain access to functional classes of reinforcers of various magnitudes.

The purpose of the current quasiexperimental, naturalistic evaluation was threefold. First, we examined the large-scale effects of the AIM curriculum when implemented daily by teachers within a public-school setting on student reports of psychological flexibility, a direct index of adherence to the ACT model. Second, we evaluated the effects of the curriculum on overall student-reported states of mindfulness as an index of the broader impact of the intervention. The final purpose of the present study was to determine the potential effects of the AIM curriculum on academic performance on state standardized tests, which is often a significant metric of school-wide success in the eyes of various stakeholders within the education system such as parents, school boards, and state government.

Method

Participants, Setting, and Materials

Participants were students from a public middle school in the midwestern United States. The school was selected for the study after a staff member reached out to the researchers inquiring about a social-emotional curriculum that could be easily delivered by classroom staff and evaluated for effectiveness. The school consisted of sixth-, seventh-, and eighth-grade classrooms, all of which participated in the AIM program during the entirety of the school year. In order for inclusion in the data analysis of this present study, students were required to provide both parental and self-consent. All students who agreed to participate and provided both consents were included in the study (i.e., there were no exclusion criteria). Students who did not provide research consent still received the intervention because the school determined the intervention would be a part of their daily curriculum; however, those students did not complete the assessment packet, and their data are not including in this report. Thus, 318 students from a total of 513 (61.98%) were included as defined by the criteria of obtaining consent and completing all the program evaluation throughout the entire school year. Common attrition was mostly due to evaluation testing not being completed at set intervals. Participants included typically developing children and children receiving special education services. Demographic information was not collected for each participant to strengthen anonymity, but information is provided on the demographics of the entire school. Racial/ethnic diversity of the middle school was 91% White, 2% Black, 3% Hispanic, 1% Asian, and 4% of two or more races; 51% of the student body came from low-income households, and 14% of the student body had individualized education programs for that school year. The distribution of gender was approximately equal.

All assessments and intervention sessions were most often conducted in the students’ homeroom or primary classrooms. These classrooms contained one teacher and anywhere from 10 to 30 students, with some rooms also containing a paraprofessional assistant. All materials were provided for the teachers, including assessment materials and intervention materials. Assessment packets were administered at the beginning of the year in August and then again at the end of the school year in May. Teachers prompted the students to answer all of the questions within the packet and assisted with any questions the students may have had. Students completed the assessment packets independently. Intervention sessions also took place within these same classrooms. All intervention materials were prepared by the researchers and given to the teachers, and these materials were kept in their classrooms throughout the school year. Materials included the AIM curriculum textbook and the assessment packet that consisted of the Avoidance and Fusion Questionnaire for Youth (AFQ-Y; Greco et al., 2008), the Child and Adolescent Mindfulness Measure (CAMM; Kuby et al., 2015), and the Strengths and Difficulties Questionnaire (SDQ; Goodman, 2001). The study took place over the entirety of the school year (approximately 175 school days).

Dependent Variables and Measures

The AFQ-Y

The AFQ-Y is a child-report measure of psychological inflexibility characterized by high levels of cognitive fusion and experiential avoidance, intended to capture the rigid, narrow responding characteristic of inflexibility (Greco et al., 2008). The AFQ-Y is a 17-item questionnaire that asks participants to rate how true each item is for them on a 5-point rating scale, with higher scores indicating higher levels of psychological inflexibility. All items within the AFQ-Y are based on an ACT model of human suffering and represent psychological inflexibility through questions about cognitive fusion (e.g., “My thoughts and feelings mess up my life”) and experiential avoidance (e.g., “I push away thoughts and feelings that I don’t like”; Greco et al., 2008). Higher scores indicate greater levels of psychological inflexibility. Research has suggested that the AFQ-Y is a developmentally sensitive measure with adequate reliability and validity for youth (Greco et al., 2008).

The CAMM

The CAMM is a 10-item questionnaire of mindfulness for children and adolescents that measures individuals’ tendencies to be more or less mindful, independent of time (Kuby et al., 2015). The CAMM assesses aspects of mindfulness through factors such as “acting with awareness” (attending to present-environmental stimuli) and “accepting without judgment” (tolerating internal/external experiences without engaging in evaluative verbal behavior; Greco et al., 2011). The CAMM is scored in reverse, meaning lower scores indicate higher levels of self-reported mindfulness. Greco et al. (2011) found good internal consistency with the CAMM, whereas Lechtenber (2012) found appropriate convergent and divergent validity in an initial validation study.

The SDQ

The SDQ is an emotional and behavioral screening tool for children and adolescents. The SDQ has 25 items, some positive and others negative, in which participants use a 3-point Likert scale to indicate how each item applies to them (Goodman, 2001). The 25 items are divided into five scales, which include the following: the emotional symptoms subscale, the conduct problems subscale, the hyperactivity/inattention subscale, the peer relationship subscale, and the prosocial behavior subscale. All but the last are summed together to generate a total difficulties subscale. Research has found the SDQ to have satisfactory reliability and validity when administered to children and adolescents (Goodman, 2001).

The AFQ-Y, CAMM, and SDQ were completed on two occasions for each participant: at the beginning of and then at the end of the school year. Each questionnaire took between 3 and 10 min to complete and was done independently by the student.

Experiment Design and Procedure

A quasiexperimental research design was used to evaluate the effects of the AIM curriculum on psychological flexibility, mindfulness, and academic outcomes before and after intervention. In addition, data regarding the academic performance of the school as a collective whole were compared with those of 2 years prior to and following the AIM intervention. School-wide data were additionally compared to the entire state’s average data to examine how this school’s performance changed in comparison to state middle-school averages before and after the AIM intervention.

Procedure

Teacher Preparation

Prior to the AIM curriculum being introduced to the students, the staff were provided with copies of the curriculum to read on their own time, as well as a 2-hr AIM training for all staff at the beginning of the year conducted by the first author. The content of this brief training included a definition of mindfulness, the threats to keeping one’s mind in the present moment, the empirical findings of the effects mindfulness can have on students, and a few examples of AIM lessons and how a teacher would deliver the program. Teachers who were targeted for more intensive content on AIM received an additional full-day training by the first author, which included discussion of AIM’s approach to functional intervention and contingency management, along with extensive role-playing and experiential activities from the AIM text. Progress monitoring to ensure the correct use of the AIM curriculum was completed throughout the course of the study by the first author, which included answering any questions or clarifying procedures of the program. Less than 10% of all teachers sought any additional support during the entirety of the school year, and no teacher received more than a total of 1 additional hr of individual consultation during this entire time. In school, treatment monitoring was performed by the school principal and affiliated staff who visited random classrooms no less than once per week to ensure delivery of the program. The density of such visits was greater at the beginning of the school year than toward the end. No programmed contingencies were in place for the staff in regard to conducting the AIM programs in their classrooms, but a verbal directive from the principal was in place to implement the program. The weekly classroom visits from administrative staff further supported implied social contingencies to complete the program. No staff were disciplined verbally or in writing for not completing the AIM program. After 5 months of implementation, a 1-hr meeting school-wide with the first author was held to address any feedback, comments, or concerns the teachers had while implementing the AIM curriculum. This meeting was separated by grade level (i.e., sixth, seventh, and eighth grade) so as to ensure that each teacher’s feedback would be specific for their grade’s students and the concerns they may have had.

Preassessment

At the beginning of the school year, teachers administered assessment packets to the students, which included the AFQ-Y, CAMM, and SDQ measures. As noted in the section on materials, students completed these assessments independently, although the classroom teacher read the directions provided with each measure and was available to answer questions if needed. Completed assessment packets were collected and provided to the researchers, who scored and summarized these responses.

AIM ACT Intervention

The intervention was the daily implementation of the AIM curriculum (Dixon & Paliliunas, 2018). The curriculum included 175 ACT lessons, which are bundled into 5-day module themes. Each of the 175 lessons is also presented across three tiers of intensity. Tier 1 is a brief general overview statement and introduces a themed message related to one of ACT’s six psychological flexibility processes. This is often done with a few sentences containing a metaphor or concept related to the flexibility process, and ideas for the child to think about during the school day. Tier 2 expands more on the prior narrative with additional content to promote discussion among students or between the teacher and student. Tier 3 further explores the concepts, often with exploratory directives to write, engage in artwork, have additional discussion, or extend the metaphorical theme in different ways. Figure 1 displays an example of the curriculum content. The general approach of AIM is to deliver Tier 1 to all students as a preventative mindfulness-based approach, whereas Tier 2 and Tier 3 are reserved for students who would be identified as needing additional supports due to a history of aggressive behavior, social and emotional challenges, or related developmental disabilities. However, there is latitude to use the program across the tiers as any teacher or implementer deems relevant to student needs or interests.

Fig. 1.

Example of the AIM curriculum

Using this tiered system, the current study included the presentation of the AIM lessons daily to the students in multiple configurations. Every morning each teacher would discuss the Tier 1 level of the AIM lesson of the day with their students for approximately 5–7 min. This was done as a universal school-wide support and to ensure all the staff and students were exploring the same ACT component and metaphors on any given day. Once the morning AIM announcement was completed, the general education classroom teachers had the option to implement more content of the AIM lesson (i.e., Tier 2 and/or Tier 3) with their students for additional individualized instruction. Special education teachers and those general education teachers who had children identified as at-risk social-emotionally were instructed to also engage in the Tier 2 or Tier 3 activity daily. Total time for this increased intensity of intervention ranged from an additional 5–30 min of AIM daily for such students. Some teachers naturally evolved into more expert roles than others and may have shared ideas, techniques, or approaches on how they were planning on implementing a day’s lesson. Such evolution of the school culture was not attempted to be control or suppressed during this study.

AIM Point System

Classrooms that were required to deliver multiple tiers of AIM’s ACT lessons also implemented the AIM contingency management system. This point-based system contains a series of slight variations on the general theme of positive behavior yielding contingent point delivery at frequent intervals of time throughout the school day. Points were redeemed for function-varying consequences, the specifics of which were determined by each teacher. Attention items may have included options such as being the line leader, participating in show-and-tell, making announcements, having extra time one on one with a peer or teacher, or telling a joke to the class. Escape items may have included being allowed to take a short break from work, putting one’s head down on the desk for a short time, receiving a free homework pass, having the teacher answer a quiz question, or taking one’s shoes off. Tangible consequences might range from small pieces of candy to access to a digital device, or extra leisure time on the computer. Finally, sensory items may have included options such as alternative seating placements or listening to music. The point system included intervals throughout the school day in which the participants had the opportunity to earn and lose points for adherence to classroom rules and responsibilities, as well as ACT-related behaviors (being flexible, accepting difficulties, remaining in the present moment, and attending to values), as defined by the classroom teacher. Point sheets allowed for the recording of points earned and lost, space to describe the behaviors, a summary of points, and a balance of points. Points were exchanged for reinforcers at scheduled intervals each day. These point sheets allowed the teachers of the included classrooms to continually monitor behavior and provide opportunities for reinforcement and feedback. The school-based point sheet would typically remain with the student throughout the day as they traveled to multiple classrooms and was completed by the relevant staff at the end of each time interval. The point system and reinforcer delivery were managed by the classroom teachers. Although a review of the point system framework is provided here because a small subset of participants in the present study were exposed to it, data regarding points are not included in the analysis.

Postassessment

At the end of the school year, teachers readministered the assessment packets to the students, which included the AFQ-Y, CAMM, and SDQ measures. Completed packets were provided to the researchers, who scored and summarized these responses. Only responses from participants who completed both the pre- and postassessments were included in the data analysis.

Results

Participant pretest and posttest scores for the AFQ-Y are summarized in Fig. 2. The results show a significant decrease in total score, t(317) = 7.932, p < 0.001, pretest: M = 20.05, SD = 12.52; posttest: M = 14.55, SD = 12.17, suggesting that the participants reported greater psychological flexibility after a year of the AIM curriculum. Participant pretest and posttest scores for the CAMM are summarized in Fig. 3. The results show a significant increase in total score, t(318) = 4.281, p < 0.001, pretest: M = 27.25, SD = 8.49; posttest: M = 29.53, SD = 8.40, suggesting student reports of greater mindfulness after a year of the AIM curriculum. Participant pretest and posttest scores for the SDQ-Difficulties subsection can be found in Fig. 4. The results reveal a nonsignificant difference in participant scores, t(310) = 1.850, p = 0.065, pretest: M = 11.78, SD = 6.315; posttest: M = 12.05, SD = 6.604. Pretest and posttest scores for the SDQ-Prosocial subsection can be found in Fig. 5. The results show a nonsignificant change, t(312) = 1.041, p = 0.296, pretest: M = 7.451, SD = 1.940; posttest: M = 7.564, SD = 2.293. Participant pretest and posttest scores for the SDQ-Hyperactivity subsection show a nonsignificant change, t(310) = 1.572, p = 0.117, pretest: M = 4.220, SD = 2.410; posttest: M = 4.320, SD = 2.342. Participant pretest and posttest scores for the SDQ-Emotional subsection show a nonsignificant change, t(311) = 0.1223, p = 0.902, pretest: M = 3.310, SD = 2.560; posttest: M = 3.203, SD = 2.563. Participant pretest and posttest scores for the SDQ-Conduct subsection show a nonsignificant change, t(311) = 0.8960, p = 0.371, pretest: M = 1.880, SD = 1.816; posttest: M = 1.920, SD = 1.851. Participant pretest and posttest scores for the SDQ-Peers did produce a significant change, t(310) = 2.570, p = 0.010, pretest: M = 2.367, SD = 1.928; posttest: M = 2.634, SD = 2.057.

Fig. 2.

Scatter dot plot of participant avoidance and fusion questionnaire for youth (AFQY) total score. Note. Markers represent scores for each participant, bars represent the mean, and whiskers represent the standard deviation

Fig. 3.

Scatter dot plot of participant child and adolescent mindfulness measure (CAMM) total score. Note. Markers represent scores for each participant, bars represent the mean, and whiskers represent the standard deviation

Fig. 4.

Scatter dot plot of participant strength and difficulties questionnaire–difficulties (SDQ-Difficulties) total score. Note. Markers represent scores for each participant, bars represent the mean, and whiskers represent the standard deviation

Fig. 5.

Scatter dot plot of participant strength and difficulties questionnaire–prosocial (SDQ-Prosocial) total score. Note. Markers represent scores for each participant, bars represent the mean, and whiskers represent the standard deviation

Standardized assessment performance was obtained for analysis through publicly available data from the state board of education for the state in which the study occurred. The school district included in this study administered the Partnership for Assessment of Readiness for College and Careers (PARCC) assessment in mathematics and English language arts (ELA; Illinois State Board of Education, 2020). The state board of education provides a “report card” in which the percentage of students whose performance is coded as not meeting performance, partially meeting performance, approaching performance, meeting performance, and exceeding performance in each of the two subjects is summarized. Using these data, the percentage of students meeting or exceeding expected performance can be compared across schools, districts, and states. Mathematics scores were obtained for the school exposed to the AIM curriculum; scores for the target school were compared to scores of the district in which it was contained and the summary of student performance at the state level, which can be seen in Fig. 6, as well as Table 1. In order to compare performance across time, scores from the year during which the intervention occurred at the target school and the prior 2 school years are reported.

Fig. 6.

State, district, and school state math testing results depicted from the publicly available data. Note. The AIM intervention occurred in 2017

Table 1.

State math testing score for 2 years prior (2015), 1 year prior (2016), and postintervention (2017) for the intervention school, their school district, and the state

Year, location	Not meeting performance	Partially met performance	Approached performance	Met performance	Exceeded performance
2015, school	17%	30%	31%	22%	1%
2016, school	19%	30%	28%	21%	2%
2017, school	17%	25%	29%	27%	3%
2015, district	31%	23%	28%	18%	1%
2016, district	17%	29%	30%	21%	2%
2017, district	16%	25%	30%	26%	2%
2015, state	15%	28%	29%	25%	3%
2016, state	16%	26%	28%	27%	4%
2017, state	16%	24%	29%	27%	5%

For state math scores 2 years prior for the entire state, 43% of all students either did not meet or partially met their performance level, whereas 57% approached, met, or exceeded their performance level. The state math testing outcomes for the school district show that 54% of all students either did not meet or partially met their performance level, whereas 47% approached, met, or exceeded their performance level. In the 2 years before the implementation of the AIM curriculum, the intervention school’s state math outcomes were as follows: 47% of all students either did not meet or partially met their performance level, whereas 54% approached, met, or exceeded their performance level. Testing 1 year prior resulted in the following state math testing scores for the entire state: 42% of all students either did not meet or partially met their performance level, whereas 59% approached, met, or exceeded their performance level. The state math testing outcomes for the school district show that 46% of all students either did not meet or partially met their performance level, whereas 53% approached, met, or exceeded their performance level. In the 2 years before the implementation of the AIM curriculum, the intervention school’s state math outcomes show that 49% of all students either did not meet or partially met their performance level, whereas 51% approached, met, or exceeded their performance level. Postintervention results of the state math testing scores for the entire state were as follows: 42% of all students either did not meet or partially met their performance level, whereas 61% approached, met, or exceeded their performance level. The state math testing outcomes for the school district show that 41% of all students either did not meet or partially met their performance level, whereas 58% approached or met their performance level. After the implementation of the AIM curriculum, the intervention school’s state math outcomes were the following: 42% of all students either did not meet or partially met their performance level, whereas 56% approached, met, or exceeded their performance level.

Using the same publicly available data, state ELA assessment scores were analyzed for the school exposed to the AIM curriculum as compared to their school district and state performance levels, which can be seen in Fig. 7, as well as Table 2. Two years prior for the entire state’s ELA scores, 34% of all students either did not meet or partially met their performance level, whereas 66% approached, met, or exceeded their performance level. The state ELA testing outcomes for the school district show that 42% of all students either did not meet or partially met their performance level, whereas 58% approached, met, or exceeded their performance level. In the 2 years before the implementation of the AIM curriculum, the intervention school’s state ELA outcomes were the following: 47% of all students either did not meet or partially met their performance level, whereas 53% approached, met, or exceeded their performance level. One year prior, the state ELA testing scores were as follows: 37% of all students either did not meet or partially met their performance level, whereas 64% approached, met, or exceeded their performance level. The state ELA testing outcomes for the school district show that 40% of all students either did not meet or partially met their performance level, whereas 61% approached, met, or exceeded their performance level. In the year before the implementation of the AIM curriculum, the intervention school’s state ELA outcomes were the following: 47% of all students either did not meet or partially met their performance level, whereas 53% approached, met, or exceeded their performance level. After the intervention, the state ELA test outcomes show that 37% of all students either did not meet or partially met their performance level, whereas 64% approached, met, or exceeded their performance level. The postintervention state ELA testing outcomes for the school district show that 36% of all students either did not meet or partially met their performance level, whereas 54% approached, met, or exceeded their performance level. After the implementation of the AIM curriculum, the intervention school’s state ELA outcomes show that 36% of all students either did not meet or partially met their performance level, whereas 64% approached, met, or exceeded their performance level.

Fig. 7.

State, district, and school state English language arts testing results depicted from the publicly available data. Note. The AIM intervention occurred in 2017

Table 2.

State English language arts testing score for 2 years prior (2015), 1 year prior (2016), and postintervention (2017) for the intervention school, their school district, and the state

Year, location	Not meeting performance	Partially met performance	Approached performance	Met performance	Exceeded performance
2015, school	19%	28%	29%	22%	2%
2016, school	19%	28%	29%	22%	2%
2017, school	11%	25%	29%	29%	6%
2015, district	18%	24%	31%	25%	2%
2016, district	18%	22%	29%	28%	4%
2017, district	13%	23%	30%	20%	4%
2015, state	14%	20%	28%	32%	6%
2016, state	16%	21%	28%	31%	5%
2017, state	16%	21%	27%	32%	5%

State math testing scores were analyzed for each grade level that was exposed to the AIM curriculum, which can be seen in Fig. 8 and Table 3. Sixth-grade students’ state math testing scores 2 years prior were as follows: 50% did not meet or partially met their performance level, whereas 49% approached or met their performance level. Seventh-grade students’ state math testing scores 2 years prior show that 51% of all students either did not meet or partially met their performance level, whereas 49% approached, met, or exceeded their performance level. Eighth-grade students’ state math testing scores 2 years prior resulted in the following: 38% of all students either did not meet or partially met their performance level, whereas 62% approached, met, or exceeded their performance level. Sixth-grade students’ state math testing scores 1 year prior resulted in the following: 53% did not meet or partially met their performance level, whereas 47% approached, met, or exceeded their performance level. Seventh-grade students’ state math testing scores 1 year prior show that 43% of all students either did not meet or partially met their performance level, whereas 61% approached, met, or exceeded their performance level. Eighth-grade students’ state math testing scores 1 year prior resulted in the following: 50% of all students either did not meet or partially met their performance level, whereas 49% approached, met, or exceeded their performance level. After the implementation of the AIM curriculum, the sixth-grade state math testing scores were the following: 45% of all students either did not meet or partially met their performance level, whereas 56% approached, met, or exceeded their performance level. After the implementation of the AIM curriculum, the seventh-grade state math scores were the following: 45% either did not meet or partially met their performance level, whereas 55% approached, met, or exceeded their performance level. After the implementation of the AIM curriculum, the eighth-grade state math scores were the following: 35% either did not meet or partially met their performance level, whereas 65% approached, met, or exceeded their performance level.

Fig. 8.

Individual grade level state math testing results for the AIM school depicted from the publicly available data. Note. The AIM intervention occurred in 2017

Table 3.

State math testing score for 2 years prior (2015), 1 year prior (2016), and postintervention (2017) for the intervention school grades 6, 7, and 8

Year, grade	Not meeting performance	Partially met performance	Approached performance	Met performance	Exceeded performance
2015, 6th grade	16%	34%	26%	23%	0%
2016, 6th grade	21%	32%	26%	19%	2%
2017, 6th grade	18%	27%	28%	27%	1%
2015, 7th grade	16%	35%	34%	14%	1%
2016, 7th grade	13%	30%	36%	21%	1%
2017, 7th grade	15%	30%	30%	21%	4%
2015, 8th grade	18%	20%	32%	28%	2%
2016, 8th grade	22%	28%	24%	22%	3%
2017, 8th grade	18%	17%	29%	33%	3%

State ELA testing scores were analyzed for each grade level that was exposed to the AIM curriculum, which can be seen in Fig. 8 and Table 4. Sixth-grade students’ state ELA testing scores 2 years prior resulted in the following: 46% either did not meet or partially met their performance level, whereas 54% approached, met, or exceeded their performance level. Seventh-grade students’ state ELA testing scores 2 years prior were the following: 47% either did not meet or partially met their performance level, whereas 54% approached, met, or exceeded their performance level. Eighth-grade students’ state ELA testing scores 2 years prior resulted in the following: 47% either did not meet or partially met their performance level, whereas 53% approached, met, or exceeded their performance level. Sixth-grade students’ state ELA testing scores 1 year prior resulted in the following: 47% either did not meet or partially met their performance level, whereas 53% approached, met, or exceeded their performance level. Seventh-grade students’ state ELA testing scores 1 year prior were the following: 43% either did not meet or partially met their performance level, whereas 48% approached, met, or exceeded their performance level. Eighth-grade students’ state ELA testing scores 1 year prior resulted in the following: 55% either did not meet or partially met their performance level, whereas 46% approached, met, or exceeded their performance level. After the implementation of the AIM curriculum, the sixth-grade state ELA testing scores were the following: 36% either did not meet or partially met their performance level, whereas 55% approached, met, or exceeded their performance level. After the implementation of the AIM curriculum, the seventh-grade state ELA scores were the following: 35% either did not meet or partially met their performance level, whereas 66% approached, met, or exceeded their performance level. After the implementation of the AIM curriculum, the eighth-grade state ELA scores were the following: 37% either did not meet or partially met their performance level, whereas 63% approached, met, or exceeded their performance level Fig. 9.

Table 4.

State English language arts testing score for 2 years prior (2015), 1 year prior (2016), and postintervention (2017) for the intervention school grades 6, 7, and 8

Year, grade	Not meeting performance	Partially met performance	Approached performance	Met performance	Exceeded performance
2015, 6th grade	16%	30%	35%	18%	1%
2016, 6th grade	15%	32%	34%	17%	2%
2017, 6th grade	10%	26%	25%	28%	2%
2015, 7th grade	24%	23%	28%	23%	3%
2016, 7th grade	13%	30%	36%	21%	1%
2017, 7th grade	11%	24%	26%	28%	12%
2015, 8th grade	16%	31%	25%	26%	2%
2016, 8th grade	27%	28%	20%	24%	2%
2017, 8th grade	12%	25%	28%	32%	3%

Fig. 9.

Individual grade level state English language arts testing results for the AIM school depicted from the publicly available data. Note. The AIM intervention occurred in 2017

Discussion

The present investigation into the potential impact of the AIM curriculum in a school building appears positive. Students reported improvements in psychological flexibility and mindfulness at the end of the 1 year of the AIM program. Additionally, school-wide state testing scores appeared to improve during the year of AIM, when compared to the school’s own data during the 2 years prior to the intervention. Mixed results were found on emotional and behavioral indices, where the reported gains tended to be limited to the domain of peer relationships. These are the first published data on the efficacy of the AIM program and suggest that possible benefits can come to school entities that embark on the implementation of the program. These data add to the existing literature that supports the use of behavior-analytic interventions for school children (Barrish et al., 1969; Parsonson, 2012) and contributes a novel value of ACT as the primary method of treatment. Furthermore, the present study implies that it may be possible for school district personnel to successfully carry out a rather conceptually deep behavioral program with minimal training and ongoing support. This last point speaks to the accessibility level of the AIM curriculum and adoptability by persons not formally trained in the behavioral sciences. Thus, it remains possible that AIM could be implemented in a variety of educational settings where practitioners who specialize in ACT, applied behavior analysis, or mindfulness may be scarce, and still produce positive outcomes at both the individual-student and school-wide levels.

After implementation of the AIM curriculum, participants demonstrated a statistically significant increase in psychological flexibility and mindfulness. Psychological flexibility has been suggested to be a protective factor against mental health disorders that are common in young people, such as mood and anxiety disorders, substance abuse disorders, and eating disorders (Coyne et al., 2011). Successful training in increasing one’s psychological flexibility might be essential for combating the rising prevalence of mental health disorders in young people, and doing so in a school setting provides an opportunity to implement such treatments as large-scale interventions. The AIM intervention also increased student levels of mindfulness and thus may provide both positive social and emotional outcomes, as well as positive academic performance. Increased levels of mindfulness have been associated with reduced stress and anxiety, including test anxiety (Cho et al., 2016), and improved academic self-efficacy and academic performance (Sampl et al., 2017). A curriculum such as AIM that addresses and affects both psychological flexibility and mindfulness in positive ways may be of value to support the social-emotional development of children and adolescents and provides a practical application for schools to implement without formal training in ACT or mindfulness.

When evaluating the effects on the participants’ emotional behavior skills, as assessed by the SDQ, there was only one subsection that resulted in significant changes pre- and postintervention. Participant scores on the peers subsection had a significant change, suggesting that after the implementation of the AIM curriculum, the participants reported having greater interaction, better relationships, and more engagement with their peers than prior to the intervention. One possible reason as to why the other subsections of the SDQ failed to result in any significant changes could be that after engaging in ACT sessions, the participants may have become more aware and accepting of their difficulties, resulting in nonsignificant changes in their self-reports. Bach and Hayes (2002) noted that patients reported higher levels of symptoms in the ACT conditions than control following the ACT intervention, and explained this phenomenon as participants becoming more accepting of symptoms that occurred would presumably also be more likely to acknowledge those symptoms rather than deny them. This phenomenon could help explain why the students’ results on the SDQ were nonsignificant postintervention; however, it is possible that this intervention simply did not have an effect on students’ impressions of these domains. Future research should explore further how increased levels of acceptance could affect individual self-report measures, as well as the effects of an AIM intervention on the SDQ with greater experimental control.

When the state mathematics and ELA assessment scores were compared, visual analysis suggests that those in the school receiving the AIM curriculum had performance improvements during the year of intervention. These positive changes were present when comparing the school to itself during prior years, as well as how the school was performing compared to the state. These results suggest that the AIM curriculum may possibly be related to the students’ improved performance on tests as compared to previous years. However, a causal relationship of the impact of the independent variable on this dependent variable was not possible in this rather naturalistic study. Future research will need more controlled examinations such as treatment versus control groups, or treatment versus control schools. Any such exploration will be riddled with methodological shortcomings, as in the present study, but over time and successive replications, such limitations become weakened. What is unique about the present results is that no modifications were made to the delivery of class content, and the intervention had no references to class or state testing content. These results suggest the possibility that by targeting verbal behavior in terms of ACT processes and mindfulness techniques through the AIM curriculum, students might perform better academically. Thus, when districts or individual schools are faced with strategic planning on how to improve less than optimal achievement scores, proposed interventions might expand beyond pure curriculum modification and also include paying attention to psychological flexibility and mindfulness.

Due to the large size of the study, the unit of analysis remained at the macro level of school change. A comparison of individual student scores was not conducted in part due to confidentiality, so the degree of change on any dependent measure for any specific student remains unknown. Additionally, there was no separation between AIM tiers in the current data, so it is unknown how the implementation of the different tiers affected the changes in certain student scores. Future studies can address this by implementing the AIM curriculum with a smaller population and evaluating the effects on those individual scores and tier levels. Additional effects of the AIM curriculum should be explored to examine the impact on measures such as GPA, conduct referrals, and more social-emotional scales. Follow-up data on the maintenance of these results are also needed to ensure the long-lasting benefits of the AIM curriculum, as well as generalization measures to ensure the adaptability of the program to student life outside of the school setting.

A final limitation of the current evaluation is the lack of frontline staff and teacher evaluations on job performance, well-being, and implementation of the curriculum. Teacher differences in implementing the lessons may have also affected the present data, as each teacher had an uncontrolled degree of latitude on approach, along with no discipline contingencies for noncompliance. It was by chance the staff all tended to deliver the AIM program throughout the year as reported by the school principal during routine monitoring, but this may be the product of implied social contingencies, something that not every school building will necessarily contain. These specific limitations of the current study reduce the internal validity of our results, yet they also provide greater external validity by allowing for the natural classroom environment to influence outcomes. Future work will surely need to enhance treatment fidelity, performance monitoring, and staff satisfaction, which when provided will increase confidence in the procedures described herein. Examinations of daily treatment durations and tier usage will also address important unanswered questions about the AIM program.

There remains a myriad of academic and social challenges for school-age children in the United States. Our results suggest that teacher implementation of AIM might have the potential to improve these domains of difficulty for students and further support the role that behavior-analytic interventions can have in school systems. Given the rise in popularity of ACT within and beyond the behavior-analytic community, it is important that this increased attention also produces increased research support. Our study on AIM is far from the final word on overall superiority in the plethora of curriculum choices for social and emotional learning; however, the results here are promising, as an intervention is only as good as its ability to be delivered within the existing ecosystem for which it was designed. For now, the utility of AIM appears hopeful, and the improvements for the children served were a welcome outcome.

Declarations

Conflict of interest

Mark R. Dixon and Dana Paliliunas receive small royalties from the sales of the AIM curriculum.