Real world executive functioning for autistic children in school and home settings

Abstract

Executive function challenges are commonly reported in the home setting for children with an autism spectrum disorder diagnosis (hereafter, autism), but little is known about these challenges in the school setting. A total of 337 youth (autism N=241, typically developing [TD] N=96) were assessed using Behavior Rating Inventory of Executive Function (BRIEF) ratings from home and school settings. Within each setting, we examined differences in specific executive function skills between diagnostic groups. Then, we examined if the autism group showed similar peak executive function impairments, associations with age, and relationships with adaptive behavior across settings. Finally, we examined interrater reliability. Autism and TD groups differed on all BRIEF scales in both settings. The Shift scale was the peak impairment in the autism group in both settings. There was also an effect of age on executive function impairment in both settings, and executive function ratings in both settings significantly predicted individual adaptive behavior domains. Interrater correlations for autistic participants were similar to interrater reliability correlations from the BRIEF standardization sample. This study shows that autistic children experience similar but not identical real world executive function challenges across school and home settings and that supports may vary by setting.

Executive functioning skills are critical for children to be able to regulate thoughts, emotions, and actions, and consequently adaptive and academic behaviors and outcomes. These skills consist of the abilities to store information in working memory, flexibly shift focus, and inhibit irrelevant responses and are often impaired in children with an autism spectrum disorder diagnosis (hereafter, autism) (Blair, 2016). Historically, executive function impairment has been thought of as a pivotal weakness explaining all of autism, and more recently as playing a role in cognitive and behavioral inflexibility (Geurts, Corbett, & Solomon, 2009; Kenworthy et al., 2008). Executive function is a critical predictor of outcomes for autistic individuals, including academic readiness and functioning (Pellicano et al., 2017; St. John, Dawson, & Estes, 2018), adaptive functioning (Bertollo & Yerys, 2019; Pugliese et al., 2015), and independence and quality of life as an adult (Bishop-Fitzpatrick et al., 2016).

Executive function impairment in autistic children¹ is shown reliably regardless of whether you measure executive function in the lab with a performance-based test or through caregiver ratings of real world performance (Lai et al., 2017; Wallace et al., 2016). Accordingly, extant research has continued to use both performance-based and informant ratings to examine executive function impairment in autism. Further, recent meta-analyses support executive function impairments in autism across multiple domains, such as working memory, flexibility, and planning (Demetriou et al., 2018; Lai et al., 2017).

Most of the studies on real world executive function impairment have used caregiver informant measures, which capture behavior from the home setting and potentially other community settings (hereinafter, “home setting” for simplicity; Wallace et al., 2016). These studies have noted a hallmark difficulty with shifting attention or deviating from routines (Granader et al., 2014; Rosenthal et al., 2013). For example, caregivers of autistic children may report that their child gets stuck on certain topics or activities (Gioia, Isquith, Guy, & Kenworthy, 2000). While broad executive function impairments characterize ASD, shifting has been proposed as a promising clinical marker from caregiver informant data given its notable ability to differentiate autistic and typically developing individuals (Leung & Zakzanis, 2014). Moreover, caregiver informant studies report that real world executive function challenges among autistic children increase with age and associate with adaptive behavior weaknesses across the range of intellectual functioning (Bertollo & Yerys, 2019; Pugliese et al., 2015; Rosenthal et al., 2013; Wallace et al., 2016). Related studies have also indicated that the association between executive function and adaptive behavior may be especially strong for autistic girls and be an important intervention target for this group (Torske, 2020; White et al., 2017).

While these caregiver ratings provide invaluable information about behavior in the home setting, they are unable to answer questions about or establish convergence with executive function in the school setting. Although a large sample of autistic children have performed worse than non-autistic children on performance-based measures of executive function given in the school setting (e.g. inhibition, working memory, and affective decision making; Kouklari, Tsermentseli, & Monks, 2018), information on informant measures of real world executive function in the school setting is still preliminary. To our knowledge, only one prior study has examined real world executive function for autistic children in the school setting, and it was limited by a small sample without a non-autistic comparison group (Freeman et al., 2017).

This is an important knowledge gap as some authors have critiqued performance-based measures as poor estimates of executive functioning in the real world, and informant report measures can provide useful complementary information when assessing executive function (Isquith, Roth, & Gioia, 2013). Qualifying convergence across executive function ratings from home and school settings is also needed as prior research shows poor interrater reliability between home and school ratings of child behavior (Achenbach et al., 2005; Azad & Mandell, 2016; Kaat & Lecavalier, 2015; Mitsis et al., 2000; Power et al., 1998; Stratis & Lecavalier). Furthermore, the school setting is unique due to a higher level of structure, which has the potential to either mitigate or exacerbate executive function impairments. Indeed, executive functioning relates to school readiness in autism (St. John et al., 2018), and school is a setting where evaluations and executive function interventions are already being successfully implemented (Kenworthy et al., 2014; Tamm et al., 2019).

Thus, we have examined informant ratings of executive function from both school and home settings for school-age autistic children to gain further insight into the stability of impairments and associations with age and adaptive behavior functioning.

This investigation addressed five key questions:

1. Do autistic and typically developing (TD) children differ on executive function ratings across home and school settings? Given robust differences identified in the home setting (Leung & Zakzanis, 2014; Wallace et al., 2016), we hypothesize this finding in both settings.

2. Do raters in the school setting identify Shift skills as a peak impairment in autistic children? This is the case for caregiver ratings in the home setting (Granader et al., 2014; Rosenthal et al., 2013), and we hypothesize this finding will extend to the school setting.

3. Does real world executive function impairment increase with age in the school setting? This age-associated change occurs in the home setting (Rosenthal et al., 2013), and we hypothesize it will also occur in the school setting.

4. Do executive function ratings in both settings explain concurrent adaptive behavior ratings in both settings? This has been found in the home setting previously (Bertollo & Yerys, 2019; Pugliese et al., 2015).

5. Are school and home ratings similar or different for autistic children across specific executive function skills (interrater reliability)? The BRIEF manual and meta-analyses on other informant reports (e.g., CBCL) suggest modest interrater correlations (Gioia et al., 2000; Stratis & Lecavalier, 2015). Here, note that the term interrater reliability has been used to refer to rater agreement across settings.

In sum, the present investigation sought to extend previous findings on real world executive functioning in autistic children from the home setting to the school setting and examine similarities and differences between settings. Importantly, such knowledge could advance identification of executive function impairment in autistic children and better inform setting-specific evaluation and interventions.

Method

Participants

A total of 337 participants (autism N=241, TD N=96) between the ages of 6 and 18 were included in the present study, which enrolled children between 2009 and 2015. The inclusion criteria were children between the ages of 6 to 18 years and expert clinical judgment confirming that all children in the autism group met DSM-IV-TR criteria for either autism, Asperger’s syndrome, or pervasive developmental disorder-not otherwise specified (American Psychiatric Association [APA], 2000). Expert clinical best estimates were based on a DSM-IV-TR and DSM-V checklist and informed by the Autism Diagnostic Observation Schedule (ADOS), the Autism Diagnostic Interview—Revised (ADI-R; Lord et al., 1994; Lord et al., 2000), Vineland Adaptive Behavior Scales, 2^nd Edition (Sparrow; Cicchetti, & Balla, 2005), and clinical impressions. Thus, it was possible for some children to not meet criteria on the ADOS but still be included in the ASD group. This method of diagnostic inclusion was used given that it parallels the broad autism diagnostic criteria used by the Collaborative Programs of Excellence in Autism network (Lainhart et al., 2006) and results in a more representative sample of the autistic population at large.

Exclusion criteria were assessed using caregiver report on a family and medical history form. For the autism group, exclusion criteria were extreme prematurity (gestational age < 32 weeks), seizures, tic disorder, or an acquired head injury, and parent report of a genetic variation with known clinical significance (e.g., Fragile X, Down Syndrome). In addition to the exclusion criteria above, children in the TD group were also excluded for taking psychoactive medication and elevated scores on the ADHD Rating Scale-IV ( >85^th percentile based on age- and sex-correction per the manual; DuPaul, Power, Anastopoulos, & Reid, 1998) or the Child and Adolescent Symptom Inventory, Fourth Edition revised (>1.5 standard deviations above the age- and sex-corrected mean per the manual; Gadow & Sprafkin, 2010).

A portion of the sample was reported previously in articles investigating association between parent ratings of executive functioning and adaptive behavior in autistic youth with lower IQ (Bertollo & Yerys, 2019) and the factor structure of the parent BRIEF (Granader et al., 2014). Overall participant characteristics by diagnostic group can be found in Table 1.

Table 1.

Participant characteristics; M(SD); Range

	Autism N=241	TD N=65	p-value	Effect Size (Hedges’ g)
Age (years)	10.1 (3.0) 6–18	10.2 (3.3) 6–18	0.88	0.02
Full Scale IQ	92.0 (23.4)a 30–158	109.1 (15.7) 79–155	p<0.001*	0.78
Gender Ratio (M:F)	217:24	54:11	0.18	-
Race Ratio (“White”: “Black”: “Other”)	200:20:13b	57:2:6c	0.43	-
Ethnicity Ratio (“Not Hispanic”: “Hispanic”)	206:20d	53:2e	0.31	-
ADHD Rating Scale-IV: Home	26.3 (12.3)f 0–53	9.0 (8.0)g 0–28	p<0.001*	1.50
ADHD Rating Scale-IV: School	19.8 (11.6)h 0–53	7.1 (7.5)i 0–27	p<0.001*	1.17
ADOS CSS	7.3 (2.1)j 1–10	-	-	-
ADI-R Total Diagnostic	3.8 (1.0)k 1–5	-	-	-
ADI-R Reciprocal Social Interaction	19.8 (6.4)l 0–30	-	-	-
ADI-R Non-Verbal Communication	8.5 (3.6)l 0–14	-	-	-
ADI-R Verbal Communication	15.4 (4.8)l 0–25	-	-	-
ADI-R Restricted Repetitive Behavior	6.0 (2.6)l 0–12	-	-	-

ADOS CSS=Autism Diagnostic Observation Schedule Calibrated Severity Score;

ADI-R=Autism Diagnostic Interview - Revised

^an=233;

^bn=8 not reported;

^cn⁼0 not reported;

^dn=15 not reported;

^en=10 not reported;

^fn=227;

^gn=91;

^hn=237;

ⁱn=95;

^jn=235;

^kn=178;

^ln=230

Measures

Executive function.

Ratings of executive function in the school and home setting were measured using the teacher and parent versions of the Behavior Rating Inventory of Executive Function (BRIEF), respectively (hereinafter, “School BRIEF” and “Home BRIEF”) (Gioia et al., 2000). For school, families chose the teacher or school professional who knew the child best as the rater, and for home, families chose the caregiver who knew the child best as the rater. The BRIEF captures executive function in everyday situations and is comprised of 86 items that fall into eight scales which are collapsed into an overall General Executive Composite (GEC), which has two underlying broad indices: the Behavioral Regulation Index (BRI) and the Metacognition Index (MI). The BRI includes Inhibit, Shift, and Emotional Control scales, and the MI includes the Initiate, Working Memory, Plan/Organize, Organization of Materials, and Monitor scales. Items have a three-point Likert scale rating, including 1 (never), 2 (sometimes), and 3 (often). Higher raw and T-scores scores indicate greater impairment; T-scores≥65 (i.e., 1.5 SDs≥the mean) indicate clinically significant ratings.

Adaptive Behavior.

Ratings of adaptive behavior in the school and home setting were measured using the teacher and parent versions of the Vineland Adaptive Behavior Scales, 2^nd Edition (VABS-II) survey, respectively (hereinafter, “School VABS-II” and “Home VABS-II”) (Sparrow, Cicchetti, & Balla, 2005). The School VABS-II was added to the study after the School BRIEF once we learned that school professionals did not feel overburdened with the study; thus the School VABS-II has a smaller available sample compared to the School BRIEF. Consistent with rater selection for the BRIEF, families chose the teacher or school professional who knew the child best as the school rater and the caregiver who knew the child as the home rater. Here, standard VABS-II scores (M=100, SD=15) are reported for 3 domains: Communication, Daily Living, and Socialization. Lower standard scores represent worse adaptive skills, where a score between 71 and 85 (SD=1.0–2.0) represents moderately low skill level and a score of 70 or lower (SD≥2.0) indicates low skill level.

Autism characteristics².

The ADI-R is a semi-structured interview with caregivers of children and adults to assess the presence of autism spectrum disorders (Lord, Rutter, & Le Couteur, 1994). The caregiver is asked about their child’s current behavior in daily life, and scores are given according to the degree to which indicated behaviors are present. Scores are given for behaviors in reciprocal social interaction, non-verbal and verbal communication, and restricted and repetitive behaviors domains. For diagnostic criteria for autism to be met, behaviors in each domain must meet criteria and abnormality in at least one domain before 36 months of age must be present.

The ADOS is a semi-structured interview consisting of modules that assess social interaction, communication, play, and imaginative use of materials to determine whether an individual meets the criteria for an autism diagnosis. Scores are given for Social Affect and Restricted and Repetitive Behavior domains, which are then totaled and converted into the CSS. CSS scores range from 1 to 10, with higher scores indicating more pronounced autism characteristics. The calibration severity score (CSS) from the ADOS was used as a covariate for autism characteristics in the present analyses.

ADHD characteristics.

The total raw scores from the ADHD Rating Scale-IV: School Version and ADHD Rating Scale-IV: Home Version were used as covariates for ADHD characteristics in the present analyses (DuPaul et al., 1998). The ADHD Rating Scale-IV is a questionnaire based on the diagnostic criteria for ADHD where each of the nine Inattention and nine Hyperactivity/Impulsivity behaviors are rated on a four-point Likert scale. Caregivers and teachers/school personnel complete the form. Total raw scores can range from 0 to 54, with higher scores indicating more severe ADHD characteristics.

Cognitive ability.

Cognitive ability was also used as a covariate in the present analyses and was assessed with the Differential Abilities Scale, Second Edition (DAS-II; Elliott, 2007) or Wechsler Intelligence Scale for Children, Fourth Edition (WISC-IV; Wechsler, 2003). Each of the assessments has a mean standard score of 100 and a standard deviation of 15 (M=100, SD=15).

Procedure

Participants were recruited from our hospital’s primary care network, specialty clinics, and the community. They were all enrolled and tested at our hospital-based research center between the years of 2009 and 2015 as part of one or more case-control studies approved by the Institutional Review Board at The Children’s Hospital of Philadelphia. During these studies, participants completed various diagnostic, cognitive, and neuropsychological assessments, while the teacher or school professional and caregiver raters completed questionnaires.

The secondary analyses conducted in the present study include all of the participants for whom a School BRIEF was completed. This was to ensure executive function ratings in the school setting could be analyzed. A minority of 38 ASD and 5 TDC eligible participants across studies did not have a School BRIEF completed. For 25 of these ASD participants, IQ scores were lower than 75 and thus study PIs in consult with study clinicians may have elected not to send out Teacher BRIEF forms given that the BRIEF has not been validated in an intellectual disability population. All participants gave informed consent for the use of their data and this secondary analysis study was approved by the Institutional Review Board at The Children’s Hospital of Philadelphia. Community members were not included in the design of the present secondary analyses, however community member involvement would be an important future step.

Analysis Plan

To assess whether the autistic and TD groups differed on executive function ratings across home and school settings, we conducted multivariate analyses of covariance (MANCOVAs) using the School BRIEF and Home BRIEF ratings for the eight sub-scales while controlling for age, gender, IQ, and ADHD characteristics. Given robust differences between autistic and TD groups identified in the home setting, the a priori hypothesis was that findings would be replicated in both settings (Leung & Zakzanis, 2014; Wallace et al., 2016).

Next, we moved onto analyses focused on the autism group. First, we conducted a one-way analysis of covariance (ANCOVA) within the autism group to examine peak impairment across the 8 subscales from School BRIEF and Home BRIEF ratings while controlling for age, gender, IQ, autism characteristics, and ADHD characteristics. The a priori hypothesis was that the Shift scale would have significantly higher scores compared to other scales in autistic participants, consistent with previous observations using Home BRIEF ratings (Granader et al., 2014; Rosenthal et al., 2013).

Next, we examined the effect of age on School and Home BRIEF ratings within the autism group. The a priori hypothesis was that executive function would be rated as more impaired as age increased. Replicating the approach of Rosenthal et al., (2013), a mixed-model ANOVA with age group (5–7, 8–10, 11–13, and 14–18) as the between-subjects factor and BRIEF scales as the within-subjects factor was conducted. Due to the present sample only including children 6 years and older, the youngest age range was adjusted to include children between the ages of 6 and 7. Subsequent one-way ANOVAs and Tukey’s hsd tests were then planned for post-hoc analyses.

Then, to assess whether school setting ratings of executive function in specific areas were predictive of school setting ratings of adaptive behavior in autistic participants beyond participant age, gender, IQ, autism characteristics, gender and autism characteristics interactions, and ADHD characteristics, we conducted a series of linear regression models using School BRIEF and School VABS-II ratings. Parallel linear models were also conducted using Home BRIEF and Home VABS-II ratings. There were 215 autistic participants and 86 TD participants included in analyses requiring School VABS-II ratings. Two TD participants were excluded from analyses requiring Home VABS-II ratings

Finally, we examined interrater reliability of BRIEF ratings in the school and home settings within the autism group by computing intra-class correlation (3,1) coefficients and compared our results to parent-teacher interrater reliability from the BRIEF standardization sample using Fisher’s r-to-z transformations (Table 21; Gioia et al., 2000).

Prior studies in autistic children have not removed participants from the autism or TD group for having an “Inconsistent” score on the inconsistency protocol of the BRIEF (Bertollo & Yerys, 2019; Freeman et al., 2017; Gioia et al., 2002; Granader et al., 2014; Isquith, Roth, & Gioia, 2013; Leung & Zakzanis, 2014; Pugliese et al., 2015; Rosenthal et al., 2013; Wallace et al., 2016). We conducted a secondary analysis removing children with those scores on either the School BRIEF or Home BRIEF for sensitivity analyses (see Tables S3-S6). This excluded 15 autistic participants and 8 TD participants.

Results

Group Characteristics

The autism and TD groups were matched on age, gender ratio, race ratio, and ethnicity ratio, but differed on full scale IQ, and ADHD characteristics (see Table 1). Specific data on socioeconomic status were not included.

Group Differences in Executive Function Ratings in School and Home

The MANCOVAs controlling for age, gender, IQ, and ADHD characteristics revealed significant diagnostic group differences for both School BRIEF and Home BRIEF ratings. (F’s>32, all p<0.001) (See Table 2). All scales were available for the full sample size in each setting, thus indicating no missing items.

Table 2.

BRIEF ratings by diagnostic group and setting (t-scores (M(SD)))

School					Home
	Autism n=241	TD n=65	p-value	Effect Size (g)	Autism n=230	TD n=65	p-value	Effect Size (g)
GEC	66.9 (11.2)	52.2 (10.7)	p<0.001	1.33	66.8 (10.4)	48.7 (13.0)	p<0.001	1.67
BRI	66.8 (13.4)	52.1 (10.8)	p<0.001	1.14	67.0 (11.7)	49.6 (12.5)	p<0.001	1.46
MI	65.0 (10.7)	51.8 (10.4)	p<0.001	1.24	65.1 (10.2)	48.2 (12.3)	p<0.001	1.58
Inhibit	61.0 (11.8)	51.1 (8.1)	p<0.001	0.89	64.1 (12.1)	49.8 (10.7)	p<0.001	1.22
Shift	70.8 (15.8)	53.3 (13.7)	p<0.001	1.13	69.5 (11.1)	50.9 (14.7)	p<0.001	1.48
Emotional Control	65.8 (14.6)	52.5 (11.6)	p<0.001	1.22	61.9 (11.8)	48.8 (10.8)	p<0.001	1.13
Initiate	64.9 (10.8)	51.2 (10.3)	p<0.001	1.27	63.6 (9.8)	48.3 (11.6)	p<0.001	1.50
Working Memory	65.0 (12.4)	51.2 (10.1)	p<0.001	1.14	66.0 (10.8)	49.8 (12.5)	p<0.001	1.44
Plan/Organize	61.6 (10.4)	51.8 (10.7)	p<0.001	0.93	62.5 (11.5)	48.2 (11.0)	p<0.001	1.25
Organization of Materials	60.0 (14.4)	50.8 (10.5)	p<0.001	0.67	57.9 (10.4)	48.5 (10.2)	p<0.001	0.91
Monitor	65.6 (11.5)	52.7 (11.2)	p<0.001	1.13	64.8 (10.8)	48.0 (13.5)	p<0.001	1.46

Peak Impairment in Autism Executive Function Ratings in School and Home

The one-way ANCOVA conducted within the autism group on the School BRIEF revealed a main effect of scale (F(7,1568)=27.53, p<0.0001, η²=0.06) after controlling for age, gender, IQ, autism characteristics, and ADHD characteristics. Subsequent planned pairwise comparisons revealed that Shift was rated significantly higher (i.e. more impaired) than the other seven scales (all p<0.01). Both Emotional Control and Monitor were found to be rated significantly higher than Inhibit, Plan/Organize, and Organization of Materials (all p<0.05), and Initiate and Working Memory were found to be rated significantly higher than Inhibit and Organization of Materials (all p<0.05).

The one-way ANCOVA conducted within the autism group on the Home BRIEF also revealed a main effect of scale (F(7,1505)=34.79, p<0.0001, η²=0.07) after controlling for age, gender, IQ, autism characteristics, and ADHD characteristics. Subsequent planned pairwise comparisons revealed that the Shift scale was rated significantly higher than all other scales (all p<0.05). The Working Memory scale was also found to be rated significantly higher than Emotional Control and Plan/Organize (all p<0.05). In contrast, the Organization of Materials scale was rated significantly lower than all other scales (all p<0.05).

Effect of Age on Autism Executive Function Ratings in School and Home

The mixed-model ANOVA with age group as the between-subjects factor and School BRIEF scale as the within-subjects factor indicated a main effect for age group (F(3,237)=4.82, p<0.01 and a main effect for scale (F(7,1659)=32.36, p<0.0001), as well as an interaction between age group and scale (F(21,1659)=4.03, p<0.0001). In contrast, the Home BRIEF revealed a main effect for scale (F(7,1582)=38.14, p<0.0001) and an interaction between age group and scale (F(21,1582)=2.20 p<0.01). Examination of heteroskedasticity did not suggest any substantial issues given all F-max <3.2.

Subsequent one way ANOVAs using the School BRIEF revealed age effects for the following scales: Shift (F(3,237)=11.05, p<0.0001), Plan/Organize (F(3,237)=6.48, p<0.001), Organization of Materials (F(3,237)=5.48, p<0.01), and Monitor (F(3,237)=3.39, p<0.05) (See Table 3 for post-hoc results).

Table 3.

Age differences in autism School BRIEF ratings

						F-value	p	Post-hoc Testing
	Total	5–7	8–10	11–13	14–18
	n=241	n=45	n=113	n=43	n=40
Inhibit	61.0 (11.8)	57.98 (8.84)	61.54 (11.62)	61.51 (14.72)	62.6 (11.66)	1.34	0.26	ns
								14–18>5–7
Shift	70.8 (15.8)	67.09 (12.35)	67.57 (12.75)	72.21 (14.62)	82.50 (21.91)	11.1	p<0.0001 *	14–18>8–10
								14–18>11–13
Emotional Control	65.8 (14.6)	62.58 (16.14)	65.88 (13.20)	69.56 (15.17)	64.93 (15.61)	1.74	0.16	ns
Initiate	64.9 (10.8)	65.13 (11.22)	63.65 (10.43)	64.0 (8.41)	68.83 (12.85)	2.41	0.07	ns
Working Memory	65.0 (12.4)	64.29 (13.18)	63.87 (12.33)	66.07 (11.08)	67.65 (12.96)	1.08	0.36	ns
								14–18>5–7
Plan/Organize	61.6 (10.4)	57.29 (9.73)	60.83 (10.18)	63.84 (9.26)	66.23 (10.96)	6.48	0.0003*	14–18>8–10
								11–13>5–7
								14–18>5–7
Organization of Materials	60.0 (14.4)	52.87 (10.29)	60.63 (13.0)	64.0 (15.77)	62.15 (17.70)	5.48	0.001*	11–13>5–7
								8–10>5–7
								14–18>5–7
Monitor	65.6 (11.5)	63.47 (10.24)	64.44 (11.09)	66.65 (11.03)	70.33 (13.11)	3.39	0.02*
								14–18>8–10

Subsequent one way ANOVAs using the Home BRIEF revealed age effects only for the Organization of Materials scale (F(3,226)=2.73, p=0.04) and post-hoc analyses using Tukey’s test revealed significantly lower (i.e. less impaired) Organization of Materials scores in the 14–18 year olds compared to the 8–10 year olds (p=0.04) (See Table S1 in Supporting Information).

Autism Executive Function and Adaptive Behavior Ratings in School and Home

The initial linear regression models conducted with the covariates of no interest (age, gender, IQ, and autism characteristics) significantly predicted each of the school-rated adaptive behavior domains: Communication, Daily Living Skills, and Socialization (overall models: F’s>11, all p<0.0001, all R²’s>0.21). Here, we use “covariate of no interest” to reflect extant knowledge of these covariates already accounting for substantial variance, and thus needing to be included in baseline models (Bertollo & Yerys, 2019). In addition, we included an interaction term between gender and autism characteristics given recent research indicating strong relationships between executive function and adaptive behavior for girls on the autism spectrum (Torske, 2020; White et al., 2017). There was a significant change in R² (ΔR²) in each of the School VABS-II domains when School BRIEF scales were added into the models: Communication (F(8,160 )=4.05, p<0.001, ΔR²=0.08), Daily Living Skills (F(8,158)=5.49, p<0.0001, ΔR²=0.12), and Socialization (F(8,176)=9.34, p<0.0001, ΔR²=0.22) (See Table 4). When the baseline model also included ADHD characteristics (overall models: F’s>20, all p<0.0001, all R²’s>0.33), only the Socialization scale had a significant change in R² (ΔR²) when School BRIEF scales were added into the models (F(8,172)=3.84, p<0.001, ΔR²=0.10)

Table 4.

Linear Regressions: Autism participants.

	School VABS-II			School VABS-II			School VABS-II
	Communication			Daily Living Skills			Socialization
	B	SE B	t	B	SE B	t	B	SE B	t
		n=193			n=191			n=209
Model 1 (Covariates):		R2=0.50			R2=0.40			R2=0.22
Age	0.33	0.25	1.32	0.52	0.32	1.59	0.44	0.28	1.62
Gender (Male)	−1.76	8.24	−0.21	−1.28	10.72	−0.12	−5.08	9.03	−0.56
IQ	0.41	0.03	13.30***	0.43	0.04	10.88***	0.21	0.03	6.33***
ADOS CSS	−0.44	1.06	−0.42	−1.04	1.38	−0.76	−1.67	1.13	−1.47
ADOS CSS * Gender	0.04	1.12	0.04	0.71	1.46	0.49	0.52	1.20	0.43
Model 2:		ΔR2=0.08			ΔR2=0.12			ΔR2=0.23
Covariates
School BRIEF
Inhibit	−0.07	0.11	−0.67	−0.12	0.14	−0.86	−0.04	0.10	−0.39
Shift	−0.03	0.07	−0.46	0.00	0.09	0.01	−0.10	0.07	−1.51
Emotional Control	−0.01	0.08	−0.12	0.01	0.10	0.15	−0.07	0.08	−0.87
Initiate	−0.17	0.11	−1.47	−0.10	0.15	−0.72	−0.15	0.11	−1.33
Working Memory	−0.05	0.10	−0.50	−0.15	0.12	−1.23	−0.10	0.10	−1.03
Plan	0.04	0.11	0.39	0.12	0.14	0.85	0.24	0.11	2.10*
Organization of Materials	−0.03	0.08	−0.45	−0.11	0.10	−1.20	−0.09	0.08	−1.23
Monitor	−0.11	0.12	−0.95	−0.24	0.15	−1.62	−0.26	0.11	−2.38*

Note.

^·p ≤ 0.1,

^*p < 0.05,

^**p < 0.01,

^***p < 0.001; The change in R² is significant for Model 2 vs. Model 1 in each adaptive behavior domain. Model 2 includes covariates and the School BRIEF.

Similarly, the initial linear regression models conducted with the covariates of no interest (age, gender, IQ, and autism characteristics) and the gender and autism characteristics interaction term significantly predicted each of the home-rated adaptive behavior domains: Communication, Daily Living Skills, and Socialization (overall models: F’s>12, all p<0.0001, all R²’s>0.20). There was a significant ΔR² in each of the Home VABS-II domains when Home BRIEF scales were added into the models: Communication (F(8,203)=8.11, p<0.0001, ΔR²=0.13), Daily Living Skills (F(8,205)=2.50, p<0.05, ΔR²=0.07), and Socialization (F(8,205)=10.72, p<0.0001, ΔR²=0.23) (See Table S2 in Supporting Information). When the baseline model also included ADHD characteristics (overall models: F’s>12, all p<0.0001, all R²’s>0.24), there was a significant ΔR² in each of the Home VABS-II domains when Home BRIEF scales were added into the models: Communication (F(8,198)=4.30, p<0.0001, ΔR²=0.07), Daily Living Skills (F(8,200)=2.00, p<0.05, ΔR²=0.06), and Socialization (F(8, 200)=6.97, p<0.0001, ΔR²=0.16).

Convergence of Autism Executive Function Ratings in School and Home

For autistic participants, results from ICC (3,1) computations indicated that there was a significant positive association between School and Home BRIEF ratings of GEC (ICC=0.22, p<0.001), BRI (ICC=0.30, p<0.0001), MI (ICC=0.30, p<0.0001), and 7 of the 8 underlying scales (no significant association was found for the Initiate scale): Inhibit (ICC=0.37, p<0.001); Shift (ICC=0.22, p<0.001); Emotional Control (ICC=0.30, p<0.0001); Initiate (ICC=0.04, p=0.27); Working Memory (ICC=0.23, p<0.001); Plan/Organize (ICC=0.25, p<0.0001); Organization of Materials (ICC=0.32, p<0.0001); and Monitor (ICC=0.28, p<0.001). In addition, Fisher’s r-to-z transformations revealed that there were no significant differences between the present correlations and interrater reliability correlations from the standardization dataset in the BRIEF Manual (Table 21; Gioia et al., 2000) for autistic participants, with the exception of the Inhibit scale (p<0.05), Initiate scale (p<0.05), Organization of Materials scale (p<0.01), and Monitor scale (p<0.05) (See Table 5).

Table 5.

Home and school interrater correlations for BRIEF Standardization Sample and Autism Group

Scale	Standardization Sample	Autism	Autism 95% CI	p-value for Standardization vs. Autism
Inhibit	0.50	0.37	0.26–0.47	0.02*
Shift	0.15	0.22	0.09–0.34	0.3
Emotional Control	0.18	0.30	0.18–0.41	0.07
Initiate	0.18	0.04	−0.09–0.17	0.04*
Working Memory	0.30	0.23	0.11–0.36	0.28
Plan/Organize	0.35	0.25	0.13–0.38	0.12
Organization of Materials	0.15	0.32	0.21–0.44	0.01*
Monitor	0.42	0.28	0.16–0.40	0.02*
Behavioral Regulation	0.31	0.30	0.18–0.42	0.87
Meta-cognition	0.34	0.26	0.13–0.37	0.21
Global Executive Composite	0.34	0.22	0.10–0.35	0.06

Discussion

This investigation currently provides the most comprehensive assessment of real world executive function in the school setting for children on the autism spectrum using a large pediatric sample. This is critical as school is a setting with a fairly rigid structure that may provide both unique executive function challenges and supports for autistic children. We addressed five key questions about real world executive function using informant measures in the school setting, and thus enhanced prior knowledge from performance-based tests (Isquith et al., 2013). For question 1 (diagnostic group differences), we found that all executive function ratings in the School BRIEF and Home BRIEF differed between autism and TD groups. For question 2 (Shift as a peak impairment), we found the Shift scale to be rated as a peak impairment for the autism group in both the school and home settings. For question 3 (executive function impairments increase with age), we found that individual executive function skills generally showed a widening gap between autism and TD groups across age in the school setting, but not the home setting. For question 4 (association with adaptive behavior functioning), we found that School BRIEF and Home BRIEF ratings were predictive of adaptive behavior ratings in the corresponding setting for the autism group. For question 5 (interrater reliability), we found significant correlations for all scales on the BRIEF but Initiate in the autism group. We also found no significant differences between present correlations in the autism group and interrater reliability correlations from the BRIEF standardization sample on Shift, Emotional Control, Working Memory, Plan/Organize, but not the Inhibit, Initiate, Organization of Materials, and Monitor scales. This finding may indicate that autistic children have more variable presentation across settings compared to the neurotypical population or that the latter four scales may yield less reliable ratings across settings than what is found in the neurotypical population. Thus, the present study was able to extend findings of executive function impairments in autism from the home setting to the school setting while also revealing similarities and differences across settings.

Our findings that real world executive function is impaired in the school setting for autistic children and that Shifting abilities are a peak impairment converge with prior studies in the home setting (Gioia, Isquith, Kenworthy, & Barton, 2002; Granader et al., 2014; Rosenthal et al., 2013; Smithson et al., 2013; Yerys et al., 2009). In addition, our findings that executive function impairments increase in older age groups in the school setting also converge with prior findings from the home setting (Rosenthal et al., 2013).

However, Shift, Plan/Organize, Organization of Materials, and Monitor scales were rated as more impaired in older age groups in the school setting in the present study whereas only the Initiate and Working Memory scales have previously been rated as more impaired in older age groups in the home setting (Rosenthal et al., 2013). Moreover, Rosenthal and colleagues’ (2013) study did not reveal the Organization of Materials scale to increase in older age groups in the home setting. These differences may be partially due to differing cognitive abilities across studies. Rosenthal and colleagues’ (2013) study only included autistic participants with an IQ score above 70 whereas the present study included a broad IQ range from 30 to 158. Alternatively, the differences may be due to Rosenthal et al. (2013) using ratings from both clinic and research evaluations. It is possible caregivers may rate behaviors differently in a clinical setting prior to obtaining a diagnosis than in a research setting where diagnosis and care has already been established. Nonetheless, the present results from the school setting extend findings that some executive function abilities in autistic children may have greater separation from the standardization sample at older ages, consequently contributing to a more noticeable divergence with neurotypical peers during adolescence (Rosenthal et al., 2013).

The present finding that the autism group’s executive function ratings in the home and school settings predict corresponding adaptive behavior ratings in Communication, Daily Living Skills, and Socialization domains also extends prior research. This study is the first to show this relationship between executive functioning and adaptive behavior in the school setting, but both concurrent and longitudinal prediction studies have demonstrated this relationship in the home setting (Bertollo et al., 2019; Gilotty et al., 2002; Pugliese et al., 2015). While prior studies have indicated that the relationship between executive function and adaptive behavior may be stronger for autistic females compared to autistic males, the present study found executive function in specific areas to be predictive of school setting ratings of adaptive behavior in autistic participants beyond gender and a gender and autism characteristics interaction (Torkse, 2020; White et al., 2017). However, the limited number of females and binary measure of gender in the current sample should be considered. Overall, these findings reveal that caregivers and school professionals are observing similar real world executive function profiles and related impairments in autistic children.

However, the addition of ADHD characteristics as a covariate led to executive function ratings only predicting adaptive behavior ratings of socialization in the school setting and all adaptive behavior ratings in the home setting. This pattern of findings indicates that there may be an aspect of ADHD characteristics that is unique from executive function ratings and differentially associated with adaptive behavior weaknesses in the context of ASD. Understanding the relationship of ADHD and adaptive functioning in both autistic and non-autistic children could further inform differential diagnosis among ASD, ADHD, and cooccurring ASD and ADHD diagnoses. Further, increased knowledge of distinct diagnostic profiles observed in the school setting may be useful for improving school-based evaluations. Future work is needed to understand what psychological, environmental, and biological factors influence this variance in adaptive behavior ratings across settings for youth on the autism spectrum.

Correspondence between executive function ratings in autism from the home and school settings was also supported by significant positive correlations between School and Home BRIEF ratings of the overall GEC, underlying BRI and MI, and seven of the eight scales. Furthermore, interrater reliability correlations from the standardization sample in the BRIEF Manual (Table 21; Gioia et al., 2000) only significantly differed from our correlations for autistic participants on the Inhibit, Initiate, Organization of Materials, and Monitor scales, indicating that reliability on the Shift, Emotional Control, Working Memory, and Plan/Organize scales aligns with what is reported from the standardization sample while the others may reflect a potential heightened difficulty of assessing executive function reliably across settings in autism.

This initial look at interrater reliability is an important step in demonstrating the utility of school ratings of the BRIEF for the autism population (See Shum et al., 2020 for similar work with an sample with increased likelihood for ADHD in China). It also contributes to an understanding of whether the BRIEF could be used in school settings for early identification of executive function impairment in autistic children and informing individualized, setting-specific interventions. Prior work indicates that examining executive functioning as a precursory step to autism diagnosis and intervention may be particularly useful for females on the autism spectrum (Torske, 2020; White et al., 2017). Further, these individualized, setting-specific interventions may also be generally needed for autistic children due to a higher likelihood for their executive function impairments to vary across settings.

While this study was strengthened by a large sample size well-characterized for autism and having both caregiver and school professional ratings, it should be noted that there were several limitations. All findings are primarily relevant to Non-Hispanic white males given their overrepresentation in the study sample. Further, results may have been influenced by collinearity between autism criteria and the Shift scale, both of which involve ability to tolerate change, make transitions, and flexibly problem-solve. Findings regarding age-related changes are also limited by the absence of longitudinal data, and thus future longitudinal studies would be a needed next step.

Information regarding both caregiver and school professional raters was limited. While caregivers were largely parents that a child lived with more than 50% of the time and school professionals were typically the teacher for a child during their study participation, detailed information about raters’ interaction with a child was not collected. Future research would benefit from analyzing whether patterns of rater and child interaction are related to ratings of executive function impairment. This analysis may be particularly relevant to examine impact of time of school year and length of rater-child relationship on executive function ratings. Further, such analysis could determine whether the age effects observed on the School BRIEF in the present study are related to younger children spending more time in the same classroom and older children transitioning between classrooms throughout the day.

Further, information regarding type of classroom (e.g. special education versus mainstream) and other reference points, such as number of other children with an autism diagnosis, was not analyzed. This information may need to be considered in future research to understand if there are systematic differences among various school professionals and between school professionals and caregivers with respect to their reference points in ratings. In addition, the present study did not explore whether co-occurring conditions other than attention deficit/hyperactivity disorder, such as anxiety, depression, or oppositional defiant disorder influenced scores in each setting. Evaluating the effect of more co-occurring conditions would be important to explore in a future study to understand potential moderating effects.

The study also did not include performance-based or neural measures of executive function to examine convergence with informant ratings. Future research could benefit from exploring whether home or school ratings of executive function are more strongly associated with these other units of analysis of executive function in an autistic sample, as recent work indicates stronger relationships with school ratings of executive function for non-autistic children (Shum et al., 2020). In addition, future analyses would benefit from extending these findings into the BRIEF’s second edition (BRIEF-2) and evaluating measurement invariance in autism for school professional ratings. Examining these questions could further demonstrate the utility of using the BRIEF to measure real world executive function in the autism population.

Conclusion

This study is the current most comprehensive examination of real world executive function for autistic children in the school setting. These findings extend previous research on impaired executive function skills in school-age autistic children by showing that a similar but not identical profile is seen for real world executive function skills in the school setting as at home. Our findings of diagnostic group differences in executive function and peak Shift impairments in both the school and home settings bolster the hypotheses that many autistic children have impaired executive function and Shifting as a primary area of weakness (Geurts, Corbett, & Solomon, 2009). Moreover, our findings that individual executive functioning skills were worse in older age groups and that executive functioning skills predicted adaptive behavior in the school setting illustrate that executive function impairment in school is a crucial target for early identification and intervention efforts. The fact that nearly all scales on the BRIEF were found to be significantly correlated across rater type suggests that it may be a valid and useful informant measure for identifying executive function impairment in autistic children and informing individual interventions at school.

Taken together, these findings are even more critical because there is already a growing interest to intervene on executive function in the school setting due to its demands (Kenworthy et al., 2014; Tamm et al., 2019) but the evidence base of real world executive function impairments in autism has not yet included ratings from school professionals for a large, controlled pediatric sample. Advancing our foundational knowledge of real world executive functioning profiles across school and home settings can drive the field to develop more targeted prevention and intervention strategies that take setting into account.