Provider Self-Reported Use and Usefulness of Intervention Strategies for Externalizing Behaviors in Youths with ASD

Abstract

Objective:

To examine predictors of implementation and perceived usefulness of four empirically supported strategies for treating externalizing behavior in youths with ASD.

Method:

Participants were 557 providers in the United States with experience treating externalizing behavior in youths with ASD. Generalized estimating equations were used to determine whether self-reported use and usefulness of four empirically supported intervention strategies (functional communication training, functional behavior analysis, visual tools/supports, token economy) were predicted by key provider characteristics: professional discipline, experience, and practice specialization (across 3 indices) in ASD. Post-hoc contrasts were performed to identify provider groups reporting greatest use and usefulness of the four strategies.

Results:

Strategies were most often used by providers with behavioral backgrounds, though perceived usefulness of strategies varied by providers’ professional discipline. Compared to providers with more than 10 years of experience, less experienced providers endorsed the highest average use and usefulness of almost all strategies. Regarding ASD practice specialization, a lower volume of ASD cases, treating fewer youths with ASD over a 5-year period, and having a high proportion of practice time working with youths with ASD reported were associated with greater use and usefulness of the strategies.

Conclusions:

Empirically supported strategies are widely used by and perceived as useful by providers who treat youths with ASD and co-occurring externalizing behaviors. Use and usefulness varies based on provider discipline, experience, and ASD practice specialization.

Externalizing behaviors are among the most common and serious problems in youths with ASD (Hill et al., 2014; Mazurek et al., 2013) These behaviors represent a chief concern for referral to psychosocial, behavioral, and psychiatric treatment, which is delivered by providers from many disciplines (e.g., behavioral analysts, psychologists, medical and allied health professionals; Dingfelder & Mandell, 2011). A diverse range of intervention strategies for treating externalizing disorders have been identified; even so, only a small proportion of these strategies are considered to be empirically supported. Furthermore, there exists a research-to-practice “gap” such that treatment by “real world” providers may not integrate empirically supported strategies (Brookman-Frazee et al., 2010; Garland et al., 2010). However, the scope of this “gap,” as it pertains to the treatment of externalizing problems in ASD, is not well understood. Specifically, little is known about use of empirically supported strategies for treating youths with co-occurring ASD and externalizing behavior presenting in usual care settings, nor what predicts the use of these strategies, even as reported by providers themselves. As such, the present study sought to investigate the use of empirically supported strategies among a diverse sample of providers in usual care settings, and to examine key predictors of use and perceived usefulness of these strategies for treating youth with co-occurring ASD and externalizing problems.

Externalizing Behaviors in ASD

Externalizing behaviors are among the most common and serious problems experienced by youths with ASD (Hill et al., 2014; Mazurek et al., 2013). Indeed, more than half of youths with ASD engage in aggressive behaviors towards others (Brown et al., 2019; Farmer et al., 2015; Kanne & Mazurek, 2011) and up to 30% of youths with ASD meet criteria for oppositional defiant disorder (Kaat & Lecavalier, 2013). Externalizing behaviors occur more frequently for youths with ASD than non-ASD youths (Bauminger et al., 2010; Farmer et al., 2015) and have been shown to persist across development (Kanne & Mazurek, 2011). Moreover, youths with ASD who engage in externalizing behaviors experience impairment across multiple domains of their social environments, including family and caregiver functioning, peer relations, and academic performance (Brown et al., 2019; Wagner et al., 2014). Aggressive and disruptive behaviors in this population are also associated with significant healthcare expenditures, such as frequent psychiatric hospitalization, day treatment programs, and long-term residential treatment (McNellis & Harris, 2014; Righi et al., 2018). Given the pervasiveness of and impairment associated with externalizing problems in this population, treatment for these behaviors represents one of the most common reasons for referral to psychosocial/behavioral treatment for youths with ASD (Stadnick et al., 2020).

Empirically Supported Treatment for Externalizing Behaviors in ASD

Within the field of behavioral health, empirically supported treatments are interventions shown to be efficacious in controlled research with a specific clinical population (Chambless & Hollon, 1998; Chambless & Ollendick, 2001). Criteria for determining whether (or to what extent) an intervention merits empirical support varies by discipline (e.g., medicine, clinical psychology, applied behavioral science) and clinical population. To that effect, researchers have identified several empirically supported interventions for treating externalizing behaviors among youths with ASD based on single case and group design methodologies. For example, Wong and colleagues (2015) identified 21 focused intervention practices for behavior problems that met the standard for empirical support, as established by the National Professional Development Center on Autism Spectrum Disorder (NPDC). Similarly, the National Standards Project (NSP) from the National Autism Center (NAC) identified 11 treatment packages demonstrating strong empirical support based on their criteria (National Autism Center, 2015). More recently, an update of NPDC’s 2015 review by the National Clearinghouse on Autism Evidence and Practice (NCAEP), identified 13 evidence-based practices for treating challenging/interfering behaviors across development (Steinbrenner et al., 2020). Viewed together, the considerable overlap between the findings of these reviews suggests strong agreement on which approaches have empirical support for treating externalizing symptoms (see Steinbrenner et al., 2020). Nonetheless, little is known about how these identified strategies are implemented in practice settings.

Research-to-Practice Gap in Treating Mental Health Comorbidities in ASD

Despite the number of empirically supported interventions that exist, research suggests that only a small percentage of children and adolescents, including those with ASD, receive treatments or practices that have empirical support (Brownson et al., 2012). This problem represents a research-to-practice “gap,” wherein empirically supported treatments are not represented in usual care settings (Garland et al., 2010). This gap is especially pronounced in the treatment of mental health comorbidities for youths with ASD (Brookman-Frazee et al., 2010; Dingfelder & Mandell, 2011), thus contributing to significant unmet service needs for this population. Despite the high prevalence of externalizing behaviors among youths with ASD, and the fact that youths presenting for mental health services will often present with behavior concerns, little is known about the provision of empirically supported intervention strategies among community providers. Moreover, the factors that predict the use of empirically supported treatments for externalizing behaviors in ASD in usual care settings are largely unknown. Understanding the usage of empirically supported interventions for externalizing behaviors and identifying predictors of intervention use represents an essential step in bridging this research-to-practice gap.

Providers’ use of empirically supported treatments relies on multiple factors pertaining to the client, provider, and organization (for a review, see Cho et al., 2019). Among these factors, provider characteristics are thought to be especially important, given that providers are most directly involved in treatment and treatment decision-making. Indeed, provider characteristics that have been linked with empirically supported treatment use in youth behavioral health treatment include professional discipline (Higa-McMillan et al., 2015) and clinical experience (Brookman-Frazee et al., 2010). Provider specialization with a certain population or presenting concern is also associated with use of empirically supported treatments. For example, providers working in specialty mental health clinics report more positive attitudes related to empirically supported treatments, which is linked with greater use of these approaches (Garland et al., 2003).

Present Study

The present study sought to explore the use of empirically supported intervention strategies by community providers to treat externalizing behavior in youths with ASD. The sample of community providers included individuals from a wide range of professional disciplines and was geographically diverse. This study also examined whether use and perceived usefulness of intervention strategies was predicted by provider discipline, experience, and ASD practice specialty variables.

Methods

Participants

Participants were 557 providers who reported working with youths with ASD between 7–22 years of age and co-occurring externalizing problems. Providers were drawn from a larger study of treatment practices in the U.S. (Kerns et al., 2019; Wainer et al., 2017). Participants were eligible if they (a) completed the screening questionnaire identifying themselves as one of the eligible provider types (see Table 1), (b) provided an email address, and (c) were located within 100-mile radii of 5 geographic areas (Drexel University, Philadelphia, PA; Rush University, Chicago, IL; St. John’s University, Queens, NY; Michigan State University, East Lansing, MI; Stony Brook University, Stony Brook, NY). Participants were recruited through several methods to ensure representation of providers across various disciplines. Participants were recruited using a nonprobability convenience sample and snowball sampling approach, leveraging extant partnerships with organizations, schools, and clinics serving youth with ASD in the geographical catchment areas. Recruitment also occurred through the professional networks of study investigators, provider listservs, advertisements, and newsletters. Ethics review board approval was granted by the 5 universities/academic medical centers associated with the respective regions. In total, 1,827 screening surveys were completed. Of these, 1,231 provider email addresses were supplied to Princeton Survey Research Associates International (PSRAI), an independent survey firm contracted to collect the online survey data.

Table 1.

Participant Demographics

Variable	n	%
Geographical Regiona
Drexel University	130	23.3
Rush University Medical Center	144	25.9
St. John’s University	133	23.9
Michigan State University	104	18.7
Stony Brook University	46	8.3
Discipline
Physician	54	9.7
Allied health professional	145	26.0
Psychologist	75	13.5
Behaviorist	105	18.9
Multi-discipline provider	88	15.8
Other	89	16.0
Missing	1	0.2
Degree
<Bachelor’s degree	12	2.2
Bachelor’s degree	68	12.2
Master’s degree	362	65.0
Doctoral degree	115	20.6
Number of years working with youths with ASD
0–10 years	301	54.0
11–20 years	204	36.6
21+ years	51	9.2
Missing	1	0.2
Practice Setting
Specialized school or classroom	318	57.1
Outpatient clinicb	152	27.3
Home/community/residential	78	14.0
General public school	61	11.0
Other setting	46	8.3
Number of practice settings
1 setting	343	61.6
2 settings	116	29.8
3 or more	48	8.6
Average number of hours/week spent working with youths with ASD in the past year
0–20 hours/week	188	33.8
21–40 hours/week	301	54.0
40+ hours/week	66	11.8
Missing	2	0.4
Percentage of practice spent working with youths with ASD
0–25%	151	27.1
26–50%	108	19.4
51–79%	111	19.9
80–100%	183	32.9
Missing	4	0.7
Number of youths with ASD served in the past year
0–29 youths	196	35.2
30–74 youths	213	38.2
75+ youths	131	23.5
Missing	17	3.1

Note. Demographic information for the providers.

^aThe Geographical Region refers to the area within a 100-mile radius of the university/academic medical centers of which the research team members were affiliated.

^bOutpatient clinic settings included outpatient clinics at community mental health centers, hospitals or medical schools, university counseling centers, and research centers

Of the 1,231 providers recruited to participate, 701 providers (56.9%) completed all survey measures; 27 cases were removed due to potential duplication (i.e., multiple submissions from the same IP address), low motivation response style, or other factors consistent with suspicious and fraudulent participation (see Lawlor et al., 2021). Of the remaining 674 providers, 557 endorsed that externalizing behavior was a core area that they regularly treated in youth with ASD over the past year. Data from this subset of providers were examined in the present study.

Caseload Demographics

Participants were asked several questions pertaining to the youths they have treated in the past year. Participants were instructed to select all response options (yes/no) that apply for questions pertaining to youth age, race/ethnicity, and socioeconomic status of the youths they served (see Table 2). Regarding gender, participants were asked to select the option that best described the gender of the youths with ASD with whom they have worked; 77.7% reported working with mostly male youth, 18.9% reported working with male and female youth equally, 3.2% reported working with mostly female youth, and 0.2% (one participant) reported they did not know. Regarding work with youth with ASD who had a known or suspected intellectual disability (ID), participants selected one option that best described their work in the past year; 51.5% reported sometimes working with youth with known or suspected ID, 37.9% reported frequent work with these youths, 7.4% reported rarely working with these youths, and 3.2% reported that they did not know.

Table 2.

Provider-Reported Demographic Characteristics of Youth with ASD Served Over the Last Year

What is the age range of children with ASD you typically worked with over the last year?	Yes (%)	No (%)
Less than 7 years old	84 (15.1)	473 (84.9)
7–9 years old	391 (70.2)	166 (29.8)
10–12 years old	348 (62.5)	209 (37.5)
13–15 years old	273 (49.0)	284 (51.0)
16–18 years old	351 (63.0)	206 (37.0)
19–22 years old	118 (21.2)	439 (78.8)
Older than 22	11 (2.0)	546 (98.0)
What is the race/ethnicity of the children with ASD whom you have primarily treated over the last year?	Yes (%)	No (%)
White	453 (81.3)	104 (18.7)
Black or African American	294 (52.8)	263 (47.2)
Hispanic or Latino	308 (55.3)	249 (44.7)
Asian	117 (31.8)	380 (68.2)
Native American	13 (2.3)	544 (97.7)
Hawaiian or Pacific Islander	21 (3.8)	536 (96.2)
Other	31 (5.6)	526 (94.4)
Unsure/Don’t know	10 (1.8)	547 (98.2)
What is the socioeconomic status (SES) of the children with ASD whom you have served over the past year?	Yes (%)	No (%)
High SES	211 (37.9)	364 (62.1)
Medium SES	435 (78.1)	122 (21.9)
Low SES	312 (56.0)	245 (44.0)
Unsure/Don’t know	36 (6.5)	521 (93.5)

Note. Demographic characteristics of the youths with ASD with whom providers worked with in the past year.

Procedure

Ethics review board approval was granted for the study procedures by the 5 universities/academic medical centers associated with the respective regions. All eligible respondents were emailed invitations and email reminders with a unique URL to access the study survey by PSRAI. All participants who attempted or completed they survey provided consent prior to beginning the survey. Respondents who completed the survey received a $40 honorarium for their participation. Data were de-identified by PSRAI before dissemination to the research staff of the present study.

Measure

Usual Care for Youth with Autism Survey (UCAS)

The UCAS (Kerns et al., 2019) is web-based survey designed to assess “usual care” of individuals with ASD aged 7–22 years. The UCAS is comprised of demographic questions pertaining to providers and their practice (see Provider Characteristics, below), the clients treated by providers, as well as an inventory of 55 intervention practices for three core treatment areas: anxiety, social deficits, and externalizing behaviors in individuals with ASD. Participants who endorsed treating youths with ASD and co-occurring externalizing behaviors within the past year (yes/no) were presented with questions pertaining to 42 intervention practices specific to the treatment of externalizing behaviors. Intervention practices were derived through a systematic literature review and two-round Delphi poll of expert ASD providers from multiple disciplines (see Kerns et al., 2019; Wainer et al., 2017). Participants reported on their use (1 = not at all, 4 = very commonly) and perceived usefulness (1 = not at all useful; 4 = very useful or “I don’t know”/“Not sure”) of intervention practices for the treatment of externalizing behaviors. Providers worked in at least one setting: a specialized school or classroom; outpatient clinic; home, community, or residential settings; a general public school; or other setting.

Provider Characteristics (UCAS)

A subset of the items on the UCAS pertaining to providers and their practice were examined in the present study.

Provider Discipline.

Providers selected their discipline from categories that were then compiled into the following for the present study: psychologists, physicians, behaviorists, allied health professionals, and “other” providers. Providers who selected multiple disciplines were categorized as “multi-discipline” providers.

Provider Experience.

Providers selected the option that represented their years of experience working with youths with ASD. Original response options on the UCAS were 0–4, 5–10, 11–15, 16–20, 20–31, or over 30 years’ experience. Groups were combined to 0–10, 11–20, and over 20 years of experience for analysis in the present study.

ASD Practice Specialty.

Providers endorsed three items that assess ASD practice specialty (i.e., the degree to which their clinical work was specialized to ASD). Providers reported on their time spent working with youths with ASD on a weekly basis. Original response options in the UCAS survey were 0–10, 11–20, 21–30, 31–40, and over 40 hours/week. Groups were combined to 0–20, 21–40, and over 40 hours/week for analysis in the present study. Providers also reported the number of youths with ASD served within the past 5 years. Response options were 0–10, 11–29, 30–49, 50–74, and 75 or more youths served. Groups were combined to 0–29, 30–74, and 75+ youths treated for the present study. Lastly, providers reported on the proportion of their practice time spent working with youths with ASD. Original response options on the UCAS survey were 0–10%, 11–25%, 26–50%, 51–79%, and 80–100% practice time. The 0–10 and 11–25% groups were combined for analysis in the present study, yielding four groups for analysis for the present study: 0–25%, 26–50%, 51–76%, and 80–100% practice time.

Identifying Empirically Supported Practices

Intervention practices included in the UCAS have varying degrees of consensus regarding their empirical support as reported by experts in the Delphi poll (Kerns et al., 2019). We utilized a two-step selection method to identify practices with strong empirical support. First, we identified practices that showed a high degree of consensus (i.e., 90% agreement) among ASD experts (Kerns et al., 2019) as demonstrating strong research support for treating externalizing symptoms. Second, we determined whether the practices met criteria for having strong empirical support for individuals with ASD, as established by one or more systematic reviews (i.e., NSP [National Autism Center, 2015]; NCAEP [Steinbrenner et al., 2020]; NPDC [Wong et al., 2015]). This method yielded four intervention practices among the 42 intervention practices identified in the UCAS specific to the treatment of externalizing behaviors with strong empirical support: functional communication training (FCT), functional behavioral assessment (FBA), visual tools/supports, and token economy.

Functional Communication Training (FCT)

FCT involves teaching a child an appropriate communicative behavior that serves the same function or purpose an inappropriate behavior, including externalizing behaviors. This communicative behavior could be verbal/vocal, or nonverbal (e.g., sign language, augmentative communication device).

Functional Behavior Assessment (FBA)

FBA is a systematic method of assessment to evaluate the function or purpose of an externalizing behavior to inform intervention. An FBA can include indirect methods (e.g., questionnaires, interviews), direct observation, and an experimental functional analysis.

Token Economy

Rewarding a child by giving them tangible symbols (e.g., points, stickers, tokens, poker chips) for completing tasks or behaving in desired ways. The child is then able to exchange tokens/points for a desired object (e.g., candy, toys) or activity (e.g., playing a video game, going outside).

Visual Tools/Supports

The use of visual tools (e.g., pictures, written words, videos, maps, timelines, calendars, or diagrams) to teach skills, behaviors, or concepts, or to increase the predictability of a situation; also known as visual prompts.

Analytic Plan

Generalized estimating equations.

Generalized estimating equations (GEE) are a type of generalized linear model that can account for possible non-independence of participant responses, such as with clustered datasets. We used GEE to identify the provider-level characteristics associated with use and usefulness of the 4 intervention strategies while accounting for shared variance associated with each of the 5 clinical sites. We predicted use and perceived usefulness of the 4 strategies (dependent variables) by the independent variables (provider characteristics): provider discipline, years of experience, and three ASD practice specialty variables. We used single-predictor GEE models for provider discipline and experience variables and multiple-predictor models for the three ASD practice specialty variables. We specified the identity link function for all GEE models with a normally distributed dependent variable, and either (1) independent and (2) unstructured correlation matrices for the dependent variable. We then compared the fit of each pair of linear models using the quasilikelihood under the independence model criterion (QIC) and selected the linear model with the lowest value. It should be noted that the Wald’s Χ² significance tests within our omnibus GEEs is not reporting on the directionality of the relationship, but rather whether there are significance differences in the response variable across levels of the predictor.

Contrasts

We conducted post-hoc contrast analyses on the independent variables (i.e., provider characteristics) that predicted use and usefulness of the four strategies (p’s <.05). For all contrasts, the comparison condition represented the group with the highest average use/usefulness. We performed post hoc pairwise contrasts using a least-significant-difference adjustment for multiple comparisons. For each set of contrasts, we selected the comparison condition based on the level of the independent variable (e.g., discipline, level of experience) that reported the greatest self-reported use and usefulness of a given strategy; all comparisons were performed relative to this comparison group.

Results

Overall Rates of Use and Usefulness

Across all respondents, self-reported use of the four strategies showed similar endorsement patterns, with a less than 5% difference between strategies for any of the response options: very commonly, commonly, occasionally, and not at all (see Table 3). Endorsement of very commonly used ranged from 55.5% (visual tools/supports) to 57.2% (token economy); commonly used ranged from 21.8% (FCT) to 23.8% (FBA); occasionally used ranged from 13.7% (FBA) to 16.5% (visual tools/supports); and not at all used ranged from 4.6% (visual tools/supports) to 8.4% (FCT). Compared to overall usefulness, somewhat greater variability was observed for providers’ overall perceived usefulness of the four strategies across five response options: very useful, somewhat useful, not too useful, not at all useful, and not sure/don’t know (see Table 3). Endorsement of very useful ranged from 59.8% (visual tools/supports) to 71.5% (FCT); somewhat useful ranged from 20.5% (FCT) to 33.5% (visual tools/supports); not too useful ranged from 2.0% (FBA) to 3.7% (FCT); and not at all useful ranged from 0.6% (visual tools/supports) to1.7% (token economy). For the not sure/don’t know response option, responses ranged from 2.0% (token economy) to 3.4% (FCT). Rates of use and usefulness by discipline are included in the supplemental materials (see Supplementary Tables 1 and 2, respectively).

Table 3.

Overall Rates of Use and Usefulness of Identified Strategies for Treating Externalizing Behaviors in ASD

	Use of Strategies
	Not at all	Occasionally	Commonly	Very Commonly
Functional behavior analysis	6.7%	13.7%	23.8%	55.8%
Functional communication training	8.4%	14.1%	21.8%	55.7%
Token economy	6.1%	13.5%	23.1%	57.2%
Visual tools/supports	4.6%	16.5%	23.4%	55.5%

	Usefulness of Strategies
	Not at all useful	Not too useful	Somewhat useful	Very useful	Not sure/don’t know
Functional behavior analysis	1.1%	2.0%	24.8%	68.8%	3.2%
Functional communication training	0.7%	3.7%	20.5%	71.7%	3.4%
Token economy	1.7%	2.4%	31.5%	62.5%	2.0%
Visual tools/supports	0.6%	2.8%	33.5%	59.8%	3.2%

Note. Overall endorsement of use and perceived usefulness of strategies across all participants.

Correlations of Self-Reported Use and Usefulness

We performed bivariate correlations between self-reported use and usefulness for each of the techniques; r’s were .45 for FBA, .48 for FCT, and .65 for both token economy and visual tools/supports (p’s <.001 for all). For self-reported use of the four techniques, r’s ranged from .24 to .38. For usefulness of the techniques, r’s ranged from .25 to .38 (p’s <.001 for all).

GEE Models and Post-hoc Contrasts

GEE models were performed to determine whether the provider discipline, experience, and ASD practice specialty variables predicted self-reported use/usefulness of the four strategies. For the contrasts, the group with the highest score on use or usefulness was the comparison condition for each set of post-hoc contrasts. Post-hoc contrasts were not performed for variables that yielded a nonsignificant GEE.

Provider Discipline

As seen in Table 4, GEE models showed that provider discipline predicted self-reported use and usefulness across all strategies (p’s <.05 for all). Behaviorists reported that they used all FBA and FCT strategies significantly more often than other providers across all comparisons (all p’s <.001; see Supplementary Table S3 and Supplementary Figure 1). Behaviorists reported that they used visual tools/supports more than all other providers (all p’s <.001), except multidiscipline providers (p = .324). Similarly, behaviorists reported more use of token economy more than physicians, allied health professionals, and other providers (all p’s = or <.001), but not psychologists (p =.216) or multidiscipline providers (p =.367). Regarding perceived usefulness of strategies, behaviorists rated FBA as more useful relative to all other providers (all p’s <.001), except psychologists (p =.156). FCT was rated as most useful by “other” providers (all p’s <.001). Visual tools/supports was rated most useful by behaviorists compared to all other providers (all p’s <.001). Token economy was rated as most useful by multi-discipline providers compared to all other providers (all p’s < .001), except psychologists (p = .084).

Table 4.

Summary of GEE Model and Fit Statistics for Single-Predictor Models: Provider Discipline and Provider Years of Experience

Predictor: Provider Discipline
Dependent Variable	Model QIC	Wald Χ2 (df)c	p
FBA Use	348.646a	833.529 (4)	<.001
FBA Usefulness	143.694a	86.375 (4)	<.001
FCT Use	365.180a	57.968 (4)	<.001
FCT Usefulness	134.167a	2016.947 (4)	<.001
Token Economy Use	396.022a	9.547 (4)	= 0.49
Token Economy Usefulness	198.275a	14.974 (4)	=.005
Visual Tools/Supports Use	365.160a	51.449 (4)	<.001
Visual Tools/Supports Usefulness	156.367a	413.299 (4)	<.001
Predictor: Provider Years of Experience
FBA Use	383.967a	3.908 (2)	=.142
FBA Usefulness	150.613b	388.433 (2)	<.001
FCT Use	401.116a	5.806 (2)	=.055
FCT Usefulness	138.277a	98.303 (2)	<.001
Token Economy Use	388.338a	22.698 (2)	<.001
Token Economy Usefulness	188.376a	14.339 (2)	<.001
Visual Tools/Supports Use	381.888a	7.459 (2)	= .024
Visual Tools/Supports Usefulness	162.113a	5.954 (2)	=.051

^aIndependent correlation matrix solution

^bUnstructured correlation matrix solution

^cWald Χ² test statistic

Note. GEE model summaries of best-fitting single-predictor models of provider discipline and years of experience predicting strategy use and usefulness, respectively. Table includes only best-fitting models and their corresponding QIC, Wald Χ², and p-value. Best-fitting models were selected based on lowest QIC value. Superscript in Model QIC column denotes the type of correlation matrix solution that yielded the best-fitting model. FBA = functional behavior assessment. FCT = functional communication training.

Provider Experience

Provider experience predicted FBA usefulness, FCT usefulness, token economy use, token economy usefulness, and use of visual tools/supports (p’s <.05 for all, see Table 4). Provider experience marginally predicted FCT use and usefulness of visual tools/supports (p’s =.055 and .051, respectively), and did not significantly predict FBA use (p =.142). Providers in the 0–10 years’ experience group self-reported use of the token economy strategy more than those from the 11–20 years’ experience group or the 21+ years’ experience providers (p =.004 and .003, respectively, see Supplementary Table 4 and Supplementary Figure 2). Providers with 21+ years of experience used visual tools/supports relative to the 11–20 years’ experience providers (p = .014) but not the 0–10 years’ experience providers (p = .986). Regarding usefulness, 0–10 years’ experience providers rated FBA as more useful than the 11–20 and 21+ years’ experience providers (p’s <.001 and =.032, respectively). Providers with 0–10 years’ experience reported more perceived usefulness of FCT and token economy relative to the 11–20 years’ experience providers (p =.003 and <.001, respectively) but not the 21+ years’ experience providers (p’s = .425 and = .516, respectively).

ASD Practice Specialty

As seen in Table 5, GEE models showed that provider specialty variables predicted self-reported use and usefulness for many of the four strategies. The three specialty variables predicted FBA use, FCT usefulness, and token economy use (p’s <.001 for all). FBA usefulness, FCT use, and visual tools/supports use were predicted by volume of youths served and practice time spent with youths with ASD (p’s <.01 for all), but not by hours/week spent working with youths with ASD (p’s = .665, .796, and .492 respectively). Token economy use and visual tools/supports usefulness were predicted by hours/week spent working with youths with ASD and practice time spent with youths with ASD (p’s <.01 for all), but not by the number of youths served over the past 5 years (p = .199 and .236, respectively).

Table 5.

Summary of GEE Model Comparisons, Fit Statistics and Predictor Statistics for Multi-Predictor Models: Provider Practice Specialty Variables

Model fit statistics and specialty predictor variables	Wald χ2 c	p
FBA Use (QIC =343.759a)
Hours/week spent working with youths with ASD	22.353 (2)	<.001
Youths with ASD served/past 5 years	40.054 (2)	<.001
Practice time spent with youths with ASD	40.791 (3)	<.001
FBA Usefulness (QIC =157.000a)
Hours/week spent working with youths with ASD	0.815 (2)	=.665
Youths with ASD served/past 5 years	15.688 (2)	<.001
Practice time spent with youths with ASD	23.471 (3)	<.001
FCT Use (QIC =353.916a)
Hours/week spent working with youths with ASD	0.457 (2)	=.796
Youths with ASD served/past 5 years	135.823 (2)	<.001
Practice time spent with youths with ASD	191.433 (3)	<.001
FCT Usefulness (QIC = 133.009b)
Hours/week spent working with youths with ASD	67.035 (2)	<.001
Youths with ASD served/past 5 years	20.057 (2)	<.001
Practice time spent with youths with ASD	2732.678 (3)	<.001
Token Economy Use (QIC =368.889a)
Hours/week spent working with youths with ASD	9.384(2)	=.009
Youths with ASD served/past 5 years	3.230 (2)	=.199
Practice time spent with youths with ASD	148.068 (3)	<.001
Token Economy Usefulness (QIC = 173.333b)
Hours/week spent working with youths with ASD	76.388 (2)	<.001
Youths with ASD served/past 5 years	184.306 (2)	<.001
Practice time spent with youths with ASD	851.662 (3)	<.001
Visual Tools/Supports Use (QIC = 366.076a)
Hours/week spent working with youths with ASD	1.419 (2)	=.492
Youths with ASD served/past 5 years	9.487 (2)	=.009
Practice time spent with youths with ASD	165.647 (3)	<.001
Visual Tools/Supports Usefulness (QIC = 158.972a)
Hours/week spent working with youths with ASD	14.428 (2)	=.001
Youths with ASD served/past 5 years	2.887 (2)	=.236
Practice time spent with youths with ASD	1568.155 (3)	<.001

^aIndependent correlation matrix solution

^bUnstructured correlation matrix solution

^cWald Χ²test statistic for best-fitting model

Note. GEE model summaries of best-fitting multiple-predictor models of provider practice specialty variables (hours/week spent working with youths with ASD, youths with ASD served in the past 5 years, and practice time spent with youths with ASD) predicting strategy use and usefulness, respectively. Table includes only best-fitting models and their corresponding QIC, Wald Χ², and p-value. Best-fitting models were selected based on lowest QIC value. Superscript next to QIC value denotes the type of correlation matrix solution that yielded the best-fitting model. FBA = functional behavior assessment. FCT = functional communication training.

Hours per week.

Providers working 21–40 hours/week with youths with ASD reported more self-reported use of FBA than providers working 0–20 hours/week (p < .001 for both) but not those providers working over 40 hours/week (p’s .141). Providers who worked 0–20 hours/week with youths with ASD self-reported more use of token economy than providers working 21–40 hours/week (p <.001) but not 40+ hours/week (p = .997). Providers working 0–20 hours/week also reported the more usefulness for FCT, token economy, and visual tools/supports than providers who worked 21–40 hours/week (p’s <.001 for all). Compared to providers working 40+ hours/week, providers working 0–20 hours/week also reported more usefulness for FCT (p =.009) and token economy (p <.001), but not for visual tools/supports (p =.078).

Volume of cases.

Providers who treated 0–29 youths with ASD over the past 5 years used FBA more than providers who treated 30–74 youths or 75+ youths over the past 5 years (p <.001 and =.031, respectively; see Supplementary Table 5 and Supplementary Figure 4). Providers in the 0–29 youths treated group used FCT more than the 75+ youths/5 years group (p <.001), but not the 30–74 youths/5 years group (p = .359). Providers who treated 0–29 youths/5 years used visual tools/supports more than the 30–74 youths/5 years group (p = .010) but not the 75+ youths/5 years group (p =.351). Regarding usefulness, providers in the 0–29 youths/5 years group reported more usefulness of FBA compared to the 30–74 youths/5 years group (p <.001) but not the 75+ youths/5 years group (p’s =.169). Providers in the 0–29 youths/5 years group reported more usefulness of FCT compared to the 75+ youths/5 years provider group (p <.001), but not the 30–74 youths/5 years group (p =.223). For token economy, providers in the 0–29 youths/5 years group reported more usefulness of this strategy compared to the 30–74 youths/5 years and 75+ youths/5 years provider groups (p’s <.001 for both).

Percentage of practice time devoted to youths with ASD.

Providers who reported that 80–100% of their practice time was spent with youths with ASD also self-reported the most use and usefulness of the strategies. For all strategies, 80–100% ASD practice providers reported more self-reported use than the 0–25% ASD practice providers (p’s <.001 for all; see Supplementary Table 5 and Supplementary Figure 5). Providers in the 80–100% ASD practice group used token economy and visual tools/supports more than the 51–79% practice groups (p’s = .018 and .049, respectively) for these strategies only. No differences were found between the 80–100% and 26–50% ASD practice groups for self-reported use of any strategy. Regarding usefulness, the 80–100% ASD practice found FBA to be more useful than the 0–25% ASD practice group (p =.022), but not the 26–50% or 51–79% ASD practice groups (p =.716 and .078, respectively). For all other strategies, the 80–100% ASD practice group reported more usefulness than all other practice groups (p’s < or = .001 for all).

Supplemental Analyses

Given the significant correlations between self-reported use and perceived usefulness of the four strategies, we performed a series of post-hoc analyses with these variables as covariates in the GEE models: self-reported use was included as a covariate for perceived usefulness and perceived usefulness was included as a covariate for self-reported use. For the models with provider discipline as a predictor variable, self-reported use was a significant covariate only for the model predicting usefulness of token economy, and perceived usefulness was a significant covariate only for the model predicting self-reported use of token economy. Self-reported use and perceived usefulness were significant covariates for all provider experience and practice specialty models. It should be noted that although the inclusion of the covariates did not affect the significance of the predictors for the discipline models, a small portion of the experience and practice specialty predictors were no longer significant with the covariates (see Supplementary Tables 6 and 7, respectively).

Discussion

Youths with ASD are treated by providers from a range of disciplines and practice settings for treatment of ASD symptoms and co-occurring conditions. This study examined the self-reported use and perceived usefulness of four empirically supported research strategies by community providers treating youths with ASD and co-occurring externalizing behaviors. Our results showed that provider discipline, experience, and ASD practice specialization predicted the self-reported use and usefulness of these strategies, with several meaningful differences across levels of these predictor variables. Across disciplines, providers self-reported that they frequently use the four empirically supported strategies examined in this study and perceived these strategies as useful for treating externalizing behaviors in youth with ASD. However, significant variability existed between disciplines regarding the self-reported use and usefulness of these strategies. Behaviorists used all strategies more frequently than other providers; they also reported the most usefulness for FBA and visual tools/supports compared to all other providers. Providers with 0–10 years of experience consistently self-reported greater use and usefulness of the strategies compared to providers with 11–20 years of experience. In general, providers with 0–10 years of experience reported use and usefulness that was similar to or exceeded use and usefulness reported by providers with 21+ years of experience. Moreover, providers who worked with youths with ASD 0–20 hours weekly reported the most self-reported use and usefulness of the strategies. Self-reported use and usefulness patterns were more variable regarding number of youths served in the past 5 years and percentage of practice spent working with youths with ASD. Together, our findings suggest broad support for empirically supported strategies for treatment of externalizing in ASD based on overall rates of use and usefulness, with notable differences depending on provider discipline, experience, and ASD practice specialization.

Provider Discipline

To a large extent, the patterns in response options were consistent with discipline. Indeed, behaviorists were the comparison group for all discipline-level contrasts examining self-reported use. Behaviorists reported more frequent use compared to all other groups, except psychologists and providers from multiple backgrounds for the token economy technique. The four techniques examined in the present study are rooted in behavioral intervention and analysis, which is likely consistent with the training and scope of practice for behavioral providers. Although not examined in the present study, previous work has shown that mental health providers’ theoretical orientations predict self-reported use of empirically supported and evidence-based treatments, with providers who have cognitive-behavioral and behavioral orientations reporting the most use (Nelson & Steele, 2007). It may be the case that behaviorists and psychologists share these orientations, which inform their use and perceived usefulness of the four strategies. It should be noted that those who self-identified as behaviorists may include behavior therapists who are “frontline” providers for youths with ASD, such as registered behavior technicians and board-certified assistant behavior analysts (Leaf et al., 2017). Thus, it is possible that the relatively high use of the four strategies is consistent with the techniques most often utilized by frontline providers.

Self-reported use of the strategies was generally lowest among allied health providers, physicians, and “other” providers, which may be expected given the scope of these providers’ practices. It notable that these providers also endorsed the lowest usefulness for some of the strategies. Lower endorsement of usefulness could be related to these providers’ experiences with these strategies, consistent with the notion that a provider’s clinical experience often predicts treatment decisions (Stewart et al., 2012; Stewart & Chambless, 2007). Even so, lower endorsement of perceived usefulness is a potential area of concern. Medical professionals are often a point of referral and can help to facilitate families to these evidence-based services (Ming et al., 2011). Therefore, if the techniques are viewed as less useful by these providers, it is possible that this could be a barrier to referral and represent a challenge in accessing empirically supported treatment. Furthermore, it may be the case that medical providers view many interventions with similar degree of usefulness (i.e., that most interventions are generally useful), with limited discernment between techniques with strong empirical or low empirical support.

Interestingly, rates of self-reported use of the techniques by physicians and allied health professionals may be higher than expected, given the perceived scope of practice for these professions. One possibility is that some of these individuals may work in acute treatment settings (e.g., inpatient psychiatric units, residential settings) that implement unit- or facility-wide behavioral interventions, such as point system (i.e., token economy) programs that are enforced by all staff. It is also possible that discernment of the strategies was lower for these professionals, as these techniques may not be within the scope of training or practice, thus contributing to an inflated endorsement rate.

Provider Experience

Provider level of experience predicted self-reported use of token economy and visual tools/supports, as well as usefulness of FBA, FCT, and token economy. The least experienced providers reported the most self-reported use and usefulness for nearly all strategies. This finding is consistent with research on the provision of empirically supported interventions (i.e., empirically supported treatments, evidence-based treatments, and evidence-based practices) in psychotherapy, insomuch that younger providers and providers with less experience report using these approaches and treatment manuals more frequently than older and more experienced providers (Becker et al., 2013; Stewart et al., 2012). Indeed, experienced clinicians are more likely than less experienced clinicians to use clinical judgement to guide treatment selection and decisions, with less emphasis on research integration and utilization of empirically supported interventions in particular (von Ranson & Robinson, 2006).

Interestingly, although the least experienced providers had the highest average self-reported use and usefulness of almost all strategies, differences between this group and the most experienced providers were non-significant on usefulness of FCT and token economy, as well as self-reported use of visual tools/supports. This finding suggests that the relationship between experience and self-reported use or usefulness of some empirically supported strategies may be curvilinear, with most- and least-experienced providers reporting more self-reported use and usefulness than mid-experience providers. One possibility is that the most-experienced providers are also involved in teaching and training duties (e.g., course instructors and clinical supervisors), leadership in ASD-related organizations, or research opportunities that necessitate knowledge of empirically supported practices in ASD treatment. Expert clinicians may also experience organizational or workplace factors that are protective against staff burnout and turnover, which includes positive climates and support regarding implementation of empirically supported interventions. A study by Beidas et al. (2016) found that providers with positive attitudes towards empirically supported interventions were twice as likely to remain at their community mental health agencies during a one-year period compared to providers with negative attitudes towards empirically supported interventions. In the current study, the self-reported use/usefulness of empirically supported treatments by expert providers may be an artifact of survival, as providers who have remained involved care of youths with ASD may be resilient to certain risk factors of staff turnover, including low organizational support for empirically supported interventions.

Lastly, it is also possible that those providers with the least experience are also the frontline providers with behavioral backgrounds with 4-year degrees or less (e.g., Registered Behavioral Technicians; Leaf et al., 2017). Thus, it is also possible that least-experience providers with less education may be less discerning in what practices they use and may therefore over-report their use of practices compared to actual use. Commonalities between least and most experienced providers may be due to discernment of practices, wherein the high use reported by most-experienced providers more closely aligns with their actual practice, and use by least-experienced providers is elevated compared to actual use.

ASD Practice Specialty

In general, a lower volume of ASD cases was associated with greater self-reported use and usefulness of empirically supported strategies. Indeed, providers who worked 0–20 hours per week self-reported more use and usefulness compared to providers working 21–40 or over 40 hours weekly. Similarly, providers who worked with a low volume of youths with ASD over the past 5 years (i.e., 0–29 youths treated) self-reported more use and usefulness of the strategies compared to providers with a higher volume of cases. At first blush, these findings seem somewhat contradictory: How are the providers who report working with youths with ASD the least reporting that they use the strategies the most? One possibility is that providers who spend less of their working hours– not practice time – working with youths with ASD have a narrower scope of practice “tools” than providers who work with a higher volume of cases. That is, a large proportion of the intervention services for lower-volume providers may be comprised mostly of these strategies. Alternatively, providers with a higher load may implement a more diverse range of strategies, and would therefore report less use relative to providers with more focused practice scope. Thus, comparisons of use should be considered on relative terms pertaining to the provider’s own practice, rather than as an absolute measure of use.

This is somewhat consistent with trends in utilization of empirically supported interventions in usual care settings implicating organizational factors – productivity requirements and caseload composition – in clinician burnout, which, in turn, is related to less use of and less positive attitudes on empirically supported interventions (Rodriguez et al., 2021). Caseload complexity (e.g., symptom severity, multiple comorbidities) is implicated in high levels of therapist burnout, negative attitudes towards empirically supported interventions, and less implementation of empirically supported interventions (Rodriguez et al., 2021; Weisz et al., 2013). To be sure, youths with ASD and co-occurring externalizing behaviors represent a complex clinical population whose challenges across multiple domains of functioning can necessitate intensive interventions across several settings (e.g., at home and school, with peers; Dovgan & Mazurek, 2019; Wagner et al., 2019). Thus, it seems reasonable to suggest that providers whose effort involves fewer ASD cases and who spend a non-majority part of their effort treating these youths are more likely to use empirically supported strategies and perceive them as useful, potentially due to less professional burnout, fewer organizational pressures, and more support for empirically supported interventions.

It bears note that weekly hours spent working with youths ASD and proportion of practice time spent working with this population represent similar, yet distinct practice variables. For example, a provider could endorse 80–100% of their practice time spent working with youths with ASD, and also report working with these youths less than 10 hours weekly, as their positions may be less than full-time or comprise of training, administrative, or research responsibilities. Nonetheless, these variables were examined as independent predictors, consistent with the assumptions of our statistical approaches.

Providers who spent 80–100% of their practice time with youths with ASD reported the most use and usefulness of all the strategies. This finding seems consistent with the notion that providers working with a specific clinical population or utilizing a narrower range of modalities may more readily implement an empirically supported intervention relevant to their scope of practice. Indeed, caseload heterogeneity in terms of presenting concerns and modality, such as those seen in community mental health clinics, is associated with less use of empirically supported interventions (Weisz et al., 2013). Although research on “specialty” clinics and providers is limited, there is some evidence to suggest that fidelity to ESTs is more easily maintained with smaller caseloads comprised of clients with similar presenting concerns (e.g., Henggeler & Schaeffer, 2016). Furthermore, a more focused scope of practice may better enable providers to consistently implement empirically supported interventions and stay current with best practices for their target population.

Limitations

Several limitations warrant comment. First, although participants were recruited across five geographically distinct sites, this sample may not be representative of providers delivering services to youths across the entire United States, as certain regions (e.g., the Southeast and Southwest) were underrepresented. Second, data were derived exclusively from provider self-report. Though survey methodology was consistent with the aims of the study, research on treatment fidelity suggests that providers may overestimate their use of empirically supported practices or deliver them with low fidelity (Hogue et al., 2015). Thus, although it is promising that providers endorse generally high use and perceived usefulness related to empirically supported treatments, it is possible that usage rates may be overestimated. Third, on a similar note, it is possible that providers over-identified the use of the strategies based on the written examples for the UCAS items. Future work may include video vignettes to demonstrate examples of strategy use. Finally, this study did not assess several organizational factors that may predict empirically supported strategy use, such as mandated use of ESTs, colleague and supervisor strategy use, and supports for EST training (Nelson & Steele, 2007). Similarly, this study did not examine whether certain demographic characteristics of providers were related to self-reported strategy use or usefulness. Demographic information was not collected to protect the respondents’ identities within potentially small care and referral networks, and because examining associations between demographic characteristics and clinical practice was not a focus of the original study. Nonetheless, it would be important to explore these variables in future work.

Conclusion

Overall, the present study found that self-reported use of empirically supported strategies and perceived usefulness of those strategies for the treatment of externalizing behaviors in ASD is predicted by key provider variables: discipline, experience, and ASD practice specialty. When one considers the functional impairment and incidence of externalizing behaviors for youths with ASD, several needs are illuminated. First, the comparatively low perceived usefulness of these strategies by medical providers suggests a need for increased knowledge of these practices, especially for those providers who help to facilitate treatment referrals. Similarly, reasons for lower use and usefulness of the strategies by mid-career providers (11–20 years of practice) warrants additional investigation, with targeted efforts on identifying barriers and promoting greater use/usefulness. Second, practice organizations should consider the role of productivity requirements, caseload complexity and heterogeneity, and volume of youths served when promoting use of empirically supported strategies and interventions broadly. Lastly, future work should consider predictors of provider burnout and turnover for those working with youths with ASD, as these youths represent a population for whom access to care remains a continual challenge. Thus, efforts aimed at provider retention and positive attitudes towards empirically supported strategies should be prioritized.

Table 6.

Summary of Generalized Estimating Equations Post-hoc Comparisons

Strategy	Discipline	Years’ experience	Hours/week spent working with youth with ASD	No. of youth with ASD served in last 5 years	Practice time % devoted to youth with ASD
FBA Use	Behaviorists > all other disciplines	NS	21–40 > 0–20	0–29 >30–74 & 75+	80–100% > 0–25%
FBA Usefulness	Behaviorists > allied health, multidiscipline, other, physicians	0–10 > 11–20 & 21+	NS	0–29 > 30–74	80–100% > 0–25%
FCT Use	Behaviorists > all other disciplines	NS	NS	0–29 > 75+	80–100% > 0–25%
FCT Usefulness	Other providers > all other disciplines	0–10 > 11–20	0–20 > 21–40 & 40+	0–29 > 75+	80–100% > 0–25%, 26–50%, 51–79%
Token Economy Use	Behaviorists > physicians, allied health, and other providers	0–10 > 11–20 & 21+	0–20 > 21–40	NS	80–100% > 0–25%, 51–79%
Token Economy Usefulness	Multi-discipline > physicians, allied health, behaviorists, other providers	0–10 > 11–20	0–20 > 21–40 & 40+	0–29 >30–74 & 75+	80–100% > 0–25%, 26–50%, 51–79%
Visual Tools/Supports Use	Behaviorists > allied health, psychologists, physicians, other providers	21+ > 11–20	NS	0–29 > 30–74	80–100% > 0–25%, 51–79%
Visual Tools/Support Usefulness	Behaviorists > all other disciplines	NS	0–20 > 21–40	NS	80–100% > 0–25%, 26–50%, 51–79%

Note. NS =, non-significant predictor; contrast summary not reported.