A Multisite, Multidisciplinary Delphi Consensus Study Describing “Usual Care” Intervention Strategies for School-Age to Transition-Age Youth With Autism

Abstract

Understanding usual care is important to reduce health disparities and improve the dissemination of evidence-based practices for youth (ages 7–22 years) with autism spectrum disorder (ASD). A barrier to describing “usual ASD care” is the lack of a common vocabulary and inventory of the practices used by a diverse provider field. To address this barrier, we gathered input from expert providers to develop an inventory of usual care practices and assess expert familiarity and perceptions of these practices as interventions for anxiety, externalizing, and social difficulties in ASD. Purposeful sampling recruited 66 expert ASD providers representing multiple disciplines from 5 sites. Via a 2-round Delphi poll, experts reviewed, suggested revisions to and rated 49 literature-derived practices on several dimensions (familiarity, usefulness, common use, research support). A revised list of 55 practices and anonymous summary of group characteristics and ratings was then returned for further review. Results yielded 55 intervention practices, 48 of which were identified as “familiar” approaches by consensus (≥ 75% endorsement). Greater variation was observed in practices identified by consensus as most often used, useful, and research supported, depending upon the target problem. Findings provide an inventory of practices, reflective of the multidisciplinary language and approaches of expert ASD providers. This inventory may be used to better assess what constitutes usual care for youth with ASD in the United States. Moreover, findings offer insights from clinical experts regarding the range and acceptability of practices that may inform and ground treatment research, dissemination, and implementation efforts.

Autism spectrum disorder (ASD) is a prevalent lifelong neurodevelopmental condition with a profound impact on affected individuals and their families (Volkmar, Rogers, Paul, & Pelphrey, 2014). Although social communication difficulties and repetitive behaviors are the core deficits of ASD, other mental health comorbidities are highly prevalent, with anxiety and externalizing disorders being among the most common and impairing (Jang, Dixon, Tarbox, & Granpeesheh, 2011; Simonoff et al., 2008). The social and mental health difficulties of youth with ASD may grow as they age, due to increasingly complex social and academic demands, growing self-awareness of difficulties and numerous other adversities, such as bullying, victimization, and inconsistent or inadequate support and transition services (Gotham, Brunwasser, & Lord, 2015; Kerns, Newschaffer, & Berkowitz, 2015; Picci & Scherf, 2015). This complexity is reflected in a wide diversity of treatment providers (e.g., psychologists, medical providers, special education teachers) and intervention practices, from those that are evidence-based (i.e., supported through research; Kazdin, 2008) to those that are preference-based (i.e., based on professional expertise, and patient values, preferences, and needs; Dowie, 1996), that are involved in the treatment of youth with ASD. This study aimed to better understand—and bridge—the gap between evidence-based care and preference-based or “usual care” in ASD by utilizing multidisciplinary expert feedback and consensus to revise and expand an inventory of literature-derived ASD intervention practices. As a secondary aim, we also evaluated consensus (or the lack thereof) among expert ASD providers regarding the usefulness, common use, and research support for these practices as tools for reducing anxiety, externalizing behavior (e.g., tantrums, aggression), and social deficits in school- to transitioned-age youth 7–22 years of age with ASD (hereto referred to as “youth”).

Research points to widespread delay in the uptake of evidence-based practices (EBPs) in community settings across various health care sectors (Brownson, Colditz, & Proctor, 2012), including therapeutic interventions for ASD (Brookman-Frazee, Taylor, & Garland, 2010; Dingfelder & Mandell, 2011). Access to quality behavioral and mental health care, in particular, remains one of the largest unmet service needs for youth with ASD, due in part to the lack of appropriately trained providers (Chiri & Warfield, 2012), which leads to observed health disparities for this population (Benevides, Carretta, & Lane, 2016). This research-to-practice gap illustrates ongoing challenges integrating research knowledge with that gained through real-world services provision to promote interventions that are not only effective but also acceptable, feasible, and sustainable in diverse community settings (Chambers, Glasgow, & Stange, 2013). Moreover, it suggests that efforts to characterize and strengthen usual care may be more successful if grounded in the perspectives, language, and expertise of providers and patients (Green et al., 2015). Some investigators have questioned the priority given to interventions identified through tightly controlled trials, which often do not reflect the clinical populations, providers, and constraints of routine practice and thus may ultimately be a poor fit for community settings (Weisz, Ugueto, Cheron, & Herren, 2013). Along these lines, it has been argued that intervention practices should be both evidence- and preference-based to achieve full implementation (Dowie, 1996).

Efforts are already under way to identify preference-based practices in the broader field of implementation science for children’s mental health. Garland, Hawley, Brookman-Frazee, and Hurlburt (2008) and Garland et al. (2010) conducted a series of studies to illustrate current practices—including EBPs and common understudied approaches (e.g., use of play techniques)—for behavioral problems in youth who present to outpatient community mental health settings. Further, recent research has favored mixed-method approaches, wherein the perspectives and expertise of stakeholders, providers and patients are used to inform implementation efforts (Drahota, Aarons, & Stahmer, 2012; Green et al., 2015).

Whereas some of these efforts have been applied to better understand and improve usual care for young children with ASD, there is limited research regarding the range and acceptability of practices currently offered to youth with ASD in community settings (Paynter & Keen, 2015). Youth with ASD receive intervention services through diverse settings, including special education, medical, specialty ASD, and developmental (e.g., speech/language and occupational therapy) and community mental health centers (Dingfelder & Mandell, 2011). Within these settings, they also encounter a range of clinical disciplines such as psychologists, behavioral therapists, educators, social workers, psychiatrists, pediatricians, nurses, speech-language pathologists, occupational therapists, physical therapists, audiologists, and neurologists (McLennan, Huculak, & Sheehan, 2008). These service settings and providers bring distinct therapeutic perspectives to youth with ASD, have different education levels and training, utilize different terminology, and employ varying standards for evaluating the need for and effectiveness of interventions (Christon, Arnold, & Myers, 2015). Given that ASD providers in community settings often use strategies with varied and sometimes no research support for young (younger than 6 years) and school-age children with ASD (Brookman-Frazee et al., 2010; Paynter & Keen, 2015), researchers must look beyond EBPs and consult with clinical experts to accurately and comprehensively assess what constitutes usual care.

Although the challenges of translating research to practice are multidetermined and related to factors within multiple levels (Bauer, Damschroder, Hagedorn, Smith, & Kilbourne, 2015), ranging from systems (e.g., health care policy, insurance mandates) to individuals (e.g., treatment preferences, demographic characteristics), provider perspectives and provider-level factors (e.g., knowledge, interest, perceived competence) may be particularly useful in (a) illuminating the activities of usual ASD care and (b) identifying preference-based practices that may or may not be supported by rigorous research. Specifically, expert providers are well positioned to educate researchers about different practices used to support youth with ASD and to identify differences in terminology used to describe these practices. The expertise of ASD providers who represent a variety of disciplines, clinical settings, and geographic locations may be essential to develop questions that will reflect the diverse services youth with ASD encounter in community settings. Furthermore, the diversity of professional disciplines and settings involved in ASD services suggests a particular role for provider input when considering methods to promote the adoption, uptake, and sustained use of EBPs (Stahmer & Aarons, 2009). Expert providers may also have a particular wealth of knowledge regarding the acceptability and feasibility of different practices relative to other usual care providers due to their focus and years of experience in the area of ASD intervention.

Another movement within dissemination and implementation research pertinent to our goal of better characterizing and identifying usual care practices is the identification of core “practice elements” within EBPs (Chorpita, Daleiden, & Weisz, 2005). Chorpita et al. (2005), as well as others (Garland et al., 2008; Wong et al., 2015), have demonstrated that EBPs often comprise a set of essential content, processes, and methods of delivery. Rather than attempting to facilitate the adoption of multiple large “treatment packages” by providers, identifying core practice elements across EBPs may allow for more effective and efficient efforts at EBP dissemination, use and sustainment (Chacko, Fabiano, Doctoroff, & Fortson, 2017). This approach may also elucidate the types of specific intervention strategies that are used by a range of providers from various disciplines, a particularly important approach when considering the large range of professional disciplines that support youth with ASD.

In summary, despite an increase in the number of school and transition-age youth with ASD being treated in community settings, little is known about which specific intervention practices are being used with this population and the extent to which various providers are aware of, use, and recommend EBPs relative to other, less researched but preference-based practices for addressing the key concerns of anxiety, externalizing behavior, and social deficits. Expert ASD providers represent a vital resource for researchers aiming to capture the broad range of practices that encompass usual care for youth with ASD and illuminate and transcend lexical barriers. They are also likely to have key insights regarding which practices, evidence-based or not, may be most acceptable and useful to community providers. As such, this study used the feedback and consensus of expert ASD providers to develop an inventory of usual care practices for youth with ASD and, as a secondary aim, to characterize this group’s familiarity with and perceptions of these practices. These aims represent essential steps toward accurately and comprehensively characterizing usual care among ASD providers in the community and to evaluating and disseminating EBPs that are appealing and feasible to providers and, thereby, deserving of further empirical attention.

METHOD

Delphi Methodology

The Delphi technique uses multiple iterations of questionnaires completed by a panel of selected experts in a particular field of interest to build consensus around a topic using both qualitative and quantitative methods (Hsu & Sandford, 2007). The Delphi process is characterized by four features: anonymity, iteration, controlled feedback, and the statistical aggregation of group resources (Rowe & Wright, 1999). Anonymity reduces the effects of group-based processes, such as social pressures to conform, that are often problematic when attempting to achieve consensus (Hsu & Sandford, 2007). Multiple iterations and controlled feedback (e.g., an anonymous summary of results) allow and encourage experts to reevaluate their initial judgments about the information they provided in previous rounds and provide qualitative feedback to provide new insights and clarify information from one round to the next (Hsu & Sandford, 2007). At the end of each round, the data provided by participants are aggregated and statistically analyzed, resulting in a final product that weighs responses of each participant equally (Rowe & Wright, 1999). Quantitative data provide an empirical approach to achieve item consensus, whereas qualitative feedback about questionnaire content improves the usability, acceptability, and comprehensiveness of the survey (Hsu & Sandford, 2007). Although consensus is a goal of the Delphi, the criterion for consensus and the number of rounds conducted should be determined a priori according to the research question and study rationale, rather than determined by when consensus is achieved (Diamond et al., 2014). Consensus criteria and number of rounds should also suggest stabilization in opinion, defined in some studies as less than a 15% change in ratings from one round to the next (von der Gracht, 2012).

Delphi methodology was used in the present study to develop a provider- and literature-informed inventory of usual care practices for youth with ASD and to develop consensus regarding which practices were considered familiar as well as most useful, commonly used, and research supported to address social difficulties, anxiety, and externalizing problems in ASD. Two rounds were planned so that experts could first review, rate, and suggest changes to an initial list of practices and then, in Round 2, rate a revised list of practices and adjust ratings after reviewing the mean ratings for each practice from Round 1.

Participants

A combination of snowball and stratified purposeful sampling (by discipline and region) was used to recruit a sample of 66 expert ASD providers representing disciplines who most often serve youth with ASD: school and clinical psychology, behavior analysis, social work, special education, psychiatry and nursing, and “other” (e.g., allied health professionals including speech language pathologists and occupational therapists; cf. Christon et al., 2015). Ninety-nine of the 159 experts invited to participate responded to the survey. Among responders, 33 were screened out due to insufficient expertise (see inclusion criteria), resulting in a final Round 1 sample of 66 experts (including 13 partial completes). Eighty percent (n = 53, including five partial completes) of Round 1 experts completed Round 2, a relatively large and robust sample exhibiting high retention for the Delphi approach (Hsu & Sandford, 2007). The majority of providers (73%, n = 48) were recruited through the professional contacts of the researchers, and another 27% (n = 18) were recruited through broader advertisement (distribution of flyers to mailing lists and networks) and a snowball approach, in which personal contacts and Delphi participants were encouraged to notify other potential expert providers in the community about the study. Inclusion criteria were (a) work with youth with ASD between 7–22 years of age, (b) greater than 50% of caseload (clients, patients, students) was youth with ASD, (c) 5 or more years of experience working with youth with ASD, and (d) treatment of 50 or more youth with ASD during the last 5 years.

Experts varied in geographic location, provider discipline, and other characteristics (see Table 1). The majority had 5 to 15 years working with youth with ASD, including regular treatment of anxiety (71.2%), externalizing behaviors (89.4%), and/or social skills deficits (92.4%). Experts worked in varied settings and reported serving diverse youth with ASD in terms of age, race/ethnicity, socioeconomic status, and potential intellectual disability. Providers reported working with a majority of male individuals with ASD (n = 56; 84.4%) and with youth from suburban (n = 55; 83.3%) and urban (n = 50; 75.8%) but fewer rural (n = 7; 12%) areas.

TABLE 1.

Expert Provider Characteristics and Clients Served

Expert Provider Characteristic	n (% of Total)	Client Characteristic	n (% of Total)
Region		Age Range (Years)a
New York City	23 (34.8%)	7–9	46 (69.7%)
Long Island	13 (19.7)%	10–12	43 (65.2%)
San Diego	11 (16.7%)	13–15	39 (59.1%)
Philadelphia	10 (15.2%)	16–18	37 (56.1%)
Chicago	9 (13. 6%)	19–22	28 (42.4%)
		19–22	28 (42.4%)
		Other	27 (40.9%)
Clinical Disciplinea		Ethnicitya
Clinical Psychologist	23 (34.8%)	White	65 (98.5%)
Behavior Analyst	19 (28.8%)	Hispanic/Latino	39 (59.1%)
Other (SLP, OT)	14 (21.2%)	African American	34 (51.5%)
School Psychologist	6 (9.1%)	Asian	30 (45.5%)
Social Worker	6 (9.1%)	Hawaiian/	4 (6.1%)
Special Education	4 (6.1%)	Pacific Islander
Teacher	2 (3%)	Other	2 (3.0%)
Psychiatrist	1 (1.5%)	Native American	1 (1.5%)
Nurse Practitioner	1 (1.5%)
Years in Practice		SESa
5–10	24 (36.4%)	High	48 (72.7%)
11–15	18 (27.3%)	Medium	57 (86.4%)
16–20	13 (19.7%)	Low	40 (60.6%)
21–30	9 (13.6%)	Don’t Know	2 (3%)
30 or More	2 (3.0%)
% Time Working With ASD Youth		Comorbid Intellectualb Disability
51–79	27 (40.9%)	Frequently Treated	33 (50%)
80–100	39 (59.1%)	Sometimes Treated	28 (42.4%)
No. of ASD Youth Served in Past 5 Years		Never/Rarely Treated	5 (7.6%)
50–74	15 (22.7%)
75 or More	51 (77.3%)

Note: N = 66. SLP = Speech-Language Pathologist; OT = Occupational Therapist; SES = socioeconomic status; ASD = autism spectrum disorder.

^aProviders endorsed more than one category or range.

^bProviders endorsed whether they frequently, sometimes, or never or rarely work with children with ASD who have a comorbid intellectual disability.

Measures

Delphi Poll Questionnaire

We began the Delphi process with a structured Delphi Poll Questionnaire (DPQ; as in Hsu & Sandford, 2007), based on a comprehensive review of the ASD treatment literature conducted for this study (see next and Wainer et al. (2017)). The DPQ was delivered online and included (a) a series of questions regarding participant expertise and training and (b) clinical characteristics of the youth with ASD, as well as a list of 49 intervention practices, each with a description of the practice and several illustrative examples. Using a 4-point Likert rating scale from 1 (not at all) to 4 (very), providers rated their familiarity with each practice. Providers also provided 4-point Likert ratings on the usefulness, common use, and research support for practices with which they were familiar (rating > 2) for only those target problems (anxiety, externalizing, social deficits) that they endorsed treating in youth with ASD.

Development of the Initial DPQ

Intervention practices included in the DPQ were informed by a systematic literature review guided by PRISMA standards (for details, see Wainer et al. (2017)), which included reviewing the National Standards Project (National Autism Center, 2015) and Wong et al. (2015) for terminology and definitions. Strategies included in the initial DPQ were derived from 49 discrete intervention practices (i.e., techniques designed to teach specific skills, such as behavioral rehearsal or cognitive restructuring) that were reliably identified (M α = .94, range = .66–1.00) in the 357 studies that were selected for inclusion (α > .90) in the review based on a coding manual developed collaboratively by the authors, who have expertise in anxiety, externalizing behavior, and social skill interventions in ASD. Several iterations of the initial DPQ were also reviewed and revised by the research team via multiple biweekly conference calls designed to enhance the clarity, content validity, and comprehensiveness of the tool. In an effort to provide a framework that is communicable across all relevant disciplines, we eschewed a priori assumptions about what constitutes “practices” in the DPQ and instead allowed the extant, multidisciplinary literature (via the literature review) to dictate what ultimately constitutes individual practices, even when sampled disciplines may differ in their formulation of such practices (e.g., homework may be considered a specific practice element in one discipline but a medium through which specific practices are delivered in another). In addition to this review by the research team, an initial draft of the DPQ was then tested and revised using Think-Aloud and Retrospective Interviews (Dillman, Smyth, & Christian, 2014) with six expert clinical providers from different disciplines. The research team interviewed each expert to clarify his or her responses to each item of the DPQ and gather feedback regarding how the DPQ could be improved. This feedback was transcribed, coded, and used to maximize the clarity, acceptability, and feasibility of the DPQ prior to initiating the Delphi.

Procedure

Delphi Round 1

Interested participants were sent a link to the DPQ via e-mail and completed Round 1 of the DPQ online. In addition to completing the DPQ, participants were encouraged to provide open-ended comments about and suggest other names they use to describe each practice, as well as list practices that were missing but utilized in clinical practice for youth with ASD. This strategy was used to improve DPQ content in two ways: (a) by ensuring that terminologies from all disciplines were represented and (b) by providing a more comprehensive and diverse (e.g., not simply limited to researched practices) provider-informed list of clinical and educational interventions for youth with ASD. Frequencies of participants’ familiarity responses to items on the DPQ, along with qualitative feedback (e.g., comments, suggestions in open text boxes), were synthesized by the research team, who removed, added, combined, modified, and renamed items and item descriptions to create a revised DPQ for Round 2.

Delphi Round 2

A revised 55-item DPQ was distributed to experts for a second round of ratings and consensus building (see the appendix for the final list of practices, practice definitions, and examples, presented to participants in Round 2). Following standard Delphi procedures, participants were informed that the DPQ had been revised based on Round 1 comments and that they would have an opportunity to revise their judgments after considering a summary of Round 1 results and also to respond to items that were revised or added. Experts were provided with statistics on participant expertise (e.g., percentage of practice devoted to ASD, number of individuals with ASD served) and mean ratings regarding the perceived utility, common use, and research support for each Round 1 practice, before providing their second round of ratings.

Analytic Plan

Consensus was operationalized a priori as 75% agreement or greater between experts in Round 2 in order to reflect the opinions of a substantial majority (as in Strang et al., 2016). This standard was applied to the final round of DPQ data to identify which practices were familiar and considered useful, commonly used, and research supported for different target problems.

RESULTS

Preliminary Analyses

T tests and chi-square analyses correcting for multiple comparisons (p = .05/15 = .003) revealed no significant differences in the expertise level (e.g., years of education; hours spent, percentage of practice and number of years devoted to youth with ASD), recruitment site, professional discipline, treatment focus (anxiety, externalizing, social skills) or patient characteristics (age, ethnicity, sex, clinician-reported comorbid intellectual disability) of those who did (n = 53) and did not (n = 13) participate in Round 2, suggesting that attrition was not selective.

Qualitative Revision of Practices

Results and feedback to the Round 1 DPQ (initially 49 items) resulted in a revised DPQ consisting of 55 intervention practices for youth with ASD (see Tables 3–5 for list of final DPQ practices and the appendix for the full names, definitions, and examples provided for each practice). This qualitative process resulted in the addition of 10 intervention practices (task analysis/chaining, differential reinforcement, motivational interviewing, parent coaching, imitating the child, communicative temptations, environment structuring, social context structuring/lunch bunch, psychoeducation, acceptance strategies), the combination of six practices identified as highly similar or overlapping into three practices (e.g., behavioral rehearsal/role playing, peer reinforcement/peer monitoring, shaping/reinforcing attempts); the deletion of one practice (peer collaboration); and changes in the names, definitions, and/or examples provided for 28 other intervention practices. Notably, though potentially related to the “didactic teaching” item in Round 1, psychoeducation was suggested as a distinct practice by providers and thus made a unique practice from didactic teaching in Round 2.

TABLE 3.

Anxiety: Round 2 Consensus around Perceived Usefulness, Use, Research Support and Familiarity with Intervention Practices, Ranked by Perceived Usefulness

Intervention Practice	% Useful Treatment	% Commonly Used	% Research Supported	% Familiar With
Relaxation	100.00%	97.00%	97.10%	90.60%
Cognitive Restructuring/Using Positive/Coping Self-Talk	100.00%	86.10%	80.60%	94.30%
Visual Tools	100.00%	85.70%	77.10%	100.00%
Stories/Vignettes	100.00%	82.80%	40.00%	100.00%
Graduated Exposure/Systematic Desensitization	97.20%	88.80%	94.40%	98.10%
Psychoeducation	97.10%	91.20%	50.00%	86.80%
Priming	97.10%	88.60%	82.90%	100.00%
Self-Management	97.10%	55.90%	38.20%	98.10%
Self-Awareness of Bodily Responses	97.00%	90.90%	84.80%	86.80%
Behavioral Rehearsal/Role-Playing	94.80%	89.40%	71.10%	98.10%
Modeling or Imitation	94.30%	80.00%	42.90%	100.00%
Shaping	94.10%	70.60%	52.90%	98.10%
Meditation	93.10%	55.20%	51.70%	69.70%
Didactic Teaching, Social Scripts, Instructional Learning	91.70%	63.90%	25.00%	98.10%
Functional Behavioral Assessment	91.50%	62.80%	37.10%	100.00%
Mindfulness Exercises	90.70%	71.90%	65.50%	79.30%
Choice Making/Providing Choices	89.20%	75.60%	29.70%	100.00%
Acceptance Strategies	88.90%	59.20%	25.90%	66.00%
Motivation by Incorporating Special Interests Into Activities	88.50%	77.10%	45.70%	98.10%
Homework	88.50%	74.30%	71.40%	100.00%
Response Prevention or Ritual Prevention	87.90%	60.60%	75.80%	90.50%
Parent Coaching	85.30%	70.60%	52.90%	96.20%
Functional Communication Training	84.40%	71.90%	46.90%	90.60%
Socratic Discussions	83.90%	61.30%	25.80%	83.00%
Positive Reinforcement	83.30%	80.50%	41.70%	100.00%
Prompt Fading	82.80%	71.40%	42.90%	100.00%
Performance Feedback	81.80%	69.70%	45.50%	92.50%
Sensory Breaks, Physical Activity, or Gross Motor Movement	80.60%	55.50%	33.30%	90.60%
Prompting	80.00%	80.00%	40.00%	100.00%
Environmental Structuring	80.00%	77.10%	31.40%	98.20%
Embedding Special Interests in Social Interaction	77.20%	57.10%	20.00%	100.00%
Noncontingent Reinforcement or Built in Breaks	76.50%	47.00%	11.80%	96.20%
Extinction	74.30%	45.70%	28.60%	100.00%
Distance Education, Telehealth, or Remote Training	74.10%	11.10%	3.70%	66.00%
Reinforcement Schedules	71.40%	54.30%	25.70%	100.00%
Motivational Interviewing	71.40%	35.00%	10.70%	75.50%
Token Economy	68.60%	51.50%	31.40%	100.00%
Peer Modeling or Peer Mentoring	68.60%	42.90%	14.30%	100.00%
Games and Activities That Require Social Interaction	67.60%	55.90%	5.90%	96.20%
Differential Reinforcement	66.60%	57.60%	30.30%	94.40%
Video/Audio Modeling	64.50%	19.40%	12.90%	84.90%
Technology-Aided Instruction and Intervention	63.70%	24.30%	6.10%	83.10%
Task Analysis/Chaining	62.90%	44.90%	34.30%	100.00%
Suppression Approach	58.60%	31.00%	0.00%	73.50%
Social Context Structuring (Lunch Bunch)	57.60%	42.40%	9.10%	84.90%
Stimulus Control	54.60%	45.40%	12.10%	94.30%
Use of Song	47.80%	13.00%	0.00%	64.20%
Games & Activities (Noncomputerized) Requiring Nonverbal Skills	44.40%	18.50%	0.00%	69.80%
Computerized Activities Requiring Nonverbal Skills	41.60%	16.70%	0.00%	56.60%
Peer Monitoring and/or Peer Reinforcement	38.50%	19.20%	7.70%	73.60%
Behavior Contract/Contingency Contract	36.90%	31.60%	5.30%	100.00%
Communicative Temptations	25.90%	25.90%	3.20%	86.80%
Imitating the Child	25.00%	25.00%	7.10%	79.20%
Response Cost	12.10%	6.00%	3.00%	94.40%
Time-Out	2.90%	17.60%	0.00%	98.10%

Note: Anxiety, N = 38. Bold font indicates practices familiar to ≥ 75% of experts. Dark gray shows ≥ 75% agreement, and light gray shows ≤ 25% agreement.

TABLE 5.

Social Skills: Round 2 Consensus Around Perceived Usefulness, Use, Research Support, and Familiarity With Intervention Strategies, Ranked by Perceived Usefulness

Intervention Practice	% Useful Treatment	% Commonly Used	% Research Supported	% Familiar With
Prompting	100%	93.5%	84.8%	100.00%
Shaping	100%	91.1%	77.8%	98.10%
Behavioral Rehearsal/Role-Playing	100%	95.9%	75.5%	98.10%
Video/Audio Modeling	100%	51.2%	69.2%	84.90%
Motivation by Incorporating Special Interests Into Activities	100%	93.4%	60.0%	98.10%
Positive Reinforcement	97.9	97.9%	66.7%	100.00%
Didactic Teaching, Social Scripts, Instructional Learning	97.9	91.4%	59.6%	98.10%
Modeling or Imitation	97.8	91.3%	87.0%	100.00%
Visual Tools	97.8	91.1%	86.7%	100.00%
Priming	97.8	93.5%	80.4%	100.00%
Prompt Fading	97.8	91.3%	78.3%	100.00%
Games and Activities That Require Social Interaction	97.8	93.4%	62.2%	96.20%
Embedding Special Interests in Social Interaction	97.8	89.1%	54.3%	100.00%
Functional Communication Training	97.6	80.9%	42.9%	90.60%
Stories/Vignettes	95.7	93.5%	28.3%	100.00%
Peer Modeling or Peer Mentoring	95.6	74%	78.3%	100.00%
Homework	95.6	80.4%	65.2%	100.00%
Token Economy	95.6	73.3%	57.8%	100.00%
Technology-Aided Instruction and Intervention	94.7	55.3%	31.6%	83.10%
Parent Coaching	93.2	68.1%	56.8%	96.20%
Psychoeducation	92.3	84.6%	48.7%	86.80%
Cognitive Restructuring/Using Positive/Coping Self-Talk	91.5	66%	19.1%	94.30%
Performance Feedback	90.5	83.4%	66.7%	92.50%
Social Context Structuring (Lunch Bunch)	89.5	71%	44.7%	84.90%
Self-Management	88.9	46.7%	35.6%	98.10%
Differential Reinforcement	88.1	73.8%	50.0%	94.40%
Reinforcement Schedules	86.9	65.3%	56.5%	100.00%
Peer Monitoring and/or Peer Reinforcement	84.8	48.5%	39.4%	73.60%
Communicative Temptations	84.7	69.2%	41.0%	86.80%
Games & Activities (Noncomputerized) Requiring Nonverbal Skills	84.4	59.4%	40.6%	69.80%
Environmental Structuring	84.1	75%	40.9%	98.20%
Task Analysis/Chaining	82.2	60%	33.3%	100.00%
Suppression Approach	78.8	72.7%	12.1%	73.50%
Functional Behavioral Assessment	76.6	48.9%	31.9%	100.00%
Socratic Discussions	75.6	56.7%	21.6%	83.00%
Choice Making/Providing Choices	75.5	67.4%	24.5%	100.00%
Computerized Games & Activities Requiring Nonverbal Skills	74.1	40.7%	3.7%	56.60%
Motivational Interviewing	73.5	32.3%	8.8%	75.50%
Stimulus Control	69.8	58.1%	20.9%	94.30%
Imitating the Child	66.7	33.4%	22.2%	79.20%
Graduated, Graded, or Habituated Exposure/Desensitization	66.7	54.2%	20.8%	98.10%
Behavior Contract/Contingency Contract	62	30%	26.0%	100.00%
Distance Education, Telehealth, or Remote Training	58	6.4%	3.2%	66.00%
Extinction	57.5	36.1%	19.1%	100.00%
Use of Song	56.7	16.7%	0.0%	64.20%
Self-Awareness of Bodily Responses	51.3	38.5%	5.1%	86.80%
Noncontingent Reinforcement or Built-In Breaks	45.4	34.1%	11.4%	96.20%
Acceptance Strategies	43.3	16.7%	20.0%	66.00%
Relaxation	34.2	29.2%	2.4%	90.60%
Mindfulness Exercises	33.4	25%	0.0%	79.30%
Exercise, Sensory Breaks, Physical Activity, Gross Motor Movement	25.6	27.9%	0.0%	90.60%
Meditation	22.6	3.2%	6.5%	69.70%
Response Prevention or Ritual Prevention	22	9.7%	4.9%	90.50%
Response Cost	18.6	25.6%	9.3%	94.40%
Time-Out	4.4	2.2%	0.0%	98.10%

Note: Social skills, N = 50. Bold font indicates practices familiar to ≥ 75% of experts. Dark gray shows ≥ 75% agreement, and light gray shows ≤ 25% agreement.

Familiar Intervention Strategies

Consensus improved from Round 1 to Round 2 of the DPQ, though not drastically (< 15% change), supporting a stabilization of opinion between rounds (see von der Gracht, 2012). The majority of intervention practices (86%; 48 practices) presented in the refined Round 2 DPQ were identified by consensus (endorsed by ≥ 75% of sample) as familiar strategies for youth with ASD (in comparison to 80% or 41 practices in Round 1). Of these, 18 practices (32%) were familiar to the entire expert sample (100% agreement) in Round 2, in contrast to only 10 practices (20%) in Round 1. Results also point to eight strategies that were less widely recognized by the expert sample (familiarity = 57%–74%): peer monitoring, meditation, acceptance, use of song, distance education/telehealth/remote training, games and activities (noncomputerized) requiring nonverbal skills, games and activities (computerized) requiring nonverbal skills, suppression. Notably, five of these strategies (meditation, acceptance strategies, distance education/telehealth/remote training, and computerized and noncomputerized games that require nonverbal skills) were rated as “not at all familiar” (as opposed to “somewhat familiar”) by 8% to 15% of the sample. In addition, 8% to 15% of experts were also “not at all familiar” with video/audio modeling and motivational interviewing techniques.

Expert consensus around the utility, common use, and research support for Round 2 practices as interventions for anxiety, externalizing behaviors, and social deficits in youth with ASD was also assessed amongst providers who treated these target problems (see Tables 2–5).

TABLE 2.

Expert Consensus (≥ 75% Agree) Regarding Strategy Use, Usefulness, and Evidence Base

Target Symptom	Anxiety Problems	Externalizing Behaviors	Social Skill Deficits
Useful	32 strategies total	33 strategies total	6 strategies total
Useful, Common, and Research Supported	Relaxation	FBA	Prompting
	Cognitive restructuring	Visual tools	Shaping
	Visual tools	Differential reinforcement	Behavioral rehearsal/Role-playing
	Graduated exposure/Desensitization	Positive reinforcement	Modeling/Imitation
	Priming	Prompting	Visual tools
	Self-awareness of bodily responses	Priming	Priming
		FCT	Prompt fading
		Choice making/Providing choices
		Reinforcement schedules
		Prompt fading
		Shaping
		Token economy
		Extinction
Useful but Limited Agreement (< 50%) Regarding the Evidence Base	Stories/Vignettes	Self-awareness of body response	FCT
	Self-management	Stories/Vignettes	Stories/Vignettes
	FBA	Exercise/Sensory breaks	Tech-aided instruction
	Choice making	Modeling/Imitation	Psychoeducation
	Motivating with special interests	Behavior rehearsal/Role-playing	Self-management
	FCT	Performance feedback	Lunch bunch
	Positive reinforcement	Tech-assisted instruction	Peer monitoring
	Prompt fading	Time-out	Communicative temptations
	Performance feedback		Games that require nonverbal
	Sensory breaks		Environmental structuring
	Prompting		Task analysis/Chaining
	Environmental structuring		FBA
	Extinction
Useful but Lack of Agreement (< 25%) Regarding Evidence Base	Didactic teaching	Cognitive restructuring	Cognitive restructuring
	Acceptance strategies	Didactic teaching	Suppression approach Socratic discussions
	Socratic discussions	Peer modeling/Mentoring
	Embedding social interests into social interaction		Choice making/Providing choices
	Noncontingent reinforcement
Useful but Limited Agreement (< 50%) Regarding Common Use	Noncontingent reinforcement	Peer modeling/Mentoring	Peer modeling/Mentoring
		Tech-aided Instruction	Tech-aided instruction
			FBA
Not Useful	Time-out	None	Time-out
	Response cost

Note: FBA = functional behavioral assessment ; FCT = functional communication training.

Anxiety

Among the 38 experts who endorsed treating anxiety, consensus indicated 32 practices as “useful” for reducing for anxiety in youth with ASD. Consensus also identified six (11%) of these useful practices as commonly used and research supported (see Tables 2 and 3). An additional 13 practices were perceived as useful but had limited consensus (< 50% agreement) regarding their evidence base for treating anxiety in ASD. Five practices were identified as useful despite a lack of consensus (< 25% agreement) regarding their evidence base. For all those practices deemed “commonly used” by consensus, there was also consensus about their usefulness. Notably, for one practice (i.e., noncontingent reinforcement) there was consensus around its usefulness but not its frequency of use (< 50% rated it as a commonly used strategy for anxiety).

Externalizing Behaviors

Amongst the 46 experts who endorsed treating externalizing behaviors, consensus identified 33 “useful” practices for reducing externalizing behavior (e.g., tantrums, aggression) in ASD (Tables 2 and 4). Of these useful practices, consensus also identified 13 (39%) as commonly used and research supported to treat externalizing behavior in ASD, a higher proportion than seen for anxiety. Eight practices were identified as useful by consensus, though there was limited consensus (< 50% agreement) regarding their evidence base, and an additional three practices were identified as useful by expert consensus despite a lack of consensus (< 25% agreement) regarding their evidence base. In addition, two practices (peer modeling/mentoring, tech-aided instruction) were identified as useful by consensus, despite some disagreement (< 50% agreement) regarding how commonly they are used to treat externalizing behaviors in ASD.

TABLE 4.

Externalizing Behavior: Round 2 Consensus around Perceived Usefulness, Use, Research Support and Familiarity with Intervention Practices, Ranked by Perceived Usefulness

Intervention Practice	% Useful Treatment	% Commonly Used	% Research Supported	% Familiar With
Functional Behavioral Assessment	100.00%	100%	95.3%	100.00%
Visual Tools	100.00%	95.1%	92.7%	100.00%
Differential Reinforcement	100.00%	89.5%	76.3%	94.40%
Environmental Structuring	100.00%	90%	62.5%	98.20%
Positive Reinforcement	97.70%	100%	84.1%	100.00%
Prompting	97.60%	95.2%	78.6%	100.00%
Priming	97.60%	88.1%	76.2%	100.00%
Parent Coaching	97.50%	75.6%	70.7%	96.20%
Functional Communication Training	97.40%	100%	94.7%	90.60%
Choice Making/Providing Choices	95.60%	91.1%	80%	100.00%
Reinforcement Schedules	95.20%	83.3%	85.7%	100.00%
Prompt Fading	95.20%	92.9%	76.2%	100.00%
Shaping	95.10%	85.4%	75.6%	98.10%
Psychoeducation	94.40%	88.9%	55.6%	86.80%
Self-Awareness of Bodily Responses	94.40%	61.1%	41.7%	86.80%
Behavior Contract/Contingency Contract	93.50%	87%	69.6%	100.00%
Stories/Vignettes	92.90%	76.2%	33.3%	100.00%
Motivation by Incorporating Special Interests Into Activities	92.70%	82.9%	51.2%	98.10%
Stimulus Control	92.50%	72.5%	72.5%	94.30%
Noncontingent Reinforcement or Built-In Breaks	92.50%	80%	62.5%	96.20%
Exercise, Sensory Breaks, Physical Activity, or Gross Motor Movement	92.30%	66.7%	41%	90.60%
Modeling or Imitation	90.50%	78.6%	47.6%	100.00%
Token Economy	90.30%	92.7%	90.2%	100.00%
Extinction	88.40%	86%	83.7%	100.00%
Self-Management	87.80%	53.7%	56.1%	98.10%
Behavioral Rehearsal/Role-Playing	84.80%	67.4%	45.7%	98.10%
Performance Feedback	84.30%	65.8%	47.4%	92.50%
Homework	83.30%	66.7%	61.9%	100.00%
Technology-Aided Instruction and Intervention	82.90%	48.6%	34.3%	83.10%
Cognitive Restructuring/Using Positive/Coping Self-Talk	81.80%	25.5%	13.6%	94.30%
Time-Out	80.50%	58.6%	39%	98.10%
Didactic Teaching, Social Scripts, Instructional Learning	79.00%	60.5%	18.6%	98.10%
Peer Modeling or Peer Mentoring	76.20%	38.1%	23.8%	100.00%
Response Cost	74.40%	69.2%	38.5%	94.40%
Relaxation	73.60%	52.6%	26.3%	90.60%
Video/Audio Modeling	72.20%	16.6%	25%	84.90%
Embedding Special Interests in Social Interaction	71.40%	59.5%	31%	100.00%
Task Analysis/Chaining	63.40%	43.9%	24.4%	100.00%
Distance Education, Telehealth, or Remote Training	63.30%	10%	6.7%	66.00%
Peer Monitoring and/or Peer Reinforcement	62.60%	28.1%	25%	73.60%
Motivational Interviewing	61.30%	29%	6.5%	75.50%
Graduated, Graded or Habituated Exposure/Systematic Desensitization	59.10%	43.2%	27.3%	98.10%
Meditation	58.30%	24.1%	13.8%	69.70%
Mindfulness Exercises	57.60%	36.4%	9.1%	79.30%
Games and Activities That Require Social Interaction	51.20%	39.1%	7.3%	96.20%
Response Prevention or Ritual Prevention	50.00%	27.5%	20%	90.50%
Acceptance Strategies	50.00%	21.5%	7.1%	66.00%
Use of Song	48.00%	8%	0%	64.20%
Communicative Temptations	45.70%	45.8%	14.3%	86.80%
Socratic Discussions	41.20%	38.4%	11.8%	83.00%
Social Context Structuring (Lunch Bunch)	40.00%	25.7%	14.3%	84.90%
Imitating the Child	37.60%	21.9%	9.4%	79.20%
Suppression Approach	29.00%	45.2%	0%	73.50%
Games & Activities (Noncomputerized) That Require Nonverbal Skills	28.60%	17.9%	3.6%	69.80%
Computerized Games & Activities That Require Nonverbal Skills	12.00%	8%	0%	56.60%

Note: Externalizing behavior, N = 48. Bold font indicates practices familiar to ≥ 75% of experts. Dark gray shows ≥ 75% agreement, and light gray shows ≤ 25% agreement.

Social Skills

Among the 50 experts who endorsed treating social deficits, consensus indicated 36 practices that were “useful” for teaching social skills to youth with ASD (Tables 2 and 5). For seven (19%) of these useful practices, there was also consensus regarding their common use and research support for building social skills in youth with ASD. Thirteen practices were identified as useful by consensus despite limited agreement (< 50%) regarding their evidence base, and an additional four practices were indicated as useful by consensus despite a lack of agreement (< 25%) regarding their evidence base. All practices rated as “commonly used” by consensus also achieved consensus regarding their usefulness. By comparison, three practices were identified as useful treatments for social deficits in ASD despite limited agreement (< 50%) regarding how commonly they are used for this purpose by providers.

DISCUSSION

Many professional disciplines with varying clinical approaches and lexicons support the behavioral health needs of youth with ASD. Expert providers, reflective of diverse disciplines and also varied practice settings and geographic locations, are particularly suited to assist researchers in navigating this complexity. Specifically, expert providers are likely aware of a broader range of practices than those tested in the research literature, which may help researchers identify different names for and applications of similar practice components across disciplines. As such, this study used two rounds of expert provider feedback to (a) revise and expand an inventory of literature-derived ASD intervention practices, and (b) to generate expert provider consensus regarding the usefulness, common use, and research support for these practices as tools for reducing anxiety, externalizing behavior, and social deficits in youth with ASD.

Findings offer an inventory of usual care practices for youth with ASD, examined and shaped by expert practitioners. From an original list of 49 literature-derived practices, expert providers suggested new practices, added terms and descriptions to existing practices, and combined similar practices to generate a revised list of 55 practices. Findings also suggest a “common core” of intervention practices familiar to expert providers representing varied geographic locations, clinical orientations, and professional capacities. Forty-eight (86%) of the 55 practices were familiar to 75% or more of the sample by the second round. Many, but not all (e.g. motivational interviewing, sensory breaks, mindfulness), of these practices reflect common elements of evidence-based ASD treatments identified by Chorpita and Daleiden (2009) and by recent systematic reviews of the literature (National Autism Center, 2015; Wong et al., 2015). Discrepancies highlight the importance of incorporating data from practitioners as well as the treatment literature when seeking to characterize usual care for youth with ASD. In contrast to reviews of evidence-based practices just noted, this study suggests a range of approaches informed by, but not limited to, the research literature. In addition, this study examined expert consensus regarding the perceived use and usefulness of practices—a novel perspective essential to identifying preference-based practices that may warrant further empirical attention.

Although most practices were familiar to most experts, findings indicated some less universal strategies (< 75% endorsement), including strategies that were “not at all familiar” to 8%–15% of experts. Lack of consensus may indicate that these approaches have not been effectively disseminated and implemented; are potentially less relevant to ASD than presumed; are particularly novel; or are specialized to certain disciplines, regions, clinical settings, or youth presentations (e.g., anxiety, externalizing disorders, older youth, nonverbal youth; Odom, 2009). Prior research suggests that the components of evidence-based interventions may vary by the target problem (e.g., autism, anxiety) and age of youth (Chorpita & Daleiden, 2009) and that implementation of EBPs may vary by provider discipline (Drahota et al., 2012). Similar explorations of the 55 different usual care practices identified herein, though beyond the scope of this article and specialized sample, represent an important direction for future research that may inform implementation efforts. The complex manner in which different factors (discipline, setting, patient population, etc.) influence provider familiarity with usual care practices will be studied in the next phase of this project via a larger, representative survey of the usual care among community ASD providers (but not necessarily experts; see Wainer et al. (2017).

Of interest, some less familiar practices, such as video modeling and technology-aided instruction, are considered EBPs for ASD (National Autism Center, 2015). Reduced familiarity with these approaches overall is consistent with the well-established lag between dissemination of clinical research and the implementation of practices (Dingfelder & Mandell, 2011). This lag may be accentuated when familiarity, proficiency, and investment in novel technologies are required for facilitating successful and sustained practice, as seen in technology-based behavioral health tools (Ramsey, Lord, Torrey, Marsch, & Lardiere, 2016). Less familiar practices, such as meditation and acceptance, may represent novel trends in treatment, consistent with a conceptual shift toward recognizing and treating anxiety and other emotional problems in ASD as well as core deficits (Pahnke, Lundgren, Hursti, & Hirvikoski, 2014).

Expert Perceptions of the Usefulness, Use, and Research Support for Usual Care Practices

There were notable areas of agreement between clinical experts and the treatment research literature around intervention practices for reducing anxiety in ASD. Encouragingly, results suggested that these EBPs have already been embraced by and proven useful to expert providers, with some receiving universal endorsements regarding their usefulness (relaxation, cognitive coping) and others high, if not 100% agreement (homework, rewards, Socratic discussion). Results also suggest that experts view some EBPs as commonly used to address anxiety in ASD, a potentially surprising result given that dissemination of EBPs is often slow (Dingfelder & Mandell, 2011). Findings may reflect a tendency for experts to over-generalize their own behavior and knowledge to others or to assume that well-established approaches for anxiety in general (e.g., exposure, cognitive restructuring) have already been extended to youth with ASD in clinical practice, though tested in research only relatively recently (Vasa et al., 2014). Using multidisciplinary terminology and behavioral examples to illustrate practice elements rather than programs may also reflect a more sensitive approach to assessing usual care, particularly among ASD providers who may use relaxation, self-talk, and habituation to help youth manage anxiety without identifying these strategies as cognitive behavioral therapy per se.

A number of practices were perceived by expert consensus to be useful but not clearly research-supported strategies for anxiety in ASD (i.e., experts were unsure of their research support). Whereas some of these strategies are already components of adapted CBTs (e.g., using social interest as motivators, positive reinforcement/rewards), suggesting that some recent findings have not yet reached ASD experts, others reflect traditional ASD-focused, behavioral interventions (e.g., prompt fading, stories/vignettes, functional behavioral analysis/assessment) that were nonetheless viewed by experts as helpful treatments for anxiety within ASD. Findings support the incorporation of behavior analytic approaches into tailored CBT for anxiety in ASD (as in Wood et al., 2009) and suggest that further integration of multidisciplinary strategies may be favored by ASD experts, a potential promising direction for the field.

Relative to anxiety and social skills, a greater number of practices related to treatment of externalizing behaviors were rated as “useful,” “commonly used,” and “research based.” Consensus for externalizing behavior strategies across these ratings likely reflects the wealth of research on interventions for problem behavior in applied behavior analysis, a field particularly associated with ASD, as well as applied behavior analysis’s focus on observable behaviors (as opposed to internal constructs like anxiety and other cognitive/affective states; Smith, 2014). Further, strong evidence is available for discrete strategies (e.g., reinforcement, shaping, token economies) as well as more comprehensive approaches (e.g., functional behavioral analysis/assessment, functional communication training) for externalizing problems in ASD (e.g., Horner, Carr, Strain, Todd, & Reed, 2002).

Social functioning is considered a core deficit of ASD; however, like anxiety, expert consensus identified fewer practices as useful, commonly used, and research supported for social skill development than were seen for externalizing behavior. Only five (prompting, shaping, visual tools, priming, and prompt fading) of the seven practices endorsed for externalizing behaviors were also identified as being useful, common, and research supported for social skills. Further, several evidence-based practices (e.g., modeling, peer modeling, rehearsal) were rated as useful for prosocial behavior and externalizing behaviors but had reduced expert consensus regarding evidence base for social skills. These findings reflect the strong behavioral tradition in the social skills literature (Romanczyk, White, & Gillis, 2005) but only recent efforts to disentangle prosocial and externalizing behaviors as distinct treatment targets in intervention research (White, Keonig, & Scahill, 2007). The lack of specific practices for treating social deficits may help explain the relatively modest effects of social interventions in ASD (Gates, Kang, & Lerner, 2017), as well as some of the mismatch between approaches and participant interest, engagement, and buy-in often seen in the community (Bottema-Beutel, Mullins, Harvey, Gustafson, & Carter, 2016)—that is, mechanistically specific, provider-endorsed, and empirically supported approaches may not yet be widely available. Thus, the present state of the literature on specific social interventions makes the comparison of provider perspectives to the current evidence base somewhat more challenging.

Of those practices that providers rated by consensus as useful for social deficits, but not necessarily well supported, it is notable that two (cognitive restructuring, Socratic discussion) are elements of cognitive-behavioral therapy (CBT). A recent meta-analysis of CBT strategies used in treating social and other symptoms of ASD found small to medium effects (Weston, Hodgekins, & Langdon, 2016), suggesting that at least some current EBPs are beginning to permeate community practice. Practices seen as useful by consensus, with mixed agreement regarding their empirical support, may reflect (a) elements of treatment packages that have not been individually tested (e.g., tech-aided instruction, psycho-education, self-management, peer monitoring, environmental structuring) and (b) practices (e.g., lunch bunch) that have not yet been empirically examined despite widespread use. Overall, identification of the perceived-useful and well-supported practices by expert consensus provides a foundation for focused ASD practitioner training and education across disciplines.

Limitations

Participants in the current study were drawn from a pool of expert providers who have extensive experience with, and knowledge of, various approaches for treating clinical concerns in youth with ASD. Experts are likely not representative of less experienced or specialized community providers (see Brookman-Frazee et al., 2010), a potential limitation. Yet this study was designed not to provide a representative picture of dissemination of intervention practices in the community but rather to develop a tool to answer such questions in future research and to shed light on how various practices are viewed and valued by expert ASD practitioners. Although a sample size of 66 participants can be considered small for traditional survey-based research, it is quite large for Delphi approaches and offers information from a range of perspectives (Hasson, Keeney, & McKenna, 2000). Indeed, even our smallest subsample of experts (n = 38), who endorsed treating anxiety in ASD, represents an appropriate sample size for this methodology. Nonetheless, future efforts to illuminate and utilize provider expertise to inform treatment research would benefit from the inclusion of more diverse, international samples, which will bring attention to health disparities as well as potentially novel intervention strategies.

A threshold of percentage agreement, determined a priori and with clear rationale, is considered an acceptable and common method for determining “consensus” within the Delphi methodology, though approaches vary (Diamond et al., 2014; Von Der Gracht, 2012). As such, consensus in this study was defined from the outset as agreement among a substantial majority (> 75%) of the sample. It is possible that a different criterion would lead to different conclusions regarding the practice recommendations. Accordingly, percentage agreements for each quality of each practice are displayed in Tables 3 to 5 so readers can examine the implications of using higher (100%) or lower (50%) consensus thresholds. Finally, the goal of this work was to develop a comprehensive list of intervention practices used to treat anxiety, externalizing behavior, and social skills deficits in ASD. Future work is needed to clarify other challenges for youth with ASD, such as depression and executive functioning deficits.

Summary and Future Research

The focus on external validity in this study, via incorporating the knowledge and perspectives of expert providers, represents a concerted effort to expand our knowledge of usual care and thus also our ability to facilitate a better fit between EBPs and usual care contexts (Stahmer, Aranbarri, Drahota, & Rieth, 2017). The final list of practices resulting from this study represents an important first step and tool for accurately surveying and describing usual care for youth with ASD in the community. As such, this clinically and empirically integrated inventory will also support and inform effectiveness, health disparities, and dissemination and implementation research, which is reliant on accurate, complete information about usual care practices, perceptions, and proficiencies (Benevides et al., 2016; Stahmer et al., 2017). Direct benefits to researchers include the ability to more accurately assess the size and scope of the research to practice gap and the needs of providers and to identify preference-based practices in the community that warrant further research (e.g., motivational interviewing, self-management, social context structuring).

A secondary aim of this Delphi study was to promote future bidirectional knowledge exchange wherein clinical expert providers and researchers collaborate to increase the relevance of ASD treatment and health disparities research, as well as dissemination and implementation efforts. Our findings provide a unique summary of what practices multidisciplinary expert providers, based on their extensive practical experience, consider promising for youth with ASD. As just noted, a critical next step will be to explore how the views of clinical experts compare to those of community providers, many of whom do not have the same level of specialization or expertise (Brookman-Frazee et al., 2010). The views of these groups are likely to be different but no less valid. Rather, both experts and community providers have important, distinct information to share with researchers regarding the complexities of serving youth with ASD, improving the dissemination and implementation of EBPs and identifying promising new approaches. Ultimately, we hope the approach outlined in this study may serve as a model for a systematic, collaborative, and iterative process to identifying needs and generating strategies for improving health care provision in many populations (Stahmer et al., 2017).

Biographies

Connor M. Kerns, Drexel University

Lauren Moskowitz, St. John's University

Tamara Rosen, Stony Brook University

Amy Drahota, Michigan State University

APPENDIX

Usual Care for Youth with Autism Spectrum Disorder (ASD): A Delphi Poll Round 2

Practice Element	Definition	Example(s)
1. Behavior Contract or Contingency Contracting	A written document that describes a desired behavior and the type and amount of reward or reinforcer that the child will receive for performing or increasing that behavior. This may be part of a Behavior Intervention Plan (BIP).	Example 1: A written agreement between a teacher and student that, if the student completes 20 math problems every day (at least 80% correct), without tearing up the worksheet, from Monday through Friday, he will receive a get-out-of-homework pass on Friday.
		Example 2: A written agreement that, if the child wears her shirt and pants all day at school (i.e., they are still on when she gets home from school), she can have ice cream for dinner.
		Example 3: A daily report card.
2. Behavioral Rehearsal/Role Playing	Having the child rehearse behavior(s) in treatment before using them in real-life situations.	Example 1: Practicing (with a provider) how to take deep breaths to relax before doing so in a real-life, stressful situation.
		Example 2: Helping a child brainstorm ways to have a difficult conversation with a classmate. The provider could role-play first as the child, then allow the child to role-play as (him/her)self.
3. Choice Making/Providing Choices	Offering a child choices or options in order to accomplish a given task. This could also include following their lead during a task or activity, to ensure it is selected/preferred by them. This may sometimes include “forced choice decision-making.”	Example: Asking a child if he would like to do his math or spelling homework first, or whether he wants to use his Star Wars pencil or monkey pencil.
4. Cognitive Restructuring or Using Positive/Coping Self-talk	Helping a child to identify negative or unhelpful thoughts (e.g. I can’t do it; I will fail; Others will laugh at me) and challenge these thoughts with more accurate or realistic perspectives. This is often a key part of cognitive-behavioral therapy; conversely, it can sometimes be called “Overt Verbalization Strategies.”	Example 1: A child believes that a friend does not like her anymore because they declined an invitation to hang out. The provider probes for evidence for and against this interpretation and helps the child to consider alternatives (e.g. I have said no to her before, even though I liked her; or Just because she said no that does not mean that she is not my friend anymore).
		Example 2: Using inner coach vs self-defeater.
5. Positive Reinforcement	Providing a reward (e.g. toy, token, praise) following a specific behavior to make the child more likely to engage in that behavior again in the future. This can also be called “contingent reinforcement.”	Example: A child receives a reward after appropriately leaving a group conversation. If the child does not appropriately leave the group conversation, he does not get access to the reward.
6. Distance Education, Telehealth or Remote Training	Using technology to teach and communicate treatment concepts from two different locations, including remote supervision or video sessions.	Example 1: A provider (located at his office) is teaching a child and parent (located in their home) about deep breathing.
		Example 2: Used in phone conversations and check ins with parents and clients.
7. Didactic Teaching, Social Scripts, Instructional Learning	Offering specific instruction and direction about how to carry out a given task or behavior. This may be done using written scripts or task analysis.	Example 1: A child could be taught how to enter, maintain and exit a conversation with a peer through step-by-step instruction.
		Example 2: Didactic instruction of social skills in a structured learning curriculum.
8. Exercise, Sensory Breaks, Physical Activity or Gross Motor Movement	Physical activity that can be used to reduce or replace problem behavior, serve as an alternative behavior, or increase appropriate behavior. This does not include use of exercise as a punishment (e.g. child must do 10 laps because they broke a rule).	Example: A provider may have a child run for 15 minutes each day in efforts to reduce hyperactivity.
9. Graduated, Graded or Habituated Exposure/ Systematic Desensitization	Helping a child to reduce a fear or aversion by gradually coming in contact with and learning to tolerate and cope with the fear/aversion in situations that cause him or her small to medium distress (e.g. seeing a dog) and then, as confidence is gained, severe amounts of distress (e.g. petting a dog).	Example 1: Fear of dogs. Exposures would gradually progress from easier situations (e.g., videos, pictures, sounds) to medium (e.g. seeing a dog, being in a room with a leashed dog) to harder ones (e.g. petting a dog, feeding a dog).
		Example 2: Fear of speaking in front of the class. Exposures would gradually progress from easier situations (e.g., imagining presenting) to medium (e.g. speaking in front of the provider, the teachers, or a few people) to harder ones (e.g. larger crowd, the entire class).
10. Extinction	Discontinuing reinforcement of a previously reinforced behavior.	Example 1: A mother specifically ignores a child’s whining to sit on her mother’s lap (extinction) but picks her child up when she asks in a “big girl” voice.
		Example 2: A child is afraid of his parents driving through a certain intersection and tantrums when they approach that intersection. Normally, his parents respond by avoiding that intersection. With extinction, his parents stop avoiding that intersection and just go through it instead (they no longer reinforce his tantrums).
11. Functional Behavioral Assessment (FBA)	A systematic method of assessment to evaluate the function or purpose of problem/challenging behavior(s) in order to inform intervention. An FBA can include indirect methods (e.g., questionnaires, interviews), direct observation (e.g., collecting ABC data) and an experimental functional analysis.	Example: A child’s aggressive behavior is assessed using an FBA. It is found that the child is unable to communicate when he is hungry and is usually given food after he becomes aggressive, suggesting that the function of his problem behavior is to get food.
12. Functional Communication Training (FCT)	Teaching a child an appropriate communicative behavior that serves the same function or purpose as the inappropriate behavior. This communicative behavior could be verbal/vocal or nonverbal (e.g. sign language, PECS, augmentative communication device, etc).	Example: A child is biting people when she wants to go to the bathroom. She is taught to select a picture of a toilet when she needs to use the bathroom instead.
13. Games and Activities that Require Social Interaction such as Cooperative or Facilitated Play	Incorporating playful activities that require direct interaction with a peer into treatment.	Example 1: In the game “Freeze”, children must follow the lead of another child to maintain an interaction or play scene.
		Example 2: In groups of two, children must build an object together. They must plan and build it together using only one hand each, requiring them to interact and problem solve to complete the activity.
14. Computerized Games and Activities that Require Nonverbal Skills	Using computer games that incorporate nonverbal communication. Nonverbal skills may include face recognition, emotion recognition, following eye gaze, and perspective taking.	Example 1: Child plays a computer game where players are asked to interact with an avatar who displays highly animated facial expressions of emotion. Points are collected when the player accurately guesses the avatar’s feelings and/or intentions.
		Example 2: “Secret Agent Society”.
15. Games and Activities (Noncomputerized) that Require Nonverbal Skills	Using games or playful activities that explicitly require the participant to examine and interpret the emotions or intentions of others.	Example 1: In “Gibberish,” children are required to interpret the tone of voice and body language of another person in order to guess what activity they are doing.
		Example 2: In “Follow My Eyes to the Prize,” children are required to use only the adult’s eye gaze and nonverbal cues to locate a hidden prize in the room.
16. Homework	Giving the child or parent activities to complete between sessions or classes in order to further practice skills learned during treatment.	Example: The provider assigns the child and/or parent homework (skills practice) to do outside of the treatment sessions. Typically, homework is reviewed during the next session.
17. Motivation by Incorporating Special Interests into Activities	Teaching therapeutic concepts by incorporating or focusing on the child’s special or circumscribed interests.	Example: Using the facial expressions of a child’s favourite cartoon character to teach emotion recognition skills.
18. Mindfulness Exercises	Paying attention and focusing on the present moment using mental focus to reduce tension.	Example 1: Encouraging a child with anxiety to focus on the present moment rather than worries or future events. A child may be taught to focus on the smell, taste, sight, sound, and feel of an experience (e.g. eating, walking, drinking).
		Example 2: Practice Yoga.
19. Meditation	Training one’s mind towards relaxation and concentration. It may be used with deep breathing and/or other mindfulness strategies (e.g. increased attention to physical sensations in the body or verbal calming techniques).	Example: As part of a treatment for school avoidance, a child may be taught to engage in daily morning meditation.
20. Modeling or Imitation	Demonstrating a desired behavior and supporting the child’s imitation of that behavior.	Example 1: A provider models a wave and says “do this.” The child imitates this action and waves back.
		Example 2: A provider models using coping strategies (e. g., deep breathing) to face and remain calm in a stressful situation (e.g., public speaking).
21. Noncontingent Reinforcement or Built in Breaks	Providing the child with a reward at pre-selected times as a preemptive intervention, regardless of whether s/he attains a specified goal.	Example: A child frequently engages in escape-maintained problem behaviors. Every 2 minutes, he is given a break (even if he did not ask for or specifically earn the break).
22. Peer Modeling or Peer Mentoring	Using peers, rather than the provider, to model appropriate behaviors.	Example: A peer model is helping a child increase play skills. The peer model will demonstrate an action on a toy and the child is rewarded when s/he imitates the peer’s actions.
23. Peer Monitoring and/or Peer Reinforcement	Having a typically developing peer observe a child with ASD and either reinforce or reward specific positive behaviors.	Example 1: Every time a student with ASD makes a request, a typically developing peer notes this behavior (i.e. monitors the child with ASD) and/or praises the child with ASD afterward (i.e. reinforces the behavior).
		Example 2: After a child with ASD engages in a target behavior (e.g., raising his hand), a peer praises him and gives him a sticker.
		Example 3: Peers ignore the child with ASD when he is engaging in the target inappropriate behavior and continue interacting with him when he is NOT engaging in the target behavior (i.e., the peers reward or reinforce him for appropriate behavior).
24. Performance Feedback	Providing feedback about the specific use (or non-use) of a skill or its components either while the skill is being performed or directly following.	Example 1: Providing feedback about the child’s use of eye-contact, facial expressions, or use of gesture and body language during a social interaction in a way that provides information to the child about the use of all, part or even none of the skills learned in order to increase the youth’s knowledge and future use of the skill.
		Example 2: Providing feedback to a child after an exposure to help challenge negative thinking (e.g. Identifying that the child used cognitive restructuring to help tolerate the exposure.)
25. Priming	Preparing a child for a task or activity by previewing the task or activity briefly in some way before teaching it. May also be called Previewing, Forecasting, Preteaching.	Example 1: In order to prepare a child to take turns during a game in class, the child is shown the game and read the rules at home the night before.
		Example 2: Showing the child a video of a doctor’s office before a doctor’s appointment.
26. Prompting	Presenting a cue to initiate a specific behavior. These cues can be given before or during the performance of the target behavior to assist in the use of a specific skill or increase the likelihood of a target behavior.	Example: Prompts, INCLUDE BUT ARE NOT LIMITED TO verbal (e.g., “say hello”), auditory (e.g., setting alarm to remind child to get dressed), tactile (e.g., device that vibrates to remind child to ask a question), visual (e.g., picture of what the child is supposed to do), gestural (e.g., pointing to sink to remind child to brush his teeth), modeling (e.g., demonstrating how to brush your teeth for the child), physical/manual (e.g., physically guiding a child’s hand to pick up the toothbrush), or positional prompts (e.g., placing a target item on the table close to the student).
27. Prompt Fading	Gradually reducing prompts until the desired response occurs in the appropriate context without any prompts.	Example: To teach a child how to wash his hands, a provider might first move or guide the child’s hands in the correct hand washing motion. Then the provider might model the task and encourage the child to imitate. Once the child is successfully imitating, the provider might provide a verbal instruction for hand-washing steps. Eventually, the child is able to complete the hand washing routine without any prompts or cues from the provider.
28. Reinforcement Schedules	Controlling the timing and frequency of rewards in order to increase a desired behavior or decrease an undesired behavior.	Example: A child who is inattentive or off-task during class may be rewarded on a variable interval schedule throughout the day to help him maintain attention. The rate of reward averages to approximately one token/30 minutes of on-task behavior.
29. Relaxation	Teaching and encouraging the use deep breathing, muscle relaxation, peaceful imagery, or other calming techniques.	Example: Teaching a child to relax their muscles when they notice that they are becoming irritable or anxious.
30. Response Cost	A type of punishment in which the child loses something rewarding or desired (e.g. a prize, sticker, money, token) following the occurrence of an unwanted or inappropriate behavior. This includes taking away a privilege.	Example 1: If a child gets up out of his seat without asking his teacher, he loses tokens that he has gained during the day.
		Example 2: If a child yells at his siblings, he loses playtime on his favorite video-game system.
31. Response Prevention or Ritual Prevention	Preventing a child from avoiding or escaping a feared or distressing situation or object. Can also be similar in implementation to “response blocking.”	Example 1: A child who had an obsession about avoiding contamination is prevented from washing their hands repeatedly or engaging in other rituals (e.g. intensive cleaning) designed to reduce this fear.
		Example 2: A child with a dog phobia stands near a dog as opposed to running away from it (his/her usual response.)
32. Self-awareness of Bodily Response	Teaching a child to recognize how different emotions feel in their bodies. This can be used as a tool to help children identify and articulate their emotions more quickly.	Example: Exploring what areas of the body are active when the child is afraid of or worried about something (e.g. heart racing, fidgety) and helping them to use these physical sensations to recognize anxiety.
33. Self-management	Teaching the child to monitor, record and/or reward his or her own behaviors. This can include self-monitoring, self-reinforcement, and self-prompting.	Example: A student is taught to tally the number of times that she initiates conversations with a peer during the school day. If she reaches a target number of initiations, she earns a reward.
34. Shaping	Reinforcing behaviors that are closer and closer “approximations” of a desired behavior until each approximation is mastered. Following the mastery of an approximation, the next closest approximation of the desired behavior is rewarded. This process is repeated until the ultimate desired behavior is achieved. Note: This also includes “reinforcing attempts,” which involves rewarding any attempt to respond, in contrast to traditional shaping in which only correct responses (or responses that are equal or better than the previous response) are reinforced.	Example 1: A provider, aiming to elicit the word “cookie” from a child, might initially reward any attempt at making a sound when asked until this is mastered. Then, they may just reward the sound “c” until it is mastered. This process is repeated with the sound “coo.” This process continues until the provider only rewards the full sound “cookie.”
		Example 2: A provider wishing to support social initiation with a peer may at first reward simply standing near a peer until this can be done regularly. Then, they may reward look or smiling at the peer. This process continues until full social initiation (as defined by the Provider) is achieved.
		Example 3: A provider may reinforce any attempt to employ the target behavior (e.g., engaging with a peer.)
35. Socratic Discussions	In this approach, the provider does not give information directly to a child but instead asks a series of leading questions (e.g. What did you do? How did it turn out? What else could have happened?) to help them understand a concept or gather desired knowledge.	Example 1: The provider asks a series of leading questions (e.g. What do other kids do? What could you do?) to help the child identify how to initiate conversation on the playground.
		Example 2: A provider asks leading questions to help the child understand how they are feeling and thinking after a specific situation.
36. Embedding Special interests in Social Interaction	Guiding or structuring a social scenario (with a peer, teacher, counselor, or other) so that the content is focused around the child’s special or perseverative interest.	Example: Initiating a conversation about Thomas the Train with a child who has a special interest or preoccupation with Thomas the Train.
37. Stimulus Control	Pairing a specific behavior with a specific stimulus/cue in order to control the behavior.	Example: When a card says “hands down”, child must keep their hands on table to get a reward. When card says “break time”, the child may engage in repetitive hand motions or other motions with his/her hands.
38. Stories/ Vignettes	Using stories or short vignettes to teach children how to problem solve and/or take the perspective of others. This includes Social Stories.	Example 1: A provider describes, through story or drawings, a personalized social situation and highlights important cues to attend to for success in the situation.
		Example 2: A child has trouble being attentive during story time and engages in many disruptive behaviors. Her provider creates a social story specifically for her that she can read before school each day that describes a child acting appropriately during story time.
39. Suppression Approach	Teaching a child to consciously refrain from excessively talking about a special or perseverative interest or engaging in stereotypes so that they can better learn or use skills (i.e. building friendships).	Example: Reminding the child that his/her special interest is great and fun but should only be spoken about for a certain amount of time or while others are interested because it can be unexpected or not interesting to peers after a time.
40. Technology-aided Instruction and Intervention	The use of technology as a method for teaching or building skills. Also called Augmentative Communication, Assistive technology, Assistive/Augmentative Communication Devices.	Example 1: An iPad app that provides speech-generating help for children.
		Example 2: A virtual reality program that teaches social skills.
41. Time-out	A type of punishment in which all sources of reward or reinforcement (e.g. attention, playing with toys) are removed. The child is moved to a setting that is “less rewarding” and/or stops receiving the reward for a period of time.	Example: A teacher removes a student from his friends for 10 minutes when s/he disrupts the class.
42. Token Economy	Rewarding a child by giving him tangible symbols (e.g., tokens, points, stickers, poker chips, “catch them being good” coupons) for completing tasks or behaving in desired ways. The child is then able to exchange tokens/points for a desired object (e.g., a toy, candy) or activity (e.g., playing a game).	Example: Receiving a sticker for every time a child initiates play with a peer. After earning a certain amount of stickers, he can exchange the stickers for 5 minutes of break time (e.g. running around the soccer field).
43. Use of Song	Embedding treatment goals or steps into the context of a song.	Example 1: Creating a song that narrates the steps of a social behavior.
		Example 2: Making up a song about taking turns.
44. Video/ Audio Modeling	Using a video or audio demonstration of the correct performance of a particular task or activity. Children then enact the demonstrated task or activity.	Example 1: A video showing two children in a conversation. The child could view this and then try to reenact this later in a real-life conversation.
		Example 2: The child or another person puts together a train set that the child will enact later in order to expand play.
45. Visual Tools or Supports	The use of visual tools (e.g. pictures, written words, videos, maps, timelines, calendars or diagrams) to teach skills, behaviors, or concepts, or to increase the predictability of a situation. Also called Visual Prompts.	Example 1: Using cartoon drawings to assist the child in understanding skill development (e.g. cartoons of people interacting to teach social skills).
		Example 2: Using a visual indictor to show how much time is left in the treatment session (e.g. a visual timer or “running man” that takes a few steps forward periodically as the session progresses).
		Example 3: Using a visual schedule to inform a child about the upcoming sequence of events or activities (e. g., picture of home, dry cleaner, bank, and back home).
		Example 4: A student is presented with an Activity Schedule - a photo album containing photos that depict the sequence of steps necessary to engage in a leisure activity.
New items added in Round 2 of DPQ (based on feedback from Round 1)
46. Task Analysis/Chaining	Task Analysis: Breaking a complex task or behavior (such as putting on your coat) into smaller discrete steps (e.g., pick up coat, put left hand in left sleeve, put right hand in right sleeve, zip). The components of the task analysis form a behavioral chain, or a sequence of related behaviors that make up a skill.	Example: Shoe Tying: 1. Place left lace across the top of the shoe, 2. Cross right lace over left lace and pull it under the left lace, 3. Make a bow with the lace on right side, 4. Wrap left lace around bow, 5. Pull through hole under bow, 6. Grab both bows and pull tight.
	Chaining: Teaching one step of the behavioral chain at a time until the entire chain of steps is mastered. Forward Chaining: Teaching the first step in the behavioral chain to mastery first and adding each subsequent component one at a time. Backward chaining: Teaching the last step in the behavioral chain first and adding each previous component one at a time.
47. Differential Reinforcement	Providing reinforcement for an appropriate behavior (DRA), physically incompatible behavior (DRI), the absence of problem behavior (DRO), or a lower rate of problem behavior (DRL), while withholding reinforcement for problem behavior.	Example 1: If a student calls out in class to get attention from the teacher, DRA would involve his teacher calling on him (reinforcing him) when he raises his hand, but ignoring him when he calls out.
		Example 2: An individual is reinforced every 5 minutes he goes without yelling (DRO).
		Example 3: If a student raises his hand only 3 times or less per class, then he can be line leader (DRL).
		Example 4: A parent provides relatively more reinforcement for approaching a feared stimulus than for running away (i.e., DRI) or more reinforcement for “brave” behaviors (e.g., coping self-talk) than anxious behaviors (e.g., reassurance-seeking) (i.e., DRA).
48. Motivational Interviewing	Facilitating and engaging intrinsic motivation within the patient in order to change the behavior.	Example: Motivating the child to change social behaviors in order to make friends.
49. Parent Coaching	Modeling the technique for the parents, then having the parents attempt the technique while offering corrective feedback.	Example: Modeling positive reinforcement for a parent, then allowing the parent to try this strategy and then providing the parents with feedback to better use the strategy.
50. Imitating the child	Mirroring the child’s actions.	Example: While a child is playing with a toy truck on the floor, the therapist also plays with a toy truck on the floor at the same time, perhaps using similar or identical movements.
51. Communicative temptations	Attempting to get a verbally delayed child to talk by enticing them with various temptations.	Example 1: Eating the child’s favorite food in front of them.
		Example 2: Putting a desired object in the child’s sight but out of reach so that the child has to make a social initiation to get the object.
52. Environmental Structuring	Modifying the environment to be optimal for reaching desired therapy goal.	Example: Removing distracting objects from a room. Limiting access to preferred objects.
53. Social Context Structuring: (Lunch bunch)	Having a group of children meet together during a specified time with a specific goal in mind.	Example: Having 3 children with ASD sit at lunch together and discuss a common area of interest.
54. Psychoeducation	The stage of training in which an individual is provided with concrete information about their condition and the way it is usually manifests, with the aim of normalizing and helping the individual (or family) to become familiar with the condition more broadly.	Example 1: Written material provided to members of a support group for parents with kids with ASD.
		Example 2: A book offered to a client that describes their disorder in accessible terms.
55. Acceptance strategies	Recognizing and accepting the existence of painful thoughts and emotions.	Example: Noticing and describing painful thoughts and feelings instead of trying to change them.