A Meta-Analysis of Video Modeling Interventions to Enhance Job Skills of Autistic Adolescents and Adults

Abstract

Background:

Autistic transition-aged youth and young adults face many societal barriers to competitive integrated employment (CIE). Existing evidence-based practices (EBPs) for autistic individuals, such as video modeling (VM), may be a viable on-the-job training method to enhance employment experiences and outcomes for this population.

Methods:

The purpose of this meta-analysis was to synthesize VM studies to teach job skills for autistic individuals. We applied the Council for Exceptional Children's (CEC) Standards for Evidence-Based Practices in Special Education to evaluate the methodological rigor of included studies using a weighted coding scheme. We further evaluated methodologically sound studies by calculating an omnibus Tau-U effect size.

Results:

Twenty articles met our inclusion criteria, and 11 of those studies were classified as methodologically sound according to the CEC's criteria. Results indicate that VM is an EBP to improve job skills of autistic individuals. The overall effect size for methodologically sound studies was strong (0.91), but most studies occurred in contrived or school-based employment settings rather than CIE settings in the local labor market.

Conclusions:

Employers, transition professionals, and related service providers can consider VM a viable method to teach job skills to autistic employees. However, additional research conducted in CIE settings is needed to better understand the effects of VM in contexts where autistic employees earn regular wages.

Lay summary

Why was this study done?

Autistic adolescents and adults often experience barriers obtaining employment in their local communities. They may also benefit from on-the-job supports for successful employment. Video modeling is one intervention technique that has been used to teach a variety of skills to autistic individuals. Video modeling involves creating short video clips that show the person how to do specific skills or tasks. We wanted to learn about how video modeling has been used to teach job skills to autistic employees.

What was the purpose of this study?

The purpose of this study was to evaluate the quality of research studies that used video modeling to teach job skills to autistic employees. Understanding how video modeling interventions can be used in employment settings may help autistic employees have more positive work experiences.

What did the researchers do?

The researchers analyzed the video modeling research studies with autistic participants aged 14 years or older. We evaluated the quality of each study, type of employment setting, type of job skill, and how much the job skills improved. We used a criteria established by a professional organization, the Council for Exceptional Children, to evaluate the quality of the research studies.

What were the results of the study?

We analyzed 20 research studies and found that video modeling was an overall effective intervention to teach job skills to autistic adolescents and adults. However, most of the studies focused on general job tasks rather than employment-related social skills. In addition, most of the studies were conducted in employment settings where the autistic employees did not earn regular wages, such as school settings or internships. We encourage future researchers to study how video modeling can be used to promote competitive integrated employment in community settings.

What do these findings add to what was already known?

Prior research studies have used video modeling to teach skills such as academic, play, and social skills to autistic children and youth. This study showed us that video modeling is an effective intervention to teach job skills to autistic adolescents and adults.

What are the potential weaknesses of the study?

There are disagreements about how to evaluate the quality of research studies in the field of special education. We used a popular criterion by the Council for Exceptional Children organization, but our results may be different from other researchers. We also did not find a large number of studies, so some of our findings should be considered with caution.

How will these findings help autistic adults now or in the future?

These findings demonstrate that video modeling is an effective on-the-job training method for autistic employees. Autistic adults can use video modeling at work to learn new job skills. Employers, job coaches, and secondary transition professionals could use brief videos to support autistic employees.

Introduction

Competitive integrated employment (CIE) is a critical component of the transition to adulthood for autistic adolescents and young adults. Although additional research is needed to better understand the relationship between employment and quality of life,¹ employment generally contributes to positive mental health outcomes, enhanced community integration, and economic self-sufficiency.² Unfortunately, CIE eludes many autistic individuals. For example, the National Autism Indicators Report found only ∼14% of autistic adults work in their local communities for pay.³ When employed, autistic adults typically earn less than minimum wage and frequently only interact with co-workers who have disabilities.⁴ Employers may also lack knowledge regarding autism and how best to support their autistic employees.⁵ Autistic employees who receive on-the-job supports are four times more likely to maintain employment compared with individuals without these supports.⁶

A variety of existing educational interventions can be used to enhance the postsecondary employment outcomes of autistic individuals in workplace environments. Additionally, vocational rehabilitation (VR) research shows that autistic individuals have better employment outcomes when proper supports and accommodations are provided.⁷ Technology-based educational and vocational interventions appear particularly relevant for increasing completion of work tasks and promoting social inclusion of autistic employees. For example, in their systematic review of vocational interventions for autistic employees, Nicholas et al.⁸ found that the majority of interventions focused on supported employment and technology and media-based interventions. Such technology-based interventions were included but were not limited to performance cue systems with audio and visual prompting,⁹ covert audio coaching,¹⁰ handheld tablets,¹¹ virtual reality,¹² and combinations of these and other interventions (i.e., intervention package¹³). Video modeling (VM) was identified in several other systematic reviews as a frequently used technology-based intervention.^8,14–17

VM is an evidence-based practice (EBP) for autistic learners with a large literature base demonstrating its efficacy for a variety of skills.¹⁸ VM is based on the idea that an individual can watch a video to learn new skills. Types of VM vary according to how the video is filmed and who serves as the model in the video. Traditional VM entails the learner watching another individual (i.e., the model) performing a skill to be learned. Point-of-view VM entails the learner viewing only the salient features of the target skill (e.g., hands manipulating a cash register). Point-of-view videos are filmed from the eye level of the learner and reduce extraneous information. Video prompting involves breaking the target behavior down into discrete steps in brief video clips. Finally, video self-modeling entails the learner observing themself performing a skill. These different types of VM may be combined based on learner preferences to obtain desired outcomes.

Systematic reviews and meta-analyses have found generally positive effects for VM for autistic individuals regardless of type of VM.^19–22 For example, Qi et al.²² systematically reviewed VM interventions that focused on social communication skills and found that it met What Works Clearinghouse single-case research design standards without reservations. Similarly, Wang et al.'s²¹ meta-analysis examined the effectiveness of peer-mediated and VM interventions for autistic children and found large effect sizes to improve social interactions. In addition, Mason et al.²⁰ meta-analyzed video self-modeling studies and found a moderate magnitude of change for a variety of outcomes, with social communication and behavioral skills having relatively larger effects. However, the majority of included participants in these reviews were preschool and elementary-aged children, and the authors noted gaps in the VM evidence base for older autistic individuals.²⁰

Despite the generally positive aforementioned studies, aggregated effects of VM on job skills in different employment contexts for autistic adults remain unclear. Required job skills can vary considerably according to the employment context. This is important because autistic transition-aged youth and adults work in a variety of employment experiences that may influence the effects of VM. In particular, the Workforce Innovation and Opportunity Act of 2014²³ in the United States outlines specific criteria for CIE experiences. To be considered CIE, individuals must receive minimum wages according to the state in which they live, the employment location is typically found in the community, and opportunities for advancement are similar to employees without disabilities.^23,24 Autistic transition-aged youth may participate in work-based learning experiences, which are supervised, school-coordinated employment activities.²⁵ Work-based learning experiences can occur in either school or community settings. Unfortunately, autistic individuals may also participate in contrived, simulated, or other employment experiences in which they are not paid or earn subminimum wages (e.g., day facilities, sheltered workshops). Research indicates that ∼42% of autistic adults participate in unpaid employment activities in day facilities.³

Although the Individuals with Disabilities Education Improvement Act (IDEIA, 2004) focuses on CIE during the transition planning process, autistic individuals are less likely to be employed in their community than their peers without disabilities.²⁶ VR services for autistic employees also tend to be among the most costly and resource-intensive.⁷ Therefore, developing a better understanding of which types of on-the-job supports are most effective could potentially assist autistic employees who receive VR services. In addition, it seems plausible that effect sizes for VM on job skills might differ from prior meta-analyses focused on VM for different outcomes due to contextual differences across studies. However, this has yet to be established, and it may be that moderating factors (e.g., employment setting, participant characteristics) influence responding.

Previous employment reviews for autistic individuals have focused on interventions to improve job performance and employment outcomes broadly,^{8,14,15,17,27} with VM interventions included in many reviews. However, researchers have described the autism and employment body of literature as limited in both quality and quantity.^8,14 For example, Seaman and Cannella-Malone¹⁴ described that the majority of vocational intervention studies focused on teaching general job skills rather than employment-related social skills. Clerical, restaurant, and retail positions were common jobs in which autistic adults worked. Findings from the National Longitudinal Transition Study-2 (NLTS2) indicated similar findings that autistic adults frequently have jobs clustered in a narrow range of occupations such as food preparation, cleaning, and maintenance.²⁸ This is unfortunate because autistic individuals can be successfully employed in more diverse jobs that require interpersonal, problem solving, and other more complex skills. Accordingly, type of job skill taught (e.g., general task acquisition vs. employment-related social skill) is another unknown moderating factor that may affect the efficacy of a VM intervention.

Finally, the methodological quality of the VM intervention literature to teach job skills for autistic individuals is unclear, and the synthesized effects of this literature are unknown. The Council for Exceptional Children's (CEC) Standards for Evidence-Based Practices in Special Education²⁹ (hereby referred to as Standards for EBPs) is an approach to categorizing EBPs in the field of special education. Although VM has been identified as an EBP for autistic learners,¹⁸ there is value in determining participant characteristics and contexts in which an established EBP will likely be effective.³⁰ The effect size or magnitude of change is also unknown for this body of literature. Therefore, the purpose of this meta-analysis was to examine the methodological rigor and synthesized effects of VM studies to improve job skills of autistic adolescents and young adults. We were guided by the following research questions:

• (1)What types of employment settings were VM studies conducted in to improve job skills of autistic individuals?
• (2)What types of job skills were targeted in studies that used VM for autistic individuals?
• (3)Does the methodological rigor of included VM studies meet the EBP criteria for improving job skills of autistic individuals according to CEC's Standards for EBPs?²⁹
• (4)What is the omnibus Tau-U effect size of methodologically sound VM employment studies for autistic individuals?

Methods

Inclusion criteria and article selection

Studies were included if they met the following criteria: (1) empirical study employing an experimental design; (2) independent variable is a VM intervention to improve job skills; (3) dependent variable of improved job skills; (4) at least one autistic participant aged 14 years and older; (5) published in English in a peer-reviewed journal between 1987 (when the first VM study was published) and January 2019 (when the search was conducted). We excluded studies focused on job obtainment (e.g., job interviewing skills) or other employment outcomes (e.g., increased hours worked per week) as the dependent variable because this review focused specifically on job skill training. We selected age 14 and older because many states mandate that transition planning begins at this age, although federal requirements are age 16 according to the IDEIA (2004). We excluded the gray literature (e.g., dissertations) that had not undergone the peer-review process.

Article selection procedures consisted of electronic, hand, and ancestral searches of the literature in an iterative process, as well as solicitation of experts who published VM research. All steps of the search process were conducted by two authors with interrater agreement (IRA) reported for each step (Fig. 1). First, we used the following search string to identify potential articles in electronic databases: all (autism) OR (autism spectrum disorder) AND all (employ*) OR (vocation*) AND (video) OR (video model*) OR (video instruction). We searched the following electronic databases: Academic Search Complete (EBSCO), APA PsycINFO (EBSCO), Educational Resources Information Center (ERIC; EBSCO), and Web of Science (Clarivate), which yielded 278 articles. Two authors read titles and abstracts to determine if articles met the inclusion criteria. We read 18 studies in full, and 16 of these 18 studies were included in the review using a consensus approach to determine article eligibility.

FIG. 1.

Study inclusion flowchart. Figure adapted from Moher et al.⁷² IRA, interrater agreement.

The two authors then conducted hand searches in academic journals that published a minimum of two included articles modeling after published systematic reviews that utilized this approach.^31–33 Hand searched journals included Developmental Neurorehabilitation, Division on Autism and Developmental Disabilities Online Journal, Education and Training in Autism and Developmental Disabilities, Focus on Autism and Other Developmental Disabilities, Journal of Vocational Rehabilitation, and Research in Autism Spectrum Disorders. We searched electronic versions of the journals if hard copies were unavailable in university library stacks. Hand searches yielded one additional article. We also conducted ancestral searches of included articles and previously conducted systematic reviews of the VM and employment literature, which yielded one additional article. Finally, we contacted three researchers with expertise in employment and/or VM to inquire about studies not yet identified (e.g., in press). We provided the experts with the inclusion criteria and list of included articles, and one expert provided two articles^34,35 she authored that met the inclusion criteria.

Coding procedures

Training

The first author trained the third author in quality indicator (QI) coding procedures using a coding matrix^36,37 and check for understanding available for free at www.ci3t.org. We determined the criterion for training as three consecutive articles coded with >85% IRA. We established a consensus process for disagreements. The third author scored a mean of 94% (range = 92.86%–96.43%) on the three training articles before coding articles included in the meta-analysis.

Demographic coding

We created a coding database to synthesize the included literature at the participant- and study-level. Participant-level coding included: (1) pseudonym, (2) gender, (3) age, (4) race or ethnicity, and (5) reported disability category. We extracted participants without an autism diagnosis and did not include them in the analysis. Study-level coding included: (1) setting where the study occurred (e.g., classroom, retail store, warehouse), (2) type of employment experience, (3) experimental design, (4) type of VM, and (5) dependent variable (i.e., job skill as described by original study authors). We operationalized the type of employment experience using the criteria from the Workforce Innovation and Opportunity Act of 2014.²³ Specifically, we categorized employment experiences as (1) CIE, (2) work-based learning in a community employment setting, and (3) contrived or school-based experiences. Internships or other unpaid experiences in the community were included in the work-based learning category. Contrived experiences were those created by researchers for purposes of a study, and school-based experiences occurred in school settings (e.g., cafeteria, copy room) rather than community employment settings. We categorized type of VM as traditional VM, point-of-view, video prompting, video self-modeling, or a combination of any two types of VM. Finally, we categorized job skills as general task acquisition relevant to the employment setting or employment-related social skill.

QI coding

The database also included coding categories pertinent to the methodological quality of this literature. Specifically, we used the eight QIs defined by CECs Standards for EBPs²⁹: (1) context and setting, (2) participants, (3) intervention agent, (4) description of practice, (5) implementation fidelity, (6) internal validity, (7) outcome measures/dependent variables, and (8) data analysis. There were 28 components related to single-case research design and 24 components related to group design within the 8 QIs. We used a weighted coding scheme developed by Lane et al.³⁸ that required the sum of all components to be divided by the sum of scored components. As required by this weighted coding scheme, we considered studies to be methodologically sound if 80% or more of the eight QIs were met. Two authors independently coded all included articles according to CEC guidelines.²⁹ The mean IRA of QI coding across all studies was 93.38% (range = 80.95%–100%).

We then examined graphs of methodologically sound studies for a potential functional relation and coded the VM intervention as having positive, mixed/neutral, or negative effects. We classified this body of literature as an EBP, potentially EBP, mixed evidence, insufficient evidence, or negative effects according to the CECs criteria.²⁹ Specifically, five methodologically sound single-subject studies with positive effects with a minimum of 20 participants must be found for VM to be considered an EBP. IRA for visual analysis was 97.14%. Finally, we analyzed studies that met 80% or more of QI standards using the weighted coding scheme to determine the magnitude of VM effects. Only two studies used group comparison designs and neither met the minimum 80% quality standard. Therefore, no group design studies were included in the effect size calculations. We determined 11 single-case design studies of the original 20 studies that were methodologically sound and included in the meta-analysis.

Meta-analysis

To conduct our meta-analysis, we extracted data from published single-case design study graphs using PlotDigitizer, Version 3.12.³⁹ We imported graphs to PlotDigitizer and calibrated the x and y axes. We identified each plotted data point for the first baseline and adjacent intervention conditions as “baseline” or “intervention” using the PlotDigitizer software. Multiple baseline or multiple probe designs resulted in extraction of potential effects across participants or behaviors according to the study. Alternating treatment designs resulted in extraction of potential effects according to different types of VM. We exported and aggregated all data sets (i.e., graph panels) into a database for analysis. We numbered each data set and labeled according to the author and publication year. We obtained a total of 66 unique data sets from the 11 methodologically sound studies.

Moderator variables

Using our demographic coding procedures, we identified and extracted relevant moderating variables from included studies for meta-analysis. At the participant-level, we identified gender, age, and disability category as moderator variables to analyze further. At the study-level, we identified type of employment experience, type of VM used, and job skill type as moderator variables. Types of employment experiences were: (1) CIE, (2) work-based learning in a community employment setting, or (3) contrived or school-based experience. We also used the same types of VM as described in the demographic coding database (i.e., traditional VM, point-of-view, video prompting, video self-modeling, or combination). Finally, we analyzed the differences between VM used to teach general task-acquisition skills versus employment-related social skills. General task-acquisition skills were those specific to completing a job that did not appear to require social communication skills or interactions (e.g., sorting mail, cleaning). Employment-related social skills were those that included interacting or communicating with co-workers, customers, or employers (e.g., greeting customers in-person or on the telephone, talking to co-workers during break time).

Effect size

We aggregated individual phase contrasts to obtain an omnibus effect size. We calculated Tau-U using software developed by Davis and Davis.⁴⁰ Tau-U combines nonoverlap phases with trend from within the intervention phase and compares each data point in the A phase to data points in the adjacent B phase.⁴¹ Tau-U analysis yields a value between 0 and 1.0, with 0–0.20 indicating small magnitude of change, 0.20–0.60 indicating moderate magnitude of change, and 0.61–0.80 indicating large magnitude of change, and 0.81 or higher indicating very large magnitude of change.⁴²

Comparing effects

We conducted an analysis of moderator variables following standard practice in analyzing categorical variables. We analyzed moderator variables with two groups using the Wilcoxon two-sample test.⁴³ We used the Wilcoxon test for gender and job skill variables. We analyzed moderator variables with three or more groups using the Kruskal–Wallis one-way analysis of variance⁴⁴ to test disability category and employment setting variables. If results showed significant differences between groups, we used a Dunn post hoc test⁴⁵ to evaluate the significance between groups. The Dunn post hoc test is beneficial when data do not meet normal distribution assumptions or do not have equal sample sizes, which is common in special education research.

Results

Participant characteristics

A total of 20 articles with 61 participants were included in the systematic review before meta-analysis. Fifty-four of the participants were male and 7 were female. The mean age of participants was 18 years (range = 13–28 years). Of studies that reported race/ethnicity, 17 participants were White, 2 participants were Asian, 2 participants were Black, 1 participant was Hispanic, and 1 participant was Native American. Race/ethnicity of 38 participants was not reported. Regarding disability category, 34 participants (56%) were identified as autistic and 13 participants (23%) were identified as autistic with co-occurring intellectual disability. Fourteen participants (18%) were identified as autistic with a secondary diagnosis (i.e., anxiety, learning disability, specific language impairment).

Employment contexts and job tasks

Table 1 includes the descriptive findings from our coding. The majority of studies (n = 12) occurred in contrived or school-based experiences. Of these 12 studies, 4 were contrived employment experiences created solely for the purposes of the research study.^46–49 Nine studies occurred in school settings such as self-contained special education classrooms,⁵⁰ teacher's staff room,^34,51 school hallways,⁵² or the school kitchen.⁵³ Seven studies occurred in work-based learning or internship experiences in the community.^52,54–59 One study entailed work-based learning in both community and school settings.⁵² Two studies occurred in CIE contexts.^60,61 Two young adults worked at restaurants for pay supported by an 18–21 transition program.⁶⁰ One young adult worked for pay at a retail store with no school or adult agency support.⁶¹

Table 1.

Descriptive Results for Included Studies

Study	Employment setting	Employment experience type	VM type	Dependent variable	Job skill type
Allen et al.46	Factory and warehouse	(3) Contrived experience by researchers	Traditional VM and audio cuing	Movements and actions of air-inflated mascot	Task
Allen et al.47	Large retail warehouse	(3) Contrived experience by researchers	Traditional VM	Movements and actions of air-inflated mascot	Task
Allen et al.48	Large retail warehouse	(3) Contrived experience by researchers	Traditional VM and point-of-view VM	Movements and actions of air-inflated mascot	Task
*Alexander et al.62	Conference room and classroom	(3) School-based experience	Point-of-view VM	Sorting mail	Task
*Bennett et al.63	Teacher workroom	(3) School-based experience	Video prompting	Clerical tasks	Task
*Bereznak et al.50	Teacher workroom and classroom living room area	(3) School-based experience	Point-of-view VM and video prompting	Number of steps completed correctly to make photocopies	Task
*Bross et al.61	Retail discount store	(1) Competitive employment	Traditional VM	Verbalization of customer service phrases	Social
Burke et al.49	Manufacturing and shipping warehouse	(3) Contrived experience by researchers	Traditional VM and video prompting	Percentage of steps completed correctly on a shipping task	Task
*Cihak and Schrader51	Teacher workroom and high school vocational	(3) School-based experience	Traditional VM and self-VM	Percentage of steps completed correctly to prepare first aid kits, photocopy, or send a fax	Task
*Cullen et al.54	Pet store, campus recreation center, dental clinic	(2) Internships (unpaid)	Traditional VM, point-of-view VM, and video prompting	Task completion accuracy	Task
English et al.55	Flower garden	(2) Nonprofit social enterprise (unpaid)	Traditional VM, video feedback, and video prompting	Percentage of steps completed correctly for gardening tasks	Task
*Goh and Bambara56	Thrift and department store	(2) Supported employment (unpaid)	Video self-modeling and video feedback	Percentage of correctly performed job task steps	Task
*Kellems and Morningstar57	Community employment settings (e.g., bowling alley, community center)	(2) Work-based learning	Traditional VM	Percentage of steps completed correctly on job tasks	Task
Mackey and Nelson58	Food warehouse and computer nonprofit organization	(2) Work-based learning	Video feedback	Decision-making, response to others, hygiene, and transition skills	Social
Rausa et al.59	Nonprofit organization	(2) Internships (unpaid)	Traditional VM	Percentage of responses completed correctly for answering a phone	Social
Van Laarhoven, et al.34	Vocational classroom, conference room, and high school staff room	(3) School-based experience	Traditional VM and point-of-view VM	Percentage of correct responses for cleaning and organization tasks	Task
*Van Laarhoven, et al.35	Hallways and doorways of high school	(3) School-based experience	Traditional VM or video feedback	Percentage of social skill used correctly	Social
*Van Laarhoven, et al.60	Restaurant	(1) Competitive employment supported with secondary program	Traditional VM and self-VM	Number of correct responses for restaurant tasks	Task
Van Laarhoven, et al.52	Restaurant, classroom, hallways, school kitchen	(2) Work-based learning and (3) school-based experience	Traditional VM and point-of-view VM	Number of correct responses and prompt level for steps of skill sequence	Task
*Yakubova and Taber-Doughty53	Classroom and kitchen areas of high school	(3) School-based experience	Point-of-view VM	Steps for solving problems during vocational tasks	Task

(1) competitive integrated employment; (2) work-based learning or internship in a community employment setting; (3) contrived or school-based experiences.

^* Indicates studies included in the meta-analysis.

VM, video modeling.

In addition, the majority of studies (n = 16) focused on general task-acquisition skills relevant to the employment settings, such as shipping items,¹¹ gardening,⁵⁵ or food preparation.⁶⁰ Three studies focused on increasing movements and actions of participants wearing air-inflated mascots (for advertising purposes).^46–48 Four studies focused on teaching clerical skills such as sorting mail, making photocopies, or sending a fax.^50,51,62,63 Four of the 20 included studies focused on some component of employment-related social skills. For example, secondary students delivered passes to teachers within a school building and demonstrated social skills such as gaining teacher attention, using a greeting phrase, and ending the conversation.³⁵ Similarly, a young adult used VM to acquire telephone skills (e.g., answering the phone with a professional greeting, taking customer orders).⁵⁹ Autistic twins participating in work-based learning experiences learned employment-related social skills such as active engagement and responding to others.⁵⁸ Finally, one young adult employed in a retail store used VM to learn customer service phrases and interactions.⁶¹

QI coding results

Table 2 is a scatterplot of QI coding results. All studies met QI 1.1. All studies met QI 2.1, and 16 studies (80%) met QI 2.2. Sixteen studies (80%) met QI 3.1. Four studies (20%) met QI 3.2. All studies met QI 4.1. Nineteen studies (95%) met QI 4.2. Twelve studies (60%) met QI 5.1. Twelve studies (60%) also met QI 5.2. Eleven studies (55%) met QI 5.3. Twelve studies (60%) met QI 6.1. All studies met QI 6.2 and 6.3. The two group design studies met 6.4. All 18 single-case design studies (100%) met QI 6.5. Seventeen of the 18 (94%) single-case design studies met QI 6.6. All single-case design studies (100%) met QI 6.7. The two group design studies (100%) met QI 6.8 and QI 6.9. All 20 studies met QI 7.1, QI 7.2, and QI 7.3. Nineteen articles (95%) met QI 7.4 and QI 7.5. Both group design studies (100%) met QI 7.6 but neither group design study met QI 8.1. All single-case design studies (100%) met QI 8.2. Finally, both group design studies (100%) met QI 8.3.

Table 2.

Summary of Quality Indicator Coding

NA, not applicable; shaded, quality indicator met; unshaded, quality indicator not met; QI 1.1, context and setting; QI 2.1, participant demographics; QI 2.2, participant disability/risk status; QI 3.1, intervention agent role; QI 3.2, intervention agent training; QI 4.1, intervention procedures; QI 4.2, intervention materials; QI 5.1, implementation fidelity; QI 5.2, implementation fidelity measures; QI 5.3, implementation fidelity dosage; QI 6.1, independent variable systematically manipulated; QI 6.2, baseline description; QI 6.3, no or limited access to the intervention; QI 6.4, describes assignment to groups; QI 6.5, three demonstrations of experimental effects at three different times; QI 6.6, baseline phases include at least three data points; QI 6.7, design controls for threats to internal validity; QI 6.8, attrition is low across groups; QI 6.9, differential attrition is low; QI 7.1, outcomes are socially important; QI 7.2, study clearly defines dependent variables; QI 7.3, study reports effects of the intervention; QI 7.4, frequency and timing outcome measures are appropriate; QI 7.5, study provides adequate internal reliability; QI 7.6, study provides evidence of validity; QI 8.1, data analysis techniques to compare two or more groups; QI 8.2, study provides single-subject graph; QI 8.3, study reports one or more appropriate effect size statistic.

Meta-analysis results

We classified 11 studies as methodologically sound according to the 80% weighted criterion.³⁸ All these studies used single-case design methodology. The weighted totals for these studies ranged from 7.17 to 8.0 of eight QIs (Table 3). One study met all eight QIs.⁵⁰ Effects from five studies were not determined because they had fewer than three autistic participants.^{35,52,54,56,61} One study with seven participants had neutral or mixed effects.⁶² The remaining five studies had positive effects.^{50,51,53,57,63} Exactly 20 autistic participants were included across the five methodologically sound single-case studies. Therefore, VM for improving job skills of autistic individuals met classification as an EBP according to CEC's Standards for EBPs.²⁹

Table 3.

Evaluation of Evidence Base of Included Studies

Study	QI absolute coding	QI weighted coding	Method. sound (≥80%)	n	Experimental design	Effect size calculated	Effect (with ≥3 participants)
Allen et al.46	5.0	6.33	No	3	ABCAC withdrawal design	No	—
Allen et al.47	5.0	5.83	No	3	AB design	No	—
Allen et al.48	5.0	5.83	No	4	MBD across participants	No	—
*Alexander et al.62	7.0	7.50	Yes	7	MP across participants	Yes	Neutral or mixed
*Bennett et al.63	7.0	7.50	Yes	5	ATD	Yes	Positive
*Bereznak et al.50	8.0	8.00	Yes	3	MP across behaviors	Yes	Positive
*Bross et al.61	7.0	7.50	Yes	1	MBD across behaviors	Yes	—
Burke et al.49	4.0	5.97	No	4	MBD across participants	No	—
*Cihak and Schrader51	7.0	7.50	Yes	4	ATD	Yes	Positive
*Cullen et al.54	7.0	7.50	Yes	1	MP across tasks and participants	Yes	—
English et al.55	5.0	6.33	No	3	MP across skills	No	—
*Goh and Bambara56	7.0	7.50	Yes	1	MP across participants	Yes	—
*Kellems and Morningstar57	7.0	7.50	Yes	4	MP across behaviors	Yes	Positive
Mackey and Nelson58	4.0	6.13	No	2	MP across participants	No	—
Rausa et al.59	4.0	5.33	No	1	MBD across behaviors	No	—
Van Laarhoven, et al.34	5.0	6.00	No	4	Nonequivalent dependent variable design	No	—
*Van Laarhoven, et al.35	7.0	7.50	Yes	2	Multiple treatments with reversal design	Yes	—
*Van Laarhoven, et al.60	6.0	7.17	Yes	1	MP across tasks	Yes	—
Van Laarhoven, et al.52	4.0	5.83	No	4	Modified pre/post-test control group design	No	—
*Yakubova and Taber-Doughty53	7.0	7.50	Yes	4	MP across participants	Yes	Positive

^* Studies included in the meta-analysis.

ATD, alternating treatments design; MBD, multiple baseline design; Method, methodologically; MP, multiple probe; n, autistic participants; QI, quality indicator; VM, video modeling.

We included data from the 11 methodologically sound studies in the magnitude of change meta-analysis. Across these 11 studies, we analyzed 66 separate effect sizes for 33 participants. Omnibus Tau-U across all VM employment studies was 0.91 (CI₉₅ = 0.84–0.97). The Wilcoxon two-sample test revealed statistically higher effects for males (z = 78, p < 0.0001), but there were only four females in the sample. The Kruskal–Wallis test revealed no statistically significant differences between types of employment experiences (chi-square = 1.62, p = 0.44). The Wilcoxon two-sample test revealed that there were no statistically significant differences between types of job skills (z = 177, p = 0.06). The Kruskal–Wallis test for disability category indicated a significant effect (chi-square = 19.64, p < 0.0001). Therefore, we conducted a Dunn post hoc analysis, which revealed a significant difference between autistic participants and autistic participants with co-occurring intellectual disability. Specifically, VM had a higher effect for autistic participants (0.95) compared with autistic participants with co-occurring intellectual disability (0.73). No other pairwise comparisons were significant.

Discussion

Our first research question aimed to better understand the different types of employment settings where VM studies were conducted (e.g., CIE, work-based learning, contrived, school-based experiences). Findings indicate that VM studies primarily occurred in contrived environments arranged by researchers or school-based experiences where participants were unpaid and/or working alongside other individuals with disabilities. In particular, 12 studies (60%) occurred in contrived or school-based employment settings. Six studies (30%) occurred in work-based learning or other nonpaid employment experiences (e.g., internships, nonprofit enterprises). Only two studies (10%) occurred in CIE settings where the autistic employee earned regular wages. This finding is concerning because noncompetitive settings may contribute to the positive effects associated with VM to teach job skills. For example, personnel in school settings likely deliver highly individualized supports to students⁶⁴ that are not always present in CIE settings. Employers in CIE settings also have reported that they could benefit from increased knowledge regarding how best to support autistic employees.⁶⁵

Regarding the second research question, we found VM interventions focused primarily on improving general job skills applicable to the employment setting. Specifically, 16 of the 20 included studies (80%) focused on task acquisition, and 4 studies (20%) focused on some form of employment-related social skills. The included studies focused primarily on narrow job skills, such as cleaning, clerical tasks, gardening, and performing in air-inflated mascots. Results do not indicate whether VM improves service-oriented job skills (e.g., co-worker interactions, problem-solving skills). This is concerning because competitive experiences present unique social communication challenges that can alter the effectiveness of VM.⁶⁶ For example, teachers and job coaches in school-based experiences can support the social communication skills of students and repair communication breakdowns (i.e., social problem solving). Furthermore, professionals in school settings are likely aware of the autistic student's disability-related needs and more likely to adjust supports for social interactions. Conversely, co-workers and employers in CIE settings may be unaware of the employee's disability status unless the employee has chosen to disclose that information.

Regarding the third research question, we found 11 methodologically sound studies using an 80% weighted criterion.³⁸ Of these 11 articles, 5 studies had positive effects with a minimum of three total participants according to the CEC's Standards for EBPs.²⁹ Accordingly, VM can be considered an EBP for improving job skills of autistic individuals. This finding is consistent with previous reviews of the literature that found sufficient evidence to qualify VM as an EBP¹⁸ but provides clarification about for whom and under what conditions VM is effective.

Finally, our fourth research question aimed to evaluate the magnitude of change associated with VM. The Tau-U revealed an omnibus score of 0.91. This finding suggests that VM is a potentially highly effective intervention for improving job skills of autistic employees. Analysis of moderator variables revealed two statistically significant differences between groups. First, we observed a difference in intervention effects based on participant gender. However, the majority of participants included in the analysis were male and only four females were included. This suggests the difference was a product of sampling bias. Second, we found a statistically significant difference between autistic participants with and without co-occurring intellectual disability. This finding suggests that VM may be more effective for autistic individuals without co-occurring intellectual disability. Modifications may be needed to enhance the effects of VM for autistic individuals with co-occurring intellectual disability. This finding is consistent with previous research indicating that cognitive ability affects CIE.^67,68 There were no significant differences for types of employment experiences. However, this finding should be interpreted with caution given very few studies occurred in CIE settings. There were also no significant differences according to job tasks. Given so few studies focused on employment-related social skills, it remains unclear whether VM will produce similar effects for employment-related social skills as general task acquisition.

Limitations

We encourage readers to consider several limitations to this meta-analysis. First, CEC's Standards for EBPs²⁹ are common but not universally accepted method for evaluating the methodological quality of a body of literature. Other organizations (e.g., Cochrane Collaboration; What Works Clearinghouse) use different standards for evaluating methodological quality of experimental research to inform conclusions about a particular intervention. In addition, we included only studies that met QI standards using the weighted coding scheme in the meta-analysis. Therefore, other researchers who use different quality standards and a more conservative coding scheme may obtain different results. Also, we believe that our search was sufficiently comprehensive to locate the relevant literature but acknowledge that different search terms might produce some additional sources. Although we believe that our findings are an accurate representation of the literature on VM for employment skills, different search procedures might have produced returns not included in this review and, accordingly, altered the findings.

Additional limitations relate to our reliance on Tau-U to calculate magnitude of change as well as the procedure for selecting moderator variables used for analysis. First, meta-analyses of VM that use different effect size metrices likely will produce different results, many of which may be difficult to interpret. Accordingly, future studies might investigate the differences and similarities of emerging metrices for estimating magnitude of changes in responding, how to interpret the results, and what theoretical implications and considerations are critical for advancing this methodological line of research. Second, we selected moderator variables for this study according to convenience. That is, we identified several potential variables to include but ultimately were only able to analyze moderating variables that were commonly reported by authors of the included studies. Although gender, disability category, employment setting, and type of job skills are important variables relevant to employment outcomes, we were not able to analyze other potentially informative variables because such details were not commonly reported by authors (e.g., level of on-the-job support provided, prior work experience).

Implications for practice and research

Results indicate that VM is an EBP for improving job skills of autistic individuals. This finding has implications for professionals who hire and support autistic employees. Importantly, VM can and should be considered a reasonable workplace accommodation as described in the Americans with Disabilities Act (1990).⁶⁹ For example, autistic employees can watch a video before or during a work shift given video models are typically no more than 2–3 minutes in duration. Video-based interventions may also promote autonomy at work if employees can access and play videos independently, but additional research in this area is warranted. Additional research is also needed on more diverse and complex job skills given the studies included in the review focused on such narrow job skills. As increasing numbers of autistic individuals join the workforce, it will become increasingly important to identify effective ways to support them.

Video-based interventions could potentially supplement or enhance existing training methods to teach autistic employees skills necessary for a specific job. For example, a job coach or employer could create a video library to support a larger number of autistic employees in a relatively quick and cost-efficient manner. Other individuals in the employment setting, such as a co-worker, could watch videos with autistic employees as a natural workplace support. Edited videos may also be used repeatedly to ensure standard performanc⁷⁰ and can be used with other employees regardless of disability status. Additional research is needed to examine training methods to teach co-workers, employers, and job coaches to enhance the implementation of video-based interventions in socially valid ways.

Conclusions

Findings from this meta-analysis indicate that the majority of VM research to improve job skills of autistic individuals occur in contrived or school-based settings and not CIE settings. Given paid employment is one predictor of positive postschool employment outcomes,⁷¹ this finding was concerning. VM met CEC's Standards for EBPs²⁹ criteria for an EBP, and the Tau-U indicated very large effects. Accordingly, employers may consider using brief videos such as those used in VM interventions as a job training method for their autistic employees. However, additional research is needed to better understand how these videos may be used to promote more inclusive workplace environments as well as the feasibility of employer-created and applied VM.

Authorship Confirmation Statement

L.A.B. conceptualized the study, conducted all search procedures, served as the primary QI coder, trained the secondary coder, and wrote and edited the article. J.C.T. conceptualized the study in collaboration with the first author and collaborated in the writing and editing of the article. J.M.H. conducted all search procedures for IRA purposes and served as the secondary QI coder. J.L.D. conducted all statistical analyses required for the study. R.A.M. revised the final article for important intellectual content. All co-authors reviewed and approved the article before submission. This article has been submitted solely to Autism in Adulthood and is not published, in press, or submitted elsewhere.

Author Disclosure Statement

No competing financial interests exist.

Funding Information

This study was funded by the National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR, grant no. 90DP0058).