Identifying Patterns of State Vocational Rehabilitation Performance in Serving Transition-Age Youth on the Autism Spectrum

Abstract

Background:

Transition-age youth on the autism spectrum (TAY-ASD) face many challenges when attempting to find and keep employment. Vocational rehabilitation (VR) is a key public source of support for employment for people with disabilities in the United States, and TAY-ASD increasingly use VR services. However, rates of VR service utilization and employment outcomes are known to vary dramatically across states for these youth, for reasons that are not fully understood.

Methods:

This study aimed to examine a set of indicators for measuring the state VR performance in serving TAY-ASD, compared with youth with other disabilities, and to identify classes of homogenous patterns of state performance across these indicators. We used latent profile analysis (LPA) to model patterns of state performance in serving TAY-ASD.

Results:

We identified five classes of states with unique patterns of performance across four key indicators (service receipt, early reach, timely services, and employment rates) and then matched states to each class based on their probability of inclusion. One class featured above average performance across all four indicators, and approximately one-fourth of states had a high probability of membership in this class.

Conclusions:

Identification of states with patterns of more efficient and effective VR service delivery for TAY-ASD will help target efforts to learn how states are delivering, organizing, and coordinating VR services for these youth. The use of methods like LPA may also be beneficial for examining performance within other autism-related service systems in the United States and internationally.

Lay Summary

Background:

Achieving employment is an important milestone on the road to adulthood. Having a job is related to financial independence, health, and well-being but can also provide a sense of belonging and opportunities for inclusion. Transition-age youth on the autism spectrum (TAY-ASD) may find that getting and keeping a job is more difficult than it is for their peers with other types of disabilities. Vocational rehabilitation (VR) is a public source of support for employment for people with disabilities in the United States, and TAY-ASD are increasingly using VR services. However, whether youth receive VR services, and whether they gain employment following VR services, is highly dependent on which state they live in. We do not yet fully understand why state VR services vary so dramatically.

About This Study:

New federal legislation, the Workforce Innovation and Opportunity Act, aims to reach students with disabilities with vocational services during secondary school (junior high and high school). Few studies exist to help us understand how well VR services are reaching students and what are the effects of these services. We tested new ways to measure VR services for TAY-ASD and also tested whether we could group states according to their results on these measurements. We wondered whether any groups of states performed better than other groups.

We used the VR data for the 50 states and Washington DC to test the following four things: how often TAY-ASD received VR services if they were eligible for them; how often these youth applied for VR services during secondary school; how often their employment plan was finished on time; and how often they got a job after VR services. We compared youth on the autism spectrum with youth with other disabilities and found that they did about the same on these measures.

What This Study Tells Us:

We identified five groups of states, which each had a unique pattern of how they performed on these measures. We named the groups—also called classes—according to their strengths. Class 1 had above average employment rates but below average performance on other measures. Class 2 had timely services, meaning that these states finished youth's employment plans on time, so that they could access services. Class 3 had both timely services and early reach to students, meaning that the students began services during secondary school. Class 4 had early reach to secondary students but low performance on other measures. Finally, Class 5 had above average performance on all measures. States in this class excelled at reaching students, developing employment plans quickly, enrolling students in services, and achieving employment by the time VR services ended. We then determined which states were most likely to belong in each class.

This study gives us another way to think about how states are doing in delivering VR services to TAY-ASD. By studying states that have better overall performance, versus others, we can identify what states might be doing differently. Learning about how some states are adapting VR services for TAY-ASD and the innovations they are using is important information for other states who wish to improve their VR services. The methods we used may also be helpful for examining the performance of other autism-related service systems in the United States and internationally.

Introduction

Employment is a key dimension of quality of life in adulthood and can be a gateway for enhancing economic self-efficacy and financial independence,¹ identity formation,² and mental health.³ For adults with intellectual and developmental disabilities (IDD), including autism spectrum disorder (ASD), working can alleviate social isolation⁴ and provide a sense of belonging and opportunities for social inclusion.⁵ Currently, well over 50,000 youth on the autism spectrum turn 18 years of age every year,⁶ and roughly 60%–80% of them expect to work following high school.^7,8 Yet, published reviews detail a host of challenges that transition-age youth on the autism spectrum (TAY-ASD) face when attempting to secure and maintain employment.^9–13 Despite these obstacles, multiple studies show that it is possible to improve employment outcomes of TAY-ASD when effective services are provided.^14–16

Vocational rehabilitation (VR) is one of the key sources of public assistance for people with disabilities in the United States who seek employment. The U.S. Department of Education's Rehabilitation Services Administration (RSA) provides grants to states to operate VR agencies in accordance with federally approved state plans for the implementation of VR services. For a comprehensive overview of VR services and related definitions, and information on how people access VR services, we refer readers to the National Autism Indicators Report: Vocational Rehabilitation.¹⁷

Transition-age youth (TAY), often defined as youth aged 16–24 years who are preparing to enter adulthood, are the largest age group of VR service users on the autism spectrum, although their outcomes are worse than those who enter VR as adults.^18,19 Less than half (47%) of those on the spectrum who entered VR at the age of 18 years or younger were employed at the time of case closure in federal fiscal year (FFY) 2011¹⁹ compared with 55% who entered at ages 19–25.^19,20 Overall, analyses suggest that approximately half of dollars spent on VR services for TAY-ASD do not result in employed youth¹⁸—a problem that is now a target of federal and state policy. TAY-ASD are particularly vulnerable to poor employment outcomes due to the unique social, communication, and behavioral characteristics of ASD; their need for highly individualized supports; and the lack of evidence-based vocational interventions available to serve this population.²¹ Given the growth of TAY-ASD in the VR population, understanding the performance of states in serving these youth is critical.

VR employment outcomes for TAY-ASD vary widely across states,^18,22 as they do for TAY with any other type of disability (TAY-Other) that affects a youth's ability to work.²³ This variation is partially explained by individual demographic and impairment characteristics, receipt of public benefits, and types of VR services received.^20,24–26 State-level factors inside and outside of VR agencies may also affect outcomes,^21,27,28 and state VR services may vary across states along several dimensions.

First, not all youth who qualify for VR services actually receive them. A youth and their legal guardian must apply for VR services, be found eligible for VR services in their state, and then complete steps necessary to receive these services. Rates of service receipt range among states from less than half of TAY-ASD to nearly 90%.^a TAY-ASD, in particular, may discontinue pursuit of services if they (and/or their family members) are unable to navigate or understand the process of accessing VR services or if they perceive a mismatch of services to their needs.²⁹

Second, not all youth who use VR services apply for them during secondary school. While approximately half of TAY-ASD enter VR during secondary school, the rate for this indicator ranges from 4% to 75% across states.^a Federal law requires transition planning for students preparing to exit special education, in part to provide early connections to vocational services when needed.³⁰ Yet, approximately one-third of districts begin transition planning at the age of 16 years or older,³¹ and some states have traditionally waited until youth exit secondary school before providing VR services.³² Multiple advisory bodies have recommended lowering the age at which transition planning formally begins to 14 years or younger^31,33,34 and beginning no later than 3 years before secondary school exit.³⁵

Third, before youth can receive VR services, the VR counselor and the individual/family must sign an Individualized Plan for Employment (IPE). Federal law states that the IPE should include the person's employment goals, the type of services the individual needs, the duration of those services, who will provide the services, and plans for evaluating progress (34 CFR § 361.45).¹⁷ Federal guidelines require completion of the IPE within 90 days after determining eligibility for VR services. Yet, timely development of IPE for TAY-ASD ranges from 19% to 83% across states.^a Experts hypothesize that excessively lengthy processes to access services are a deterrent to VR service receipt for TAY-ASD,²⁹ similar to the general population of youth with disabilities.³⁶

The federal Workforce Innovation and Opportunity Act (WIOA) of 2014 reauthorized VR grants to states and placed more focus on providing earlier and more effective vocational interventions for youth with disabilities. WIOA requires states to allocate 15% of VR funds for new pre-employment transition services delivered primarily during high school, such as job exploration counseling and work-based learning experiences that are either school- or community-based, for students who are eligible or potentially eligible for VR services (34 CFR §361.65). Despite these actions and specific calls for earlier attention to the school-to-work transition for TAY-ASD,³¹ we are still in need of methods for measuring and tracking how WIOA will impact the early vocational experiences of TAY-ASD across states.

Models currently used to identify states that are “high performing” often rely on singular indicators, such as the employment rate for persons who received state IDD services.³⁷ Honeycutt et al. created a more comprehensive state-level summary statistic to reflect VR outcomes of those with IDD. However, this work pre-dated recent revisions to the RSA data, which allow us to determine whether youth were secondary students at the time of VR application.³⁸ Furthermore, we do not know how well prior methods work for tracking the performance of TAY-ASD who may face unique employment challenges.

The purposes of this study were to examine a set of indicators for measuring the state VR performance in serving TAY-ASD and to identify classes of homogenous patterns of state performance across these indicators. Our analyses focus on the RSA data from years that preceded official approval of state plans for WIOA implementation (submitted in 2016).³⁹ Therefore, the results of this study equate to pre-WIOA baseline data on the VR experiences and outcomes of TAY-ASD across states in comparison to those of TAY-Other. Our research questions were: (1) How stable were the state means for the selected indicators across years, and did these indicators perform similarly for TAY-ASD versus TAY-Other? (2) Do states cluster into clear groups with specific patterns of performance in serving TAY-ASD? Ability to group states using analyses of VR indicators may help target further investigation of states that have applied effective approaches to VR service delivery, organization, and coordination for TAY-ASD.

Methods

Data set

We analyzed administrative data from the RSA Case Service Report (RSA-911). The RSA-911 contains information about all individuals with a VR case that was closed within a given year, including information on demographics, disability characteristics, VR services received, and whether employment was attained. VR cases are generally closed after an individual maintains employment for 90 days (or longer if specified in the IPE) or if the person is not actively participating in services or for another reason stated by the individual. The analysis of this secondary data was deemed exempt by the Drexel University Institutional Review Board.

We combined 3 years of data for youth whose cases were closed in fiscal years 2014–2016 and who were 14–24 years when they applied for VR services. This age range encompasses the years during which transition services are delivered across states, as some states begin the transition from special education at age 14 and some states allow continuation of special education services until ages 24 or 25. There were 51,436 TAY-ASD noted as the primary or secondary cause of their work impairment and 532,707 TAY who had any disability other than autism as the primary or secondary cause of their work impairment (TAY-Other). Examples of other impairments include intellectual disability, depression, epilepsy, traumatic brain injury, or other sensory, physical, or mental impairments.⁴⁰ Analyses were limited to those who lived in the 50 states and Washington DC. We included youth who had a job at the time of VR application. These youth may have been requesting VR services needed to find a new job that better matched their interests or skills or a job in an integrated setting that included people with and without disabilities (if currently working in a nonintegrated setting).

Measures

Our selection of indicators was anchored in a conceptual framework borrowed from the field of evaluating the performance of human service programs.^41,42 We measured three aspects of program performance—outputs, quality, and outcomes.⁴¹ Outputs refer to the quantity of services provided in terms of either units of service provided (volume) or the number of individuals who received services. Quality indicators measure the proportion of outputs that meet predefined standards in domains, such as reliability and responsiveness (timeliness). Outcomes measure the effectiveness of programming in terms of results and accomplishments of program participants. We created four indicators of the state VR performance using variables in the 2014–2016 RSA-911 data sets:

Among youth eligible for VR services:

• 1.Service receipt (output): The percentage of youth who actually received any VR services between eligibility determination and case closure.

Among youth who received any VR services before case closure:

• 2.Early reach (quality–reliability): The percentage of youth who were secondary students when they entered VR.
• 3.Timeliness (quality–responsiveness): The percentage of youth who had an IPE developed within 90 days of eligibility determination.
• 4.Employment (outcome): The percentage of youth who were employed at the time of case closure. RSA defines employment as maintaining a job in competitive or supported employment for 90 days in an integrated setting.

Statistical analyses

We conducted bivariate analyses to examine the characteristics of TAY-ASD and TAY-Other who exited VR in FFY 2014, 2015, or 2016. Next, we assessed the stability of selected indicators for TAY-ASD and TAY-Other by first calculating the mean frequency for each indicator by state for each year. We identified frequencies one standard deviation (SD) above or below the mean of state means, summarized frequencies across states in quintiles, and examined the stability of each state's performance across the 3 years. We chose to use a 3-year estimate (FFY 2014–2016) for the remaining statistical analyses. The technique of pooling multiple years of data has been used in previously published analyses of the RSA-911 data,^23,33 as VR indicator values are known to fluctuate mildly from year to year between and within states.¹⁸ Pooling years is also preferable as high or low values on individual indicators within any one year may not be indicative of better or worse overall performance.³²

We used latent profile analysis (LPA) to identify groups of states with homogenous performance across our four indicators. LPA is a latent variable technique that identifies the patterns of relationships between measured variables to create “classes” that represent underlying unobserved (latent) constructs.^43,44 LPA is used when the observed variables are continuous, and the latent variable is modeled to be categorical, as is the case in this study.⁴⁵ Latent variable techniques have been successfully used to group states according to the patterns of policies.⁴⁶ All analyses were performed using Stata 15.

Identification of groupings of states using LPA may be superior to using a summary statistic for several reasons. LPA is a model-based approach that can be tested for goodness of fit. Using LPA allows for modeling the relationships among the individual indicators and accounts for the possibility that the indicators of state VR performance are markers for other unmeasured, or latent, characteristics of performance indicators. Once classes of performance are identified, each state is then assigned a probability of membership within each class. The probability not only accounts for the uncertainty of the modeling approach but also allows for ranking of states within a class based on the degree of probability of membership.

We constructed models with successive numbers of classes, from two to six, and compared the Bayesian information criterion (BIC), the Akaike information criterion (AIC), and entropy statistics to quantitatively determine the optimal number of classes.⁴⁷ AIC and BIC are measures of true likelihood to model-fitted likelihood; smaller AIC and BIC are indicative of a more likely model.⁴⁸ Entropy is a measure of separation between classes, with higher entropy values indicating a higher degree of certainty of class assignment.⁴⁷ We then examined the classes created by the model to assess whether they were conceptually meaningful. We removed models from consideration if multiple classes had marginal probabilities of membership lower than 10%, as these models contained several classes that likely represented fewer than five states. The final LPA model was chosen based on fit statistics and qualitative assessment to determine the most parsimonious and conceptually plausible model.⁴³

After selecting a final model, we assessed the probability of states' class assignments using the posterior predicted probabilities of the appropriate model. We assigned each state to the class with the highest predicted probability of membership for that state. Next, we ranked states within each class using the predicted probability of class membership, which had the advantage of acknowledging class uncertainty.

Results

Characteristics of TAY at VR application

Table 1 summarizes the characteristics of all TAY-ASD and TAY-Other who were found eligible for VR services and whose VR cases were closed in FFY 2014–2016. These youth were eligible for VR services, but not all received VR services before case closure. TAY represented 35.5% of all case closures between FFY 2014 and 2016, and TAY-ASD comprised 10.9%% of these TAY cases. Half (49.8%) of TAY-ASD applied for VR services between the ages of 14 and 18 years compared with slightly over half (53.9%) of TAY-Other. A higher proportion of TAY-ASD had an Individualized Education Program (IEP) during secondary school (93.8% vs. 90.0%)—a legal document delineating their special education goals and services. Significantly less TAY-ASD (4.5% vs. 7.4%) did not have an IEP or a 504 plan—a legal document specifying accommodations a student needs to access general education—compared with TAY-Other. However, TAY-ASD were only slightly more likely than TAY-Other to apply for VR services during secondary school (52.0% vs. 49.5%).

Table 1.

Characteristics of Transition-Age Youth Whose Vocational Rehabilitation Case Closed in Federal Fiscal Years 2014–2016

	TAY-ASD n = 51,436		TAY-Other disabilities n = 532,707
	n	%	n	%
Male	42,784	83.2	308,506	58.0a
Age at application, years
14	149	0.3	1573	0.3
15	645	1.3	6525	1.2
16	3311	6.4	32,905	6.2
17	9405	18.3	103,499	19.4
18	12,075	23.5	142,986	26.8
19	7264	14.1	67,829	12.7
20	5796	11.3	43,962	8.3
21	4927	9.6	38,165	7.2
22	3261	6.3	33,291	6.3
23	2522	4.9	31,398	5.9
24	2081	4.1	30,574	5.7
Race
White	42,852	83.3	373,772	70.2a
Black	5558	10.8	129,279	24.3a
Other/multiple	3018	5.9	29,377	5.5
Hispanic or Latino	3879	7.5	90,653	17.0a
Highest education completed (at VR application)
Less than high school	17,671	34.4	213,924	40.4a
Special education certificate or current special education student	15,172	29.5	128,249	24.2a
High school	13,510	26.3	138,676	26.2
Some postsecondary	4836	9.4	45,059	8.5a
Postsecondary degree or certificate	222	0.4	3171	0.6a
Secondary student at the time of application	26,725	52.0	260,633	49.5a
Had an IEP or section 504 planb
IEP	25,063	93.8	234,486	90.0a
504 plan	452	1.7	6927	2.7a
No IEP or 504 plan	1210	4.5	19,220	7.4a
Source of referral to VR
Educational institution	20,413	55.3	198,827	56.4a
Self-referral	6130	16.6	62,732	17.8a
Community rehabilitation program	1430	3.9	9640	2.7a
Other	8964	24.3	81,651	23.1a
Employed at the time of application	4072	7.9	57,684	11.0a

^a Significant difference at p < 0.001.

^b IEP and 504 plans measured for those who were secondary students at the time of application to VR.

IEP, Individualized Education Program; TAY-ASD, transition-age youth on the autism spectrum; VR, vocational rehabilitation.

Question 1. How stable were the state means for the selected indicators across years, and did these indicators perform similarly for TAY-ASD versus TAY-Other?

State-level means for the four indicators were relatively stable across individual years of the RSA data (available upon request). Across both disability groups, for three individual years of state data points, across all four indicators, there was only one instance of a state with a mean value that fell one SD above the mean in one year and one SD below the mean in another year. Similarly, for each metric, only five to six states had means that fluctuated between the two upper and two lower quintiles of performance across years. The performance of TAY-ASD and TAY-Other was similar on these indicators across individual years (available upon request).

Three-year estimates for TAY-ASD are shown in Table 2 for each metric. Examining national means, TAY-ASD received VR services more frequently (69.9%) than TAY-Other (62.1%) and were more frequently employed at VR exit (58.3% vs. 53.2%). Approximately half (52.6%) of TAY-ASD and half (51.4%) of TAY-Other entered VR services during secondary school. However, the percentage of TAY-ASD whose IPE was developed in a timely manner was lower (55.9%) compared with that of TAY-Other (63.4%).

Table 2.

Three-Year Estimates (Federal Fiscal Years 2014–2016) for Key Indicators

	TAY-ASD				TAY-Other disabilities
State	Service receipt	Early reach	Timely services	Employed at VR exit	Service receipt	Early reach	Timely services	Employed at VR exit
Alabama	75.9%	53.5%	72.6%	73.6%	74.0%	58.4%	83.0%	66.9%
Alaska	81.4%	51.7%	55.3%	61.1%	62.4%	40.2%	55.3%	50.9%
Arizona	60.7%	55.2%	44.7%	49.3%	51.8%	53.9%	39.0%	47.5%
Arkansas	57.4%	31.2%	78.3%	53.8%	57.0%	21.5%	85.3%	67.4%
California	73.1%	52.8%	83.6%	55.1%	76.6%	64.3%	82.3%	55.3%
Colorado	64.7%	55.1%	37.7%	68.7%	53.1%	55.7%	47.8%	58.2%
Connecticut	73.0%	54.3%	49.8%	48.8%	64.2%	44.8%	54.6%	44.8%
Delaware	76.2%	56.6%	64.3%	66.5%	64.2%	60.4%	64.7%	66.3%
District of Columbia	53.0%	68.6%	71.9%	28.6%	42.2%	51.9%	85.1%	37.1%
Florida	70.1%	65.3%	52.2%	40.2%	68.3%	62.8%	54.2%	31.6%
Georgia	57.0%	36.4%	43.7%	68.4%	50.2%	34.0%	52.6%	65.6%
Hawaii	67.1%	25.0%	58.8%	44.0%	54.0%	23.4%	56.5%	38.5%
Idaho	67.7%	49.6%	57.6%	52.2%	64.9%	48.1%	68.6%	48.5%
Illinois	80.8%	64.4%	55.3%	45.7%	78.2%	68.8%	58.2%	44.8%
Indiana	70.3%	53.6%	56.1%	56.7%	57.7%	53.7%	62.5%	48.1%
Iowa	80.6%	22.3%	22.5%	60.3%	72.0%	21.8%	17.1%	60.4%
Kansas	68.4%	33.0%	59.8%	59.2%	54.3%	29.2%	63.5%	42.6%
Kentucky	59.1%	35.1%	26.8%	52.6%	56.0%	34.1%	48.2%	47.2%
Louisiana	58.8%	41.9%	56.0%	57.4%	50.3%	42.0%	67.6%	53.1%
Maine	54.9%	61.9%	28.1%	47.4%	46.8%	62.4%	34.8%	37.7%
Maryland	63.0%	66.8%	44.9%	67.7%	55.6%	55.4%	52.3%	61.5%
Massachusetts	73.5%	63.9%	32.8%	59.8%	67.6%	62.1%	41.9%	55.0%
Michigan	80.9%	68.5%	70.0%	49.8%	73.3%	69.3%	75.1%	45.7%
Minnesota	73.0%	68.8%	47.7%	66.3%	62.3%	69.0%	53.8%	61.4%
Mississippi	61.1%	71.1%	64.9%	36.9%	72.3%	58.9%	84.4%	44.8%
Missouri	62.6%	47.7%	49.6%	68.1%	54.8%	48.4%	58.1%	61.3%
Montana	68.6%	11.8%	71.5%	51.4%	61.9%	18.8%	63.0%	37.0%
Nebraska	65.4%	39.3%	54.3%	75.7%	54.8%	37.1%	60.9%	64.0%
Nevada	64.4%	26.0%	52.5%	62.7%	57.3%	39.9%	63.5%	49.1%
New Hampshire	65.1%	45.5%	44.7%	54.2%	57.7%	43.0%	52.1%	43.5%
New Jersey	50.6%	10.3%	65.0%	64.5%	46.0%	5.9%	67.5%	62.9%
New Mexico	61.3%	49.0%	54.1%	38.8%	60.5%	54.7%	58.3%	31.5%
New York	73.2%	72.2%	62.3%	59.2%	57.6%	70.8%	63.6%	58.4%
North Carolina	64.5%	25.8%	68.4%	66.9%	54.5%	23.3%	74.8%	54.4%
North Dakota	56.1%	59.8%	32.2%	69.0%	46.9%	58.8%	40.9%	69.9%
Ohio	72.2%	65.1%	51.2%	51.4%	56.6%	56.8%	52.7%	43.6%
Oklahoma	74.1%	76.6%	72.9%	50.8%	61.2%	67.4%	72.7%	51.9%
Oregon	63.8%	35.9%	41.9%	69.0%	59.3%	47.4%	56.1%	66.1%
Pennsylvania	88.5%	41.7%	79.5%	54.5%	71.8%	38.6%	82.7%	56.1%
Rhode Island	69.6%	51.4%	50.0%	61.1%	51.7%	42.0%	70.6%	59.6%
South Carolina	82.3%	63.4%	73.7%	47.6%	74.0%	53.8%	80.3%	50.8%
South Dakota	75.7%	59.1%	68.9%	73.5%	58.5%	63.0%	67.8%	60.7%
Tennessee	50.7%	42.1%	36.5%	60.0%	44.7%	35.8%	40.8%	56.5%
Texas	66.7%	42.9%	59.4%	63.3%	60.5%	38.6%	69.1%	56.0%
Utah	71.1%	69.0%	54.4%	67.1%	57.1%	41.1%	64.5%	58.5%
Vermont	85.0%	39.1%	75.8%	60.9%	80.6%	35.5%	83.2%	50.0%
Virginia	79.0%	64.0%	32.5%	61.8%	71.0%	61.1%	41.2%	55.2%
Washington	59.2%	32.3%	36.0%	75.4%	49.2%	26.8%	39.2%	68.1%
West Virginia	66.3%	71.9%	28.4%	51.3%	56.7%	67.4%	51.2%	51.5%
Wisconsin	69.5%	42.4%	42.1%	68.6%	52.2%	32.8%	34.2%	61.2%
Wyoming	72.4%	47.6%	75.0%	67.9%	60.4%	40.3%	68.7%	58.5%
National average	69.6%	52.6%	55.9%	58.3%	62.1%	51.4%	63.4%	53.2%
Mean of state means	68.2%	49.8%	54.3%	58.1%	59.7%	47.0%	60.1%	53.3%
Standard deviation of state means	8.9	16.1	15.4	10.4	9.1	15.5	15.2	9.7
Point gap between highest–lowest state mean	30.7	60.4	61.1	47.1	38.4	64.9	68.2	38.4

Service receipt, % who received VR services after being found eligible; Early reach, % who were secondary students at the time of application to VR; Timely services, % with employment plan developed within 90 days of eligibility determination; Employment, % with competitive or supported employment in an integrated setting at the time of VR case closure.

Gaps in percentage points between the highest and lowest state means for TAY-ASD varied from 30.7 percentage points for service receipt to 61.1 percentage points for timely IPE development. Interstate gaps in point estimates were lower for TAY-ASD cases compared with TAY-Other for all indicators with exception of the employment rate. The interstate gap in employment rates was nearly 10 points higher for TAY-ASD (47.1) versus TAY-Other (38.4), indicating greater variation across states for employment outcomes of TAY-ASD.

Question 2. Do states cluster into clear groups with specific patterns of performance in serving TAY-ASD?

Table 3 displays the fit statistics from each LPA model. BIC increased with each increase in class number, but AIC remained relatively stable. The largest AIC decrease occurred between the five- and six-class models. Entropy increased as the number of classes in the model increased, although the four-class model had slightly lower entropy than the three-class model. These fit statistics point to a six-class solution as the best fit to the data. However, by evaluating the probability of class membership, there were two solutions that had classes with fewer than 10% membership, a five- and a six-class solution. The six-class solution had three classes with less than 10% predicted membership; thus, the five-class solution was chosen for the final model.

Table 3.

Fit Statistics for Latent Profile Analysis Models Used to Determine Ideal Number of Classes

	Distributions across classes
No. of classes in model	1	2	3	4	5	6	Log-likelihood	df	BIC	AIC	Entropy
2	35.1	64.9	n/a	n/a	n/a	n/a	143.5037	13	−235.8938	−261.0075	0.70881685
3	43.3	11.1	45.6	n/a	n/a	n/a	149.6713	18	−228.5698	−263.3426	0.78882856
4	34.2	10.7	42.6	12.4	n/a	n/a	154.4031	23	−218.3743	−262.8063	0.78187267
5	26.8	11.5	4.2	32.2	25.3	n/a	160.1545	28	−210.2179	−264.3090	0.82400787
6	8.1	18.8	3.9	7.5	42.0	19.7	166.5752	33	−203.4001	−267.1503	0.91539812

AIC, Akaike information criterion; BIC, Bayesian information criterion; df, degrees of freedom.

Figure 1 depicts the profile of performance for each class across the four key indicators compared with the national means for these indicators. We assigned names to each class based on their individual areas of strength, as indicated by average frequencies that fell above the national mean for one or more indicators, within an overall pattern of performance.

FIG. 1.

Graphic indicator profiles for a five-class solution, and final primary class membership for states with posterior predicted probabilities of membership.

Approximately 26.8% of states had a primary classification in Class 1 (above average employment rates). Class 1 states exhibited above average employment rates for TAY-ASD at the time of VR exit (64.1%, SE = 2.8), but below average performance on service receipt (63.4%, SE = 9.2), timeliness (43.3%, se = 4.0), and early reach (39.3%, SE = 4.2). Fewer states (11.5%) were classified as Class 2 (timely services), which featured above average performance on timeliness of IPE development (66.5%, SE = 4.8) after TAY-ASD were determined eligible for services. While the employment rate for Class 2 states (56.9%, SE = 3.7) was only slightly below average and receipt of services was below average (62.3%, SE = 2.8). The rate of entering VR during secondary school was far below average (22.7%, SE = 4.8). Class 3 (timely services and early reach with below average service receipt and employment) was the smallest class with 4.2% of states assigned as members. States in Class 3 were above average for both timeliness (67.3%, SE = 7.8) and reach to secondary students (68.5%, SE = 8.0)—both indicators of quality. However, Class 3 states were below average in service receipt for TAY-ASD (57.5%, SE = 4.8) and far below average in employment at VR exit (33.3%, SE = 6.1). Class 4 (early reach) was the largest class with 32.2% of states assigned as members. This class had above-age reach to students (60.3%, SE = 3.2). Both service receipt (68.3%, SE = 2.4) and employment at VR exit (56.0%, SE = 2.8) were only slightly below average. However, timeliness (45.3%, SE = 3.6) was well below average. Class 5 (timely services and early reach with above average service receipt and employment) contained approximately one-quarter of states (25.3%). This class featured above average performance across all indicators: service receipt (77.8%, SE = 2.0), reach to students (56.7%, SE = 3.0), timeliness (69.6%, SE = 3.9), and employment at VR exit (59.2%, SE = 2.6).

Figure 1 also presents predictions and probability of state membership for each class. For the majority of states (76%), the probability of membership in the assigned class was in the range of 80%–90%. Across classes, only five states had probabilities of class membership that were in a borderline range (∼50%). Class 4 contained the most states but had a lower overall degree of certainty, as the probability of membership fell below the 80% level for 6 of the 17 states.

Discussion

Compared with TAY-Other, TAY-ASD had higher national averages for service receipt and employment rates, similar rates of application to VR during secondary school, but lower rates of timely IPE development. Studies using earlier data sets also found higher employment rates for TAY-ASD, compared with TAY-Other, following VR services.¹⁸ In this study, the interstate variation in employment rates at VR exit was more pronounced for TAY-ASD, similar to prior studies,²² suggesting that high levels of state-level variation in outcomes of TAY-ASD have persisted over the past decade.

We identified five classes of state VR performance for TAY-ASD during FFY 2014–2016, grouped here according to their performance profiles for the purpose of discussion. Slightly less than half of states (n = 24) were members of Classes 2, 3, and 4. States in these classes delivered timely services (via IPE development within the required 90 days) or had early reach to secondary students or both timely services and early reach. The profiles of these classes may reflect state VR agencies that focus on execution of processes. However, this strength did not necessarily translate into an increased rate of delivery of services to those who were found eligible or higher employment rates.

Over half of states (n = 27) belonged to Classes 1 and 5, both featured above average employment outcomes. Class 1 states had an above average employment rate despite having below average timely services, service receipt, and early reach. Class 5 represented one-quarter of states and featured above average performance across all indicators. Class 5 states excelled in output, quality, and outcome indicators, pointing to potentially superior planning and execution of services, outreach activities, and connections needed to help TAY-ASD become employed. Class 5 was also the sole class in which TAY-ASD experienced above average VR service receipt. Although others have found that states with high rates of VR service receipt had lower rates of employment at VR exit,²² states in Class 5 had both higher than average service receipt and above average employment rates.

Implications

Three main research implications arise from our study. First, LPA offers a viable method for determining groups of states with distinct patterns of performance using indicators in the RSA-911 data set. The states assigned to Class 5 could be characterized as “high-performing states.” We are hesitant to use this label as the results of latent class analyses are not necessarily expected to be predictive or stable across future data sets. However, pooling data years assisted in stabilizing minor fluctuations, enhancing our confidence in the identified patterns. The current class assignments provide important clues that may guide questions for future state-level comparisons. LPA methods may be beneficial for studying the performance of other autism-related service systems within the United States and internationally.

Second, our finding of distinct patterns of state VR performance highlights a need for information about how states are serving TAY-ASD. Determining whether and how “high-performing” states are modifying their VR practices or programs to meet the needs of TAY-ASD would provide critical context crafting for understanding the differences in outcomes. There have been increasing calls to develop standards for serving youth in VR, improve partnerships between the education system and VR, and increase training for VR counselors who serve youth.^32,34,49 Crafting of standards specific to TAY-ASD is contingent upon first identifying state-level innovations that could be contributing to higher levels of performance.

Third, this study suggested a more expansive slate of indicators for measuring the VR experiences of TAY-ASD. With the advent of WIOA and its focus on early vocational preparation, the need to identify youth-focused VR indicators has become particularly salient. Additionally, holistic pictures of youth employment are likely better achieved using multiple indicators versus reporting solely on employment rates. However, the overall lack of appropriate RSA indicators for measuring the youth experiences with VR is concerning. A recent consortium of states engaged in Partnership in Employment grants recommended to “Require RSA to develop standards and indicators for VR in serving transition-age youth, including youth with significant disabilities” (p. 272).⁵⁰ Others have noted a need for indicators measuring the cross-agency collaboration of VR with youth-serving systems.⁵¹ We also need measures detailing work-based learning experiences or exposure to internships or apprenticeships, which may be particularly critical for TAY-ASD who wish to sample job environments. Finally, responses for existing indicators (e.g., reasons for exiting VR) are typically not tailored to youth, making it difficult to assess their experiences. The challenge of identifying useful employment indicators for youth on the autism spectrum is likely not unique to U.S. service systems and may be best solved by applying lessons learned from international efforts in outcomes measurement.

In the related fields of social science and public health, child indicators have evolved from a focus on “minimums” (e.g., child survival, basic needs) to nuanced outcomes measured in composite indices of well-being with a focus on promoting child development.⁵² These composite indices are now often tailored to regions and subgroups and are tied to informing policymakers, as seen in the Annie E. Casey Foundation KIDS COUNT reports and Child Trend reports.⁵³ Youth-focused vocational indicators are in need of similar development to inform advances in federal and state employment policy and to meet the calls for indicators that best reflect the performance of TAY-ASD.

Limitations and strengths

We note here that the findings from the analysis of the RSA-911 data, like all administrative data, are not representative of the broader population of autistic individuals, and the findings cannot be generalized beyond TAY who received VR services during the study years. We further note that the RSA-911 data only include people who have applied for VR services and were found eligible. Therefore, we do not know whether states differ in the percentage of people who apply for VR services and are found ineligible. Also, while we chose to include descriptive statistics for all states, some “states” such as DC have many fewer cases, so the results should be interpreted with caution. Nevertheless, the RSA-911 data provide the largest window into the VR experiences of TAY-ASD and allow comparisons by state.

We did not measure the factors associated with the classes of states we identified. Our current understanding of institutional-, community-, and policy-level factors that may predict why states differ in vocational services and outcomes is limited.⁵⁴ A recent Delphi survey of national autism employment experts revealed five categories of modifiable state-level factors that might affect VR outcomes of TAY-ASD: capacity to provide VR services and job opportunities, efficient and effective VR processes, innovative state-level VR practices and policies, interagency efforts to improve employment outcomes, and VR staff training and competency.²⁹ Factors within these categories merit further investigation as examples of potential complex latent factors that may underlie the classes identified in this study.

Conclusions

This study offers a new schema for using VR indicators and latent profile methods to group states according to their performance along the dimensions of timeliness, outreach, service receipt, and employment outcomes. Effective transitions from school into work and postsecondary education are critical stepping stones to achieving stability and productivity in adult social roles; thus, the importance of developing and tracking comprehensive sets of youth employment indicators cannot be overstated as part of the broader effort to advance the quality of lives of youth and adults with ASD. The five classes of states identified in this study are not an end point, but a beginning for further study to identify how higher performing groups of states are achieving better outcomes for TAY-ASD.

Authorship Confirmation Statement

A.M.R., J.E.R., and P.T.S. conceptualized and designed this study. J.E.R. and J.P. worked on the data analysis protocol, and J.E.R. conducted the analysis. A.M.R. and J.E.R. interpreted the data, in consultation with J.P. K.N.-L. and A.L. provided context regarding disability employment policy and experiences within states. A.M.R. drafted and revised this article with input from all coauthors. All coauthors reviewed and approved the article before submission. This article has been submitted solely to this Journal and is not published, in press, or submitted elsewhere.

Author Disclosure Statement

No competing financial interests exist.