Developing a Vocational Index for Adults with Autism Spectrum Disorders

Abstract

Existing methods of indexing the vocational activities of adults with autism spectrum disorders (ASD) have made significant contributions to research. Nonetheless, they are limited by problems with sensitivity and reliability. We developed an index of vocational and educational outcomes that captures the full range of activities experienced by adults with ASD, and that can be reliably coded across studies using specific decision rules. To develop this index, we used employment, vocational, and educational data collected from nearly 350 adults with ASD at 6 times over 12 years, as part of a larger longitudinal study. The resulting index consists of 11 categories coded on a 9-point scale, ranging from competitive employment and/or postsecondary educational program to no vocational/educational activities.

Introduction

Given the surge of children and adolescents with autism spectrum disorders (ASD) expected to exit secondary school in the next decade, researchers and policy makers are increasingly extending their focus beyond childhood and considering the development of individuals with ASD as they enter and move through adulthood. The number of children diagnosed with ASD began rising rapidly nearly 20 years ago, in the early 1990’s (Gurney et al. 2003; IDEAdata.org 2010), and children from that generation are now exiting the school system. They enter a world of adult services that is plagued by long waiting lists and unprepared to meet their unique needs (Howlin et al. 2005). In part because of the inadequacy of adult services, employment tends to be low among adults with ASD, with studies finding anywhere from 4 to 13 % competitively employed (Ballaban-Gil et al. 1996; Eaves and Ho 2008; Howlin et al. 2004). Our preliminary study of a small sub-sample of young adults with ASD who had recently exited high school (n = 66) found similar low rates of employment, with 17 % competitively employed (Taylor and Seltzer 2011). The rate of employment among adults with ASD looks slightly more optimistic when including any type of employment outside of the home (e.g., including working with supports or sheltered workshop work), at around 37 %; however, even this is approximately one-half of what would be expected for same-aged adults without a disability (Newman et al. 2011).

Two approaches have been used in past research to quantify employment and vocational outcomes among adults with ASD. The first method was dichotomizing adults with ASD into those who are versus those who are not employed in the community (Mawhood and Howlin 1999; Howlin et al. 2005; Lawer et al. 2009; Garcia-Villamisar and Hughes 2007; Garcia-Villamisar et al. 2000, 2002; Cimera and Cowan 2009). This approach was most often used when examining the impact of vocational rehabilitative services or interventions aimed at employment. The other common approach was the creation of an ordinal index of “adult status” or adult outcome (Howlin et al. 2004; Eaves and Ho 2008; Farley et al. 2009; Billstedt et al. 2005; Esbensen et al. 2010), originally proposed by Lotter (Lotter 1978) and ranging from “very good outcome” to “very poor outcome,” or the like. Although this approach often also takes social relationships and general independence into account, its vocational aspect generally assigns top ratings for independent employment, mid-level ratings for supported or sheltered employment, and the lowest ratings for no vocational activities.

These existing approaches have made significant contributions to our knowledge about employment and vocational outcomes among adults with ASD, and they have raised important issues about the barriers to employment faced by this population (for a review, see Howlin 2005). However, as detailed below, both approaches are subject to limitations that constrain their usefulness in contemporary research. As increasing public attention is focused on the needs of adults with ASD, it is critical to develop and implement an index of vocational outcomes that reliably captures the range of their vocational activities, as well as change over time.

The present study describes the creation of a new, more comprehensive index of employment, vocational, and educational activities for adults with ASD, developed through detailed examination of these activities collected from nearly 350 adults with ASD over a 12-year period. An index that reflects the full range of vocational and educational activities experienced by these adults will improve our ability to examine trajectories of development during adulthood, as well as to measure the impact of interventions and services aimed at promoting independence.

Limitations of Current Approaches

Lack of Sensitivity

Both of the existing approaches collapse over a range of outcomes that blur meaningful differentiations for adults with ASD. For the dichotomous employment variable, in most studies, people who are working in the community with supports are categorized the same as those who are working in the community in competitive positions without the aid of formal supports (e.g., Lawer et al. 2009). Studies examining the economic costs of ASD or focusing on the independence of the adult will likely find the distinction between working with versus without supports meaningful. Similarly, the dichotomous approach categorizes adults who are in sheltered workshops or adult day centers the same way as those who have no vocational or occupational activities (i.e., as not working). For many adults with ASD, spending time in a sheltered workshop or other sheltered setting may be a more desirable and adaptive outcome than having no vocational activities whatsoever.

Although the ordinal index of adult outcome presents a more nuanced view of employment and vocational outcomes for adults with ASD, studies using such an approach also have collapsed across types of vocational activities. For example, in some studies, supported work in the community and sheltered workshop work are combined into one category (Howlin et al. 2004; Eaves and Ho 2008). Although both types of work require supports, separating those who have community employment (even with support) from those who are working in sheltered (i.e., segregated) settings is likely important in examining adult outcomes. There are many iterations of how distinct categories have been combined in past research, but the main point is that past approaches have combined experiences that are qualitatively different, which limits the ability to observe change over time.

Subjectivity and Inconsistency of Ratings

An additional limitation of past approaches is that the categories are sometimes operationally defined in a way that risks subjectivity in coding. For example, in one past study using the adult outcome index, “the need for regular support” in work activities is categorized as a “fair” outcome and “the need of a very high level of support” is categorized as a “poor” outcome (Farley et al. 2009). Another study considers an “obvious severe handicap and no independent social progress” as a “poor” outcome and an “obvious very severe handicap and unable to lead any kind of independent existence” as a “very poor” outcome (Billstedt et al. 2005). Although some studies have been able to achieve high interrater reliability within a study using such categorizations (e.g., Farley et al. 2009), it is likely that the categories may be interpreted differently in other studies. Further, the criteria for specific categories (e.g., good vs. very good) often differ from study to study (e.g., Howlin et al. 2004; Farley et al. 2009; Billstedt et al. 2005). Thus, subjectivity in ratings and differences in operational definitions across studies make it difficult to compare outcomes across populations of research participants as well as to validly determine the impacts of interventions.

Defining Success Normatively

A third limitation of the existing approaches is that optimal outcomes are defined by normative standards. For example, in most studies, a “very good” outcome on the adult outcome index involves a paid job without supports (Howlin et al. 2004; Eaves and Ho 2008; Farley et al. 2009). When using this criterion, most adults with ASD are found to have “poor” or “very poor” outcomes, with percentages of such outcomes generally over 50 % and as high as 79 % in some studies (Billstedt et al. 2005; Howlin et al. 2004; Eaves and Ho 2008; Farley et al. 2009). Importantly, when outcome is defined normatively, it is highly correlated with IQ. Adults with ASD and a comorbid intellectual disability almost always receive poorer outcome scores than those without an intellectual disability (Gillberg and Steffenburg 1987; Eaves and Ho 2008; Farley et al. 2009). By applying normative standards, these indices do not account for potential variation in success for those adults who have intellectual disability.

Not Reflecting Multiple Positions or Number of Hours Working

Finally, no previous study of adult outcomes has incorporated the measurement of multiple jobs or placements that are often held at the same time by adults with ASD. For example, currently, it is not uncommon for adults to simultaneously be spending time at a supported job in the community, a sheltered workshop, and volunteering, all on a part time basis within the work-week. Most previous studies (Howlin et al. 2004; Eaves and Ho 2008) assign research participants to one vocational activity level (e.g., a score of 0 for competitive employment; a score of 2 for supported/sheltered employment), but do not specify how to code a sample member who has two or more part-time positions at different levels. Thus, a comprehensive approach to measuring vocational outcomes for adults with ASD must include decision rules for assigning a single code for people with multiple part-time positions. Further, it is necessary to take into account the number of hours that adults with ASD are working at their employment positions so that working just a few hours a week would not get the same code as working closer to full time.

The Present Study

In the present study, we describe the development of a Vocational Index that addresses many of the limitations of previous approaches. To do this, we used a longitudinal dataset of approximately 400 families of adolescents and adults with ASD in two states (Seltzer et al. 2003, 2011; Taylor and Seltzer 2011). As part of this larger study, data were gathered about each individual’s employment, vocational activities, and educational activities at six points over a 12-year period. At any given time, most individuals had multiple part-time jobs and vocational placements, with varying amounts of time spent at each activity, and many experienced significant turnover in vocational activities. This resulted in an extraordinarily complex and dynamic employment dataset. The goal of our index is to capture the heterogeneity in vocational and educational activities, and provide coding rules that can result in a series of mutually exclusive categories from which outcomes can be reliably coded and life course trajectories described.

Methods

Sample

The present study used a subsample (n = 343) of adolescents and adults with ASD drawn from our larger longitudinal study of families of adolescents and adults with ASD (N = 406; Seltzer et al. 2003, 2011; Taylor and Seltzer 2011). The criteria for inclusion in the larger study were that the son or daughter with ASD was age 10 or older (age range = 10–52 at the beginning of the study in 1998), had received an ASD diagnosis (autistic disorder, Asperger disorder, or pervasive developmental disorder) from an independent educational or health professional, and had a researcher-administered Autism Diagnostic Interview-Revised (ADI-R; Lord et al. 1994) profile consistent with the diagnosis. Nearly all of the sample members (94.6 %) met the ADI-R lifetime criteria for a diagnosis of autistic disorder. Case-by-case review of the other sample members (5.4 %) determined that their ADI-R profile was consistent with their ASD diagnosis (i.e., meeting the cutoffs for reciprocal social interaction and repetitive behaviors for Asperger disorder, and for reciprocal social interaction and either impaired communication or repetitive behaviors for PDD-NOS). Half of the participants lived in Wisconsin (n = 202) and half in Massachusetts (n = 204). We used identical recruitment and data-collection methods at both sites. Families received information about the study through service agencies, schools, and clinics; those who were interested contacted a study coordinator and were subsequently enrolled. Six waves of employment data have thus far been collected: four waves collected every 18 months from 1998 to 2003, spanning a 4.5 year period, a fifth wave collected in 2008, and a sixth wave collected in 2010. At each time point, data were collected from the primary caregiver, who was usually the mother, via in-home interviews that typically lasted 2–3 h and via self-administered questionnaires.

To develop our index, we focused on the subsample (n = 343) of individuals with ASD who had at least one time point of post-high school employment, vocational, and educational data (the remaining individuals either were still in high school at the most recent time point of data collection or dropped out of the study prior to exiting high school). The individuals with ASD included in this subsample averaged 22.84 years of age (SD = 9.58) at the start of the study, with a range from 10 to 52 years. Just over 73 % were male and 62 % were living with their parents at the initial time point. One quarter (25 %) were reported to have a seizure disorder, and 74.3 % of individuals were verbal, as indicated by daily functional use of at least three-word phrases. Approximately 71 % had an intellectual disability. The median income for families at the start of the study was between $50,000 and $60,000. Just over 94 % of parents were Caucasian.

Procedures

At each time point of data collection, parent respondents (primarily mothers) were asked to indicate which vocational or educational activities their son or daughter with ASD participated in during the week on a regular basis (from a list including competitive employment, sheltered workshop, community integration program, school, volunteering, etc.). At 5 of the 6 time points, parents were asked to indicate the number of hours their son or daughter spent in each indicated activity each week (this question was omitted from Time 4 due to a clerical error).

We also examined all open-ended data at each time point and extracted all information that was relevant to the son or daughter’s vocational/educational activities, including data about employment from the Vineland Screener (Sparrow et al. 1993), which was administered at three waves (Times 4, 5, and 6). Finally, a series of questions about job history at the most recent time point provided useful data (a comprehensive list of questions used at each time point to determine the Vocational Index categories is available from the first author).

Next, the extracted qualitative information was used to develop mutually exclusive vocational/educational categories, which were applied to each time point for each person. The development of the categories was informed by examining vocational categories used in past research on adults with ASD (Howlin et al. 2004; Taylor and Seltzer 2011), and the categories were refined and finalized by examining the range of possible vocational/educational placements in this sample. After categories were developed, the first author applied the categories to each time point after high school exit for each person in the study. After this initial determination, the first and second author reviewed 40 % of the cases, to ensure there was agreement about how to code each case.

An independent rater then recoded a random selection of 10 % of cases. Agreement of categories coded by the independent rater with the initial category determination was excellent at 94 % (Cronbach’s alpha = .92). All discrepancies were discussed and resolved. Finally, the two authors examined patterns of categories over all 6 time points to determine the order of categories from most to least independent. Although the ordering for some of the categories was self-evident (independent employment is more independent than a sheltered workshop), others were not (is volunteering more similar to working in the community or to having no vocational/educational activities?), and inspection of the data across the study period was informative.

Results

The final categories and ordering are presented in Table 1. Since many individuals had multiple post-high school activities, we developed decision rules to yield eleven mutually exclusive categories, coded on a scale of 1–9 (several categories are given the same code as described below). Following the existing models of adult outcome indices (Howlin et al. 2004; Billstedt et al. 2005; Eaves and Ho 2008; Farley et al. 2009), these categories reflect the level of independence inherent in the educational or vocational settings. In addition, our categories reflect whether the adult had a total number of vocational/educational activities of greater than or less than 10 h a week. The percentages of adults with ASD in each vocational/educational category at each time point of our study are presented in Table 2. Note that the denominator changes from time point to time point reflecting the increasing number of cases who exited high school prior to each point of measurement as well as sample attrition during the study. Thus, Table 2 should not be used to examine the progression of vocational activities, as any observed changes in frequencies are a combination of change in activities within a person and changes in the sample composition over time.

Table 1.

Vocational index, from most to least independent

	Score	Category
Most independent	9	Employment in the community without supports greater than 10 h a week
	9	Postsecondary, degree-seeking educational program greater than 10 h a week
	8	Postsecondary, degree-seeking educational program or employment in the community without supports—total activities 10 h a week or less
	7	Employed in the community with supports greater than 10 h a week. No time spent in sheltered settings.
	6	Employed in the community with supports (no time spent in sheltered settings)—total activities 10 h a week or less
	5	Sheltered vocational setting and supported community employment—total activities greater than 10 h a week
	4	Sheltered vocational setting and volunteering in the community—total activities greater than 10 h a week
	4	Sheltered vocational setting (workshop or day activity center) with no community employment/volunteering—greater than 10 h a week.
	3	Sheltered vocational setting—total activities 10 h a week or less
	2	Volunteering with no other activities or postsecondary non-degree seeking education with no other activities
Least independent	1	No vocational/educational activities

Table 2.

Distribution of vocational index categories across the study for those who have at least one time point of vocational data (n = 343)

Code	Category	Time 1	Time 2	Time 3	Time 4	Time 5	Time 6	Average
	Number in high school	177	148	120	87	10	0
	Number who exited high school	164	174	194	200	246	239
9	Employed in the community without supports (%)	6.7	4.6	5.7	9.0	6.9	11.3	7.6
9	Postsecondary degree-seeking program (%)	2.4	4.6	6.2	7.5	9.3	4.2	5.9
8	Degree-seeking education/employment without supports—10 h/week or fewer (%)	0.0	2.9	1.5	1.5	1.6	3.8	2.0
7	Employed in the community with supports (%)	12.8	14.4	11.3	8.0	9.3	10.5	10.8
6	Employed in the community with supports—10 h/week or fewer (%)	4.9	4.0	4.1	5.5	1.6	2.9	3.7
5	Sheltered vocational setting and community employment (%)	12.8	12.1	13.9	12.5	13.4	8.8	12.2
4	Sheltered vocational setting and volunteering (%)	6.1	2.9	6.2	1.5	8.5	9.2	6.0
4	Sheltered vocational setting only (%)	32.3	36.8	33.0	35.5	27.6	31.0	32.4
3	Sheltered vocational setting—10 h/week or fewer (%)	3.7	5.2	2.6	2.0	3.7	2.1	3.1
2	Volunteering or postsecondary non-degree seeking education only (%)	5.5	4.0	5.2	3.5	4.5	4.6	4.5
1	No vocational/educational activities (%)	12.8	8.6	10.3	13.5	13.4	11.7	11.8
	Number of attrition/missing (%)	2	21	29	56	87	104

Percentages reflect the number in each category divided by those who had exited high school at each time point (e.g. Percent in sheltered setting only at Time 1 is 53/164 or 32.3 %). Average percentages were calculated by dividing the sum in each category across all time points by the sum of those who exited high school across all time points

Category Descriptions

Employment in the Community Without Supports (Score of 9)

The highest rating on our Vocational Index was given to those adults who were working in the community in competitive jobs. This category reflects adults who are employed in the community, without receiving formal or informal on-the-job supports or adult day services, and who are not enrolled in postsecondary degree-seeking educational programs. Only between 4.6 to 11.3 % of adults in our sample were classified into this category during the multiple time points of our study. Types of work most often performed by these adults included factory work (e.g., operating a shrink-wrap machine or unloading trucks), retail (e.g., minimum wage job at a “big box” type store), kitchen work, or custodial work. To be coded into this category, adults with ASD were required to be working in the community more than 10 h a week.

Postsecondary Degree-Seeking Educational Program (Score of 9)

The highest rating was also given to those adults who were enrolled in a postsecondary degree-seeking program (scored equally high as working in the community without on-the-job supports; see above). The postsecondary degree-seeking category consists only of adults who are taking classes toward some type of postsecondary degree. The percentage of adults with ASD who were classified into this category at each time point ranged from 2.4 to 9.3 % (see Table 2). These individuals were enrolled in a wide range of programs such as a print technology and digital media major at a technical college, history or accounting majors at four-year universities, and culinary school. Although there was significant overlap in employment without supports and postsecondary educational programs (see below), we thought it most important to capture postsecondary education in our scale of mutually exclusive categories, and thus adults with ASD who were in a degree-seeking program and also working in the community were coded into the education category. Note that individuals were not coded into this category if they spent a total of 10 h or less per week in the educational program and employment.

We gave this category the same rating as working in the community without supports for three reasons. First, both are developmentally normative outcomes. Second, as reported in our earlier study (Taylor and Seltzer 2011), many adults who were in degree-seeking programs were also employed in the community. In fact, nearly two-thirds (64 %) of adults with ASD in postsecondary degree-seeking programs were employed in the community without supports while enrolled in the educational program. This percentage is comparable to the rate of employment for youth without disabilities attending postsecondary programs (Aud et al. 2011). Third, although just under one-half of adults with ASD were employed without supports at the time point after they exited the postsecondary program, nearly 90 % were employed without supports by the second time point after exiting the program (i.e., 36 months after exiting, on average). Thus, it appears that attending a postsecondary degree-seeking program is associated with subsequent competitive employment, although it may take additional time for youth with ASD to obtain a job compared with the norm.

Postsecondary Degree-Seeking Education or Employment in the Community Without Supports—10 h a Week or Less (Score of 8)

In order to separate out those adults with ASD who had minimal employment or educational activities, we reduced the score in our Vocational Index by one point for adults who were either in a postsecondary degree-seeking program or employed in the community without supports for 10 h a week or less. A cut-off of 10 h per week was chosen by examining nationally-representative data on employment among young adults with ASD. Employed adults with ASD worked an average of 24 h a week (Newman et al. 2011), and we felt that “minimal” employment was best reflected by identifying those adults who were working 50 % or less of the average. Because hours were collected in 10-h increments in our data (1–10 h, 11–20 h, 21–30 h, 31+ hours), we thus defined the minimal category as those who were working 10 h or less. This approach allowed us to identify adults with ASD who were averaging fewer than 2 h a day of employment/educational activities, while still taking into account the level of independence needed to participate in these activities. Across the time points of our study, anywhere from 0 to 3.8 % of adults were classified into this category.

Employed in the Community with Supports (Score of 7)

Our next category included adults who worked in the community with supports. We gave this category the highest independence rating after competitive employment and degree-seeking programs because on-the-job supports are intended to allow the acquisition of skills needed to eventually work independently at that job. Excluded from this category were adults who had part-time supported positions in the community but also spent part of the week in a segregated sheltered setting. To be coded into this category, the total amount of supported employment was required to be over 10 h a week.

Across the six time points of our study, this category constituted 8.0–12.8 % of post-high school activities (see Table 2). Although the majority of adults in this category were receiving formal supports from an agency, adults were also included if they were receiving on-the-job supports from families (such as families accompanying them to the workplace, the adult working in a family business where it is clear that support is being provided, or families providing instrumental support so that the adult can pursue his or her own business). Examples of supported jobs included working at a video store, prepping a restaurant for lunch, working at a warehouse distribution center, shredding confidential information, doing dishes at a nursing home, and working in a grocery store. Adults who had two jobs in the community—one with and one without on-the-job supports, were coded into this category, reflecting the maximum amount of supports necessarily for them to carry out their vocational activities.

Employment in the Community with Supports—10 h a Week or Fewer (Score of 6)

As in the preceding categories, adults who were working in the community with supports, and whose total hours of employment were 10 h a week or fewer, were coded into this category and thus deducted one point based on the small number of hours in which they were engaged in employment.

Sheltered Vocational Setting and Supported Community Employment (Score of 5)

Adults with ASD who were spending time at a segregated sheltered vocational setting (sheltered workshop or adult day center) and also working for pay with supports in the community, with total activities over 10 h a week, were given a mid-level score on our index. From 8.8 to 13.9 % of adults were classified into this category over the study.

At each time point, approximately 20–25 % of the adults who were spending time at a sheltered vocational setting were also working a supported job in the community. To determine whether to give those with community employment a higher score than those who were in a sheltered vocational setting only, we conducted a detailed examination of whether working in the community in addition to attending a sheltered setting was related to community employment at a subsequent point of the study. Of those who were in a sheltered vocational setting with some community employment, approximately 20 % subsequently moved into a supported employment position without any sheltered work. Thus, we felt that supported employment in addition to sheltered work was indeed related to subsequent employment for a minority of adults, and decided to give adults in this category a score that was one point higher than those adults who were spending time in a sheltered vocational setting without any community employment.

Sheltered Vocational Setting with Volunteering (Score of 4)

We next coded those adults with ASD who were spending time in a sheltered vocational setting but also volunteering in the community (1.5–9.2 % of the sample). Similar to the preceding category, we examined the data in detail to determine whether volunteering in addition to a sheltered setting was related to subsequent employment, and thus should be coded in the same category as participating in a sheltered vocational setting with community employment. Of those who were in a sheltered vocational setting with some community volunteering, less than 10 % moved into a supported employment position without any sheltered work at a later time point of the study. Therefore, we decided to give adults who were spending time in a sheltered setting and also volunteering in the community the same code as those adults who were only spending time in a sheltered vocational setting, as it rarely was related to a subsequent higher level of vocational independence.

Sheltered Vocational Setting Only (Score of 4)

We next coded adults who were spending time in sheltered vocational settings without community employment or volunteering. This was by far the most prevalent category, with 27.6 to 36.8 % of adults only spending time in sheltered settings at each time point of the study. As in the earlier categories, adults were coded into this category if their total number of vocational/educational activities was greater than 10 h a week. Further, adults were only coded into this category if it was clear that they were leaving their residence to attend a sheltered vocational setting; one-on-one aides who took the adult on community outings or chores performed at their residence were not included.

In most past studies, individuals working in a sheltered workshop were given a higher code than those who attended an adult day center (e.g., Howlin et al. 2004; Eaves and Ho 2008). In our data, however, these two types of programs were nearly interchangeable, because agencies often provide both in the same setting. Furthermore, whether an individual participated in an adult day program or a sheltered workshop appears to depend more on the availability of work contracts and differences in regional resources, and less on the abilities of the adult him or herself. Thus, we considered both sheltered workshops and adult day centers as sheltered vocational settings, giving them the same code.

Sheltered Vocational Setting—10 h a Week or Fewer (Score of 3)

Adults with ASD who were spending time in a sheltered vocational setting, and whose total hours of vocational/educational activities were 10 h a week or fewer, were coded into this category and thus deducted one point. Across the study, 2.0–5.2 % of adults fell into this category.

Volunteering Only or Postsecondary Non-Degree Seeking Education Only (Score of 2)

For a small number of adults in the sample, the only vocational/educational activities were volunteering in the community or taking postsecondary courses that were not related to obtaining a degree. Across the time points of the study, 3.5–5.5 % of adults were classified into this category. Before assigning a score to this category, we once again conducted a detailed examination of our data to determine whether either of these activities tended to be related to subsequent employment in the community. Neither volunteering only nor taking non-degree seeking classes led to community employment (with or without supports) in more than a few cases. Furthermore, the majority of the time (upwards of 80 % across nearly all time points), adults were participating in these activities 10 h a week or less. Thus, we coded them as a 2 on our 9-point Vocational Index.

No Vocational/Educational Activities (Score of 1)

Our lowest score was given to those adults with ASD who had no vocational or educational activities. From 8.6 to 13.5 % of adults were classified into this category over the course of the study. It is important to note that our index reflects vocational/educational activities only. Thus, adults with ASD who had recreational activities, but no vocational or educational activities, were coded into this category (e.g. adults whose only activities involved recreational outings in the community with a one-on-one aide or with parents).

Consistency of the Vocational Index Categories Over Time

In order to determine whether our Vocational Index categories appeared to be equally applicable across the study period (from 1998 to 2010), we examined the distribution of categories for a subsample of 93 adults who had exited high school before the beginning of the study, and who continued to participate at the most recent time point. These findings are presented in Table 3. This sub-sample has the advantage of reflecting changes over 12 years within an adult cohort for whom we had complete data. As shown in Table 3, the vocational categories remained fairly stable over time. Thus, it appears that the categories apply to vocational and educational opportunities equally well in 2010 as in 1998 even though the individuals occupying each category changed over the study period.

Table 3.

Distribution of vocational index categories across the study time points for those adults with ASD who had exited high school before the start of the study and were still participating at the most recent time point (n = 93)

Code	Category	Time 1	Time 2	Time 3	Time 4	Time 5	Time 6	Average
9	Employed in the community without supports (%)	9.8	5.5	6.6	8.8	7.5	6.5	7.4
9	Postsecondary degree-seeking program (%)	2.2	3.3	1.1	0.0	0.0	0.0	1.1
8	Degree-seeking education/employment without supports—10 h/week or fewer (%)	0.0	1.1	1.1	1.1	0.0	1.1	0.7
7	Employed in the community with supports (%)	10.9	11.0	9.9	12.1	7.5	9.7	10.2
6	Employed in the community with supports—10 h/week or fewer (%)	3.3	2.2	2.2	4.4	3.2	2.2	2.9
5	Sheltered vocational setting and community employment (%)	15.2	16.5	17.6	16.5	17.2	10.8	15.6
4	Sheltered vocational setting and volunteering (%)	7.6	3.3	8.8	1.1	6.5	5.4	5.4
4	Sheltered vocational setting only (%)	29.3	36.3	33.0	39.6	30.1	39.8	34.7
3	Sheltered vocational setting—10 h/week or fewer (%)	4.3	6.6	3.3	2.2	5.4	2.2	4.0
2	Volunteering or postsecondary non-degree seeking education only (%)	4.3	3.3	4.4	3.3	6.5	7.5	4.9
1	No vocational/educational activities (%)	13.0	11.0	12.1	11.0	16.1	15.1	13.1
	Number of attrition/missing	1	2	2	2	0	0

Percentages reflect the number in each category/(93—Number of attrition/missing). Average percentages were calculated by dividing the sum in each category across all time points by the sum of those who exited high school across all time points

Discussion

As more and more individuals with ASD exit high school and enter the adult service system, it is increasingly important to capture the diverse range of possible vocational outcomes in a way that can be reliably implemented from study to study and that can meaningfully reflect change. This study adds to the literature by providing such an index with specific categories and decision rules, developed by examining employment, vocational, and education information from nearly 350 adults with ASD collected over 12 years. Although existing methods for categorizing vocational outcomes of adults with ASD have made important contributions to our knowledge about the vocational prospects of these adults, a more sensitive and reliable index will aid comparison of outcomes from sample to sample, as well as allow for the examination of change over time.

Comparison to Other Vocational Indices

There are a number of ways that our Vocational Index differs from the existing methodologies used to classify vocational and employment outcomes. First, our index takes into account the full range of vocational activities experienced by adults with ASD, allowing for a more detailed differentiation of vocational activities than is often supported by the existing employment/vocational measures. Second, we factored the post-secondary educational activities of adults with ASD into our Vocational Index. This is an important addition, as college attendance is becoming an increasingly viable option for these adults due in part to legislation such as the Higher Education Opportunity Act of 2008, as well as government-funded postsecondary programs such as the Model Comprehensive Transition and Postsecondary Programs for Students with Intellectual Disabilities. Third, we developed detailed decision rules to allow for consistent classification of adults with ASD who have more than one type of vocational activity simultaneously. Finally, our index takes into account whether adults with ASD are spending minimal hours in vocational activities.

It is worth noting that, when comparing the resulting vocational categories in our index to that of Howlin et al. (2004), there were two main category determinations that differed in their order. Of the variations on the adult outcome index, we chose Howlin’s as a comparison because it offers the most explicit coding rules. In the Howlin et al. (2004) study, supported and sheltered employment were given the same rating, and having no occupation was rated the same as “in a special centre” (which is likely equivalent to adult day programs in the United States). As already discussed, in our US sample, adults with ASD often attended sheltered workshops and adult day programs interchangeably, and thus these two were not differentiated. The second difference occurred when determining the placement of volunteering on our index. Howlin et al. (2004) gave voluntary work the second highest code, falling after employment or self-employed, but before supported/sheltered employment. When examining our data in detail, it appeared that volunteering was not a pathway to employment, and instead was often more of a recreational activity or a last attempt to provide some sort of minimal vocational activity for the adult with ASD. Thus, we gave a lower code for adults who were volunteering with no other vocational or educational activities.

Limitations of Our Vocational Index

We expect that our Vocational Index will be a powerful tool that enables researchers to capture the diversity of employment, vocational, and educational activities experienced by adults with ASD. However, it is subject to limitations. Although the Vocational Index was developed using a large sample of adults with ASD, all adults in the study came from two states within the United States (Wisconsin and Massachusetts). Nevertheless, given the wide diversity of vocational activities in our sample, we feel confident that the range of activities of these adults is representative of the possible range of activities of adults who live in other geographical areas (even if the distribution of adults in each of these types of activities differs by region). However, it is possible that situations would arise that would not easily follow the specified decision rules and coding scheme. Furthermore, similar to other studies that examined vocational outcomes for adults with ASD, our index takes a normative perspective when placing an order on vocational outcomes. Thus, competitive employment and postsecondary degree-seeking educational programs were given the highest score, with scores decreasing as more supports are needed to carry out the activities. As we will explain below, we feel that this index is best used in conjunction with more subjective measures of adult outcome that take into account the qualitative aspects of employment in whatever vocational setting is appropriate for an individual’s abilities (Taylor 2009).

Recommendations for Use and Directions for Future Research

There are a number of ways that our employment index can be used to capture outcomes for adults with ASD as well as change in outcomes over time. First, we recommend that this index be used in conjunction with other measures of successful adulthood. Independence in vocational settings is an important aspect of success, but so too is having a high level of performance at work. Good work habits, appropriate behavior in the workplace, minimal absenteeism, as well as satisfaction with work are all aspects of vocational success that could be incorporated into future measurement approaches and would be less tied to IQ. Furthermore, a good fit (or “job match”) between the person with ASD and his or her work environment is an important consideration when assessing vocational outcomes. Gerhardt (2011) defines a “job match” as the extent to which a particular job meets the individual’s needs in terms of challenge, interest, comfort, camaraderie, status, hours, pay, and benefits. Factors such as these are rarely considered when assessing outcomes for adults with ASD, but could be used alongside more traditional indicators of vocational independence. Unfortunately, we did not measure these variables in our larger longitudinal study and thus were unable to incorporate them as aspects of vocational success into our index.

Another way to move beyond a normative view of successful adult outcomes could be to fully incorporate the number of hours worked per week as a separate indicator of adult success. It is likely that an adult with ASD who has good work habits and few behavior problems, and who enjoys his/her work, might work more hours regardless of the independence inherent in that vocational setting. This person, then, might warrant a score indicating more success in vocational outcomes than someone else who is in that same type of vocational setting but working fewer hours. Of course, it is important to note that there are many system-level factors that limit the number of hours that an adult with ASD works, such as the type and extent of supports offered in the local geographic area or economic considerations. For example, in our data, adults with ASD in Wisconsin were more likely than those in Massachusetts to have no vocational or educational activities (14.9 vs. 8.3 %), whereas adults in Massachusetts were more likely to be spending time in sheltered vocational settings (56.0 % in Massachusetts vs. 45.8 % in Wisconsin; state-level data available from first author). Regardless, a separate consideration of number of hours engaged in vocational activities might be a useful way to capture the subjective vocational experience of adults with ASD.

Another way to use this Vocational Index is to define success not only by level at any given point in time, but also by trajectories over time. Using this index, it would be possible to characterize those individuals who experience relative stability in their level of employment, as well as those who become more independent over time (e.g., moving from a sheltered setting to a supported position in the community) and those who become less independent (e.g., moving from a competitive job to volunteering or no vocational/educational activities). An adult with ASD who has many changes in vocational categories over time, who has downward mobility on our scale, or who is consistently engaged at the minimal level might be considered less successful than a person whose absolute level is lower but who has stability, is upwardly mobile, or who is working an increasing amount of hours over time. The present data lend themselves to such an approach, and we will pursue this type of analysis as a next step.

Finally, this index can be used not only to characterize outcomes of adults with ASD, but also to examine factors that predict vocational outcomes. Studies that have examined success for adults with ASD tend to find more positive outcomes predicted by higher IQ scores and better early language abilities (Eaves and Ho 2008; Howlin et al. 2004; Billstedt et al. 2007; Lord and Bailey 2002). Future research should examine whether these same factors are also important predictors of our Vocational Index. But while knowing an individual’s IQ and early language abilities may help predict vocational outcomes for adults with ASD, this information is less helpful in considering ways to improve outcomes. Thus, future research should also examine malleable factors that may be related to better vocational outcomes for these adults. For example, Farley et al. (2009) found that adults with ASD with higher levels of adaptive behavior had more favorable outcomes. Further, they attributed the higher rate of positive outcomes in their sample to the overall high level of community inclusion. Malleable factors such as adaptive behavior or social participation should be further examined, as they may provide better avenues for intervention than more static factors like IQ or early language.

Conclusions

The present study adds to the extant literature by developing an index of vocational and educational activities for adults with ASD that consists of clear coding rules while still encompassing the diversity of their activities. Using a measure of vocational outcomes that can be reliably coded from study to study is important, as it will contribute to our ability to examine differences in these outcomes among adults with ASD living in different geographic areas. Furthermore, by reliably capturing the full range of employment, vocational, and educational experiences of adults with ASD, an index such as this will allow researchers to better capture change in vocational outcomes over time due to maturation, social change, or intervention.