Assessing Global and Autism-relevant Quality of Life in Autistic Adults: A Psychometric Investigation Using Item Response Theory

Abstract

Although many interventions and services for autistic people have the ultimate goal of improving quality of life (QoL), there is relatively little research on how best to assess this construct in the autistic population, and existing scales designed for non-autistic individuals may not assess all meaningful facets of QoL in the autistic population. To address this need, the autism spectrum QoL form (ASQoL) was recently developed as a measure of autism-relevant quality of life. However, the psychometrics of the ASQoL have not been examined beyond the authors’ initial validation study, and important properties such as measurement invariance/differential item functioning (DIF) have not yet been tested. Using data from 700 autistic adults recruited from the Simons Foundation’s SPARK cohort, the current study sought to perform a comprehensive independent psychometric evaluation of the ASQoL using item response theory, comparing its performance to a newly-proposed brief measure of global QoL (the WHOQOL-4). Our models revealed substantial DIF by sex and gender in the ASQoL, which caused ASQoL scores to grossly underestimate the self-reported QoL of autistic women. Based on a comparison of latent variable means, we demonstrated that observed sex/gender differences in manifest ASQoL scores were the result of statistical artifacts, a claim that was further supported by the lack of significant group differences on the sex/gender-invariant WHOQOL-4. Our findings indicate that the ASQoL composite score is psychometrically problematic in its current form, and substantial revisions may be necessary before valid and meaningful inferences can be made regarding autism-relevant aspects of QoL.

Lay Summary

Quality of life (QoL) is an extremely important outcome for autistic people, but many of the tools that are used to measure it do not take into account how QoL may be different for autistic people. Using data from 700 autistic adults, we examined the measurement properties of the autism spectrum quality of life form (ASQoL), a new measure of QoL designed specifically for autistic people. Our results indicate that the ASQoL shows a pronounced sex/gender bias, which causes it to underestimate QoL in autistic women. This bias needs to be eliminated before the ASQoL can be successfully used to measure QoL in the autistic population.

Introduction

Quality of Life (QoL) is defined as “an overall general well-being that comprises objective descriptors and subjective evaluations of physical, material, social, and emotional well-being together with the extent of personal development and purposeful activity, all weighted by a personal set of values” (Felce & Perry, 1995; Karimi & Brazier, 2016). This concept represents a fundamental outcome measure in autism research (Provenzani et al., 2019), and within the neurodiversity paradigm, increased subjective QoL has been suggested as the ultimate goal of any intervention or service provided to autistic individuals (Brown et al., 2020; Burgess & Gutstein, 2007; Kapp, 2018; Robertson, 2009). Research to date on QoL in autistic people has primarily compared the QoL of autistic and non-autistic people across multiple domains, invariably demonstrating that autistic children, adolescents, and adults on average have lower QoL than the general population (Ayres et al., 2018; van Heijst & Geurts, 2015). More recent work in this area has also attempted to determine which clinical, demographic, and environmental factors that most significantly contribute to this reduced QoL (Deserno et al., 2019; Kim & Bottema-Beutel, 2019; Oakley et al., 2020). However, a major limitation of this body of literature is the fact that the measures of QoL used in autistic people were designed for the general population and may not fully represent the factors that autistic people themselves find most salient or meaningful (Erez & Gal, 2020; McConachie et al., 2018, 2019).

In order to enhance the overall validity of QoL measurement in the autistic population, McConachie and colleagues (2018, 2019) conducted focus groups of autistic adults to examine the content of the World Health Organization Quality of Life–Brief Version (WHOQOL-BREF; The WHOQOL Group, 1998a) and WHOQOL Disabilities Module (WHOQOL-DIS; Power et al., 2010) and propose additional facets of autism-relevant QoL that were not adequately assessed. Based on these focus groups, these researchers developed a supplementary QoL item pool (termed the Autism Spectrum QoL [ASQoL]) to be used in conjunction with the WHOQOL-BREF and WHOQOL-DIS when measuring QoL in autistic adults (McConachie et al., 2018). The study additionally provided initial psychometric validation for an eight-item ASQoL score composite, which ostensibly measures the facets of QoL that autistic people find to be missing from the WHOQOL-BREF and WHOQOL-DIS.

While the development of the ASQoL represents a valuable scientific advancement in the study of QoL in autism, the measurement properties of this form have yet to be tested outside of the original validation study by McConachie et al. (2018). Based on these initial findings, the ASQoL appears to have adequate internal consistency reliability, test-retest reliability, a unidimensional factor structure, and strong convergent validity with all four WHOQOL-BREF domains. However, a number of important psychometric properties have yet to be addressed, including the degree to which the ASQoL measures underlying QoL construct equally across different subsets of the autistic population (known as differential item functioning [DIF] or measurement invariance; Vandenberg & Lance, 2000; Wicherts, 2016) and the degree to which ASQoL scores are predicted by known contributors to reduced QoL in autism, such as increased social difficulties, depression, and anxiety. To further the field’s understanding of this promising measure, the current study sought to assess the psychometric properties of the ASQoL in a large US-based sample of verbal autistic adults, focusing specifically on the questionnaire’s latent structure, clinical correlates, and DIF between subgroups of the autistic population. As our study also included several “Global QoL” items from the WHOQOL-BREF, a secondary aim was to assess the psychometric properties of this item pool, determining whether a composite of these items could be validated as a measure of global QoL in the autistic population.

Methods

Participants

Autistic adults were recruited from the Simons Foundation’s Simons Powering Autism Research Knowledge (SPARK) cohort (Feliciano et al., 2018) using the SPARK Research Match service. All participants (a) were between the ages of 18 and 46 years at the time of study invitation, (b) self-reported a prior professional diagnosis of autism spectrum disorder or equivalent condition (e.g., Asperger syndrome, PDD-NOS), and (c) were designated as their own legal guardians (i.e., “dependent” adults in the SPARK cohort were excluded). As a condition of participation in the SPARK cohort, all individuals in our study resided within the United States. Data were collected during early 2019 as part of a larger survey study on repetitive thinking in autistic adults (project number RM0030Gotham). Data from this larger cohort have been reported in several previous publications (Williams, McKenney, et al., 2020; Williams, Everaert, et al., 2020; Williams & Gotham, 2020a, 2020b). Participants were compensated $50 in Amazon gift cards for completion of the study. A total of 1,012 individuals enrolled in the study, 700 of whom were included in the current analyses. Individuals were excluded from the current study if they (a) did not self-report a professional diagnosis of autism on the demographics form, (b) reported demographic variables (e.g., race/ethnicity, sex at birth) that were inconsistent with those originally reported to SPARK, (c) did not complete the QoL measures included in the survey battery, (d) indicated careless responding as determined by incorrect answers to two instructed-response items (e.g., Please respond ‘Strongly Agree’ to this question.), or (e) answered “Yes” or “Suspected” to a question regarding being diagnosed with Alzheimer’s disease (which given the age of participants in our study almost certainly indicated random or careless responding). All participants gave informed consent, and all study procedures were approved by the institutional review board at Vanderbilt University Medical Center.

Measures

Autism Spectrum Quality of Life (ASQoL)

The ASQoL is a 9-item measure of self-reported QoL designed specifically for use in autistic adults (McConachie et al., 2018). The measure consists of one “Global QoL” item (Are you at ease (OK) with ‘Autism’ as an aspect of your identity?), as well as eight items addressing specific facets of QoL, including barriers to accessing services, friendships, sources of support, and sensory issues. Items are rated on a 5-point Likert scale with varying response options, and three of the items are reverse scored. The eight-item composite score (excluding the “Global” item, which correlates poorly with the remainder of the items) has demonstrated sound psychometric properties in this population, including convergence with other measures of health-related QoL (McConachie et al., 2018). In our sample, the eight items contributing to the composite were modestly interrelated, with a mean polychoric inter-item correlation ($\overline{IIC}$) of 0.362 (range: 0.144–0.686). The “Global QoL” item 9 was poorly related to the other items ($\overline{IIC}$=0.134, range: −0.040–0.242), and thus it was excluded from all further analyses of the measure’s psychometric properties.

Global Quality of Life Measure

In order to measure global quality of life, we administered items from the WHOQOL-BREF, a widely-used quality of life measure that has previously been validated in autistic adults (McConachie et al., 2018). Items are rated on a 5-point Likert scale with varying response options. The full WHOQOL-BREF contains 26 items: 2 global items (health and QoL) and 24 additional items organized into four domains of physical health, mental health, social relationships and environment. In general population samples, the WHOQOL-BREF has a bifactor latent structure that is completely invariant across genders (Perera et al., 2018). To reduce participant burden in the current study, we administered a subset of five items that were strong indicators of the “general QoL” factor (mean Item Explained Common Variance [I-ECV] = 0.76, range: 0.69–0.88) (Perera et al., 2018). These five items included the “Global QoL” item 1 (How would you rate your quality of life?), as well as items 5 (How much do you enjoy life?), 6 (To what extent do you feel your life to be meaningful?), 17 (How satisfied are you with your ability to perform your daily living activities?), and 19 (How satisfied are you with yourself?). In our sample, these items were strongly related, ($\overline{IIC}$ = 0.647, range: 0.462–0.756), indicating a high degree of homogeneity.

Additional Clinical Measures

As a part of the larger survey battery, participants also completed multiple other self-report questionnaires, including measures of autism symptomatology, co-occurring psychopathology, and personality, all of which we hypothesized would be associated with reduced QoL in autistic adults. Autistic traits were quantified using the Social Responsiveness Scale–Second Edition: Adult Self-report (SRS-2; Constantino & Gruber, 2012), from which the total T-score was derived. Depression and anxiety symptoms were quantified using the Beck Depression Inventory–II autism-specific T-score (Williams, Everaert, et al., 2020) and Generalized Anxiety Disorder–7 (GAD-7) total score (Kroenke et al., 2010), respectively. Trait neuroticism was measured using a 10-item scale derived from the international personality item pool (Goldberg et al., 2006), which we refer to as the IPIP-N10 (Williams & Gotham, 2020b). Lastly, trait alexithymia was measured using the T-score from the eight-item Toronto Alexithymia Scale (TAS-8; Williams & Gotham, 2020b), which is the only alexithymia measure to be thoroughly validated in the autistic population to date. Reliabilities of these forms in the current sample have been reported previously (Williams & Gotham, 2020b).

Statistical Analyses

Confirmatory Factor Analysis and Model Fit Evaluation

We first sought to assess the latent structures of both QoL measures, using confirmatory factor analysis (CFA) to determine whether each set of items could be explained by a single latent “QoL” factor. As the ASQoL contains three items that are scored in reverse (items 6, 7, and 8), we additionally examined the fit of a bifactor CFA model (Markon, 2019), which contained one “General Autism-relevant QoL” factor as well as a specific factor to capture shared variance among the three reverse-scored items. All CFA models were fit using the lavaan R package (Rosseel, 2012) using the robust diagonally weighted least squares estimator with a mean- and variance-corrected test statistic (i.e., WLSMV estimation; Li, 2016). CFA model fit was evaluated using the chi-squared test, as well as the categorical maximum likelihood-estimated comparative fit index (CFI_cML), Tucker-Lewis index (TLI_cML), and root mean square error of approximation (RMSEA_cML) proposed by Savalei (2020), as typical WLSMV fit indices often fail to detect meaningful misspecification (Xia & Yang, 2019). We additionally calculated the population-unbiased standardized root mean square residual (SRMR_u; Maydeu-Olivares, 2017; Shi et al., 2020) and weighted root mean square residual (WRMR; DiStefano et al., 2018) for each model. Although we reject the idea of stringent fit index cutoffs (see Marsh et al., 2004; McNeish et al., 2018; Tomarken & Waller, 2003), we interpreted our findings with regard to the benchmarks of CFI_cML/TLI_cML>0.95, RMSEA_cML<0.08, SRMR_u<0.05, and WRMR<1.0 indicating adequate model fit. In cases where fit indices suggested that a model did not fit adequately, we further examined areas of local misfit using the modification index method of Saris et al. (2009), using this information to further refine the model.

Item Response Theory Analyses

Best-fitting factor models for both measures were then converted into analogous unidimensional or multidimensional item response theory (IRT) models. Specifically, we fit QoL item data to a graded response model (Gibbons et al., 2007; Samejima, 1969) using maximum marginal likelihood estimation as implemented in the mirt R package (Chalmers, 2012). We then assessed model fit using the limited-information C₂ statistic (Cai & Monroe, 2014), C₂-based approximate fit indices (CFI_C2 and RMSEA_C2), and SRMR, with values of CFI_C2>0.975, RMSEA_C2<0.089, and SRMR<0.05 suggested to indicate good model fit (Maydeu-Olivares & Joe, 2014). Local independence between items was examined using the standardized local dependence (LD) χ² statistic (Chen & Thissen, 1997), with LD- χ² values of 10 or higher indicative of significant LD (Toland et al., 2017). Once these measurement models were finalized, we estimated maximum a posteriori (MAP) latent trait scores (denoted θ), which are interpretable as Z-scores relative to the full sample of 700 autistic adults.

After ensuring the adequate fit and local independence of both IRT models, we then investigated differential item functioning (DIF) across subgroups of the autistic population. In particular, we examined whether the ASQoL and WHOQoL items functioned differentially between groups based on the following factors: biological sex (male vs. female), gender identity (male vs. female), age (>30 years vs. ≤30 years), race (non-Hispanic White vs. others), education level (no higher education vs. some college or more), history of autism-specific services, history of individualized education plan or other special education services, age of autism diagnosis (>18 years vs. ≤18 years), current anxiety disorder, current depressive disorder, lifetime diagnosis of attention deficit-hyperactivity disorder (ADHD), level of trait neuroticism (based on a median split), and level of alexithymia (TAS-8 T-score ≥60 vs. <60). DIF testing was conducted using the iterative Wald procedure proposed by Cao et al. (2017) and implemented in R by the first author (Williams, 2020). The Benjamini-Hochberg (1995) false discovery rate (FDR) correction was applied to all omnibus Wald tests, and only those with p_FDR<0.05 were flagged as demonstrating significant DIF. Significant omnibus Wald tests were then followed up with post-hoc tests of each individual parameter to determine which of these values differed significantly between groups (Stover et al., 2019). Notably, as the null hypothesis of exactly zero DIF is always false at the population level (Cohen, 1994), we calculated DIF effect sizes (Meade, 2010), which allowed us to determine the magnitude of DIF in each item. In particular, we calculated the expected score standardized difference (ESSD), a standardized DIF effect size that can be interpreted on the same scale as Cohen’s d (Meade, 2010). Items with observed ESSD values of greater than ±0.5 (indicative of “medium” effects) were thus flagged as demonstrating practically significant DIF. Additionally, the sum total of all DIF for a given contrast (i.e., differential test functioning [DTF]) was quantified using the expected test score standardized difference (ETSSD).

To further test the validity of these newly proposed QoL latent trait scores, we examined the relationships between these scores and autistic traits (SRS-2), depression (BDI-II), anxiety (GAD-7), neuroticism (IPIP-N10), and alexithymia (TAS-8), all of which we hypothesized to be negatively correlated with both global and autism-relevant QoL. We also hypothesized that there would be positive correlations between each QoL score and the ASQoL “Global QoL” item (Are you at ease (OK) with ‘Autism’ as an aspect of your identity). To determine whether the two QoL measures correlated differently with any given clinical outcome, we statistically compared correlation coefficients using the confidence interval method proposed by Zou (2007) and implemented in the cocor R package (Diedenhofen & Musch, 2015).

Results

Participants and Demographics

As with other samples drawn from this same SPARK study, adults in our sample were predominantly non-Hispanic White, college-educated, and diagnosed with autism as young adults (Table 1). A minority of participants in the sample (35.8%) reported receiving educational services for autism while in school, and slightly over half of respondents (53.3%) had received any autism-specific services in their lifetimes. Current depressive and anxiety disorders (defined as symptoms in the past three months and/or receiving ongoing treatment) were reported in 59.0% and 71.0% of the sample, respectively. Global quality of life, as indexed by WHOQOL item 1 (How would you rate your quality of life?), was “Good” or “Very good” in 57.7% of the sample (see Table 1 for full demographics).

Table 1.

Demographic and Clinical Characteristics of Sample

	Males (n = 259)	Females (n = 441)	Total (N = 700)
Age (Years)	30.95 (7.37)	30.85 (6.84)	30.89 (7.04)
Age of Autism Diagnosis (Years)	18.76 (12.13)	20.43 (10.50)	19.81 (11.15)
Gender Identity
Cisgender	239 (92.6%)	380 (86.2%)	619 (88.4%)
Transgender	2 (0.8%)	13 (2.9%)	15 (2.1%)
Non-binary	17 (6.6%)	48 (10.9%)	65 (9.3%)
Race/Ethnicity
Asian	11 (4.2%)	12 (2.7%)	23 (3.3%)
Black/African American	9 (3.5%)	18 (4.1%)	27 (3.9%)
Native American/Alaska Native	7 (2.7%)	20 (4.5%)	27 (3.9%)
Native Hawaiian/Pacific Islander	2 (0.8%)	0 (0%)	2 (0.3%)
White	203 (78.4%)	374 (84.8%)	577 (82.4%)
Other Race	7 (2.7%)	16 (3.6%)	23 (3.3%)
Hispanic/Latino	18 (6.9%)	36 (8.2%)	54 (7.7%)
Special Education Services for Autism	108 (41.7%)	139 (31.5%)	247 (35.3%)
Any Autism-specific Services	155 (59.8%)	213 (48.3%)	368 (52.6%)
Current Depressive Disorder	134 (51.7%)	279 (63.3%)	413 (59.0%)
Current Anxiety Disorder	151 (58.3%)	346 (78.5%)	497 (71.0%)
Clinical Questionnaires
SRS-2 Total T-score	69.25 (10.75)	72.31 (9.91)	71.18 (10.33)
BDI-II Autism-specific T-score	48.34 (9.08)	51.73 (9.13)	50.48 (9.25)
GAD-7 Total Score (0–21)	6.96 (5.71)	9.19 (5.76)	8.37 (5.84)
TAS-8 T-score	57.83 (10.52)	59.69 (11.06)	59.00 (10.89)
IPIP-N10 Total Score (1–5)	3.43 (0.82)	3.72 (0.69)	3.61 (0.75)
ASQoL Mean Score (1–5)	3.37 (0.79)	3.11 (0.81)	3.20 (0.81)
WHOQOL-4 Mean Score (1–5)	3.17 (0.91)	3.13 (0.89)	3.14 (0.90)
Global Quality of Life
Very Good	34 (13.1%)	61 (13.8%)	95 (13.6%)
Good	118 (45.6%)	191 (43.3%)	309 (44.1%)
Neither Poor nor Good	71 (27.4%)	111 (25.2%)	182 (26.0%)
Poor	28 (10.8%)	66 (15.0%)	94 13.4%)
Very Poor	8 (3.1%)	12 (2.7%)	20 (2.9%)

Note. Continuous variables are presented as M (SD), and categorical variables are presented as n (%). “Males” and “Females” refer to biological sex, as self-reported in the SPARK demographics survey. SRS-2 = Social Responsiveness Scale–Second Edition; BDI-II = Beck Depression Inventory–II; GAD-7 = Generalized Anxiety Disorder–7; TAS-8 = eight-item Toronto Alexithymia Scale; IPIP-N10 = ten-item neuroticism questionnaire; ASQoL = Autism Spectrum Quality of Life form; WHOQOL-4 = four-item global quality of life composite.

Confirmatory Factor Analysis

A unidimensional model of the eight core ASQoL items (as proposed by McConachie et al., 2018) fit the data poorly, with all fit indices falling short of the proposed benchmarks (χ²(20)=501.02, CFI_cML=0.787, TLI_cML=0.702, RMSEA_cML=0.170, SRMR_u=0.103, WRMR=2.424). Examination of local misspecifications indicated that the three reverse-scored items shared substantial additional variance, warranting an additional common factor. A bifactor model, in which these items loaded onto an additional shared-method factor demonstrated substantially improved fit (χ²(17)=94.21, CFI_cML=0.970, TLI_cML=0.951, RMSEA_cML=0.069, SRMR_u=0.041, WRMR=0.969), and thus this model was used in all further analyses. In our sample, model-based reliability (ω_T=0.835) and general factor saturation (ω_H=0.723) were acceptable for the eight-item ASQoL composite score (Rodriguez et al., 2016).

A unidimensional factor model of the five WHOQOL items demonstrated borderline adequate fit, suggesting the presence of non-trivial misspecification (χ²(5)=84.59, CFI_cML=0.960, TLI_cML=0.920, RMSEA_cML=0.161, SRMR_u=0.037, WRMR=1.049). Examination of local misspecification demonstrated that WHOQOL item 17 (How satisfied are you with your ability to perform your daily living activities?) had large unmodeled negative residual correlations with items 6 (estimated parameter change [EPC]=−0.131) and 5 (EPC=−0.101). Based on these findings, item 17 was removed from the WHOQOL composite, greatly improving fit (χ2(2)=3.82, CFI_cML=0.999, TLI_cML=0.996, RMSEA_cML=0.040, SRMR_u=0.007, WRMR=0.247). Despite the small number of items, the four-item WHOQOL composite (WHOQOL-4) demonstrated strong model-based reliability (ω=0.882).

Item Response Theory Models

The eight ASQoL items were fit to a bifactor graded response model, which demonstrated adequate overall fit (C₂(17)=54.84, CFI_C2=0.984, RMSEA_C2=0.056, SRMR=0.050). However, LD was detected between ASQoL items 1 (Do you have enough support from others to make important decisions?) and 4 (Do you have enough support, if or when you need it, to help you deal with problems?), likely due to both items assessing social support (LD-χ²=18.18, Cramer’s V = 0.21). To accommodate this LD, we chose to combine the scores from items 1 and 4 into a 9-point polytomous testlet, which was then modeled as one item. Doing so eliminated all LD (all LD-χ²<4.37) and caused no meaningful deterioration in model fit (C₂(11)=39.34, CFI_C2=0.982, RMSEA_C2=0.061, SRMR=0.052). Item parameters for this model can be found in Supplemental Table S1. Marginal reliability for the ASQoL latent general factor score was adequate (ρ_xx=0.832), with acceptable estimated score reliabilities (r_xx>0.7) within the latent trait range [−2.92,2.11].

A unidimensional graded response model was fit to the WHOQOL-4 items, demonstrating nearly perfect fit (C₂(2)=2.47, CFI_C2=1.0, RMSEA_C2=0.018, SRMR=0.013), and no significant positive LD. Notably, significant negative LD was seen between WHOQOL items 5 and 6 (LD-χ²=−14.73, V=0.19), but such dependency can safely be ignored due to its inability to upwardly-bias item slope parameters (Toland et al., 2017). Item parameters for this model can be found in Supplemental Table S2. Marginal reliability of WHOQOL-4 latent trait scores was strong (ρ_xx=0.890), and acceptable reliability was demonstrated within the latent trait range [−2.68, 2.42].

Differential Item Functioning

None of the eight ASQoL items functioned differentially according to age (all p_FDR>0.828), race/ethnicity (all p_FDR>0.791), education level (all p_FDR>0.377), receipt of autism services (all p_FDR>0.051), history of special education services for autism (all p_FDR>0.097), age of autism diagnosis (all p_FDR>0.135), presence of co-occurring depression (all p_FDR>0.792), or lifetime ADHD diagnosis (all p_FDR>0.075). However, significant DIF/DTF was found according to sex (3 items; ETSSD=0.281), gender (4 items; ETSSD=0.284), presence of a current anxiety disorder (2 items; ETSSD=−0.073), high/low neuroticism (2 items; ETSSD=−0.118), and high/low alexithymia (2 items; ETSSD=0.217). The ASQoL items that demonstrated statistically significant DIF between these subgroups are presented in Table 2.

Table 2.

ASQoL Items with Significant Differential Item Functioning Between Subgroups of Autistic Adults

ASQoL Item #	Grouping Variable	χ2	df	p FDR	UIDS	ESSD	Parameters
6	Sex	24.22	6	0.001	0.451	0.606†	d2, d3, d4 (all F>M)
7	Sex	44.40	6	<0.001	0.614	0.935†	d1, d2, d3, d4 (all F>M)
8	Sex	29.48	6	0.002	0.432	0.480	d2, d3 (all F>M)
3	Gender	13.17	5	0.022	0.107	0.206	d1 (F>M)
6	Gender	19.84	6	0.006	0.411	0.549†	d3, d4 (all F>M)
7	Gender	35.95	6	<0.001	0.560	0.856†	d1, d2, d3, d4 (all F>M)
8	Gender	17.54	6	0.010	0.398	0.463	d2, d3 (all F>M)
1/4 (testlet)	Anxiety	29.02	9	0.001	0.335	0.057	d2 (Anx>NoAnx)
7	Anxiety	18.16	6	<0.001	0.489	−0.847†	d3, d4 (all NoAnx>Anx)
3	Neuroticism	14.95	5	0.011	0.266	−0.574†	d1, d2 (all LowN>HighN)
7	Neuroticism	26.84	6	<0.001	0.394	−0.551†	a2, d1, d2, d3 (all LowN>HighN)
7	Alexithymia	56.53	6	<0.001	0.720	1.299†	a1, d1, d2, d3, d4 (all HighA>LowA)
8	Alexithymia	16.25	6	0.012	0.350	0.439	a1 (HighA>LowA)

Note. Results indicate omnibus Wald tests of differential item functioning (DIF) using the iterative anchor-selection method of Cao et al. (2017). P-values are corrected using the Benjamini-Hochberg false discovery rate procedure (p_FDR). Parameters that were significantly different between groups when tested alone with follow-up Wald tests (p_FDR < 0.05) are indicated in the Parameters column. UIDS = unsigned item difference in the sample (absolute expected item score difference between individuals of the same latent trait level); ESSD = expected score standardized difference (in Cohen’s d metric); a₁/a₂ = slope parameter; d₁-d₄ = item intercept parameters (i.e., item “difficulty” parameters). Larger values of a parameters indicate that the item is more strongly related to the latent trait in one group, whereas larger values of d parameters indicate that a given item response is endorsed at lower latent trait levels in one group relative to the other. F = female; M = male; Anx = co-occurring anxiety disorder; NoAnx = no co-occurring anxiety disorder; LowN = below median neuroticism score; HighN = above median neuroticism score; HighA = alexithymia T-score ≥60; LowA = alexithymia T-score < 60.

^†Practically significant DIF (i.e., |ESSD| > 0.5)

Groups based on biological sex showed the most pronounced differences, with large DIF for item 7 (Do sensory issues in the environment make it difficult to do things you want to do?; ESSD=0.935), moderate DIF for item 6 (Do you feel there are barriers when accessing health services?; ESSD=0.606), and borderline-moderate DIF for item 8 (Do you feel there are barriers to your needs being met in ‘official’ situations (e.g. at the benefits office, at work, with your landlord, etc.)?; ESSD=0.480). All three of these items exhibited differences in their intercept parameters, with autistic women more likely to endorse lower QoL than men at the same latent trait level. To further examine the effect of this DIF on group-level comparisons, we compared ASQoL scores between groups based on sex using (a) observed ASQoL total scores and (b) latent mean ASQoL scores derived from the IRT model that allowed item parameters to vary by group. Based on observed ASQoL scores, autistic women reported significantly lower autism-relevant QoL than autistic men (d=−0.324, CI_95% [−0.478,−0.169]). However, a comparison of latent mean scores indicated that reported QoL was actually numerically higher in autistic women, although the 95% confidence interval of the difference did not exclude zero (d=0.119, CI_95% [−0.035,0.272]). The difference between the two standardized difference values was substantial (Δd=0.443), and the overall bias in comparisons was even larger than predicted by the DTF statistic (ETSSD=0.281). DIF by gender showed a similar pattern, with items 6–8 again underestimating QoL for women of a given trait level (Table 2). DIF by gender was also found in ASQoL item 3 (How secure do you feel about your financial situation?; ESSD=0.206), although the effect was small and practically ignorable for both this item and item 8 (ESSD=0.463). Self-identified autistic women reported significantly lower quality of life when assessed using the ASQoL total score (d=−0.289, CI_95% [−0.449,−0.129]) but not latent group means (d=0.070, CI_95% [−0.089,0.230]). As with DIF by biological sex, the difference in effect sizes (Δd=0.359) was larger than predicted by the DTF statistic (ETSSD=0.284). Less pronounced though still practically significant DIF was noted for groupings based on alexithymia, level of neuroticism, and current anxiety disorder, with item 7 demonstrating the most consistent DIF across groups (Table 2).

Unlike the ASQoL items, the WHOQOL items demonstrated invariance across all tested subgroups, with no statistically or practically significant DIF by age (all p_FDR>0.093), sex (all p_FDR>0.855), gender (all p_FDR>0.903), race/ethnicity (all p_FDR>0.825), education level (all p_FDR>0.819), receipt of autism services (all p_FDR>0.963), history of special education services for autism (all p_FDR>0.929), age of autism diagnosis (all p_FDR>0.991), current anxiety (all p_FDR>0.704), current depression (all p_FDR>0.762), lifetime ADHD (all p_FDR>0.973), level of neuroticism (all p_FDR>0.128), or level of alexithymia (all p_FDR>0.905). Moreover, all estimated ESSD values were smaller than 0.375, indicating that there were not any effects of practical significance that simply did not reach the threshold of statistical significance. To demonstrate the impact of this lack of DIF on group comparisons, we additionally compared WHOQOL-4 QoL scores between groups based on sex using both total scores and model-based latent means/variances. In this comparison, there were extremely small and non-significant sex differences in QoL regardless of whether it was measured according to the WHOQOL-4 total score (d=−0.046, CI_95% [−0.201,0.108]) or latent means from the DIF model (d=0.038, CI_95% [−0.116,0.193]; Δd=0.084).

Validity Testing

As the ASQoL demonstrated relatively poor performance in the DIF testing, we chose to focus primarily on the convergent and divergent validity of the model-based WHOQOL-4 latent trait score. However, in order to provide a comparator for the WHOQOL-4, we additionally examined the relationships between each covariate and the ASQoL total score, comparing external correlations between the two QoL measures. Correlations between the WHOQOL-4 latent trait score, ASQoL total score, and other variables of interest are presented in Table 3.

Table 3.

Correlations Between Quality of Life Scores and Other Self-reported Clinical Outcomes

	WHOQOL-4 (r [95% CI])	ASQoL (r [95% CI])	Δr [95% CI]	Cohen’s q
SRS-2 Total T-Score	−0.419 [−0.478, −0.356]	−0.588 [−0.635, −0.537]	0.169 [0.114, 0.225]	0.228 (small)
BDI-II T-Score	−0.696 [−0.732, −0.656]	−0.545 [−0.595, −0.490]	−0.151 [−0.202, −0.103]	0.248 (small)
GAD-7 Total Score	−0.522 [−0.574, −0.466]	−0.491 [−0.545, −0.433]	−0.031 [−0.086, 0.024]	0.042 (negligible)
IPIP-N10 Mean Score	−0.568 [−0.616, −0.515]	−0.507 [−0.560, −0.450]	−0.061 [−0.121, −0.001]	0.086 (negligible)
TAS-8 T-Score	−0.347 [−0.410, −0.280]	−0.434 [−0.492, −0.371]	0.087 [0.028, 0.147]	0.103 (small)
ASQoL “Global” Item	0.294 [0.225, 0.361]	0.196 [0.124, 0.267]	0.098 [0.035, 0.162]	0.104 (small)

Note. All associations were measured using Pearson correlations, except for those with the ASQoL “Global” item, which utilized polychoric correlations. WHOQOL-4 = four-item global quality of life composite; ASQoL = Autism Spectrum Quality of Life form; Δr = difference between correlations with WHOQOL-4 and ASQoL (i.e., r_WHOQOL-4 – r_ASQoL). SRS-2 = Social Responsiveness Scale–Second Edition; BDI-II = Beck Depression Inventory–II; GAD-7 = Generalized Anxiety Disorder–7; TAS-8 = eight-item Toronto Alexithymia Scale; IPIP-N10 = ten-item neuroticism questionnaire; Cohen’s q = difference in Fisher-transformed correlation coefficients (<0.1 = negligible, 0.1–0.3 = small, 0.3–0.5 = medium; >0.5 = large).

Correlations between the two QoL measures and other clinical variables were largely similar in magnitude, with all correlation differences smaller than 0.17 (Table 3). Notably, the WHOQOL-4 latent trait score correlated highly with the ASQoL total score (r=0.604, CI_95% [0.555,0.649]), supporting the convergent validity of both QoL constructs. The ASQoL total score correlated most highly with the SRS-2 T-score (r=−0.588, CI_95% [−0.635,−0.537]), and this correlation was significantly larger than the SRS-2/WHOQOL-4 correlation (Δr=0.169, CI_95% [0.114,0.225]). Conversely, the WHOQOL-4 score correlated most strongly with the BDI-II latent trait score (r= −0.696, CI_95% [−0.732,−0.656]), and this correlation was significantly larger than the ASQoL/BDI-II correlation (Δr=−0.151, CI_95% [−0.202, 0.103]). Unlike the other psychopathology measures, the GAD-7 did not correlate significantly more with one QoL measure over the other (Δr=−0.031, CI_95% [−0.086,0.024]). The two QoL measures also differed significantly in their associations with personality traits, with the ASQoL total score demonstrating a larger correlation with alexithymia and the WHOQOL-4 demonstrating a larger correlation with neuroticism. Lastly, both measures correlated only modestly with the ASQoL “Global QoL” item, although the polyserial correlation between this item and the WHOQOL-4 score was significantly higher than that with the ASQoL score (Δr_poly=0.098, CI_95% [0.035,0.162]).

Discussion

The ASQoL is a recently developed tool that seeks to assess several aspects of QoL that autistic people identified as particularly salient. However, aside from one initial validation study, the psychometric properties of this measure have not been explored in depth. In the current study, we investigated the latent structure, reliability, and validity of the ASQoL in a large online sample of autistic adults recruited from SPARK. Although the measure demonstrated adequate fit to a bifactor model, good reliability, and criterion validity, several of the items functioned differentially across multiple subgroups of autistic adults. Most notably, ASQoL items 6, 7, and 8 demonstrated substantial DIF (i.e., response bias) according to sex and gender, leading to artifactual sex/gender differences of moderate size in ASQoL total scores. In comparison, the WHOQOL-4, which exhibited no DIF across any tested category, did not display meaningful sex differences at the level of latent trait scores or manifest total scores, further suggesting that the differences seen in ASQoL scores were the result of a statistical artifact rather than true sex differences in reported QoL. Thus, while we wholeheartedly believe that a complete understanding of QoL in autism requires additional facets representing autistic experiences, additional psychometric work is necessary to develop valid measures of autism-relevant QoL that function equivalently across meaningful subgroups of the autistic population (Wicherts, 2016).

Among the ASQoL items demonstrating substantial DIF between groups, item 7 (Do sensory issues in the environment make it difficult to do things you want to do?) was found to function differentially in five of the contrasts with moderate to large effect sizes. Thus, while autistic adults frequently report sensory difficulties as contributing to reduced quality of life (Lin & Huang, 2019; McConachie et al., 2018, 2019), the ASQoL item assessing general sensory issues appears to be interpreted quite differently across various groups of autistic adults. This finding is somewhat unsurprising, given the heterogeneity of sensory differences seen in autism both within and across modalities (Uljarević et al., 2017; Williams, He, et al., 2020). Moreover, as atypical responses to sensory stimuli in autism likely involve multiple other higher-order processes such as emotion regulation and attention (Edgington et al., 2016; Green & Wood, 2019; Thielen & Gillebert, 2019; Williams, He, et al., 2020), it is likely the case that different individuals have substantially different definitions of what “sensory issues in the environment” actually entail. Although we are only able to speculate about the reasons why DIF may occur across the autistic population, we believe that reducing the vagueness of items assessing sensory challenges (e.g., by focusing each item on a single sensory modality) will likely improve the ability of such items to be interpreted uniformly across autistic adults.

Despite the psychometric limitations of the ASQoL in the current study, the 4-item WHOQOL composite score (WHOQOL-4) performed surprisingly well, demonstrating very strong reliability, validity, and excellent fit to the measurement model. Moreover, the WHOQOL-4 items, which assess general life satisfaction and global QoL, did not demonstrate any DIF across the 12 contrasts tested in the current study. Interestingly, this measure correlated somewhat poorly (r<0.3) with the ASQoL “Global QoL” item regarding autistic identity, indicating that autistic identity may be a construct largely separate from (though positively correlated with) general and autism-relevant quality of life (McDonald, 2017). While the abbreviated WHOQOL-4 necessarily provides less information than the 26-item WHOQOL-BREF from which it was derived, this measure represents a viable option to quickly and easily assess global QoL when circumstances would prevent the administration of the full WHOQOL-BREF. Additionally, the WHOQOL-4 could serve as a more reliable measure of nonspecific QoL than WHOQOL-BREF item 1, which is typically used as a standalone measure of global QoL (The WHOQOL Group, 1998a). Although normative scores are not currently available for the WHOQOL-4, a simple heuristic that can be used is to calculate the mean of the four items, producing a composite score ranging from 1 (“Very poor”) to 5 (“Very good”). Given its brevity and favorable psychometric properties in autistic adults, the WHOQOL-4 is an attractive tool for measuring QoL in both clinical and research settings. Future work should attempt to determine the degree to which WHOQOL-4 ratings are influenced by various facets of well-being, as measured by a more comprehensive battery such as the WHOQOL-100 (The WHOQOL Group, 1998b) or the recently-developed PROMIS Autism Battery–Lifespan (Holmes et al., 2020).

This study is not without limitations. First and foremost, the sample of autistic adults surveyed was generally unrepresentative of the wider autistic population, being predominantly female, adult-diagnosed, and college-educated. However, item parameters/DIF tests remain valid when the sample is not representative of the underlying population (Embretson, 1996), and the relative abundance of females in the current study increased the statistical power of DIF tests across sex and gender groups. Additionally, the ASQoL was not administered alongside the WHOQOL-BREF and WHOQOL-DIS, as originally intended by the ASQoL authors (McConachie et al., 2018). It is therefore possible that when administered alongside these additional forms, the ASQoL items could function differently due to the assessment context. Nevertheless, the DIF/DTF observed in the current study was large enough to substantially bias the eight-item ASQoL composite score, and it is unlikely that such pronounced DIF is solely the result of item order effects. Thus, we recommend that researchers not utilize the eight-item ASQoL total score until the form is revised to address its psychometric shortcomings. Future research is desperately needed to develop and validate novel measures of autism-related QoL that are invariant across all major subgroups of the autistic population.

Conclusion

The current study represents the first independent psychometric assessment of the ASQoL and the first to compare it to a similarly short measure of global QoL (the novel WHOQOL-4) in a large sample of verbal autistic adults. The development of the ASQoL represents an important step forward in the measurement of autistic adults’ quality of life, as it is the first measure to incorporate the perspectives of autistic individuals when considering what constitutes a “good QoL” for this population. However, in its current form, the measure demonstrates significant psychometric problems, including differential functioning of many of the items across key subgroups of the autistic population (including men and women, individuals with and without co-occurring mental health conditions, and individuals with varying levels of trait alexithymia). Alternatively, the WHOQOL-4 displayed complete invariance across these same subgroups, providing strong psychometric support for the use of this global QoL measure in autistic adults. Although the study of autism-relevant QoL is still in its infancy, additional focus on the measurement of this construct is essential to making valid inferences about the subjective well-being of autistic individuals.