Commentary: The observed association between autistic severity measured by the Social Responsiveness Scale (SRS) and general psychopathology – a response to

Introduction

In their analysis of the accumulated data from the clinically ascertained Simons Simplex Collection (SSC), Hus et al. (2013) provide a large-scale clinical replication of previously reported associations (see Constantino, Hudziak & Todd, 2003) between quantitative autistic traits [as measured by the Social Responsiveness Scale (SRS)] and variation in behavior problem symptomatology, using one of the most widely used instruments in the world, the Child Behavior Checklist (CBCL). One of the more resounding affirmations of such an association comes from a recently published analysis of genetically informative data involving some 20,000 children and adults, demonstrating that the genetic influences on autistic syndromes substantially overlap with those influencing attention deficit hyperactivity disorder and other psychiatric syndromes (Lundstrom et al., 2011). Reexamination of the association between autistic traits and other behavioral symptoms in a large clinical sample affords a unique window of observation on the nature of overlap of neuropsychiatric syndromes.

One way to interpret the SRS-CBCL association is to view it as a methodological confound involving ‘non-ASD-specific factors’ indexed by the CBCL, for which the authors advocate statistically adjusting SRS scores. Indeed, they show that within the restricted range of scores encompassed by the clinical subjects of the SSC, the SRS-total-score association with the CBCL was comparable in magnitude to that with the Autism Diagnostic Interview-Revised Current Behavior Algorithm Total (ADI-C, a parent-report developmental history measure), but stronger than that with the Autism Diagnostic Observation Schedule (ADOS Calibrated Severity Score). This by itself underscores potentially important differences in what is measured by the accumulated observations of parents over time (SRS, ADI-C) versus direct structured observations of clinicians (ADOS). Next, on the basis of an observed lack of association between the CBCL and an orthogonal measure of social developmental delay, the Vineland Adaptive Behavioral Scale, (which, itself, does not specifically index autistic symptomatology), the authors infer that non-ASD-specific behavioral variation measured by the CBCL influences variation in SRS ratings.

A central question that cannot be resolved by the design of the Hus et al. (2013) study, however, is whether the causal arrow points in the opposite direction, that is, the behavioral symptoms, which appear to ‘predict’ autistic severity scores on the SRS, might actually be caused by the autistic syndrome. If the true nature of autistic impairment is that it travels with (and in fact induces) a range of behavioral disability, then controlling a severity measurement for the presence of such disability – or failing to capture the full extent of abnormality in any ‘autism’ measure – could actually distort the measurement of the phenotype. Consider the observation that impairments in motor coordination significantly aggregate in children with autism and correlate with severity of autistic social impairment (Hilton, Zhang, Whilte, Klohr & Constantino, 2012). Given that motor impairments are not included in the current diagnostic criteria for autism and are therefore ‘non-ASD-specific,’ the logical consequence of an argument for statistical correction would be to adjust severity ratings of social-communicative impairment for variation in motor coordination – a practice which could seriously confound attempts to link behavior with underlying genetic or neurobio-logic variation when direction of causation is unknown.

Beyond the issue of the direction of causality, the analyses presented in Hus et al. (2013) evoke several important caveats when evaluating overlap in psychopathology constructs. The first is to consider the purpose for which a given measurement instrument is most applicable within a given sample. Brief quantitative measures such as the SRS are designed to most efficiently characterize symptom severity in relatively weakly selected samples (e.g., general clinical settings, school, community, and population samples). The Hus et al. (2013) data demonstrate that in the SSC, the SRS sensitively distinguishes ASD cases from non-ASD cases at the group level; mean scores for undiagnosed siblings are lower than the population average (males T-score M = 43.7, females T-score M = 44.3), consistent with a sampling design in which siblings of the probands were screened to ensure that they were unaffected (in a sense, the sibling sample was selected for low levels of the very traits that the SRS was designed to detect). Mean scores for ASD cases were over two pooled standard deviations higher than those of undiagnosed siblings (males T-score M = 80.6, females T-score M = 89.63).

This level of discrimination is strong and raises a second key issue about overlap: the question of whether a potential confound remains detectable after accounting for ASD diagnosis. All overlap between autism-related traits and other psycho-pathological traits can be theoretically apportioned into: (a) overlap that directly relates to the presence versus absence of ASD diagnosis; and (b) extraneous influences of unrelated psychopathological traits on symptom measurement. As noted above in the example for motor functioning, removing true score variance that is related to categorical ASD discrimination (ASD vs. no ASD) would defeat the purpose of the measurement.

In our own study (Frazier et al. in press), we used Interactive Autism Network data to show that the magnitude of the relationships between potential confounds and SRS scores is greatly reduced (all partial correlations < .13) after accounting for ASD diagnosis. For this, we implemented a two-stage approach. In stage 1, we examined a range of demographic and clinical factors to determine which ones influence SRS scores, using a linear regression model in the same manner employed by Hus et al. This approach alone, however, does not consider the possibility that such ‘non-ASD-specific’ factors may themselves be caused by autistic symptomatology. Therefore, in stage 2, we implemented a hierarchical regression in which categorical ASD diagnosis, followed by the ‘non-ASD-specific’ factors (identified in stage 1), was incorporated. We note that the only difference between the models implemented across the two stages is that the latter included categorical ASD diagnostic status: it specifically examined whether the ‘non-ASD-specific’ factors continued to predict SRS scores after accounting for ASD diagnosis. If not, then the relationships between those factors and SRS scores are more likely due to their association with the presence of autism. In many scenarios, it will be appropriate to adjust for these residual relationships (possibly using a continuous norming approach), but the residual correlations that we observed after accounting for ASD diagnosis were substantially lower in magnitude than those reported by Hus et al., and therefore less likely to substantively contaminate autism symptom measurement.

Even for confounds that reach a nontrivial threshold for magnitude-of-effect, it is not always clear which ones should actually be implemented in adjustments to a severity score. Demographic factors (age, sex, race/ethnicity) represent the most straightforward example, but in some scenarios, even these adjustments are not ideal (e.g., longitudinal studies of the development of autism symptoms across the life span). For other potential confounds, the appropriateness of an adjustment depends on the context of measurement. If a researcher is interested in identifying individuals with high autism symptom levels in the absence of psychopathology that is causally unrelated to ASD, then it would be reasonable to consider adjustment for these other psychopathological constructs.

The Hus et al. (2013) analyses raise additional psychometric and analytic issues, such as rater influences on construct measurement, evaluation of the magnitude of correlation in samples with wide versus narrow symptom ranges, and the effects of missing data on bivariate correlations and multivariate regressions. Each can have substantial impact on the relationship between potential confounds and parent-reported autism symptom scores; future work developing and validating autism symptom measures will need to pay careful attention to each of these issues.

There are always potential trade-offs in specificity that are incurred by the implementation of brief measurement methods. Given the marked heterogeneity of autistic syndromes, the phenotypic variation inherent within known genetic causes, and the continuous distribution of autistic severity observed in general populations across cultures, it remains unclear at this juncture what a ‘gold standard’ for autistic severity should encompass. Even if one were to assume that there exists natural separation between what constitutes ‘autism’ and what constitutes independent dimensions of behavioral variation confounding its ascertainment, then one might expect to observe such separation empirically. Factor and latent class analysis of SRS data in multiple studies (see Constantino (2011) have consistently failed to demonstrate separable components of dysfunction beyond one-to-two heritable factors that, respectively, reflect the social-communicative and repetitive behavioral symptoms of autism and that capably differentiate autism-affected from unaffected populations of children (Frazier et al., 2012). At the same time – and outside the traditional range of clinical affectation – subtle variations in distributions of SRS scores differentiate undiagnosed relatives of children with familial autism from relatives of children with other psychiatric conditions (Constantino, 2011).

Time will tell what the boundaries of autism are and are not, and those boundaries are changing more rapidly than even the most judicious revisions to diagnostic instruments can keep pace with. In the meantime, ascertainment of severity in autism might best remain broadly conceived, flexible, and inclusive. Scientists should allow the steady accumulation of data on all aspects of phenotypic characterization of children with ASD to guide decision-making about which parameters of behavioral variation most appropriately and feasibly index autistic severity in characterizing the condition over time, and in linking behavioral variation with underlying genetic and neurobiologic mechanisms.