Advancing measurement of ASD severity and social competence: a reply to

The Social Responsiveness Scale (SRS) is currently being used in clinical and genetic studies of autism as both a screener and as a quantitative measure of autistic traits. Our paper (Hus, Bishop, Gotham, Huerta and Lord, 2013) assessed the influence of non-specific factors on SRS scores in order to aid researchers in their interpretations of these scores. In their commentary, Constantino and Frazier (2013) argue that the strong influence of behavior problems on the SRS represents the overlap between neuropsychiatric syndromes, and that behavioral symptoms reflected in SRS scores “might actually be caused by the autistic syndrome” (p.1). They cite evidence for substantial overlap in genetic associations between ASD, ADHD and other child psychiatric disorders. Our concern, however, is that if scores on a measure do not distinguish between general behavior problems and autism symptoms, questions about overlap cannot even begin to be answered. Moreover, what was not noted in the commentary was the fact that we found equally large effects of behavior problems and smaller effects of social competence on SRS scores in typical siblings, who did not have ASD.

It is widely recognized that children with ASD have varying levels of general behavior problems, and that many have comorbid conditions, such as ADHD, language delay and intellectual disability (Simonoff et al., 2008; Lundstrom et al., 2011). However, for researchers seeking to identify causal or risk-related genetic influences, behavioral measures not confounded by these other behaviors are needed in order to draw conclusions that a particular finding is contributing to specific risk for ASD. Similarly, for researchers who want to evaluate associations between particular regions of the brain and core autism symptoms or social competence, evidence that measures such as the SRS are strongly influenced by general behavior problems would seem to present a problem in identifying the specificity of that region.

Thus, our goal in Hus et al (2013) was to determine the influences of non-specific factors known to affect scores on other ASD measures (Charman et al., 2007) in order to provide more informed interpretations of SRS scores. Given the SRS’s widespread use in genetic and neurobiological research, we hoped that we could increase the degree to which the SRS measured social competence and ASD symptoms, and thereby extend its utility in drawing associations between behavioral phenotypes and underlying biology. In previous work, we found this approach useful in improving the validity of scores on the Autism Diagnostic Observation Scale (ADOS). After the introduction of the ADOS, we learned that expressive language level and age (e.g. deBildt et al., 2004) strongly influenced raw ADOS totals. Through expressive language and age-based algorithms (Gotham, Risi, Pickles, & Lord, 2007) and the introduction of severity scores calibrated by the same dimensions, we were able to improve the degree to which the ADOS domain scores represent autism severity within the context of a clinical observation. The calibrated severity scores in the ADOS (Gotham, Pickles, & Lord, 2009; Hus, Gotham, & Lord, 2012) allow researchers to expand the boundaries of constructs of social-communication deficits and repetitive behaviors and quantify difficulties across an interval scale, rather than simply providing categorical cut-offs – contributing to the need for dimensional measures highlighted by Constantino & Frazier (2013).

We approached the current analysis of the SRS from the same point of view. The SRS is particularly valuable because it provides a range of scores even within a typical population. We wondered if we could better understand what child factors could be controlled in order to make the SRS a more specific measure of social competence. Recently, Duku and colleagues (2012) took on a similar task with the SRS, resulting in selection of a subset of 30 items intended to measure social impairments in preschool children, but which still correlated with the CBCL internalizing and externalizing scales, r = 0.65 – 0.68.

We started by looking for how the SRS related to the social domain of the Vineland Adaptive Behavior Scales, a parent report measure widely used as a measure of social competence (Klin et al., 2007; Gillespie-Lynch et al., 2011). We chose social competence because the SRS is commonly referred to as a quantitative measure of social reciprocity that provides scores that can describe both very competent social skills, as well as the full range of social impairments (see Constantino & Gruber, 2012). As shown in the Hus et al (2013) paper, SRS scores are related to Vineland social domain scores and to current ADI scores, but hardly more than they are to CBCL scores in children with ASD (see Tables 3 & S1). In typical siblings, the SRS was most strongly related to the CBCL, with less strong, but significant relationships to developmental level and social competence. Further analyses, not discussed in the commentary, suggested that the effects of behavior problems were not explained by an influence of behavior problems on social competence in either the children with ASD or their typical siblings (see the online supplement in Hus et al, 2013).

In our search for how to better use SRS scores as a measure of social competence or ASD severity, we also considered the effects of factors that we know influence scores on other autism instruments (e.g., the ADOS) including language level, age and IQ. Based on clinical experience that there are items on the SRS that parents of less verbally able or more intellectually disabled children often find difficult to answer, we felt it was particularly important to explore how language level and IQ affected SRS scores. After carrying out these analyses and submitting the original paper, we were asked by reviewers to analyze other instruments. We did in fact go on to carry out similar analyses for the Autism Diagnostic Interview-Revised (ADI-R; Hus & Lord, 2012) and were able to show that ADI-R scores were also strongly related to age and language level. Because these relationships were predictable and easily quantifiable, ADI-R totals and domain scores could potentially be calibrated, as we had done for the ADOS, to provide better measures of domain severity (Hus & Lord, 2012).

For the SRS, such a calibration is less straightforward. Behavior problems, age, language level and IQ should all be considered before interpreting scores as a quantitative measure of social competence or ASD. While this is not impossible to do, it means that, as it stands, individuals who receive high SRS scores may encompass both those with significant ASD symptoms (irrespective of behavior problems), as well as those with high levels of behavior problems (but who may have relatively strong social skills; see Hus et al.(2013), Figure 1). This is not a question of boundaries, but one of precision of measurement.

Measurement factors are recognized as important throughout neurobiology (e.g., Zhu et al., 2009); as mental health researchers, we need to advocate for giving equal weight to measurement issues in describing behavior. Parent factors are widely acknowledged as limitations to using questionnaires over interviews or observational measures. For example, a recent study by Bennett and colleagues (2012) demonstrated strong effects of maternal depression on the SRS, with a weaker effect on the ADI-R and no effect on the ADOS. We need more studies about the properties of all the ASD measures and we need work from research groups independent of those of us who authored the instruments. Improving measurement is a source of advancement in science; we need to support this endeavor and to support transparency in this approach. Researchers and clinicians are well aware of the enormous variability in ASD. It is logical that measures of ASD symptoms will need to take into account individual differences (e.g., language level, IQ, etc.). To not do this, is to miss an opportunity to understand how best to conceptualize and assess different dimensions of severity in ASD.