Child and family characteristics moderate agreement between caregiver and clinician report of autism symptoms

Abstract

Rates of autism spectrum disorder (ASD) and age at first diagnosis vary considerably across the US, and are moderated by children’s sex, race, ethnicity, and availability of services. We additionally suggest that degree of caregiver-clinician agreement on ASD symptoms may play a role in ASD assessment. Since gold standard ASD assessment integrates caregiver-reported developmental history with clinician observations, differential agreement between reporters across demographic groups may contribute to a host of detrimental outcomes. Here, we investigate whether caregiver-clinician agreement on ASD symptoms varies according to child and family characteristics. Comprehensive data from 2,759 families in the Simons Simplex Collection were analyzed. Linear models were created with caregiver reports predicting clinician reports, and moderating effects of child characteristics and family factors were examined. Poorer reporter correspondence was observed when children had higher IQ scores, stronger adaptive behavior, and more behavioral difficulties. Greater disagreement was also associated with African American racial status (for younger children), lower household income, and paternal social difficulties (for older children). Children’s biological sex did not moderate caregiver-clinician agreement. Marked disagreement between caregivers and clinicians could lead to suboptimal or insufficient intervention services and negative experiences for families throughout development. Such families may also be less likely to qualify for research studies, and therefore be underrepresented in the ASD literature. Modified assessment procedures may be required to improve assessment accuracy and family experiences.

Introduction

Autism spectrum disorder (ASD) is an early-emerging neurodevelopmental disorder defined by deficits in social communication and patterns of restricted or repetitive behaviors or interests (American Psychiatric Association, 2013). Although the average age of ASD diagnosis in the U.S. is 4–5 years (CDC, 2014; Shattuck et al., 2009), many parents report social or communication concerns as early as age 6 months (Bolton, Golding, Emond, & Steer, 2012). Recent estimates place ASD prevalence at 1.5% of the population (Christensen et al., 2016), but both age and rates of ASD diagnoses vary tremendously across different demographic groups. Perhaps best documented are sex differences, with a male-to-female ratio of 4.5:1 in ASD prevalence (Christensen et al., 2016), and an older mean age of diagnosis among girls compared to boys (Shattuck et al., 2009). Similarly, rates of ASD differ across racial and ethnic groups, with African American and Latino children being less likely to receive an ASD diagnosis than White children (Christensen et al., 2016). To date, such discrepancies have been attributed to true differences in ASD prevalence across subgroups (Werling & Geschwind, 2013), disparities in access to appropriate evaluation services (Zuckerman et al., 2014), and phenotypic differences that lead to under-identification among some children (Hiller, Young, & Weber, 2014; Kopp & Gillberg, 2011).

Like many psychiatric diagnoses, ASD is defined by a constellation of behavioral symptoms observed over the course of development. Initial diagnosis and subsequent evaluations of skills and deficits therefore rely upon observations and reports from multiple informants. Gold standard ASD evaluation integrates developmental information from caregivers with thorough observational assessments by trained clinicians (Ozonoff, Goodlin-Jones, & Solomon, 2005). Underlying this process is an understanding that caregivers and clinicians provide both unique and overlapping information necessary for understanding individuals’ unique profiles of strengths and difficulties (Risi et al., 2006). Although perspectives across reporters may match closely for some families, others may experience more striking differences between perspectives—for example, if a clinician perceives a difficulty that a family views as normative, or vice versa. Such divergences between reporters’ perspectives may place at least some families at risk for detrimental consequences across the lifespan. Perhaps most importantly, families for whom there is poorer agreement between reporter perspectives may be at elevated risk for (1) inaccurate diagnoses and inappropriate service provision, (2) poorer family experiences with healthcare professionals, and (3) underrepresentation in ASD research. Research in this area has been extremely limited to date, with very little exploration as to how convergence or divergence between caregivers’ and clinicians’ perspectives might influence individual and family outcomes. Thus, our understanding of this issue is far from complete.

Since demographic factors relate to ASD diagnosis, they may also relate to perceptions and interpretations of ASD-related behaviors more broadly, with some families more likely than others to experience significant divergences in perspectives. Two common and well-regarded measures of caregiver and clinician report of ASD symptoms are the Autism Diagnostic Interview–Revised (ADI-R; Lord, Rutter, & Le Couteur, 1994) and the Autism Diagnostic Observation Schedule (ADOS; Lord, Rutter, DiLavore, & Risi, 2003), respectively. Early work with these measures indicated fair agreement for both ASD symptom levels (Risi et al., 2006) and diagnoses (de Bildt et al., 2004), with correlations between caregiver and clinician reports in the range of r=.6–.7 (LeCouteur et al., 2008; Risi et al., 2006). However, studies also reveal marked variability in agreement across research samples (Gray et al., 2008; Ventola et al., 2006), from a low of r=.28 to a high of r=.95 based on sample characteristics (Risi et al., 2006). Child characteristics such as cognitive level appear to matter, as studies have suggested lower correlations between ADOS and ADI-R scores, and poorer sensitivity of clinical cut-offs on some measures, in samples with intellectual disability than in samples with average cognitive ability (Havdahl et al., 2016; Risi et al., 2006). Behaviorally, significant ADHD symptoms appear to increase the age at which children are diagnosed with ASD (Yee & Millichap, 2015). More generally, possible demographic moderators of reporter agreement include socioeconomic status and child race/ethnicity—both of which are associated with ability and desire to pursue ASD evaluations and the age at which an ASD diagnosis is received (Christensen et al., 2016; Emerson, Morrell, & Neece, 2016; Mazurek et al., 2014; Zuckerman et al., 2014).

Following from this discussion, our goal was to explore potential child- and family-level moderators of agreement between caregiver and clinician reports of ASD symptoms. We operationalize caregiver report with the Autism Diagnostic Interview—Revised (ADI-R; Lord et al., 1994), and clinician report with the Autism Diagnostic Observation Schedule (ADOS; Lord et al., 2003). Although diagnostic processes in the clinical and research communities vary and may not routinely include both measures, these instruments are standardized, well-researched, and frequently used in both clinical and research settings. Thus, they represent a solid foundation from which to explore caregiver-clinician agreement.

Method

Participants

Data were obtained through the Simons Simplex Collection (SSC), a national consortium spanning 12 sites across the U.S. (Fischbach & Lord, 2010). The study protocol was approved by each site’s Human Subjects Division, and all families provided informed consent prior to participating. Families were enrolled if they had one child with ASD between ages 4 and 18 years, and no history of diagnosed or suspected ASD in their immediate or extended family. Exclusionary criteria included a nonverbal mental age below 18 months, presence of known genetic conditions (e.g., Fragile X), histories of neurological disease or significant head injury, significant sensory or motor impairment, extensive pregnancy or birth complications, gestational age below 36 weeks at birth, birth weight under 2000 grams, and a primary language other than English.

This procedure yielded a sample of N=2,759 children and adolescents (375 female) with a mean age of 108.3 months (SD=42.8, range 48–216). Research reliable clinicians diagnosed children and adolescents with ASD using CPEA criteria (Lainhart et al., 2006), which includes the ADOS, ADI-R, and expert clinical judgment, with rigorous reliability procedures within and between SSC sites. Severity of symptoms varied across the sample, with a mean calibrated severity score (Gotham, Pickles, & Lord, 2009) of 7.39 (SD=1.7, range 4–10). Self-reported racial/ethnic backgrounds were as follows: African American (4.0%), Asian (4.0%), Native American or Hawaiian (0.3%), White (78.5%), other (4.5%), more than one race (7.8%). An additional 0.8% declined to disclose their racial/ethnic identity. See Table 1 for participant characteristics.

Table 1.

Descriptive statistics for participating families

	Mean (SD)	Range
Child Variables
Age (months)	108.3 (42.8)	48–216
ADI-R Current behavior total score	27.47 (10.1)	1–60
ADOS total score	15.19 (5.2)	7–28
ADOS Calibrated Severity Score	7.44 (1.7)	4–10
Verbal IQ score	78.04 (31.3)	5–167
Nonverbal IQ score	84.54 (26.2)	9–161
Full scale IQ score	81.17 (28.0)	7–167
Vineland Adaptive Behavior Composite	73.14 (12.1)	27–115
CBCL Behavior Problems Total	62.35 (9.1)	27–92
Parent Variables
Maternal age at child’s birth (years)	31.35 (4.96)	16.08–45.25
Paternal age at child’s birth (years)	33.50 (5.72)	17.08–57.58
Maternal BAPQ	85.89 (20.9)	36–169
Paternal BAPQ	96.38 (21.8)	40–179
Maternal SRS-ARV	29.56 (20.4)	0–135
Paternal SRS-ARV	29.78 (22.7)	0–172

Notes: ADI-R = Autism Diagnostic Interview, Revised; ADOS = Autism Diagnostic Observation Schedule; CBCL = Child Behavior Checklist; BAPQ = Broader Autism Phenotype Questionnaire; SRS-ARV = Social Responsiveness Scale – Adult Research Version

Measures

Diagnostic measures

Consistent with SSC data collection protocols, families typically completed the ADI-R and ADOS during a single visit and so reflect concurrent assessment of the child’s ASD symptoms by both reporters. Both the ADI-R and ADOS demonstrate high sensitivity and specificity in identifying ASD (Gotham et al., 2007; Lord et al., 1997).

Caregiver reports of ASD symptoms were assessed with the ADI-R, a semi-structured interview that evaluates current and historical difficulties in domains of social development, communication, and restricted or repetitive behaviors and interests. It should be noted that, as a semi-structured interview, the ADI-R does require some interpretation by clinicians regarding caregivers’ comments and observations. The ADI-R yields scores in each of three domains: social, communication, and restricted/repetitive behavior and interests. Scores across domains were summed for a total ADI-R score using the published “current behavior” algorithm. The number of items contributing to this algorithm differs slightly for children ages 4 years through 9 years, 11 months vs. children ages 10 years and older. To account for this, ADI-R total scores were computed using age-appropriate algorithms (younger group n=1783, older group n=976). Within each age group, we also summed subscores for the social and communication domains to yield a single score for social-communication difficulties. This procedure allowed for ADI-R total scores, as well as domain-specific social-communication and restricted/repetitive subscores.

Clinician report of ASD symptoms was measured with the ADOS, a semi-structured play-based assessment of the same three areas of functioning. For purposes of this study, ADOS total scores were computed using the revised algorithm (Gotham, Risi, Pickles, & Lord, 2007), which sums 14 items across domains of communication, reciprocal social interaction, and restricted/repetitive behaviors. Although the precise items contributing to the algorithm score vary across modules, the number of contributing items is equivalent across them. This algorithm also yields subscores reflecting social-communication difficulties (social affect subtotal) and restricted/repetitive behaviors.

Child characteristics

Child characteristics were identified as potential moderators of caregiver-clinician agreement. Demographic factors included children’s age at assessment, biological sex, birth order, and caregiver-reported race (with sufficient sizes to examine effects for families reporting African American, Asian, and White race). Behavioral characteristics included full-scale IQ, assessed with the Differential Abilities Scale, 2^nd Edition (Elliott, 2007), the Wechsler Abbreviated Scale of Intelligence (Wechsler, 1999) or Mullen Scales of Early Learning (Mullen, 1984); adaptive behavior, assessed with the composite standard score from the Vineland Adaptive Behavior Scales, 2^nd Edition (Vineland-II; Sparrow, Cicchetti, & Balla, 2005); and behavioral difficulties, assessed with the Total Problems T-score from the age-appropriate version of the Child Behavior Checklist (Achenbach, 1991).

Family characteristics

Family characteristics included annual household income (dichotomized below/above $80,000), parents’ ages at child’s birth, and parents’ education (dichotomized below/above college completion). Finally, parents’ social functioning was assessed with the Broader Autism Phenotype Questionnaire (Hurley, Losh, Parlier, Reznick, & Piven, 2007), which yields a self-report estimate, and with the total score from the Social Responsiveness Scale—Adult Research Version (SRS-ARV; Constantino & Todd, 2005) as reported by the parent’s partner. With the exception of annual household income and parental education, all variables were entered as continuous variables.

Analytic Approach

We chose to first examine simple correspondences between reports of ASD symptoms using a correlational approach. This was followed by regression-based moderator analyses to test potential moderators of caregiver-clinician correspondence. Because the ADI-R Current Behavior algorithm incorporates differing numbers of items depending on children’s ages, younger (ages 4:0 to 9:11) and older (10 years and older) groups were analyzed using separate but parallel approaches. For each age group, caregiver-clinician correspondence was assessed through a series of linear regression models in two steps:

• Step 1 – Individual moderator models. Each putative moderator (e.g., child IQ) was entered into a separate regression model along with the ADI-R total score and the Moderator × ADI-R interaction term as predictors of ADOS total scores. In this step, significant interaction terms indicate moderation of caregiver-clinician agreement.

• Step 2 – Combined moderators models. Interaction terms that were significant in Step 1 were included (along with their respective main effects) together in a combined regression model to compare their relative contributions in predicting ADOS total scores. In this second step, significant interaction terms indicate a moderator that maintains its effect above-and-beyond other significant effects.

Since our focus is on identifying factors that affect agreement between reporters, we focus primarily on moderating effects in which child/family characteristics interact with ADI-R Current scores to predict ADOS total scores. Consistent with this approach, we devote less attention to main effects (i.e., child/family characteristics that predict ADOS total scores but do not interact with ADI-R scores) within the following models.

Results

Correlations Between Reports of ASD Symptoms

Caregiver- and clinician-reported ASD symptoms were correlated significantly for both younger, r=.39, p<.001, and older, r=.40, p<.001, participants. Domain-specific correlations between the ADI-R and ADOS suggested that the two reports were moderately correlated with respect to social-communication difficulties (younger: r=.38, p<.001; older: r=.39, p<.001), with smaller correlations with respect to restricted/repetitive behavior (younger: r=.16, p<.001; older: r=.27, p<.001). Comparisons of correlation strengths indicated that reporter correspondence was significantly greater for social-communication difficulties compared to restricted/repetitive behaviors in both the younger, z=7.28, p<.001, and older, z=3.00, p<.01, groups.

Moderation Analyses

Entered alone, caregiver reports of ASD symptoms on the ADI-R current scores predicted clinician assessments with the ADOS total score for both younger, β=.39, t=27.7, p<.001, and older children, β=.40, t=17.3, p<.001. ADI-R current scores accounted for 15.4%, F(1, 1779)=324.5, p<.001, and 15.5%, F(1, 896)=165.97, p<.001, respectively, of the variance in ADOS scores, suggesting influence from additional factors. These factors were subsequently examined through moderation analyses.

Moderators: Younger Children

Step 1 – Individual moderators models

For the younger age group (4:0–9:11), we observed main effects of child IQ, β=−.26, t=−3.53, p<.001, African American racial status, β=.19, t=2.60, p=.009, White racial status, β=−.17, t=−2.40, p=.017, and mother BAPQ scores, β=−.19, t=−2.81, p=.005. Higher ADOS total scores (greater autism severity) were associated with lower IQ, parent-report of African American racial status, and lower maternal BAPQ scores (see Table 2).

Table 2.

Main and interactive effects of putative moderators on caregiver-clinician agreement for the younger group

Main Effect of Moderator			Main Effect of ADI-R		Interactive Effect		Variance
Moderator	β	t-value	β	t-value	β	t-value	Model adj. r2
Child Age	−.081	−1.138	.348	3.782***	.045	.422	.156
Child Sex	−.081	−1.249	.260	3.589***	.183	1.905	.156
Child Birth Order	.109	1.566	.441	8.927***	−.102	−1.179	.155
Child Race/Ethnicity
African American	.187	2.597**	.401	18.068***	−.142	−1.966*	.157
Asian	.038	.586	.392	17.509***	.002	.023	.154
White	−.167	−2.397*	.321	6.84***	.137	1.695	.157
Child Full Scale IQ	−.256	−3.526***	.438	6.127***	−.246	−3.029**	.348
Child Adaptive Behavior	.025	.391	1.018	7.490***	−.692	−5.885***	.245
Child Behavior Problems	−.003	−.042	.714	5.236***	−.346	−2.135*	.170
Household Income	−.041	−.572	.329	4.342***	.088	.903	.154
Maternal Age at Birth	−.079	−1.116	.143	1.007	.278	1.781	.156
Paternal Age at Birth	−.083	−1.220	.174	1.383	.248	1.773	.155
Maternal Education	−1.07	−1.529	.254	3.254**	.183	1.864	.155
Paternal Education	−.033	−.481	.322	4.220***	.098	.991	.154
Maternal BAPQ	−.193	−2.807**	.286	3.225**	.152	1.337	.165
Paternal BAPQ	−.060	−.842	.390	3.858***	.006	.051	.156
Maternal SRS-ARV	−.044	−.649	.414	10.932***	−.041	−.528	.159
Paternal SRS-ARV	.014	.217	.436	12.572***	−.110	−1.520	.161

Notes:

^*p<.05,

^**p<.01,

^***p<.001.

BAPQ = Broader Autism Phenotype Questionnaire (self-report; Hurley et al., 2007). SRS-ARV = Social Responsiveness Scale – Adult Research Version (partner-report; Constantino & Todd, 2005)

Interactions were significant for IQ scores, β=−.25, t=−3.03, p=.002, Vineland adaptive behavior scores, β=−.69, t=−5.89, p<.001, CBCL total scores, β=−.35, t=−2.14, p=.03, and African American racial status, β=−.14, t=−1.97, p=.049. Thus, we observed moderating effects of child IQ, adaptive behavior, behavioral difficulties, and race, such that caregiver-clinician agreement of ASD symptom levels was strongest when children had lower IQ scores, lower adaptive behavior, fewer behavioral difficulties, and were not African American (see Figure 1).

Figure 1.

For younger children, caregiver-clinician agreement was strongest when children had lower IQ scores, lower adaptive behavior, fewer behavioral difficulties, and were not African American. Note that variables were entered as continuous moderators and dichotomized only for purposes of visual display.

Step 2 – Combined moderators model

For the younger group, the subsequent model containing IQ, adaptive behavior, behavior problems, and African American status plus their respective ADI-R interaction terms accounted for 36.1% of the variance in ADOS total scores, F(9, 1774)=112.4, p<.001. For this model, the interaction between ADI-R scores and behavior problems remained significant, β=−.41, t=−2.71, p=.007, as did the interaction between ADI-R and African American race, β=−.17, t=−2.62, p=.009. Thus, for younger children, behavioral difficulties and African American race emerged as the most robust moderators of caregiver-clinician symptom agreement, with stronger agreement between caregivers and clinicians for children with fewer behavioral difficulties and without caregiver-reported African American ethnicity.

Moderators: Older Children and Adolescents

Step 1 – Individual moderators models

For the older group (10:0–18:0), main effects were significant only for IQ, β=−.20, t=−2.16, p=.031. Higher ADOS scores (greater severity) corresponded with lower IQ among older participants (see Table 3).

Table 3.

Main and interactive effects of putative moderators on caregiver-clinician agreement for the older group

Main Effect of Moderator			Main Effect of ADI-R		Interactive Effect		Variance
Moderator	β	t-value	β	t-value	β	t-value	Model adj. r2
Child Age	−.108	−1.105	.192	1.046	.232	1.123	.155
Child Sex	−.132	−1.422	.260	2.577*	.193	1.408	.156
Child Birth Order	−.013	−.126	.329	4.213***	.115	.903	.160
Race/Ethnicity
African American	−.033	−.347	.386	12.407***	.150	1.564	.168
Asian	.041	.407	.390	12.433***	.024	.238	.158
White	.023	.239	.478	7.001***	−.161	−1.459	.168
Child Full Scale IQ	−.200	−2.156*	.480	6.233***	−.334	−3.393**	.388
Child Adaptive Behavior	−.036	−.405	.919	5.545***	−.644	−4.493***	.297
Child Behavior Problems	.101	1.111	1.163	5.237***	−.844	−3.288**	.194
Household Income	−.173	−1.740	.184	1.684	.283	2.039*	.158
Maternal Age at Birth	.067	.700	.348	1.752	.054	.250	.162
Paternal Age at Birth	.004	.045	.191	.962	.231	1.072	.166
Maternal Education	−.032	−.338	.369	3.608***	.037	.274	.154
Paternal Education	−.079	−.830	.329	3.184**	.094	.681	.154
Maternal BAPQ	−.134	−1.380	.319	2.427*	.111	.660	.159
Paternal BAPQ	.022	.232	.503	3.440**	−.130	−.745	.156
Maternal SRS-ARV	−.001	.011	.439	8.150***	−.091	−.806	.160
Paternal SRS-ARV	.075	.784	.485	9.619***	−.216	−2.021*	.169

Note:

^*p<.05,

^**p<.01,

^***p<.001.

BAPQ = Broader Autism Phenotype Questionnaire (self-report; Hurley et al., 2007). SRS-ARV = Social Responsiveness Scale – Adult Research Version (partner-report; Constantino & Todd, 2005)

Interactions indicating moderating effects were significant for child IQ, β=−.33, t=−3.39, p=001, adaptive behavior, β=−.64, t=−4.49, p<.001, behavior problems, β=−.84, t=−3.29, p=.001, family household income, β=.28, t=2.04, p=.042, and father social difficulties, β=−.22, t=−2.02, p=.044. Caregiver-clinician agreement was strongest for children who had lower IQ scores, lower adaptive behavior, fewer behavioral difficulties, higher family income, and fathers with fewer social difficulties (see Figure 2).

Figure 2.

For older children, caregiver-clinician agreement was strongest when children had lower IQ scores, lower adaptive behavior, fewer behavioral difficulties, higher family income, and fathers with fewer social difficulties. Note that household income was entered as a dichotomous variable, while other moderators shown here were entered as continuous variables and dichotomized only for purposes of visual display.

Step 2 – Combined moderators model

For the older age group, the combined model again contained child IQ, adaptive behavior, and behavior problem scores, but also family income and father social difficulties, plus their corresponding ADI-R interaction terms. This model accounted for 40.1% of the variance in ADOS scores, F(11, 840)=52.1, p<.001. Of the moderators entered, behavior problems, β=−.43, t=−1.79, p=.074, and father social difficulties, β=−.19, t=−1.92, p=.055, approached significance, but no moderators emerged as statistically significant when entered simultaneously. Thus, although agreement between reporters was moderated by child IQ, adaptive behavior, behavioral difficulties, family income, and father social difficulties, none of these moderators arose as significantly more influential than the rest when compared directly against one another.

Discussion

Our study is among the first to examine correspondence and divergence between caregiver and clinician reports of ASD symptoms. We identify several child and family characteristics that moderate agreement between these reporters’ perspectives. For both younger (9 years and under) and older (10 years and older) children, caregivers and clinicians showed significant but modest correspondences between their reports of ASD symptoms, with caregiver reports accounting for approximately 15% of variance in clinician assessments. Caregiver-clinician agreement was moderated by a number of child and family factors that increased shared variance to approximately 40%. Moderating effects of child IQ, adaptive behavior, and behavioral difficulties were observed in both groups, such that children with lower IQs, lower adaptive behavior, or fewer behavioral difficulties had better correspondence between reporters. From a practical standpoint, children with the converse–higher IQs, stronger adaptive skills, and more behavioral concerns–were more likely to have poor agreement between caregiver reports and clinician assessments, and consequently may be at elevated risk for detrimental outcomes. To the extent that our findings extend beyond symptom description to include ASD diagnostic outcomes, a point to which we return below, these findings are consistent with concerns that current assessment practices might miss “high-functioning” individuals with ASD (those with strong cognitive and adaptive skills) as well as those with comorbid symptoms such as anxiety, inattention, or hyperactivity that might complicate differential diagnosis (Lai & Baron-Cohen, 2015; Yee & Millichap, 2015).

Beyond shared moderators, other factors were unique to our two age groups. For younger children, African American race was associated with lower agreement—an effect that remained over-and-above other moderators. Previous literature documents that African American children eventually diagnosed with ASD receive their diagnosis an average of 18 months later than White children (Mandell, Listerud, Levy, & Pinto-Martin, 2002), undergo more visits with clinicians before receiving a diagnosis (Mandell, Ittenbach, Levy, & Pinto-Martin, 2007), and more often receive a different diagnosis (e.g., ADHD) prior to ASD (Mandel et al., 2007). Our finding of poorer symptom agreement between caregivers and clinicians suggests one mechanism through which these discrepancies arise, placing African American children with ASD at a disadvantage for identifying areas of need and obtaining appropriate service referrals. With respect to initial diagnosis, it is difficult to overstate the implications of these findings—overwhelming evidence underscores the critical importance of early and intensive intervention for children with ASD (Dawson et al., 2010; Rogers et al., 2014), and such delays in diagnosis (and thus ASD-specific care) have meaningful effects on trajectories and ultimate outcomes of children’s social, communication, and cognitive skills (Dawson et al., 2010; Dawson et al., 2012).

A different set of family factors emerged as significant for older children and adolescents, perhaps suggesting an age-related shift in relative effects of different moderators. In our older subgroup, poorer agreement was associated with lower family income. This may reflect socioeconomic influences on parents’ perspectives on behaviors suggestive of ASD. For instance, although speculative, families with fewer economic resources might have limited access to routine medical care that would otherwise shape their expectations for their child’s development and thus their interpretation of behavior. Regardless of the cause, findings related to family income and child race are concerning, as they likely add a layer of complexity to disparities in accessing specialized evaluation, research, and support services for ASD, even at later ages.

We also observed an effect of fathers’ social difficulties in our older group, such that better agreement emerged when fathers were more competent socially, but we did not observe a comparable effect for mothers. Interpretation of this effect is tentative without knowledge of which parent provided ADI-R data; in some families, mothers reported on both child and father characteristics, with potential common reporter variance. Nonetheless, findings regarding fathers’ social difficulties present an interesting parallel to recent work revealing that fathers—but not mothers—underreport their own broader phenotype traits (Sasson, Faso, Parlier, Daniels, & Piven, 2014). Such findings highlight the importance of incorporating multiple sources of information across home and school settings (e.g., teachers) into diagnostic decisions, as each reporter’s social experiences and skills provide a unique interpretive lens for a child’s behavior.

Child age, sex, and birth order did not moderate agreement between reporters. These null findings are surprising, particularly regarding sex. Since its recognition (e.g., Kanner, 1943), ASD has been diagnosed more frequently in boys than girls, leading some researchers to suggest that current referral and diagnostic processes may overlook girls with significant symptoms (Lai, Baron-Cohen, 2015), perhaps due to sex-specific ASD phenotypes (Kopp & Gillberg, 2011), patterns of comorbidity (Hiller et al., 2014), molecular genetic substrates (Halladay et al., 2015), or behavioral expectations (Shattuck et al., 2009). Our current findings provide no evidence of an influence of sex in caregiver-clinician agreement.

Limitations

We suspect that null findings, particularly with regard to biological sex, are due in part to the nature of our sample. By design, the final SSC sample included only children/adolescents who met strict and rigorous research inclusion standards, meeting ASD diagnostic thresholds on both the ADOS and the ADI-R. This approach yields a sample with high confidence of true ASD, but also introduces a potential floor effect because the range of ASD severity is somewhat truncated. As a result, we likely underestimate moderating influences of some variables. Our findings are conservative in that sense, and a broader sample including children without an established ASD diagnosis might reveal additional moderators.

Similarly, because of sample characteristics, our findings cannot speak directly to factors affecting initial diagnosis of ASD, given that children with wider discrepancies between reporters’ perspectives are less likely to have met the research inclusion diagnostic thresholds and consequently would have been excluded from the sample. Our analyses are best interpreted as relating to variability in symptom description within a sample that had strong agreement on child diagnosis. The implications for diagnostic outcomes are more speculative and will await exploration in additional, community-based samples where individuals might meet diagnostic criteria on one but not both measures. It is also the case that, although frequently used in research settings, the ADI-R and the ADOS are not used universally in clinical settings, due in part to constraints on financial resources, training commitments, and administration time. Thus, we cannot determine whether our results generalize to other caregiver and clinician assessment tools that are more common to clinical settings, particularly those which vary by format (e.g., questionnaire vs. interview, written vs. verbal) or length.

The simplex nature of our sample may also affect generalizability of conclusions. Consistent with the goals of the SSC project more broadly (Fischbach & Lord, 2010), families in the SSC dataset have only one child with ASD, presumed to have ASD of de novo genetic etiology. Moderating factors may differ in multiplex families (those in which a second child has ASD), as parents’ familiarity with ASD or family history of ASD might alter their perspective of their child’s behavior and development, and so alter relative contributions of moderators. Future research with multiplex families will be particularly interesting given phenotypic differences in parents’ social and communication skills across simplex vs. multiplex families (e.g., Gerdts, Bernier, Dawson, & Estes, 2013).

Finally, we must consider possible effects of measurement issues on our findings. Divergence between caregiver and clinician reports could reflect differences in perception (as we discuss here), true variability in children’s behavior across settings, or a combination of both, and our current analyses cannot differentiate between these possibilities. It may be that some children display greater variability in behavior from one setting to another, yielding caregiver and clinician reports that differ but are both accurate. This cannot be determined based on available data, although we emphasize that both child and parental characteristics appear to contribute to these differences. Similarly, although we use ADI-R scores for caregiver reports in this study, semi-structured interviews filter caregiver comments through clinician’s interpretations. Finally, scoring procedures on both the ADOS and the ADI-R yield scores of 0, 1, or 2 for the majority of items, which are then summed according to empirically derived algorithms (Gotham et al., 2007). Although higher algorithm scores are frequently interpreted as indicating ASD severity, correspondence between scores and ASD severity is not absolute, as higher totals may reflect a broader range of ASD symptoms rather than severity of symptoms. Our findings must be interpreted with this caveat in mind.

Implications

Because our data were derived from a research diagnostic process, the most proximal implication is for scientific knowledge of ASD. Families for whom reporters diverge on ASD symptoms may be less likely to meet study inclusion criteria and qualify for research studies, and consequently may be poorly represented in the research literature. Many studies require participants to meet symptom thresholds on both caregiver-report and observational measures, and thus caregivers and clinicians must both indicate sufficient symptoms for study entry. If some subgroups within the population experience wider divergence in reporter perspective, they may be excluded systematically from research that could provide direct benefits to them, and provide more comprehensive understanding of ASD etiology and experience.

Extrapolating tentatively from our data into clinical diagnostic and referral processes, significant disagreement between caregivers and clinicians could promote a number of detrimental effects. Most immediately, marked disagreement in symptom reports at an initial evaluation could result in inaccurate diagnosis. For instance, a clinician may fail to recognize symptoms that are apparent to a caregiver, and may delay appropriate evaluation and supports (Zuckerman, Lindly, & Sinche, 2015). Even years after initial diagnosis, vastly different views of children’s skill deficits and strengths across caregivers and clinicians could result in inappropriate or inadequate supports and services that fail to meet individuals’ medical, educational, or behavioral needs.

More subtly, poor caregiver-clinician agreement may influence families’ experiences with healthcare professionals and attitude toward supports throughout development. Limited existing research addressing discrepancies between reporters suggests that clinicians might prioritize their own ADOS observations over caregiver reports (Risi et al., 2006). Evaluation for ASD is inherently stressful for families (Crane, Chester, Goddard, Henry, & Hill, 2015), and stress is likely exacerbated when caregivers and clinicians disagree about the presence or severity of ASD symptoms or proposed supports/interventions. Consistent with this, qualitative evidence reveals that pediatrician invalidation of parental concerns related to initial ASD diagnosis constitutes a marked source of stress and may affect parent mental health and trust in the provider (Zuckerman et al., 2014). Furthermore, parents’ experiences during the diagnostic process and satisfaction with its outcome likely influence their reception of the ASD diagnosis and their willingness to pursue treatment recommendations (Reed & Osborne, 2012).

Future Directions

Following from this discussion, we highlight a number of important avenues for further examination. First, other potential moderators such as medical comorbidities or known genetic events should be considered, as families with ongoing medical or genetic issues may monitor and interpret their child’s behavior differently from those without such awareness. In addition, caregiver characteristics such as mental health concerns (e.g., depression, anxiety), language proficiency, familiarity with child development, or comfort interacting with medical personnel more broadly might influence how caregivers perceive and describe strengths and concerns related to ASD. Recent research suggests that language match between parents and children may be an important consideration as well (Vanegas, Magaña, Morales, & McNamara, 2016), as parents who have a different primary language from their child (e.g., parents for whom Spanish is primary but whose children speak English as their primary language) may interpret or report their child’s social and communication skills differently, tending to report greater skill and fewer deficits compared to clinician observation (Vanegas et al., 2016). Findings such as this underscore the need to consider individual differences among family members in addition to family-level factors, as well as to consider how these influence instrument validity (Magaña & Smith, 2013; Vanegas et al., 2016).

From a practical standpoint, critical questions remain regarding the degree to which moderators of caregiver-clinician agreement influence referral and diagnosis, and alter ongoing care and treatment over the course of development. Despite their potential importance, these questions have not been addressed directly in the literature. Yet, for families, agreement with their healthcare providers—and variables affecting that agreement—may be critical to their reception of and adjustment to an ASD diagnosis, motivation to pursue recommended intervention services, and ability to find a medical home for their child. If agreement is systematically poorer for some families, they may be at greater risk of negative outcomes in these important areas. Identifying factors that predict such outcomes could help clinicians and researchers more easily identify families at risk and adjust their clinical approach accordingly. For example, clinicians might shift the relative weight they attribute to a family’s report of their child’s behavior versus their own observations, engage in additional conversation about the nature and course of ASD, prioritize transparency in their assessments (e.g., having parents observe assessments with a professional who can explain procedures in the moment), or obtain information from additional observers (e.g., teachers, coaches). Similarly, researchers might consider whether their inclusion protocols bias their samples against families with characteristics identified here, such as low household income or African American ethnicity, and develop strategies to mitigate inequities. Such strategies have the potential to improve clinical outcomes as well as representativeness in research, two critical goals as we continue to enrich our understanding of causes, correlates, and outcomes related to ASD.

Lay Summary.

Evaluation of autism spectrum disorder (ASD) incorporates both caregiver and clinician perspectives of symptoms, and disagreement between these perspectives could lead to poorer outcomes for families. Using data from 2,759 families, we show that caregiver-clinician agreement on ASD symptoms is poorer for children with higher cognitive and adaptive skills, more behavioral difficulties, lower household income, and African American racial status. These children may be at higher risk for misdiagnosis, poorer family experiences during evaluations, and poorer representation in ASD research.