The broader autism phenotype in mothers is associated with increased discordance between maternal-reported and clinician-observed instruments that measure child autism spectrum disorder

Abstract

Autism spectrum disorder (ASD) diagnosis relies on parent-reported and clinician-observed instruments. Sometimes, results between these instruments disagree. The broader autism phenotype (BAP) in parent-reporters may be associated with discordance. Study to Explore Early Development data (N=712) were used to address whether mothers with BAP and children with ASD or non-ASD developmental disabilities were more likely than mothers without BAP to ‘over-’ or ‘under-report’ child ASD on ASD screeners or interviews compared with clinician observation or overall impression. Maternal BAP was associated with a child meeting thresholds on a maternal-reported screener or maternal interview when clinician ASD instruments or impressions did not (risk ratios: 1.30 to 2.85). Evidence suggests acknowledging and accounting for reporting discordances may be important when diagnosing ASD.

The diagnostic criteria for autism spectrum disorder (ASD) are impairment in social communication and interaction, and restricted and repetitive behaviors and interests (RRBI) (American Psychiatric Association 2013). When diagnosing ASD in children, the ASD diagnostic evaluation process relies on a caregiver (usually a parent) reporting on child behavioral and developmental traits as well as on a clinician observing the child’s social abilities and behavior. Generally, one or more clinicians synthesize all available information to reach a diagnosis based on the Diagnostic and Statistical Manual 5^th edition criteria for ASD (American Psychiatric Association 2013; Huerta and Lord 2012).

ASD evaluation instruments, whether clinician observation or parent reports, have strengths and limitations. Clinicians are often experts in ASD evaluation and can compare a child to typically developing children or children with ASD. However, there is only a brief time for clinicians to observe a child during an evaluation, they can only assess current behaviors at a single point in time, and they may be constrained by a clinical setting (e.g. cannot observe daily living skills, interacting with peers) (Lord et al. 2000; Noterdaeme et al. 2002; Westman Andersson et al. 2013). Parents (or other caregivers) are usually the most aware of the child’s development, health status, and current behaviors. Parents are often advocates and experienced reporters on the child’s health conditions (Boshoff et al. 2016; DePape and Lindsay 2015). But, ASD interview instruments based on parent report also have potential shortcomings. The order in which questions are asked can affect response (Jones et al. 2015), and responses about the child’s developmental history might be influenced by current child behavior, developmental level, and demographic characteristics (Hus and Lord 2013). Informant language ability may also affect response, particularly for non-native English speakers (Vanegas et al. 2016; Huerta and Lord 2012). Using parent interviews in combination with clinician observation enables collection of a wider range of information while minimizing potential error of relying on only one brief clinical observation of child ASD symptoms or only on parental report (Wiggins et al. 2015; de Bildt et al. 2004; Le Couteur et al. 2008; Falkmer et al. 2013; Mazefsky and Oswald 2006). Sometimes these two approaches produce discordant information. Past studies found discordance to be associated with a child’s age (de Bildt et al. 2004; Ventola et al. 2006) and RRBI (Wiggins et al. 2015; Le Couteur et al. 2008; Ventola et al. 2006).

Reporting discordance between parents and clinicians or parent and child self-report in other psychiatric disorders may provide background into potential reasons for discordance between interview and observation instruments when evaluating ASD. Mothers with a psychiatric condition like depression (Rothen et al. 2009; De Los Reyes and Kazdin 2005; Verweij et al. 2011; Vandeleur et al. 2015; Ringoot et al. 2015; Hughes and Gullone 2010; Heun et al. 1998; Chilcoat and Breslau 1997; Gartstein et al. 2009; Daryanani et al. 2015), anxiety (Kendler et al. 1991; Briggs-Gowan et al. 1996; Pereira et al. 2015; De Los Reyes and Kazdin 2005), or ADHD (Rothen et al. 2009) were more likely to report traits of that condition in their child as compared to a clinician observation or child interview (Chilcoat and Breslau 1997; Hughes and Gullone 2010). These traits may be reported more frequently because mothers with psychiatric conditions could view the behavior of others more negatively (Torbjörn Ohrt 1999), or children of parents with psychiatric conditions have traits of that psychiatric condition and the parent is more finely attuned when reporting (Richters 1992; Chilcoat and Breslau 1997). Learning more about reporting discordance in these other disorders has provided insight into child psychosocial adjustment (Hoza et al. 2002), family dynamics (De Los Reyes and Kazdin 2006), and intervention efficacy (Lerner et al. 2012; Mikami et al. 2010; De Los Reyes and Kazdin 2008).

Similarly, the presence of broader autism phenotype (BAP) may be a source of differential reporting. BAP is a sub-clinical collection of quantitative autism traits seen in family members of children with ASD (Bishop et al. 2004a; Bishop et al. 2004b; Bora et al. 2016; de Jonge et al. 2015; Shi et al. 2015; Lyall et al. 2014; Sasson et al. 2013b; Sasson et al. 2013a; Berthoz et al. 2013; Mohammadi et al. 2012; Ruta et al. 2012; Szatmari et al. 2000; Gerdts and Bernier 2011). Traits of BAP most often include difficulties with pragmatic language, reciprocal social interaction, and social cognition, as well as behavioral and cognitive rigidity (Sucksmith et al. 2011). Studies have shown that individuals with BAP may also have more problems with anxiety, articulation, empathy, language development, and social initiation and response, compared to people without BAP (Sucksmith et al. 2011; Landa et al. 1991; Berthoz et al. 2013; Lamport and Zlomke 2014). These socio-communicative traits may impact how an informant (the person who reports on a child’s traits) reports on a child (Crocetti et al. 2016; Deoliveira et al. 2005). Additionally, maternal BAP is a predictor of increased parental stress (Derguy et al. 2016; Ingersoll and Hambrick 2011) and anxiety (Lau et al. 2014), which both may have an effect on an informant’s reporting (Muller et al. 2011). Research is needed to examine if autism-like traits in a mother, who most commonly acts as an informant when reporting on child ASD, are a factor in discordance between child ASD evaluation instruments.

Our objective was to evaluate whether maternal BAP is associated with discordance between a child being at risk for ASD on a maternal-reported screening instrument or meeting ASD criteria on an interview instrument as compared to a clinician observation instrument or a measure of clinician overall certainty that the child has ASD (‘best estimate’). We evaluated this discordance by examining whether the maternal report was more or less likely to meet these criteria than clinician observation or ‘best estimate’. Additionally, we evaluated each of five BAP-related domains (social awareness, social motivation, social communication, social cognition, and autistic mannerisms) to see whether a particular area of BAP traits influence discordance. Lastly, we explored whether discordance is associated with maternal self-reported history of a diagnosis of depression or anxiety disorders to better understand whether discordance is specific to BAP or to overall maternal psychiatric conditions more generally.

Methods

Study to Explore Early Development

We used data collected in the first phase of the Study to Explore Early Development (SEED). SEED is a multi-site, community-sampled, case-control study with the purpose of characterizing the autism behavioral phenotype and associated behavioral, medical, and developmental conditions and investigating genetic and environmental risk factors for ASD (Schendel et al. 2012). SEED included six sites: California, Colorado, Georgia, Maryland, North Carolina, and Pennsylvania. Data on children born between 2003 and 2006 were collected between 2007 and 2012. Eligible children were between 30 and 68 months at the time of developmental assessment, were born and resided in the study catchment area at the time of first study contact, and lived with a knowledgeable caregiver who was able to communicate the child’s developmental history in English (or in Spanish in California and Colorado) (Schendel et al. 2012).

Three groups of children were sampled in SEED. The first two groups were children with 1) a past diagnosis or other indication of ASD or 2) a non-ASD developmental disability or delay (DD). These children were identified from multiple education and health providers in the study areas that diagnose and serve children with a broad range of DDs including health clinics, early intervention programs, and special education programs. The third group of children was randomly sampled from birth records at each study site (population controls). Identified families were sent a written invitation to participate and a follow-up invitation telephone call was conducted.

ASD evaluation

During the preliminary phone call with SEED staff, children’s caregivers (99.0% biological mothers) completed the Social Communication Questionnaire (SCQ) (Rutter et al. 2003a), an autism screening instrument. The SCQ is comprised of 40 yes or no questions aimed at assessing a child’s socio-communication ability. The recommended cut-off score that indicates ASD risk is 15 points. Because of the young age of included children (3 to 5 years of age), SEED used a cut-off of ≥11 to define a positive autism screen in order to maximize case finding (Wiggins et al. 2007).

After caregivers completed the SCQ, all children had a developmental assessment that included the Mullen Scales of Early Learning. Children with a past diagnosis of ASD or a positive score on the SCQ received a full diagnostic evaluation. DD and population controls children who screened negative on the SCQ were given a full evaluation if a clinician suspected ASD during the developmental assessment.

For the full ASD evaluation, clinicians conducted the Autism Diagnostic Interview-Revised (ADI-R) (Rutter et al. 2003b) with the child’s caregiver during an in person visit. The ADI-R is a 93-item, 150-minute semi-structured interview that obtains comprehensive information from the caregiver in three domains of child development: social skills, communication skills, and RRBI. The ADI-R also obtains information on whether developmental delays or deficits were noted in the first three years of life (Lord et al. 1994; Rutter et al. 2003b). SEED used the standard ADI-R algorithm to determine whether the child met ADI-R criteria for ASD (Rutter et al. 2003b). Although the ADI-R may not be appropriate for children with mental age less than 24 months (Rutter et al. 2003b), it was still incorporated into SEED final case algorithms for those children (Wiggins et al. 2015). Inter-site reliability was 99% and intra-site reliability was 87% for the ADI-R based on quarterly reliability exercises that included rescoring videotaped ADI-Rs by supervising clinicians (Wiggins et al. 2015; Schendel et al. 2012).

SEED used the Autism Diagnostic Observation Schedule (Lord et al. 2012) as the clinician observation instrument. During the ADOS, clinicians interact with and observe a child for over 40 minutes while creating social opportunities that elicit social communication and social interaction that allow the clinician to record RRBI. The ADOS has specific modules based on age and verbal ability (Lord et al. 2000); SEED used module 1 for children with no or few words and no phrase speech, module 2 for children with phrase speech who were not verbally fluent, and module 3 for verbally fluent children (Wiggins et al. 2015). Standard ADOS algorithms were used for each module to determine ASD classification. Inter-site reliability was 99% and intra-site reliability was 99% based on quarterly reliability exercises that included rescoring videotaped ADOS exams by supervising clinicians (Wiggins et al. 2015; Schendel et al. 2012).

The Ohio State University Autism Rating Scale (OARS) served as the global clinical judgment instrument. The OARS is a tool filled out by clinicians; they use all available information to measure symptom severity, degree of impairment, and clinician certainty in child’s ASD diagnosis (The Ohio State University (OSU) Research Unit on Pediatric Psychopharmacology 2005). This tool was specifically modified to a five point Likert scale for SEED. We dichotomized this scale into “uncertain” (scores of 1–3 or a note that ASD symptoms are better accounted for by another disorder) and “certain” (scores of 4 or 5) (Wiggins et al. 2015; Schendel et al. 2012). For this study, a score of 4 or 5 indicated a case, and the OARS served as our ‘best estimate’ of clinician certainty of ASD. Additionally, since the OARS incorporates all available information and is similar to how a diagnosis is determined in a clinical setting, it acted as a diagnostic ‘gold standard’ in our analyses. Details of these instruments are presented in Online Supplement 1. Further information on SEED procedures can be found in Schendel et al. (2012) and Wiggins et al. (2015).

BAP instrument

BAP was assessed using the Social Responsiveness Scale-Adult (SRS-A) (Constantino 2002). Although not originally designed to measure BAP, the SRS-A has shown good consistency with other BAP and qualitative autism trait instruments (Nishiyama et al. 2014; Ingersoll et al. 2011a). The SRS-A is a 65-item Likert scale questionnaire that takes 15-20 minutes to complete and assesses the caregiver’s social and communication traits. In SEED, the caregivers were asked to have a spouse (or friend or relative if no spouse) complete the SRS-A on them, as recommended for this instrument. A strength of the SRS-A is its ability to measure five distinct domains of social responsiveness: social awareness, social cognition, social communication, social motivation, and autistic mannerisms (Constantino 2002). The SRS-A has strong internal validity, exhibiting a Cronbach’s alpha internal consistency coefficient of 0.95 (Ingersoll et al. 2011a; Constantino and Todd 2005). Scores are shown to be independent of intelligence quotient (IQ) and age (Constantino 2002; Constantino et al. 2009; Ingersoll et al. 2011a). Raw overall and domain SRS-A scores were standardized to T-scores, which have a mean of 50 and a standard deviation of 10. For this study we used the standard ‘mild range’ recommendation of ≥60 to indicate BAP and those with scores <60 to indicated no BAP (Constantino and Todd 2005). Domain scores were dichotomized using the same T-score standardization.

Discordance

We created variables to indicate discordance between the maternal screener (SCQ) or maternal interview (ADI-R) and the clinician observation (ADOS) or clinician ‘best estimate’ (OARS) using the SEED cut-off scores for each instrument. Our four comparisons were the SCQ versus ADOS, SCQ versus OARS, ADI-R versus ADOS, and ADI-R versus OARS. If the two instruments were in agreement (both indicated ASD risk or ASD or both indicated no ASD risk or ASD) then they were considered ‘not discordant’. Since the ASD evaluation is a complex process that takes place over a limited amount of time, we do not want to imply that a mother’s report or clinician observation on child ASD is the ‘correct one’; however, since we are more concerned about influence of maternal characteristics on reporting, we use the clinician completed instruments as our reference group when presenting our results. Therefore, we define ‘over-reporting’ as the maternal report meeting the instrument’s threshold when the clinician did not and ‘under-reporting’ when the maternal report did not meet the threshold while the clinician did.

Study Sample

For this study, we included all children in SEED identified from educational or medical providers that serve children with DDs (N=2,541). We then excluded siblings (since inclusion would violate independence) (N=61); children without a completed SCQ (N=26), ASD evaluation (N=1424) or maternal SRS-A (N=167); and children whose mother did not act as the sole informant on the ADI-R or SCQ (N=151). Our final analytic sample was 712 mother-child dyads.

Although some children with history of DD did not receive the full evaluation, we elected to include those who did in our sample. We chose to include these children because if a child screened negative on the SCQ and a clinician had suspicion of ASD at the clinic visit, the child still received the full ASD evaluation (N=51). We ran a sensitivity analysis excluding these children from our sample to determine the impact of inclusion.

Analytic approach

Demographic information and maternal-reported psychiatric history were collected in SEED using a maternal interview, self-completed questionnaires, and data abstraction from child’s birth and medical records. Maternal psychiatric history was collected using the maternal medical history form where a mother self-reported whether she had previously received a physician diagnosis of certain conditions. We calculated means and percentages for these demographic characteristics by BAP status and by ADOS vs. ADI-R discordance as a representative outcome.

We used log-binomial regression to estimate percent discordance and evaluated whether discordance (‘over-reporting’ or ‘under-reporting’ compared to not discordant) differed by maternal BAP status using α=0.05 to indicate statistically significant differences. A model was run for ‘over-reporting’ which excluded those who ‘under-reported’ and a model was run for ‘under-reporting’ which excluded those who ‘over-reported’.

Since 167 mothers were missing the SRS-A (18.6% of mothers who met other eligibility criteria), we ran a sensitivity analysis to examine potential selection bias. We predicted missing SRS-A based on demographic variables then used individual probabilities to calculate inverse probability weights. We reran our analysis using these inverse probability weights to evaluate the impact of this missingness.

Based on BAP being a well-validated construct that is not affected by demographics like age or education (Gerdts and Bernier 2011), the SRS-A being independent of age and IQ (Constantino and Todd 2005), and scores not varying by race or ethnicity (Constantino and Todd 2005; Constantino and Gruber 2012), we do not believe that there are confounders for which to adjust in our main analyses. However, it is possible that language difficulties affected both completion of the SRS-A and the SCQ or ADI-R. We ran sensitivity analyses to evaluate how our estimates would change if we excluded those who indicated that their preferred language was Spanish (n=47).

For our secondary objectives, we ran models with all five SRS-A to evaluate effects of a single domain controlling for the others. To explore whether maternal self-reported history of a diagnosis of depression or an anxiety disorder (referred to as depression diagnosis or anxiety diagnosis) had an association with discordance, we ran models with depression diagnosis or anxiety diagnosis as an independent variable instead of BAP status. Being that this analysis is more exploratory in nature and the literature is limited on whether BAP confounds the relationship between depression or anxiety and reporting on ASD measures, we elected to calculate effect estimates that were not conditional on maternal BAP status. We qualitatively examined whether effect estimates differed from the BAP estimates.

Results

In our analytic sample of 712 mother-child dyads that met all entry criteria, 67 mothers (9.4%) met BAP criteria and 645 did not (91.6%). Table 1 presents demographic variables by overall BAP status and discordance status [Table 1]. Among mothers with BAP, 40.3% were black, 23.9% were Hispanic, and 19.4% had ≥16 years of education. Among mothers without BAP, 19.8% were black, 14.8% were Hispanic, and 51.0% had ≥ 16 years of education. Mothers who ‘over-reported’ were 68.6% white and 22.9% had <12 years of education. Mothers with no discordance were 60.9% white and 7.1% had <12 years education.

Table 1.

Distribution of socio-demographic characteristics for families referred into the Study to Explore Early Development (SEED), by maternal broader autism phenotype status and ADI-R–ADOS discordance

	BAP+a N=67		BAP−b N=645		Not discordantc N=553		‘Over-reporting’d N=35		‘Under-reporting’e N=124
	N	%	N	%	N	%	N	%	N	%
Child Case status
ASD	40	59.7	439	68.1	410	74.1	0	0.0	69	55.7
DD	25	37.3	174	27.0	122	22.1	29	83.9	48	38.7
Possible Casef	2	3.0	32	5.0	21	3.8	6	17.1	7	5.7
Child Sex
Male	51	76.1	514	79.7	442	79.9	28	80.0	95	76.6
Female	16	23.9	131	20.3	111	20.1	7	20.0	29	23.4
Maternal race
White	33	49.3	462	62.3	385	60.9	24	68.6	75	60.5
Black	27	40.3	147	19.8	139	22.0	5	14.3	25	20.2
Asian	3	4.5	51	6.9	43	6.8	1	2.9	9	7.3
Other	3	4.5	46	6.2	39	6.2	3	8.6	8	6.5
Multi-racial	1	1.5	35	4.7	26	4.1	2	5.7	7	5.6
Maternal ethnicity
Hispanic	16	23.9	95	14.8	86	17.4	8	22.9	17	13.7
Not-Hispanic	51	76.1	549	85.3	466	84.4	27	77.1	107	86.3
Missing			1		1
Maternal education (years)
<12	13	19.4	42	6.5	39	7.1	8	22.9	8	6.5
12 to <16	41	61.2	274	42.5	252	45.6	16	45.7	47	37.9
>=16	13	19.4	329	51.0	262	47.4	12	34.3	69	55.6
Maternal self-reported history of depression diagnosis
Yes	23	36.5	158	24.8	147	27.1	8	23.5	26	21.8
No	40	63.5	479	75.2	400	73.7	26	76.5	93	78.2
Missing	4		8		6		1		5
Maternal self-reported history of an anxiety disorder diagnosis
Yes	11	17.7	84	13.2	70	12.9	8	21.6	17	14.2
No	51	82.3	551	86.8	473	87.1	26	70.3	103	85.8
Missing	5		10		10		1		4
Site
California	15	22.4	95	14.7	80	14.5	11	31.4	19	15.3
Colorado	9	13.4	132	20.5	116	21.0	5	14.3	20	16.1
Georgia	10	14.9	130	20.2	115	20.8	4	11.4	21	16.9
Maryland	11	16.4	81	12.6	80	14.5	5	14.3	7	5.6
North Carolina	8	11.9	135	20.9	97	17.5	8	22.9	38	30.6
Pennsylvania	14	20.9	72	11.2	65	11.8	2	5.7	19	15.3
Child age (months)
Mean, SD	60.3	6.0	59.5	6.6	59.7	6.4	61.6	5	58.4	7.4
Missing			3		2		1
Maternal age (years)
Mean, SD	34.1	6.4	36.1	5.7	35.9	5.8	33.8	6.1	36.3	5.7
Missing			1		1
Number of children in household
Mean, SD	2.5	1.3	2.2	1.0	2.3	1	2.4	1.37	2.3	1.0
Missing	2		8		8		1		1
Past child ASD diagnosis
Yes	37	55.2	458	71.2	407	73.9	16	45.7	72	58.1
No	30	44.8	185	28.8	144	26.1	19	54.3	52	41.9
Missing			2		2

ASD: autism spectrum disorder;

BAP: broader autism phenotype;

DD: developmental disability;

SRS-A: Social Responsiveness Scale- Adult;

SCQ: Social Communication Questionnaire;

OARS: The Ohio State University Autism Rating Scale

SD standard deviation

Sample: referred into SEED, had completed SRS-A, SCQ, and full evaluation Excludes siblings

^aBAP+ is when a mother scores ≥60 on the SRS-A

^bBAP− is when a mother scores<60 on the SRS-A

^cNot discordant is when the SCQ or ADI-R has the same result as the ADOS or OARS

^d‘Over-reporting’ is when the SCQ or ADI-R meet SEED criteria for ASD risk or ASD while the ADOS or OARS does not

^e‘Under-reporting’ is when the SCQ or ADI-R does not meet SEED criteria for ASD risk or ASD while the ADOS or OARS does

^fPossible Cases are children whose final case status could not be determined

In our total sample, 624 children met the threshold for ASD risk on the SCQ (87.7%), 456 met ADI-R criteria for ASD (64.0%), 544 met ADOS criteria (76.4%), and 466 had a clinician best estimate of ASD based on the OARS (65.4%). Discordance was common in our sample (Table 2), as our comparison with the least discordance (ADI-R vs. ADOS in BAP-mothers) was 21.9% discordant. The most common discordance was ‘over-reporting’ between the SCQ and ADOS or SCQ and OARS in mothers with BAP (32.8% and 40.3% respectively). Qualitatively, ‘over-reporting’ was less common when the ADI-R was the maternal measure compared to the SCQ. Phi coefficients for correlation between discordance outcomes were moderate to low (ranging from φ =0.60 to φ =0.09). [Table 2].

Table 2.

Discordance between maternal and clinician ratings on child autism spectrum disorder screening and evaluation in children referred into the Study to Explore Early Development (SEED), by maternal broader autism phenotype status

	BAP+ a N=67			BAP− b N=645
Discordance	N	Percent	95% CI	N	Percent	95% CI
SCQ vs. ADOS
‘Over-reporting’ c	22	32.8	23.3, 46.3	130	20.2	17.3, 46.3
‘Under-reporting’ d	4	6.0	2.3, 15.4	68	10.5	8.4, 13.2
Not discordant e	41	61.2	50.6, 74.1	447	69.3	65.8, 73.0
SCQ vs. OARS
‘Over-reporting’	27	40.3	26.3, 33.4	191	29.6	30.1, 53.9
‘Under-reporting’	3	4.5	1.5, 13.5	57	8.8	6.9, 11.3
Not discordant	37	55.2	44.5, 68.5	397	61.6	57.9, 65.4
ADI-R vs. ADOS
‘Over-reporting’	8	11.9	6.2, 22.9	27	4.2	6.2, 22.9
‘Under-reporting’	10	14.9	8.4, 26.4	114	17.7	15.0, 20.9
Not discordant	49	73.1	63.2, 84.6	504	78.1	75.0, 81.4
ADI-R vs. OARS
‘Over-reporting’	12	17.9	10.7, 29.9	69	10.7	8.6, 13.4
‘Under-reporting’	8	11.9	6.2, 22.9	84	13.0	10.7, 15.9
Not discordant	47	70.2	60.0, 82.0	492	76.3	73.1, 79.6

CI: confidence interval;

BAP: broader autism phenotype;

SCQ: Social Communication Questionnaire;

ADI-R: Autism Diagnostic Interview-Revised;

ADOS: Autism Diagnostic Observation Schedule; OARS: The Ohio State University Autism Rating Scale;

^aBAP+ is when a mother scores ≥60 on the SRS-A

^bBAP− is when a mother scores<60 on the SRS-A

^c‘Over-reporting’ is when the SCQ or ADI-R meet SEED criteria for ASD risk or ASD while the ADOS or OARS does not

^d‘Under-reporting’ is when the SCQ or ADI-R does not meet SEED criteria for ASD risk or ASD while the ADOS or OARS does

^eNot discordant is when the SCQ or ADI-R has the same result as the ADOS or OARS

Maternal BAP was significantly associated with ‘over-reporting’ compared to no discordance when comparing the SCQ to the ADOS (risk ratio (RR): 1.63, 95% confidence interval (CI): 1.12, 2.37), and the ADI-R to the ADOS (RR: 2.85, 95% CI: 1.35, 6.03) [Figure 1]. The effect estimate for the SCQ versus the OARS (RR 1.26, 95%CI: 0.95, 1.77) and ADI-R compared to the OARS (RR: 1.65, 95% CI: 0.97, 2.81) were similarly elevated, but did not reach statistical significance. There were no significant differences when assessing ‘under-reporting’, but the effect estimates suggest less ‘under-reporting’ by mothers with BAP when reporting on the SCQ (SCQ vs. ADOS RR: 0.57, 95% CI: 0.21, 1.50; SCQ vs. OARS RR: 0.51, 95% CI: 0.16, 1.57).

Figure 1.

Risk ratios comparing discordance between maternal and clinician ratings on child autism spectrum disorder screening and evaluation instruments in children referred into the Study to Explore Early Development (SEED), by maternal broader autism phenotype status

No BAP and Clinician observations/estimates is the referent category

SCQ: Social Communication Questionnaire;

ADI-R: Autism Diagnostic Interview-Revised;

ADOS: Autism Diagnostic Observation Schedule;

OARS: The Ohio State University Autism Rating Scale;

Over: ‘Over-reporting’;

Under: ‘Under-reporting’;

‘Over-reporting’ is when the SCQ or ADI-R meets SEED ASD risk or ASD criteria while ADOS or OARS does not

‘Under-reporting’ is when the SCQ or ADI-R does not meet SEED ASD risk or ASD criteria while ADOS or OARS does

Sensitivity analysis that restricted to children with a past ASD diagnosis (results not presented) and that weighted to control for potential bias due to missing SRS-As (Online Supplement 2) both showed minimal differences compared to the primary analysis. Excluding mothers whose preferred language was Spanish (Online Supplement 3) attenuated effect estimates for ‘over-reporting’ and slightly increased our effects for ‘under-reporting.’ Although there were some differences, restricting to English speakers did not change our interpretation of overall results, so we present our full sample estimates. Using a SCQ cut-off of 15 (instead of 11) increased our effect estimates for ‘over-reporting’ between the SCQ-ADOS and the SCQ-ADI-R comparisons by approximately 10% (results not presented).

We also evaluated the association between the five SRS-A domains and discordance. Among all mothers in our sample, 7.0% had a T score ≥60 in the social awareness domain, 13.8% in the social cognition domain, 9.3% in the social communication domain, 13.9% in the autistic mannerism domain, and 10.5% in the social motivation domain. Social cognition (RR: 1.94, 95% CI: 1.06, 3.55) and social awareness (RR: 2.1, 95% CI: 1.15, 3.82) were significantly associated with ‘over-reporting’ on the ADI-R compared to the OARS (Table 3). No other comparisons, including for ‘under-reporting’ (not shown), met statistical significance. [Table 3]

Table 3.

‘Over-reporting’ discordance between maternal and clinician ratings on child autism spectrum disorder screening and evaluation instruments in the Study to Explore Early Development, by Social Responsiveness Scale-Adult domains

	RR	95% CI
SCQ vs. ADOS
Overall	1.63	1.12, 2.37
Social awareness	0.65	0.36, 1.17
Social cognition	1.09	0.67, 1.78
Social communication	1.55	0.88, 2.73
Autistic mannerisms	1.17	0.75, 1.82
Social motivation	0.94	0.57, 1.54
SCQ vs. OARS
Overall	1.30	0.95, 1.77
Social awareness	1.23	0.80, 1.89
Social cognition	1.28	0.88, 1.86
Social communication	1.13	0.68, 1.88
Autistic mannerisms	0.97	0.68, 1.40
Social motivation	0.67	0.42, 1.07
ADI-R vs. ADOS
Overall	2.85	1.35, 6.03
Social awareness	1.33	0.40, 4.39
Social cognition	0.50	0.13, 1.92
Social communication	3.03	0.65, 14.14
Autistic mannerisms	1.73	0.63, 4.77
Social motivation	0.50	0.13, 1.68
ADI-R vs. OARS
Overall	1.67	0.96, 2.92
Social awareness	2.10	1.15, 3.82
Social cognition	1.94	1.06, 3.55
Social communication	0.64	0.26, 1.60
Autistic mannerisms	1.29	0.70, 2.37
Social motivation	0.51	0.20, 1.30

RR: risk ratio;

CI: confidence interval;

BAP: broader autism phenotype;

SCQ: Social Communication Questionnaire;

ADI-R: Autism Diagnostic Interview-Revised;

ADOS: Autism Diagnostic Observation Schedule

OARS; The Ohio State University Autism Rating Scale:

^aRR for ‘over-reporting’ compares domain positive mothers to domain negative mothers Bold indicates statistical significance at an alpha=0.05 level

In our sample, 36.5% of mothers with BAP reported ever having a depression diagnosis versus 24.8% of BAP- mothers. Nevertheless, depression diagnosis was not associated with either ‘over-reporting’ or ‘under-reporting’ (Table 4) [Table 4]. For anxiety diagnosis, 17.7% of mothers with BAP and 13.2% of mothers without BAP reported a previous diagnosis. Risk of ‘over-reporting’ between the ADOS and the ADI-R by anxiety diagnosis was elevated but did not meet statistical significance (RR: 1.97, 95% CI: 0.92, 4.20). Compared to those without anxiety diagnosis, those with anxiety diagnosis also had non-significant elevated risk of ‘over-reporting’ on the SCQ compared to the ADOS (RR: 1.35, 95% CI: 0.95, 1.92) and to the OARS (RR: 1.27, 95% CI: 0.96, 1.77).

Table 4.

Risk ratios and 95% confidence intervals for discordance between maternal and clinician ratings on child autism spectrum disorder screening and evaluation instruments in the Study to Explore Early Development (SEED), by maternal self-reported history of depression or anxiety, or maternal broader autism phenotype

	SCQ				ADI-R
	‘Over-reporting’ a		‘Under-reporting’ b		‘Over-reporting’		‘Under-reporting’
	RR	95% CI	RR	95% CI	RR	95% CI	RR	95% CI
	Maternal Depression
ADOS	0.93	0.68, 1.28	1.02	0.95, 1.10	0.85	0.39, 1.83	0.80	0.53. 1.19
OARS	0.99	0.72, 1.36	0.93	0.53, 1.64	1.14	0.73, 1.78	0.90	0.57, 1.42
	Maternal Anxiety
ADOS	1.35	0.95, 1.92	0.66	0.30, 1.47	1.97	0.92, 4.20	1.09	0.69, 1.73
OARS	1.27	0.96, 1.68	0.99	0.47, 2.08	1.13	0.64. 2.0	1.10	0.64, 1.89
	Maternal BAP
ADOS	1.63	1.12, 2.37	0.57	0.21, 1.50	2.85	1.35, 6.03	0.84	0.47, 1.53
OARS	1.30	0.95, 1.77	0.51	0.16, 1.57	1.67	0.96, 2.92	0.92	0.46, 1.81

RR: risk ratio;

CI: confidence interval;

SCQ: Social Communication Questionnaire;

ADI-R: Autism Diagnostic Interview-Revised

ADOS: Autism Diagnostic Observation Schedule

OARS: The Ohio State University Autism Rating Scale

^a‘Over-reporting’ is when the SCQ or ADI-R meets SEED ASD risk or ASD criteria while ADOS or OARS does not

^b‘Under-reporting’ is when the SCQ or ADI-R does not meet SEED ASD risk or ASD criteria while ADOS or OARS does Bold indicates statistical significance at an alpha=0.05 level

Discussion

In a sample of children with past diagnosis of ASD or other non-ASD DDs, we found that discordance between maternal report instruments (a screening questionnaire or diagnostic interview) and clinician observation or clinician ‘best estimate’ of ASD was common. For our purposes, we used the clinician observation or ‘best estimate’ as our referent category and used ‘over-reporting’ or ‘under-reporting’ as a way to characterize how maternal reporting relates to observations and opinions of clinicians.

Mothers with BAP were significantly more likely than mothers without BAP to ‘over-report’ on the SCQ versus both the ADOS and the OARS, and on the ADI-R versus the ADOS. These results are in agreement with literature for other psychiatric disorders that show that a mother with a psychiatric condition may report more characteristics of the condition in her child than child self-report or clinician observation (Rothen et al. 2009; Hughes and Gullone 2010; Chilcoat and Breslau 1997; Gartstein et al. 2009; De Los Reyes et al. 2011). Qualitatively, effect estimates for ‘over-reporting’ were slightly higher when using the ADI-R compared to the SCQ. This may be a result of the instruments delivery in SEED (in-person vs. telephone), length (150 minutes vs. 15-20 minutes), or purpose (diagnostic interview vs. screening instrument).

We found that mothers with BAP were less likely than mothers without BAP to ‘under-report’ when reporting on the SCQ (compared to ADOS or OARS), but effect estimates were not statistically significant. ‘Under-reporting’ could be less likely because of the nature of the SCQ as a screening instrument, which we chose to have high sensitivity at the expense of specificity, or because of our exclusion of children with DD who scored negative and did not have ASD characteristics. Ultimately, we do not believe this sampling is differential by BAP status. Based on BAP traits like aloofness, it is possible that a person with BAP would be more likely to ‘under-report’ another’s ASD symptoms because of differences in social understanding; however, our results did not support this.

Although we found significant ‘over-reporting’ on the SCQ compared to both the ADOS and the OARS comparing mothers with and without BAP, none of the specific SRS-A domains were significantly associated with such ‘over-reporting’ after controlling for the other domains. It is possible that the overall construct identified by total SRS-A score is more important than any individual domain when assessing discordance on the SCQ. Our sample size may have limited our ability to precisely estimate effects by domain. The autistic mannerisms domain was significantly associated with ‘over-reporting’ on the ADI-R versus the ADOS, and the social cognition domain was associated with ‘over-reporting’ on the ADI-R versus the OARS. It is possible that effects of traits that comprise those two domains are more pronounced in the longer, more intensive ADI-R interview process, but confidence intervals were wide. Further work is needed to examine whether specific BAP traits in those domains, or domains from other BAP instruments, are associated with discordance.

Even though mothers with depression may ‘over-report’ child depression symptomatology (Rothen et al. 2009; De Los Reyes and Kazdin 2005; Richters 1992) and maternal depression is associated with BAP (Ingersoll et al. 2011b), we found no effect of maternal depression diagnosis on discordance between the instruments we evaluated. Our findings are limited to those who reported a past diagnosis of depression, which may not capture the full extent of maternal depression; it may be of future interest to examine associations with dysphoria or trait-based depression since depressive traits could mediate the relationship between BAP and discordance. As for maternal anxiety diagnosis, we saw patterns suggesting ‘over-reporting’ when using the ADI-R compared to the ADOS or the OARS. It is possible that the ADI-R facilitates state anxiety (emotional arousal in response to fear or stressful situations (Endler and Kocovski 2001)) in those with a reported history of an anxiety diagnosis due to the in-depth nature of questioning and time required to complete the interview. It is also possible that trait anxiety (which we did not measure) is exacerbated during the ADI-R and this affects reporting of ASD symptoms in the child. Future research is needed to evaluate this question and associations with BAP using trait-based anxiety measures.

In the context of our results, it should be noted that our sample was comprised primarily of children with past history of DD or ASD. Once concern is raised about a child’s development, a mother may be more likely to push for a diagnosis and report in such a way to receive maximum services, running counter to ‘under-reporting’ child traits (Ryan and Runswick-Cole 2008; McKeever and Miller 2004). Therefore, it might be informative to evaluate discordance among mothers of children with ASD symptoms who have never been through the ASD evaluation process to evaluate these patterns without the mother’s past experience.

Future work is needed to examine the potential causes of these BAP-related discordances in reporting ASD characteristics. One hypothesis is that the ‘over-reporting’ we see could be due to social desirability (Tracey 2016; Henderson et al. 2012), where the social factors of BAP in mothers makes them more susceptible to giving answers that they think the clinician wants to hear, as compared to mothers without BAP. As yet, no empirical studies have evaluated the association between BAP and social desirability bias and SEED did not utilize instruments that assessed respondent social desirability. We examined self-reported past history of a diagnosis of depression or anxiety, but state-based emotional traits that may not be diagnosed and are associated with BAP may play a role in how a mother completes an instrument. Further, mothers may notice or be less tolerant of traits they themselves have and this could lead to ‘over-reporting,’ as has been seen when studying depression (Torbjörn Ohrt 1999). Similarly, a mother with BAP may be a better reporter because she is more finely attuned to her child’s presentation, being that she may have experienced similar socio-communication difficulties. This ‘over-reporting’ may be due to mothers with BAP having children who have a different ASD presentation that is more difficult to assess in the limited clinical observation setting. This question will be evaluated in future work using SEED data.

This study has some limitations. Our sample was restricted to children who were identified through ASD and DD education and health providers, which prevents us from making inferences about the larger SEED sample or the general population. Additionally, the SEED sample is more educated and less frequently Hispanic than the general population, which limits our generalizability. In our study, mothers’ with BAP versus mothers without differ in demographic variables like education, race, and age. However, based on our causal framework, which posits that BAP is independent of demographics like education and maternal age, we do not suspect confounding. Covariate differences may be because our effects are mediated through education, past child ASD diagnosis, or maternal age. When examining effects of culture, specifically language difference, our sensitivity analysis showed slight attenuation of effects when restricting to mothers whose preferred language was English. Next steps include examining cultural difference, measured as ethnicity or race, as an effect measure modifier of the BAP discordance relationship. Further, it is likely that the same clinician completed all instruments, potentially increasing correlation between instruments. Nevertheless, SEED’s thorough evaluation procedure and strong inter- and intra-rater reliability minimize this potential bias. Mothers were asked to have a spouse or friend complete the SRS-A on her, but the relationship to the mother was often left blank on the form (78.8% of responses). If the SRS-A was completed through self-report, it could create misclassification, as self-reports tend to overstate social responsiveness (De la Marche et al. 2015). We do not believe that if a percentage of mothers did self-report that it would substantially affect our results because the difference in self-reporting would be non-differential by BAP (De la Marche et al. 2015), and the extent of any informant effect would be minimal (Constantino and Gruber 2012). Mother’s self-reported their past psychiatric medical diagnoses and we were unable to verify this reporting with medical records. We were unable to assess whether the effect that we see is different if a father or other caregiver reports. This may be of interest for future work.

This study also has major strengths and can inform future research and the ASD diagnostic processes. Our study sample was derived from multiple sites and from various types of education and health providers, not solely clinic-derived. Having a broader population sampling scheme allows for more diversity in included ASD phenotype and family demographics. Additionally, we have data on full diagnostic evaluations with data reported by informants and clinicians, regardless of screener score. Moreover, data from four separate instruments with different uses (screeners, interviews, observation, best estimate) were included, allowing for us to assess discordance between instruments with different goals and methods. We believe that this is one of the first studies to assess BAP as a source of discordance between ASD evaluation instruments.

Our study found that mothers with BAP were more likely to ‘over-report’ and indicate that a child meets an instrument’s criteria for ASD when a clinician does not reach those conclusions. This result was not seen when assessing maternal depression diagnosis and was evident only when comparing the ADI-R and the ADOS for maternal anxiety disorder diagnosis. Future work is needed to address whether the phenotypic profile of children with ASD whose parents have BAP differs from that of children with ASD whose parents do not have BAP, which may explain some of this observed discordance. Additionally, much is still unknown on how people with BAP report on others, regardless of whether they are reporting on ASD; more work can be done to explore how people with BAP act as informants. Based on our results, clinicians may need to be cognizant of maternal socio-communicative ability when synthesizing available information and accounting for instrument discordance when deciding on a final diagnosis for child ASD.