Influence of Family Demographic Factors on Social Communication Questionnaire Scores

Abstract

This study examined the effect of demographic factors on Social Communication Questionnaire (SCQ) scores in children aged 30–68 months. Diagnoses of ASD were made after a gold standard evaluation that included the Autism Diagnostic Observation Schedule (ADOS), and the Autism Diagnostic Interview Revised (ADI-R). The relationship of demographic variables to SCQ scores was compared in two source populations: (a) children recruited from clinical and educational sources serving children who have ASD or other developmental disorders (CE) and (b) children recruited from birth certificates to represent the general population (BC). The impact of the demographic variables—child sex, child age, maternal language, maternal ethnicity, maternal education, maternal race, and household income—on total SCQ score were studied to examine their impact on the SCQ’s performance. Demographic factors predicting the SCQ total score were used to generate ROCs. Factors that had a significant influence on SCQ performance were identified by examining the area under the ROCs. Optimal SCQ cut-points were generated for significant factors using the Youden’s Index. Overall male sex, lower household income, lower maternal education and Black race predicted higher SCQ scores. In this sample, the most common optimum value for the SCQ cut-point across the different sociodemographic groups was 11.

Standardized screeners for autism spectrum disorder (ASD) are important for assessing potential ASD risk in clinical settings, and for determining ASD risk in population-based studies. To the extent that screeners are meant to identify ASD risk in children for further clinical evaluation, it has been recommended that cut-points be set to maximize sensitivity in order to identify the largest number of possible cases [Norris & Lecalvalier, 2010]. However, in population-based studies, a balance between sensitivity and specificity may be a better option than maximizing sensitivity, which can produce a high rate of false positives that require costly follow-up evaluations, or maximizing specificity, where screeners may fail to detect substantial numbers of children with the condition, resulting in a biased sample and a lengthy period for recruitment. However, the goal of selecting efficient cut-points for study recruitment can be complicated by the fact that the ability of screeners to distinguish high and low risk individuals can vary across population subgroups [Ransohoff & Feinstein, 1978; Whiting, Rutjes, Reitsma, Glas, Bossuyt, & Kleijnen, 2004]. Currently, little is known about the effect of demographic factors on the performance of ASD screeners.

The Social Communication Questionnaire [SCQ; Rutter, Bailey, & Lord, 2003] is a caregiver report screener for ASD that has been widely used in clinical studies [e.g., Allen, Silove, Williams, & Hutchins, 2007; Wiggins, Bakeman, Adamson, & Robins, 2007], as well as in some epidemiological studies [e.g., Johnson et al., 2011; Schendel et al., 2012]. Several studies have documented the impact of demographic variables on the performance of the SCQ. It has been noted that the SCQ may be better at identifying children over seven years, than younger ones [Corsello et al., 2007; Norris & Lecalvalier, 2010]. For younger (<4 years) children, studies have found that an SCQ score of 11 maximizes sensitivity and specificity [Allen et al., 2007; Barnard-Brak, Brewer, Chesnut, Richman, & Schaeffer, 2016; Lee, David, Rusyniak, Landa, & Newschaffer, 2007; Wiggins et al., 2007]. Although these studies indicate child age can impact the performance of the SCQ, there is less evidence about how other demographic characteristics of children and parents may influence the SCQ score.

Additional demographic factors found to be associated with SCQ scores include, child sex, in which girls tended to score lower on the SCQ than boys [Constantino, Zhang, Frazier, Abbacchi & Law, 2010]. In addition, a recent study found lower family income and maternal education are associated with lower SCQ specificity [Moody et al., 2017]. Another study found lower parental education was associated with higher SCQ scores [Tsai, Harrington, Lung, & Lee, 2017]. African American race and Hispanic ethnicity have also been found to affect the performance of the SCQ [Moody et al., 2017]. In addition Hispanic ethnicity and the administration of a screener in Spanish have been found to affect the performance of the MCHAT [Windham et al., 2014].

This study examined how demographic factors affected the performance of the SCQ in the context of a population-based study. There have been few attempts to examine the SCQ’s cut-points for different populations. Such knowledge would improve our understanding of how this tool performs in epidemiological studies, and how its use may need to be tailored to facilitate efficient recruitment for specific populations. This study added to our knowledge of how the SCQ’s sensitivity and specificity can vary across demographic characteristics and by identifying optimal cut-points for different demographic characteristics.

Method

Study Design

SCQ scores and demographic data were collected in the Study to Explore Early Development-Phase I (SEED), a multi-site, case-control study exploring the phenotypes and determinants of ASD; the methods for which have been described in detail previously [Schendel et al., 2012; Wiggins et al., 2015a]. The SEED protocol was reviewed and approved by Institutional Review Boards at each study site. This study examined the relationship of maternal race, ethnicity, primary language, and education; household income, and child sex and age to SCQ scores in two source populations: (a) children recruited from clinical and educational (CE) sources serving children who have ASD or other developmental disorders and (b) children recruited from birth certificates (BC) to represent the general population.

Eligibility, Ascertainment and Recruitment

Children were eligible for SEED if they were born between September 1, 2003 and August 31, 2006 in a study catchment area located in one of six participating states (California, Colorado, Georgia, Maryland, North Carolina, Pennsylvania), resided there at the time of first study contact, and lived with a knowledgeable caregiver (a family member or caregiver at least 18 years of age at enrollment who had resided with and consistently cared for the child since he or she was 6 months of age or younger), who was competent to communicate orally in English or, in California and Colorado, either English or Spanish. Characteristics of each of the study catchment areas, and of children born in the study catchment areas for relevant study years have been described [Schendel et al., 2012]. To maintain the appropriate age range for validated study instruments, children were enrolled so as to be 30 to 68 months of age at the time of completion of the developmental evaluation.

Children with potential ASD or DD (non-ASD developmental delay or disorder) were ascertained from clinical and educational (CE) settings serving or evaluating children with developmental problems and included children who had received an ASD or a related diagnosis from a clinical provider or had received early intervention or special education services for an ASD or a related condition [Schendel et al., 2012]. Related diagnoses included, but were not limited to, intellectual disability (ID) and language delay. Using a broad diagnostic net to define “related diagnoses and conditions” ensured that both previously diagnosed and undiagnosed children with ASD were included. Caregivers of children with an ASD diagnosis from a clinical provider could also refer their child for potential enrollment. Children identified at each site by randomly sampling state birth certificates were eligible for inclusion in the general population source group (BC); these comprised births during the cohort period and born to mothers who were resident in the study catchment area.

Families were initially contacted by letter. Interested families were screened for eligibility during an introductory phone call. Those deemed eligible were administered the SCQ Current Form [Rutter et al., 2003] during the same phone call. The total SCQ score was used to identify children at risk for ASD. Children, who scored positive on the SCQ screener, who had a previous ASD diagnosis, or who had ASD symptoms observed during the study activities, were considered at risk for ASD and underwent an ASD assessment. In this way, even if children screened negative on the SCQ, the clinicians could still use their clinical judgment to triage children to a full evaluation. The SCQ is a 40-item caregiver-report screening questionnaire that asks caregivers to report on the presence of child behavior that is characteristic of ASD. The SCQ takes about 10 min to complete and requires no training to administer. Each item is checked ‘yes’ or ‘no’ and assigned a value of 1 (presence of behavior) or 0 (absence of behavior). Total scores can range from 0 to 39, as the first item is not included in the total score. Internal consistency for the SCQ total score is 0.81 [Snow & Lecavalier, 2008]. SCQ data were scored using the Internet System for Assessing Autistic Children (ISAAC; http://www.autismtools.org). To maximize sensitivity for identification of young children with ASD this study used an SCQ score of 11 to define children at risk for ASD, which would lead to further evaluation [Wiggins et al., 2007]. In addition, children scoring below 11 on the SCQ received the ASD evaluation if they had a prior record of ASD or were noted by a study clinician to show indications of ASD during the basic examination administered to all groups.

Study Procedures and Study Group Classification

Extensive data were collected via multiple methods (caregiver-completed interviews, self-administered forms and questionnaires about the child, family and household; clinical assessments of the child’s development and morphology; and bio-samples from the child, and both parents, if available). Standardized procedures were used across the six sites. Data collection was uniform for all study groups except that those previously diagnosed with an ASD or determined to be at risk for an ASD via the SCQ screen completed additional developmental assessments in the clinic.

Based on recruitment source and the historical and clinical data collected, study children were assigned to final diagnostic categories—ASD, DD, or POP (non-ASD population control), the methods for which have been detailed elsewhere [Wiggins et al., 2015a]. In brief, all enrolled children who completed the clinical visit were administered the Mullen Scales of Early Learning [MSEL; Mullen, 1995]. The assessment protocol for full evaluation included the Autism Diagnostic Observation Schedule [ADOS; Lord, Rutter, DiLavore, & Risi, 1999; Lord et al., 2000], a semi-structured ASD diagnostic observation of children with a mental age of at least 15 months (for the purposes of this study) [Gotham, Risi, Pickles, & Lord, 2007] and the Autism Diagnostic Interview Revised [ADI-R; Rutter et al., 2003], a standardized ASD diagnostic interview for caregivers of children with a mental age of at least 24 months.

This study used a diagnostic algorithm, based on the results of the ADOS and ADI-R, which was the diagnostic gold standard for this cohort [Wiggins et al., 2015a]. Briefly, the algorithm classified children with ASD if they met ASD criteria on both the ADOS and ADI-R or met ASD criteria on the ADOS and one of three alternate criteria on the ADI-R (i.e., met criteria on the social domain and was within two points on the communication domain, met criteria on the communication domain and was within two points on the social domain, or met criteria on the social domain and had two points noted on the behavioral domain). A detailed description of the SEED ASD classification algorithm can be found in Wiggins et al. [2015a].

Children in the final diagnostic category classification of DD had a developmental delay or disability, were judged unlikely to have ASD based on initial SCQ screen or observation, or scored positively for ASD based on screening, but did not meet study classification criteria for ASD based on a clinical evaluation.

Children who received the final diagnostic category classification of POP had been recruited from birth certificates, and were determined either not at risk for ASD on initial screen or observation, or were found to be at risk, but did not meet ASD classification criteria on confirmatory instruments. An Indeterminate group was comprised of children determined to be at risk for ASD who did not complete confirmatory testing or whose low mental age did not allow the child to be assigned a final diagnostic category classification. These children were not included in the analyses for this study. Behavioral and demographic profiles of the enrolled samples have been described [Wiggins et al., 2015b; DiGuiseppi et al., 2016].

Sociodemographic Data Collection

For each enrolled child, a knowledgeable caregiver completed a detailed telephone or, in rare instances, an in-person interview that collected comprehensive data about family background, education and household characteristics (“caregiver interview”). If the biologic mother was the respondent (99%), she additionally completed interview questions about her reproductive history and her pregnancy with the enrolled child. The caregiver interview was completed at a median of 1.4 months (interquartile range: 1.0–2.5 months) after enrollment. Specific questions, included; for maternal race, selection from a fixed set of categories (American Indian or Alaska Native, Asian, Black, or African American, Native Hawaiian or Other Pacific Islander, White), maternal ethnicity (Hispanic Yes/No), language usually spoken at home (English, Spanish, or other language), highest year of schooling completed by the mother at the time of the interview, and household income in the past 12 months, collected as categorical variables (Table 1). Child age was categorized as above or equal to 55.7 months and below 55.7 months, which was the mean age for the sample.

Table 1.

Distribution of Sociodemographic Variables for Children Recruited from Clinical and Educational Sources Serving Children Who Have ASD or Other Developmental Disorders (CE) and Children Recruited from Birth Certificates to Comprise a General Population Comparison Group (BC)

	CE				BC			All participants
	N	Percent	Meana	SD	N	Percent	Meana	SD	N	Percent	Meana	SD
Child sex
Male	1203	72.8	12.28	7.43	487	53.9	4.78	3.89	1690	66.1	10.11	7.43
Female	450	27.2	9.82	7.29	417	46.1	4.02	3.60	867	33.9	7.03	6.50
Child age
Less than 55.7 mos.	665	40.2	12.19	7.68	395	43.7	4.47	3.56	1060	41.5	9.32	7.46
> = 55.7 mos.	988	59.8	11.21	7.30	509	56.3	4.39	3.93	1497	58.5	8.89	7.13
Household income
<$30,000	373	23.5	14.77	6.98	125	14.3	7.48	5.00	498	20.2	12.94	7.26
$30,000–$70,000	434	27.4	12.03	7.41	186	21.2	4.61	3.57	620	25.2	9.81	7.34
$70,000–$110,000	401	25.3	10.01	7.21	251	28.7	3.97	3.36	652	26.5	7.69	6.71
$110,000+	376	23.7	9.73	7.35	314	35.8	3.43	2.96	690	28.0	6.87	6.58
Maternal education
High school or less	297	18.0	13.76	6.92	85	9.4	7.27	5.50	382	15.0	12.32	7.16
Some college	472	28.6	12.85	7.21	222	24.6	5.45	4.15	694	27.2	10.48	7.26
Bachelor's degree	501	30.4	10.76	7.49	329	36.4	4.05	3.19	830	32.5	8.10	6.97
Master's or greater	379	23.0	9.44	7.41	267	29.6	3.16	2.58	646	25.3	6.84	6.67
Primary language
Spanish	81	4.9	12.26	6.04	16	1.8	6.06	3.23	97	3.8	11.24	6.12
Not Spanish	1572	95.1	11.57	7.54	888	98.2	4.40	3.78	2460	96.2	8.98	7.30
Maternal race
Black	321	20.6	13.69	6.65	109	12.5	6.70	4.22	430	17.7	11.92	6.84
Other	184	11.8	12.21	7.63	88	10.1	5.10	4.40	272	11.2	9.91	7.52
White	1056	67.6	10.85	7.64	676	77.4	3.93	3.48	1732	71.2	8.15	7.19
Maternal ethnicity
Hispanic	216	13.1	12.38	6.70	79	8.7	5.16	3.52	295	11.5	10.45	6.81
Not-Hispanic	1437	86.9	11.49	7.58	825	91.3	4.36	3.79	2262	88.5	8.89	7.31

^aUntransformed SCQ total scores.

ASD, autism spectrum disorder; mos., months.

Data Analysis

Analyses conducted for this study were limited to children who received a final study classification of ASD, DD, or POP, whose caregiver responded to the caregiver interview and completed the clinic visit. Among these families, data were missing on maternal characteristics for <2% of children (percent of missing data ranged from 0.1 to 1.3% for the different characteristics measured), and on the household for <5% (1.4–4.8%). Analyses were performed using participants with complete data. The internal consistency of the SCQ was evaluated using Cronbach’s alpha, which indicates how closely the SCQ’s items are related.

In the analyses, all predictors were treated as categorical variables, while the SCQ total score was continuous. We used univariate ANOVA to assess the influence of each of the categorical demographic factors on the SCQ score. We were interested in factors that may influence the sensitivity and specificity of the SCQ screener used for our sample. Our sample spans child ages, has varying levels of maternal education and family income, includes different races and ethnicities, and users of Spanish and English as a primary language. These each could theoretically influence a caregiver’s report of behaviors queried by the SCQ. We first assessed the influence of each of the categorical demographic factors on the SCQ score. All predictors were of interest, and were included in the linear regression analyses of the relationship of these predictors to the SCQ scores using SPSS General Linear Model.

Sensitivity and specificity for identifying final classification of ASD were computed separately for the CE and BC groups. Sensitivity and specificity for ASD were also computed for strata defined by demographic factors within the CE group. Only specificity was computed for the BC group because the number of children diagnosed with ASD in that group was too small (n = 18) to allow examination of sensitivity. Also for the CE group, receiver operating characteristic (ROC) curves were computed using a nonparametric approach, stratified by demographic variables to display the impact of the demographic variables on the performance of the SCQ. Differences between areas under the ROC curves (AUC) were also examined across categories of the demographic factors, using STATA roccomp. The AUC is one index of the accuracy of an ROC and can be interpreted as the average value of sensitivity for all the possible values of specificity. A maximum AUC of 1 would mean that the SCQ is perfect in the differentiation between ASD and non-ASD, while a value of 0.5 would indicate it is a random predictor.

To identify alternate cut-points for the SCQ, the Youden’s Index (J) was computed by demographic subgroups within the CE group. The Youden’s Index J statistic summarizes the sensitivity and specificity at multiple cut-points (Hajian-Tilaki 2013). The maximum value of the J statistic was used to select the optimum SCQ cut-point for differentiating between ASD and non-ASD when equal weight was given to sensitivity and specificity.

Analyses were conducted using SPSS version 24 and STATA version 12.

Results

A total of 3,769 index children were enrolled in SEED, of whom 2,557 (67.8%) had a valid SCQ, completed the clinic visit and were classified as ASD or non-ASD. The characteristics of the study samples, including mean SCQ scores for the demographic variables, are presented in Table 1. Mean ages and standard deviations (SD) for the children in the CE and BC groups were 55.8 (7.2) and 55.6 (7.6) months, respectively. Mean total SCQ scores for the CE and BC groups were 11.3 (7.4) and 4.4 (3.7) respectively. Cronbach’s alpha for the SCQ was 0.89 for the CE group and 0.77 for the BC group. For each of the source groups, the SCQ distributions were skewed to the left and non-normal. This was addressed using square-root transformed SCQ total scores in the regression analyses. Although the transformation did not produce a normal distribution for the SCQ, the residuals for the regression of the predictors on the SCQ were found to be normal for both source groups. Multicollinearity also was examined and judged to be low, as VIF was under 1.60 for all predictors, for both groups. Regression analyses, for both groups, found higher SCQ scores were associated with male sex, lower household income, lower maternal education, and Black and Other maternal race (Table 2). For the CE group only, higher SCQ scores were also significantly associated with younger child age (Table 2). Spanish language and Hispanic ethnicity were not statistically significant for either group. These last two variables were not included in subsequent analyses.

Table 2.

Regression Analysis of Predictors on SCQ Scores of Children Recruited from Clinical and Educational Sources and Children Recruited from Birth Certificates

	Clinical and educational						Birth certificate
			CI (95)						CI (95)
	B	SE	Lower	Upper	t	P	B	SE	Lower	Upper	t	P
Child sex
Male	0.455	0.068	0.321	0.589	6.680	<0.001	0.218	0.061	0.099	0.337	3.585	<0.001
Female
Child age
Less than 55.7 mos.	0.209	0.062	0.088	0.331	3.380	<0.001	0.070	0.062	−0.051	0.191	1.135	0.257
> = 55.7 mos.
Household income
<$30,000	0.584	0.113	0.363	0.805	5.180	<0.001	0.539	0.128	0.288	0.789	4.222	<0.001
$30,001–$70,000	0.274	0.089	0.100	0.449	3.080	0.002	0.177	0.089	0.003	0.352	1.991	0.047
$70,001–$110,000	0.024	0.085	−0.143	0.191	0.282	0.778	0.062	0.076	−0.088	0.212	0.815	0.415
$110,000+
Maternal education
High school or less	0.455	0.120	0.219	0.691	3.780	<0.001	0.576	0.140	0.302	0.850	4.128	<0.001
Some college	0.401	0.090	0.223	0.578	4.429	<0.001	0.341	0.094	0.157	0.526	3.635	<0.001
Bachelors	0.230	0.081	0.071	0.389	2.834	0.005	0.198	0.075	0.051	0.345	2.651	0.008
Masters or greater
Primary language
Spanish	−0.425	0.321	−1.056	0.205	−1.324	0.186	−0.321	0.419	−1.143	0.501	−0.766	0.444
Not Spanish
Maternal race
Black	0.228	0.085	0.061	0.395	2.677	0.008	0.295	0.109	0.081	0.510	2.701	0.007
Other	0.242	0.095	0.056	0.428	2.550	0.011	0.234	0.103	0.033	0.435	2.280	0.023
White
Maternal ethnicity
Hispanic	0.235	0.131	−0.022	0.492	1.793	0.073	0.104	0.137	−0.165	0.374	0.760	0.447
Not Hispanic

B, unstandardized coefficient B; SCQ, social communication questionnaire; ASD, autism spectrum disorder; mos., months.

The performance of the SCQ for predicting ASD was examined. At a cut-point of 11 the SCQ sensitivity and specificity in the CE group (n = 1653) were 88% (95% CI [0.86, 0.91]) and 74% (95% CI [0.71, 0.77]) respectively. Sensitivity and specificity by demographic factors for the CE group were also examined (Table 3); specificity was substantially lower for the risk factors identified above (Table 3). In the BC group (n = 904), sensitivity for ASD was 78% (95% CI [0.56, 0.93]) and specificity was 95% (95% CI [0.94, 0.96]). For the BC group, specificity by demographic factors was high ranging from 81% for Black maternal race to 99% for the highest levels of maternal education (Table 3).

Table 3.

Sensitivity and Specificity of SCQ ≥11 for ASD for CE Group and Sensitivity for BC Group by Demographic Factors

	CE group								BC group
		CI (95)			CI (95)					CI (95)
	Sens	Lower	Upper	Spec	Lower	Upper	Total ASD cases	N	Spec	Lower	Upper	Total ASD cases	N
Child sex
Male	88%	0.85	0.91	71%	0.67	0.74	553	1203	94%	0.92	0.96	14	487
Female	89%	0.83	0.94	80%	0.75	0.84	123	450	96%	0.96	0.96	4	417
Child age
Less than 55.7 months	90%	0.86	0.93	76%	0.71	0.80	295	665	95%	0.93	0.97	7	395
> = 55.7 months	87%	0.84	0.91	73%	0.84	0.91	381	988	95%	0.93	0.97	11	509
Household income
<$30,000	93%	0.88	0.96	49%	0.42	0.56	161	373	82%	0.74	0.88	8	125
$30,000–$70,000	92%	0.88	0.95	70%	0.65	0.76	177	434	96%	0.93	0.98	1	186
$70,000–$110,000	89%	0.83	0.93	88%	0.83	0.91	161	401	97%	0.95	0.99	5	251
>$110,000	79%	0.72	0.85	89%	0.84	0.92	158	376	98%	0.96	0.99	4	314
Maternal education					0.84	0.92
High School diploma or less	95%	0.89	0.98	54%	0.47	0.61	111	297	82%	0.72	0.89	3	85
Associate degree or some college	90%	0.86	0.94	63%	0.57	0.69	211	472	91%	0.87	0.95	7	222
Bachelor degree	89%	0.84	0.93	84%	0.80	0.88	204	501	98%	0.95	0.99	6	329
Masters or advanced degree	80%	0.73	0.86	88%	0.83	0.92	147	379	99%	0.99	0.99	2	267
Spanish language
Yes	—	-	-	55%	0.42	0.68	23	81	—	-	-	0	16
No	88%	0.85	0.90	75%	0.72	0.78	653	1572	95%	0.93	0.96	14	58
Maternal hispanic ethnicity
Yes	92%	0.84	0.96	65%	0.57	0.73	83	216	96%	0.90	0.99	1	79
No	88%	0.85	0.90	75%	0.72	0.78	593	1437	95%	0.93	0.96	17	825
Maternal race
Black	91%	0.85	0.95	52%	0.45	0.59	138	321	81%	0.72	0.87	1	109
Other	89%	0.81	0.94	78%	0.69	0.86	87	184	98%	0.93	1.00	5	88
White	87%	0.84	0.90	80%	0.77	0.83	419	1056	97%	0.96	0.98	12	676

SCQ, social communication questionnaire; CE, clinical and educational sources; BC, birth certificate source; ASD, autism spectrum disorder; Sens, sensitivity; Spec, specificity.

The ROC analyses revealed that the SCQ is about equally accurate for older and younger children in our CE sample (Table 4, Fig. 1). The differences in AUCs were statistically significant for all but child age, although effect sizes were small (Table 4). AUCs were higher for females than males (Table 4, Fig. 2), for higher levels of maternal education (Table 4, Fig. 3), greater income (Table 4, Fig. 4), and higher for White and Multi-racial or Other race than for Black mothers (Table 4; Fig. 5). Based on computation of Youden J, the most common optimum value for the SCQ cut-point in the different demographic factors was 11 (Table 5). Optimal cut-points, higher than 11, were identified for lower levels of education and income, as well as, Black race (Table 5).

Table 4.

Comparison of AUCs by Demographic Variables for CE Group

			CI (95)
Variables	AUC	SE	Lower	Upper	X2	P	Cramer V
Child sex					8.25	0.004	0.071
Male	0.867	0.010	0.847	0.887
Female	0.918	0.014	0.890	0.946
Child age					0.28	0.600	0.013
< 55.7 months	0.888	0.013	0.863	0.913
> = 55.7 months	0.879	0.011	0.858	0.901
Maternal Education					24.59	<0.001	0.122
HS diploma or less	0.850	0.023	0.805	0.895
Associate degree/some college	0.855	0.017	0.822	0.888
Bachelor degree	0.937	0.010	0.917	0.957
Masters or advanced degree	0.900	0.017	0.867	0.933
Maternal race					12.96	0.002	0.091
Black	0.814	0.023	0.768	0.860
Other	0.889	0.026	0.837	0.938
White	0.905	0.009	0.887	0.923
Household income					19.58	<0.001	0.111
<$30,000	0.831	0.021	0.790	0.872
$30,000–$70,000	0.897	0.015	0.868	0.926
$70,000–110,000	0.936	0.012	0.912	0.959
$110,000+	0.912	0.015	0.880	0.941

AUC, Area under ROC; CE, Clinical Educational Source.

Figure 1.

ROC curve for SCQ score in relation to child age.

Figure 2.

ROC curve for SCQ score in relation to child sex.

Figure 3.

ROC curve for SCQ score in relation to maternal education.

Figure 4.

ROC curve for SCQ score in relation to household income.

Figure 5.

ROC curve for SCQ score in relation to maternal race.

Table 5.

Optimal SCQ Cut-Points by Demographic Factors for CE Group

Variables	SCQ score	Sensitivity	1—Specificity	Youden J
Child sex
Male	≥11	0.881	0.292	0.588
Female	≥11	0.894	0.202	0.692
Child age
< 55.7 months	≥11	0.895	0.241	0.654
> = 55.7 months	≥11	0.874	0.275	0.599
Maternal education
HS diploma or less	≥15	0.820	0.242	0.578
Associate degree or some college courses	≥13	0.810	0.249	0.561
Bachelor degree	≥11	0.892	0.158	0.734
Masters or advanced degree	≥11	0.796	0.121	0.675
Maternal race
Black	≥14	0.790	0.290	0.500
Other	≥11	0.874	0.199	0.674
White	≥11	0.885	0.216	0.669
Household income
<$30,000	≥14	0.845	0.325	0.519
	≥16	0.745	0.226	0.519
$30,000–$70,000	≥11	0.921	0.296	0.625
$70,000–110,000	≥11	0.888	0.125	0.763
$110,000+	≥11	0.791	0.115	0.676

SCQ, social communication questionnaire; CE, clinical and educational sources; Sen, sensitivity; Spec, specificity.

Discussion

This study found that child sex, household income, race, and maternal education were associated with SCQ scores in both source groups for the adjusted model. In general, our analyses demonstrated that a cut-point of 11 was optimal for most of the demographic factors examined in these analyses, suggesting that our use of a cut-point of 11, for more extensive evaluation, was appropriate for this study. At a cut-point of 11 the SCQ demonstrated a higher sensitivity for ASD (88%) among children seen for developmental concerns than among children (78%) in the general population. In contrast specificity was lower (74%) for the CE than for BC (95%). These differences reflect groups categorized by a priori ASD risk. This is relevant when considering the target source population in which the screener will be applied in individual studies. These results suggest that for young children the SCQ (at a cut-point of 11) would be more sensitive in detecting ASD when used for children with identified developmental concerns, but more specific in rejecting ASD for children in the general population.

Specificity for the SCQ was consistently higher in the BC group than for the CE group. For the CE group, at a cut-point of 11, specificity was markedly affected by demographic factors, ranging from 49% to 89% and 54% to 88% across categories of household income and maternal education respectively. It is noteworthy that specificity for Black respondents was 52% compared to 80% for Whites. In addition, the SCQ’s accuracy for children in the CE group was affected by family income, maternal education, maternal race, and child sex. Overall, these results suggest that studies recruiting from clinical populations with greater proportions of lower household income, lower maternal education, or Black race may find it advantageous to adopt an SCQ score of more than 11 points in order to reduce inaccurate ASD screens. These results are consistent with evidence that lower maternal education and minority status are associated with higher scores on the Modified Checklist for Autism in Toddlers (MCHAT) and its revision (Khowaja, Hazzard, & Robins, 2015; Windham et al., 2014), and indicate that these demographic groups may be prone to false positive ASD screen results.

In addition, we found that the internal reliability of the SCQ is substantially lower for the BC group (0.77) than for the CE group (0.89). An alpha of 0.77 is noteworthy for being at the low end of acceptable reliability (Cortina, 1993). We are not aware of another study that reported a similarly low alpha for the SCQ, so perhaps our BC group is unique, in some ways, as compared to samples in other studies.

Spanish language did not have a significant impact on the SCQ score. Other studies have suggested that mother’s language may affect the likelihood of children being detected with developmental problems through surveillance (Zuckerman, Boudreau, Lipstein, Kuhlthau, & Perrin, 2009) or of screening positive for ASD (Windham et al., 2014). It seems likely that in our sample, the effects of income and maternal education were more powerful than the language spoken. The small number of Spanish speakers included in this study could also be a factor here. It is not clear why caregivers from low education, low income or with Black race tended to endorse more positive symptoms of ASD on screening tests. It is possible that misinterpretation of the items is responsible for higher scores on these screeners. However, clarifying the wording of items and giving examples did not eliminate this problem in the MCHAT-R (Khowaja et al., 2015). In addition, the desire to be enrolled in the study to obtain payments or clinician evaluation could be contributing factors. Another explanation may be related to the tendency of persons with lower socioeconomic status to have lower rates of participation in scientific studies (Galea & Tracy, 2007). Perhaps low education, low income, and minority families who enter studies, such as this, have a higher threshold for concern about their children’s development and as a result their children have more significant developmental problems than caregivers with similar levels of education and income, who do not participate. Clearly further examination of this issue is needed.

Although higher SCQ cut-points might provide better specificity for those with less education or lower income, a cut-point of 11 was found to maximize sensitivity and specificity for most levels of this study’s variables, which suggests that the cut-point of 11 is likely to have worked very well for most of the sociodemographic groups in our sample. However, for other studies, desired cut-points for each of socioeconomic status stratum may be different based on clinical or other considerations, including the costs of evaluation of false positives. Future studies may therefore choose alternate SCQ cut-points depending upon the research questions, study design, and population to be enrolled. A limitation of our study warrants consideration. This study did not conduct an evaluation for ASD in most children who scored below 11 points on the SCQ. As a consequence, it is likely that specificity was upwardly biased because children who scored below the cut-off were assumed to be correctly identified as not having ASD. However, this limitation is mitigated by the fact that children scoring below 11 on the SCQ received the ASD evaluation if they had a prior report of ASD or were noted by a study clinician to show indications of ASD. As a result, 13% of children with negative SCQs were evaluated for ASD, of whom 12% were identified as having ASD.

Conclusions

The SCQ was found to have low specificity for certain subgroups in the CE group. To the extent that race, household income and maternal education influence the SCQ’s optimal cut-point, future epidemiological studies that use the SCQ to identify participants or to determine group assignment may want to consider how the demographic characteristics of their study population may influence enrollment and thus the performance of the SCQ as a screener in that population. Although generally shown to be a good choice across demographic subgroups in this study, the SCQ cut-point of 11 was associated with poor specificity for families with lower income and education than for families with greater income and higher levels of education. That said, the cut-point of 11 showed satisfactory sensitivity, for the study’s two subgroups, and was likely a good choice to screen for ASD in our sample.

Lay Summary.

Screeners are used to help identify children who are more likely to have ASD than their peers. Ideally screeners should be accurate for different groups of children and families. This study examined how well the Social Communication Questionnaire (SCQ) predicts ASD. We found that male sex, lower household income, lower maternal education and Black race were associated with higher SCQ scores. In this study an SCQ cut-point of 11 worked best across the different sociodemographic groups in our sample.