Skip to main content
NCBI home page
Wiley Open Access Collection logoLink to Wiley Open Access Collection
. 2022 Mar 12;15(6):988–995. doi: 10.1002/aur.2704

Assessing the utility of electronic measures as a proxy for cognitive ability

Tess Levy 1,2, Bari Britvan 1,2, Hannah Grosman 3, Ivy Giserman‐Kiss 4, Kristin Meyering 5, Jordana Weissman 1,2, Danielle Halpern 1,2, Jessica Zweifach 1,2, M Pilar Trelles 1,2,6, Jennifer H Foss‐Feig 1, Alexander Kolevzon 1,2,6,7, Stephan J Sanders 8, Elise B Robinson 9,10,11, Joseph D Buxbaum 1,2,7,12,13, Somer Bishop 8, Paige M Siper 1,2,6,✉
PMCID: PMC9314947

INTRODUCTION

Large‐scale genomic studies have been paramount to discovering genes associated with autism spectrum disorder (ASD). The most recent ASD gene discovery study included ~12,000 cases and identified over 100 genes associated with autism (Satterstrom et al., 2020). However, there remains a gap in the ability to systematically obtain critical phenotypic information beyond a categorical diagnosis. The clinical characterization burden, both in time and resources, has hindered the analytic potential of existing genetic databases in which key variables, such as cognition, is inconsistently captured. Additionally, the travel burden for participation in these studies limits sample diversity.

Cognitive ability is one of the strongest indicators of ASD symptom severity and prognosis (Ben‐Itzchak et al., 2014; Flanagan et al., 2012; Harris & Handleman, 2000), and intellectual disability is strongly associated with rare de novo variants (Brkić et al., 2020; Iossifov et al., 2014; O'Roak et al., 2012). Identifying ways to increase feasibility of intellectual quotient (IQ) measurement is highly relevant to the field. While clinician‐administered cognitive assessments are considered the gold‐standard, several “next‐best” options exist that can provide a proxy of cognitive ability and require fewer resources. Abbreviated cognitive assessments, including the Wechsler Abbreviated Scale of Intelligence (WASI‐II; Wechsler, 2011) and the Kaufman Brief Intelligence Test (Kaufman, 1990), are fair alternatives for measuring general intellectual functioning (Canivez et al., 2009; Hays et al., 2002). However, administration still requires an in‐person visit and trained examiner. An electronic delivery platform would increase sample diversity. For example, Simons Foundation Powering Autism Research (SPARK; Feliciano et al., 2018), an autism genetics study, allows families to participate remotely by mailing in saliva samples, however, estimates of cognitive ability are not prospectively captured. Identifying a proxy for cognitive variability via remote assessments would be a notable advance. Additionally, genetic studies often use “trio” analysis by including both parents of the proband, or siblings when both parents are not available. Data on family member controls' cognitive ability is another rarely available, relevant datapoint in analyzing and interpreting genomic data. We aimed to measure the viability of in‐person electronic assessments with the goal of eventually transitioning to remote administration. We assessed the accuracy of two electronic tests, the electronic Peabody Picture Vocabulary Test, Fourth Edition (Dunn & Dunn, 2007) to assess verbal ability and an electronic Visual Reasoning test (Germine et al., 2012) to assess nonverbal ability, in predicting scores on a clinician‐administered cognitive assessment in individuals with ASD and their immediate family members.

METHODS

Participants

Ninety‐seven individuals including probands, parents, and typically developing siblings enrolled in this study. Only participants who completed the WASI‐II and at least one of the electronic measures were included in analyses. Five probands were excluded because they did not obtain basal scores on WASI‐II subtests. These five probands were unable to complete the e‐PPVT‐4, and four of five were unable to complete the VR. The final sample included 92 participants: 19 probands 6–24 years old (10.68 ± 4.9), 46 parents (46.5 ± 7.0), and 27 siblings (11.56 ± 4.8; Table 1). This study was approved by the Mount Sinai Program for the Protection of Human Subjects. Informed consent was obtained for all participants, and assent was obtained when appropriate.

TABLE 1.

Cohort demographics and descriptive statistics

All participants Probands Parents Siblings
Sample size 92 19 46 27
Age, years 28.6 (18.7) 10.7 (4.9) 46.5 (7.0) 11.6 (4.8)
% female 54.3% 39.7% 50% 74%
WASI‐II full‐scale IQ 108.79 (17.3) 90.21 (23.6) 112.37 (11.1) 115.78 (11.1)
WASI‐II verbal comprehension index 109.34 (18.9) 87.74 (23.6) 114.04 (13.1) 116.50 (11.8)
WASI‐II perceptual reasoning index 105.68 (15.8) 93.63 (22.0) 107.30 (10.9) 111.40 (13.7)
e‐PPVT‐4 standard score 102.02 (15.9) 87.58 (23.4) 103.78 (9.9) 109.19 (11.0)
VR standard score 98.16 (14.7) 87.44 (17.0) 100.6 (11.6) 101.2 (14.9)
SRS‐2 Total T‐Score 53.34 (14.9) 76.47 (11.7) 48.80 (8.8) 44.63 (5.9)
Vineland‐II adaptive behavior composite n/a 72.37 (11.4) n/a n/a
ADOS‐2 comparison n/a 6.78 (1.8) n/a n/a
Full cognitive assessment full‐scale IQ n/a 93.56 (24.7) n/a n/a
Full cognitive assessment verbal IQ n/a 94.83 (24.6) n/a n/a
Full cognitive assessment nonverbal IQ n/a 94.71 (29.1) n/a n/a
Open in a new tab

Note: Scores shown as mean (standard deviation).

Abbreviations: e‐PPVT‐4: electronic Peabody Picture Vocabulary Test, Fourth Edition; IQ: Intellectual quotient; Probands: individuals with autism spectrum disorder; VR: Visual Reasoning test; WASI‐II: Wechsler Abbreviated Scales of Intelligence, Second Edition.

Procedure

Participants completed (a) a clinician‐administered WASI‐II (n = 92; Wechsler, 2011), (b) an electronic version of the PPVT‐4 (e‐PPVT‐4; n = 92; Dunn & Dunn, 2007) and (c) an electronic Visual Reasoning test (VR; n = 91) developed by TestMyBrain.org (Chaytor et al., 2021; Germine et al., 2012; Singh et al., 2021). Electronic assessments were completed in the clinic (2017–2019) on a 15‐in. laptop with an external mouse, which was used to select response on both tests. Testing was supervised by a research coordinator who was responsible for setting up the assessments. The coordinator did not assist or prompt in any capacity. The two electronic assessments were completed in the same block of time, with the order alternated for each participant. The WASI‐II full‐scale IQ was calculated based on all four subtests. WASI‐II verbal comprehension and perceptual reasoning (nonverbal) indices were also calculated. The e‐PPVT‐4 is a standardized assessment of receptive vocabulary where individuals were verbally instructed to point to pictures of specific objects, actions, and concepts, among four choices. The e‐PPVT‐4 yields an overall standard score. The VR test, which is made up of matrix reasoning tasks, was chosen for the electronic measure of nonverbal ability; matrix reasoning tasks are among the best measures of fluid and general intelligence (Carroll, 1997). The VR test was designed to be a measure of cognitive functioning, and specifically of attention, perception, cognitive control, and working memory (Passell et al., 2019). It has similar reliability to the WASI‐II Matrix Reasoning test (Cronbach's alpha = 0.77; Passell et al., 2019). Additionally, the VR test correlates with the scholastic aptitude test (SAT) Math (rho = 0.41, n = 1345, 95% CIs [0.37, 0.45]); and to a lesser degree with Verbal (rho = 0.22, n = 1358, 95% CIs [0.17, 0.27]) and Vocabulary (rho = 0.31, n = 10,000, 95% CIs [0.29, 0.33]; Passell et al., 2019). The VR test asked participants to solve visual puzzles by selecting an answer among five options to complete a visual pattern. Z‐scores were calculated via norms provided by TestMyBrain, and then used to calculate standard scores.

In addition to the WASI‐II, probands were administered a full clinician‐administered cognitive assessment based upon age and functioning level; participants received the Wechsler Intelligence Scales for Children, Fifth Edition (n = 13; Wechsler, 2014), Wechsler Adult Intelligence Scales, Fourth Edition (n = 2; Drozdick et al., 2018), Stanford‐Binet Intelligence Scales, Fifth Edition (n = 3; Roid & Pomplun, 2012), or the Mullen Scales of Early Learning (n = 1). The Vineland Adaptive Behavior Scales, Second Edition (Sparrow, 2011; Sparrow et al., 2005) was administered to all probands to assess ID, as deficits in cognitive and adaptive functioning are required for a diagnosis of ID.

Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM‐5) (American Psychiatric Association, 2013) ASD diagnoses were confirmed based on an evaluation with a board‐certified psychiatrist or licensed psychologist, incorporating information from the Autism Diagnostic Observation Schedule, Second Edition (n = 18; Lord et al., 2012) and the Autism Diagnostic Interview‐Revised (n = 8; Lord et al., 1994) or the Social Communication Questionnaire (n = 11; Rutter et al., 2003). All 19 probands had a confirmed DSM‐5 ASD diagnosis. Parents and siblings were screened for ASD using the Social Responsiveness Scale‐2nd Edition (SRS‐2); and none were excluded.

Analysis

Spearman's rank‐order correlation coefficients (ρ) were used to examine the relationship between WASI‐II full‐scale IQ and both the e‐PPVT‐4 and VR. Correlations between the e‐PPVT‐4 and WASI‐II verbal comprehension index and between the VR and WASI‐II perceptual reasoning index were assessed. Mean difference (absolute value) between standard scores of an individuals' e‐PPVT‐4 or VR score with their WASI‐II score was calculated. The proportion of cases in which e‐PPVT‐4 or VR fell within an individual's WASI‐II full‐scale IQ 95% confidence interval was evaluated. Additionally, to assess if scores on the e‐PPVT‐4 or VR predicted classification ranges on the WASI‐II full‐scale IQ, groups were divided based on scores under 85 (below average), scores 85–115 (average), and scores over 115 (above average). Analyses assessed the electronic measures with full‐scale IQ to examine a potential relationship with overall IQ, as well as with the respective verbal/nonverbal index. Analyses were completed on the whole cohort and within family member type.

RESULTS

The WASI‐II and e‐PPVT‐4 were completed by 92 participants, and the VR by 91 participants (Figure 1). Mean scores (Table 1) were significantly lower in probands relative to both parents and siblings on the WASI‐II (p = 0.002, p < 0.001), e‐PPVT‐4 (p = 0.003, p < 0.001), and VR (p = 0.002, p = 0.013). There was no significant difference between parent and sibling scores.

FIGURE 1.

FIGURE 1

Open in a new tab

Results from the WASI‐II, e‐PPVT‐4, and VR for the cohort. Bars represent the average score in the cohort, error bars signify the standard deviation, and individual points represent participant scores. WASI‐II verbal represents the verbal comprehension index; WASI‐II nonverbal represents the percentual reasoning index. All results are measured in standard scores, which have a population mean of 100 and standard deviation of 15. e‐PPVT‐4, Electronic‐Peabody Picture Vocabulary Test, Fourth Edition; IQ: Intellectual quotient; WASI‐II: Wechsler abbreviated scale of intelligence, second edition

Correlations between WASI‐II and electronic assessments

WASI‐II full‐scale IQ was strongly correlated with both e‐PPVT‐4 (ρ = 0.712, p < 0.001) and VR (ρ = 0.630, p < 0.001; Table 2, Figure 2a). The e‐PPVT‐4 was strongly correlated with the WASI‐II verbal comprehension index (ρ = 0.685, p < 0.001) and the VR was strongly correlated with the WASI‐II perceptual reasoning index (ρ = 0.608, p < 0.001).

TABLE 2.

Spearman's correlations

Assessments All Probands Parents Siblings
e‐PPVT‐4 and WASI‐II full‐scale IQ 0.712** 0.756** 0.645** 0.436*
e‐PPVT‐4 and WASI‐II verbal comprehension index 0.685** 0.754** 0.585** 0.383*
VR and WASI‐II full‐scale IQ 0.630** 0.730** 0.623** 0.446*
VR and WASI‐II perceptual reasoning index 0.608** 0.722** 0.659** 0.393*
Open in a new tab

Note: Spearman's rank correlation coefficient's (ρ) are displayed.

Abbreviations: e‐PPVT‐4: electronic Peabody Picture Vocabulary Test, Fourth Edition; IQ: Intellectual quotient; Probands: individuals with autism spectrum disorder; VR: Visual Reasoning test; WASI‐II: Wechsler Abbreviated Scales of Intelligence, Second Edition.

FIGURE 2.

FIGURE 2

Open in a new tab

(a) Participant's WASI‐II full‐scale IQ plotted by e‐PPVT‐4 score or VR test score. Blue data points represent probands, orange represent siblings, and red represent parents. The dashed line represents a perfect correlation. Box plot on the X axis represents the mean of the WASI‐II full‐scale IQ for the whole cohort, box plot on the Y axis represents the mean of the e‐PPVT‐4 and VR test, respectively. (b) Mean difference between the e‐PPVT‐4 or VR test with the WASI‐II full‐scale IQ. Dotted horizonal lines represent standard deviations. (c) Proportion of cases where the e‐PPVT‐4 or VR test correctly classified WASI‐II full‐scale IQ. Below average are standard scores ≤85, average are scores between 86 and 114, and above average are ≥115. e‐PPVT‐4, Electronic‐Peabody Picture Vocabulary Test, Fourth Edition; IQ: Intellectual quotient; WASI‐II: Wechsler abbreviated scale of intelligence, second edition

Mean difference between WASI‐II and electronic measures

A comparison of the discrepancy in standard scores between the e‐PPVT‐4 and VR with the WASI‐II were calculated for each participant. The mean difference between e‐PPVT‐4 and full‐scale IQ was 10.7, and the mean difference for the VR and full‐scale IQ was 13.4, where both electronic measures tended to yield lower scores than in‐person testing (Table 3, Figure 2b).

TABLE 3.

Mean difference and classification accuracy by 95% CI and range

Calculation All Probands Parents Siblings
Mean difference e‐PPVT‐4 and WASI‐II full‐scale IQ 10.7 (6.9) 13.3 (7.9) 10.1 (6.1) 10.0 (7.2)
e‐PPVT‐4 and WASI‐II verbal comprehension index 11.6 (7.8) 11.1 (7.6) 11.9 (8.2) 11.6 (7.5)
VR and WASI‐II full‐scale IQ 13.4 (10.1) 11.5 (9.2) 12.7 (9.1) 15.8 (12.0)
VR and WASI‐II perceptual reasoning index 11.4 (9.2) 12.0 (9.3) 9.0 (7.3) 15.0 (11.0)

95% confidence interval classification accuracy

 e‐PPVT‐4 within WASI‐II full‐scale IQ 30.4% 21.1% 32.6% 33.3%
e‐PPVT‐4 within WASI‐II verbal comprehension index 35.9% 31.6% 37.0% 37.0%
VR within WASI‐II full‐scale IQ 27.5% 38.9% 23.9% 25.9%
VR within WASI‐II perceptual reasoning index 47.3% 44.4% 52.2% 40.7%
Score range classification accuracy e‐PPVT‐4 below average 71% 83% 0% N/A
e‐PPVT‐4 average 57% 50% 59% 56%
e‐PPVT‐4 above average 69% 100% 67% 67%
VR below average 33% 75% 0% 0%
VR average 59% 67% 58% 57%
VR above average 86% 100% 75% 100%
Open in a new tab

Note: Mean difference, the absolute value of the difference between the standard scores—displayed as Mean (Standard deviation). 95% Confidence Interval, displayed as the proportion of times an individual's e‐PPVT‐4 and VR standard score was within their WASI‐II 95% confidence interval. Classification accuracy, displayed as the proportion of times an individual's e‐PPVT‐4 and VR classification (below average, average, above average) predicted their WASI‐II classification.

Abbreviations: e‐PPVT‐4: electronic Peabody Picture Vocabulary Test, Fourth Edition; IQ: Intellectual quotient; Probands: individuals with autism spectrum disorder; VR: Visual Reasoning test; WASI‐II: Wechsler Abbreviated Scales of Intelligence, Second Edition.

95% confidence interval accuracy

The proportion of cases that e‐PPVT‐4 and VR scores accurately classified based on WASI‐II 95% confidence intervals are displayed in Table 3. Participant's e‐PPVT‐4 and VR score were in the WASI‐II full‐scale IQ 95% confidence interval 30.4% and 27.5% of cases, respectively.

Score range classification accuracy

The e‐PPVT‐4 correctly classified WASI‐II full‐scale IQ score ranges in 71% of below average scores, 57% of average scores, and 69% of above average scores (Table 3, Figure 2c). The VR correctly classified WASI‐II scores in 33% of below average, 59% of average, and 86% of above average cases. Classification accuracy by participant group is displayed in Table 3.

DISCUSSION

This study explored the potential of measuring cognition using electronic assessments. Two computer‐based measures, the e‐PPVT‐4, a measure of receptive vocabulary, and the VR test, a measure of attention, perception, cognitive control, and working memory, were administered to individuals with autism and their immediate family members and compared to clinician‐administered IQ tests. The e‐PPVT‐4 and VR showed moderate to strong correlations with full‐scale IQ on an abbreviated measure of intellectual functioning (WASI‐II), and with verbal and perceptual reasoning indices, respectively, suggesting measurement of similar constructs. This highlights the utility of these measures in large scale genomic studies. However, scores on electronic measures were nearly one standard deviation below WASI‐II scores and only fell within the WASI‐II 95% confidence interval less than a third of the time, indicating poor precision of measurement. The e‐PPVT‐4 better predicted below average scores (71%) compared to the VR test (33%) and both tests similarly predicted IQs within the average range (57%–59%). Results indicate the electronic measures should not be used clinically as a proxy for IQ.

With the goal of increasing available data for genomic studies, inclusion of information from profoundly affected individuals is critical. However, 20% of probands (5/24) were unable to complete the electronic measures due to functional capacity; all these participants received a diagnosis of intellectual disability. If other factors are ruled out (e.g., attention, motor), incomplete results may be a useful indicator of low cognitive ability. Correlations from parents and siblings were weaker than those for probands, likely in part due to the lower variability of scores within groups.

Overall, correlations indicate these electronic assessments adequately relate to functioning on the WASI‐II at the group level; however, results lack the precision needed to estimate scores at the individual level. This is consistent with prior literature illustrating poor predictive ability between various IQ measures, despite significant correlations (Bell et al., 2001). Overall, there was not a clear difference in performance between the e‐PPVT‐4 and VR test; however, the e‐PPVT‐4 tended to better capture the more severely affected individuals, which is an important consideration. Future studies with larger and more variable samples can further evaluate these electronic assessments.

LIMITATIONS

Our sample size was modest, and therefore does not capture the extent of the heterogeneity in real‐world ASD populations. Specifically, while a larger proportion (38%) of probands in our originally recruited cohort had ID, many were unable (five of nine individuals with ID) to complete electronic assessments, leaving the resulting cohort with underrepresentation of ID (21% ID compared to 33% expected [Baio et al., 2018; Shaw et al., 2020]). Additionally, electronic measures were completed in the clinic under controlled conditions and results may not generalize to remote administration.

CONCLUSION

Electronic assessments show strong correlations with IQ, however, poor precision in estimating absolute IQ scores. Nevertheless, electronic assessments may provide a useful means to fill gaps within existing autism databases and enhance future large‐scale genomic research in the field. In contrast, our results indicate these measures should not be used as a proxy for clinical assessment of individual IQ scores.

CONFLICT OF INTEREST

Alexander Kolevzon receives research supports from AMO Pharma and consults to Acadia, Alkermes, Neuren, and GW Pharma. He serves on Scientific Advisory Boards for Ovid Therapeutics, Jaguar Therapeutics, and Ritrova Therapeutics. Joseph D. Buxbaum consults to BridgeBio, holds a patent for IGF‐1 in Phelan‐McDermid syndrome, holds an honorary professorship from Aarhus University Denmark, receives research support from Takeda and Oryzon and is a journal editor for Springer Nature.

AUTHOR CONTRIBUTIONS

Somer Bishop, Stephan J. Sanders, Elise B. Robinson, Joseph D. Buxbaum, and Paige M. Siper were responsible for the conception and design of the study. Paige M. Siper, Ivy Giserman‐Kiss, Danielle Halpern, Jessica Zweifach, Maria del Pilar Trelles, Hannah Grosman, Kristin Meyering, Jordana Weissman, Alexander Kolevzon, and Jennifer H. Foss‐Feig acquired the data. Tess Levy, Bari Britvan, Paige M. Siper, Somer Bishop, Hannah Grosman, Ivy Giserman‐Kiss, and Jennifer H. Foss‐Feig participated in the analysis and/or interpretation of data. Tess Levy, Bari Britvan, and Paige M. Siper drafted the manuscript. Somer Bishop, Stephan J. Sanders, Elise B. Robinson, Jennifer H. Foss‐Feig, Alexander Kolevzon, Joseph D. Buxbaum, Paige M. Siper, and Tess Levy were responsible for revising the manuscript for critically for important intellectual content. All authors have approved the version of the manuscript to be published.

ETHICS STATEMENT

All participants or their legal guardians provided informed consent prior to study participation. Assent from minors was collected when appropriate. The study was approved by the Mount Sinai Program for the Protection of Human Subjects (Study ID: 16‐0494).

ACKNOWLEDGMENTS

The authors would like to thank all of the families who generously participated in this study.

Levy, T. , Britvan, B. , Grosman, H. , Giserman‐Kiss, I. , Meyering, K. , Weissman, J. , Halpern, D. , Zweifach, J. , Trelles, M. P. , Foss‐Feig, J. H. , Kolevzon, A. , Sanders, S. J. , Robinson, E. B. , Buxbaum, J. D. , Bishop, S. , & Siper, P. M. (2022). Assessing the utility of electronic measures as a proxy for cognitive ability. Autism Research, 15(6), 988–995. 10.1002/aur.2704

Funding information Autism Science Foundation; Hilibrand Family Foundation

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.

REFERENCES

  1. American Psychiatric Association . (2013). Diagnostic and statistical manual of mental disorders (DSM‐5®). American Psychiatric Publication. [Google Scholar]
  2. Baio, J. , Wiggins, L. , Christensen, D. L. , Maenner, M. J. , Daniels, J. , Warren, Z. , Kurzius‐Spencer, M. , Zahorodny, W. , Robinson Rosenberg, C. , White, T. , Durkin, M. S. , Imm, P. , Nikolaou, L. , Yeargin‐Allsopp, M. , Lee, L. C. , Harrington, R. , Lopez, M. , Fitzgerald, R. T. , Hewitt, A. , … Dowling, N. F. (2018). Prevalence of autism Spectrum disorder among children aged 8 years ‐ autism and developmental disabilities monitoring network, 11 sites, United States, 2014. Morbidity and Mortality Weekly Report, 67(6), 1–23. 10.15585/mmwr.ss6706a1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bell, N. L. , Lassiter, K. S. , Matthews, T. D. , & Hutchinson, M. B. (2001). Comparison of the Peabody picture vocabulary test‐third edition and Wechsler adult intelligence scale‐third edition with university students. Journal of Clinical Psychology, 57(3), 417–422. 10.1002/jclp.1024 [DOI] [PubMed] [Google Scholar]
  4. Ben‐Itzchak, E. , Watson, L. R. , & Zachor, D. A. (2014). Cognitive ability is associated with different outcome trajectories in autism spectrum disorders. Journal of Autism and Developmental Disorders, 44(9), 2221–2229. 10.1007/s10803-014-2091-0 [DOI] [PubMed] [Google Scholar]
  5. Brkić, D. , Ng‐Cordell, E. , O'Brien, S. , Scerif, G. , Astle, D. , & Baker, K. (2020). Gene functional networks and autism spectrum characteristics in young people with intellectual disability: A dimensional phenotyping study. Molecular Autism, 11(1), 98. 10.1186/s13229-020-00403-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Canivez, G. L. , Konold, T. R. , Collins, J. M. , & Wilson, G. (2009). Construct validity of the Wechsler abbreviated scale of intelligence and wide range intelligence test: Convergent and structural validity. School Psychology Quarterly, 24(4), 252–265. 10.1037/a0018030 [DOI] [Google Scholar]
  7. Carroll, J. B. (1997). Psychometrics, intelligence, and public perception. Intelligence, 24(1), 25–52. [Google Scholar]
  8. Chaytor, N. S. , Barbosa‐Leiker, C. , Germine, L. T. , Fonseca, L. M. , McPherson, S. M. , & Tuttle, K. R. (2021). Construct validity, ecological validity and acceptance of self‐administered online neuropsychological assessment in adults. The Clinical Neuropsychologist, 35(1), 148–164. 10.1080/13854046.2020.1811893 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Drozdick, L. W. , Raiford, S. E. , Wahlstrom, D. , & Weiss, L. G. (2018). Flanagan, In D. P. & McDonough (Eds.) E. M., The Wechsler Adult Intelligence Scale—Fourth Edition and the Wechsler Memory Scale—Fourth Edition.. Contemporary intellectual assessment: Theories, tests, and issues, (pp. 486–511). New York, NY: The Guilford Press. [Google Scholar]
  10. Dunn, L. , & Dunn, D. (2007). Peabody picture vocabulary test (4th ed.). NCS Pearson. [Google Scholar]
  11. Feliciano, P. , Daniels, A. M. , Green Snyder, L. , Beaumont, A. , Camba, A. , Esler, A. , Gulsrud, A. G., Mason, A., Gutierrez, A., Nicholson, A., Paolicelli, A. M., McKenzie, A. P., Rachubinski, A. L., Stephens, A. N., Simon, A. R., Stedman, A., Shocklee, A. D., Swanson, A., … Chung, W. K. (2018). SPARK: A US cohort of 50,000 families to accelerate autism research. Neuron, 97(3), 488–493. 10.1016/j.neuron.2018.01.015 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Flanagan, H. E. , Perry, A. , & Freeman, N. L. (2012). Effectiveness of large‐scale community‐based intensive behavioral intervention: A waitlist comparison study exploring outcomes and predictors. Research in Autism Spectrum Disorder, 6(2), 673–682. 10.1016/j.rasd.2011.09.011 [DOI] [Google Scholar]
  13. Germine, L. , Nakayama, K. , Duchaine, B. C. , Chabris, C. F. , Chatterjee, G. , & Wilmer, J. B. (2012). Is the web as good as the lab? Comparable performance from Web and lab in cognitive/perceptual experiments. Psychonomic Bulletin & Review, 19(5), 847–857. 10.3758/s13423-012-0296-9 [DOI] [PubMed] [Google Scholar]
  14. Harris, S. L. , & Handleman, J. S. (2000). Age and IQ at intake as predictors of placement for young children with autism: A four‐ to six‐year follow‐up. Journal of Autism and Developmental Disorders, 30(2), 137–142. 10.1023/a:1005459606120 [DOI] [PubMed] [Google Scholar]
  15. Hays, J. R. , Reas, D. L. , & Shaw, J. B. (2002). Concurrent validity of the Wechsler abbreviated scale of intelligence and the Kaufman brief intelligence test among psychiatric inpatients. Psychological Reports, 90(2), 355–359. 10.2466/pr0.2002.90.2.355 [DOI] [PubMed] [Google Scholar]
  16. Iossifov, I. , O'Roak, B. J. , Sanders, S. J. , Ronemus, M. , Krumm, N. , Levy, D. , Stessman, H. A. , Witherspoon, K. T. , Vives, L. , Patterson, K. E. , Smith, J. D. , Paeper, B. , Nickerson, D. A. , Dea, J. , Dong, S. , Gonzalez, L. E. , Mandell, J. D. , Mane, S. M. , Murtha, M. T. , … Wigler, M. (2014). The contribution of de novo coding mutations to autism spectrum disorder. Nature, 515(7526), 216–221. 10.1038/nature13908 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Kaufman, A. S. (1990). Kaufman brief intelligence test: KBIT: AGS. American Guidance Service Circle Pines. [Google Scholar]
  18. Lord, C. , Rutter, M. , DiLavore, P. , Risi, S. , Gotham, K. , & Bishop, S. (2012). Autism diagnostic observation schedule–(ADOS‐2) (2nd ed.). Western Psychological Corporation. [Google Scholar]
  19. Lord, C. , Rutter, M. , & Le Couteur, A. (1994). Autism diagnostic interview‐revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism and Developmental Disorders, 24(5), 659–685. 10.1007/bf02172145 [DOI] [PubMed] [Google Scholar]
  20. O'Roak, B. J. , Vives, L. , Girirajan, S. , Karakoc, E. , Krumm, N. , Coe, B. P. , Levy, R. , Ko, A. , Lee, C. , Smith, J. D. , Turner, E. H. , Stanaway, I. B. , Vernot, B. , Malig, M. , Baker, C. , Reilly, B. , Akey, J. M. , Borenstein, E. , Rieder, M. J. , … Eichler, E. E. (2012). Sporadic autism exomes reveal a highly interconnected protein network of de novo mutations. Nature, 485(7397), 246–250. 10.1038/nature10989 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Passell, E. , Dillon, D. G. , Baker, J. T. , Vogel, S. C. , Scheuer, L. S. , Mirin, N. L. , Rutter, L. A. , Pizzagalli, D. A , & Germine, L. (2019). Digital cognitive assessment: Results from the TestMyBrain NIMH Research Domain Criteria (RDoC) field test battery report. [Google Scholar]
  22. Roid, G. H. , & Pomplun, M. (2012). The Stanford‐binet intelligence scales. The Guilford Press. [Google Scholar]
  23. Rutter, M. , Bailey, A. , & Lord, C. (2003). The social communication questionnaire: Manual. Western Psychological Services. [Google Scholar]
  24. Satterstrom, F. K. , Kosmicki, J. A. , Wang, J. , Breen, M. S. , de Rubeis, S. , An, J. Y. , Peng, M. , Collins, R. , Grove, J. , Klei, L. , Stevens, C. , Reichert, J. , Mulhern, M. S. , Artomov, M. , Gerges, S. , Sheppard, B. , Xu, X. , Bhaduri, A. , Norman, U. , … Buxbaum, J. D. (2020). Large‐scale exome sequencing study implicates both developmental and functional changes in the neurobiology of autism. Cell, 180(3), 568–584.e23. 10.1016/j.cell.2019.12.036 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Shaw, K. A. , Maenner, M. J. , Baio, J. , Washington, A. , Christensen, D. L. , Wiggins, L. D. , Pettygrove, S. , Andrews, J. G. , White, T. , Rosenberg, C. R. , Constantino, J. N. , Fitzgerald, R. T. , Zahorodny, W. , Shenouda, J. , Daniels, J. L. , Salinas, A. , Durkin, M. S. , & Dietz, P. M. (2020). Early identification of autism spectrum disorder among children aged 4 years—early autism and developmental disabilities monitoring network, six sites, United States, 2016. Morbidity and Mortality Weekly Report, 69(3), 1–11. 10.15585/mmwr.ss6903a1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Singh, S. , Strong, R. W. , Jung, L. , Li, F. H. , Grinspoon, L. , Scheuer, L. S. , Passell, E. J. , Martini, P. , Chaytor, N. , Soble, J. R. , & Germine, L. (2021). The TestMyBrain digital neuropsychology toolkit: Development and psychometric characteristics. Journal of Clinical and Experimental Neuropsychology, 43(8), 786–795. 10.1080/13803395.2021.2002269 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Sparrow, S. S. (2011). Vineland adaptive behavior scales. In Kreutzer J. S., DeLuca J., & Caplan B. (Eds.), Encyclopedia of clinical neuropsychology (pp. 2618–2621). Springer New York. [Google Scholar]
  28. Sparrow, S. S. , Cicchetti, D. V. , & Balla, D. A. (2005). Vineland adaptive behavior scales Vineland‐II: Survey forms manual. Pearson Minneapolis. [Google Scholar]
  29. Wechsler, D. (2011). Wechsler abbreviated scale of intelligence second edition (WASI‐II). Pearson. [Google Scholar]
  30. Wechsler, D. (2014). Wechsler intelligence scale for children: Grundlagen, Testauswertung und Interpretation/[Übers. durch Pearson Assessment & Information GmbH, Frankfurt/M.]. Manual 1 (5th ed.). London: Pearson. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.

Articles from Autism Research are provided here courtesy of Wiley

RESOURCES