Skip to main content
Proceedings of the National Academy of Sciences of the United States of America logoLink to Proceedings of the National Academy of Sciences of the United States of America
. 2020 Apr 27;117(19):10609–10613. doi: 10.1073/pnas.1919050117

Neonatal CSF vasopressin concentration predicts later medical record diagnoses of autism spectrum disorder

Ozge Oztan a, Joseph P Garner a,b, John N Constantino c,d,1, Karen J Parker a,1,2
PMCID: PMC7229671  PMID: 32341146

Significance

Autism is a poorly understood brain disorder of childhood onset. It is characterized by social impairments and diagnosed behaviorally. Early diagnosis fosters better developmental outcomes through behavioral intervention. However, long clinic wait times to undergo behavioral evaluation contribute to delayed diagnoses and treatments. Emerging evidence indicates that low cerebrospinal fluid concentration of the “social” neuropeptide vasopressin is a marker of social impairment in children with autism. Here we report that cerebrospinal fluid vasopressin concentration is significantly lower in newborns diagnosed with autism later in childhood. These preliminary findings suggest that a biomarker of autism may be present before behavioral symptoms emerge. If replicated, this approach could be useful for assessing autism risk and facilitating early intervention in high-risk individuals.

Keywords: autism, vasopressin, cerebrospinal fluid, social, oxytocin

Abstract

Autism spectrum disorder (ASD) is a brain disorder characterized by social impairments. ASD is currently diagnosed on the basis of behavioral criteria because no robust biomarkers have been identified. However, we recently found that cerebrospinal fluid (CSF) concentration of the “social” neuropeptide arginine vasopressin (AVP) is significantly lower in pediatric ASD cases vs. controls. As an initial step in establishing the direction of causation for this association, we capitalized upon a rare biomaterials collection of newborn CSF samples to conduct a quasi-prospective test of whether this association held before the developmental period when ASD first manifests. CSF samples had been collected in the course of medical care of 0- to 3-mo-old febrile infants (n = 913) and subsequently archived at −70 °C. We identified a subset of CSF samples from individuals later diagnosed with ASD, matched them 1:2 with appropriate controls (n = 33 total), and quantified their AVP and oxytocin (OXT) concentrations. Neonatal CSF AVP concentrations were significantly lower among ASD cases than controls and individually predicted case status, with highest precision when cases with comorbid attention-deficit/hyperactivity disorder were removed from the analysis. The associations were specific to AVP, as ASD cases and controls did not differ in neonatal CSF concentrations of the structurally related neuropeptide, OXT. These preliminary findings suggest that a neurochemical marker of ASD may be present very early in life, and if replicated in a larger, prospective study, this approach could transform how ASD is detected, both in behaviorally symptomatic children, and in infants at risk for developing it.


Autism spectrum disorder (ASD) is a brain disorder characterized by social interaction impairments and the presence of restricted, repetitive behaviors. ASD is currently diagnosed behaviorally because its disease biology remains poorly understood. Consequently, there are no laboratory-based diagnostic tests to detect, or medications to treat, ASD’s core features. Even in the absence of effective medications, earlier ASD diagnoses foster better developmental outcomes through intensive behavioral intervention (1, 2). Although ASD can be diagnosed reliably at 2 y of age (3), sadly, due to many factors (e.g., long clinic wait times to undergo behavioral evaluation), the average age of an ASD diagnosis in the United States is 4 y, and far older in economically disadvantaged areas that lack medical specialists (4, 5). By the time a child is showing behavioral abnormalities and has received a formal ASD diagnosis, cumulative delays in the early processing of basic social stimuli have contributed to an atypical trajectory of poor social learning and abnormal social skill acquisition that is immensely difficult to overcome. The capability of rapidly detecting ASD based on a patient’s biological markers, when a child’s symptoms emerge—or even more optimistically, before the disorder manifests behaviorally—would revolutionize ASD detection and enable timely intervention.

Our team has recently pioneered ASD biomarker discovery efforts in cerebrospinal fluid (CSF). Most prior ASD biomarker research has been restricted to blood, which requires less invasive collection procedures, but is less representative of brain biochemistry than CSF. We have previously found that CSF [but not blood (6)] concentrations of the “social” neuropeptide arginine vasopressin (AVP) accurately differentiate pediatric ASD cases from controls (aged 1.5 to 19 y), and that autistic children with the lowest CSF AVP concentrations have the greatest social symptom severity (7, 8). Here we conducted a quasi-prospective study to determine whether this neurochemical difference exists before the condition develops, within a pediatric community cohort for whom CSF had been acquired in early infancy. Specifically, this study exploited a one-of-a-kind archival biomaterials collection of frozen “leftover” CSF that had been obtained during standard of care from n = 1,632 infants, aged 0 to 3 mo (n = 913 of whom we could unambiguously link to their electronic medical record) (911). Electronic medical records were reviewed to assemble a matched case-control cohort of patients with and without childhood ASD. Each ASD case was matched to two controls, resulting in n = 11 ASD case-control trios. We then quantified subjects’ CSF AVP and oxytocin (OXT) concentrations. [OXT was included as a “negative control” because it is nearly structurally identical to AVP but has not been found to differ between individuals with and without ASD in our prior CSF-based research (7).]

Results

A flow diagram depicting study participant eligibility assessment and selection is presented in Fig. 1. Two ASD cases and one control did not have sufficient CSF sample volumes to assay. Additionally, for each analysis, matches were excluded if there were no viable biological data from at least one ASD case and one of its controls. Individuals subsequently diagnosed with ASD later in childhood (n = 9) had significantly lower mean neonatal CSF AVP concentrations compared to those who did not later receive an ASD diagnosis (n = 17) (F1, 16 = 7.980; P = 0.0122; partial η2p2] = 33.3%) (Fig. 2A). This effect held when within-match normalized values were used (F1, 24 = 6.856; P = 0.0151; ηp2 = 22.2%) (Fig. 2B), and was even stronger when trios containing individuals with comorbid attention-deficit/hyperactivity disorder (ADHD) were excluded from the analysis (F1, 12 = 55.52; P < 0.0001; ηp2 = 82.2%) (as shown in Fig. 2B). In contrast, mean neonatal CSF OXT concentrations did not differ between infants later diagnosed with ASD (n = 6) and those who were not (n = 12) (F1, 11 = 0.0288; P = 0.8684; ηp2 = 0.3%) (Fig. 2A). The same result was seen when within-match normalized values were used (F1, 16 = 0.3854; P = 0.5435; ηp2 = 2.4%). To assess power, we calculated the least-significant number required to detect each neuropeptide effect size for the higher-powered within-match normalized values analyses. Least-significant number for AVP was 17 subjects (vs. the 26 analyzed), and for OXT was 182 subjects (vs. the 18 analyzed).

Fig. 1.

Fig. 1.

Flow diagram depicting study participant eligibility assessment and selection.

Fig. 2.

Fig. 2.

Neonatal CSF neuropeptide concentrations in individuals subsequently diagnosed with ASD or not. (A) Mean neonatal CSF AVP concentration is lower in ASD cases (n = 9) compared to controls (n = 17), whereas mean CSF OXT concentration does not differ between groups (n = 6 ASD cases, n = 12 controls). (B) Neonatal CSF AVP concentrations (normalized for the mean and SD of each matched trio) are plotted for matched ASD cases and controls (indicated by a vertical line). Matches in which the ASD case also had comorbid ADHD are distinguished on the x axis. (C) Neonatal CSF AVP concentration significantly predicts individual ASD case and control outcomes. ASD cases (blue) plotted above, and controls (orange) plotted beneath, the dashed line (which represents 50% probability) are correctly classified. The four misclassified individuals correspond to one ASD case and one control each from the last two ASD + ADHD trios shown in B, and are shaded in both panels. The three female individuals are plotted as diamonds in B and C.

We next tested the extent to which neonatal CSF neuropeptide concentrations individually predicted ASD outcome. Neonatal CSF AVP concentrations correctly predicted seven of nine ASD cases and 15 of 17 controls (likelihood ratio [LR] χ2 = 6.216; P = 0.0127; range odds ratio [OR] = 0.03769; ηp2 = 45.0%) (Fig. 2C). Notably, the two misclassified individuals were among four trios containing ASD cases with comorbid ADHD. If these trios were excluded from analysis, then neonatal CSF AVP concentration perfectly predicted five of five ASD cases and nine of nine controls (LR χ2 = 18.25; P < 0.0001). This effect was also specific to AVP, as neonatal CSF OXT concentration was unable to predict a subsequent ASD diagnosis in this study cohort (n = 18) (LR χ2 = 0.4298; P = 0.5121; range OR = 0.3922; ηp2 = 6.2%).

Discussion

This study revealed that 0- to 3-mo-old neonates later diagnosed with ASD had significantly lower CSF AVP concentrations compared to those who did not later receive an ASD diagnosis. Neonatal CSF AVP concentration also individually identified ASD cases with high precision in this archival collection. These findings extend our prior CSF AVP research conducted in behaviorally symptomatic children to suggest that this putative ASD biomarker may be present very early in life, before behavioral symptoms emerge. If replicated, assessment of CSF AVP concentration in infants may be useful for identifying those at risk for developing ASD, prioritizing patients for early clinical monitoring, specifying biologically tractable subgroups, and facilitating intervention in high-risk individuals.

The present findings were specific to AVP, as we found no evidence that the nearly structurally identical neuropeptide OXT differed between groups or predicted individual ASD risk. These findings are consistent with those of our previous studies implicating CSF AVP (but not CSF OXT) as a marker of impaired sociality in naturally low-social male rhesus monkeys and in two independent cohorts of ASD-diagnosed children (two-thirds of whom were male) (7, 8). Preclinical research has shown that AVP plays a critical role in regulating prosocial behavior, particularly in male mammals (1215). Given that ASD affects four times as many males as females (5), a better understanding of brain AVP signaling impairments may provide insight into male-biased vulnerability to this social disorder.

It is well established that experimental dysregulation of the brain AVP signaling pathway produces robust social deficits in mice and voles (13, 16). Natural silencing of AVP gene activity from birth is also associated with robust social developmental deficits in Brattleboro rats (17, 18), suggesting that early impairment in brain AVP signaling may similarly contribute to the pathogenesis of social deficits in young humans. However, AVP itself has yet to be implicated as a high confidence ASD risk gene (https://gene-archive.sfari.org/database/human-gene/AVP). It therefore seems more plausible that the AVP signaling pathway may be one of several common biological pathways impacted by the convergence of multiple and diverse ASD susceptibility genes (19). Conceivably, such a phenomenon would explain why 1) CSF AVP concentration is closely linked to behavioral symptom severity in idiopathic ASD patients (7), and 2) AVP treatment improves social abilities in idiopathic ASD patients with polygenic disease burden (20).

The accuracy of low neonatal CSF AVP concentration to correctly identify ASD cases in this study, although robust, was imperfect, as two of nine ASD cases and 2 of 17 controls were misclassified by the analysis. It bears mention that these misclassified individuals were all from the ASD + ADHD trios. Although few detection methods are 100% accurate, it seems likely that in a disorder as clinically heterogeneous as ASD, other as yet unidentified CSF proteins could be incorporated into a multidimensional biomarker panel by which to more sensitively detect ASD and risk for developing it. Such an approach would also guard against misclassification of healthy individuals (given ASD’s low base rate), and could be used to better distinguish ASD from other neuropsychiatric illnesses (e.g., ASD vs. ADHD vs. ASD + ADHD).

There are several limitations of this study that merit comment. First, although our participants were drawn from a large sample (n = 913), our final study cohort was small (n = 33), and ASD diagnoses were ascertained by medical record review. Although there is 93% overlap between community diagnosis and clinical confirmation of an ASD diagnosis (21), it is essential that our preliminary findings be replicated in a larger-scale prospective study, inclusive of gold-standard ASD research diagnostic assessments. We do note here, however, that the recognition rate of ASD in our sample (11 of 913) is in keeping with the population prevalence of ASD at that time, in the same state (Missouri), as monitored by the Centers for Disease Control and Prevention (22). Second, our case-control study design did not accommodate a disease differential, and thus could have unintentionally overestimated the CSF AVP effect in this study. Future work, inclusive of other brain disorders with and without social impairment, will be required to test whether low CSF AVP concentration is indeed specific to ASD, as opposed to a more general “signature” of altered brain development (and, thus, potentially equally predictive of other disorders). Third, our participants were a sample of convenience due to the challenges involved in obtaining CSF from infants for research purposes. No infant in our study was completely healthy; it is therefore possible that their febrile status in some way altered their CSF AVP concentrations, but there is no reason to think that this would have occurred in a systemically different manner between the two groups. Finally, ascertainment bias in participant selection conceivably could have led us to overlook neonates with low CSF AVP concentrations that nevertheless developed typically. We think this is unlikely because we oversampled controls for just this reason, and so the range of control CSF AVP concentrations is most likely a reasonable reflection of the control population as a whole (but one of course that warrants replication).

We note that the decision to disseminate these preliminary results issues from a number of critical scientific considerations. That a previously identified neurochemical marker of ASD may be present very early in life, before behavioral symptoms emerge, is of high scientific significance, particularly given no reliable biomarker of ASD yet exists. Although the relationship between the AVP signaling pathway and ASD is associative, it nevertheless may have profound implications for understanding the cause and potential prevention of this debilitating condition. Finally, human CSF is a difficult-to-acquire biomaterial. To our knowledge, the archival collection of neonatal CSF described in this study—with accompanying opportunity to ascertain patient outcomes through 12 y of age—represents the only prospective sample in the world of its kind, large enough to capture the number of cases represented here, and one that will take a decade to reproduce prospectively. These considerations all drive to the same conclusion: A larger-scale prospective study is now warranted, inclusive of disease differentials and refined behavioral characterization to replicate and extend these preliminary findings. Should this approach continue to hold promise, neonatal assessment of CSF AVP concentration in high-risk individuals may stand to revolutionize how ASD is evaluated and treated, enabling earlier detection, improved diagnostic yield, and reduced time to treatment.

Materials and Methods

Participants.

Eligible study participants included n = 1,632 individuals who, as infants aged 0 to 3 mo, had presented in the St. Louis Children’s Hospital with minor febrile illness and underwent standard-of-care lumbar puncture. CSF samples were centrifuged and placed immediately on ice. “Leftover” CSF aliquots were reserved in lieu of disposal by the clinical diagnostic laboratory and stored at −70 °C thereafter, as described elsewhere (9). Washington University in St. Louis’s Institutional Review Board authorized use of biological materials with a waiver of consent, with additional authorization required to subsequently access medical records pertinent to these samples. For the present study, we obtained a second waiver of consent to perform a medical record review, and approval to send de-identified CSF samples to Stanford University for quantification and statistical analysis (#201805167). Stanford University also approved execution of this study (#53007).

Review of medical records of the patients whose identifying information matched that recorded on the sample labels (n = 913) enabled identification for the present study individuals subsequently diagnosed with ASD or not. For this review, we included children who remained within the St. Louis Children’s Hospital medical record system and had medical record entries, such that the quasi-prospective cohort from which the cases and controls were selected exclusively included children for whom presence or absence of an ICD-9 diagnostic code for what is currently construed as Diagnostic and Statistical Manual of Mental Disorders, 5th edition (DSM-5) (23) Autism Spectrum Disorder (299.0, 299.8) was available. Individuals deemed controls were those who developed typically, and had no documentation of ASD in their medical record through age 12 y. Each ASD case was then matched to two controls on the basis of sex, birth year, and ethnicity, resulting in n = 11 ASD case-control trios (n = 33; n = 3 females, n = 30 males). No children with documented bacterial infections of CSF in the neonatal period (i.e., meningitis) were included in this study. We note that most neonates for whom CSF is obtained in the medical workup of fever are found to have negative CSF cultures, and viral syndromes as causes of fever. The reason for obtaining CSF in the medical workup of neonates with fever is that it can be the only presenting sign of meningitis, which constitutes a medical emergency if the laboratory examination of CSF confirms the diagnosis.

Neonatal CSF Neuropeptide Quantification.

De-identified second-aliquot CSF samples were collected at St. Louis Children’s Hospital between 1995 and 1999. Samples were immediately frozen and archived at −70 °C, where they remained until quantification. CSF samples were subsequently sent to Stanford University where AVP and OXT concentrations were quantified using commercially available enzyme immunoassay kits (Enzo Life Sciences). These kits are highly specific and exclusively recognize AVP and OXT, respectively, and not related peptides (i.e., the AVP cross-reactivity with OXT is <0.001%; the OXT cross-reactivity with AVP is <0.02%). A research scientist without knowledge of the experimental conditions performed sample preparation and neuropeptide quantification using established procedures as previously described (7, 8). Briefly, CSF samples were directly assayed for AVP and OXT and run in duplicate (100 µL per well) with a tunable microplate reader (Molecular Devices) for 96-well format. Given that the “leftover” volume of CSF available for research studies in neonates is often extremely limited, we prioritized conducting the AVP assay first, and the OXT assay second, based on our previous findings (7). This yielded n = 9 ASD cases and n = 17 controls for AVP, and n = 6 ASD cases and n = 12 controls for OXT.

Statistical Analyses.

Data were analyzed using JMP v.14 (SAS Institute). We initially tested for group differences in CSF neuropeptide concentrations using a general linear model (GLM) blocked by matched trio. Statistical quality-control checks from this analysis showed that the within-match variance differed between matched trios, violating a fundamental assumption of GLM: That is, that the predicted error of a data point is independent of its expected value. The optimal solution to this issue is to use weighted-least squares (WLS)-GLM, with the inverse of the within-group variance as the weight. This approach models not just the mean of the matched trio but also the variance (24, 25). We first performed (and report) these analyses. However, this WLS-GLM approach potentially sacrifices power at smaller sample sizes (because error degrees-of-freedom are lost to match) and could not be easily adapted to our planned subsequent logistic regression analyses. Because WLS is essentially equivalent to calculating normalized values within each match, we normalized (z-scored) the biological data for each neuropeptide within each trio and confirmed using GLM that both normalization solutions yielded similar results. We then used the simpler (z-scored) statistical approach for all additional analyses. Group differences in CSF neuropeptide concentrations were next power-tested by figuring least-significant number. As more trios had to be excluded for CSF OXT than AVP due to missing data, CSF OXT and AVP concentrations were tested separately. Effect sizes are reported as ηp2 as this effect size is dimensionless, comparable across analyses, and can be converted into other effect-size measures readily (26). Finally, we used logistic regression models to test whether neonatal CSF neuropeptide concentrations predicted individual probability of a subsequent ASD diagnosis. Effect sizes are reported as range OR, and their equivalent ηp2 following (27). Note that ORs cannot be calculated for perfectly predictive logistic regression models.

Data Availability Statement.

The data reported in this article are available in Dataset S1.

Supplementary Material

Supplementary File
pnas.1919050117.sd01.pdf (61.3KB, pdf)

Acknowledgments

We thank Teddi Gray for assistance with medical record review. This study was supported by the NIH (Grants MH56317 and HD087011) and Stanford’s Department of Psychiatry.

Footnotes

Competing interest statement: The Board of Trustees of the Leland Stanford Junior University filed a patent application related to biological measures studied herein (PCT/US2019/019029 “Methods for diagnosing and determining severity of an autism spectrum disorder”). This patent has not been granted, nor licensed, and no study author is receiving any financial compensation at this time.

This article is a PNAS Direct Submission.

This article contains supporting information online at https://www.pnas.org/lookup/suppl/doi:10.1073/pnas.1919050117/-/DCSupplemental.

References

  • 1.Reichow B., Barton E. E., Boyd B. A., Hume K., Early intensive behavioral intervention (EIBI) for young children with autism spectrum disorders (ASD). Cochrane Database Syst. Rev. 10, CD009260 (2012). [DOI] [PubMed] [Google Scholar]
  • 2.Rogers S. J., Vismara L., “Interventions for infants and toddlers at risk for autism spectrum disorder” in Handbook of Autism and Pervasive Developmental Disorders, Volkmar F. R., Rogers S. J., Paul R., Pelphrey K. A., Eds. (American Cancer Society, ed. 4, 2014). [Google Scholar]
  • 3.Lord C., et al. , Autism from 2 to 9 years of age. Arch. Gen. Psychiatry 63, 694–701 (2006). [DOI] [PubMed] [Google Scholar]
  • 4.Antezana L., Scarpa A., Valdespino A., Albright J., Richey J. A., Rural trends in diagnosis and services for autism spectrum disorder. Front. Psychol. 8, 590 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Baio J., et al. , Prevalence of autism spectrum disorder among children aged 8 years—Autism and developmental disabilities monitoring network, 11 sites, United States, 2014. MMWR Surveill. Summ. 67, 1–23 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Carson D. S., et al. , Arginine vasopressin is a blood-based biomarker of social functioning in children with autism. PLoS One 10, e0132224 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Oztan O., et al. , Cerebrospinal fluid vasopressin and symptom severity in children with autism. Ann. Neurol. 84, 611–615 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Parker K. J., et al. , Arginine vasopressin in cerebrospinal fluid is a marker of sociality in nonhuman primates. Sci. Transl. Med. 10, eaam9100 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Constantino J. N., Murphy D. L., Monoamine metabolites in ‘leftover’ newborn human cerebrospinal fluidA potential resource for biobehavioral research. Psychiatry Res. 65, 129–142 (1996). [DOI] [PubMed] [Google Scholar]
  • 10.Constantino J. N., Morris J. A., Murphy D. L., CSF 5-HIAA and family history of antisocial personality disorder in newborns. Am. J. Psychiatry 154, 1771–1773 (1997). [DOI] [PubMed] [Google Scholar]
  • 11.Constantino J. N., Murphy D. L., Morris J. A., Family psychiatric history, cerebrospinal fluid monoamine metabolites, and temperament in infants. Biol. Psychiatry 45, 626–632 (1999). [DOI] [PubMed] [Google Scholar]
  • 12.Winslow J. T., Hastings N., Carter C. S., Harbaugh C. R., Insel T. R., A role for central vasopressin in pair bonding in monogamous prairie voles. Nature 365, 545–548 (1993). [DOI] [PubMed] [Google Scholar]
  • 13.Donaldson Z. R., Spiegel L., Young L. J., Central vasopressin V1a receptor activation is independently necessary for both partner preference formation and expression in socially monogamous male prairie voles. Behav. Neurosci. 124, 159–163 (2010). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Albers H. E., The regulation of social recognition, social communication and aggression: Vasopressin in the social behavior neural network. Horm. Behav. 61, 283–292 (2012). [DOI] [PubMed] [Google Scholar]
  • 15.Parker K. J., Lee T. M., Central vasopressin administration regulates the onset of facultative paternal behavior in microtus pennsylvanicus (meadow voles). Horm. Behav. 39, 285–294 (2001). [DOI] [PubMed] [Google Scholar]
  • 16.Bielsky I. F., Hu S.-B., Szegda K. L., Westphal H., Young L. J., Profound impairment in social recognition and reduction in anxiety-like behavior in vasopressin V1a receptor knockout mice. Neuropsychopharmacology 29, 483–493 (2004). [DOI] [PubMed] [Google Scholar]
  • 17.Paul M. J., et al. , Atypical social development in vasopressin-deficient Brattleboro rats. eNeuro 3, ENEURO.0150-15.2016 (2016). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Schank J. C., Early locomotor and social effects in vasopressin deficient neonatal rats. Behav. Brain Res. 197, 166–177 (2009). [DOI] [PubMed] [Google Scholar]
  • 19.Voineagu I., et al. , Transcriptomic analysis of autistic brain reveals convergent molecular pathology. Nature 474, 380–384 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Parker K. J., et al. , A randomized placebo-controlled pilot trial shows that intranasal vasopressin improves social deficits in children with autism. Sci. Transl. Med. 11, eaau7356 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Lee H., et al. , Accuracy of phenotyping of autistic children based on Internet implemented parent report. Am. J. Med. Genet. B. Neuropsychiatr. Genet. 153B, 1119–1126 (2010). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Autism and Developmental Disabilities Monitoring Network Surveillance Year 2008 Principal Investigators; Centers for Disease Control and Prevention , Prevalence of autism spectrum disorders–Autism and developmental disabilities monitoring network, 14 sites, United States, 2008. MMWR Surveill. Summ. 61, 1–19 (2012). [PubMed] [Google Scholar]
  • 23.American Psychiatric Association , Diagnostic and Statistical Manual of Mental Disorders (American Psychiatric Association, ed. 5, 2013). [Google Scholar]
  • 24.Neter J., Kutner M. H., Nachtsheim C. J., Wasserman W., Applied Linear Statistical Models (Irwin, Chicago, 1996). [Google Scholar]
  • 25.Littell R. C., Stroup W. W., Freund R. J., SAS for Linear Models, (SAS Institute, 2002). [Google Scholar]
  • 26.Gaskill B. N., Garner J. P., Power to the people: Power, negative results and sample size. J. Am. Assoc. Lab. Anim. Sci. 59, 9–16 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Lenhard W., Lenhard A., Calculation of Effect Sizes (Psychometrica, 2016). [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary File
pnas.1919050117.sd01.pdf (61.3KB, pdf)

Data Availability Statement

The data reported in this article are available in Dataset S1.


Articles from Proceedings of the National Academy of Sciences of the United States of America are provided here courtesy of National Academy of Sciences

RESOURCES