Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Jun 7.
Published in final edited form as: Dev Psychopathol. 2021 Dec 7;33(5):1665–1684. doi: 10.1017/s0954579421000651

Translating RDoC to Real-World Impact in Developmental Psychopathology: A Neurodevelopmental Framework for Application of Mental Health Risk Calculators

Leigha A MacNeill 1,2, Norrina B Allen 2,3, Roshaye B Poleon 2, Teresa Vargas 4, K Juston Osborne 4, Katherine S F Damme 4, Deanna M Barch 5,6,7, Sheila Krogh-Jespersen 1,2, Ashley N Nielsen 8, Elizabeth S Norton 1,2,9, Christopher D Smyser 7,8,10, Cynthia E Rogers 6,10, Joan L Luby 6, Vijay A Mittal 1,2,4,11,12, Lauren S Wakschlag 1,2
PMCID: PMC8794223  NIHMSID: NIHMS1709646  PMID: 35095215

Abstract

The National Institute of Mental Health Research Domain Criteria’s (RDoC) has prompted a paradigm shift from categorical psychiatric disorders to considering multiple levels of vulnerability for probabilistic risk of disorder. However, the lack of neurodevelopmentally-based tools for clinical decision-making has limited RDoC’s real-world impact. Integration with developmental psychopathology principles and statistical methods actualize the clinical implementation of RDoC to inform neurodevelopmental risk. In this conceptual paper, we introduce the probabilistic mental health risk calculator as an innovation for such translation and lay out a research agenda for generating an RDoC- and developmentally-informed paradigm that could be applied to predict a range of developmental psychopathologies from early childhood to young adulthood. We discuss methods that weigh the incremental utility for prediction based on intensity and burden of assessment, the addition of developmental change patterns, considerations for assessing outcomes, and integrative data approaches. Throughout, we illustrate the risk calculator approach with different neurodevelopmental pathways and phenotypes. Finally, we discuss real-world implementation of these methods for improving early identification and prevention of developmental psychopathology. We propose that mental health risk calculators can build a needed bridge between RDoC’s multiple units of analysis and developmental science.

Keywords: risk calculator, developmental change, psychopathology, RDoC, prevention/intervention

Introduction

One in 6 children between the ages of 2 and 8 have a diagnosed mental, behavioral, or developmental disorder (Cree et al., 2018), and 73.9% of young adults with a psychiatric disorder received their diagnosis before age 18 (Kim-Cohen et al., 2003). Despite the high prevalence of mental health problems and their early onset, the field of developmental psychopathology is still plagued by the need for a science of “when to worry” (Wakschlag et al., 2019). Although dimensional approaches to studying psychopathology are beneficial for characterizing clinical phenomenology (Sroufe, 1990), there is a gap in identifying thresholds that denote which individuals to target for prevention or treatment and when. On one hand, most children who exhibit problem behavior in early development do not go on to develop a mental health disorder. On the other hand, many children with severe, chronic psychopathology exhibit vulnerability and/or symptoms in early childhood (e.g., conduct problems; Shaw & Taraban, 2016). For example, elevated preschool irritability (a transdiagnostic developmental indicator of lifespan psychopathology risk; Wakschlag et al., 2018) increases risk of preadolescent internalizing and externalizing psychopathology more than sevenfold (Wiggins et al., under review). Yet, routine care often fails to detect children who show such precursors of psychopathology and even fewer are identified for treatment (Sheldrick, Merchant, & Perrin, 2011).

The National Institute of Mental Health’s Research Domain Criteria (RDoC) is a heuristic framework designed to characterize psychopathology by integrating domains of functioning that cut across multiple levels of analysis for identifying mental health problems. This goal is well aligned with the rationale of longitudinal developmental research put forth by Baltes and Nesselroade (1979), in which within-person change in a construct co-occurs with changes in other constructs. Thus, RDoC domains may be a useful framework for how developmental psychopathologists can begin to determine which constructs can be used for predicting disorder. However, the lack of developmentally-based tools specifically designed for decision-making (e.g., when to refer) has been one key limitation of RDoC’s impact on developmental disorders in real-world settings. Reliably differentiating which vulnerable children and youth have an increased likelihood of developing a disorder from those who will have natural course remission or develop adaptively due to environmental supports is critical for designing programs for prevention/intervention. Well-validated, developmentally-based, and generalizable clinical supports are therefore essential for detecting probabilistic risk of psychopathology, particularly at critical or sensitive periods of development during which brain and behavior are most malleable.

The development of a risk assessment tool for future pragmatic clinical and research use may be an optimal method for early prediction of mental disorders. Clinical risk calculators have the potential to generate both categorical risk classification as well as a personalized estimate of one’s probability of developing a mental health disorder from a set of risk factors. Estimating an individual’s risk for developing a disease is a hallmark of personalized medicine. Risk calculators for predicting physical health outcomes can support a clinician’s reasoning for the intensity of the prevention/intervention strategy needed based on a personalized risk estimate for an individual patient (Cannon et al., 2016). The Framingham risk calculator is one of the most widely used risk calculators for prevention of cardiovascular disease, which has optimized precision medicine. It has transformed the standard of care in cardiovascular disease by leveraging the lowest burden (e.g., most available, lowest cost, most parsimonious) risk indices for risk prediction (Pencina & D’Agostino, 2012). Although risk calculators have become a popular risk estimation tool in physical health domains, they have been under-utilized in mental health (Bernardini et al., 2017; Wakschlag et al., under review). The mental health field could greatly benefit from this approach, given the robust support for the efficacy of prevention programs during early and/or vulnerable periods of development (Bernardini et al., 2017; Dawson, Ashman, & Carver, 2000; Luby et al., 2019).

Risk calculators must be both sensitive (i.e., identifying individuals with psychopathology as positive) and specific (i.e., identifying individuals without psychopathology as negative). A key goal of the mental health risk calculator is to identify the probabilistic risk that an individual will have impairing psychopathology or convert to a disorder from a high-risk state. We focus on probabilistic risk specifically, rather than absolute certainty, given the typical variation in developmental populations (Wakschlag et al., 2019). Probabilistic risk prediction models estimate the probability that individuals will develop a disorder at a future time point by using a set of pre-defined risk indices. These probabilistic models are critical to providing estimates of future risk among patients for whom the future occurrence of the outcome is unknown at the time of clinical decision-making. Each individual is assigned (a) an estimated probability of developing a disorder over a length of time based on these set of risk indices, and (b) a categorical risk classification to be used for clinical decision-making (Pencina & D’Agostino, 2012).

Neurodevelopmental vulnerability indicators (i.e., brain and behavior that underlie human capacity to engage in, manage, and adapt to everyday experiences that drive daily health and functioning) are transdiagnostic, and therefore, may make key targets for capturing the heterogeneous emergence of developmental psychopathology (Rogers et al., 2017; Wakschlag et al., 2018). But patients, particularly young children, are characteristically difficult to assess, and measures of brain structure and function have not been proven clinically useful in mental disorders to date, in part due to non-specificity for individual patients and disorders. Here, probabilistic risk calculators serve as an invaluable bridge between predictors identified in more basic psychopathology studies and a practice of identifying the fewest indicators needed for effective decision making in the context of real-world implementation. Further, such methods could identify those subjects in need of more intensive and costly neuropsychiatric testing and determine their value added for diagnostic prediction. By detecting the constructs and units of analysis requisite for identifying the subset of individuals that will go on to develop clinical disorders, we may one day be able to prevent disorder onset. In turn, we can decrease the immense public health burden of mental health treatment and its social corollaries (McDaid, Park, & Wahlbeck, 2019).

By embedding development in risk calculators, and leveraging the methodological rigor of the RDoC framework, we can cultivate a population-based strategy for actualizing mental health. We have six primary goals in this conceptual paper. The first goal is to define risk calculator parameters and introduce two risk calculator exemplars in the vulnerable phases of the clinical sequence. Second, we discuss an RDoC-informed approach for selection of neurodevelopmental markers and environmental indicators across multiple units of analysis for risk prediction, accounting for measurement intensity. Specifically, we propose that RDoC’s constructs and their units of analysis can help guide the selection and measurement of risk indicators. Third, we provide strategies for embedding developmental principles in risk calculator designs. Fourth, we discuss considerations for choosing mental health outcomes. Fifth, we offer strategies for generalizing risk prediction, which includes leveraging integrative data strategies. Finally, we discuss how neurodevelopmentally-oriented risk calculators can be used to accelerate clinical translation to mental health disorder prevention.

1. Risk Calculator Parameters and Exemplars in Vulnerable Phases of the Clinical Sequence

A risk model performs well if individuals with the outcome have a higher predicted risk than those who do not. Three key measures of risk model performance are the concordance (c) statistic, discrimination slope, and model calibration. The c-statistic for a binary outcome is the area under the receiver operating characteristic (ROC) curve (AUC) and is the most common statistic for discriminating performance of risk calculators (D’Agostino, Griffith, Schmid, & Terrin, 1997). The AUC provides a value from 0 to 1 that represents the ability of the parameter or risk score to distinguish between having the disorder and not having the disorder (or any binary outcome of interest). An AUC between 0.8 and 1.0 is considered good discrimination performance, 0.7 to 0.8 is moderate, 0.6 to 0.7 is fair, and 0.5 represents no better than chance. The discrimination slope is an index of improvement in model sensitivity and specificity (Pencina et al., 2008). Calibration is a measure of how closely predicted probabilities align with real experience (D’Agostino et al., 1997), such that perfect calibration occurs when predicted and observed risk are equal.

In the field of cardiovascular prevention, clinical guidelines recommend routine cardiovascular risk assessment. The prevention guidelines then include not only absolute levels of clinical risk factors but also a patient’s global risk using the Pooled Cohort Equation (PCE) to guide treatment initiation and intensity. The PCE has consistently demonstrated good discrimination with a c-statistic >0.75 across a broad range of populations. Importantly, the PCE includes basic demographic and clinical risk factor levels that are easily and reliably obtained as part of routine clinical care. In general, more sophisticated and expensive biomarkers or genetic risk scores have added little to the discrimination of the PCE model.

In the mental health domain, risk calculators may be most clinically useful when applied in vulnerable phases of neurodevelopment, during which we see the onset of key behaviors and characteristics as well as heightened interindividual variability. These phases often coincide with the “grow out of it” or “watch and wait” mentalities (Luby, 2012; Wakschlag et al., 2015) that hinder intervention and canalize behavioral tendencies leading to impairment. The Mental Health, Earlier roadmap proposed by Wakschlag and colleagues (2019) aims to reduce the research-to-practice gap by calling for interdisciplinary developmental science to work toward mental health prevention. Specifically, they propose two pillars: Earlier and Healthier. The Earlier pillar presents a shift toward transdiagnostic and probabilistic diagnosis processes to distinguish between normative (and potentially transient) behavior and maladaptive development. The Healthier pillar is grounded on prevention rather than a reactive treatment approach. By determining an individual’s probabilistic risk for psychopathology across multiple levels of analysis, we can better prevent the emergence of disorder. Throughout this report, we discuss the application of the risk calculator approach to two different neurodevelopmental pathways and phenotypes: 1) psychotic syndromes in youth, and 2) common internalizing/externalizing problems in early childhood. We find these to be a useful contrast illustratively because these syndromes have distinct phenomenology and etiologies. Further, the former presents a rare and severe disorder that usually onsets in late adolescence/early adulthood and the latter represents the most common forms of psychopathology that typically onset in early childhood. Although not exhaustive, Table 1 provides an overview of key studies presenting risk algorithms predicting a range of psychopathologies including psychosis, bipolar disorder (see Silva Ribeiro et al., 2020 for review), depression, Attention-Deficit/Hyperactivity Disorder (ADHD), and to a lesser extent, internalizing/externalizing problems.

Table 1.

Examples of Commonly Used and Otherwise Interesting Risk Calculators in Psychiatric & Psychological Research

Study Replication/Validation Age Population Characteristics Risk Indicators & Measurement Method Outcome &
Measurement Method
Findings

Birmaher et al., 2018 None yet 6–17 years with at least 1 follow-up before conversion Youth diagnosed with BP-NOS – the Course and Outcome of Bipolar Youth (COBY) study Demographics & family history by survey/parent or individual report

Duration of BP illness by self-report

Various survey & parent report scales to assess BP symptoms, mood, behavior & functioning
COBY Risk Calculator: Conversion from BP-NOS to BP I/II based on the stated risk indicators 53.6% of COBY youths converted to BP-I/II (76.0% of which converted within 5 years)

The COBY risk calculator predicted risks of conversion that were highly consistent with the observed conversion outcomes

Earlier onset BP-NOS, familial hypomania/mania, high mania, anxiety, & mood lability symptoms were all important predictors of conversion
Cannon et.al, 2008 See Cannon et al. (2016) & Carrión et al. (2016) as example follow-ups to NAPLS project approach to predicting conversion Mean age
18 years across 2.5 years follow-up
Prodromal symptomatic for psychosis – Structured Interview for Prodromal Syndromes (SIPS) verified Structured clinical interview (DSM-IV) to assess demographics, family history, individual mental health history, & psychiatric diagnoses/symptomology

Genetic risk for SZ
Cox proportional hazards model to predict conversion to psychosis by SIPS criteria 35.0% conversion rate to full psychosis

Genetic risk for SZ, high levels of unusual thought content, suspicion/paranoia, social impairment, & a history of substance abuse contributed to psychosis prediction
Cannon et al., 2016 Carrión et.al, 2016

Osborne & Mittal, 2019

Zhang et.al, 2018
12–30 years (mean age 18 years) followed up to 2 years CHR population from the North American Prodrome Longitudinal Study (NAPLS2) Demographics & family history by survey

Behavioral test battery to assess SZ symptomology e.g. unusual thought content & processing

Social functioning, stress & trauma exposure by survey
Conversion to psychosis

Proportional hazards (PH) model to predict likelihood of conversion to psychosis based on risk indicators
16.0% conversion rate to full psychosis over 2-year period

All but three of the risk indicators were found to be significant in predicting conversion to psychosis – stressful life events, traumas, & family history of SZ were not significant predictors
Caye et al., 2020 (Validated in three external cohorts) Assessed at age 7 or 10 & in last available assessment at age 17 Derived from the Avon Longitudinal Study of Parents and Children (ALSPAC). Validated in the Environmental Risk (E-Risk) Longitudinal Twin Study, the 1993 Pelotas Birth Cohort, & the Multimodal Treatment Study of Children with ADHD (MTA) Female sex, socioeconomic status, maternal depression, IQ, maltreatment, psychopathology, & single-parent family measured in childhood via parent & child reports, behavioral assessments, & interviews Logistic regression to predict ADHD measured via parent report (ALSPAC) Prevalence of ADHD ranged from 8.1–12% in population-based samples

AUC predicting adult ADHD of 0.82

Risk model did not predict anxiety or MDD
Ciarleglio et al., 2019 None yet Mean age
20.08 years followed up to 2 years
CHR population from the early detection & intervention for mental disorders program at the New York State Psychiatric Institute

Participants identified as CHR based on the SIPS criteria
All SIPS symptom domains, social & role functioning, violent ideation, & behavior, history of trauma, demographics, genetic & familial risk by interview Conversion to psychosis by SIPS criteria

LASSO-logistic regression to predict transition to psychosis
32.2% conversion rate to full psychosis over a 2 year period

Evidence for a practical & clinically useful risk calculator
Fusar-Poli et al., 2017 Fusar-Poli et al., 2019b

Irving et al., 2021

Oliver et.al, 2020

Oliver et.al, 2021
Mean age 32.97 years (mean follow-up was 1588 days) Active users of South London and the Maudsley (SLaM) NHS Foundation Trust services in 2008–2015 & had received first index diagnosis of any nonorganic & nonpsychotic mental disorder Demographics by automatic Clinical Record Interactive Search Cox multivariable PH model to assess the risk of developing any nonorganic ICD-10 psychotic disorder & time to psychosis development The multivariable model significantly predicted psychosis onset

Increased age & male sex were significantly associated with an increased risk of psychosis

Relative to White, Black, Asian, mixed, & other races/ethnicities were linked to increased risk of psychosis development
Fusar-Poli et al., 2019a None yet Mean age 32.97 years (mean follow-up was 1588 days) All patients had received a first index diagnosis of a non-organic/non-psychotic mental disorder within South London & Maudsley (SLaM) NHS Trust in the period 2008–2015. Demographics by automatic Clinical Record Interactive Search Development of any ICD-10 psychotic disorder

Fractional polynomial analysis of nonlinear association between incidence rate of developing any disorder & age bands
Compared to the original model, other than an age effect, there were no large changes – predictors remained significantly associated with an increased risk of psychosis (Male, Black, Asian, mixed, & other ethnicities)

BP, mood disorders, & transient psychotic disorders showed a comparable & higher risk of psychosis

The refined risk calculator model significantly predicted psychosis onset & showed statistically higher external validity than the original model (Fusar-Poli et.al, 2017)
Lee et al., 2020 None yet 15–35 years at baseline CHR population from the Seoul Youth Clinic (SYC)

Participants identified as CHR based on the SIPS criteria
Functional decline, IQ, California Verbal Learning Test score, Strange Stories test score, & Social

Functioning Scale (social engagement & withdrawal) by clinical & cognitive functioning assessments
LASSO-penalized Cox regression to assess the risk of developing a psychotic disorder

Transition to psychosis determined if subject met the Presence of Psychotic Syndrome (POPS) criteria for SIPS
18.0% conversion rate to full psychosis over more than 10 year period

Demonstrated fair predictive ability
Meehan et al., 2020 None yet Assessed at 5, 10, 12, & 18 years Environmental Risk (E-Risk) Longitudinal Twin Study cohort born in 1994–1995 Demographics & familial background by mother report

Child victimization exposure by caregiver reports & interviews

Mental health history by mother report e.g., self-harm/suicide attempts

Survey scales/ inventories & adult reports to assess symptomology & disorder traits e.g., Personality, ADHD, CD, & Anxiety

Parenting & familial interactions by adult reports & speech samples

Social interactions by survey & child self-reports
Internalizing disorder (e.g., generalized anxiety, major depressive, PTSD) by psychiatric interview

Externalizing disorder (e.g., ADHD, conduct & alcohol, cannabis or tobacco dependence) by psychiatric interview

Thought disorder (e.g., 1 of 7 psychotic symptoms rooted in delusions & hallucinations) by psychiatric interview

Regularized logistic regression used for prediction models of age-18 psychiatric outcomes. LASSO regularized regression used to identify subsets of predictors that maximized prediction accuracy for each outcome
60.4% (n=334) of victimized children met diagnostic criteria for any psychiatric disorder at age 18

Victimized children were significantly more likely to present with any psychiatric or thought disorder than their non-victimized peers

Three internally validated models demonstrated adequate discrimination (0.66–0.73)
Rocha et al., 2021 (Validated in two external cohorts) Assessed at 15 & 18–19 years Derived from the 1993 Pelotas Birth Cohort. Validated in the Environmental Risk (E-Risk) Longitudinal Twin Study and the Dunedin Multi-disciplinary Health & Development Study. No evidence of current or previous depression. As a pragmatic approach, mostly self-reported sociodemographic predictors were used (inherent characteristics, problematic behavior indicators, & markers of household dysfunction) Multivariate prognostic model using binary logistic regression predicting MDD at 18–19 years, measured via interview 3.1% (Pelotas), 17.7% (E-Risk), & 16.8% (Dunedin) had MDD at 18–19 years

C-statistic of the models in the Pelotas cohort ranged from 0.76–0.79

Discrepancies between samples occurred (Dunedin C-statistic = 0.63, E-Risk C-statistic = 0.59)
Studerus et al., 2020 None yet Mean age 25 years followed up to 5 years CHR population from Basel Früherkennung von Psychosen (FePsy) study

Participants identified as CHR based on the Basel Screening Instrument for Psychosis (BSIP)
Age, sex, years of school education, 5 BPRS-E subscale scores, & the BPRS-E total score Conversion to psychosis assessed by the
Brief Psychiatric Rating Scale-Expanded (BPRS-E)

Joint models to assess the risk of developing a psychotic disorder
21.0% conversion rate to full psychosis over 5 year period

Dynamic prediction may perform better than static prediction
Luby, Rogers, & Wakschlag (proposed) None yet Assessed across 0–54
months.
Infants from the Mental Health, Earlier Synthetic Cohort (MHESC) consisting of multiple extramural cohorts from Northwestern University and Washington University in St. Louis Emotional regulation & developmental functioning by survey & direct behavioral assessments

Stress exposure by survey & parent interview

Parenting, parental mental health, & family relations by survey & direct behavioral assessments

Neural functioning & development by neuroimaging techniques (MRI, EEG)

Child psychopathology by survey & parent interview
Internalizing/
Externalizing problems by survey & interview

Diagnoses of any mental health disorder by interview
(In ongoing data collection)
Worthington et al., 2021 None yet 12–35 years (mean age
18.70)
Subsample of CHR population from the North American Prodrome Longitudinal Study (NAPLS2) with cortisol samples

All participants met criteria for Prodromal risk syndrome by the Criteria of Prodromal States (COPS) & SIPS
Unusual thoughts, suspiciousness, symbol coding, verbal learning, social functioning decline, baseline age, family history, total traumas, & stressful life events by survey scales & other clinical assessments including SIPS & Scale of Psychosis-risk Symptoms (SOPS)

Baseline cortisol by saliva sample
Conversion to psychosis by SIPS criteria

Multivariate Cox proportional hazards regression model to assess the risk of developing a psychotic disorder
13.0% conversion rate to full psychosis over 2-year period

The addition of cortisol to the original NAPLS2 calculator provided 7% improvement in predictive accuracy over the base NAPLS2 risk calculator
Zhang et al., 2019 Osborne & Mittal, 2019; Zhang et.al, 2019 Mean age
20.90
years at baseline
CHR population from the ShangHai-At-Risk-for-Psychosis (SHARP) program Unusual thoughts, suspiciousness, social anhedonia, expression of emotion, ideational richness, dysphoric mood, & functional decline by interview Conversion to psychosis by SIPS criteria

Multivariate proportional hazards model for predicting psychosis
24.0% conversion rate to full psychosis over a 1-year period

Provides a simple-to-use risk calculator that only requires SIPS items

Notes: BP-NOS = bipolar disorder not otherwise specified; BP = bipolar disorder; DSM = Diagnostic and Statistical Manual of Mental Disorders, fourth edition; MHESC = Mental Health, Earlier Synthetic Cohort; SZ = schizophrenia; CHR = clinical high risk; ADHD = attention-deficit/hyperactivity disorder; MDD = Major Depressive Disorder; ICD = International Statistical Classification of Diseases and Related Health Problems; LASSO = Least Absolute Shrinkage and Selection Operator; CD = conduct disorder; PTSD = post-traumatic stress disorder; MRI = magnetic resonance imaging; EEG = electroencephalography;

Predicting Psychosis.

A majority of the research on mental health risk calculators, albeit small, is on the conversion to psychosis from prodromal stages (Cannon et al., 2008; Cannon et al., 2016; Carrión et al., 2016; Ciarleglio et al., 2019; Fusar-Poli et al., 2017; Fusar-Poli et al., 2019a; Fusar-Poli et al., 2019b; Irving et al., 2021; Lee et al., 2020; Oliver et al., 2020; Oliver et al., 2021; Osborne & Mittal, 2019; Studerus, Beck, Fusar-Poli, & Riecher-Rössler, 2020; Worthington et al., 2021; Zhang et al., 2018; Zhang et al., 2019). Individuals at clinical high-risk (CHR) for psychosis typically exhibit attenuated positive (e.g., hallucinations) and negative (e.g., anhedonia) symptoms and decreased socio-occupational functioning (McGlashan, Walsh, & Woods, 2010). Most patients show milder forms of these symptoms (e.g., brief hallucinations or mild anhedonia) before the onset of fully psychotic symptoms (Fusar-Poli et al., 2012). Understanding the risk factors that distinguish those who develop the disorder within the broader group of individuals with CHR syndrome may facilitate the prevention of the transition from CHR to psychosis and improve the course of illness in people who do convert (Fusar-Poli et al., 2013). There is high variability in symptom progression in CHR individuals, and those that do not convert are still likely to develop a host of maladaptive outcomes. Therefore, CHR individuals are a population that is well-suited to the individualized probabilistic prediction of a risk calculator approach (Osborne & Mittal, 2019). Thus far, several risk indicators for psychosis prediction in the CHR population have been identified, including individual history of mental health problems, earlier onset of non-psychotic mental disorders, and various demographic factors (Fusar-Poli et al., 2017; Fusar-Poli et al., 2019a).

The North American Prodrome Longitudinal Study (NAPLS) and Shanghai At-Risk for Psychosis (SHARP) program have developed two notable risk calculators (NAPLS-2 and SIPS-RC, respectively) that provide probabilistic risk scores for conversion to a psychotic disorder for individuals at CHR. The NAPLS-2 calculator provided risk scores for psychosis across two years (Cannot et al., 2016). The NAPLS-2 calculations are based on a multivariate proportional hazards regression model that includes: 1) age, 2) Symbol-Coding score (Keefe et al., 2004), 3) the Hopkins Verbal Learning Test-Revised (HVLT-R), 4) negative life event sum scores, 5) a drop in global functioning over the prior year, 6) severity of unusual thought content and suspiciousness, 7) family history of psychosis, and 8) total number of life-time experienced traumas. Notably, the NAPLS Risk score reflects an amalgamation of both changing features, such as age, cognition, symptom severity, and global function drop, with stable features of risk, such as family history of psychosis. As a result, an individual with relatively stable ratings in all areas may “progress” in risk over time by virtue of age alone.

The SIPS-RC calculator included participants with at least a 1-year follow-up assessment (Zhang et al., 2019). The SIPS-RC derives risk estimates from four dimensions: 1) current positive symptom severity, 2) current negative symptom severity, 3) low levels of dysphoric mood (general symptoms), and 4) deterioration of functioning over the past four months. When examining the relative performance of the two risk calculators in an independent sample, the NAPLS-2 and SIPS-RC demonstrated moderate to fair discrimination performance for distinguishing CHR converters and non-converters, respectively (Osborne & Mittal, 2019). However, when examining the calculators’ abilities for predicting positive symptom progression, both calculators provided moderate discrimination ability, with the SIPS-RC (AUC = .76) providing qualitatively better prediction of positive symptom progression than the NAPLS-2 (AUC = .71). Differences in discrimination performance between the two calculators may reflect cross-culture differences in risk prediction. Indeed, when validating the NAPLS-2 risk calculator in the Chinese SHARP sample, a similar reduction in discrimination performance was observed for the NAPLS-2 (AUC = .63; Zhang et al., 2018).

The existing risk calculators for predicting psychosis have shown promise for adopting a developmental perspective. For example, the NAPLS-2 calculator is based on a study that reduced hundreds of possible variables to a small number, including developmentally relevant constructs (e.g., childhood trauma). A developmentally-informed RDoC approach has the potential to build on these foundations. For example, currently in the NAPLS-2 calculator, life events and trauma do not offer a high predictive value for later conversion to psychosis. Rather than summing Childhood Trauma and Abuse Questionnaire items to yield an index (Cannon et al., 2016; Janssen et al., 2004), a potentially more powerful approach might be to capture timing, duration, severity, or degree of impact of trauma or stressors. Moreover, rather than treating trauma as a uniform construct, researchers could consider distinguishing between conceptually and empirically distinct types (e.g., neglect or threat; Vargas, Conley, & Mittal, 2020; Vargas & Mittal, 2018; Vargas, Zou, Conley, & Mittal, 2019). As such, the predictive value of trauma and life events could be underestimated in current NAPLS-2 calculations, given the relevance of sensitive and critical periods of neurodevelopment in informing degrees of impact of environmental exposures. Further incorporating a developmental perspective to assessing risk could be helpful in this regard.

Predicting internalizing and externalizing problems in early childhood.

Although existing calculators predicting adult clinical outcomes have considered events in childhood, there is a growing need to expand this risk evaluation during early development. Meehan and colleagues (2020) used a risk calculator model to identify significant risk predictors of psychiatric outcomes in Environmental Risk (E-Risk) populations aged 7 – 18 years, identifying individuals of victimization being the most at risk for the development of any psychiatric disorder. Risk calculators predicting depression and ADHD in youth and young adults have also been tested, demonstrating c-statistics above 0.7 in test samples (but poorer discrimination in some validation samples; e.g., Caye et al., 2020; Rocha et al., 2021). Research has also shown that some of the problem behaviors identified in these older age groups (e.g., the majority of severe and chronic internalizing and externalizing problems; Meehan et.al, 2020; Damme et al., 2021) onset with symptoms or vulnerability patterns in early childhood, laying the groundwork for identification far earlier in the developmental sequence (Wakschlag et al., 2018).

The first few years of life are a time of heightened neurodevelopmental plasticity, and interventions delivered during this time may be most effective at both the cost and impact level (Campbell et al., 2014). The significant overlap between the normative misbehaviors of early childhood and problems in self-regulation has been a major impediment to early identification (Wakschlag, Tolan, & Leventhal, 2010). To differentiate normative variation from clinical risk markers presaging chronic psychopathology, researchers must identify the constellation of modifiable risk factors that identify the subset of young children with early problems in self-regulation that will advance to persistent clinical problems (Luby et al., 2019). This includes functioning in other domains of development and social-ecological factors that may modify the likelihood that risk will result in adaptive or maladaptive outcomes.

Although risk calculators have yet to be developed for psychopathology prevention in early childhood, our preliminary data indicate the potential utility of this approach. Using data harmonized from two early childhood datasets, we examined the utility of multiple-level early childhood behavioral and ecological risk factors in preschool to predict internalizing/externalizing disorders at preadolescence. The transdiagnostic behavioral indicator, irritability, provided the most discriminative power, and behavioral risk was amplified by exposure to environmental adversity (Wakschlag et al., under review). To achieve power and precision necessary for generating a tool for dissemination, we are conducting the Mental Health, Earlier Synthetic Cohort (MHESC) Study. This is a new initiative designed to generate and validate a developmentally-based risk calculator, for which we are currently in the early development stages. The goal is to compare innovative computational and epidemiologic data science methods to accelerate clinical translation of neurodevelopmental discovery during infancy toward generalizable risk prediction for preschool psychopathology. We are pooling three independent, extramural datasets to form the first clinically-enriched “synthetic” neuroimaging cohort for generation of neurodevelopmentally-based clinical risk algorithms (N=1,020, followed from birth-54 months). Unlike prototypical risk calculators for psychosis prediction, risk calculators in early childhood, such as the MHESC calculator, are designed to operate in a relatively normative population. The MHESC Study will focus on transdiagnostic indicators of emotion dysregulation, as emotion dysregulation underlies the development of common and modifiable internalizing and externalizing problems and is measurable early in life.

A key emotion dysregulation indicator of emerging psychopathology in the MHESC study is irritability. Irritability is detectable in infancy and early childhood, and it can be reliably assessed at the survey, behavior, and paradigm levels in these developmental periods and has established neural correlates (Wakschlag et al., 2019). Non-invasive neural measures such as electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) have provided important insight into the development of brain structure and function underlying emotion dysregulation, including irritable behaviors (Brotman, Kircanski, Stringaris, Pine, & Leibenluft, 2017; Morawetz et al., 2020; Nielsen, Wakschlag, & Norton, under review). For instance, fMRI studies demonstrate that young children who are moderately irritable but not impaired recruit the prefrontal cortex for regulating frustration, but children who have severe impairment show less activation in this region (Grabell et al., 2018). Similarly, structural MRI has revealed that severe irritability in early school age children predicted reduced gray matter volumes in areas related to emotion (e.g., amygdala, medial prefrontal cortex) at preadolescence (Damme et al., 2021). With respect to EEG and Event-Related Potentials (ERPs), children who are persistently irritable at age 3 have shown a heightened error-related negative (ERN) component (associated with error monitoring) at age 6, which in turn, predicted greater internalizing problems at age 9. Further, a blunted ERN was related to more externalizing problems (Kessel et al., 2016).

2. An RDoC-Informed Approach for Selection of Neurodevelopmental Markers and Environmental Indicators

Practical guidelines for incorporating markers into a risk calculator have been established, such as defining the population and outcomes of interest, as well as modeling and testing selection to evaluate incremental value (D’Agostino, 2012). The most rudimentary but necessary condition for including a marker is its statistical significance (Pencina et al., 2008), but that alone does not guarantee that the marker will improve the discrimination of a risk prediction tool. Given the dearth of developmental mental health risk calculators that exist, RDoC principles can be particularly helpful for guiding initial decision making for risk indicator inclusion.

A core tenet of the RDoC framework is the use of multiple units of analysis when measuring a construct within a particular domain. RDoC delineates behavioral and neural domains of positive valence, negative valence, cognitive, social processes, arousal/regulatory, and sensorimotor systems. Each of these domains has a set of constructs that can be measured across multiple units. For example, a researcher may measure the construct of fear in the negative valence system at the circuit unit (e.g., ventromedial prefrontal cortex), the physiological unit (e.g., pupillary dilation), the survey unit (e.g., parent report of temperament), and the paradigm unit (e.g., Laboratory Temperament Assessment Battery). The researcher could then use these scores in a risk calculator algorithm to assess the added predictive value of each unit for the development of an anxiety disorder.

As previously mentioned, the MHESC Study will use indicators of emotion dysregulation to predict probabilistic risk for psychopathology. Emotion regulation indicators span RDoC domains that include negative valence, positive valence, arousal and regulatory, and cognitive systems. Negative valence systems reflect how an individual responds to stressful or aversive situations (e.g., fear), and positive valence systems pertain to positive motivational contexts (e.g., responses to reward). For example, highly irritable children show alterations in brain regions associated with reward processing, error monitoring, and emotion regulation when they expect a reward that is not received (Perlman et al., 2015). Arousal and regulatory systems pertain to an individual’s sensitivity to external and internal stimuli, functionally facilitating engagement with the environment in a context specific manner. These systems could facilitate or hamper emotion regulatory efforts. Cognitive functions intrinsic to emotion regulation dissociate components of RDoC cognitive system domains. The cognitive domains most relevant to emotion regulatory capacities would include cognitive control, particularly the inhibition subconstruct, and working memory, particularly the active maintenance and flexible updating subconstructs.

Irritability, when conceptualized within an RDoC framework, stems from alterations across a swathe of domains as well, including negative valence domain (e.g., frustrative non-reward), as well as positive valence domain (e.g., errors in reward prediction), and cognitive domain impairments including attention and language subdomains (Bell, Bryant, Boyce, Porter, & Malhi, 2021). In addition, irritability may engage emotion regulation indicators, thus comprising parts of the above discussed arousal/regulatory, cognitive control/inhibition, and working memory/active maintenance and flexible updating. Using an RDoC approach, a key goal of the MHESC Study is to measure constructs such as irritability and its correlates across multiple units of analysis, including circuits (e.g., natural-sleep MRI), physiology (e.g., EEG, eye-tracking), behaviors (e.g., in-lab observations), parent reports, and paradigms (e.g., in-lab experimental tasks).

These indicators and their units of analysis have been extensively studied in developmental populations and are considered robust predictors of child mental health outcomes (Cicchetti, 2008). Notably, the bulk of evidence on neural circuitry is from older children. However, there is an accruing body of literature on neural circuitry in infancy and early childhood underlying risk for developing psychopathology. For instance, amygdala resting state functional connectivity in neonates has predicted internalizing symptoms at 2 years of age (Rogers et al., 2017). Less amygdala reactivity has been related to greater depression in preschoolers, and aberrant Default Mode Network connectivity has been identified in preschoolers with a history of depression (Gaffrey et al., 2018; Gaffrey, Luby, Botteron, Repovš, & Barch, 2012). On the whole, however, there are high levels of variability in findings and a lack of standardization of neural measures in young populations. For the MHESC, we will synthesize the neural circuitry findings across the lifespan with extant knowledge from early childhood to select the appropriate neural biomarkers for inclusion in the risk algorithms. It is important to include these neural indicators in the development of the MHESC given that risk calculators may have the greatest clinical value in these younger age groups. Neural measures are seldom incorporated in risk calculators due to increased burden and low reproducibility of findings, thus it is imperative that any findings from the MHESC Study, particularly with respect to neural correlates, be robust, replicated, and externally validated before they are included in a risk calculator for public health use (see Section 5).

Although RDoC spans a wide range of constructs examined on a spectrum of normal to abnormal, it has focused on capturing deviations of degree. For instance, anxiety becomes a disorder when an individual demonstrates excessive fear or anxiety, but some anxiety, particularly in response to stress, is normal and adaptive (e.g., vigilance toward impending threat). However, less emphasized in RDoC are deviations in kind, or phenomena that are not evident in normative development (Mittal & Wakschlag, 2017). Deviations of kind include compulsive behaviors evident in obsessive-compulsive disorder (OCD). Deviations of degree and kind should both be considered when choosing risk indicators and should align with the outcome of interest.

Accounting for environmental factors that amplify or attenuate risk.

An individual’s dysregulation is nested within dynamic environmental contexts and is an important predictor of later adaptation, thus a tiered approach is necessary for risk prediction. Neurodevelopmental risk is probabilistic, such that multiple individual and contextual mechanisms shape early characteristics and behaviors over the course of development to influence outcomes (Mittal & Wakschlag, 2017). These risk factors do not occur in isolation, but rather operate together to inform development (Rutter, 1987). Further, given the large intraindividual and interindividual variability in development of mental health problems, particularly in early development, an analysis across multiple systems and modalities is essential for identifying profiles of risk that can inform screening and intervention (Finlay-Jones et al., 2019).

Pediatric care settings have only recently started considering environmental factors in the picture of a child’s health, even though there is robust evidence to support that early adversity and environmental stress influence mental health problems and the efficacy of interventions (Shonkoff, 2010; Shonkoff, Boyce, Levitt, Martinez, & McEwen, 2021). Epidemiological perspectives suggest that multiple ecological subsystems of the individual are the key predictors for mental health cumulative risk scores, such that accrual of adversity over time decreases developmental competence (Sameroff, Seifer, & McDonough, 2004). These subsystems include parent mental illness and psychological well-being, parent attitudes and beliefs about child development, responsive and sensitive parenting, and social determinants of health such as racial discrimination (Fowler, Tompsett, Braciszewski, Jacques-Tiura, & Baltes, 2009; Goodman et al., 2011; Sameroff, Seifer, Barocas, Zax, & Greenspan, 1987; Teti, O’Connell, & Reiner, 1996). Although many of these factors amplify risk for psychopathology, there has been less of an emphasis in mental health risk factors on resilience-promoting factors (Yates, Egeland, & Sroufe, 2003). For example, warm and responsive parenting interacts with early regulatory abilities in a bi-directional manner, and this type of parenting is the main target of most evidence-based early prevention/intervention programs (Kochanska & Kim, 2013; Smith et al., 2020; Waller et al., 2014). Further, even though developmental psychopathology research has traditionally included factors both intrinsic and extrinsic to the child in predicting mental health outcomes (Cicchetti & Dawson, 2002), mental health risk calculators typically do not include these extrinsic, contextual factors. An important goal of the MHESC Study is to test the added value of social determinants of health, parenting, and family context indicators. In our proof-of-concept risk calculator predicting preadolescent psychopathology from preschool indicators (Wakschlag et al., under review), adding environmental adversity to the model accounting for demographics and behavioral risk improved model discrimination.

RDoC conceptualizations stress the value of environmental, social, and developmental context for emotional experience, regulation, and symptom presentation (Barrett, 2012; Barrett, Mesquita & Gendron, 2011; Wilson-Mendenhall et al., 2011). The arousal domain, for example, enables sensitivity to internal or external stimuli, such that context-sensitive action can be undertaken. In this case, chronic high arousal preparing the organism for context sensitive action in the face of repeated exposure to environmental threats could constitute an adaptive response. However, chronic environmental threat exposure occurring during sensitive developmental periods could ultimately contribute to emerging vulnerability for psychopathology. As such, constructs that relate to emotion regulation could interact with and be impacted by both development and environment.

Which methods when?

Multiple levels of analysis are important for best identifying risk for psychopathology, but assessment at all levels for all children may not be feasible, practical, or translatable. Indicators of emotion dysregulation are measurable in the first year of life, pervasive across mental health disorders, and associated with disruptions in the prefrontal cortex (Beauchaine, 2015; Beauchaine & Cicchetti, 2019; Finlay-Jones et al., 2019). Yet questions remain regarding which or what combination of indicators are sufficient for predicting mental health risk, as well as when and for whom these indicators are most predictive. In this context, “sufficient” means the most parsimonious and least intensive/burdensome set of indicators that add clinically meaningful explanatory value. By leveraging multiple units of analysis in a risk calculator approach to developmental psychopathology research, we can determine the units that are needed for identifying children across the spectrum of risk to streamline measurement and reduce burden at the clinical level.

Risk prediction models prioritize low burden or less intense measures, warranting the inclusion of more burdensome or intense measures only when they evidence substantial added predictive value (Lloyd-Jones, 2010). For example, the MHESC Study aims to better understand whether more cost- and resource-intensive methods have significant added value to early mental health risk prediction. The first risk calculator algorithm will be derived solely from commonly used survey data to optimize feasibility for future use in primary care settings. Then, we will investigate the statistical and clinical incremental utility of more intensive assessment for future use in mental health specialty settings. This algorithm sequentially tests the added predictive value of methods of intermediate-high intensity (from direct observational or performance-based assessments to MRI) for most precise, least burdensome risk prediction. This sequential method will enable us to establish for which children surveys alone are adequate, when behavioral methods have added value, and/or whether EEG and/or MRI have sufficient added value for children with higher clinical uncertainty (e.g., those that fall in the middle of the dimensional spectrum) after replication and validation to warrant consideration of use. In addition, dimensional approaches and capturing heterogeneity lead to the question of what works for whom. As RDoC and developmental psychopathology frameworks point away from a “you have it or you don’t” approach to clinical risk towards a vulnerability spectrum conceptualization, it is possible that children at neither clinical extreme will need more intensive measurements or additional domains to precisely determine probabilistic risk. For instance, using a stoplight metaphor (Smith et al., 2018), children in green (low risk) may only receive additional testing as part of their regular well-child visit, children in red (high risk) may receive immediate referral for prevention/intervention or mental health specialty, and children in yellow (higher clinical uncertainty) may prompt the additional intermediate-high intensity measures.

An alternative approach in risk algorithm development is to incorporate tools that are practical for implementation across all potential risk levels. For example, these calculators may only use demographic information and clinical/neurocognitive functioning measures in the forms of surveys and short tests (Cannon et al., 2015). Digital technologies for capturing behavioral assessments may be optimal for reaching under-represented and low-resourced families. Online platforms such as Lookit (Scott & Schulz, 2017) have pushed past the typical constraints of in-person data collection by capturing a wide range of child behaviors from participants representative of the United States population with respect to race, parent education, and income. Prioritizing cognitive tasks that adults can complete online could help the critical effort of reaching non-help-seeking individuals. Using lower-burden, cost-effective, and easily accessible assessments may increase the likelihood of widespread adoption in the real world. Calculators that leverage these types of assessments will also be more accessible to adolescents and young adults who have concerns about their mental health, facilitating their ability to make informed decisions for seeking information and support.

3. Strategies for Embedding Developmental Principles in Risk Calculator Designs

Developmental psychopathologists use individual characteristics to predict which individuals will grow into healthy, prosocial, competent adults, as well as for whom mental health problems might arise. Importantly, some of these characteristics change across seconds, days, and months, and some of them remain relatively stable across the life course. Change, or the absence of it, is woven into the fabric of risk for psychopathology, but is not always prioritized in the actual calculation of risk. There is recent evidence that accounting for longitudinal change in blood pressure does indeed improve precision of probabilistic risk prediction in cardiovascular disease (Pool, Ning, Wilkins, Lloyd-Jones, & Allen, 2018). RDoC, while providing a comprehensive framework for identifying risk factors that promote psychopathology, has not yet attempted to account for such intraindividual variability and change.

Risk status is situated in trajectories of biobehavioral change across development, rather than a characteristic or behavior at one point in time (Wakschlag et al., 2015). Capturing all risk indicators at one point in time can lead to an over- or under-identification of impairment. On one hand, treating normative variation in behavior as a clinical marker may squander limited resources. On the other hand, clinically-concerning behavior may not occur until later in development for some children, which may lead to an under-identification of disorder. Further, even when change is considered, many studies have simplified development to a change score in the two-time point approach, which can smooth out important discontinuities inherent to development (Adolph, Robinson, Young, & Gill-Alvarez, 2008). Longitudinal designs with time points that are spaced out over long periods may also mask development as it emerges (Adolph et al., 2008; Boker & Nesselroade, 2002; Collins, 2006; Hertzog & Nesselroade, 2003; Damme et al., 2021). Developmental processes and skill acquisition occur at rapid rates (e.g., days), are highly variable, and change depending on the construct, person, and time period assessed. Incorporating repeated measures in a risk calculator framework can facilitate the differentiation between transient maladaptive behaviors and stable impairment.

Examples from emotion dysregulation and psychosis research demonstrate that measurement of risk at only one time point can lead to biased results. Irritable behavior is a risk factor for the development of psychopathology, but only when it is pervasive, intense, and occurs in developmentally unexpectable contexts (Wakschlag et al., 2019). Further, pathogenic forms of dysregulated tantrums, such as destruction, are far less common, but they can be important indicators for sensitive and specific clinical identification (Wiggins et al., 2018). However, the presence of tantrums per se in early childhood is normative and approximately 1/4–1/3 of young children do not persist into the school years (Wakschlag et al., 2015). Although children who score high in irritability are at higher risk of chronic irritability, and less than 1% of children with low irritability develop higher levels later in childhood (Wiggins et al., 2020), the children who are at intermediate risk may require repeated assessments to determine event rates for psychopathology at later timepoints (Wiggins, Briggs-Gowan, Brotman, Leibenluft, & Wakschlag, 2020).

With respect to psychosis, 15–35% of CHR individuals between the ages 12 and 35 develop fully psychotic symptoms across 2 years (Fusar-Poli et al., 2012). Notably, this is also a population that is typically in the adolescent and young-adult developmental period, as the onset of schizophrenia characteristically occurs between mid to late adolescence and late adulthood (i.e., early thirties; Jones, 2013; McGorry, Purcell, Goldstone, & Amminger, 2011; Patel, Leathem. Currin & Karlsgodt, 2021; Rajji, Ismail, & Mulsant, 2009; Walker et al., 2013). Given the large heterogeneity in risk factors and dynamic developmental window for developing fully psychotic symptoms, a developmentally informed calculator leveraging multiple measurement occasions could have significant potential. For example, Worthington and colleagues (2021) included salivary cortisol as a new predictor in the established NAPLS-2 risk calculator and observed marked improvement from the original model. This finding speaks to the relevance of stress-sensitivity for emerging psychosis. But importantly, increases in stress-sensitivity are developmentally normative during the psychosis-risk period, as cortisol increases each year in adolescence to play an important role in modulating pubertal development (Mittal & Walker, 2019). Because sensitivity is a moving target during the risk period, there is an opportunity here for detecting obvious and conceptually relevant divergence. The CHR period is a time of clinical change (by definition, most meeting criteria have recently emergent and/or escalating attenuated psychosis symptoms). It is also a time of profound developmental change, overlaying the adolescent and young-adult period. Future risk calculators that take into account developmental trajectories (e.g., using variables that capture a subgroup of youth who are showing a steep incline in stress sensitivity) may reveal the subgroup at highest risk for transition to psychosis (Dean et al., 2016).

Further, the degree to which a behavioral measure is dysfunctional changes depending on the developmental stage in question. As previously mentioned, irritable behavior is normative in early childhood, as this is a point in development when children are learning to regulate their own emotions and behavior with guidance from parents (Rothbart & Bates, 1998). Therefore, young children are still novices and are prone to tantrums and outbursts. These behaviors become clinically concerning when they are frequent, high in intensity, occur throughout multiple contexts, and impair other functioning (Wakschlag et al., 2015). However, with age and experience, self-regulation is typically well integrated in children’s emotional, cognitive, and behavioral responding. Thus, the presence of a tantrum per se at older ages may be cause for concern, where quality of behavior (e.g., dysregulated tantrum) is especially discriminative at younger ages (Wakschlag et al., 2010). Regarding psychosis, motor behaviors such as clumsiness may disappear as children reach other milestones, but the latent vulnerability for psychosis remains and can be detected with sensitive paradigms in adults (Mittal & Wakschlag, 2017). Researchers can better incorporate developmental change processes into their risk algorithms by choosing meaningful sampling rates when collecting data, modeling change in a theory-informed way, and considering a risk indicator’s capacity for change.

Choosing sampling rates in the context of developmental change.

Many factors can be considered for determining the optimal sampling rate, including number of time points, spacing between time points, and when to begin sampling. If meaningful, theoretically driven choices are not made at the data collection level, the risk algorithm may yield biased results with little predictive value. First, researchers should consult theoretical models regarding the shape and timescale of change for informing their decisions (Boker & Nesselroade, 2002). If theory dictates a more complex anticipated pattern of change, more frequent measurement occasions are often required (Ram & Grimm, 2007). For instance, to model quadratic change, at least three time points are needed. Second, the sampling rate should capture the age of onset for the behavior(s) in question, and ideally, start sampling before the expected age of onset. Too large a time interval and the model will erroneously estimate the age of onset by missing when most variability occurs. For instance, Adolph and colleagues (2008) found that when changing the sampling interval of infant motor behavior from the daily to monthly level, there were increased errors in estimating how early infants reached stable expression. Third, data should be collected as frequently as possible. This is particularly true for binary risk indicators, as distortions in their trajectories are especially susceptible to gross sampling intervals (Adolph et al., 2008). Once researchers use theory to inform the sampling rate and collect the data at these time points, they can use the scores to determine how many measurement occasions are needed to reduce false-positives, the optimal interval length for optimizing risk prediction, and how early to begin sampling. Refining the risk calculator in this way will help researchers identify the minimum number of occasions requisite for sensitive and specific risk prediction, as there will come a point when additional occasions will not improve the model. Further, limiting the number of assessments will mitigate the burden for both families and care providers when the calculator is ready for public health use.

Capturing the shape of developmental change.

Repeated measures data open up possibilities for examining multiple change processes that can be used to predict mental health disorders. In particular, growth curve analyses can be leveraged for describing interindividual differences in intraindividual change (Ram & Grimm, 2007). Two common parameters that can be extracted from these growth curve models to predict risk are the individual’s intercept (i.e., initial or average level of behavior) and their linear slope (rate of change). However, development is ripe with both continuities and discontinuities in behavior, from gradual and incremental changes in development of white matter in the brain (Giedd et al., 1999) to surges in cognitive performance bookended by periods of little to no change (e.g., MacNeill, Ram, Bell, Fox, & Pérez-Edgar, 2018). Therefore, researchers should use their knowledge of developmental theory to consider using parameters from non-linear trajectories (e.g., polynomial, exponential, and logistic).

Statistical methods that identify different trajectories of development can be used to form heterogeneous groups meaningful for risk prediction (Nagin, 2005). For instance, Allen and colleagues (2014) used latent mixture modeling to identify five unique groups of blood pressure trajectories through early adulthood to predict the development of atherosclerosis in middle age. They found that groups with elevated blood pressure over time had greater odds of having a high score of coronary artery calcification in middle age compared to the group with low-stable blood pressure. Allen and colleagues (2020) have also examined CVH trajectory groups beginning in childhood, finding relations with subclinical atherosclerosis levels in middle age. In their study of child socioemotional development, Liu and colleagues (Liu et al., 2018) identified six patterns of child anger from 9 months to 7 years of age: low/stable rank, average/stable rank, average/decreasing rank, average/increasing rank, high/decreasing rank, and high/stable rank. They found that children in the high/stable angry group had greater levels of internalizing and externalizing problems at age 8 compared to those in the average/stable, average/decreasing, and high/decreasing groups. These studies demonstrate that latent group modeling can inform our understanding of the development of disease and disorder over time, where such scores could be incorporated into a mental health risk calculator.

Two other aspects of within-person change that can be considered for identifying disorder are an individual’s timing and rate of change. Timing refers to how mature an individual is in relation to their same-age peers, and rate is how quickly or slowly an individual progresses. Individuals can be considered early, average, or late maturers based on their progress relative to others, and they can also be considered fast, average, or slow depending on their pace from one point to another. Findings from logistic growth curve models demonstrate relations among timing and rate that inform between-person differences that have potential downstream consequences for developmental outcomes. For instance, earlier timing (at what age youth reach inflection point, or 50% growth) and faster rate (slope of curve at the inflection point) in pubertal development have been associated with youth’s internalizing and externalizing problems (Marceau, Ram, Houts, Grimm, & Susman, 2011). Across the second half of the infant’s first year, timing and rate of cognitive task performance are correlated, such that infants growing faster reach the inflection point (50% of the way between no ability and mastery) earlier. This rate of change is related to linear change in EEG power across this period (MacNeill et al., 2018). In a study of dysregulated fear, fear sensitivity scores (50% fear) across fear contexts were related inhibition ratings across early childhood (Buss, Davis, Ram, & Coccia, 2018). In sum, how children compare to their peers in their timing of milestones and the rates at which they reach such milestones contribute to physiological change and problem behavior. There is a lack of established norms for the development of many emotional features relevant to risk for mental disorder. Thus, identifying prototypical and normative variation in milestone acquisition may be a logical next step for the examination of developmental psychopathology.

Capturing capacity to change.

In addition to rate of change, risk indicators can also reflect the individual’s capacity to change, or dynamic characteristics of the individual (Ram & Gerstorf, 2009). They may capture trait-like capabilities of the individual or their potential to change, such as their plasticity, lability, rigidity, and robustness. For instance, within-family lability in parent-adolescent connectedness is linked to risk for depression, anxiety, antisocial behavior, drunkenness, and marijuana use one year later (Fosco, Mak, Ramos, LoBraico, & Lippold, 2019).

Importantly, some variables may not change at all during the period of study or throughout the individual’s life course. These time-invariant, between-person predictors of risk include demographic characteristics, such as sex and crime status. They are typically easier to measure than psychological or social constructs that change over time, making them easy additions to the risk calculator. They also may serve as important moderators in the relations between risk indicators and mental health outcomes, as constellations of risk indicators may only inform risk for particular demographic subgroups.

Although such demographic information is important to account for and can help identify those individuals who are most at risk, it may not be advantageous to include multiple stable measures in a risk prediction model if they cannot be targets of prevention/intervention efforts. For example, the current NAPLS risk calculator includes a range of indicators, but several will never change (e.g. history of trauma, first-degree relative with psychosis). Sensitivity to change is inherent in markers that prioritize development. The possibility of testing whether interventions lower a risk calculator score is currently limited by calculators that rely heavily on static markers or those with limited potential for tapping into clinically relevant variation and change.

Further, although demographic information may be relatively easy to obtain from participants, these variables likely reflect complex social constructs that may, in turn, perpetuate disparities. For instance, race and ethnicity have been historically included in risk prediction algorithms (Obermeyer, Powers, Vogeli, & Mullainathan, 2019), yet social factors such as racism and forms of exclusion and mistreatment are what likely hold predictive power for mental health outcomes. Risk calculators that include racial and/or ethnic categories should be appropriately justified. Racial categories alone should not be used to interpret psychological processes because they do not provide conceptual information about behaviors and attitudes of the study participants (Helms, Jernigan, & Mascher, 2005). Rather, risk calculators should include variables that directly measure minoritized individual’s successes, experiences, and exposures to racism and mistreatment. Lastly, risk calculators should be tested and validated in samples that have adequate representation across racial and ethnic groups, which can be facilitated by leveraged integrative data strategies (see Section 5).

4. Choosing Mental Health Outcomes

As previously mentioned, both individual- and environmental-level factors contribute to mental health problems across development and thus underlie many disorders. Although this non-specific vulnerability can pose challenges for diagnosing a particular disorder while ruling out other disorders, population-based studies can take advantage of the transdiagnostic approach by identifying risk factors that cut across multiple disorders to impact “the greatest proportion of disease burden” (Finlay-Jones et al., 2019). These efforts may be particularly fruitful in early childhood, a time when mental health disorders are often undifferentiated and behavioral heterogeneity within a diagnostic group is high (Kim & State, 2014). Researchers implementing risk calculators in early childhood may consider examining broadband self-regulation problems as outcomes, rather than narrow disorders, as they are common and cut across several domains of functioning (Wakschlag et al. 2019). Self-regulation problems also widely span the cognitive control and working memory RDoC constructs.

This transdiagnostic approach may not be fitting of all risk calculators, where some may be designed to predict risk for a specific disorder with a concerted effort to rule out other highly comorbid disorders. To improve specificity, a risk calculator may choose to include a measure on which patients of the target disorder perform normally, but people with related disorders evidence impairment. Risk calculators for psychosis in older adolescents and young adults have used this more narrow approach to identify conversion to psychosis in particular. The priority has been to focus on the group of individuals who meet criteria for the CHR syndrome. In doing so, they may leverage behavioral paradigms that distinguish CHR pathology from other forms of psychopathology that may manifest similarly yet require different treatment (Gold et al., 2020). For instance, tasks that elicit hedonic responses may differentiate schizophrenia from mood disorders.

Traditionally, risk calculator outcomes are binary, such that risk indicators predict whether an individual either has or does not have a disorder. RDoC adopts a transdiagnostic approach for developmental risk prediction where dysfunction is a continuum. Initially, it may appear that the risk calculator framework with its binary diagnosis works to constrain diagnostic systems rather than broaden or deepen them. However, criteria for developing a disorder can be shifted in different iterations of the risk calculator algorithm for comparison. Varied thresholds can be derived via psychometric methods informed by clinically meaningful thresholds. For example, the threshold for “watch and wait” monitoring by pediatricians may be lower than the threshold for psychopharmacologic treatment. Further, risk calculators do not need to be limited to whether an individual does or does not have a diagnosis. Rather, researchers could consider developing a risk calculator for predicting whether or not an individual needs treatment. Treatment thresholds may be lower than diagnosis thresholds, and individuals who do not quite meet clinical diagnosis risk criteria may still benefit or need the treatment, depending on what resources are available and the risks and side effects of such treatment.

As an alternative to binary diagnostic criteria, which may not be optimal in early childhood given heightened within-person variability in preschool psychopathology (Kotov et al., 2017), researchers may consider using a bifactor modeling approach to generate a general psychopathology latent factor, or “p factor” (Caspi et al., 2014). The p factor reflects shared psychopathology variance similar to the general IQ “g factor”, in that it reflects the co-occurrence of all measured psychopathology symptoms. Thus, using the p factor may be a more optimal way to capture and treat clinical phenomena at young ages compared to specific binary DSM disorders (Caspi et al., 2014; Caspi & Moffitt, 2018; Martel et al., 2017; Michelini et al., under review). Treatment approaches would focus on transdiagnostic risk factors for psychopathology that emerge from the p factor. Further, transdiagnostic manuals may be easier for clinicians to use as opposed to training on several specified treatment protocols (Walkup, Matthews, & Greene, 2017). We will use a p factor approach in the MHESC Study to predict probabilistic risk of impairing internalizing/externalizing psychopathology at preschool age from risk factors during the infant and toddler periods. To generate clinical thresholds for the p factor for predictive utility to clinical outcomes, scores can be validated in relation to the dichotomized diagnosis score.

5. Strategies for Generalizing Risk Prediction

Individuals must be relatively healthy to capture the onset of the disease within the time measured (Pencina & D’Agostino, 2012). Large samples sizes help to attain the necessary power and precision for the identification of a mental health disorder. Researchers using multiple methods to chart development over time often must sacrifice large sample sizes due to time and budgetary constraints. Therefore, integrative data analysis methods, such as pooling observed data from multiple cohorts to create a synthetic cohort, may be an ideal method for retaining richness in both data quality and quantity. This integration of data from independent studies has the potential to increase enrollment of underrepresented populations in research studies and cross-validate or replicate across datasets to increase generalizability. These methods can bring together scientists from different domains to springboard interdisciplinary research and solve methodological challenges, as well as pool resources to run a large study that would otherwise be too costly or unfeasible. These integrative methods do not compromise the aims of the individual studies, but rather leverage ongoing or existing studies that are similar in nature (Chandler et al., 2015; Fortier et al., 2017).

Modern statistical approaches such as the synthetic cohort approach use a full-information multivariate framework for preserving the variability in development that might otherwise be erased if timepoints were missing. Further, individuals can be analyzed across all time points using multiple imputation by leveraging their data from other time points (Ning et al., in press; Siddique et al., 2015). The dataset in the risk calculator analysis is thus a combination of the observed and imputed data so that each participant has a data point for every measure at every time point (Siddique et al., 2015).

It is inevitable that independent study cohorts do not use all the same measures for assessing the same construct. Data can be harmonized, such that study-specific variables are systematically pooled into the analytical dataset, resulting in a set of common variables across all the samples that are considered the same construct. Common methods for harmonization include algorithmic transformation (combinable variables or categories), simple calibration model (e.g., transforming weight from pounds to ounces), standardization models (e.g., z-score transformations), latent variable models (combining two different scales into a latent variable), Item Response Theory methods, and multiple imputation models (Curran & Hussong, 2009; Fortier et al., 2017; Siddique et al., 2015). To implement this approach, each cohort must have constructs in common that serve as anchors for the harmonization process. For instance, to examine the added value of responsive parenting, each study must have a measure of responsive parenting even if that measure differs across cohorts (Curran & Hussong, 2009; Luby et al., 2019).

Successful harmonization initiatives of ongoing studies require that investigators work together to determine the overarching research questions and hypotheses, what domains of study are necessary for addressing these questions, and how data will be collected in each study (Chandler et al., 2015). The latter includes aligning on study protocols for measures that require pre-harmonization. Although survey measures can often be standardized to reflect similar scores and behavioral observations can be recoded (as long as they elicit the desired behavior), experimental tasks and neural measures are sensitive to even small differences in the lab environment or stimulus presentation that can have major effects on the final results. Pre-alignment of such measures may involve travel to all study sites to set up the experiment, observing protocols, and training study staff (Norton et al., under review). Throughout the study, fidelity trips across sites may also be needed to ensure protocols are maintained. Other critical steps for the harmonization process include monitoring data collection and participation, creating measures documentation and data codebooks for the integrated dataset, and determining how and when data will be deposited and harmonized for analysis (Chandler et al., 2015). Importantly, although agreeing on common research questions and psychological constructs is necessary for the creation of a synthetic cohort, the harmonization approach provides flexibility in how data are collected to leverage unique sample characteristics. Further, harmonization does not require individual investigators to compromise on the unique scope of their work (Chandler et al., 2015).

To test for generalizability of model performance, the risk calculator algorithms should be internally validated, as well as externally validated in an independent dataset (Simon, 2006). Internal validation involves splitting one’s sample into derivation and test samples to estimate reproducibility of the algorithm. Various methods of internal validation can be applied (e.g., sample splitting, bootstrapping, and cross-validation), but regardless of method, the test is performed in the original population (König, Malley, Weimar, Diener, & Ziegler, 2007). The test set can be used to define thresholds for determining level of clinical risk. The predicted and observed risks can then be compared in the test sample to determine the accuracy of the new model. After this internal validation, researchers should validate the risk algorithms in an external dataset to examine model accuracy before they apply this algorithm in real-world settings. External validation is required for rigorous assessment of the generalizability for any proposed risk prediction tool (König et al., 2007).

The MHESC Study is an example of using synthetic cohorts in risk calculator creation. It uses a synthetic cohort approach by combining multiple, ongoing extramural cohorts at Northwestern University and Washington University in St. Louis. Investigators at each site have pioneered neurodevelopmental approaches to early childhood psychopathology, multi-modal infant neuroimaging, computational and deep learning methods, and epidemiologic risk prediction. Activities in both studies were extensively pre-aligned to capture the central transdiagnostic infant risk phenotype of emotion dysregulation, as well as the central transdiagnostic clinical outcome of general psychopathology risk (internalizing/externalizing problems) using multiple methods including surveys, behavioral observations, eye-tracking, EEG, and MRI. On-site training and oversight by study investigators have enabled each site to collect the multi-modal data. The MHESC Study capitalized on the neural measures that were part of the original studies, so rather than needing to introduce new neural measures, we increased the number of occasions at which we collected these measures. Each study uses identical neural and behavioral measures. Although some survey measures differ, all studies capture the same constructs of interest, allowing for harmonization of survey data post-collection. This synthetic approach adjusts for sample heterogeneity and tests its empirical salience. Using diverse pooled samples with methods that rigorously account for such differences enhances the generalizability of the risk calculator for future public health use. The risk calculator algorithm will be internally validated, as well as externally validated in the normative sample collected in the Baby Connectome Project (Howell et al., 2019).

6. Accelerating Clinical Translation to Mental Health Disorder Prevention

Risk prediction algorithms should be useful across settings, from public health (e.g., screening for a disorder) to mental health treatment (e.g., diagnosis, prognosis, clinical decision making). The final risk calculator for pragmatic clinical use should be the most reliable and parsimonious algorithm, which may be based on the intended end-user, ambiguity of clinical presentation, and cost and/or risk of the intervention.

Primary care.

Risk calculator algorithms, taking into account both individual- and context-level factors, can be implemented in routine care at children’s pediatric check-ups to facilitate quick decision-making for prevention or clinical services. Behavior in early childhood is highly variable, such that health care professionals often tell parents to watch and wait for more pronounced problem behaviors to develop and remain stable over time. Using a personalized risk estimate for their patients, health care providers can make a faster, empirically-informed decision to identify and act when faced with clinical uncertainty (ASCEND Institute, https://www.telethonkids.org.au/ascend). If providers could easily differentiate youth who are at risk from those who are not, based on a risk algorithm score, they could refer patients to mental health services more efficiently. For mental health measures to be successfully implemented in routine primary care, they must balance brevity with the ability to capture modifiable, transdiagnostic risk factors for the outcome of interest (Smith et al., 2019). Therefore, survey-only or a combination of survey and online cognitive assessments that generate instantaneous scores may be most feasible for primary care.

Mental health specialty settings.

For clinical practice, mental health risk calculators have the potential to provide empirically-informed parameters for therapeutic decision-making based on multi-level risk makers in a standardized, cost-effective manner. These guidelines are currently underdeveloped, and establishing them could ameliorate clinical and public health practice. A risk calculator in a mental health specialty clinic may need to be more comprehensive than a risk calculator for primary care, because individuals coming to the clinic may already have an indication that problems exist and are thus seeking further diagnostic support. In this setting, a risk calculator may also be of particular value for determining whether groups of intermediate risk (higher clinical uncertainty) need added evaluation to increase clinical certainty. In such cases, clinicians may employ intensive methods (e.g., behavioral observations, EEG, MRI) to acquire a more complete understanding of risk.

Public access.

Although many children and adolescents have a primary care physician, these visits can be infrequent, particularly as children get older. Further, they may not always serve as a space where parents and youth feel they can speak openly about potential mental health problems. Risk calculators could be made publicly available in order to reach vulnerable individuals and families who would benefit most. Risk calculators for physical health risk, such as the Framingham Risk Calculator for Coronary Heart Disease, are already available for anyone on the internet to use. Researchers may also consider assessments for the risk calculator that can be completed in the home of any young person at risk for mental health problems. For instance, an individual can use an online platform that delivers surveys and cognitive batteries that feed into the risk calculation score. Such online distributions will shift risk prediction from the few clinical sites in town to any individual with access to the internet.

An important caveat is that once reliable developmental risk calculators are established in research, resources for accessing evidence-based treatment must be made easily available before the risk calculators can be used in practice. Without access to this infrastructure, those with a high-risk score will not receive the help they require and deserve. Unfortunately, the availability of these resources and their support in payment systems lag far behind the empirical evidence base. Further, a number of structural (e.g., financial costs, parent availability, transportation) and attitudinal (e.g., stigma, thoughts that problems will get better on their own) barriers prevent timely access to treatment (Reardon et al., 2017). Children from low-income families and historically disadvantaged populations, as well as children whose parents have mental health issues, may disproportionately face more barriers to initial mental health treatment (e.g., Ofonedu, Belcher, Budhathoki, & Gross, 2017). An important resource that could be developed alongside the calculator is a non-academic, web-based dissemination platform for strategic health messaging and evidenced-based information on primary-care-based mental health risk and promotion. These materials must be accessible to the population at risk and informed by stakeholders.

Risk of stigma.

The ultimate goal of many risk identification algorithms is to implement them at a large scale in existing health care systems during critical and sensitive periods when interventions are likely to be most successful (e.g., early childhood, transition to adolescence). Although early identification can prevent the onset of disorder or mitigate symptoms, labeling children with a disorder early on can lower their social and academic standing and foster stereotypes, prejudice, and discrimination (Kaushik, Kostaki, & Kyriakopoulos, 2016). Seeking out this support requires that individuals and caregivers disclose their mental health problems, which can exacerbate the stress of all involved (Karnieli-Miller et al., 2013). Children and adolescents rarely seek out mental health resources on their own, therefore their caregivers, broader family, peers, and culture play a role in cultivating expectations that can lead to or prevent treatment (Roberts, Attkisson, & Rosenblatt, 1998). In many cases, parents blame themselves for their child’s disorder, which contributes to their efficacy as a parent and affective distress (Eaton, Ohan, Stritzke, & Corrigan, 2016; Eaton, Stritzke, Corrigan, & Ohan, 2020).

Many parents, however, still see the value in early identification. For instance, studies have found that parents of Head Start children perceive screening as important and that this early identification has benefits such as closer parent-teacher relationships and better parenting (Nelson et al., 2011). Developmental risk calculator approaches must consider the bioethics behind risk communication. Culturally-informed framing that focuses on a child’s strengths, as opposed to a deficit-based approach, should be used to guide families through mental health treatment (Finlay-Jones et al., 2019). Before a risk calculator is employed, researchers should consider collecting qualitative data from individuals and/or their families to better understand the stigma surrounding mental health diagnoses, identify key barriers and motivators to seeking and receiving treatment, and understand cultural variation in norms and expectations surrounding mental health. These data can be collected through individual interviews and focus groups.

Reducing false-positives.

Given both the burden of intense assessments and the potential stigma attached to mental health problems, risk for false-positives is a large ethical concern tethered to early identification of mental health (Ozonoff, 2015). In addition to misallocation of valuable mental health resources, false-positives may cause parents undue worry and difficulty maintaining healthy relationship with their child (Hewlett & Waisbren, 2006). Developmental risk calculators that differentiate children and youth with high probability of mental health problems versus those with naturally remitting problems will reduce the likelihood of false-positives. This personalized approach is designed to limit over-pathologizing of normative variation, particularly in early development. Prevention/intervention resources may be limited, and those will benefit the most (e.g., those at highest risk) should be the receiver of such resources. The risk calculator model should have high discrimination ability to form such a subgroup (Steyerberg et al., 2013). High discrimination is also essential if prevention efforts could potentially have adverse effects on those individuals with lower risk of future diagnosis (Cannon et al., 2016). Importantly, treatment in the form of medication, when inappropriately applied, can have harmful consequences.

When resources are abundant and/or cost-effective, one may consider lowering the threshold for outcome identification, particularly if the intervention can be beneficial for all or side effects are benign (Cannon et al., 2016). For instance, strategies for directly improving self-regulation may be beneficial for all children (Smith & Dishion, 2014). Interventions centered on positive parenting in the domains of positive behavior support, monitoring and limit-setting, and family relationship building can enhance children’s self-regulation, ultimately leading to the prevention of later internalizing and externalizing disorders (Smith & Dishion, 2014). Although the concern of false-positives is an important one for reducing burden at the systems-level, interventions that improve parenting and/or promote children’s self-regulation are likely to be beneficial for all children and families regardless of risk status.

Conclusions

This conceptual paper provided a framework for incorporating a developmental perspective into risk calculator algorithms for use in a wide range of settings to promote child and adolescent mental health. We summarize the recommendations highlighted throughout the paper in Figure 1. Development and broad-based dissemination of developmental mental health risk calculators will help primary care physicians and specialty clinicians provide personalized risk estimates to their patients and their families to assist them in making complicated treatment (prevention and intervention) decisions. The risk calculator approach incorporates several key RDoC tenets (e.g., utilizing multiple units of analysis as predictors across each of the domains, examining transdiagnostic outcomes). Adopting a developmental risk calculator approach can help translate RDoC science into clinically useful metrics to strengthen: a) our understanding of how individual and environmental risk indicators predict the developmental course of mental health, b) our prevention/intervention efforts by reaching children earlier, and c) the knowledge base and decision-making for children and youth with high clinical uncertainty. A developmental risk calculator that is pragmatic, personalized, and developmentally-sensitive can optimize the characterization of neurodevelopmental vulnerability for identification of mental health risk to improve prevention and early intervention toward enhancing child and adolescent well-being.

Figure 1.

Figure 1.

Summary of recommendations for generating developmentally-informed mental health risk calculators.

Footnotes

In press: Development & Psychopathology

References

  1. Adolph KE, Robinson SR, Young JW, & Gill-Alvarez F. (2008). What is the shape of developmental change? Psychological Review, 115, 527–543. doi: 10.1037/0033-295X.115.3.527 [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Allen NB, Krefman AE, Labarthe D, Greenland P, Juonala M, Kähönen M, . . . Lloyd-Jones DM (2020). Cardiovascular health trajectories from childhood through middle age and their association with subclinical atherosclerosis. JAMA Cardiology, 5, 557–566. doi: 10.1001/jamacardio.2020.0140 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Allen NB, Siddique J, Wilkins JT, Shay C, Lewis CE, Goff DC, . . . Lloyd-Jones D. (2014). Blood pressure trajectories in early adulthood and subclinical atherosclerosis in middle age. Jama, 311, 490–497. doi: 10.1001/jama.2013.285122 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Baltes PB, & Nesselroade JR (1979). Longitudinal research in the study of behavior and development. New York, NY: Academic Press. [Google Scholar]
  5. Barrett LF (2012). Emotions are real. Emotion, 12, 413–429. doi: 10.1037/a0027555 [DOI] [PubMed] [Google Scholar]
  6. Barrett LF, Mesquita B, & Gendron M. (2011). Context in emotion perception. Current Directions in Psychological Science, 20, 286–290. [Google Scholar]
  7. Beauchaine TP (2015). Future directions in emotion dysregulation and youth psychopathology. Journal of Clinical Child and Adolescent Psychology, 44, 875–896. doi: 10.1080/15374416.2015.1038827 [DOI] [PubMed] [Google Scholar]
  8. Beauchaine TP, & Cicchetti D. (2019). Emotion dysregulation and emerging psychopathology: A transdiagnostic, transdisciplinary perspective. Developmental Psychopathology, 31, 799–804. doi: 10.1017/s0954579419000671 [DOI] [PubMed] [Google Scholar]
  9. Bell E, Bryant RA, Boyce P, Porter RJ, & Malhi GS (2021). Irritability through Research Domain Criteria: An opportunity for transdiagnostic conceptualisation. BJPsych Open, 7, e36. doi: 10.1192/bjo.2020.168 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Bernardini F, Attademo L, Cleary SD, Luther C, Shim RS, Quartesan R, & Compton MT (2017). Risk prediction models in psychiatry: Toward a new frontier for the prevention of mental illnesses. The Journal of Clinical Psychiatry, 78, 572–583. doi: 10.4088/JCP.15r10003 [DOI] [PubMed] [Google Scholar]
  11. Birmaher B, Merranko JA, Goldstein TR, Gill MK, Goldstein BI, Hower H, . . . Keller MB (2018). A risk calculator to predict the individual risk of conversion from subthreshold Bipolar symptoms to Bipolar Disorder I or II in youth. Journal of the American Academy of Child and Adolescent Psychiatry, 57, 755–763.e754. doi: 10.1016/j.jaac.2018.05.023 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Boker SM, & Nesselroade JR (2002). A method for modeling the intrinsic dynamics of intraindividual variability: Recovering the parameters of simulated oscillators in multi-wave panel data. Multivariate Behavioral Research, 37, 127–160. doi: 10.1207/s15327906mbr3701_06 [DOI] [PubMed] [Google Scholar]
  13. Brotman MA, Kircanski K, Stringaris A, Pine DS, & Leibenluft E. (2017). Irritability in youths: A translational model. American Journal of Psychiatry, 174, 520–532. doi: 10.1176/appi.ajp.2016.16070839 [DOI] [PubMed] [Google Scholar]
  14. Buss KA, Davis EL, Ram N, & Coccia M. (2018). Dysregulated fear, social inhibition, and respiratory sinus arrhythmia: A replication and extension. Child Development, 89, e214–e228. doi: 10.1111/cdev.12774 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Campbell F, Conti G, Heckman JJ, Moon SH, Pinto R, Pungello E, & Pan Y. (2014). Early childhood investments substantially boost adult health. Science, 343, 1478–1485. doi: 10.1126/science.1248429 [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Cannon TD, Cadenhead K, Cornblatt B, Woods SW, Addington J, Walker E, . . . Heinssen R. (2008). Prediction of psychosis in youth at high clinical risk: A multisite longitudinal study in North America. JAMA Psychiatry - Archives of General Psychiatry, 65, 28–37. doi: 10.1001/archgenpsychiatry.2007.3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Cannon TD, Chung Y, He G, Sun D, Jacobson A, van Erp TG, . . . Heinssen R. (2015). Progressive reduction in cortical thickness as psychosis develops: A multisite longitudinal neuroimaging study of youth at elevated clinical risk. Biological Psychiatry, 77, 147–157. doi: 10.1016/j.biopsych.2014.05.023 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Cannon TD, Yu C, Addington J, Bearden CE, Cadenhead KS, Cornblatt BA, . . . Kattan MW (2016). An individualized risk calculator for research in prodromal psychosis. The American Journal of Psychiatry, 173, 980–988. doi: 10.1176/appi.ajp.2016.15070890 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Carrión RE, Cornblatt BA, Burton CZ, Tso IF, Auther AM, Adelsheim S, . . . McFarlane WR (2016). Personalized prediction of psychosis: External validation of the NAPLS-2 psychosis risk calculator with the EDIPPP Project. The American Journal of Psychiatry, 173, 989–996. doi: 10.1176/appi.ajp.2016.15121565 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Caspi A, Houts RM, Belsky DW, Goldman-Mellor SJ, Harrington H, Israel S, . . . Moffitt TE (2014). The p Factor: One general psychopathology factor in the structure of psychiatric disorders? Clinical Psychological Science, 2, 119–137. doi: 10.1177/2167702613497473 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Caspi A, & Moffitt TE (2018). All for one and one for all: Mental disorders in one dimension. The American Journal of Psychiatry, 175, 831–844. doi: 10.1176/appi.ajp.2018.17121383 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Caye A, Agnew-Blais J, Arseneault L, Gonçalves H, Kieling C, Langley K, ... & Rohde LA (2020). A risk calculator to predict adult attention-deficit/hyperactivity disorder: generation and external validation in three birth cohorts and one clinical sample. Epidemiology and Psychiatric Sciences, 29, 111–122. doi: 10.1017/S2045796019000283 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Chandler RK, Kahana SY, Fletcher B, Jones D, Finger MS, Aklin WM, . . . Webb C. (2015). Data collection and harmonization in HIV research: The seek, test, treat, and retain initiative at the national institute on drug abuse. American Journal of Public Health, 105, 2416–2422. doi: 10.2105/ajph.2015.302788 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Ciarleglio AJ, Brucato G, Masucci MD, Altschuler R, Colibazzi T, Corcoran CM, . . . & Girgis RR (2019). A predictive model for conversion to psychosis in clinical high-risk patients. Psychological Medicine, 49, 1128–1137. doi: 10.1017/S003329171800171X [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Cicchetti D. (2008). Multi-level analysis perspective on research and development and psychopathology. In Beauchaine TP & Hinshaw SP (Eds.), Child and adolescent psychopathology (pp. 27–57). Hoboken, NJ: Wiley [Google Scholar]
  26. Cicchetti D, & Dawson G. (2002). Editorial: Multiple levels of analysis. Development and Psychopathology, 14, 417–420. doi: 10.1017/S0954579402003012 [DOI] [PubMed] [Google Scholar]
  27. Collins LM (2006). Analysis of longitudinal data: The integration of theoretical model, temporal design, and statistical model. Annual Review of Psychology, 57, 505–528. doi: 10.1146/annurev.psych.57.102904.190146 [DOI] [PubMed] [Google Scholar]
  28. Cree RA, Bitsko RH, Robinson LR, Holbrook JR, Danielson ML, Smith C, . . . Peacock G. (2018). Health care, family, and community factors associated with mental, behavioral, and developmental disorders and poverty among children aged 2–8 Years - United States, 2016. Morbidity and Mortality Weekly Report, 67, 1377–1383. doi: 10.15585/mmwr.mm6750a1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Curran PJ, & Hussong AM (2009). Integrative data analysis: The simultaneous analysis of multiple data sets. Psychological Methods, 14, 81–100. doi: 10.1037/a0015914 [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. D’Agostino RB Sr. (2012). Cardiovascular risk estimation in 2012: Lessons learned and applicability to the HIV population. The Journal of Infectious Diseases, 205, S362–367. doi: 10.1093/infdis/jis196 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. D’Agostino R. B., Jr., Griffith JL, Schmid CH, & Terrin N. (1997). Measures for evaluating model performance. In: Proceedings of the Biometrics Section, American Statistical Association, Biometrics Section Alexandria, VA, 253–258. [Google Scholar]
  32. Damme KSF, Wakschlag LS, Briggs-Gowan MJ, Norton ES, & Mittal VA (2021). Developmental patterning of irritability enhances prediction of psychopathology in pre-adolescence: Improving RDoC with developmental science. Preprint. doi: 10.1101/2020.04.30.070714 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Dawson G, Ashman SB, & Carver LJ (2000). The role of early experience in shaping behavioral and brain development and its implications for social policy. Development and Psychopathology, 12, 695–712. doi: 10.1017/s0954579400004089 [DOI] [PubMed] [Google Scholar]
  34. Dean DJ, Orr JM, Bernard JA, Gupta T, Pelletier-Baldelli A, Carol EE, & Mittal VA (2016). Hippocampal shape abnormalities predict symptom progression in neuroleptic-free youth at ultrahigh risk for psychosis. Schizophrenia Bulletin, 42, 161–169. doi: 10.1093/schbul/sbv086 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Eaton K, Ohan JL, Stritzke WGK, & Corrigan PW (2016). Failing to meet the good parent ideal: Self-stigma in parents of children with mental health disorders. Journal of Child and Family Studies, 25, 3109–3123. doi: 10.1007/s10826-016-0459-9 [DOI] [Google Scholar]
  36. Eaton K, Stritzke W, Corrigan P, & Ohan J. (2020). Pathways to self-stigma in parents of children with a mental health disorder. Journal of Child and Family Studies, 29. doi: 10.1007/s10826-019-01579-2 [DOI] [Google Scholar]
  37. Finlay‐Jones A, Varcin K, Leonard H, Bosco A, Alvares G, & Downs J. (2019). Very early identification and intervention for infants at risk of neurodevelopmental disorders: A transdiagnostic approach. Child Development Perspectives, 13, 97–103. doi: 10.1111/cdep.12319 [DOI] [Google Scholar]
  38. Fortier I, Raina P, Van den Heuvel ER, Griffith LE, Craig C, Saliba M, . . . Burton P. (2017). Maelstrom Research guidelines for rigorous retrospective data harmonization. International Journal of Epidemiology, 46, 103–105. doi: 10.1093/ije/dyw075 [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Fosco GM, Mak HW, Ramos A, LoBraico E, & Lippold M. (2019). Exploring the promise of assessing dynamic characteristics of the family for predicting adolescent risk outcomes. Journal of Child Psychology and Psychiatry, 60, 848–856. doi: 10.1111/jcpp.13052 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Fowler PJ, Tompsett CJ, Braciszewski JM, Jacques-Tiura AJ, & Baltes BB (2009). Community violence: A meta-analysis on the effect of exposure and mental health outcomes of children and adolescents. Developmental Psychopathology, 21, 227–259. doi: 10.1017/s0954579409000145 [DOI] [PubMed] [Google Scholar]
  41. Fusar-Poli P, Bonoldi I, Yung AR, Borgwardt S, Kempton MJ, Valmaggia L, . . . McGuire P. (2012). Predicting psychosis: Meta-analysis of transition outcomes in individuals at high clinical risk. JAMA Psychiatry - Archives of General Psychiatry, 69, 220–229. doi: 10.1001/archgenpsychiatry.2011.1472 [DOI] [PubMed] [Google Scholar]
  42. Fusar-Poli P, Borgwardt S, Bechdolf A, Addington J, Riecher-Rössler A, Schultze-Lutter F, . . . Yung A. (2013). The psychosis high-risk state: A comprehensive state-of-the-art review. JAMA Psychiatry, 70, 107–120. doi: 10.1001/jamapsychiatry.2013.269 [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Fusar-Poli P, Davies C, Rutigliano G, Stahl D, Bonoldi I, & McGuire P. (2019a). Transdiagnostic individualized clinically based risk calculator for the detection of individuals at risk and the prediction of psychosis: Model refinement including nonlinear effects of age. Frontiers in Psychiatry, 10, 313. doi: 10.3389/fpsyt.2019.00313 [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Fusar-Poli P, Werbeloff N, Rutigliano G, Oliver D, Davies C, Stahl D, ... & Osborn D. (2019b). Transdiagnostic risk calculator for the automatic detection of individuals at risk and the prediction of psychosis: second replication in an independent national health service trust. Schizophrenia Bulletin, 45, 562–570. doi: 10.1093/schbul/sby070 [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Fusar-Poli P, Rutigliano G, Stahl D, Davies C, Bonoldi I, Reilly T, & McGuire P. (2017). Development and validation of a clinically based risk calculator for the transdiagnostic prediction of psychosis. JAMA Psychiatry, 74, 493–500. doi: 10.1001/jamapsychiatry.2017.0284 [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Gaffrey MS, Barch DM, Bogdan R, Farris K, Petersen SE, & Luby JL (2018). Amygdala reward reactivity mediates the association between preschool stress response and depression severity. Biological Psychiatry, 83, 128–136. doi: 10.1016/j.biopsych.2017.08.020 [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Gaffrey MS, Luby JL, Botteron K, Repovš G, & Barch DM (2012). Default mode network connectivity in children with a history of preschool onset depression. Journal of Child Psychology and Psychiatry, 53, 964–972. doi: 10.1111/j.1469-7610.2012.02552.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Giedd JN, Blumenthal J, Jeffries NO, Castellanos FX, Liu H, Zijdenbos A, . . . Rapoport JL (1999). Brain development during childhood and adolescence: A longitudinal MRI study. Nature Neuroscience, 2, 861–863. doi: 10.1038/13158 [DOI] [PubMed] [Google Scholar]
  49. Gold JM, Corlett PR, Strauss GP, Schiffman J, Ellman LM, Walker EF, . . . Mittal VA (2020). Enhancing psychosis risk prediction through computational cognitive neuroscience. Schizophrenia Bulletin, 46, 1346–1352. doi: 10.1093/schbul/sbaa091 [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Goodman SH, Rouse MH, Connell AM, Broth MR, Hall CM, & Heyward D. (2011). Maternal depression and child psychopathology: A meta-analytic review. Clinical Child and Family Psychology Review, 14, 1–27. doi: 10.1007/s10567-010-0080-1 [DOI] [PubMed] [Google Scholar]
  51. Grabell AS, Li Y, Barker JW, Wakschlag LS, Huppert TJ, & Perlman SB (2018). Evidence of non-linear associations between frustration-related prefrontal cortex activation and the normal:Abnormal spectrum of irritability in young children. Journal of Abnormal Child Psychology, 46, 137–147. doi: 10.1007/s10802-017-0286-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Helms JE, Jernigan M, & Mascher J. (2005). The meaning of race in psychology and how to change it: A methodological perspective. American Psychologist, 60, 27–38. doi: 10.1037/0003-066X.60.1.27 [DOI] [PubMed] [Google Scholar]
  53. Hertzog C, & Nesselroade JR (2003). Assessing psychological change in adulthood: An overview of methodological issues. Psychology and Aging, 18, 639–657. doi: 10.1037/0882-7974.18.4.639 [DOI] [PubMed] [Google Scholar]
  54. Hewlett J, & Waisbren SE (2006). A review of the psychosocial effects of false-positive results on parents and current communication practices in newborn screening. Journal of Inherited Metabolic Disease, 29, 677–682. doi: 10.1007/s10545-006-0381-1 [DOI] [PubMed] [Google Scholar]
  55. Howell BR, Styner MA, Gao W, Yap PT, Wang L, Baluyot K, . . . Elison JT (2019). The UNC/UMN Baby Connectome Project (BCP): An overview of the study design and protocol development. Neuroimage, 185, 891–905. doi: 10.1016/j.neuroimage.2018.03.049 [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Irving J, Patel R, Oliver D, Colling C, Pritchard M, Broadbent M, ... & Fusar-Poli P. (2021). Using natural language processing on electronic health records to enhance detection and prediction of psychosis risk. Schizophrenia Bulletin, 47, 405–414. doi: 10.1093/schbul/sbaa126 [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Janssen I, Krabbendam L, Bak M, Hanssen M, Vollebergh W, de Graaf R, & van Os J. (2004). Childhood abuse as a risk factor for psychotic experiences. Acta Psychiatrica Scandinavica, 109, 38–45. doi: 10.1046/j.0001-690x.2003.00217.x [DOI] [PubMed] [Google Scholar]
  58. Jones PB (2013). Adult mental health disorders and their age at onset. The British Journal of Psychiatry Supplement, 54, s5–10. doi: 10.1192/bjp.bp.112.119164 [DOI] [PubMed] [Google Scholar]
  59. Kaat AJ, Blackwell CK, Estabrook R, Burns JL, Petitclerc A, Briggs-Gowan MJ, . . . Wakschlag LS (2019). Linking the Child Behavior Checklist (CBCL) with the Multidimensional Assessment Profile of Disruptive Behavior (MAP-DB): Advancing a dimensional spectrum approach to disruptive behavior. Journal of Child and Family Studies, 28, 343–353. doi: 10.1007/s10826-018-1272-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Karnieli-Miller O, Perlick DA, Nelson A, Mattias K, Corrigan P, & Roe D. (2013). Family members’ of persons living with a serious mental illness: Experiences and efforts to cope with stigma. Journal of Mental Health, 22, 254–262. doi: 10.3109/09638237.2013.779368 [DOI] [PubMed] [Google Scholar]
  61. Kaushik A, Kostaki E, & Kyriakopoulos M. (2016). The stigma of mental illness in children and adolescents: A systematic review. Psychiatry Research, 243, 469–494. doi: 10.1016/j.psychres.2016.04.042 [DOI] [PubMed] [Google Scholar]
  62. Keefe RS, Goldberg TE, Harvey PD, Gold JM, Poe MP, & Coughenour L. (2004). The brief assessment of cognition in schizophrenia: Reliability, sensitivity, and comparison with a standard neurocognitive battery. Schizophrenia Research, 68, 283–297. doi: 10.1016/j.schres.2003.09.011 [DOI] [PubMed] [Google Scholar]
  63. Kessel EM, Meyer A, Hajcak G, Dougherty LR, Torpey-Newman DC, Carlson GA, & Klein DN (2016). Transdiagnostic factors and pathways to multifinality: The error-related negativity predicts whether preschool irritability is associated with internalizing versus externalizing symptoms at age 9. Development and Psychopathology, 28, 913–926. doi: 10.1017/S0954579416000626 [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Kim YS, & State MW (2014). Recent challenges to the psychiatric diagnostic nosology: A focus on the genetics and genomics of neurodevelopmental disorders. International Journal of Epidemiology, 43, 465–475. doi: 10.1093/ije/dyu037 [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. Kim-Cohen J, Caspi A, Moffitt TE, Harrington H, Milne BJ, & Poulton R. (2003). Prior juvenile diagnoses in adults with mental disorder: Developmental follow-back of a prospective-longitudinal cohort. JAMA Psychiatry - Archives of General Psychiatry, 60, 709–717. doi: 10.1001/archpsyc.60.7.709 [DOI] [PubMed] [Google Scholar]
  66. Kochanska G, & Kim S. (2013). Difficult temperament moderates links between maternal responsiveness and children’s compliance and behavior problems in low-income families. Journal of Child Psychology and Psychiatry, 54, 323–332. doi: 10.1111/jcpp.12002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. König IR, Malley JD, Weimar C, Diener HC, & Ziegler A. (2007). Practical experiences on the necessity of external validation. Statistics in Medicine, 26, 5499–5511. doi: 10.1002/sim.3069 [DOI] [PubMed] [Google Scholar]
  68. Kotov R, Krueger RF, Watson D, Achenbach TM, Althoff RR, Bagby RM, . . . Zimmerman M. (2017). The Hierarchical Taxonomy Of Psychopathology (HiTOP): A dimensional alternative to traditional nosologies. Journal of Abnormal Psychology, 126, 454–477. doi: 10.1037/abn0000258 [DOI] [PubMed] [Google Scholar]
  69. Lee TY, Hwang WJ, Kim NS, Park I, Lho SK, Moon SY, ... & Kwon JS (2020). Prediction of psychosis: model development and internal validation of a personalized risk calculator. Psychological Medicine. doi: 10.1017/S0033291720004675 [DOI] [PMC free article] [PubMed] [Google Scholar]
  70. Liu C, Moore GA, Beekman C, Pérez-Edgar KE, Leve LD, Shaw DS, . . . Neiderhiser JM (2018). Developmental patterns of anger from infancy to middle childhood predict problem behaviors at age 8. Developmental Psychology, 54, 2090–2100. doi: 10.1037/dev0000589 [DOI] [PMC free article] [PubMed] [Google Scholar]
  71. Lloyd-Jones DM (2010). Cardiovascular risk prediction: Basic concepts, current status, and future directions. Circulation, 121, 1768–1777. doi: 10.1161/circulationaha.109.849166 [DOI] [PubMed] [Google Scholar]
  72. Luby J, Allen N, Estabrook R, Pine DS, Rogers C, Krogh-Jespersen S, . . . Wakschlag L. (2019). Mapping infant neurodevelopmental precursors of mental disorders: How synthetic cohorts & computational approaches can be used to enhance prediction of early childhood psychopathology. Behaviour Research and Therapy, 123, 103484. doi: 10.1016/j.brat.2019.103484 [DOI] [PMC free article] [PubMed] [Google Scholar]
  73. Luby JL (2012). Dispelling the “they’ll grow out of it” myth: Implications for intervention. The American Journal of Psychiatry, 169, 1127–1129. doi: 10.1176/appi.ajp.2012.12081037 [DOI] [PubMed] [Google Scholar]
  74. MacNeill LA, Ram N, Bell MA, Fox NA, & Pérez-Edgar K. (2018). Trajectories of infants’ biobehavioral development: Timing and rate of A-Not-B performance gains and EEG maturation. Child Development, 89, 711–724. doi: 10.1111/cdev.13022 [DOI] [PMC free article] [PubMed] [Google Scholar]
  75. Marceau K, Ram N, Houts RM, Grimm KJ, & Susman EJ (2011). Individual differences in boys’ and girls’ timing and tempo of puberty: Modeling development with nonlinear growth models. Developmental Psychology, 47, 1389–1409. doi: 10.1037/a0023838 [DOI] [PMC free article] [PubMed] [Google Scholar]
  76. Martel MM, Pan PM, Hoffmann MS, Gadelha A, do Rosário MC, Mari JJ, . . . Salum GA (2017). A general psychopathology factor (P factor) in children: Structural model analysis and external validation through familial risk and child global executive function. Journal of Abnormal Psychology, 126, 137–148. doi: 10.1037/abn0000205 [DOI] [PubMed] [Google Scholar]
  77. McDaid D, Park AL, & Wahlbeck K. (2019). The economic case for the prevention of mental illness. Annual Review of Public Health, 40, 373–389. doi: 10.1146/annurev-publhealth-040617-013629 [DOI] [PubMed] [Google Scholar]
  78. McGlashan T, Walsh B, & Woods SW (2010). The psychosis-risk syndrome: handbook for diagnosis and follow-up: Oxford University Press. [Google Scholar]
  79. McGorry PD, Purcell R, Goldstone S, & Amminger GP (2011). Age of onset and timing of treatment for mental and substance use disorders: Implications for preventive intervention strategies and models of care. Curr Opin Psychiatry, 24, 301–306. doi: 10.1097/YCO.0b013e3283477a09 [DOI] [PubMed] [Google Scholar]
  80. Meehan AJ, Latham RM, Arseneault L, Stahl D, Fisher HL, & Danese A. (2020). Developing an individualized risk calculator for psychopathology among young people victimized during childhood: A population-representative cohort study. Journal of Affective Disorders, 262, 90–98. doi: 10.1016/j.jad.2019.10.034 [DOI] [PMC free article] [PubMed] [Google Scholar]
  81. Michelini G, Gair K, Tian Y, Dougherty LR, Goldstein BL, MacNeill LA . . . Kotov R (under review). Do general and specific factors of preschool psychopathology predict preadolescent outcomes? A transdiagnostic hierarchical approach. [DOI] [PMC free article] [PubMed] [Google Scholar]
  82. Mittal VA, & Wakschlag LS (2017). Research domain criteria (RDoC) grows up: Strengthening neurodevelopment investigation within the RDoC framework. Journal of Affective Disorders, 216, 30–35. doi: 10.1016/j.jad.2016.12.011 [DOI] [PMC free article] [PubMed] [Google Scholar]
  83. Mittal VA, & Walker EF (2019). Advances in the neurobiology of stress and psychosis. Schizophrenia Research, 213, 1–5. doi: 10.1016/j.schres.2019.08.030 [DOI] [PubMed] [Google Scholar]
  84. Morawetz C, Riedel M, Salo T, Berboth S, Eickhoff S, Laird A, & Kohn N. (2020). Multiple large-scale neural networks underlying emotion regulation. Neuroscience & Biobehavioral Reviews, 116, 382–395. doi: 10.1016/j.neubiorev.2020.07.001 [DOI] [PubMed] [Google Scholar]
  85. Nagin D. (2005). Group-based modeling of development Harvard University Press. [Google Scholar]
  86. Nelson BB, Chung PJ, DuPlessis HM, Flores L, Ryan GW, & Kataoka SH (2011). Strengthening families of children with developmental concerns: Parent perceptions of developmental screening and services in Head Start. Ethnicity & Disease, 21, S1–89-93. [PMC free article] [PubMed] [Google Scholar]
  87. Nielsen A, Wakschlag L, & Norton E (under review). Linking irritability and functional brain networks: A transdiagnostic case for expanding consideration of development and environment in RDoC. [DOI] [PMC free article] [PubMed] [Google Scholar]
  88. Ning H, Krefman A, Zhao L, Wilkins JT, Lloyd-Jones DM, Siddique J, & Allen NB (in press.). Development and validation of a large synthetic cohort for the study of cardiovascular health across the lifespan. In Press. American Journal Experts. [DOI] [PMC free article] [PubMed] [Google Scholar]
  89. Norton ES, MacNeill LA, Harriott EM, Allen NB, Krogh-Jespersen S, Smyser CD, . . . Wakschlag LS (under review.). EEG/ERP as a pragmatic method to expand the reach of infant-toddler neuroimaging in HBCD: Promises and challenges. [DOI] [PMC free article] [PubMed] [Google Scholar]
  90. Obermeyer Z, Powers B, Vogeli C, & Mullainathan S. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366, 447–453. doi: 10.1126/science.aax2342 [DOI] [PubMed] [Google Scholar]
  91. Ofonedu ME, Belcher HME, Budhathoki C, & Gross DA (2017). Understanding barriers to initial treatment engagement among underserved families seeking mental health services. Journal of Child and Family Studies, 26, 863–876. doi: 10.1007/s10826-016-0603-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  92. Oliver D, Spada G, Colling C, Broadbent M, Baldwin H, Patel R, . . . Fusar-Poli P. (2021). Real-world implementation of precision psychiatry: Transdiagnostic risk calculator for the automatic detection of individuals at-risk of psychosis. Schizophrenia Research, 227, 52–60. doi: 10.1016/j.schres.2020.05.007 [DOI] [PMC free article] [PubMed] [Google Scholar]
  93. Oliver D, Wong CMJ, Bøg M, Jönsson L, Kinon B, Wehnert A, . . . Fusar-Poli P. (2020). Transdiagnostic individualized clinically-based risk calculator for the automatic detection of individuals at-risk and the prediction of psychosis: External replication in 2,430,333 US patients. Translational Psychiatry, 10, 364. doi: 10.1038/s41398-020-01032-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  94. Osborne KJ, & Mittal VA (2019). External validation and extension of the NAPLS-2 and SIPS-RC personalized risk calculators in an independent clinical high-risk sample. Psychiatry Research, 279, 9–14. doi: 10.1016/j.psychres.2019.06.034 [DOI] [PMC free article] [PubMed] [Google Scholar]
  95. Ozonoff S. (2015). Editorial: Early detection of mental health and neurodevelopmental disorders: The ethical challenges of a field in its infancy. Journal of Child Psychology and Psychiatry, 56, 933–935. doi: 10.1111/jcpp.12452 [DOI] [PubMed] [Google Scholar]
  96. Patel PK, Leathem LD, Currin DL, & Karlsgodt KH (2021). Adolescent neurodevelopment and vulnerability to psychosis. Biological Psychiatry, 89, 184–193. doi: 10.1016/j.biopsych.2020.06.028 [DOI] [PMC free article] [PubMed] [Google Scholar]
  97. Pencina MJ, & D’Agostino RB Sr. (2012). Thoroughly modern risk prediction? Science Translational Medicine, 4, 131fs110. doi: 10.1126/scitranslmed.3004127 [DOI] [PubMed] [Google Scholar]
  98. Pencina MJ, D’Agostino RB Sr., D’Agostino RB Jr., & Vasan RS (2008). Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond. Statistics in Medicine, 27, 157–172; discussion 207–112. doi: 10.1002/sim.2929 [DOI] [PubMed] [Google Scholar]
  99. Perlman SB, Jones BM, Wakschlag LS, Axelson D, Birmaher B, & Phillips ML (2015). Neural substrates of child irritability in typically developing and psychiatric populations. Developmental Cognitive Neuroscience, 14, 71–80. doi: 10.1016/j.dcn.2015.07.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  100. Pool LR, Ning H, Wilkins J, Lloyd-Jones DM, & Allen NB (2018). Use of long-term cumulative blood pressure in cardiovascular risk prediction models. JAMA Cardiology, 3, 1096–1100. doi: 10.1001/jamacardio.2018.2763 [DOI] [PMC free article] [PubMed] [Google Scholar]
  101. Rajji TK, Ismail Z, & Mulsant BH (2009). Age at onset and cognition in schizophrenia: Meta-analysis. British Journal of Psychiatry, 195, 286–293. doi: 10.1192/bjp.bp.108.060723 [DOI] [PubMed] [Google Scholar]
  102. Ram N, & Gerstorf D. (2009). Time-structured and net intraindividual variability: Tools for examining the development of dynamic characteristics and processes. Psychology and Aging, 24, 778–791. doi: 10.1037/a0017915 [DOI] [PMC free article] [PubMed] [Google Scholar]
  103. Ram N, & Grimm KJ (2007). Using simple and complex growth models to articulate developmental change: Matching theory to method. International Journal of Behavioral Development, 31, 303–316. doi: 10.1177/0165025407077751 [DOI] [Google Scholar]
  104. Reardon T, Harvey K, Baranowska M, O’Brien D, Smith L, & Creswell C. (2017). What do parents perceive are the barriers and facilitators to accessing psychological treatment for mental health problems in children and adolescents? A systematic review of qualitative and quantitative studies. European Child & Adolescent Psychiatry, 26, 623–647. doi: 10.1007/s00787-016-0930-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  105. Roberts RE, Attkisson CC, & Rosenblatt A. (1998). Prevalence of psychopathology among children and adolescents. The American Journal of Psychiatry, 155, 715–725. doi: 10.1176/ajp.155.6.715 [DOI] [PubMed] [Google Scholar]
  106. Rocha TBM, Fisher HL, Caye A, Silva LADD, Arseneault L, Barros FCLFD, ... & Kieling, C. C. (2021). Identifying adolescents at risk for depression: A prediction score performance in cohorts based in 3 different continents. Journal of the American Academy of Child and Adolescent Psychiatry, 60, 262–273. doi: 10.1016/j.jaac.2019.12.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  107. Rogers CE, Sylvester CM, Mintz C, Kenley JK, Shimony JS, Barch DM, & Smyser CD (2017). Neonatal amygdala functional connectivity at rest in healthy and preterm infants and early internalizing symptoms. Journal of the American Academy of Child and Adolescent Psychiatry, 56, 157–166. doi: 10.1016/j.jaac.2016.11.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
  108. Rothbart MK, & Bates JE (1998). Temperament. In Handbook of child psychology: Social, emotional, and personality development, Vol. 3, 5th ed. (5 ed., Vol. 3, pp. 105–176). Hoboken, NJ, US: John Wiley & Sons, Inc. [Google Scholar]
  109. Rutter M. (1987). Psychosocial resilience and protective mechanisms. The American journal of orthopsychiatry, 57, 316–331. doi: 10.1111/j.1939-0025.1987.tb03541.x [DOI] [PubMed] [Google Scholar]
  110. Sameroff A, Seifer R, & McDonough SC (2004). Contextual contributors to the assessment of infant mental health. In Handbook of infant, toddler, and preschool mental health assessment. (pp. 61–76). New York, NY, US: Oxford University Press. [Google Scholar]
  111. Sameroff AJ, Seifer R, Barocas R, Zax M, & Greenspan S. (1987). Intelligence quotient scores of 4-year-old children: Social-environmental risk factors. Pediatrics, 79, 343–350. [PubMed] [Google Scholar]
  112. Scott K, & Schulz L. (2017). Lookit (Part 1): A new online platform for developmental research. Open Mind, 1, 4–14. doi: 10.1162/OPMI_a_00002 [DOI] [Google Scholar]
  113. Shaw DS, & Taraban L. (2016). New directions and challenges in preventing conduct problems in early childhood. Child Development Perspectives, 11. doi: 10.1111/cdep.12212 [DOI] [PMC free article] [PubMed] [Google Scholar]
  114. Sheldrick RC, Merchant S, & Perrin EC (2011). Identification of developmental-behavioral problems in primary care: A systematic review. Pediatrics, 128, 356–363. doi: 10.1542/peds.2010-3261 [DOI] [PubMed] [Google Scholar]
  115. Shonkoff JP (2010). Building a new biodevelopmental framework to guide the future of early childhood policy. Child Development, 81, 357–367. doi: 10.1111/j.1467-8624.2009.01399.x [DOI] [PubMed] [Google Scholar]
  116. Shonkoff JP, Boyce WT, Levitt P, Martinez FD, & McEwen B. (2021). Leveraging the biology of adversity and resilience to transform pediatric practice. Pediatrics, 147. doi: 10.1542/peds.2019-3845 [DOI] [PubMed] [Google Scholar]
  117. Siddique J, Reiter JP, Brincks A, Gibbons RD, Crespi CM, & Brown CH (2015). Multiple imputation for harmonizing longitudinal non-commensurate measures in individual participant data meta-analysis. Statistics in Medicine, 34, 3399–3414. doi: 10.1002/sim.6562 [DOI] [PMC free article] [PubMed] [Google Scholar]
  118. Silva Ribeiro J, Pereira D, Salagre E, Coroa M, Santos Oliveira P, Santos V, ... & Vieta E. (2020). Risk calculators in bipolar disorder: A systematic review. Brain Sciences, 10, 525. doi: 10.3390/brainsci10080525 [DOI] [PMC free article] [PubMed] [Google Scholar]
  119. Simon R. (2006). A checklist for evaluating reports of expression profiling for treatment selection. Clinical Advances in Hematology and Oncology, 4, 219–224. [PubMed] [Google Scholar]
  120. Smith JD, Berkel C, Jordan N, Atkins DC, Narayanan SS, Gallo C, . . . Bruening MM (2018). An individually tailored family-centered intervention for pediatric obesity in primary care: Study protocol of a randomized type II hybrid effectiveness-implementation trial (Raising Healthy Children study). Implementation Science, 13, 11. doi: 10.1186/s13012-017-0697-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  121. Smith JD, Cruden GH, Rojas LM, Van Ryzin M, Fu E, Davis MM, . . . Brown CH (2020). Parenting interventions in pediatric Primary care: A systematic review. Pediatrics, 146. doi: 10.1542/peds.2019-3548 [DOI] [PMC free article] [PubMed] [Google Scholar]
  122. Smith JD, & Dishion TJ (2014). Mindful parenting in the development and maintenance of youth psychopathology. In Transdiagnostic treatments for children and adolescents: Principles and practice. (pp. 138–158). New York, NY, US: The Guilford Press. [Google Scholar]
  123. Smith JD, Wakschlag L, Krogh-Jespersen S, Walkup JT, Wilson MN, Dishion TJ, & Shaw DS (2019). Dysregulated irritability as a window on young children’s psychiatric risk: Transdiagnostic effects via the family check-up. Development and Psychopathology, 31, 1887–1899. doi: 10.1017/s0954579419000816 [DOI] [PMC free article] [PubMed] [Google Scholar]
  124. Sroufe LA (1990). Considering normal and abnormal together: The essence of developmental psychopathology. Development and Psychopathology, 2, 335–347. doi: 10.1017/S0954579400005769 [DOI] [Google Scholar]
  125. Steyerberg EW, Moons KG, van der Windt DA, Hayden JA, Perel P, Schroter S, . . . Altman DG (2013). Prognosis Research Strategy (PROGRESS) 3: Prognostic model research. PLOS Medicine, 10, e1001381. doi: 10.1371/journal.pmed.1001381 [DOI] [PMC free article] [PubMed] [Google Scholar]
  126. Studerus E, Beck K, Fusar-Poli P, & Riecher-Rössler A. (2020). Development and validation of a dynamic risk prediction model to forecast psychosis onset in patients at clinical high risk. Schizophrenia Bulletin, 46, 252–260. doi: 10.1093/schbul/sbz059 [DOI] [PMC free article] [PubMed] [Google Scholar]
  127. Teti DM, O’Connell MA, & Reiner CD (1996). Parenting sensitivity, parental depression and child health: The mediational role of parental self‐efficacy. Early Development and Parenting: An International Journal of Research and Practice, 5, 237–250. doi: 10.1002/(sici)1099-0917(199612)5:4 [DOI] [Google Scholar]
  128. Vargas T, Conley RE, & Mittal VA (2020). Chronic stress, structural exposures and neurobiological mechanisms: A stimulation, discrepancy and deprivation model of psychosis. International Review of Neurobiology, 152, 41–69. doi: 10.1016/bs.irn.2019.11.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  129. Vargas T, & Mittal VA (2018). Issues affecting reliable and valid assessment of early life stressors in psychosis. Schizophrenia Research, 192, 465–466. doi: 10.1016/j.schres.2017.04.021 [DOI] [PubMed] [Google Scholar]
  130. Vargas T, Zou DS, Conley RE, & Mittal VA (2019). Assessing developmental environmental risk factor exposure in clinical high risk for psychosis individuals: Preliminary results using the individual and structural exposure to stress in psychosis-risk states scale. Journal of Clinical Medicine, 8. doi: 10.3390/jcm8070994 [DOI] [PMC free article] [PubMed] [Google Scholar]
  131. Wakschlag LS, Estabrook R, Petitclerc A, Henry D, Burns JL, Perlman SB, . . . Briggs-Gowan ML (2015). Clinical implications of a dimensional approach: The normal:Abnormal spectrum of early irritability. Journal of the American Academy of Child and Adolescent Psychiatry, 54, 626–634. doi: 10.1016/j.jaac.2015.05.016 [DOI] [PMC free article] [PubMed] [Google Scholar]
  132. Wakschlag LS, Luby JL, Pool LR, MacNeill LA, Adam H, Barch DM, . . . Allen NB (under review). Closing the research-to-practice gap in early identification: Proof-of-concept for a personalized early childhood mental health risk calculator. [Google Scholar]
  133. Wakschlag LS, Perlman SB, Blair RJ, Leibenluft E, Briggs-Gowan MJ, & Pine DS (2018). The neurodevelopmental basis of early childhood disruptive behavior: Irritable and callous phenotypes as exemplars. The American Journal of Psychiatry, 175, 114–130. doi: 10.1176/appi.ajp.2017.17010045 [DOI] [PMC free article] [PubMed] [Google Scholar]
  134. Wakschlag LS, Roberts MY, Flynn RM, Smith JD, Krogh-Jespersen S, Kaat AJ, . . . Davis MM (2019). Future directions for early childhood prevention of mental disorders: A road map to mental health, earlier. Journal of Clinical Child & Adolescent Psychology, 48, 539–554. doi: 10.1080/15374416.2018.1561296 [DOI] [PMC free article] [PubMed] [Google Scholar]
  135. Wakschlag LS, Tolan PH, & Leventhal BL (2010). Research Review: ‘Ain’t misbehavin’: Towards a developmentally-specified nosology for preschool disruptive behavior. Journal of Child Psychology and Psychiatry, 51, 3–22. doi: 10.1111/j.1469-7610.2009.02184.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  136. Walker E, Trotman H, Goulding S, Holtzman C, Ryan A, Macdonald A, . . . Brasfield J. (2013). Developmental mechanisms in the prodrome to psychosis. Development and Psychopathology, 25, 1585–1600. doi: 10.1017/S0954579413000783 [DOI] [PMC free article] [PubMed] [Google Scholar]
  137. Walkup JT, Mathews T, & Green CM (2017). Transdiagnostic behavioral therapies in pediatric primary care: Looking ahead. JAMA Psychiatry, 74, 557–558. doi: 10.1001/jamapsychiatry.2017.0448 [DOI] [PubMed] [Google Scholar]
  138. Waller R, Gardner F, Viding E, Shaw DS, Dishion TJ, Wilson MN, & Hyde LW (2014). Bidirectional associations between parental warmth, callous unemotional behavior, and behavior problems in high-risk preschoolers. Journal of Abnormal Child Psychology, 42, 1275–1285. doi: 10.1007/s10802-014-9871-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  139. Wiggins JL, Briggs-Gowan MJ, Brotman MA, Leibenluft E, & Wakschlag LS (2020). Toward a developmental nosology for disruptive mood dysregulation Disorder in early childhood. Journal of the American Academy of Child and Adolescent Psychiatry, 60, 388–397. doi: 10.1016/j.jaac.2020.04.015 [DOI] [PMC free article] [PubMed] [Google Scholar]
  140. Wiggins JL, Briggs-Gowan MJ, Estabrook R, Brotman MA, Pine DS, Leibenluft E, & Wakschlag LS (2018). Identifying clinically significant irritability in early childhood. Journal of the American Academy of Child & Adolescent Psychiatry, 57, 191–199. e192. doi: 10.1016/j.jaac.2017.12.008 [DOI] [PMC free article] [PubMed] [Google Scholar]
  141. Wiggins JL, Ureña Rosario A, Krogh-Jespersen S, MacNeill LA, Walkup JT, Smith JD, & Wakschlag LS (under review). Advancing early irritability as a pragmatic, transdiagnostic marker of early-onset, chronic mental disorder: Prevalence, stability, and predictive utility. [Google Scholar]
  142. Wiggins JL, Rosario A, Krogh-Jespersen S, Briggs-Gowan MJ, Smith JD, & Wakschlag L (under review.). Advancing early irritability as a transdiagnostic marker of mental disorder via a pragmatic approach: Prevalence, stability, and predictive utility. Submitted manuscript. [Google Scholar]
  143. Wilson-Mendenhall CD, Barrett LF, Simmons WK, & Barsalou LW (2011). Grounding emotion in situated conceptualization. Neuropsychologia, 49, 1105–1127. doi: 10.1016/j.neuropsychologia.2010.12.032 [DOI] [PMC free article] [PubMed] [Google Scholar]
  144. Worthington MA, Walker EF, Addington J, Bearden CE, Cadenhead KS, Cornblatt BA, . . . Cannon TD (2021). Incorporating cortisol into the NAPLS2 individualized risk calculator for prediction of psychosis. Schizophrenia Research, 227, 95–100. doi: 10.1016/j.schres.2020.09.022 [DOI] [PMC free article] [PubMed] [Google Scholar]
  145. Yates TM, Egeland B, & Sroufe LA (2003). Rethinking resilience: A developmental process perspective. In Resilience and vulnerability: Adaptation in the context of childhood adversities. (pp. 243–266). New York, NY, US: Cambridge University Press. [Google Scholar]
  146. Zhang T, Li H, Tang Y, Niznikiewicz MA, Shenton ME, Keshavan MS, . . . Wang J. (2018). Validating the predictive accuracy of the NAPLS-2 psychosis risk calculator in a clinical high-risk sample from the SHARP (Shanghai At Risk for Psychosis) program. The American Journal of Psychiatry, 175, 906–908. doi: 10.1176/appi.ajp.2018.18010036 [DOI] [PubMed] [Google Scholar]
  147. Zhang T, Xu L, Tang Y, Li H, Tang X, Cui H, . . . Wang J. (2019). Prediction of psychosis in prodrome: Development and validation of a simple, personalized risk calculator. Psychological Medicine, 49, 1990–1998. doi: 10.1017/s0033291718002738 [DOI] [PubMed] [Google Scholar]

RESOURCES