Causation and prediction in child and adolescent mental health research

Abstract

There have been recent advances in prediction model research in psychiatry. Several validated prediction models are available to support the diagnosis and prognosis of mental health conditions. Prediction model research is translational in nature and the translation pathway includes well conducted epidemiological studies. These studies provide essential information needed to develop new prediction models. Despite the recent advances in prediction model research in psychiatry, there are still important limitations that hamper translation into health care. Comprehensive and transparent reporting as well as an increased focus on external validation, implementation research and stakeholder involvement may help close the translational gap.

EDITORIAL

One fundamental task of evidence‐based child and adolescent mental health research is to identify the underlying causes of mental health conditions. As randomized clinical trials, the gold standard approach for addressing confounding and establishing causal effects, are unfeasible or unethical in many situations, observational research designs are often used in causal studies (Ohlsson & Kendler, 2020). Causal studies using observational designs typically test one or more pre‐specified hypotheses and several pre‐specified models about causal relationships, while controlling for potential confounding factors. The papers in the July issue of Journal of Child and Adolescent Psychology and Psychiatry (JCPP) Advances from Gustavson Kristin, (2021) and Myers Lynnea, (2021) and the commentary by Taylor and Mark, (2021) draw important attention to the critical issue of unmeasured (e.g., familial) confounding in causal studies and the added value of using genetically‐informative study designs, such as sibling comparison studies (Ohlsson & Kendler, 2020).

Another overarching goal of evidence‐based child and adolescent mental health research is to establish accurate prediction of diagnosis, outcomes and treatment response. Investigating prediction and causation (as described above) involves different sets of research questions. In contrast to causal studies, there is no hypothesis‐testing in studies of prediction, and no need to control for confounding. Furthermore, while causal studies typically test many different models to explore confounding as well as possible mediation and effect modification that together may provide support for a causal hypothesis, prediction studies typically focus on identifying one optimal prediction model for a pre‐specified purpose. Prediction models are generally developed with the aim to support health care providers in estimating the risk of a specific condition being present (diagnostic models) or of a specific outcome occurring in the future (prognostic models) (Collins, Reitsma, Altman, & Moons, 2015). Other, more specific aims of prediction model studies, when applied to mental health conditions, involves developing clinical prediction tools capable of providing more precise psychiatric diagnosis, establishing accurate prognosis about future outcomes, and predicting response to pharmacological and non‐pharmacological treatment.

Useful prediction models have been available for years in some areas of medicine, such as in cardiovascular medicine (e.g., Framingham Risk Score), while prediction models in psychiatry have advanced only quite recently. To date, there are validated prediction models available to support the diagnosis and prognosis of mental health conditions, (e.g., psychosis) and to predict treatment response (Salazar de Pablo et al., 2021). Nevertheless, important knowledge gaps remain to be addressed and there is still a lack of clinical translation. The ambition with this editorial is to highlight the main findings from three publications in the July issue of JCPP Advances and discuss these in the context of the overarching goals of contemporary prediction model research.

Identify factors that predict public health relevant outcomes (prognostic models). Two of the papers in the July issue of JCPP Advances represent useful examples of how well‐conducted epidemiological studies can inform the field about public‐health relevant outcomes in particular high‐risk groups (e.g., those experiencing victimisation or COVID‐19 school lockdown).

First, the publication by Armitage, (2021) in the July issue of JCPP Advances focused on moderators of adult wellbeing following victimisation. They used data from the Avon Longitudinal Study of Parents and Children (ALSPAC) on depressive symptoms and wellbeing at age 23 years, as well as data on victimisation in adolescence, and 15 protective factors across development. Of the 15 protective factors (including different indicators at the individual‐, family‐, and peer‐level), perceived childhood scholastic competence was the only factor that mitigated some of the negative effects of victimisation. The study by Armitage, (2021) is one of the first to explore a wide range of protective factors in predicting adult mental health and wellbeing following victimisation. An increased focus on protective factors is an important goal for future research in child and adolescent mental health conditions.

Second, the timely and important study by Mansfield, (2021) provides insight into the mental health and wellbeing of different groups during the partial school closures of the first UK COVID‐19 school lockdown in 2020. This large‐scale, cross‐sectional study (over 11,000 pupils) surveyed school pupils across Southern England. Among many things, this study found that pupils most likely to report deteriorations in wellbeing were female, had reported socio‐economic deprivation, had previous mental health support or had upcoming examinations.

The two papers discussed above have several potential clinical implications in the area of victimization (e.g., school‐based interventions aimed at increasing perceptions of scholastic competence may reduce the burden of victimisation on later wellbeing) and COVID‐19 school lockdowns (e.g., the risk groups identified would benefit from a broad curriculum of support for education and wellbeing), but they also represent examples of how epidemiology studies can be used to build prognostic models. It is increasingly recognized that prediction model research typically progresses on a translational pathway from basic science to health care implementation. Basic information about the distribution and effect size of potential predictors are essential in the process of developing clinically useful prognostic models (Delgadillo & Lutz, 2020).

Identify factors that predict diagnosis (diagnostic models). The publication by Stroth, (2021) in the July issue of JCPP Advances applies a diagnostic prediction modelling framework for Autism Spectrum Disorder (ASD). This is an important step towards addressing the urgent need for tools enabling health care professionals in the primary care sector to identify children for referral to ASD specialists. Diagnosing ASD is complex and time consuming, especially in children who have verbal difficulties, and “delayed diagnosis” is a critical problem. Knowledge of the most discriminative features may therefore help in the development of training tools that assist clinicians to identify suspicion of ASD in nonverbal and minimally verbal children. Based on machine learning strategies, Stroth, (2021) analysed items from a behavioural observation (ADOS) and a clinical anamnestic interview (ADI‐R) in a well‐characterized clinical population of children. The findings indicate that focussing attention on just a few diagnostic features may yield sufficiently high quality in the classification decision compared to the full item set contained in ADOS and ADI‐R. The authors stress the importance of future research to break down these most discriminative subsets of diagnostic items in more detail and translate them into a training tool for clinicians. One critical limitation that the authors highlight in the manuscript is the need of external replication in an independent sample, ideally using a large sample. Small sample size and lack of external validation is a general problem in prediction model research that needs to be addressed in future research.

Conclusions and future directions for prediction model research. In psychiatry research, increased efforts have recently been devoted to the development of new prediction models. Several validated prediction models are available to support the diagnosis and prognosis of mental health conditions. Prediction model research is translational in nature and includes well‐conducted epidemiology studies. These studies provide essential building‐blocks for developing prediction models. Despite the recent advances in “psychiatric” prediction model research, there are still important limitations that hampers translation into health care.

First, it is well‐established that the quality of reporting of prediction model studies is poor (Collins et al., 2015). Comprehensive and transparent reporting of information on all aspects of the prediction model is needed to assess risk of bias and the potential usefulness of prediction models. Recommendations for the reporting of studies a multivariable prediction model are already in place (TRIPOD) (Collins et al., 2015).

Second, there has been an increased interest in using machine learning approaches in prediction model research (Fusar‐Poli, Hijazi, Stahl, & Steyerberg, 2018). This is a promising direction in prediction modelling research. The increased availability of “big data” (e.g., large‐scale cohort data, electronic health record databases, large‐scale detated multi‐site data) combined with sophisticated machine learning approaches is probably one explanation for the recent advances. Nevertheless, prediction modelling using machine learning is a relatively new direction in psychiatry and it remains to be established if such advanced analytical methods will outperform “classic” types of prediction models (Salazar de Pablo et al., 2021). It also needs to be highlighted that comprehensive and transparent reporting is even more important with complex models.

Third, prediction models tend to perform better in the sample in which they were created and a critical measure is therefore how the model performs in an independent sample (Fusar‐Poli et al., 2018). Unfortunately, only a small proportion of the total prediction models developed in the context of psychiatry are externally validated (Salazar de Pablo et al., 2021). The field probably needs to shift focus, at least to some extent, from developing new predictions models to external validation and updating existing models.

Fourth, only a very small proportion of the total prediction models developed in psychiatry is considered for clinical implementation (Salazar de Pablo et al., 2021). An increased focus on implementation research is needed to address this translational gap. A stronger link to real clinical needs may also help address this translational gap. Already at the design stage of a prediction model study, the research team need to define; who will use, how to use, and when to use the proposed prediction model. Early stakeholder (e.g., patients, providers, payers, and policymakers) involvement may help in this translational process. For example, people with lived experience have particular insights into their difficulties that researcher might miss. Researchers need to ask what stakeholders want and work towards translating those requests into research and subsequently into outcomes that improve people's lives.

Addressing the above limitations should hopefully help in generating more successful examples of prediction models in psychiatry that travels the full pathway from basic science to health care implementation.

2. CONFLICTS OF INTEREST

Henrik Larsson reports receiving grants from Shire Pharmaceuticals; personal fees from and serving as a speaker for Medice, Shire/Takeda Pharmaceuticals and Evolan Pharma AB outside the submitted work; and sponsorship for a conference on attention‐deficit/hyperactivity disorder from Shire Pharmaceuticals outside the submitted work. He is Editor‐in‐Chief of JCPP Advances. [Corrections made on 22 June 2022, after first online publication: This Conflicts of Interest statement has been updated in this version.]