Abstract
Background
Despite efforts to predict suicide risk in children, the ability to reliably identify who will engage in suicide thoughts or behaviours (STB) has remained unsuccessful.
Aims
We apply a novel machine learning approach and examine whether children with STB could be differentiated from children without STB based on a combination of traditional (sociodemographic, physical health, social environmental, clinical psychiatric) risk factors, but also more novel risk factors (cognitive, neuroimaging and genetic characteristics).
Method
The study sample included 5,885 unrelated children (50% female, 67% white, 9-11 years) from the Adolescent Brain Cognitive Development (ABCD) study. We performed penalized logistic regression analysis to distinguish between: 1. Children with current or past STB, 2. Children with a mental illness but no STB (clinical controls), and 3. Healthy control children (no STB and no history of mental illness). The model was subsequently validated on 7 independent sites of the ABCD study (N=1712).
Results
Results showed that we were able to distinguish the STB group from healthy controls and clinical controls (AUROC: 0.79-0.81 and 0.70-0.78 respectively). However, we could not distinguish children with suicidal ideation from those who attempted suicide (AUROC: 0.49-0.59). The factors that differentiated the STB group from the clinical control group included family conflict, prodromal psychosis symptoms, impulsivity, depression severity and history of mental health treatment.
Conclusions
This work highlights that mostly clinical psychiatric factors were able to distinguish children with STB from children without STB. Future research is needed to determine if these variables prospectively predict subsequent suicidal behaviour.
Keywords: suicide, youth, machine learning, children, penalized logistic regression
Introduction
Suicidal thoughts and behaviours in adolescence
Despite national and international prevention efforts aimed at reducing suicide risk, the rate of suicide still continues to rise globally (1). Suicidal thoughts and behaviours typically emerge during adolescence, and their incidence rates rise sharply from childhood to adolescence (2). Suicide is the second leading cause of death for young people between 10 and 24 years of age (3,4). To better target prevention and intervention efforts, we must increase our understanding of risk factors for suicidal thoughts and behaviours in children and adolescents.
Risk factors for suicidal thoughts and behaviours in adolescence
Two studies have investigated risk factors associated with suicidal thoughts and behaviours in a very large sample of children between the ages of 9 and 11 (N=11,875) in the Adolescent Brain Cognitive Development Study (5–7). DeVille et al. (6) examined social-environmental factors using generalized linear mixed-effects models and revealed that higher levels of family conflict was associated with suicidal ideation, while low parental monitoring was associated with both ideation and attempt. Janiri et al. (5) examined a broader range of potential risk and protective factors for suicidality using logistic regression and also showed that higher levels of family conflict was a risk factor for suicidality, the presence of child psychopathology and longer weekend screen time were also found to be risk factors, while greater parental supervision and positive school involvement were protective factors. The findings that poor family coherence and support are associated with suicidal ideation in children are in line with the interpersonal-psychological model and the integrated motivational-volitional model of suicidal behaviour, in which the feeling of being alone and non-supported (thwarted belongingness) is an important risk factor for suicidal ideation and attempt (8,9). However, sociodemographic and clinical factors that have been identified previously to be associated with STBs, have not led to improved prediction of STBs (10). Therefore, there is a need for research into novel measures associated with STBs such as genetics or regional brain activity, which have been shown to play a role in suicidal thoughts and behaviour in adolescents (11). In addition, these studies have not examined whether a combination of factors, instead of examining associations per risk factor, distinguishes children with and without suicidal thoughts or behaviour. Combining different types of risk factors may improve classification over use of individual risk factors.
Aims
To address these gaps, we examine whether a combination of a broad range of traditional risk factors (sociodemographic, physical health, social environmental, clinical psychiatric characteristics) and novel risk factors (cognitive, neuroimaging and genetic characteristics) in a sample of almost 6,000 unrelated children in the ABCD study could together differentiate children with a lifetime history of suicidal thoughts and/or suicide attempt and two control groups. Since a large number of children in the suicidal thoughts and behaviour (STB) group also have a psychiatric disorder, the control groups were: 1) children without psychiatric disorder (healthy controls; HC), 2) children with psychiatric disorder but no history of suicidal thoughts or behaviour (clinical controls; CC). To this end, we used binomial penalized logistic regression and a feature selection approach, which can determine which type of measures contribute most to the classification of STB. In addition to examining risk for suicidal thoughts and behaviours, it is important to identify factors that distinguish between individuals who only think about suicide (suicidal ideation), and those who attempt suicide (e.g. 12,13). This is relevant as it has been shown that only one third of individuals with suicidal thoughts actually attempt suicide (14) and the identification of factors that differentiate these individuals may further inform targeted prevention and intervention efforts. Therefore, as a final aim, we examined which factors differentiated children with (a history of) suicidal ideation, but no history of suicidal behaviour, and those that have attempted suicide during their lives.
Methods
Participants
All data included in this study were collected as part of the Adolescent Brain Cognitive Development (ABCD) study (Annual Release 2.1;https://nda.nih.gov/abcd). Data were drawn from the baseline measurement of the ABCD study, which included data from 11,875 children between the ages of 9 and 11 assessed at 22 sites across the United States. The recruitment method and inclusion and exclusion criteria of the ABCD study are described elsewhere (7). All adolescents provided written assent and their parents provided consent. The Institutional Review Board of the University of California at San Diego approved the study protocol and data collection and is responsible for ethical oversight.
In the current study, we only included unrelated children, leading to a sample size of 9,985 children (see Supplemental Note 1 and Supplemental Figure S1). In addition, 9 children were excluded due to missing sociodemographic data, and 4000 children were excluded due to missing neuroimaging data or excluded due to low quality of neuroimaging data (as suggested by the ABCD team). This resulted in a total sample size of 5885 children for the current analysis.
Definition of outcome groups
Suicidal thoughts and behaviours (interrupted, aborted or actual suicide attempt) and psychiatric diagnoses were assessed using the child-and parent-reported of the computerized Kiddie Schedule for Affective Disorders and Schizophrenia for DSM-5 (KSADS-5) (15). As previous findings showed low correspondence between parent- and child-reported STBs in the ABCD sample (5), we created two STB outcome variables, one for each reported (see Supplemental Note 2 for more details on group definitions). Children with comorbid psychiatric disorders were not excluded from the STB group. The definitions for the HC and CC group were the same across parent and child outcome variables, however, due to differences in the STB outcome, the sample of the HC and CC groups also differed.
For the parent-reported STB outcome variable, we created three groups based on the K-SADS-5 diagnostic information: 1) healthy control group (no parent-reported or child-reported psychiatric diagnosis was present and no parent-reported lifetime history of suicidal thoughts and behaviour; N=2,415); 2) clinical control group (a parent-reported or child-reported psychiatric diagnosis was present, but there was no lifetime parent-reported history of suicidal thoughts or behaviour; N=2,976); 3) STB group (lifetime parent-reported suicidal thoughts or behaviour was reported; N=494).
The child-reported STB outcome variable included the following three groups: 1) healthy control group (no parent-reported or child-reported psychiatric diagnosis was present and no child-reported lifetime history of suicidal thoughts or behaviour; N=2,367); 2) clinical control group (a parent-reported or child-reported psychiatric diagnosis was present, but there was no lifetime child-reported history of suicidal thoughts or behaviour; N=2,985); 3) STB group (lifetime child-reported suicidal thoughts or behaviour was reported; N=528).
In addition, ancillary analyses were performed on the individuals that were in the same group according to both the parent and child outcome variables (see Supplemental Note 3).
For secondary analyses, we created two additional outcome variables to distinguish children with lifetime suicidal thoughts (ideation) from children with a history of suicidal behaviour (attempt). The child-reported outcome variable included 461 children with self-reported suicidal ideation but no history of attempt and 67 children with a self-reported history of suicide attempt. The parent-reported suicide ideation and suicide attempt outcome variables included 464 children with suicidal ideation but no history of attempt, and 30 children with a history of suicide attempt (see Supplemental Note 4 for more details on group definitions).
Risk factors
Seven sociodemographic, 13 physical health, 28 socioenvironmental, 56 clinical psychiatric, 14 cognitive functioning, 88 neuroimaging and five genetic variables were included, based on available literature, as classifiers of group status (for a detailed overview of all included measures please see Supplemental Note 5,6 and Supplemental Table S1).
Statistical analysis
Training and independent validation datasets
In order to perform the binomial penalized logistic regression analysis, a training dataset, consisting of 2/3 of the data, and a validation dataset (or hold-out sample), consisting of 1/3 of the data, were created by randomly splitting the data according to the data collection site to ensure the generalization of model performance to independent sites (see Figure 1). When the groups were based on the child-reported STB the training dataset consisted of 4168 children and the validation dataset of 1712 children (see Supplemental Table 2). The training dataset based on parent-reported STB included 4172 children and the validation dataset 1713 children.
Figure 1. Flowchart to describe the analysis procedure.
The ABCD data was split into a training and test set. The training set was used to do a penalized logistic regression in ten-fold cross validation and repeat this ten times with 4 different combinations of the Lasso and Ridge penalty. Features that had a coefficient higher than 0 in 90% or more of the repeats were selected to create a Ridge logistic regression to differentiate groups. This Ridge model was then tested on the test dataset. In addition, the same procedure was repeated only including risk factors from one modality.
Classification of group status in the training set
Binomial penalized logistic regression analysis was performed using the package glmnet in R (16). This was applied to a combination of all measures in the training set to distinguish between (1) the healthy control group; (2) the clinical control group and (3) the STB group. The binomial penalized logistic regression builds a sparse model by adding a penalty which prevents overfitting. This approach combines two types of penalties or regularizations. A Ridge penalty shrinks coefficients, making their contribution to the model small, and a Lasso penalty forces some coefficients to zero, meaning that the feature is not selected for the model. A combination of the two penalties allows for feature selection as well as for features to have a small contribution to the model. Binomial penalized regression was performed with different penalties (alpha levels: 0.25, 0.5, 0.75 and 1), varying between a Lasso penalty (alpha=1) and a combination of Lasso and Ridge penalties (elastic net; alpha’s between 0.25 and 0.75). Ten-fold cross-validation (CV) was applied by dividing the training dataset into 10 sets, and within each CV fold, 9 out of 10 sets were combined to form the training set and 1 was used as the test set. This was repeated 10 times. The glmnet package determined the optimal lambda value by identifying the lambda associated with the minimum Brier score. In each CV fold, we imputed missing values using the caret package (17) in the test set and training set separately, in order to prevent data leakage. Binomial analyses comparing two groups were run (HC vs. CC, CC vs. STB and HC vs STB groups). Binomial analyses were performed instead of multinomial analyses, as a set of clinical psychiatric measures were only non-zero in the CC and STB groups. As the STB group was smaller than the clinical control and healthy control groups, we under-sampled these larger groups within each CV fold to match the size of the STB group by randomly selecting cases from the HC and CC groups. In additional analyses we performed the abovementioned analyses again using a nested alpha function (please see supplemental note 7 for a description and the results).
The performance of the model was assessed using the area under the receiver operating characteristics curve (AUROC). AUROC represents the proportion of times an individual from a positive class (e.g. STB group) is ranked below an individual from a negative class (e.g. HC). In addition, sensitivity, specificity, average of the sensitivity and specificity (accuracy) were calculated. Permutation testing (by comparing the AUROC against the AUROC of the same procedure repeated 1000 with permuted group labels) was used to examine if the model performed significantly above chance level classification. To identify the features that contributed most to the classification model, the features that had a coefficient of more than 0 in at least 90% of the subsamples were selected.
Generalization to the independent validation set
The features that were selected in at least 90% of the subsamples at each alpha (0.25, 0.50, 0.75, 1.00) in the training dataset were used to classify group membership in the independent validation set. This validation set consisted of 7 sites from the ABCD study that were kept separate to ensure independence. This analysis was done to test the generalizability of the classification model to independent sites and participants. The selected features at each alpha were used in a Ridge logistic regression in the whole training set; this model was then tested on the independent validation set.
Modality-specific classification
In order to examine the individual contribution of the different modalities to the classification of the STB groups, we repeated the aforementioned analysis, but only including specific types of measures, thus performing separate analyses for sociodemographic, physical health, social environmental, clinical psychiatric, cognitive functioning, neuroimaging and genetic measures.
Factors that differentiate ideators from attempters
To examine which factors differentiate between children with a history of attempt from those with suicidal ideation but no history of suicide attempt, the abovementioned binomial penalized logistic regression analysis was performed again with a different outcome variable. For this analysis, the dataset was again divided into a training set and validation set using the same site split as in the main analysis, and the same approach (including the binomial penalized logistic regression with CV, feature selection and Ridge regression) was used to test generalizability; however, only 5 folds were used because the sample size was smaller.
Results
Sample characteristics
Age, sex, lifetime psychiatric diagnosis and self-reported suicidal thoughts or behaviours are presented in Table 1 for the three groups (healthy controls, clinical controls, and STB) based on child-reported and parent-reported suicidal thoughts or behaviours.
Table 1. Sample characteristics for the child- and parent-reported STB group.
| Child-reported STB groups | Parent-reported STB groups | |||||
|---|---|---|---|---|---|---|
| Suicidal ideation/ attempt N = 528 | Suicidal ideation/attempt N = 494 | |||||
| Age (in months) (SD) | 119 (7.52) | 120 (7.60) | ||||
| Sex (% female) | 45.1 % | 38.5% | ||||
| Lifetime BD (parent-reported) | 44 (8.3%) | 69 (14.0%) | ||||
| Lifetime BD (child-reported) | 113 (21.4%) | 68 (13.8%) | ||||
| Lifetime DD (parent-reported) | 62 (11.7%) | 115 (23.3%) | ||||
| Lifetime DD (child-reported) | 74 (14.0%) | 43 (8.7%) | ||||
| Lifetime ADHD (parent-reported) | 154 (29.2%) | 205 (41.5%) | ||||
| Lifetime Psychotic disorder (parent-reported) | 19 (3.6%) | 38 (7.7%) | ||||
| Lifetime Panic disorder (parent-reported) | 10 (1.9%) | 18 (3.6%) | ||||
| Lifetime Social anxiety (parent-reported) | 35 (6.6%) | 67 (13.6%) | ||||
| Lifetime Eating disorder (parent-reported) | 56 (10.6%) | 52 (10.5%) | ||||
| Lifetime Specific phobia (parent-reported) | 162 (30.7%) | 199 (40.3%) | ||||
| Lifetime GAD (parent-reported) | 60 (11.4%) | 107 (21.7%) | ||||
| Lifetime ANX (child-reported) | 49 (9.3%) | 26 (5.3%) | ||||
| Lifetime OCD (parent-reported) | 87 (16.5%) | 108 (21.9%) | ||||
| Lifetime PTSD (parent-reported) | 60 (11.4%) | 86 (17.4%) | ||||
| Lifetime Substance use disorder (parent-reported) | 1 (0.2%) | 3 (0.6%) | ||||
| Lifetime passive suicidal ideation (child-reported) | 386 (73.1%) | 423 (85.6%) | ||||
| Lifetime active non-specific suicidal ideation (child-reported) | 275 (52.1%) | 232 (47.0%) | ||||
| Lifetime active suicidal ideation + method (child-reported) | 77 (14.6%) | 59 (11.9%) | ||||
| Lifetime active suicidal ideation + intent (child-reported) | 35 (6.6%) | 27 (5.5%) | ||||
| Lifetime active suicidal ideation + plan (child-reported) | 28 (5.3%) | 10 (2.0%) | ||||
| Lifetime preparatory actions toward suicidal behaviour (child-reported) | 24 (4.5%) | 33 (6.7%) | ||||
| Lifetime interrupted suicide attempt (child-reported) | 8 (1.5%) | 4 (0.8%) | ||||
| Lifetime aborted suicide attempt (child-reported) | 28 (5.3%) | 6 (1.2%) | ||||
| Lifetime actual suicide attempt (child-reported) | 37 (7.0%) | 23 (4.7%) | ||||
Note: ADHD: Attention Deficit Hyperactivity Disorder; ANX: anxiety disorder; BD: bipolar disorder; DD: depressive disorder; GAD: generalized anxiety disorder; OCD: obsessive-compulsive disorder; PTSD: post-traumatic stress disorder; SD: standard deviation
Classification of STB group
Classification of STB group: Cross validation model performance
Results of the analysis using the child-reported STB group measures are presented in Table 2. AUROC values were highest when differentiating the HC and STB groups (range: 0.79-0.80 across the different alpha levels), and were lowest for the comparison between HC and CC groups (AUROC range: 0.68-0.69, Supplemental Figure 2).
Table 2. Classification of STB groups (child-reported and parent-reported): Results of binomial penalized logistic regression analysis.
| Comparison | Alpha | AUROC | SD AUROC | Sensitivity | Specificity | Accuracy | PPV | NPV |
|---|---|---|---|---|---|---|---|---|
| Child-reported STB | ||||||||
| HC vs. CC | | | |||||||
| 0.25 | 0.687 | 0.022 | 0.692 | 0.578 | 0.635 | 0.621 | 0.652 | |
| 0.50 | 0.687 | 0.026 | 0.696 | 0.576 | 0.636 | 0.621 | 0.654 | |
| 0.75 | 0.689 | 0.029 | 0.695 | 0.575 | 0.635 | 0.621 | 0.653 | |
| 1.00 | 0.688 | 0.028 | 0.693 | 0.579 | 0.636 | 0.622 | 0.653 | |
| HC vs. STB | ||||||||
| 0.25 | 0.795 | 0.042 | 0.688 | 0.768 | 0.728 | 0.748 | 0.711 | |
| 0.50 | 0.796 | 0.046 | 0.684 | 0.771 | 0.727 | 0.749 | 0.709 | |
| 0.75 | 0.796 | 0.049 | 0.681 | 0.761 | 0.721 | 0.740 | 0.705 | |
| 1.00 | 0.797 | 0.049 | 0.683 | 0.772 | 0.728 | 0.750 | 0.709 | |
| CC vs. STB | ||||||||
| 0.25 | 0.705 | 0.054 | 0.593 | 0.693 | 0.643 | 0.659 | 0.630 | |
| 0.50 | 0.715 | 0.059 | 0.602 | 0.710 | 0.656 | 0.674 | 0.640 | |
| 0.75 | 0.705 | 0.058 | 0.599 | 0.696 | 0.648 | 0.663 | 0.635 | |
| 1.00 | 0.712 | 0.061 | 0.596 | 0.708 | 0.652 | 0.671 | 0.637 | |
| Parent-reported STB | ||||||||
| HC vs. CC | ||||||||
| 0.25 | 0.684 | 0.027 | 0.675 | 0.585 | 0.630 | 0.619 | 0.643 | |
| 0.50 | 0.686 | 0.028 | 0.678 | 0.586 | 0.632 | 0.621 | 0.645 | |
| 0.75 | 0.685 | 0.030 | 0.680 | 0.584 | 0.632 | 0.620 | 0.646 | |
| 1.00 | 0.686 | 0.026 | 0.676 | 0.588 | 0.632 | 0.621 | 0.645 | |
| HC vs. STB | ||||||||
| 0.25 | 0.811 | 0.045 | 0.667 | 0.805 | 0.736 | 0.774 | 0.707 | |
| 0.50 | 0.809 | 0.048 | 0.672 | 0.794 | 0.733 | 0.765 | 0.708 | |
| 0.75 | 0.810 | 0.047 | 0.666 | 0.801 | 0.733 | 0.770 | 0.706 | |
| 1.00 | 0.808 | 0.048 | 0.668 | 0.789 | 0.729 | 0.760 | 0.704 | |
| CC vs. STB | ||||||||
| 0.25 | 0.765 | 0.057 | 0.635 | 0.749 | 0.692 | 0.716 | 0.672 | |
| 0.50 | 0.769 | 0.052 | 0.642 | 0.760 | 0.701 | 0.728 | 0.680 | |
| 0.75 | 0.761 | 0.053 | 0.643 | 0.748 | 0.696 | 0.718 | 0.677 | |
| 1.00 | 0.774 | 0.049 | 0.649 | 0.760 | 0.704 | 0.730 | 0.684 | |
Note: CC: clinical control group; HC: healthy control group; STB: suicidal thoughts and behaviours group, PPV: positive predictive value, NPV: negative predictive value
A similar pattern was observed for the results of the analyses using the parent-reported STB group measures (see Table 2), with the highest AUROC observed for the HC vs. STB comparison (range: 0.80-0.81) and lowest for the HC vs. CC comparison (range: 0.68-0.69). ROC curves, cross-validation curves and Brier scores are presented in the Supplemental Figures S3–5 and Supplemental Table S4.
Feature selection
Results of the feature selection analysis are presented in Supplemental Table S5 and S6 for the child-reported STB groups and parent-reported STB groups, respectively. While the same measures were included in both analyses, the factors that distinguished the child-reported STB group from the clinical controls were family conflict, prodromal psychotic symptoms, impulsivity (UPPS-P negative urgency and lack of planning subscales) and the CBCL depression subscale score. The factors that differentiated the clinical controls from the parent-reported STB group included the CBCL depression subscales (anxious depression, DSM5 depression), CBCL conduct disorder subscale score, CBCL internalizing and externalizing broad band scores and a history of mental health service use or treatment. Plots of the stability of each predictor within repeated cross-validation folds are presented in the Supplemental Figure S6. In addition, we examined the results using a stricter feature selection approach (see Supplemental note 8 for a description and results).
Generalization to the independent validation dataset
The AUROC in the independent validation dataset (7 separate ABCD sites) using the most contributing features selected (see above), was in line with the AUROCs achieved in the training dataset (see Supplemental Tables S7 and S8). Classifying HC vs. STB, the AUROC ranged between 0.78-0.80 using the child-reported STB group measure and between 0.81-0.83 using the parent-reported STB group measure, using the features that were selected in the training dataset at different alphas. Classifying HC vs. CC, the AUROC ranged between 0.70-0.71 using the child-reported group measure and between 0.70-0.71 using the parent-reported measure. Finally, classifying CC vs. STB, the AUROC ranged between 0.70-0.72 when using the child-reported measure and between 0.70-0.71 when using the parent-reported measure.
Modality-specific classification
Results of these analyses are presented in Table S9 and S10 for the child-reported and parent-reported STB group analyses, respectively. For both the classification of the child- and parent reported STB group status, the clinical psychiatric (AUROC range child-reported: 0.67-0.69; parent-reported: 0.77-0.79), physical health (AUROC range child-reported: 0.58-0.73; parent-reported: 0.62-0.78), cognitive functioning (AUROC range child-reported: 0.58-0.72; parent-reported: 0.52-0.65) and social environmental factors (AUROC range child-reported: 0.61-0.74; parent-reported: 0.61-0.73) best classified STB groups, in contrast to neuroimaging (AUROC range child-reported: 0.49-0.52; parent-reported: 0.48-0.52), sociodemographic measures (AUROC range child-reported: 0.53-0.59; parent-reported: 0.53-0.61) and genetic characteristics (AUROC range child-reported: 0.51-0.57; parent-reported: 0.52-0.55). Similar to the aforementioned results, the highest AUROC values were observed for the HC vs. STB comparison. Statistical analyses of the performance of the different modalities can be found in supplemental note 9 and tables S11 and S12.
Classification of ideators versus attempters
Results of the analysis used to classify child-reported suicidal ideation versus suicidal attempt are presented in Table 3. AUROC values varied between 0.54 and 0.58 across the different alpha levels. Results of the same analysis, but using the parent-reported group measure showed similar results (AUROC range: 0.49-0.54; see Table 3). As the results show that it is not possible to distinguish these two groups, no further feature selection or modality specific classification was performed.
Table 3. Classification of suicidal ideation versus suicidal behaviour (child-reported and parent-reported): Results of binomial penalized logistic regression analysis.
| Comparison | Alpha | AUROC | SD AUROC | Sensitivity | Specificity | Accuracy | PPV | NPV |
|---|---|---|---|---|---|---|---|---|
| Child-reported STB | ||||||||
| Ideation versus suicidal behaviour | ||||||||
| 0.580 | 0.146 | 0.659 | 0.460 | 0.559 | 0.549 | 0.574 | ||
| 0.50 | 0.559 | 0.130 | 0.691 | 0.413 | 0.552 | 0.541 | 0.572 | |
| 0.75 | 0.546 | 0.104 | 0.722 | 0.352 | 0.537 | 0.527 | 0.559 | |
| 1.00 | 0.558 | 0.119 | 0.707 | 0.371 | 0.539 | 0.529 | 0.559 | |
| Parent-reported STB | ||||||||
| Ideation versus suicidal behaviour | ||||||||
| 0.486 | 0.195 | 0.634 | 0.358 | 0.496 | 0.497 | 0.495 | ||
| 0.50 | 0.521 | 0.179 | 0.729 | 0.313 | 0.521 | 0.515 | 0.536 | |
| 0.75 | 0.519 | 0.168 | 0.692 | 0.318 | 0.505 | 0.504 | 0.508 | |
| 1.00 | 0.533 | 0.213 | 0.705 | 0.318 | 0.512 | 0.509 | 0.519 | |
Note: PPV: positive predictive value, NPV: negative predictive value
Discussion
Main findings
In a large sample of almost 6,000 unrelated children, we examined whether a combination of non-biological (sociodemographic, physical health, clinical psychiatric, cognitive, psychosocial) and biological (neuroimaging and genetic) factors could differentiate healthy children, children with psychiatric disorder but no history of suicidal thoughts or behaviour, and children with a lifetime history of suicidal thoughts or suicide attempt (the STB group). Binomial penalized logistic regression analysis showed that the STB group could be distinguished from the HC and CC groups (AUROC range 0.79-0.81 and 0.70-0.78 respectively), but the ability to differentiate the CC and HC group was less accurate (AUROC range 0.68-0.69). These results may be explained by the fact that the children with most severe psychiatric symptoms may have been included in the STB group, thereby reducing the differences between the CC and HC groups. Our model generalized to independent data (separate ABCD recruitment sites (AUROC range 0.70-0.83)). The analyses for groups based on parent- and child-reported measures were performed separately, as a recent study (5) showed low correspondence between parent-reported and child-reported measures of suicidal thoughts and behaviours in the ABCD study. The AUROCs of these analyses were very similar, as we were able to distinguish the groups based on both the parent- and child-reported measures. Children with a lifetime history of suicidal ideation could not be distinguished from those with a lifetime history of suicide attempt (AUROC range 0.49-0.58).
Predictive value
The classification of STB and identifying contributing risk factors are important aims in suicide research as it may help identify those at risk and help target prevention and intervention efforts. In this study, when discriminating the STB group from the CC group, the PPV varied between 0.65-0.73, while the NPV varied between 0.63 and 0.69. This means that around 3 out of 10 children were misclassified as belonging to the STB group while they had no history of STB, and similarly, around 3 out of 10 children were misclassified as belonging to the CC group while they did have a history of STB. While the PPV in our study, is higher than observed in a meta-analysis which used psychological and biological risk instruments to predict suicidal behaviour (20), the sensitivity observed is lower than the sensitivity of existing suicide scales in predicting suicide attempt (18,19), and therefore our classification model is not yet sufficient to be used as a clinical decision tool. Risk assessment using traditional suicide scales may therefore outperform our multimodal prediction. Our findings are in line with three meta-analyses that showed that (a combination of) psychological or biological measures were limited in their ability to predict suicide or suicidal behaviour (19–21) showing that classification of suicidal thoughts and behaviours is complex, and adding to the current debate around precision medicine in suicide research (e.g. 22).
Classification per modality
When the risk factors were divided into separate modalities to examine their unimodal predictive characteristics, the AUROC values for social environmental, physical, cognitive, and clinical psychiatric modalities were higher than the AUROC values for neuroimaging, genetic and sociodemographic modalities. This finding was in line with the strongest contributing features when all predictors were combined in one analysis, as these features were mainly from the social environmental, and clinical psychiatric categories. The fMRI-based measures included did not seem to contribute to the classification of children with STB. In contrast to these findings, previous studies have found that functional brain alterations in the prefrontal cortex are related to STB (23,24) and contribute to the classification of suicidal youth (25). However, our findings are consistent with a neuroimaging-specific evaluation of this same cohort, in which no association was found between suicidal thoughts and behaviours and functional neuroimaging measures (26). These discrepant findings between ABCD and other studies could potentially be explained by the younger age of participants in the ABCD study, the fact that ABCD is a population study, or methodological issues that have been described elsewhere (27). In addition, the polygenic risk scores included in the analysis also did not contribute to classification, which is in line with previous studies that show that PRS for MDD only explained a small part of the variation in self-injurious behaviour (28).
Individual features that contribute to classification of STBs
Most features that contributed to the model classifying HC and STB also contributed when classifying CC from HC. When the STB group was differentiated from the CC group, family conflict, prodromal psychosis, severity of mental health symptoms and measures of impulsivity were amongst the features that contributed most to the model’s predictions. These findings highlight the potential need for clinicians to consider alternative interventions, including family-based psychological interventions to decrease family conflict (29) or neuropsychological training to increase cognitive control and planning abilities; and emotional regulation skills, distress tolerance training or mindfulness-based interventions in order to decrease negative urgency and modulate impulsivity in suicidal individuals. Surprisingly, parent-reported child mental health service use, predicted parent-reported STB, but not child-reported STB, further highlighting the low correspondence between parent- and child-reported STB.
Classification of ideation vs. attempt
Understanding which children will experience suicidal thoughts or attempt suicide has important implications for suicide prevention and clinical practice (30). In this cross-sectional study, we were unable to differentiate children with suicidal thoughts from children with a history of suicidal behaviour, potentially suggesting a shared etiology between ideation and attempt in this age group. A large study in 16 year-olds showed that, compared to adolescents with suicidal thoughts, those that attempted suicide were more often exposed to self-harm by friends or family members, were more likely to be diagnosed with a psychiatric disorder, more often were female, exposed to trauma, more impulsive and had specific personality characteristics (i.e. high sensation seeking and low conscientiousness) (31). A second large study conducted among adolescents and young adults showed that acquired capability, impulsivity, mental imagery about death and exposure to suicidal behaviour were more common in those who attempted suicide compared to ideators (13). Meta-analyses showed that traumatic life events, history of abuse, drug use disorders, and alterations in decision making and impulsivity were more common in attempters than ideators, while depression, alcohol use, hopelessness, and sociodemographic variables did not differ between attempters and ideators (32,33). We were unable to include some of the aforementioned variables in our logistic regression model, which may explain why our classification performance was poorer than that observed in previous studies. The variables that contributed to the classification of STB from CC, were unable to distinguish ideation from an attempt, as they may be related to suicidal thoughts and behaviour in general, and do not differ between ideators and attempters. In addition, only 67 children reported a history of suicide attempt and only 30 parents reported that their child had a history of suicide attempt, which may have limited our power to detect small effects. Finally, the young age of these participants may have added additional noise to the classification, as a larger fraction of the ideation group may attempt suicide in the future compared to studies with older participants.
Strengths and limitations
This is the first study to combine multimodal features to classify children with suicidal thoughts and behaviour from the CC and HC participants in the ABCD study, and builds on previous work by Janiri et al. and DeVille et al. (5,6). Compared to these previous studies, the strengths of this study include the large sample size of unrelated participants, the availability of many different types of predictors, including clinical, sociodemographic, biological and cognitive measures, and the use of an ecologically valid control group consisting of children with a psychiatric disorder.
An additional strength is rigorous validation using cross validation and an independent out-ofsample validation which avoids overly optimistic results due to overfitting in the training set. The findings need to be interpreted in the light of a few limitations, including the cross-sectional nature of the data. Firstly, longitudinal data collection for participants enrolled in the ABCD study is planned at 2-year intervals for a total of 10 years, and future studies may build on these baseline models to predict suicidal thoughts or behaviour throughout adolescence. Secondly, no measures of the severity or frequency of suicidal ideation or behaviour were available, which limited our ability to examine specific subgroups with varying suicidal severity. Thirdly, both static (e.g. PRS), early life (e.g. negative life events), and transient (e.g. psychiatric symptoms) factors were included as risk factors in the current study as they can all contribute to risk for STB. While static risk factors are unmodifiable and may not represent immediate targets for suicide prevention, theymay be important for identification and classification of those at risk. In contrast, more dynamic risk factors may represent better direct targets for suicide prevention. Ecological Monetary Assessment (EMA) lies at the dynamic end of the spectrum and could potentially detect risk in real time, and could therefore be considered in future studies. Fourthly, the independent holdout sample used in this study was a single random partition of the ABCD data. To ensure generalisability the model should be tested in yet another independent dataset, preferably a different dataset where similar measures were collected. Finally, in this study, we showed the limited contribution of biological measures to classification, however, we may have missed interesting associations as we included these measures as continuous measures across the entire range. Future studies on these biological measures could consider using a more sophisticated approach by first stratifying groups by certain clinical and/or biological characteristics and then selecting a classification model which would be based on this individual’s characteristics.
Conclusion
While results of this study revealed modest classification of STB-based groups in children, which limits the use of this model as a clinical decision tool, this study did reveal risk factors for STBs in children and points to potential treatment targets. Our study shows that social environment (family conflict), cognitive (impulsivity) and clinical measures (e.g., severity of prodromal psychosis symptoms, severity of depression) differentiate children with and without a history of suicidal thoughts and behaviour. More studies in a larger sample of attempters are needed to confirm whether the factors identified in our study differentiate those with ideation from those with a history of attempt and prospectively predict subsequent suicidal behaviour. In addition, future studies could determine whether including additional variables (e.g., suicide-related measures) improves classification. This work highlights the need for clinicians to monitor children who present multiple risk factors and may inform future socio-environmental interventions that may contribute to suicide prevention in at-risk children.
Supplementary Material
Funding and Disclosure
This work was supported by the MQ Brighter Futures Award MQBFC/2 (LS) and the National Institute of Mental Health of the National Institutes of Health under Award Number R01MH117601 (LS, NJ). LS is supported by a NHMRC Career Development Fellowship (1140764).
Data used in the preparation of this article were obtained from the Adolescent Brain Cognitive DevelopmentSM (ABCD) study (https://abcdstudy.org), held in the NIMH Data Archive (NDA). This is a multisite, longitudinal study designed to recruit more than 10,000 children age 9-10 and follow them over 10 years into early adulthood. The ABCD Study is supported by the National Institutes of Health and additional federal partners under award numbers U01DA0401048, U01DA050989, U01DA051016, U01DA041022, U01DA051018, U01DA051037, U01DA050987, U01DA041174, U01DA041106, U01DA041117, U01DA041028, U01DA041134, U01DA050988, U01DA051039, U01DA041156, U01DA041025, U01DA041120, U01DA051038, U01DA041148, U01DA041093, U01DA041089, U24DA041123, U24DA041147. A full list of supporters is available at https://abcdstudy.org/federal-partners.html. A listing of participating sites and a complete listing of the study investigators can be found at https://abcdstudy.org/consortium_members/. ABCD consortium investigators designed and implemented the study and/or provided data but did not necessarily participate in analysis or writing of this report. This manuscript reflects the views of the authors and may not reflect the opinions or views of the NIH or ABCD consortium investigators. The ABCD repository grows and changes over time. The ABCD data used in this report came from 10.15154/1520786. DOIs can be found at nda.nih.gov.
Footnotes
Author contributions
LvV, YT: conceptualised the study, conducted the data analysis, and wrote the majority of the manuscript. RD: contributed to the analysis of the data and critically revised the manuscript. AC: contributed to the analysis and assisted in writing the manuscript. AA-P, JR, NJ, ME: critically revised the manuscript. LS: conceptualised the study, provided guidance for interpreting the findings, and critically revised the manuscript
Declaration of interests: None
A preprint of this manuscript has previously been published on MedRxiv.
References
- 1.Alicandro G, Malvezzi M, Gallus S, La Vecchia C, Negri E, Bertuccio P. Worldwide trends in suicide mortality from 1990 to 2015 with a focus on the global recession time frame. Int J Public Health. 2019 Jun;64(5):785–95. doi: 10.1007/s00038-019-01219-y. [DOI] [PubMed] [Google Scholar]
- 2.Hawton K, Saunders KEA, O’Connor RC. Self-harm and suicide in adolescents. Lancet. 2012 Jun 23;379(9834):2373–82. doi: 10.1016/S0140-6736(12)60322-5. [DOI] [PubMed] [Google Scholar]
- 3.World Health Organization. Preventing suicide: A global imperative. World Health Organization; 2014. [Google Scholar]
- 4.Centers for Disease Control and Prevention (CDC) Web-based injury statistics query and reporting system (WISQARS) 2017. Available from: https://www.cdc.gov/injury/wisqars/fatal.html.
- 5.Janiri D, Doucet GE, Pompili M, Sani G, Luna B, Brent DA, et al. Risk and protective factors for childhood suicidality: a US population-based study. Lancet Psychiatry. 2020 Apr;7(4):317–26. doi: 10.1016/S2215-0366(20)30049-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.DeVille DC, Whalen D, Breslin FJ, Morris AS, Khalsa SS, Paulus MP, et al. Prevalence and Family-Related Factors Associated With Suicidal Ideation, Suicide Attempts, and Self-injury in Children Aged 9 to 10 Years [Internet] JAMA Network Open. 2020;3:e1920956. doi: 10.1001/jamanetworkopen.2019.20956. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Volkow ND, Koob GF, Croyle RT, Bianchi DW, Gordon JA, Koroshetz WJ, et al. The conception of the ABCD study: From substance use to a broad NIH collaboration. Dev Cogn Neurosci. 2018 Aug;32:4–7. doi: 10.1016/j.dcn.2017.10.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Ribeiro JD, Joiner TE. The interpersonal-psychological theory of suicidal behavior: current status and future directions. J Clin Psychol. 2009 Dec;65(12):1291–9. doi: 10.1002/jclp.20621. [DOI] [PubMed] [Google Scholar]
- 9.O’Connor RC. The Integrated Motivational-Volitional Model of Suicidal Behavior [Internet] Crisis. 2011;32:295–8. doi: 10.1027/0227-5910/a000120. [DOI] [PubMed] [Google Scholar]
- 10.Franklin JC, Ribeiro JD, Fox KR, Bentley KH, Kleiman EM, Huang X, et al. Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research. Psychol Bull. 2017 Feb;143(2):187–232. doi: 10.1037/bul0000084. [DOI] [PubMed] [Google Scholar]
- 11.Vargas-Medrano J, Diaz-Pacheco V, Castaneda C, Miranda-Arango M, Longhurst MO, Martin SL, et al. Psychological and neurobiological aspects of suicide in adolescents: Current outlooks. Brain Behav Immun Health. 2020 Aug;7:100124. doi: 10.1016/j.bbih.2020.100124. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Klonsky ED, May AM. The Three-Step Theory (3ST): A New Theory of Suicide Rooted in the “Ideation-to-Action” Framework. Int J Cogn Ther. 2015 Jun 1;8(2):114–29. [Google Scholar]
- 13.Wetherall K, Cleare S, Eschle S, Ferguson E, O’Connor DB, O’Carroll RE, et al. From ideation to action: Differentiating between those who think about suicide and those who attempt suicide in a national study of young adults [Internet] Journal of Affective Disorders. 2018;241:475–83. doi: 10.1016/j.jad.2018.07.074. [DOI] [PubMed] [Google Scholar]
- 14.Nock MK, Borges G, Bromet EJ, Alonso J, Angermeyer M, Beautrais A, et al. Cross-national prevalence and risk factors for suicidal ideation, plans and attempts. Br J Psychiatry. 2008 Feb;192(2):98–105. doi: 10.1192/bjp.bp.107.040113. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Kobak KA, Kratochvil CJ, Stanger C, Kaufman J. Computerized screening of comorbidity in adolescents with substance or psychiatric disorders. Anxiety Disorders and Depression (La Jolaa, CA) [Internet] 2013 Available from: https://www.researchgate.net/grant/Computerized-Screening-for-Comorbidity-in-Adolescents-with-Substance-or-Psychiatric-Disorders_2371516. [Google Scholar]
- 16.Friedman J, Hastie T, Tibshirani R. Regularization paths for generalized linear models via coordinate descent. J Stat Softw. 2010;33(1):1. [PMC free article] [PubMed] [Google Scholar]
- 17.Kuhn M. Building predictive models in R using the caret package. J Stat Softw. 2008;28(5):1–26. Others. [Google Scholar]
- 18.Lindh ÅU, Dahlin M, Beckman K, Strömsten L, Jokinen J, Wiktorsson S, et al. A Comparison of Suicide Risk Scales in Predicting Repeat Suicide Attempt and Suicide: A Clinical Cohort Study. J Clin Psychiatry [Internet] 2019 Nov 19;80(6) doi: 10.4088/JCP.18m12707. [DOI] [PubMed] [Google Scholar]
- 19.Runeson B, Odeberg J, Pettersson A, Edbom T, Jildevik Adamsson I, Waern M. Instruments for the assessment of suicide risk: A systematic review evaluating the certainty of the evidence. PLoS One. 2017 Jul 19;12(7):e0180292. doi: 10.1371/journal.pone.0180292. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Carter G, Milner A, McGill K, Pirkis J, Kapur N, Spittal MJ. Predicting suicidal behaviours using clinical instruments: systematic review and meta-analysis of positive predictive values for risk scales. Br J Psychiatry. 2017 Jun;210(6):387–95. doi: 10.1192/bjp.bp.116.182717. [DOI] [PubMed] [Google Scholar]
- 21.Belsher BE, Smolenski DJ, Pruitt LD, Bush NE, Beech EH, Workman DE, et al. Prediction Models for Suicide Attempts and Deaths: A Systematic Review and Simulation. JAMA Psychiatry. 2019 Jun 1;76(6):642–51. doi: 10.1001/jamapsychiatry.2019.0174. [DOI] [PubMed] [Google Scholar]
- 22.Whiting D, Fazel S. How accurate are suicide risk prediction models? Asking the right questions for clinical practice. Evid Based Ment Health. 2019 Aug;22(3):125–8. doi: 10.1136/ebmental-2019-300102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Schmaal L, van Harmelen A-L, Chatzi V, Lippard ETC, Toenders YJ, Averill LA, et al. Imaging suicidal thoughts and behaviors: a comprehensive review of 2 decades of neuroimaging studies. Mol Psychiatry. 2020 Feb;25(2):408–27. doi: 10.1038/s41380-019-0587-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Gosnell SN, Fowler JC, Salas R. Classifying suicidal behavior with resting‐state functional connectivity and structural neuroimaging. Acta Psychiatr Scand. 2019 Jul 20;140(1):20–9. doi: 10.1111/acps.13029. [DOI] [PubMed] [Google Scholar]
- 25.Just MA, Pan L, Cherkassky VL, McMakin DL, Cha C, Nock MK, et al. Machine learning of neural representations of suicide and emotion concepts identifies suicidal youth. Nature Human Behaviour. 2017 Oct 30;1(12):911–9. doi: 10.1038/s41562-017-0234-y. [DOI] [PMC free article] [PubMed] [Google Scholar] [Retracted]
- 26.Vidal-Ribas P, Janiri D, Doucet GE, Pornpattananangkul N, Nielson DM, Frangou S, et al. Multimodal neuroimaging and suicidality in a US population-based sample of school-aged children [Internet] doi: 10.1101/19013193. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Bissett PG, Hagen MP, Poldrack RA. A cautionary note on stop-signal data from the Adolescent Brain Cognitive Development [ABCD] study [Internet] [cited 2020 Oct 6];2020 :2020.05.08.084707. doi: 10.7554/eLife.60185. Available from: https://www.biorxiv.org/content/10.1101/2020.05.08.084707v1.abstract. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Maciejewski DF, Renteria ME, Abdellaoui A, Medland SE, Few LR, Gordon SD, et al. The Association of Genetic Predisposition to Depressive Symptoms with Non-suicidal and Suicidal Self-Injuries. Behav Genet. 2017;47:3–10. doi: 10.1007/s10519-016-9809-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Weinstein SM, Cruz RA, Isaia AR, Peters AT, West AE. Child- and Family-Focused Cognitive Behavioral Therapy for Pediatric Bipolar Disorder: Applications for Suicide Prevention. Suicide Life Threat Behav. 2018 Dec;48(6):797–811. doi: 10.1111/sltb.12416. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Klonsky ED, Qiu T, Saffer BY. Recent advances in differentiating suicide attempters from suicide ideators. Curr Opin Psychiatry. 2017 Jan;30(1):15–20. doi: 10.1097/YCO.0000000000000294. [DOI] [PubMed] [Google Scholar]
- 31.Mars B, Heron J, Klonsky ED, Moran P, O’Connor RC, Tilling K, et al. What distinguishes adolescents with suicidal thoughts from those who have attempted suicide? A population-based birth cohort study. J Child Psychol Psychiatry. 2019;60(1):91–9. doi: 10.1111/jcpp.12878. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.May AM, Klonsky ED. What distinguishes suicide attempters from suicide ideators? A meta-analysis of potential factors. Clinical Psychology: Science and Practice. 2016;23(1):5–20. [Google Scholar]
- 33.Saffer BY, Klonsky ED. Do neurocognitive abilities distinguish suicide attempters from suicide ideators? A systematic review of an emerging research area. Clin Psychol Sci Pract. 2018 Mar 28;25(1):e12227 [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.