Machine learning and Bayesian network analyses identifies associations with insomnia in a national sample of 31,285 treatment-seeking college students

Abstract

Background

A better understanding of the relationships between insomnia and anxiety, mood, eating, and alcohol-use disorders is needed given its prevalence among young adults. Supervised machine learning provides the ability to evaluate which mental disorder is most associated with heightened insomnia among U.S. college students. Combined with Bayesian network analysis, probable directional relationships between insomnia and interacting symptoms may be illuminated.

Methods

The current exploratory analyses utilized a national sample of college students across 26 U.S. colleges and universities collected during population-level screening before entering a randomized controlled trial. We used a 4-step statistical approach: (1) at the disorder level, an elastic net regularization model examined the relative importance of the association between insomnia and 7 mental disorders (major depressive disorder, generalized anxiety disorder, social anxiety disorder, panic disorder, post-traumatic stress disorder, anorexia nervosa, and alcohol use disorder); (2) This model was evaluated within a hold-out sample. (3) at the symptom level, a completed partially directed acyclic graph (CPDAG) was computed via a Bayesian hill-climbing algorithm to estimate potential directionality among insomnia and its most associated disorder [based on SHAP (SHapley Additive exPlanations) values)]; (4) the CPDAG was then tested for generalizability by assessing (in)equality within a hold-out sample using structural hamming distance (SHD).

Results

Of 31,285 participants, 20,597 were women (65.8%); mean (standard deviation) age was 22.96 (4.52) years. The elastic net model demonstrated clinical significance in predicting insomnia severity in the training sample [R² = .44 (.01); RMSE = 5.00 (0.08)], with comparable performance in the hold-out sample (R² = .33; RMSE = 5.47). SHAP values indicated that the presence of any mental disorder was associated with higher insomnia scores, with major depressive disorder as the most important disorder associated with heightened insomnia (mean |SHAP|= 3.18). The training CPDAG and hold-out CPDAG (SHD = 7) suggested depression symptoms presupposed insomnia with depressed mood, fatigue, and self-esteem as key parent nodes.

Conclusion

These findings provide insights into the associations between insomnia and mental disorders among college students and warrant further investigation into the potential direction of causality between insomnia and depression.

Trial registration

Trial was registered on the National Institute of Health RePORTER website (R01MH115128 || 23/08/2018).

Sleep disturbance is often conceptualized as a transdiagnostic mechanism observed across a range of mental disorders [1] and, in some cases, was even included as a diagnostic criterion [2]. Unlike earlier conceptualizations of insomnia as merely a symptom or consequence of other mental health issues, an emerging perspective suggests that sleep and other mental disorders are intricately intertwined and bidirectional [3–5]. Such findings are not unexpected given that patients who reported sleep–wake disorders, notably and most commonly, insomnia, exhibited higher rates of comorbidity (e.g., 40% of those with insomnia vs. 16.4% with no sleep difficulties reported having additional disorders) [6]. Systematic reviews have also found associations between sleep disturbance and most mental disorders, including all anxiety disorders, depression, alcohol use disorder, and eating disorders [7–10]. Similarly, meta-analytic evidence found comorbidity was associated with increased rapid eye movement  pressure alterations and inhibited sleep depth [11]. However, there is a relative lack of representation of college samples despite their high prevalence of sleep difficulties. For instance, over 60% of U.S. students were categorized as having poor sleep quality [12], with an estimated 7.7% meeting criteria for insomnia [13]. Consequently, shorter durations of sleep and irregular sleep–wake schedules were significantly associated with a lower grade point average [14], along with poor sleep quality and increased odds of developing a mental disorder (e.g., OR = 2.36 [15]). Given the interrelationships between insomnia and several mental disorders within this population, further teasing apart of its relations may help us understand which mental disorder is most associated with heightened insomnia among U.S. college students and their probable directional relationships.

Embracing such complexity requires a nuanced approach. Unlike unregularized approaches (e.g., linear regression), supervised machine learning methods allow the opportunity to aggregate disparate small variable effects to inform clinical outcomes while also accounting for complex, interactive, or non-linear effects [16]. Elastic net regularization (elastic net), in particular, is preferable to unregularized approaches given its ability to minimize adverse effects of multicollinearity and reduce the probability of spurious, false positive associations [17]. Therefore, elastic net has led to parsimonious models with greater stability and accuracy and higher out-of-sample predictive performance relative to linear regression [18, 19]. Consequently, such models were used in recent studies to understand the relationship between insomnia and interacting symptoms. For example, utilizing baseline data from a controlled trial, Bard et al. [20] evaluated functional impairment and the relative importance of depressive and anxiety symptoms among insomnia patients. Also, Lyall et al. [21] employed actigraphy and mental health data from the UK Biobank to determine the most important sleep features (e.g., sleep duration, chronotype) related to depression and whether patients with poorer outcomes could be identified.

Further evidence is thus warranted in determining whether higher levels of insomnia are most related to the presence or absence of some disorders compared to others. McCallum et al. [22] used simple regression and found that both generalized anxiety disorder (GAD) and major depressive disorder (MDD) maintained independent associations with sleep disturbance, even after correcting for confounds, and were found to hold the greatest contribution among nine mental disorders. However, adopting a supervised machine learning approach using elastic net, which balances bias-variance trade-off may be a better approach.

Despite the advent of machine learning frameworks such as the seminal SHAP (Shapley Additive exPlanations) [23], which allow for interpretation of the magnitude of a variable’s impact on model predictions, revealing the structure of relations is a formidable challenge. Network analysis is one methodological approach suited for such an endeavor, given its telos of disentangling the complex dynamics of self-reinforcing causal interactions between symptoms [24]. Broadly, in this approach, a network comprises symptoms (nodes) and the associations between them (edges). In other words, an edge between nodes represents a conditional dependent relationship between two symptoms while keeping all other symptoms in the network constant [25]. Within this approach, hypotheses posit symptoms as causal agents that promote the development of other symptoms and, when unabated, go beyond a critical threshold and develop into a new harmful equilibrium known as a mental disorder [26, 27].

Insomnia as a node or a set of nodes has appeared in many prior cross-sectional network analyses, providing snapshots of associations between symptoms. Extant studies included examining insomnia's network structure [28–34] but also insomnia’s relationships with single disorders, such as MDD [35–38], post-traumatic stress disorder (PTSD) [39], psychosis [40], and schizophrenia [41], or with transdiagnostic factors, such as hyperarousal [42] or personality traits [43]. However, cross-sectional networks have also been developed between insomnia and multiple disorders, most commonly between MDD and GAD [20, 44–49] or with the further addition of PTSD [50], but also between prolonged grief disorder (PGD) and PTSD [51]. Most studies utilized the graphical Gaussian model (GGM), in other words, an undirected network of partial correlation coefficients, along with the graphical LASSO (least absolute shrinkage and selection operator [52]), as a regularization technique to avoid spurious, false-positive edges. However, as pointed out by Williams and Rast [53] and further highlighted by McNally et al. [54], graphical LASSO was developed and optimized for high-dimensional settings with more variables than the number of participants, which often is not the case in typical network structures thus leading to unwarranted sparsity. Moreover, despite efforts, the conventional GGM approach employed in such studies makes few inferences on potential directionality.

Conversely, Bayesian network analysis, such as directed acyclic graphs (DAGs), allows for estimating directed networks built on cross-sectional data. Although DAGs cannot confirm temporal precedence, it can provide preliminary clues to identify the direction of probabilistic dependence between edges [55]. In other words, if an edge originates from node X and connects to node Y (i.e., X → Y), node Y's presence suggests node X's presence more strongly than vice versa. Whereas the node considered the "parent" (X) might be present without its "offspring" (Y), the presence of the offspring indicates the presence of the parent. However, the assertion of causality is predicated on multiple conditions: these include the absence of any bidirectional causal relations (such as X causing Y and Y causing X) or causal loops (such as X causing Y, Y causing Z, and Z causing X); and second, the absence of any significant variables missing from the dataset [54]. To our knowledge, two studies on insomnia and common comorbidities have taken such a Bayesian approach. In one of these studies, Zhang et al. [56] elucidated associations between insomnia and depression and health-related behaviors (e.g., internet use, physical inactivity, smoking, alcohol consumption) among adolescents in China. In the other study, Yu et al. [57] examined the relationships between sleep disturbance and mental health (e.g., anxiety, depression, loneliness, well-being, health attitudes) among adults in China. However, whether such associations can be generalized to other demographic groups or other mental disorders requires further evaluation.

The current exploratory study thus intended to fill these gaps by first examining the relative importance of 7 disorders (MDD, GAD, social anxiety disorder [SAD], panic disorder [PD], PTSD, anorexia nervosa [AN], and alcohol use disorder [AUD]) association with insomnia within a nationally representative sample of treatment-seeking U.S. college students. We were then interested in exploring a Bayesian network analysis of the most predicted disorder at a symptom level. Thus, we used a straightforward 4-step statistical approach: (1) at the diagnostic level, an elastic net model, along with SHAP, was employed to illuminate the relative importance of mental disorders associated with insomnia and functioned as variable selection for further modeling; (2) the model was tested for generalizability in a hold-out sample; (3) at the symptom level, DAGs characterized the structure, relations, potential importance, and probable direction among insomnia and its most associated disorder; (4) inspired by Bard et al. [20], who randomly partitioned their data into training and hold-out samples to evaluate the replicability of their GGMs, we tested for generalizability of the DAGs by assessing (in)equality within a hold-out sample using structural hamming distance (SHD).

Methods

Participants

The current study was a secondary analysis of 39,194 treatment-seeking participants across 26 U.S. colleges and universities who participated in screening for an ongoing randomized controlled trial investigating the effectiveness of a transdiagnostic coached mobile mental health intervention that used population-level screening for engaging college students in tailored services for preventing and treating anxiety, depression, and eating disorders (clinicaltrial.gov; ID: NCT04162847). Participants were eligible for the screen if they were ≥ 18 years of age, enrolled at one of the 26 participating universities, provided informed consent to participate, and passed a one-item attention check. See Fitzsimmons-Craft et al. [58] for a more detailed description of eligibility criteria. Participants were excluded for only previewing the survey (n = 1), not responding to (n = 5,513) or denying (n = 503) consent for screening, being under 18 years of age (n = 63), not reporting their age (n = 1,154), not being an undergraduate student (n = 629) or not reporting their year in school (n = 46). The final sample was a nationally representative sample of 31,285 undergraduate students. All data for the present study were collected prior to selection for the randomized controlled trial or intervention delivery.

Procedures

Students enrolled at participating universities received an email invitation to complete a brief survey on health and well-being between October 2019 and November 2021. Emails were sent to either the entire student population or a random subset of the student population and either to undergraduate students from all years (17 schools) or only years 1 or 2 (9 schools). Emails informed students that based on their responses, they may be eligible for a subsequent study involving random assignment to conditions designed to support mental health. Emails included a link to an online screening survey via Qualtrics. Participating students were entered into a raffle to win one of several $100 gift cards. The study was approved by the institutional review board of all authors’ universities and administrators at each participating school.

Measures

All models were based on data captured pre-intervention delivery and included insomnia, MDD, GAD, PTSD, SAD, PD, AN, and AUD.

Insomnia was assessed using the Insomnia Severity Index (ISI; [59]). The ISI has seven questions with 5-point Likert scale responses, which are summed to produce a total score between 0 and 28, with higher scores indicating greater insomnia severity. Cronbach’s alpha for the present study was .884. Its internal consistency, concurrent validity, and sensitivity to clinical improvements in insomnia patients are well established [60].

MDD was assessed using the Patient Health Questionnaire-9 (PHQ-9; [61]). Participants reported frequency of depressive symptoms over the past two weeks on 9 items with four-point scales ranging from 0 (“Not at all”) to 3 (“Nearly every day”). Total score ranges from 0 to 27. Cronbach’s alpha for the present study was .877. Participants screened positive for probable MDD if they scored 10 or higher, maintaining a sensitivity of .88 and specificity of .85 [62].

PTSD was assessed using the Primary Care PTSD Screen (PC-PTSD; [63]), which has total scores ranging from 0 to 4. Participants screened positive for probable PTSD if they scored three or higher, which demonstrated a sensitivity of .78 and a specificity of .89 [63]. Cronbach’s alpha for the present study was .806.

GAD was assessed using the Generalized Anxiety Disorder Questionnaire-IV (GAD-Q-IV; [64]), maintaining a .82 specificity and .89 sensitivity, and has a total score ranging from 0 to 12. SAD was assessed using the Social Phobia Diagnostic Questionnaire (SPDQ; [65]), maintaining a .85 specificity and .82 sensitivity, and has a total score ranging from 0 to 27 [65]. PD was assessed using the Panic Disorder Self-Report (PDSR; [66]), maintaining a 1.00 specificity and .89 sensitivity, and has a total score ranging from 0 to 24 [66]. Cronbach’s alpha for the present study was .856, .97, and .959, respectively. These measures all assessed full diagnostic criteria based on the Diagnostic and Statistical Manual of Mental Disorders, 5th edition [2]. Participants screened positive for a disorder if they endorsed all diagnostic criteria. GAD-Q-IV, SPDQ, and PDSR demonstrate strong retest reliability, good convergent and discriminant validity, and a kappa agreement of .67, .66, and .93, with structured interviews, respectively.

AN was assessed by the Weight and Shape Concerns Scale (WCS; [67]). Total scores for the weight/shape concerns scale range from 0 to 100. Participants screened positive for probable AN if they scored 59 or higher and had a current body mass index ≤ 18.45, based on self-reported height and weight. Cronbach alpha for the present study was .797. These criteria have been used in prior online screening studies [68].

AUD was assessed using the Alcohol Use Disorders Identification Test Consumption (AUDIT; [69]). The instrument contains three questions about alcohol consumption with 4-point Likert scale responses, which are summed to obtain a total score ranging from 0 to 12. Cronbach’s alpha for the present study was .85. To identify probable AUD, we used the clinical cut-off of 4 or higher for participants assigned male at birth and 3 or higher for participants assigned female or intersex. This system had .88 sensitivity and .75 specificity for males and .87 sensitivity and .85 specificity for females [70].

Statistical analysis

Pre-processing

The data were randomly partitioned into a 70% split as a training set and a 30% hold-out set to evaluate the final models in completely unseen new cases. Missing values for the included variables in our sample were low and were similar across sets (6.26% and 6.41%, respectively).

Nonetheless, to tackle missing data for all analyses, a machine learning approach for imputation was employed, specifically utilizing nonparametric missing value imputation via random forests facilitated by the R package mice [71]. Imputations were aggregated across 10 multiple imputed datasets, each with 100 iterations, to minimize biased error calculations and produce stable estimates. Random forest imputations were done separately for the training and hold-out sets. Minimal recoding adjustments were made before each imputation to maintain the inherent relationships between variables (as recommended by van Ginkel et al. [72]). Moreover, to prevent “data leakage” of variable distributions between sets, all pre-processing steps were done separately for training and hold-out sets. Topological overlap between node pairs was also screened for and removed if found via the “goldbricker” function within the R package networktools [73].

Supervised machine learning (Elastic net regularization)

Elastic net development

Elastic net regularization is a form of conventional regression that combines both ridge and LASSO norms. It provides a penalization term to balance stability and parsimony with a lambda hyperparameter. In this way, it  determines magnitude and an alpha hyperparameter regulating the balance between the two norms. Tuning of alpha and lambda was conducted using a resampling grid search. The final model was selected using repeated tenfold cross-validation to minimize biased estimates of the true error and assess the stability of model performance [74]. Tenfold cross-validation partitions the sample into 10 subsets, 9 of which are used in the training process and then tested on the remaining subset [75]. This process is iterated for the remaining 10 subsets, building new models until each of the 10 subsets is used only once in the training and testing data. This procedure then repeats the 10 folds by 10 repeats for a total of 10 models. The final model is then averaged to produce a single estimate. Final alpha and lambda values were selected based on the smallest value of root mean square error (RMSE) and were used to estimate model coefficients.

In the current study, the elastic net model considered 7 disorders (MDD, GAD, PTSD, SAD, PD, AUD, AN) as binary variables (i.e., presence vs absence) and insomnia as a continuous outcome (i.e., total ISI score). This approach was used because we were first interested in understanding which clinical disorder would be most associated with insomnia symptoms. Imbalance within the outcome was also addressed by applying the synthetic minority over-sampling technique for regression with Gaussian noise (SMOGN; [76]), which randomly undersamples high-frequency cases and oversamples rare cases using SmoteR and Gaussian noise to generate a more balanced proportion of cases within the continuous outcome and improve prediction accuracy. Imbalance occurs when machine learning models favor predictions from high-frequency cases and ignore rare cases, given preferences for high accuracy even if purely by chance. All analyses were conducted in R version 4.3.1 using the caret package [77].

Elastic net evaluation

The cross-validated elastic net model built from the training sample was evaluated by being applied to individuals within the hold-out sample to predict insomnia severity. Importantly, individuals within the hold-out sample were not utilized as part of the development and tuning of the elastic net model. RMSE determined the accuracy of the model, i.e., the magnitude of error. Lower values represent higher accuracy. The coefficient of determination (R²) was also used, given evidence of R² being the most informative metric within regression-based supervised machine learning [78]. R² determined predictability, i.e., the proportion of variance within the outcome explained by the elastic net model. Values are interpreted as percentages and range from 0 to 1, with higher values representing higher predictability. The current study adopted the benchmark set by Uher et al. [79], who found an R² of 6.3 or higher inferred clinical significance.

Elastic net variable importance

Methods for explainable artificial intelligence were run using SHAP (Shapley Additive exPlanation) values [23] to facilitate the interpretability of the elastic net model. SHAP values assign a value to each variable that represents the average contribution of that variable across all possible combinations of variables. The average SHAP value across all participants is 0, but the average absolute SHAP value informs relative variable importance. We are not aware of any thresholds that could serve as an empirical cut-off for variable selection using SHAP values. Given our use of SHAP values in determining subsequent modeling, we selected any variable above 1 in SHAP values to be included in the network analyses.

Bayesian networks (directed acrylic graphs)

Network estimation

DAG analyses were run via the hill-climbing algorithm from the R package, “bnlearn” [80], to determine the potential directionality and conditional dependencies among symptoms. DAGs return a network comprising symptoms (nodes) and the relations between them (edges). To create the DAGs, a bootstrap function computes the structural aspect of a network by adding edges, removing them, and reversing their direction to optimize a goodness-of-fit score (i.e., Bayesian information criterion [BIC]). This step determines whether an edge exists; however, it does not calculate the weights of edges. To do so, we randomly restarted the process with different candidate edges linking different symptom pairs, perturbing the system. To ensure robustness, we used 50 restarts (as per Briganti et al., [55]) and 100 permutations (as implemented by McNally et al. [81, 82]). In the current study, we employed a Bayesian network via a completed partially directed acrylic graph (CPDAG), a type of Markov equivalence class that encodes identical conditional dependencies between DAGs and accounts for drawbacks of equivalent separate DAGs [83]. Insomnia was included in the DAG analyses as a single-sum score derived from the ISI representing insomnia severity, with all items of the PHQ minus item 3 [insomnia/hypersomnia] to prevent multicollinearity.

Network stability

To verify the stability of the resultant network, we bootstrapped 10,000 samples, computed a network for each sample, and averaged all 10,000 networks to obtain the final network. Following the reasoning of Briganti et al. [55], we first determined the structure of the network and then ascertained the direction of each edge. The bnlearn program computes a BIC value for each edge. The thickness of an edge corresponds to its absolute BIC value and, hence, its importance to model fit. The larger the absolute BIC value, the more damaging it would be to the model fit if one were to remove the edge from the network. Accordingly, high absolute BIC values indicate how important an edge is to the model that best characterizes the data structure. In line with Sachs et al. [84], if an edge ran from symptom X to symptom Y in at least 85% of the bootstrapped networks, this edge appeared in the final, averaged network. After which, if an edge ran from symptom X to symptom Y in at least 51% of the bootstrapped networks, its direction was depicted using an arrow pointing from node X to node Y. Accordingly, such significance thresholds promoted the stability of the final averaged network and led to sparse networks that ensured genuine edges. Lastly, we then computed the identical network but had edge thickness reflect the probability for which the depicted direction of the edge occurred.

Network confirmatory analysis

Often, the reliability and replicability of parameter estimates in cross-sectional network analyses are not considered and are, at the least, questionable (e.g., [85–88]). For the current investigation, three steps were taken to ensure model stability: (1) random perturbations to avoid local maxima and optimize goodness-of-fit index (i.e., BIC values); (2) bootstrapping 10,000 different DAGs to determine strength and direction of the edges; (3) using significance thresholds outlined in Sachs et al. [84]. Supplementing our bootstrapped stability tests, a confirmatory analysis was run by repeating steps 1–3 for a CPDAG built on a hold-out sample and then assessing for (in)equality between the CPDAG developed on the training sample. To test for the similarity between the training and hold-out CPDAGs, structural hamming distances (SHD) were used, which quantified the sum number of changes between nodes, arcs, and directions that were required of a network for it to turn into the one that was being compared [89]. In other words, calculating the true positive, false positive, and false negative arcs by comparing the training network to the hold-out network, considered the "true" standard network. This allowed for testing whether the network estimation was roughly consistent across both data subsets, further suggesting generalizability.

Results

Sample characteristics

Screening sample characteristics for the entire sample are presented in Table 1, and diagnostic severity, split between training and hold-out samples, may be found in Table 2. Most participants identified as female (63.4%), heterosexual (72.7%), white (65.7%), and non-Hispanic (67.5%).

Table 1.

Sample distribution across demographic characteristics

	N	%
Sex (at birth)
Male	9950	31.8%
Female	20,597	65.8%
Gender
Male	9798	31.3%
Female	19,823	63.4%
Other	1664	5.3%
Sexual orientation
Heterosexual	22,759	72.7%
Lesbian/Gay	1002	3.2%
Bisexual	3453	11.0%
Other	3211	10.3%
Race
White	20,563	65.7%
Black or African American	1992	6.4%
American Indian or Alaskan Native	164	0.5%
Asian	4236	13.5%
Native Hawaiian or Pacific Islander	83	0.3%
Multiracial	1790	5.7%
Ethnicity
Non-Hispanic	18,673	59.6%
Hispanic	3143	10.0%
Parental education
Bachelor's degree or higher	21,126	67.5%
Less than bachelor's degree	9160	29.3%
Financial difficulty
Not Very Hard	13,075	41.8%
Somewhat Hard	9570	30.5%
Hard	4913	15.7%
Very hard	2833	9.1%
Year in school
Undergraduate 1	12,807	40.9%
Undergraduate 2	9014	28.8%
Undergraduate 3	5007	16.0%
Undergraduate 4	3725	11.9%
Undergraduate 5	732	2.3%
School sector
Public	26,783	85.6%
Private	4502	14.4%

N = 31,285

Table 2.

Diagnostic severity by data partition

Disorder	Training sample		Hold-out sample
	Clinical	Non-clinical	Clinical	Non-clinical
Insomnia	18.49 (3.18)	6.84 (4.04)	18.39 (3.19)	6.86 (4.07)
MDD	15.50 (4.43)	4.94 (2.73)	15.42 (4.37)	5.01 (2.74)
GAD	10.36 (1.23)	3.30 (2.92)	10.35 (1.24)	3.31 (2.93)
SAD	19.37 (3.14)	10.65 (6.45	19.51 (3.06)	8.75 (5.26)
PD	17.20 (2.75)	2.19 (4.71)	17.22 (2.73)	2.13 (4.58)
PTSD	3.55 (0.50)	0.65 (0.82)	3.57 (0.49)	0.67 (0.83)
AN	78.14 (12.00)	40.05 (25.67)	75.50 (10.48)	40.46 (25.59)
AUD	5.13 (1.83)	0.62 (0.89)	5.10 (1.81)	0.62 (0.90)

Training sample (n = 21,899); hold-out sample (n = 9,386)

Elastic net regularization

A total of 21,899 participants were included in the training models, and 9,386 participants were included in the hold-out models. Run on the full training sample, the elastic net model derived from repeated tenfold cross-validation was associated with an optimal alpha parameter of 0.1 and a lambda parameter of 0.008 (via RMSE criterion). The final elastic net model demonstrated clinical significance in predicting insomnia severity in the training sample [R² = .44 (.01), RMSE of 5.00 (0.08)], with comparable variance explained in the hold-out sample (i.e., completely unseen new cases; R² = .33, RMSE of 5.47). Results of the SHAP variable importance ranking are displayed in Fig. 1, in which SHAP values illustrated that MDD (SHAP = 3.18, cf. β = 6.39) was the most important disorder associated with insomnia, thus apt for further modeling, followed by GAD (SHAP = 0.96, cf. β = 2.30) and PTSD (SHAP = 0.96, cf. β = 2.18).1 It should be noted, however, that the presence of any of the seven mental disorders was associated with higher insomnia scores.

Fig. 1.

SHAP variable importance Note. To determine relative importance, we used a gold standard interpretability method termed SHapley Additive explanation (SHAP). SHAP values provide a more comprehensive understanding of each variable’s contribution to the model's predictions

Directed acrylic graphs

The CPDAG built on training data (N = 21,899), as displayed in Fig. 2, shows a chain of symptoms dependent on the parent node of depressed mood, which directly predicted fatigue, anhedonia, poor self-esteem, concentration problems, eating problems, psychomotor disturbance, suicidal ideation, or insomnia. That is, depressed mood had no incoming edges (i.e., in-degree = 0) but had eight outgoing edges (i.e., out-degree = 8). The most important arrows connected depressed mood to fatigue (with a change in BIC of -4067.81 and a directional probability of 50.52%) and depressed mood to poor self-esteem (with a change in BIC of -3294.17 and a directionality probability of 55.71%). Accordingly, fatigue emerged as a key step in the cascading node with one incoming arrow (i.e., in-degree = 1) and five direct descendants (out-degree = 5): anhedonia, poor self-esteem, concentration problems, eating problems, or insomnia. There were seven total paths for insomnia (depressed mood, fatigue, anhedonia, poor self-esteem, concentration problems, eating problems, or psychomotor disturbance). In other words, all depression symptoms, except for suicidal ideation, presupposed insomnia. That is, insomnia was more likely when depressed mood, fatigue, anhedonia, poor self-esteem, concentration problems, eating problems, or psychomotor disturbance were present than vice versa. Suicidal ideation occurred only through depressed mood, poor self-esteem, or psychomotor disturbance. This could have arisen from eating problems or concentration problems, and depressed mood or poor self-esteem. Suicidal ideation or insomnia were the only symptoms without any descendants and, thus, were not a prerequisite for any other symptoms. Additional DAGs with arrow thickness denoting directional probability using Sachs et al.'s [84] approach were also run and can be found in Fig. 3. BIC values employed in Fig. 2 and directional probability values in Fig. 3 are listed in Table 3.

Fig. 2.

CPDAG importance. Note. Built on the training sample (n = 21,899). Arrow thickness denotes a change in the Bayesian Information Criterion (BIC; a relative measure of a model’s goodness-of-fit) arising from the proportion of the averaged 10,000 bootstrapped networks wherein that arrow is removed from the network. In other words, the more an arrow contributed to the model fit, the thicker it is

Fig. 3.

CPDAG directional probability. Note. Built on the training sample (n = 21,899). Edge thickness signifies directional probabilities arising from the proportion of the averaged 10,000 bootstrapped networks wherein that arrow was pointing in that direction, or, in other words, confidence that the direction of prediction flows in the direction depicted in the graph

Table 3.

Model fit importance and directional probability of CPDAG

From	To	BIC	Directional probability
Depressed mood	Fatigue	-4067.81	50.52%
Poor self esteem	Concentrate problems	-422.89	52.42%
Poor self esteem	Suicidal ideation	-706.11	54.61%
Concentrate problems	Anhedonia	-258.38	54.63%
Fatigue	Poor self esteem	-425.75	55.18%
Depressed mood	Poor self esteem	-3294.17	55.71%
Poor self esteem	Anhedonia	-70.23	56.48%
Fatigue	Concentrate problems	-844.15	57.60%
Depressed mood	Concentrate problems	-389.82	58.12%
Insomnia	Psychomotor problems	-136.30	58.40%
Eating problems	Insomnia	-257.32	58.46%
Depressed mood	Suicidal Ideation	-959.96	60.10%
Fatigue	Anhedonia	-450.74	61.36%
Depressed mood	Anhedonia	-1745.17	61.87%
Anhedonia	Eating problems	-77.82	62.60%
Suicidal Ideation	Psychomotor disturbance	-262.77	67.22%
Eating problems	Psychomotor disturbance	-101.78	68.04%
Concentrate problems	Eating problems	-232.44	70.12%
Anhedonia	Insomnia	-21.12	70.28%
Poor self esteem	Eating problems	-243.91	72.67%
Suicidal ideation	Insomnia	-19.41	73.82%
Concentrate problems	Psychomotor disturbance	-661.91	75.42%
Poor self esteem	Psychomotor disturbance	-70.02	76.02%
Depressed mood	Psychomotor disturbance	-22.40	76.96%
Fatigue	Eating problems	-718.30	77.39%
Depressed mood	Eating problems	-75.04	77.89%
Concentrate problems	Insomnia	-104.80	78.12%
Poor self esteem	Insomnia	-17.72	80.42%
Fatigue	Insomnia	-461.96	85.09%
Depressed mood	Insomnia	-14.26	85.49%

Built on training sample (n = 21,899). BIC Bayesian Information Criterion. Importance to model fit is characterized by BIC values, in other words, the change in BIC when the arrow in reference is removed from the network. Larger negative scores reflect increased importance for model fit. Directional probability values characterize the frequency of arrow and direction in reference within 10,000 bootstrapped networks

Structural distance

As observed in Fig. 4, 7 needed operators were found (SHD = 7; i.e., adding or deleting an undirected edge, and adding, removing, or reversing the orientation of an edge; [89]) within the hold-out CPDAG (N = 9386) to match the training CPDAG (N = 21,899). The parent node of depressed mood and fatigue, as a cascading node, along with its five direct descendants, anhedonia, poor self-esteem, concentration problems, eating problems, or insomnia, remained the same across networks. However, there were false positives for which directions switched within the hold-out network as compared to the training network or the “true network.” These arrows were concentration problems related to anhedonia or insomnia related to psychomotor disturbance. Accordingly, within the hold-out network, insomnia attained one direct descendant, signifying that psychomotor disturbance was more likely when insomnia was present than vice versa. Suicidal ideation also gained two descendants: psychomotor disturbance or insomnia. In other words, suicidal ideation occurred only through depressed mood or poor self-esteem and directly predicted insomnia or psychomotor disturbance. Thus, nodes without any descendants switched from suicidal ideation and insomnia to psychomotor disturbance within the hold-out sample, implying that suicidal ideation was not a prerequisite for other symptoms.

Fig. 4.

Similarity between training and hold-out CPDAGs. Note: training sample (n = 21,899); hold-out sample (n = 9,386). CPDAG = completed partially directed acyclic graph. Green arrows = true positives; Red arrows = false positives; SHD = structural hamming distance. SHD assesses the similarity between two CPDAGS and represents the number of edge insertions, deletions or flips to transform one graph to another graph. Lower SHD values represent higher similarity

Discussion

The present study set out to investigate associations between insomnia and common mental disorders within a large nationally representative sample of treatment-seeking U.S. college students to (a) determine whether higher levels of insomnia were most related to the presence or absence of some disorders compared to others and (b) tease apart the potential directionality between insomnia and symptoms of its most associated disorder.

We first used a broad range of disorders (MDD, GAD, PTSD, SAD, PD, AUD, AN) to predict insomnia severity using elastic net regularization. The elastic net model accounted for 33% (R² = .33) of the variance in insomnia, in part due to the inclusion of MDD, which SHAP values identified as the top disorder most associated with insomnia. GAD and PTSD, respectively, were also listed as secondary and tertiary disorders contributing to the model’s performance but to a lesser degree. Findings were in parallel with Bard et al. [20], who found MDD symptoms (e.g., low energy, depressive affect via PHQ-9) to be key features across multiple domains of sleep functioning and impairment as compared to anxiety (GAD-7; [90]) and insomnia symptoms (SCI-9; [91]). Our results also converge with McCallum et al. [22], who found GAD, MDD, and PTSD, respectively, as the top contributors to sleep disturbance, although findings switched between the first and secondary top contributors. Discrepancies may be due in part to sample differences, as we utilized a representative sample of college students in the American population as compared to McCallum et al. [22], who noted self-selection bias within their Australian general community sample. It may also have been due to measurement error, given the prior study's usage of non-validated self-report checklists based on DSM-5 criteria as compared to the present study, which used valid and reliable diagnostic self-report measures with adequate kappa agreement with structured interviews (e.g., GAD-Q-IV, SPDQ, PDSR, PC-PTSD). In addition, our analytic approaches diverged from theirs, given that the present study derived variable importance via the explanatory power of a machine learning model with all disorders contained in the model as compared to p values from separate regressions for each disorder tested.

DAG analyses were conducted to offer additional insight as to how MDD symptoms may have led to insomnia. Depressed mood was found to be the most important parent symptom, directly predicting fatigue, anhedonia, poor self-esteem, concentration problems, eating problems, psychomotor disturbance, suicidal ideation, or insomnia. Stated differently, the presence of fatigue, anhedonia, poor self-esteem, concentration problems, insomnia, eating problems, or psychomotor disturbance each presupposed the presence of depressed mood more than vice versa. In a typical DAG structure, higher upstream nodes are given greater predictive priority, whereas downstream nodes carry less activation potential and are less likely to influence other symptoms in the network. These findings suggested that insomnia was seemingly dependent on other downstream symptoms in the network, indicating that the occurrence of insomnia more likely depended on the presence of MDD symptoms rather than vice versa. Notably, network estimation related to parent nodes was consistent across both training and hold-out samples, further suggesting replicability. However, caution is warranted when inferring nodes with no descendants (i.e., not a prerequisite for other symptoms) as discrepancies between samples were observed. Future simulation studies are needed to determine the typical conditions when differences in network estimations arise between data subsets and their implications on validity.

Nonetheless, our findings are consistent with DSM-5 guidelines on MDD [2], suggesting that depressed mood is a hallmark feature of MDD and is one of the two main symptoms required for assigning a positive diagnosis [92]. Moreover, findings of depressed mood as a parent symptom aligned with extant network reviews on MDD [87, 93–95] and investigations that set out to identify MDD’s most important central symptoms (e.g., [96–99]). Results also aligned with studies on the association between MDD and insomnia [34, 37, 50, 57]. Insomnia was commonly found to be a robust risk factor for both first episode and recurrent depressive episodes [100]. Mechanistically speaking, Harvey [101] denoted that such associations occurred due to the presence of a bidirectional cycle. Disturbances in mood and symptoms during the day disrupt nighttime sleep, whereas sleep deprivation worsens mood regulation and symptoms the following day [4], creating a vicious cycle. Such cycles further persist, given that individuals with mood disorders are vulnerable to disruptions in biological rhythms and that external stressors can lead to such disruptions in biological rhythms [102]. Accordingly, college populations may be prone to such cycles, considering their increased physiological changes, heavy academic workload, and psychosocial stressors [15].

DAG analyses also implicated the presence of insomnia as probabilistically dependent on the presence of either fatigue or poor self-esteem. These findings are in line with existing centrality findings of depressed mood, fatigue, and self-esteem symptoms emerging across western [97, 103–105] and eastern cultures [106–108]. Furthermore, other findings implicated depressed mood directly leading to fatigue [57] or indirectly impacting insomnia through fatigue [109]. In fact, fatigue has been reported as the highest bridge symptom linking depression and insomnia symptoms [35, 36, 48, 110].

Our findings also have implications for treatment targets among patients with comorbidities. Results suggested that the interrelationships of depressed mood, fatigue, and self-esteem presupposed insomnia. Untreated comorbid insomnia with depression has been shown to maintain the risk of relapse due to both disorder’s links with mood dysregulation [101, 111]. As such, the presence of both disorders should be assessed during population-level screening and patient management. However, and notably, depression treatment does not inherently ameliorate insomnia. For example, sleep-related complaints are often the most common residual symptoms after antidepressant treatment [112], warranting targeted insomnia treatment [113–115]. Interventions for insomnia often necessitate specific behavioral strategies (e.g., sleep hygiene), which are not inherent to pharmacotherapy or cognitive behavioral therapy (CBT) for MDD (for review, see [112]). As CBT for insomnia (CBT-I) has also been shown to be effective in treating both insomnia and depressive symptoms among those who have both, further randomized clinical trials are needed to determine if treatment combinations are better than either approach alone, for example, evaluating treatment efficacy comparing CBT-I and CBT for depression to CBT-I vs. CBT for depression.

The current study is not without caveats and deserves careful consideration. All analyses were based on observational and exploratory data rather than experimental. Although Bayesian learning methods can enable probabilistic causal inferences, networks derived from such data cannot make strong inferences of causation from cross-sectional data. To make such inferences within the network paradigm requires additional assumptions (e.g., [116–118]). Also, our CPDAG models rested on several key assumptions inherent to Bayesian networks, including the assumption of causal relations among symptoms and acyclicity and that no important variables were excluded from the network. There were reasons to suspect that the acyclicity assumption may have been violated, given the degree of potential reverse directionality. Here, arrows deemed most important seemed relatively thin, indicating that the arrow was pointing in both directions in a substantial percentage of bootstrapped networks. For example, depressed mood and fatigue almost certainly had a bidirectional influence on one another. Accordingly, the edge connecting depressed mood to fatigue pointed in that direction in 50% of the 10,000 bootstrapped networks. The direction of the association between these two variables may thus have tipped in both directions, implying a possible 'hidden' cycle within an acyclic graph. The impact of violating the assumption of acyclicity is unknown but, at a minimum, implies that the current DAG analyses failed to detect feedback loops. Accordingly, failing to meet acyclicity may have also led to skewed interpretations of true positive, false positive, and false negative arc comparisons between training and hold-out samples, given that there could have been bi-directionality. Hence, a major limitation of the present findings is it may be treated as only a simplified snapshot of probabilistic causal relations. We are not aware of any studies that used temporal network analyses to investigate directional relationships between MDD and insomnia. Although Jordan et al. [119] found evidence of longitudinal associations between sleep processes and depressed mood within a naturalistic setting, results may not generalize to patients with MDD due to sample recruitment strategies (i.e., homogeneous sample of nurses) and measure choices (e.g., a single-item indicator of depressed mood). Future studies could improve upon our approach by gathering time-series data to detect feedback loops to elucidate potential bidirectional dependencies between depression and insomnia.

The present study reveals associations related to insomnia and common comorbidities within a nationally representative sample of treatment-seeking U.S. college students. Results illuminated MDD as the most important association that contributed to heightened insomnia with the interrelationships of depressed mood, fatigue, and self-esteem, presupposing insomnia. These findings serve as a foundation for generating hypotheses rather than conclusive causal evidence emphasizing the need for further research into the temporal precedence between insomnia and common comorbidities. The presented approach to computing Bayesian network analyses may be helpful when developing supervised machine learning models in teasing apart the structure of relations and probable directional relationships between features and predicted outcomes.

Abbreviations

AN

Anorexia nervosa

AUD

Alcohol use disorder

AUDIT

Alcohol use disorders identification test

BIC

Bayesian information criterion

CBT-I

Cognitive behavioral therapy for insomnia

CPDAG

Completed partially directed acrylic graph

DAG

Directed acyclic graphs

GAD

Generalized anxiety disorder

GAD-Q-IV

Generalized Anxiety Disorder Questionnaire-IV

GGM

Graphical Gaussian model

ISI

Insomnia Severity Index

LASSO

Least absolute shrinkage and selection operator

MDD

Major depressive disorder

PC-PTSD-5

Primary Care PTSD Screen

PD

Panic disorder

PDSR

Panic Disorder Self-Report

PGD

Prolonged grief disorder

PHQ-9

Patient Health Questionnaire-9

PTSD

Post-traumatic stress disorder

R²

Coefficient of determination

RMSE

Root mean square error

SAD

Social anxiety disorder

SHAP

Shapley Additive exPlanations

SHD

Structural hamming distance

SMOGN

Synthetic minority over-sampling technique for regression with Gaussian noise

SPDQ

Social Phobia Diagnostic Questionnaire

WCS

Weight and Shape Concerns Scale

Authors’ contributions

AC: Conceptualization, Methodology, Software, Formal analysis, Data Curation, Visualization, Writing – original draft preparation, Writing – review & editing. SYB, MHSN: Data Curation, Writing – original draft preparation. EEFC, DE, DEW, CBT: Investigation, Writing – review and editing, Project administration, Funding acquisition. MGN: Conceptualization, Methodology, Validation, Investigation, Writing – review and editing, Supervision, Project administration, Funding acquisition. All authors read and approved the final manuscript.

Funding

The current study was funded by the National Institute of Mental Health (R01MH115128 || 23/08/2018).

Availability of data and materials

The dataset used and/or analyzed during the current study are available from the corresponding author upon reasonable request.

Declarations

Ethics approval and consent to participate

All participants gave informed consent to participate in the study. The protocol was approved by the Institutional Review Boards (IRB) at the coordinating universities (IRB ID #: 201901073).

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.