Gender Differences in Machine Learning Models of Trauma and Suicidal Ideation in Veterans of the Iraq and Afghanistan Wars

Abstract

Suicide rates among recent veterans have led to interest in risk identification. Evidence of gender-and trauma-specific predictors of suicidal ideation necessitates the use of advanced computational methods capable of elucidating these important and complex associations. In this study, we used machine learning to examine gender-specific associations between predeployment and military factors, traumatic deployment experiences, and psychopathology and suicidal ideation (SI) in a national sample of veterans deployed during the Iraq and Afghanistan conflicts (n = 2,244). Classification, regression tree analyses, and random forests were used to identify associations with SI and determine their classification accuracy. Findings converged on several associations for men that included depression, posttraumatic stress disorder (PTSD), and somatic complaints. Sexual harassment during deployment emerged as a key factor that interacted with PTSD and depression and demonstrated a stronger association with SI among women. Classification accuracy for SI presence or absence was good based on the receiver operating characteristic area under the curve, men = .91, women = .92. The risk for SI was classifiable with good accuracy, with associations that varied by gender. The use of machine learning analyses allowed for the discovery of rich, nuanced results that should be replicated in other samples and may eventually be a basis for the development of gender-specific actuarial tools to assess SI risk among veterans.

The suicide rate among Iraq and Afghanistan war veterans has resulted in calls to improve suicide risk identification (Kemp & Bossarte, 2012). Research among veterans of these wars has consistently demonstrated associations between suicidal ideation (SI) and psychopathology including posttraumatic stress disorder (PTSD) and depression (Gradus, Street, Suvak, & Resick, 2013; Jakupcak et al., 2009; Lemaire & Graham, 2010; Pietrzak et al., 2010), alcohol abuse (Gradus, Street, et al., 2013; Pietrzak et al., 2010). The broad exposure to potentially traumatic deployment events associated with the insurgency warfare nature of these conflicts, combined with an observed increase in the suicide rate among recent veterans specifically, has also resulted in an interest in the association between various forms of trauma and SI in this population. Studies in this population have shown that combat experiences (Gradus, Street, et al., 2013; Maguen et al., 2011) and traumatic brain injuries (TBIs; Gradus et al., 2015; Wisco et al., 2014) are associated with SI. Further, there is a growing body of literature documenting an association between sexual harassment during military service and SI across recent veteran samples. Studies have documented associations between sexual harassment during deployment and SI while accounting for mental health symptoms among female veterans specifically (Gradus, Street, et al., 2013). Recently, sexual harassment during deployment was shown to be associated with SI in a combined sample of male and female veterans after adjusting for combat exposure (Monteith, Menefee, Forster, Wanner, & Bahraini, 2015). Among military personnel and veterans currently enrolled in college classes, moderately sized associations were found for premilitary and military sexual trauma and SI in a gender-stratified sample, although some were nonsignificant, likely owing to the small sample size (Bryan, Bryan, & Clemans, 2015).

Suicide and associated thoughts and behaviors are known to have complex etiology. Yet, traditional regression models are not equipped to identify complex interacting risk structures empirically; rather, they are designed for null hypothesis testing of a priori-specified associations (Hald, 1998). The literature examining the complex associations between gender, potentially traumatic deployment experiences, psychopathology, and SI in recent veterans has been limited by the use of traditional regression methods, which are designed to characterize only a small piece of a complex casual picture. Supervised machine learning classification algorithms, such as classification tree analyses and random forests, are capable of identifying, validating, and determining the predictive accuracy of large sets of risk factors, thus conferring the ability to elucidate a broader causal picture than traditional regression while encompassing predictors from varied domains (Hastie, Tibshirani, & Friedman, 2001). Studies have demonstrated that supervised machine learning procedures accurately classify suicidal behavior among nonmilitary clinical samples (Baca-Garcia et al., 2006; Delgado-Gomez, Blasco-Fontecilla, Sukno, Ramos-Plasencia, & Baca-Garcia, 2012; Lukaschek et al., 2014; Mann et al., 2008) in U.S. Army soldiers (Kessler et al., 2014, 2016), suicide among Veterans Health Administration (VHA) patients with depression (Ilgen et al., 2009), and suicide among VHA patients overall (McCarthy et al., 2015). No study has used machine learning to examine predictors of SI among Iraq and Afghanistan veterans irrespective of VHA user status, an important consideration, given that only 58% of veterans from these conflicts use VHA care (Department of Veterans Affairs, Epidemiology Program, Post-Deployment Health Group, 2014).

We previously published two reports examining probable psychiatric diagnoses as mediators of the association between negative deployment experiences (e.g., combat exposure, sexual harassment, TBI) and SI (Gradus, Street, et al., 2013; Gradus et al., 2015). For men and women, probable psychopathology fully mediated associations between negative deployment experiences and SI, with two exceptions. First, there is an independent association between sexual harassment during deployment and SI that was specific to women, even after probable psychopathology was considered (Gradus, Street, et al., 2013). Second, among men, an association between TBI and SI remained after probable psychopathology was added to the model (Gradus et al., 2015). Taken together, these results indicate that risk for SI is associated with multiple pathways that might be gender-specific, but the existence of complex interactions between important predictors makes it difficult to further elucidate these pathways with traditional regression.

In the current work, we expand upon this previous work and apply classification tree and random forest analyses to examine variables from a variety of domains (e.g., predeployment factors, military and demographic characteristics, deployment experiences, psychopathology) to identify gender-specific associations with SI among a national sample of veterans deployed in support of the Iraq and Afghanistan conflicts.

Method

Participants and Procedure

A description of the parent study has been published (Street, Gradus, Vogt, Giasson, & Resick, 2013). Potential participants (n = 6,000) were randomly sampled (within gender) from the VHA Environmental Epidemiology Service roster (women oversampled). Participants were deployed in support of the conflicts in Iraq and Afghanistan, but not necessarily to Iraq or Afghanistan, as some military members were deployed elsewhere during these conflicts (e.g., Kuwait). Participants included 1,139 male and 1,209 female veterans (48.6% response rate, accounting for ineligible nonresponders). Participants who reported SI, but no or few current mental health symptoms were removed from the sample (1,062 males and 1,099 females), as these participants are likely reporting history of SI occurring before the time of the survey. Because predictors of suicidal ideation and suicide attempts would be expected to differ, participants in the parent study who reported more severe suicidal behavior (i.e., suicide plans or suicide attempt; n = 73) were excluded from the present analyses. Given other work establishing gender differences in SI predictors among these veterans (Gradus, Street, et al., 2013), we conducted gender-stratified analyses. Table 1 displays characteristics of the sample. Veterans were contacted with this mail survey using Dillman’s tailored design method (Dillman, 2007). This study was approved by the VA Boston Healthcare System Institutional Review Board (Boston, MA).

Table 1.

Characteristics of the Sample, Stratified by Suicidal Ideation and Gender

Variable	Men (n = 1,062)				Women (n = 1,099)
	Suicidal ideation (n = 179)		No suicidal ideation (n = 883)		Suicidal ideation (n = 191)		No suicidal ideation (n = 908)
	M	SD	M	SD	M	SD	M	SD
Age, years	34.67	9.98	37.68	10.11	32.97	8.42	34.93	9.01
Number of deployments	1.46	0.68	1.45	0.79	1.32	0.54	1.35	0.69
Total time deployed (in months)	14.17	8.86	11.87	8.44	11.74	6.63	10.57	7.62
	n	%	n	%	n	%	n	%
4-year college degree or more education	53	29.8	289	32.9	66	34.5	446	49.2
In a romantic relationship	121	68.0	714	82.2	125	66.1	684	75.5
Have children	106	60.6	613	70.0	110	57.9	515	56.8
Any premilitary trauma	140	83.3	523	64.0	131	75.7	507	59.1
Deployed to Iraq or Afgahanistan	150	86.2	662	77.7	146	79.8	295	33.6
Army or Marines military branch	145	81.0	554	63.0	132	69.8	507	56.2
Active duty military component	104	58.4	413	46.9	115	60.8	441	48.9
Enlisted rank	150	88.8	713	83.5	158	85.4	703	79.3
Any combat experiences	162	90.5	690	78.7	161	84.7	623	68.8
Any aftermath of battle experience	159	88.8	667	76.0	162	85.3	629	69.7
Any sexual harassment	29	16.3	73	8.3	134	70.2	384	42.4
Any general harassment	136	77.7	579	66.3	173	91.5	654	72.5
Any lack of unit support	122	68.2	412	46.8	136	71.2	493	54.4
Any life and family concerns	172	97.7	816	93.3	184	96.3	856	94.7
Positive traumatic brain injury screen	61	39.6	115	14.4	33	19.8	69	8.1
Probable depression	139	77.7	175	19.9	173	90.6	225	25.0
Probable PTSD	101	56.7	116	13.2	103	53.9	106	11.7
Probable anxiety	102	57.3	121	13.7	106	56.1	138	15.2
Problematic alcohol use	94	53.1	167	19.0	67	35.3	123	13.6
Moderate or high somatic symptoms	130	73.1	254	29.0	133	69.9	306	33.9
VA health care user	103	66.9	283	42.6	109	61.6	342	44.7
Difficulty with postdeployment Adjustment	175	97.8	723	82.2	186	97.9	757	83.7

Note. PTSD = posttraumatic stress disorder; VA = Veteran’s Affairs.

Measures

Demographics and premilitary variables

Demographics were self-reported by participants and included age and education level grouped according to (a) high school/general educational development test, (b) vocational/technical training, (c) some college, (d) 4-year college graduate, (e) some graduate or professional school, and (f) graduate/professional degree. Race was dichotomized (Caucasian and other races), relationship status was dichotomized (in a relationship: yes/no), and parental status was similarly dichotomized. A modified scale from the Deployment Risk and Resiliency Inventory (DRRI) was used to assess eight potentially traumatic experiences prior to the military (e.g., witnessed someone being assaulted or killed; Vogt, Proctor, King, King, & Vasterling, 2008).

Military and deployment variables

Military characteristics included number of deployments, total time deployed, deployment to Iraq or Afghanistan versus elsewhere (dichotomous), branch (1 = Marines, 2 = Army, 3 = Navy, 4 = Air Force, 5 = Coast Guard, 6 = more than one branch), component (active duty vs. Reserves/Guard) and rank (enlisted vs. officer).

Deployment experiences during any deployment were assessed via the DRRI and dichotomized according to any endorsement on each scale (Vogt et al., 2008). The Combat Experiences Scale includes 15 items that assess exposure to combat experiences (e.g., directing fire at the enemy). The Aftermath of Battle Scale includes 15 items that assess exposure to postbattle experiences (e.g., taking care of wounded). The Sexual Harassment Scale is a 7-item scale that measures exposure to sexual harassment and assault during deployment. The General Harassment Scale includes seven items that assess harassment during deployment based on gender, ethnicity, or other social characteristics. The Unit Support Scale includes 12 items that assess perception of social support from unit members and leadership during deployment. The Life and Family Concerns Scale is a 14-item scale that assesses concerns about life and family during deployment. Cronbach’s alpha for these scales ranged from .84 to .94.

The Traumatic Brain Injury Quick Screen for Veteran Populations assesses probable deployment-related TBI (Arlinghaus & Hickey, 2005). Consistent with the American Congress of Rehabilitation Medicine and VA/Department of Defense (DoD) clinical practice guidelines (Department of Veterans Affairs/DoD, 2009), participants were coded as having probable TBI if they reported at least one deployment-related event in which they experienced an alteration or loss of consciousness as a result of a blast or non blast-related head injury.

Postmilitary psychological symptomatology and other predictors

The Center for Epidemiologic Studies Depression Scale (CES-D; Radloff, 1977) was used to measure past week depression symptoms with a Likert response scale (1 = none of the time or less than one day, 4 = 5–7 days). A score of 10 was used to categorize participants into probable depression/no depression groups (Andresen, Malmgren, Carter, & Patrick, 1994). Cronbach’s alpha for the CES-D was .90.

The Posttraumatic Stress Disorder Checklist, Military Version (PCL-M; Weathers, Litz, Herman, Huska, & Keane, 1993) was used to assess the 17 DSM-IV symptoms of PTSD (American Psychiatric Association, 2000). Participants rated how bothered they were by each symptom within the past month using a Likert scale (1 = not at all, 5 = extremely bothered). A score of 50 was used to categorize the sample into probable PTSD or no PTSD groups (Weathers et al., 1993). Cronbach’s alpha for the PCL was .97.

Anxiety symptoms were measured using the anxiety sub-scale of the Depression Anxiety Stress Scales (DASS; P. F. Lovibond & S. H. Lovibond, 1995; S. H. Lovibond & P. F. Lovibond, 1995). Participants indicated how much 14 statements about situational anxiety, autonomic arousal, skeletal musculature effects, and subjective experience of anxious affect applied to them using response options from 1 (did not apply to me at all) to 4 (applied to me very much, or most of the time). A score of 7 was used to categorize participants into normal anxiety/symptomatic anxiety groups (S. H. Lovibond & P. F. Lovibond, 1995). Cronbach’s alpha for the DASS was .93.

The 4-item CAGE Questionnaire (Ewing, 1984) was used to assess problematic alcohol use. It asks participants to report on their drinking habits (e.g., “Have you felt you ought to cut down on your drinking?”); a score of 2 was used to categorize participants within groups of problematic alcohol use/no problematic alcohol use (Bush, Shaw, Cleary, Delbanco, & Aronson, 1987). Cronbach’s alpha for the CAGE was .74.

Somatic symptoms were assessed via the Patient Health Questionnaire (PHQ-15; Kroenke, Spitzer, & Williams, 2002) a 15-item measuring assessing severity of somatic symptoms. Participants were asked to rate the severity of each symptom over the past month as either 0 (not bothered at all), 1 (bothered a little), or 2 (bothered a lot) and for the current study total responses were categorized into minimal (0–4 symptoms), low (5–9 symptoms), medium (10–14 symptoms), and high (15–30 symptoms; Kroenke et al., 2002). Cronbach’s alpha for the PHQ-15 was .88.

Health care use of overall and VA care specifically were assessed via two questions asking whether participants had seen a professional for help with an emotional problem since returning from deployment and whether participants had used a VA facility for any of that care.

Postdeployment adjustment was assessed via questions that asked participants to rate how much they agreed with five statements (e.g., “The transition from my deployment back to my regular life was extremely difficult”) on a Likert scale ranging from 1 = strongly disagree to 5 = strongly agree. Endorsement of one or more items on this scale was classified as “postdeployment adjustment issues”; otherwise participants were classified as not having postdeployment adjustment issues. Cronbach’s alpha for the postdeployment adjustment scale was .89.

Postmilitary SI

The 4-item Suicidal Behaviors Questionnaire-Short Form (SBQ-SF; Brown, 2001) was used to assess suicidal behaviors and thoughts occurring since the most recent deployment. The question “Since your most recent deployment to OEF/OIF [Operation Enduring Freedom/Operation Iraqi Freedom], have you thought about or attempted to kill yourself?” was used to categorize SI for the current analyses. Responses to this question were dichotomized into no SI (i.e., response of “no”) and yes SI (responses of “It was just a passing thought”; “I briefly considered it, but not seriously”; “I thought about it and was somewhat serious”).

Data Analysis

Classification tree analysis (Hastie et al., 2001), chosen for its ability to construct complex prediction models without a priori specification, was used to build profiles of participants who did and did not experience postdeployment SI, stratified by gender. Single classification trees were plotted as an initial evaluation of the structure of the data. Results were further evaluated using random forests, which are predictive models for the outcome constructed from repeated classification tree analysis using a bootstrapping procedure to evaluate robustness of results (Hastie et al., 2001). Each forest was built with 1,000 trees, 5 variables sampled as split candidates at each node, and a minimum of 20 observations needed to attempt a split. Variable importance (using the area under the curve [AUC] method with out-of-bag trees only; Janitza, Strobl, & Boulesteix, 2013) was plotted using dot plots. To produce the dot plots, the AUC of each tree is calculated under the original data, then recalculated after scores on the test variable are randomly permuted, which destroys its association with the outcome; the difference in AUC reflects the degree to which the test variable contributes to the tree’s predictive accuracy, with a large difference indicating an important variable. The x-axis of the dot plots shows the average difference in AUC (i.e., the drop in accuracy associated with shuffling that variable) over the trees in the RF. Box plots were also derived to examine the similarity in SI profiles between the single classification tree and random forest results. Finally, AUCs were calculated (using out-of-bag trees only) to determine the model fit. The analytic sample was restricted to participants who responded to the suicidal ideation survey item, thus there were no missing data for the outcome variable. The tree-fitting algorithm does not delete cases with missing data in a predictor variable; these cases are ignored when evaluating the predictor for the best binary split. If a predictor with missing data is selected for implementing a split, cases with missing values are sorted into daughter nodes using values from a surrogate predictor: an alternate variable that most closely duplicates the same division of cases. Trees were constructed using the ctree algorithm (Hothorn, Hornik, & Zeileis, 2006) in the R statistical environment (R Core Team, 2016) using p values with α = .01.

Results

SI Among Male Veterans

Figure 1 displays the variables associated with SI among men in the classification tree analysis. Men with probable depression, anxiety, and somatic symptoms, and who were deployed to Iraq or Afghanistan specifically had a high probability of SI (node 19; 55.4%), indicating a meaningful interaction between these variables. Other notable pathways included men with probable depression only (node 14; 33.3%), and men with 20% of a greater probability of SI among men in this sample (with adequate sample sizes in each group).

Figure 1.

Classification tree for the prediction of suicidal ideation among men. Each oval node represents a predictor variable for which the model found a meaningful association with suicidal ideation risk. Oval nodes connected by a path denote an interaction between those predictors. Each rectangular bin at the bottom (terminal node) represents the group of people with the characteristic profile in the branches above. Within the rectangular bins: y = (the proportion of people in that bin who did not have suicidal ideation, the proportion of people in that bin who had suicidal ideation). PTSD = posttraumatic stress disorder.

In the variable importance plot derived from the random forest multiple variables fall to the right of the dashed line in the plot, indicating that there was evidence of associations of these variables with SI upon repeated constructions of the tree in validation subsamples (Figure 2). The box plots comparing the SI probability among members of each terminal node in the classification tree analysis with the results derived from the random forest are displayed in Figure 3. The results for nodes 5 and 18 show that the initial classification tree overestimated the probability of SI among these groups. The remaining box plots show that the random forest probabilities were mostly consistent with the probabilities of SI found in the classification tree, with the potential exception of node 16 in which it appears that the original tree may have underestimated the probability. The overall random has an AUC of .91.

Figure 2.

Variable importance for predicting suicial ideation among men. The dashed vertical line represents the absolute value of the lowest rank variable importance. Variables with a dot to the right of the line contributed to predictive accuracy in the random forest, with higher numbers on the x-axis indicating stronger associations with suicidal ideation. The x-axis shows the average difference in the area under the curve (AUC; i.e., the drop in accuracy associated with shuffling that variable) comparing the AUC of each tree under the original data, with the AUC after scores on the test variable are randomly permuted. PTSD = posttraumatic stress disorder; TBI = traumatic brain injury; VA = Veteran’s Affairs.

Figure 3.

Box plots comparing the single classification tree results with the bootstrapped random forest results among men. The number above box plots indicates corresponding terminal node in the single classification tree (see Figure 1). The dashed line indicates probability of suicidal ideation from the single tree for individuals in that node. The bold line in the center of each box represents the median probability of SI from the random forest for the same individuals. RF = random forest.

SI Among Female Veterans

The classification tree for SI among women in the sample is shown in Figure 4. The highest probability of SI was among women who had probable depression, had been sexually harassed during deployment, and had probable PTSD (node 6; 61.2%). Moderate probabilities of SI were also found for women who had probable depression (node 3; 30.8%), and probable depression and were sexually harassed during deployment (node 5; 42.1%;). Lower probabilities of SI were found for other pathways among women who were not depressed.

Figure 4.

Classification tree for the prediction of suicidal ideation among women. Each oval node represents a predictor variable for which the model found a meaningful association with suicidal ideation risk. Oval nodes connected by a path denote an interaction between those predictors. Each rectangular bin at the bottom (terminal node) represents the group of people with the characteristic profile in the branches above. Within the rectangular bins: y = (the proportion of people in that bin who did not have suicidal ideation, the proportion of people in that bin who had suicidal ideation). PTSD = posttraumatic stress disorder.

The results of the variable importance plot show that other variables may be important to prediction as well (e.g., probable anxiety, general harassment during deployment), although the results indicate that depression and PTSD have the strongest association with SI (Figure 5). Figure 6 displays box plots that compare the probability of SI within each node derived from the classification tree with the probability of SI derived from the random forest. The random forest revealed SI probabilities for each node that were generally consistent with the classification tree results. The overall AUC for the random forest model was .92 among women.

Figure 5.

Variable importance for predicting suicidal ideation among women. The dashed vertical line represents the absolute value of the lowest rank variable importance. Variables with a dot to the right of the line contributed to predictive accuracy in the random forest, with higher numbers on the x-axis indicating stronger associations with suicidal ideation. The x-axis shows the average difference in the area under the curve (AUC; i.e., the drop in accuracy associated with shuffling that variable) comparing the AUC of each tree under the original data, with the AUC after scores on the test variable are randomly permuted. PTSD =posttraumatic stress disorder; TBI =traumatic brain injury; VA = Veteran’s Affairs.

Figure 6.

Box plots comparing the classification tree results with the bootstrapped random forest results among women. Number above box plots indicates corresponding terminal node in the single classification tree (Figure 4). The dashed line indicates the probability of suicidal ideation (SI) from the single tree for individuals in that node. The bold line in the center of each box represents the median probablity of SI from the random forest for the same individuals. RF = random forest.

Discussion

This study is the first to use classification tree and random forest analyses to characterize variables associated with SI in a national sample of Iraq and Afghanistan war veterans. This data-driven method allows for the simultaneous examination of predictors from a variety of domains and for the discovery of unanticipated associations with SI. The literature to date has largely documented hypothesis-driven, null-hypothesis-tested associations between gender, trauma, psychiatric symptomatology, and SI among veterans, which highlighted sexual harassment, PTSD, depression, and alcohol abuse as key predictors (Bryan et al., 2015; Gradus, Street, et al., 2013; Gradus et al., 2015; Jakupcak et al., 2009; Lemaire & Graham, 2010; Monteith et al., 2015; Pietrzak et al., 2010). This study extends that work by demonstrating that a supervised machine learning approach allows for the discovery of complex variable combinations associated with SI with key differences found for male and female veterans.

Among men, the classification tree analysis, which serves as an initial visualization of the data, and the random forest analysis confirm that there are multiple predictor profiles associated with SI; we found five risk-factor combinations associated with a 20.0% or greater probability. For example, among male veterans, probable depression, probable anxiety, more than mild somatic symptoms, and deployment to Iraq and Afghanistan specifically present a key combination associated with increased SI. With regard to trauma specifically, among males without probable depression or PTSD, premilitary trauma demonstrated an important association with SI, but only in combination with probable alcohol use. Other forms of trauma were not represented in the classification tree analysis; however, probable TBI and general harassment were identified as having potentially important overrepeated iterations in the random forest analyses. These variables will be important to consider in future replications of this work. Importantly, two groups that were found to have the high probabilities of SI (node 5, probable PTSD and more than a 4-year college degree; node 18, probable depression, probable anxiety, somatic symptoms, and a deployment outside Iraq or Afghanistan) were not corroborated by the random forest. This suggests that the risk profile of members of these nodes is spurious (likely due to the sample sizes of these subgroups) and should be interpreted with caution. The other inconsistency between the classification tree and random forest findings was for node 16 (probable depression and anxiety, but little or no somatic symptoms). In this case, the box plot revealed that the initial classification tree had underestimated the probably of SI in this subgroup; it will be important to replicate these results. The current study demonstrates that many nuanced associations and new avenues for identifying veterans in most need of intervention may be missed by relying on traditional regression methods that require investigator specification of potentially important predictors.

A relatively homogenous picture emerged among women, with the classification tree displaying three variables that were most strongly associated with SI in this sample: probable depression, probable PTSD, and sexual harassment during deployment. The comparison of SI probabilities from the terminal nodes in the classification tree with those from the random forest confirms these findings. This is consistent with previous research which has found that PTSD and depression are associated with SI among veterans (Gradus, Street, et al., 2013; Jakupcak et al., 2009; Lemaire & Graham, 2010; Pietrzak et al., 2010). This is further consistent with the growing body of literature demonstrating sexual harassment is associated with SI in this population, including our own work using traditional regression-based methods in this sample (Belik, Stein, Asmundson, & Sareen, 2009; Bryan et al., 2015; Gradus, Shipherd, Suvak, Giasson, & Miller, 2013; Gradus, Street, et al., 2013; Monteith et al., 2015). Compared with our results for male veterans, in which a machine learning approach greatly enriched previous findings, our results for women demonstrate that probable PTSD, probable depression, and sexual harassment during deployment show consistent and robust associations with SI. The differences in results across men and women when comparing traditional regression and machine learning methods highlight the ability of data-driven statistical methods to glean rich results by obviating the need for a priori investigator specification of predictors. The variable importance plot further suggests that other variables emerged as important upon multiple repetitions of the model building algorithm (e.g., some forms of trauma). Continued inclusion of these variables in future studies that replicate our findings is important to determine if they display a more prominent association with SI in other samples of the female veterans who were deployed as part of these conflicts.

Ultimately, this line of work could have significant implications for identification, and subsequent prevention and treatment through our ability to target subgroups of a population for whom the probability of suicidal ideation is particularly high. Evidence from machine learning studies in this area builds a clearer picture of who is at particular risk. Awareness of factors that increase risk for suicidal ideation is of particular importance among a population like military veterans who may be reluctant to disclose suicidal ideation (Vannoy et al., 2016). This specific study contributes to that literature by demonstrating that clinicians treating this population of veterans should be aware that among male and female patients there may be different characteristics which are associated with SI. Of particular note, the type of potentially traumatic events experienced both predeployment and during deployment may play a critical role.

The current work presents with key limitations. Data were cross-sectional; thus we cannot generalize to the prediction of future events. Importantly, the study queried participants about SI at any time since their most recent deployment, but only recent mental health; therefore, it is possible in theory that participants reporting SI are reporting about past ideation that is no longer present despite continued mental health symptomatology. To address this possible issue the current analyses are restricted to participants who reported both SI and current mental health symptomatology; however, this should be kept in mind when interpreting our results. In addition, data were obtained via self-report and are subject to the biases found in that form of assessment. Our response rate (accounting for an estimated proportion of ineligibility among nonresponders) was just under 50.0%. Although this response rate is consistent with other large survey studies of OEF/OIF veterans (Smith, Smith, Gray, & Ryan, 2007; Tanielian & Jaycox, 2008) and we previously reported few demographic differences between our responders and nonresponders (Street et al., 2013), it is important to keep in mind that our findings may be impacted by nonresponse bias. Finally, although a data-driven approach gives us the ability to examine important associations without the need for the a priori specification of predictors, a limitation to this method, and machine learning methods in general, is that the computer will find a model that fits the data every time. In our analyses, we employed the standard safeguards against finding a model whose algorithms search for optimal solutions that fit the data at hand, but lack generalizability. In the current study, we utilized a form of validation whereby results are replicated across multiple bootstrap samples to determine consistency across replication samples, providing a distribution of AUC estimates along with information about features that commonly replicate across models. These represent commonly utilized methods to guard against overfitting and overinterpretation of results (Hastie et al., 2001).

Despite these limitations, the current study is the first to identify distinct gender-specific combinations of risk factors including trauma, psychopathology, and military experiences that have varying associations with SI, revealing important differences. Such a consideration is important to the mental health of veterans, as the number of women in the military is projected to increase in the coming decades (Department of Veterans Affairs, Office of Public Affairs Media Relations, 2010). Future research should continue to use machine learning and other novel methods to elucidate gender-specific risk factors for SI and other suicidal behaviors in military, veteran, and civilian samples and expand upon the present study through the use of longitudinal data.