Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Oct 1.
Published in final edited form as: Suicide Life Threat Behav. 2016 Nov 1;47(5):589–602. doi: 10.1111/sltb.12307

Neurocognitive Function and Suicidal Behavior in U.S. Army Soldiers

James A Naifeh 1, Matthew K Nock 1, Robert J Ursano 1, Patti L Vegella 1, Pablo A Aliaga 1, Carol S Fullerton 1, Christina L Wryter 1, Steven G Heeringa 1, Murray B Stein 1, Ronald C Kessler, On behalf of the Army STARRS Collaborators1
PMCID: PMC5411316  NIHMSID: NIHMS849421  PMID: 27801502

Abstract

This prospective cohort study used administrative data from the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS) to examine associations between neurocognitive functioning and subsequent suicidal events among Regular Army enlisted soldiers during the years 2004–2009. Cases were all soldiers who completed the Army’s Automated Neuropsychological Assessment Metrics (ANAM) computerized testing battery prior to documented suicide attempt (n=607), ideation (n=955), or death (n=57). Controls were an equal-probability sample of 9,893 person-months from other soldiers. Exploratory factor analysis of five ANAM tests identified a general neurocognitive factor that excluded the mathematic processing test (MTH). When examined separately in logistic regression analyses that controlled for sociodemographics and mental health diagnosis, both the general neurocognitive factor (β= −0.197 to −0.521; p<0.01) and MTH (β= −0.024 to −0.064; p<0.05) were associated with all outcomes. When both predictors were examined simultaneously, the general neurocognitive factor continued to be associated with all outcomes (β= −0.164 to −0.417; p<0.05) and MTH continued to be associated with suicide attempt (β= −0.015; p=0.046) and ideation (β= −0.014; p=0.018). These clear and robust associations suggest that future research must continue to examine the extent to which objective neurocognitive tests may enhance understanding and prediction of suicide risk.


Impairment in various aspects of neurocognitive functioning has been found in suicide attempters relative to other psychiatric patients and healthy controls (Jollant, Lawrence, Olie, Guillaume, & Courtet, 2011; Richard-Devantoy, Berlim, & Jollant, 2014a, 2014b). In addition to their value in the search for endophenotypes of suicidal behavior (Courtet, Gottesman, Jollant, & Gould, 2011; Mann et al., 2009), neurocognitive measures have potential to enhance risk detection. The objective nature of neurocognitive tests offers advantages over current risk assessment methods based on clinician observation and self-report (Nock et al., 2013). Suicide prediction, and issues around self-report, have become an especially important priority in the U.S. Army, which experienced a sharp increase in suicidal behavior during the wars in Afghanistan and Iraq (Schoenbaum et al., 2014; Ursano, Kessler, Heeringa, et al., 2015). Indeed, many soldiers are reluctant to report problems due to concerns about stigma and negative career impact (Warner et al., 2011), creating a significant obstacle to prevention programs that rely on self-disclosure.

Despite widespread implementation of screening (Appenzeller, Warner, & Grieger, 2007; Warner et al., 2007a, 2007b) and prevention programs (Ramchand, Acosta, Burns, Jaycox, & Pernin, 2011), identifying soldiers at risk of suicide ideation, attempt, or death, remains a significant challenge. These efforts could be improved by leveraging the vast array of objective, administrative data the Army collects on its soldiers. Whereas prior studies using administrative data have produced valuable information related to the sociodemographic, service-related, and mental health correlates of suicidal behavior (Bachynski et al., 2012; Bell, Harford, Amoroso, Hollander, & Kay, 2010; Black, Gallaway, Bell, & Ritchie, 2011; Gilman et al., 2014; Kessler et al., 2015; Logan, Skopp, Karch, Reger, & Gahm, 2012; Schoenbaum et al., 2014; Street et al., 2015; Ursano, Kessler, Heeringa, et al., 2015; Ursano, Kessler, Stein, et al., 2015), many administrative variables with potential to enhance risk detection have yet to be examined. Among these are indicators of neurocognitive functioning from tests administered to soldiers prior to deployment to provide baseline data in the event of deployment-related traumatic brain injury.

Here we examine associations of neurocognitive functioning with subsequent suicide-related outcomes among Regular Army enlisted soldiers using administrative data from the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS; www.armystarrs.org) (Ursano et al., 2014). Whereas previous studies used small clinical or convenience samples, we present a population-level prospective study of neurocognitive functioning and suicide risk. We examined the association of neurocognitive functioning and subsequent risk of suicide attempt, controlling for sociodemographic and mental health covariates. Analyses were then repeated to examine associations with suicide ideation and death.

METHOD

Sample

The Army STARRS Historical Administrative Data Study integrates 38 Army and Department of Defense (DoD) administrative data systems, including those in which suicidal events (ideation, attempts and death) are medically documented. It includes individual-level person-month records for all soldiers on active duty between January 1, 2004 and December 31, 2009 (n=1.66 million) (Kessler et al., 2013). The current longitudinal cohort study focused on records for the 975,057 Regular Army soldiers on active duty during this time (excluding activated Army National Guard and Army Reserve). Data were analyzed using a discrete-time survival framework with person-month as the unit of analysis (Willett & Singer, 1993), such that each month in the career of a soldier was treated as a separate observational record. Cases were limited to enlisted soldiers who completed neurocognitive testing prior to a suicidal event, resulting in 607 suicide attempters, 955 suicide ideators, and 57 suicide decedents. Cases with documentation of multiple suicidal events were classified based on the first occurrence of the most severe type of event (i.e., prioritizing suicide death over attempt over ideation) (eFigure 1). Using an equal-probability 1:200 sample of control person-months stratified by gender, rank, time in service, deployment status (never, currently, previously), and historical time, we identified those who completed neurocognitive testing prior to their sampled person-month record (n=9,893). Control person-months excluded officers (including warrant officers), soldiers with a documented suicidal event (attempt, ideation, death), and person-months during which a soldier died (e.g., due to combat, homicide, accident, illness). Each control person-month was assigned a weight of 200 to adjust for under-sampling. Soldiers with multiple neurocognitive assessment records were excluded, as we could not determine why the tests were re-administered (e.g., technical problems; Cernich, Brennana, Barker, & Bleiberg, 2007) or how those factors may have affected test validity.

Measures

Suicidal events

Suicide attempters were identified using: records from the Department of Defense Suicide Event Report (DoDSER) (Gahm et al., 2012), a DoD-wide surveillance mechanism that aggregates information on suicidal behaviors via a standardized form completed by medical providers at DoD treatment facilities; and ICD-9-CM E95x diagnostic codes (E950–E958; indicating self-inflicted poisoning or injury with suicidal intent) from the Military Health System Data Repository (MDR), Theater Medical Data Store (TMDS), and TRANSCOM (Transportation Command) Regulating and Command & Control Evacuation System (TRAC2ES), which together provide healthcare encounter information from military and civilian treatment facilities, combat operations, and aeromedical evacuations. The E959 code (late effects of a self-inflicted injury) was excluded, as it confounds the temporal relationships between the predictor variables and the suicide attempt (Walkup, Townsend, Crystal, & Olfson, 2012). Suicide ideators were identified using DoDSER records and MDR, TMDS, and TRAC2ES records containing the ICD-9-CM V62.84 code indicating suicidal ideation, coding options which were not in use prior to 2006. Suicide decedents were identified using records from the Armed Forces Medical Examiner Tracking System (AFMETS) (eTable 1).

Neurocognitive functioning

Neurocognitive functioning was assessed by the Army’s Automated Neuropsychological Assessment Metrics (version 4) Traumatic Brain Injury Battery (ANAM4™ TBI), a computerized battery of tests assessing neurocognitive functioning (e.g., response speed, attention/concentration, immediate and delayed memory, spatial processing, decision processing speed and efficiency) (C-SHOP, 2007). The system operates in Microsoft Windows™ on IBM-compatible notebook and desktop computers (C-SHOP, 2007). ANAM test results are stored within an administrative data system included in the Army STARRS HADS (eTable 1). We selected five tests from the Army’s ANAM4 TBI-MIL battery: Code Substitution–Learning (CDS), assessing associative learning; Procedural Reaction Time (PRO), assessing processing speed; Mathematical Processing (MTH), assessing working memory; Matching to Sample (M2S), assessing visual spatial memory; and Code Substitution–Delayed (CDD), assessing delayed memory. We excluded Simple Reaction Time, a test of visuomotor processing speed and attention, as it has lower cognitive processing demands than other tests in the battery and is rarely associated with traditional neurocognitive measures and constructs (C-SHOP, 2007). We also excluded the Go/No-Go test, a recent addition to the battery assessing response inhibition, because it was administered to very few soldiers in our 2004–2009 sample. We measured test performance using throughput, a continuous score based on the number of correct responses per unit of available response time, combining measures of both speed and accuracy (C-SHOP, 2007). Throughput is a general performance index measuring cognitive efficiency (Thorne, 2006), and is believed to best reflect the processes underlying ANAM tests (Short, Cernich, Wilken, & Kane, 2007).

Sociodemographic and mental health factors

Given that both neurocognitive test performance and suicidal behavior correlate with sociodemographic and mental health factors, we included such factors as a priori covariates in our analyses. Sociodemographic variables (gender, age at neurocognitive testing, education, race/ethnicity) were drawn from the DoD Defense Manpower Data Center (DMDC) Master Personnel and Transaction Files (eTable 1). Using MDR, TMDS, and TRAC2ES records, we created an indicator variable for previous mental health diagnosis from ICD-9-CM mental disorder codes (e.g., major depression, posttraumatic stress disorder, personality disorders), excluding postconcussion syndrome, tobacco use disorder, and supplemental V-codes (eTable 2).

Analysis procedures

All analytic procedures were conducted using SAS version 9.3 (SAS Institute Inc., 2011). Analyses focused first on the prediction of suicide attempts. We then tested the extent to which findings replicated in the prediction of suicide ideation and suicide death. This allowed us to test whether neurocognitive functioning is predictive of suicide attempts specifically, or of suicidal thoughts and behaviors more generally.

We removed outliers from the suicide attempt case-control sample based on the same criteria used in establishing ANAM normative/reference groups for the Army population (C-SHOP, 2007). Specifically, scores on each test were excluded if they: (1) exceeded six standard deviations from the mean reaction time, or (2) were in the top 1% of speed and simultaneously in the bottom 1% of accuracy (i.e., percent correct). We also removed scores that were likely invalid due to a low percentage of correct responses (≤ 56%) (C-SHOP, 2007).

Given the ANAM’s design as a measure of general cognitive functioning (C-SHOP, 2007), as well as factor analytic studies indicating that tests in the ANAM library can be represented by a small number of common domains or functions (Bleiberg, Kane, Reeves, Garmoe, & Halpern, 2000; Kabat, Kane, Jefferson, & DePino, 2001), an exploratory factor analysis with maximum likelihood extraction and promax rotation was conducted using the five ANAM throughput scores (CDS, PRO, MTH, M2S, CDD). We examined associations of the resulting factor score with subsequent suicide attempt using a series of logistic regression analyses. Based on evidence that suicidal events among soldiers increased during the study period (Schoenbaum et al., 2014; Ursano, Kessler, Heeringa, et al., 2015), all regression equations included dummy predictors for calendar year and month. Coefficients of other predictors can consequently be interpreted as averaged within-month associations. Prospective associations of the neurocognitive predictors with each suicide-related outcome were first examined in separate univariate models (controlling only for historical time). Those analyses were repeated in multivariate models that controlled for socio-demographics (gender, age at neurocogintive testing, education, race/ethnicity), and history of mental health diagnosis prior to testing. We then examined neurocognitive predictors and covariates simultaneously in multivariate models predicting each outcome.

RESULTS

Neurocognitive functioning and suicide attempt

Among those who completed neurocognitive testing, sociodemographic correlates of suicide attempt included being: female, less educated, White Non-Hispanic, younger at the time of neurocognitive testing, and receiving a mental health diagnosis prior to testing (Table 1). Approximately 80% of suicide attempters and 83% of controls completed neurocognitive testing in the prior 12 months, with no difference between groups in time since test administration, t(661.1)=0.10, p=0.92. A total of 329 soldiers (25 attempters, 304 controls) were excluded due to invalid scores or as outliers on one or more tests.

Table 1.

Characteristics of Regular Army enlisted suicide attempters, ideators, and decedents who completed neurocognitive testing, versus controls.1

Cases

Controls
(N = 9,893)
Suicide Attempters
(N = 607)
Suicide Ideators
(N = 955)
Suicide Decedents
(N = 57)
N % N % χ2 N % χ2 N % χ2





Gender 30.11* 4.51* 3.55
  Female 874 8.8 92 15.2 103 10.8 1 1.8
  Male 9,019 91.2 515 84.8 852 89.2 56 98.2
Age at testing 129.40* 72.08* 11.19*
  17–20 1,499 15.2 171 28.2 215 22.5 13 22.8
  21–24 3,404 34.4 250 41.2 358 37.5 17 29.8
  25–29 2,481 25.1 114 18.8 232 24.3 20 35.1
  30–34 1,261 12.7 39 6.4 75 7.9 1 1.8
  35–39 850 8.6 23 3.8 51 5.3 3 5.3
  40+ 398 4.0 10 1.6 24 2.5 3 5.3
Education 101.07* 148.44* 1.15
  < High School2 1,696 17.1 195 32.1 301 31.5 10 17.5
  High School 7,445 75.3 387 63.8 619 64.8 43 75.4
  Some College 395 4.0 17 2.8 20 2.1 1 1.8
  College+ 357 3.6 8 1.3 15 1.6 3 5.3
Race 13.12* 17.69* 4.78
  White 6,519 65.9 440 72.5 690 72.3 44 77.2
  Black 1,641 16.6 80 13.2 126 13.2 8 14.0
  Hispanic 1,190 12.0 64 10.5 96 9.9 2 3.5
  Asian 377 3.8 14 2.3 32 3.4 2 3.5
  Other 166 1.7 9 1.5 11 1.2 1 1.8
Mental health diagnosis
prior to testing3
152.21* 291.04* 8.51*
  Yes 2,647 26.8 297 48.9 489 51.2 25 43.9
  No 7,246 73.2 310 51.1 466 48.8 32 56.1
1

Case-control sample includes enlisted Regular Army enlisted soldiers (i.e., excluding officers and members of the U.S. Army National Guard and Army Reserve) on active duty during the years 2004–2009. Cases are the subset of soldiers who completed neurocognitive testing prior to their first administratively documented suicide attempt. Controls are soldiers who completed the neurocognitive testing prior to their sampled person-month record, representing a subset of a 1:200 stratified probability sample of all active duty Regular Army person-months in the population, exclusive of soldiers with a non-fatal suicidal behavior and all person-months involving a death (i.e., due to suicide, combat, homicide, injury, or illness). All records in the 1:200 control sample were assigned a weight of 200 to adjust for the under-sampling of months not associated with a suicidal behavior.

2

< High School includes: General Educational Development credential (GED), home study diploma, occupational program certificate, correspondence school diploma, high school certificate of attendance, adult education diploma, and other non-traditional high school credentials.

3

Mental health diagnosis prior to testing was determined based on ICD-9 mental disorder codes (Appendix B).

4

Based on throughput scores from the Army’s Automated Neuropsychological Assessment Metrics (ANAM) battery.

*

p < 0.05, two-tailed.

Means and standard deviations of ANAM throughput scores are included in Table 2. All bivariate correlations between throughput scores were significant (r=0.18–0.66), with MTH having the only correlations below 0.30. Using the entire case-control sample, exploratory factor analysis with maximum likelihood extraction indicated a single, unrotated factor solution based on an eigenvalue >1 (3.36) and the scree plot. All tests had adequate item loadings (>0.40) except for MTH (.35). We excluded MTH and repeated the factor analysis with the remaining four tests, which again supported a single-factor solution and generated similar item loadings (Table 3). In subsequent regression analyses we used the factor score from this “general neurocognitive factor” (based on CDS, PRO, M2S, and CDD), as well as the MTH throughput score, as our neurocognitive predictors.

Table 2.

Means and standard deviations of Automated Neuropsychological Assessment Metrics (ANAM) throughput scores among Regular Army enlisted suicide attempters, ideators, and decedents versus controls.1,2

Cases

Controls
(N = 9,893)
Suicide
Attempters
(N = 607)
Suicide Ideators
(N = 955)
Suicide Decedents
(N = 57)
ANAM Tests M (SD) M (SD) M (SD) M (SD)





Code Substitution–Learning (CDS) 53.3 (11.1) 52.1 (11.7) 51.5 (11.8) 50.6 (11.8)
Procedural Reaction Time (PRO) 100.6 (14.3) 98.1 (15.5) 97.9 (15.8) 97.0 (17.1)
Mathematical Processing (MTH) 20.5 (6.1) 19.2 (5.9) 19.2 (6.3) 18.3 (6.0)
Matching to Sample (M2S) 35.3 (10.8) 33.7 (10.6) 34.1 (10.8) 33.1 (9.8)
Code Substitution–Delayed (CDD) 47.2 (15.5) 46.5 (15.4) 46.3 (15.3) 42.5 (15.0)
1

Case-control sample includes enlisted Regular Army enlisted soldiers (i.e., excluding officers and members of the U.S. Army National Guard and Army Reserve) on active duty during the years 2004–2009. Cases are the subset of soldiers who completed neurocognitive testing prior to their first administratively documented suicide attempt. Controls are soldiers who completed the neurocognitive testing prior to their sampled person-month record, representing a subset of a 1:200 stratified probability sample of all active duty Regular Army person-months in the population, exclusive of soldiers with a non-fatal suicidal behavior and all person-months involving a death (i.e., due to suicide, combat, homicide, injury, or illness). All records in the 1:200 control sample were assigned a weight of 200 to adjust for the under-sampling of months not associated with a suicidal behavior.

Table 3.

Bivariate correlations and factor loadings of Automated Neuropsychological Assessment Metrics (ANAM) throughput scores among Regular Army enlisted suicide attempters and controls.1

ANAM Throughput Score Correlations2 Single-Factor Loadings3



ANAM Test CDS PRO MTH M2S All Tests
Included
MTH
Excluded



Code Substitution–Learning (CDS) 0.85 0.86
Procedural Reaction Time (PRO) 0.50* 0.61 0.59
Mathematical Processing (MTH) 0.29* 0.33* 0.35
Matching to Sample (M2S) 0.46* 0.41* 0.24* 0.57 0.56
Code Substitution–Delayed (CDD) 0.66* 0.39* 0.18* 0.41* 0.73 0.74
1

Case-control sample includes enlisted Regular Army enlisted soldiers (i.e., excluding officers and members of the U.S. Army National Guard and Army Reserve) on active duty during the years 2004–2009. Cases are the subset of soldiers who completed neurocognitive testing prior to their first administratively documented suicide attempt. Controls are soldiers who completed the neurocognitive testing prior to their sampled person-month record, representing a subset of a 1:200 stratified probability sample of all active duty Regular Army person-months in the population, exclusive of soldiers with a non-fatal suicidal behavior and all person-months involving a death (i.e., due to suicide, combat, homicide, injury, or illness). All records in the 1:200 control sample were assigned a weight of 200 to adjust for the under-sampling of months not associated with a suicidal behavior.

2

Correlations within the suicide ideation and suicide decedent case-control samples were nearly identical to those presented above.

3

Based on exploratory factor analysis with maximum likelihood estimation.

*

p < .05, two-tailed.

In logistic regression models that controlled only for historical time, lower scores on both the general neurocognitive factor (β= −0.128; p=0.005) and MTH (β= −0.037; p<0.0001) predicted subsequent suicide attempt (Table 4). When sociodemographics and mental health diagnosis prior to testing were added as covariates, both the general neurocognitive factor (β= −0.197; p<0.0001) and MTH (β= −0.024; p=0.001) continued to show a significant association with suicide attempt. The predictors remained significant when they were entered simultaneously in the same multivariate model, although their effects were decreased : general neurocognitive factor, (β= −0.164; p=0.001); MTH (β= −0.015; p=0.046) (Table 5).

Table 4.

Univariate associations of neurocognitive functioning with subsequent suicide attempt, ideation, and death among Regular Army enlisted soldiers.1,2

Neurocognitive Predictors3 Suicide Attempt
(N = 607)
Suicide Ideation
(N = 955)
Suicide Death
(N = 57)
β p β p β p




ANAM Factor Score −0.128 0.005 −0.232 <0.0001 −0.423 0.005
Mathematical Processing (MTH) −0.037 <0.0001 −0.037 <0.0001 −0.066 0.006
1

Case-control sample includes enlisted Regular Army enlisted soldiers (i.e., excluding officers and members of the U.S. Army National Guard and Army Reserve) on active duty during the years 2004–2009. Cases are the subset of soldiers who completed neurocognitive testing prior to their first administratively documented suicide attempt. Controls are soldiers who completed the neurocognitive testing prior to their sampled person-month record, representing a subset of a 1:200 stratified probability sample of all active duty Regular Army person-months in the population, exclusive of soldiers with a non-fatal suicidal behavior and all person-months involving a death (i.e., due to suicide, combat, homicide, injury, or illness). All records in the 1:200 control sample were assigned a weight of 200 to adjust for the under-sampling of months not associated with a suicidal behavior.

2

Logistic regression models examined univariate associations (controlling only for historical time) of neurocognitive functioning with each outcome (suicide attempters, ideators, and decedents).

3

General factor score is based on the throughput scores of 4 tests: Code Substitution (CDS), Procedural Reaction Time (PRO), Matching to Sample (M2S), and Code Substitution Delayed (CDD). The Mathematical Processing (MTH) throughput score was examined as a separate variable.

*

p < 0.05

Table 5.

Multivariate associations of neurocognitive with subsequent suicide attempt, ideation, and death among Regular Army enlisted soldiers.1,2

Suicide Attempt
(N = 607)
Suicide Ideation
(N = 955)
Suicide Death
(N = 57)
Neurocognitive Predictors3 β p β p β p




Entered Separately
  General neurocognitive factor −0.197 <0.0001 −0.287 <0.0001 −0.521 0.001
  Mathematical Processing (MTH) −0.024 0.001 −0.027 <0.0001 −0.064 0.024
Entered Simultaneously
  General neurocognitive factor −0.164 0.001 −0.256 <0.0001 −0.417 0.011
  Mathematical Processing (MTH) −0.015 0.046 −0.014 0.018 −0.044 0.081
1

Case-control sample includes enlisted Regular Army enlisted soldiers (i.e., excluding officers and members of the U.S. Army National Guard and Army Reserve) on active duty during the years 2004-2009. Cases are the subset of soldiers who completed neurocognitive testing prior to their first administratively documented suicide attempt. Controls are soldiers who completed the neurocognitive testing prior to their sampled person-month record, representing a subset of a 1:200 stratified probability sample of all active duty Regular Army person-months in the population, exclusive of soldiers with a non-fatal suicidal behavior and all person-months involving a death (i.e., due to suicide, combat, homicide, injury, or illness). All records in the 1:200 control sample were assigned a weight of 200 to adjust for the under-sampling of months not associated with a suicidal behavior.

2

Logistic regression analyses controlled for historical time, gender, education, race, age at testing, and mental health diagnosis prior to testing. Neurocognitive predictors were first entered individually, then simultaneously, in models that controlled for these covariates.

3

General neurocognitive score is based on the throughput scores of 4 tests: Code Substitution (CDS), Procedural Reaction Time (PRO), Matching to Sample (M2S), and Code Substitution Delayed (CDD). The Mathematical Processing (MTH) throughput score was examined as a separate variable.

*

p < 0.05

Neurocognitive functioning performance and suicide ideation or death

These analyses were repeated separately with suicide ideators and suicide decedents. Among those who completed neurocognitive testing, ideator characteristics differed from those of controls in a pattern similar to attempters, whereas differences between decedents and controls were significant only for age at testing and mental health diagnosis prior to testing (Table 1). Ideators (t[1,091.2]=0.17, p=0.86) and decedents, (t[9,948]=0.80, p=0.42) did not differ from controls in time since test administration. ANAM testing occurred within the prior 12 months for 79.9% of ideators and 84.2% of decedents (vs. 83.0% of controls). Identification of test outliers resulted in the exclusion of 364 soldiers from the ideator analyses (60 cases, 304 controls) and 310 soldiers from the decedent analyses (4 case, 306 controls). All bivariate correlations between throughput scores in the ideator and decedent case-control samples were significant and nearly identical to those reported for attempters in Table 3.

In univariate models that controlled only for historical time, suicide ideation was predicted by poorer performance on the general neurocognitive factor (β= −0.232; p<0.0001) and MTH (β= −0.037; p<0.0001) (Table 4). When sociodemographics and mental health diagnosis prior to testing were added as covariates, both the general neurocognitive factor (β= −0.287; p<0.0001) and MTH (β= −0.027; p=0.001) continued to show a significant association with suicide attempt. The predictors remained significant when they were entered simultaneously in the same multivariate model: general neurocognitive factor, (β= −0.256; p<0.0001); MTH (β= −0.014; p=0.018) (Table 5).

Suicide death was also predicted by poorer performance on the general neurocognitive factor (β= −0.423; p=0.005) and MTH (β= −0.066; p=0.006) in univariate analyses. The associations persisted when sociodemographics and mental health diagnosis prior to testing were added as covariates: general neurocognitive factor (β= −0.521; p=0.001); MTH (β= −0.064; p=0.024). Results were similar when these predictors were entered simultaneously, however, only the general neurocognitive factor remained significant (β= −0.417; p=0.011), whereas MTH trended toward significance (β= −0.044; p=0.081) (Table 5).

Effects of mood at the time of testing

It is possible that the observed association between neurocognitive performance and suicidality could be due to participants’ mood during the time of test administration (e.g., those with low mood may have slower performance). As such, we repeated the multivariate analyses with participant mood included as an additional covariate (along with sociodemographics and mental health diagnosis prior to testing). The ANAM battery includes a self-assessment of current mood across seven dimensions: vigor (high energy level), happiness (positive disposition), depression (dysphoria), anger (negative disposition), fatigue (low energy level), anxiety (anxiety level), and restlessness (motor agitation). For each mood category, respondents are presented with a series of six adjectives (e.g., Shaky) and asked to rate the degree to which each adjective describes how they feel using a 7-point Likert scale (0=Not at all to 6=Very much). Adjective ratings are then averaged to create a score for each mood category (Johnson, Vincent, Johnson, Gilliland, & Schlegel, 2008).

The previous results were unchanged when examining the general neurocognitive factor and MTH separately: both variables predicted suicide attempt, ideation, and death after controlling for sociodemographics, mental health diagnosis prior to testing, and mood at the time of testing (results not shown). Results were similar when the predictors entered simultaneously, however, only the general neurocognitive factor remained a significant predictor of attempt (β= −0.175; p<0.0001), ideation (β= −0.135; p=0.008), and death (β= −0.368; p=0.025), whereas MTH was no longer associated with any outcome (β= −0.010 to −0.043; p=0.092–0.16).

DISCUSSION

We examined prospective associations between neurocognitive functioning and the subsequent onset of suicide attempts, suicide ideation, and suicide deaths. This examination yielded two key findings. First, the results revealed associations between decreased neurocognitive functioning, as measured by a general neurocognitive factor and mathematical processing, and increased risk of each of the suicide-related outcomes assessed. Second, and importantly, these associations remained even after adjusting for sociodemographics and mental health diagnosis prior to testing.

These prospective findings across multiple suicide-related outcomes, which remained after adjusting for meaningful covariates, reveal a clear and robust relationship between decreased neurocognitive functioning and suicidality. These effects were observed in a representative sample of predominantly young, healthy soldiers, suggesting that neurocognitive testing, in combination with other predictors, may contribute useful information about future risk for medically serious suicidal events. Additional research is needed to identify the extent to which suicide risk is associated with impairment in specific cognitive domains. While useful as a general measure of neurocognitive functioning, the Army’s ANAM TBI battery is most likely not optimal for parsing out the specific neurocognitive deficits associated with suicide risk.

Although mathematical processing was not significantly associated with suicide death after adjusting for the general neurocognitive factor, power to detect significant effects among decedents was likely limited by the small number of cases, a frequent and long-recognized problem due to the low base rate of suicide deaths (Pokorny, 1983; Rosen, 1954). It is also possible that the discrepant findings for fatal versus non-fatal suicidal events indicate that tests of mathematical processing are not sensitive to the cognitive profile of soldiers who die by suicide. Conversely, the findings may represent legitimate differences between those populations, which, despite many overlapping risk factors, are not identical. Consistent with prior studies (Nock et al., 2008), soldiers with documented suicide ideation or attempt were more likely than controls to be female, whereas those who died by suicide were more likely to be male. Fatal and non-fatal suicidal behaviors are also associated with different patterns of psychiatric morbidity and level of suicidal intent (Beautrais, 2001, 2003; Brent et al., 1988). We were unable to account for suicidal intent in this analysis of Army/DoD administrative data, but prior studies have found that risk factors for self-injury differ based on whether or not there was intent to die (Nock & Kessler, 2006). On the other hand, there is evidence from clinical, neurobiological, and family heritability studies that those who make a suicide attempt or die by suicide have similarities not shared with ideators (Brent & Mann, 2005; Linehan, 1986; Mann, 2003). These issues may be resolved with future studies that are able to include a larger number of suicide decedents.

The current study has five noteworthy limitations. First, the findings may not be representative of all enlisted soldiers, as baseline neurocognitive testing is typically conducted only with soldiers who are preparing to deploy. Similarly, the findings may not generalize to officers, which were excluded from the sample due to the small proportion with neurocognitive assessment data. Second, our use of medically documented outcomes means we likely captured the most severe events, but not those that were never reported. Soldiers and civilians with suicide ideation or attempt often do not receive treatment (Bruffaerts et al., 2011; Kessler, Berglund, Borges, Nock, & Wang, 2005; McKibben et al., 2014) and, therefore, would not be captured by medical records. Suicidal events that never come to medical attention may have different associations with neurocognitive functioning. Ideation, in particular, may go unreported to healthcare providers. We also cannot account for undocumented suicidal behavior that occurred prior to testing (e.g., pre-enlistment suicidality), or how those experiences may have influenced test scores. Third, our analyses did not control for a number of other potentially important variables, including deployment history and other life stressors (Nock et al., 2013; Ursano, Kessler, Stein, et al., 2015). Fourth, although the ANAM’s Simple Reaction Time test was excluded due to its low cognitive demands and weak associations with traditional neurocognitive measures and constructs (C-SHOP, 2007), several studies have found it is a sensitive indicator of cognitive changes and impairments (Cernich, Reeves, Sun, & Bleiberg, 2007; Reeves et al., 2006; Warden et al., 2001). Inclusion of that test may have altered the results. Finally, we were unable to examine the Go/No-Go test, which had not been administered to enough of the soldiers in our sample to include in the current study. It will be important for future studies to include this test, given the potential relevance of impulsivity to suicidal behavior (Jollant et al., 2011).

With these limitations in mind, our findings raise the possibility that neurocognitive testing may have value in understanding and assisting in the detection of suicide risk among soldiers by providing objective indicators that can supplement current strategies. However, its unique value in clinical decision-making for individual soldiers is extremely limited, if it exists at all. Such data are probably most useful as components of a risk prediction (e.g., machine learning) algorithm that draws on other risk indicators from a wide range of sources (e.g., Kessler et al., 2015), and for furthering our understanding of the neurobiology of suicide risk.

Although neurocognitive testing is currently administered prior to deployment, it is possible that risk detection could be aided by collecting baseline neurocognitive data at an earlier point (e.g., during accession). Many soldiers report a pre-enlistment history of suicidal behavior and mental disorders (Rosellini et al., 2015; Ursano, Heeringa, et al., 2015), and the initial months of Army service are a high risk period for suicide attempts (Ursano, Kessler, Stein, et al., 2015). The utility of earlier neurocognitive screening in detecting suicide risk will be further examined in the Army STARRS New Soldier Study (Ursano et al., 2014), a survey of soldiers in their first week of basic training that includes an assessment of neurocognitive domains found to be associated with suicidal behavior and other adverse mental health outcomes (e.g., Thomas et al., 2013).

It is important to note that the ANAM is not designed to measure suicide risk. Risk detection might be substantially improved by incorporating tests of cognitive domains that have demonstrated stronger and more consistent associations with suicidal behavior, such as decision-making, problem-solving, verbal fluency, and memory (Jollant et al., 2011; Richard-Devantoy et al., 2014a, 2014b), or tests designed to measure aspects of suicide-specific cognition (Cha, Najmi, Park, Finn, & Nock, 2010; Harrison, Stritzke, Fay, Ellison, & Hudaib, 2014; Nock et al., 2010). In addition, recent DoD efforts to develop mobile neurocognitive assessment platforms (Elsmore, Reeves, & Reeves, 2007; Lathan, Spira, Bleiberg, Vice, & Tsao, 2013) may eventually provide opportunities to monitor neurocognitive correlates of suicide risk among soldiers in forward-deployed environments where standard computer-based test administration is unfeasible. This might be an important capability given the impairments in cognitive functioning that service members can experience during combat deployment (Vasterling et al., 2006) or while operating in other extreme environments (Lathan et al., 2013).

Conclusions

These preliminary findings raise the possibility that decreased neurocognitive functioning could indicate a diathesis for suicidal thoughts and behavior (Lowe et al., 2007). Future studies should examine the utility of other, more specific neurocognitive tests in risk detection among soldiers, and whether associations between specific neurocognitive domains and suicide-related outcomes are modulated by different experiences and environmental exposures (e.g., combat, interpersonal conflict, legal problems).

Supplementary Material

01

Acknowledgments

The Army STARRS Team consists of Co-Principal Investigators: Robert J. Ursano, MD (Uniformed Services University of the Health Sciences) and Murray B. Stein, MD, MPH (University of California San Diego and VA San Diego Healthcare System). Site Principal Investigators: Steven Heeringa, PhD (University of Michigan) and Ronald C. Kessler, PhD (Harvard Medical School). National Institute of Mental Health (NIMH) collaborating scientists: Lisa J. Colpe, PhD, MPH and Michael Schoenbaum, PhD. Army liaisons/consultants: COL Steven Cersovsky, MD, MPH (USAPHC) and Kenneth Cox, MD, MPH (USAPHC)

Other team members: Pablo A. Aliaga, MA (Uniformed Services University of the Health Sciences); COL David M. Benedek, MD (Uniformed Services University of the Health Sciences); K. Nikki Benevides, MA (Uniformed Services University of the Health Sciences); Paul D. Bliese, PhD (University of South Carolina); Susan Borja, PhD (NIMH); Evelyn J. Bromet, PhD (Stony Brook University School of Medicine); Gregory G. Brown, PhD (University of California San Diego); Laura Campbell-Sills, PhD (University of California San Diego); Catherine L. Dempsey, PhD, MPH (Uniformed Services University of the Health Sciences); Carol S. Fullerton, PhD (Uniformed Services University of the Health Sciences); Nancy Gebler, MA (University of Michigan); Robert K. Gifford, PhD (Uniformed Services University of the Health Sciences); Stephen E. Gilman, ScD (Harvard School of Public Health); Marjan G. Holloway, PhD (Uniformed Services University of the Health Sciences); Paul E. Hurwitz, MPH (Uniformed Services University of the Health Sciences); Sonia Jain, PhD (University of California San Diego); Tzu-Cheg Kao, PhD (Uniformed Services University of the Health Sciences); Karestan C. Koenen, PhD (Columbia University); Lisa Lewandowski-Romps, PhD (University of Michigan); Holly Herberman Mash, PhD (Uniformed Services University of the Health Sciences); James E. McCarroll, PhD, MPH (Uniformed Services University of the Health Sciences); James A. Naifeh, PhD (Uniformed Services University of the Health Sciences); Tsz Hin Hinz Ng, MPH (Uniformed Services University of the Health Sciences); Matthew K. Nock, PhD (Harvard University); Rema Raman, PhD (University of California San Diego); Holly J. Ramsawh, PhD (Uniformed Services University of the Health Sciences); Anthony Joseph Rosellini, PhD (Harvard Medical School); Nancy A. Sampson, BA (Harvard Medical School); LCDR Patcho Santiago, MD, MPH (Uniformed Services University of the Health Sciences); Michaelle Scanlon, MBA (NIMH); Jordan W. Smoller, MD, ScD (Harvard Medical School); Amy Street, PhD (Boston University School of Medicine); Michael L. Thomas, PhD (University of California San Diego); Leming Wang, MS (Uniformed Services University of the Health Sciences); Christina L. Wassel, PhD (University of Pittsburgh); Simon Wessely, FMedSci (King’s College London); Christina L. Wryter, BA (Uniformed Services University of the Health Sciences); Hongyan Wu, MPH (Uniformed Services University of the Health Sciences); LTC Gary H. Wynn, MD (Uniformed Services University of the Health Sciences); and Alan M. Zaslavsky, PhD (Harvard Medical School).

Army STARRS was sponsored by the Department of the Army and funded under cooperative agreement number U01MH087981 with the U.S. Department of Health and Human Services, National Institutes of Health, National Institute of Mental Health (NIH/NIMH). The contents are solely the responsibility of the authors and do not necessarily represent the views of the Department of Health and Human Services, NIMH, the Veterans Administration, Department of the Army, or the Department of Defense.

REFERENCES

  1. Appenzeller GN, Warner CH, Grieger T. Postdeployment health reassessment: A sustainable method for brigade combat teams. Military Medicine. 2007;172(10):1017–1023. doi: 10.7205/milmed.172.10.1017. [DOI] [PubMed] [Google Scholar]
  2. Bachynski KE, Canham-Chervak M, Black SA, Dada EO, Millikan AM, Jones BH. Mental health risk factors for suicides in the US Army, 2007–8. Injury Prevention. 2012;18:405–412. doi: 10.1136/injuryprev-2011-040112. [DOI] [PubMed] [Google Scholar]
  3. Beautrais AL. Suicides and serious suicide attempts: Two populations or one? Psychological Medicine. 2001;31:837–845. doi: 10.1017/s0033291701003889. [DOI] [PubMed] [Google Scholar]
  4. Beautrais AL. Suicide and serious suicide attempts in youth: A multi-group comparison study. American Journal of Psychiatry. 2003;160:1093–1099. doi: 10.1176/appi.ajp.160.6.1093. [DOI] [PubMed] [Google Scholar]
  5. Bell NS, Harford TC, Amoroso PJ, Hollander IE, Kay AB. Prior health care utilization patters and sucide among U.S. Army soldiers. Suicide and Life-Threatening Behavior. 2010;40:407–415. doi: 10.1521/suli.2010.40.4.407. [DOI] [PubMed] [Google Scholar]
  6. Black SA, Gallaway MS, Bell MR, Ritchie EC. Prevalence and risk factors associated with suicides of army soldiers 2001–2009. Military Psychology. 2011;23:433–451. [Google Scholar]
  7. Bleiberg J, Kane RL, Reeves DL, Garmoe WS, Halpern E. Factor analysis of computerized and traditional tests used in mild brain injury research. The Clinical Neuropsychologist. 2000;14(3):287–294. doi: 10.1076/1385-4046(200008)14:3;1-P;FT287. [DOI] [PubMed] [Google Scholar]
  8. Brent DA, Mann JJ. Family genetic studies, suicide, and suicidal behavior. American Journal of Medical Genetics Part C. 2005;133C:13–24. doi: 10.1002/ajmg.c.30042. [DOI] [PubMed] [Google Scholar]
  9. Brent DA, Perper JA, Goldstein CE, Kolko DJ, Allan MJ, Allman CJ, et al. Risk factors for adolescent suicide: A comparison of adolescent suicide victims with suicidal inpatients. Archives of General Psychiatry. 1988;45(6):581–588. doi: 10.1001/archpsyc.1988.01800300079011. [DOI] [PubMed] [Google Scholar]
  10. Bruffaerts R, Demyttenaere K, Hwang I, Tat Chiu W, Sampson N, Kessler RC, et al. Treatment of suicidal people around the world. British Journal of Psychiatry. 2011;199:64–70. doi: 10.1192/bjp.bp.110.084129. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. C-SHOP. ANAM4 TBI: User manual. Norman, OK: Center for the Study of Human Operator Performance, University of Oklahoma; 2007. [Google Scholar]
  12. Cernich AN, Brennana DM, Barker LM, Bleiberg J. Sources of error in computerized neuropsychological assessment. Archives of Clinical Neuropsychology. 2007;22S:S39–S48. doi: 10.1016/j.acn.2006.10.004. [DOI] [PubMed] [Google Scholar]
  13. Cernich AN, Reeves DL, Sun W, Bleiberg J. Automated Neuropsychological Assessment Metrics sports medicine battery. Archives of Clinical Neuropsychology. 2007;22S:S101–S114. doi: 10.1016/j.acn.2006.10.008. [DOI] [PubMed] [Google Scholar]
  14. Cha CB, Najmi S, Park JM, Finn CT, Nock MK. Attentional bias toward suicide-related stimuli predicts suicidal behavior. Journal of Abnormal Psychology. 2010;119(3):616–622. doi: 10.1037/a0019710. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Courtet P, Gottesman II, Jollant F, Gould TD. The neuroscience of suicidal behaviors: What can we expect from endophenotype strategies? Translational Psychiatry. 2011;1:e7. doi: 10.1038/tp.2011.6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Elsmore TF, Reeves DL, Reeves AN. The ARES® test system for palm OS handheld computers. Archives of Clinical Neuropsychology. 2007;22S:S135–S144. doi: 10.1016/j.acn.2006.10.009. [DOI] [PubMed] [Google Scholar]
  17. Gahm GA, Reger MA, Kinn JT, Luxton DD, Skopp NA, Bush NE. Addressing the surveillance goal in the National Strategy for Suicide Prevention: The Department of Defense Suicide Event Report. American Journal of Public Health. 2012;102(Suppl 1):S24–S28. doi: 10.2105/AJPH.2011.300574. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Gilman SE, Bromet EJ, Cox KL, Colpe LJ, Fullerton CS, Gruber MJ, et al. Socio-demographic and career history predictors of suicide and suicide mortality in the United States Army 2004–2009. Psychological Medicine. 2014;44(12):2579–2592. doi: 10.1017/S003329171400018X. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Harrison DP, Stritzke WGK, Fay N, Ellison TM, Hudaib A-R. Probing the implicit suicidal mind: Does the death/suicide implicit association test reveal a desire to die, or a diminished desire to live? Psychological Assessment. 2014;26(3):831–840. doi: 10.1037/pas0000001. [DOI] [PubMed] [Google Scholar]
  20. Johnson DR, Vincent AS, Johnson AE, Gilliland K, Schlegel RE. Reliability and construct validity of the Automated Neuropsychological Assessment Metrics (ANAM) mood scale. Archives of Clinical Neuropsychology. 2008;23:73–85. doi: 10.1016/j.acn.2007.10.001. [DOI] [PubMed] [Google Scholar]
  21. Jollant F, Lawrence NL, Olie E, Guillaume S, Courtet P. The suicidal mind and brain: A review of neuropsychological and neuroimaging studies. World Journal of Biological Psychiatry. 2011;12(5):319–339. doi: 10.3109/15622975.2011.556200. [DOI] [PubMed] [Google Scholar]
  22. Kabat MH, Kane RL, Jefferson AL, DePino RK. Construct validity of selected Automated Neuropsychological Assessment Metrics (ANAM) battery measures. The Clinical Neuropsychologist. 2001;15(4):498–507. doi: 10.1076/clin.15.4.498.1882. [DOI] [PubMed] [Google Scholar]
  23. Kessler RC, Berglund P, Borges G, Nock M, Wang PS. Trends in suicide ideation, plans, gestures, and attempts in the United States, 1990–1992 to 2001–2003. Journal of the American Medical Association. 2005;293(20):2487–2495. doi: 10.1001/jama.293.20.2487. [DOI] [PubMed] [Google Scholar]
  24. Kessler RC, Colpe LJ, Fullerton CS, Gebler N, Naifeh JA, Nock MK, et al. Design of the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS) International Journal of Methods in Psychiatric Research. 2013;22(4):267–275. doi: 10.1002/mpr.1401. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Kessler RC, Warner CH, Ivany C, Petukhova MV, Rose S, Bromet EJ, et al. Predicting suicides after psychiatric hospitalization in US Army soldiers: The Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS) JAMA Psychiatry. 2015;72(1):49–57. doi: 10.1001/jamapsychiatry.2014.1754. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Lathan C, Spira JL, Bleiberg J, Vice J, Tsao JW. Defense Automated Neurobehavioral Assessment (DANA): Psychometric properties of a new field-deployable neurocognitive assessment tool. Military Medicine. 2013;178:365–371. doi: 10.7205/MILMED-D-12-00438. [DOI] [PubMed] [Google Scholar]
  27. Linehan MM. Suicidal people: One population or two? Annals of the New York Academy of Sciences. 1986;487:16–33. doi: 10.1111/j.1749-6632.1986.tb27882.x. [DOI] [PubMed] [Google Scholar]
  28. Logan J, Skopp NA, Karch D, Reger MA, Gahm GA. Characteristics of suicides among US Army active duty personnel in 17 US states from 2005 to 2007. American Journal of Public Health. 2012;102(Suppl 1):S40–S44. doi: 10.2105/AJPH.2011.300481. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Lowe M, Harris W, Kane RL, Banderet L, Levinson D, Reeves D. Neuropsychological assessment in extreme environments. Archives of Clinical Neuropsychology. 2007;22S:S89–S99. doi: 10.1016/j.acn.2006.10.010. [DOI] [PubMed] [Google Scholar]
  30. Mann JJ. Neurobiology of suicidal behavior. Nature Reviews Neuroscience. 2003;4:819–828. doi: 10.1038/nrn1220. [DOI] [PubMed] [Google Scholar]
  31. Mann JJ, Arango VA, Avenevoli S, Brent DA, Champagne FA, Clayton P, et al. Candidate endophenotypes for genetic studies of suicidal behavior. Biological Psychiatry. 2009;65(7):556–563. doi: 10.1016/j.biopsych.2008.11.021. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. McKibben JBA, Fullerton CS, Herberman-Mash HB, Nock MK, Naifeh JA, Kessler RC, et al. Suicidal behaviors and the use of mental health services among active duty Army soldiers. Psychiatric Services. 2014;65(3):374–380. doi: 10.1176/appi.ps.201200460. [DOI] [PubMed] [Google Scholar]
  33. Nock MK, Borges G, Bromet EJ, Cha CB, Kessler RC, Lee S. Suicide and suicidal behavior. Epidemiologic Reviews. 2008;30:133–154. doi: 10.1093/epirev/mxn002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Nock MK, Deming CA, Fullerton CS, Gilman SE, Goldenberg MN, Kessler RC, et al. Suicide among Soldiers: A review of psychosocial risk and protective factors. Psychiatry. 2013;76:97–125. doi: 10.1521/psyc.2013.76.2.97. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Nock MK, Kessler RC. Prevalence of and risk factors for suicide attempts versus suicide gestures: Analysis of the National Comorbidity Survey. Journal of Abnormal Psychology. 2006;115(3):616–623. doi: 10.1037/0021-843X.115.3.616. [DOI] [PubMed] [Google Scholar]
  36. Nock MK, Park JM, Finn CT, Deliberto TL, Dour HJ, Banaji MR. Measuring the suicidal mind: Implicit cognition predicts suicidal behavior. Psychological Science. 2010;21(4):511–517. doi: 10.1177/0956797610364762. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Pokorny AD. Prediction of suicide in psychiatric patients: Report of a prospective study. Archives of General Psychiatry. 1983;40(3):249–257. doi: 10.1001/archpsyc.1983.01790030019002. [DOI] [PubMed] [Google Scholar]
  38. Ramchand R, Acosta J, Burns RM, Jaycox LH, Pernin CG. The war within: Preventing suicide in the U.S. military. Santa Monica, CA: RAND Corporation; 2011. [PMC free article] [PubMed] [Google Scholar]
  39. Reeves DL, Bleiberg J, Roebuck-Spencer T, Cernich AN, Schwab K, Ivins B, et al. Reference values for performance on the Automated Neuropsychological Assessment Metrics V3.0 in an active duty military sample. Military Medicine. 2006;171:982–994. doi: 10.7205/milmed.171.10.982. [DOI] [PubMed] [Google Scholar]
  40. Richard-Devantoy S, Berlim MT, Jollant F. A meta-analysis of neuropsychological markers of vulnerability to suicidal behavior in mood disorders. Psychological Medicine. 2014a;44:1663–1673. doi: 10.1017/S0033291713002304. [DOI] [PubMed] [Google Scholar]
  41. Richard-Devantoy S, Berlim MT, Jollant F. Suicidal behaviour and memory: A systematic review and meta-analysis. World Journal of Biological Psychiatry. 2014b doi: 10.3109/15622975.2014.925584. Epub ahead of print. [DOI] [PubMed] [Google Scholar]
  42. Rosellini AJ, Heeringa SG, Stein MB, Ursano RJ, Chiu WT, Colpe LJ, et al. Lifetime prevalence of DSM-IV mental disorders among new soldiers in the U.S. Army: Results from the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS) Depression and Anxiety. 2015;32:13–24. doi: 10.1002/da.22316. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Rosen A. Detection of suicidal patients: An example of some limitations in the prediction of infrequent events. Journal of Consulting Psychology. 1954;18(6):397–403. doi: 10.1037/h0058579. [DOI] [PubMed] [Google Scholar]
  44. SAS Institute Inc. SAS® 9.3 Software. Cary, NC: SAS Institute Inc.; 2011. [Google Scholar]
  45. Schoenbaum M, Kessler RC, Gilman SE, Colpe LJ, Heeringa SG, Stein MB, et al. Predictors of suicide and accident death in the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS) JAMA Psychiatry. 2014;71(5):493–503. doi: 10.1001/jamapsychiatry.2013.4417. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Short P, Cernich A, Wilken JA, Kane RL. Initial construct validation of frequently employed ANAM measures through structural equation modeling. Archives of Clinical Neuropsychology. 2007;22S:S63–S77. doi: 10.1016/j.acn.2006.10.012. [DOI] [PubMed] [Google Scholar]
  47. Street AE, Gilman SE, Rosellini AJ, Stein MB, Bromet EJ, Cox KL, et al. Understanding the elevated suicide risk of female soldiers during deployments. Psychological Medicine. 2015;45:717–726. doi: 10.1017/S003329171400258X. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Thomas ML, Brown GG, Gur RC, Hansen JA, Nock MK, Heeringa SG, et al. Parallel psychometric and cognitive modeling analyses of the Penn Face Memory Test in the Army Study to Assess Risk and Resilience in Servicemembers. Journal of Clinical and Experimental Neuropsychology. 2013;35(3):225–245. doi: 10.1080/13803395.2012.762974. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Thorne DR. Throughput: a simple performance index with desirable characteristics. Behavior Research Methods. 2006;38(4):569–573. doi: 10.3758/bf03193886. [DOI] [PubMed] [Google Scholar]
  50. Ursano RJ, Colpe LJ, Heeringa SG, Kessler RC, Schoenbaum M, Stein MB. The Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS) Psychiatry. 2014;72(2):107–119. doi: 10.1521/psyc.2014.77.2.107. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Ursano RJ, Heeringa SG, Stein MB, Jain S, Raman R, Sun X, et al. Prevalence and correlates of suicidal behavior among new soldiers in the US Army: Results from the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS) Depression and Anxiety. 2015;32:3–12. doi: 10.1002/da.22317. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Ursano RJ, Kessler RC, Heeringa SG, Cox KL, Naifeh JA, Fullerton CS, et al. Nonfatal suicidal behaviors in U.S. Army administrative records, 2004–2009: Results from the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS) Psychiatry. 2015;78:1–21. doi: 10.1080/00332747.2015.1006512. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Ursano RJ, Kessler RC, Stein MB, Naifeh JA, Aliaga PA, Fullerton CS, et al. Suicide Attempts in the U.S. Army during the wars in Afghanistan and Iraq, 2004–2009. JAMA Psychiatry. 2015 doi: 10.1001/jamapsychiatry.2015.0987. in press, Epub ahead of print. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Vasterling JJ, Proctor SP, Amoroso P, Kane RL, Heeren T, White RF. Neuropsychological outcomes of army personnel following deployment to the Iraq war. Journal of the American Medical Association. 2006;296(5):519–529. doi: 10.1001/jama.296.5.519. [DOI] [PubMed] [Google Scholar]
  55. Walkup JT, Townsend L, Crystal S, Olfson M. A systematic review of validated methods for identifying suicide or suicidal ideation using administrative or claims data. Pharmacoepidemiology and Drug Safety. 2012;21(Suppl 1):174–182. doi: 10.1002/pds.2335. [DOI] [PubMed] [Google Scholar]
  56. Warden DL, Bleiberg J, Cameron KL, Ecklund J, Walter J, Sparling MB, et al. Persistent prolongation of simple reaction time in sports concussion. Neurology. 2001;57(3):524–526. doi: 10.1212/wnl.57.3.524. [DOI] [PubMed] [Google Scholar]
  57. Warner CH, Appenzeller GN, Grieger T, Belenkiy S, Breitbach J, Parker J, et al. Importance of anonymity to encourage honest reporting in mental health screening after combat deployment. Archives of General Psychiatry. 2011;68:1065–1071. doi: 10.1001/archgenpsychiatry.2011.112. [DOI] [PubMed] [Google Scholar]
  58. Warner CH, Breitbach JE, Appenzeller GN, Yates V, Grieger T, Webster WG. Division mental health in the new brigade combat team structure: Part I. Predeployment and deployment. Military Medicine. 2007a;172(9):907–911. doi: 10.7205/milmed.172.9.907. [DOI] [PubMed] [Google Scholar]
  59. Warner CH, Breitbach JE, Appenzeller GN, Yates V, Grieger T, Webster WG. Division mental health in the new brigade combat team structure: Part II. Redeployment and postdeployment. Military Medicine. 2007b;172(9):912–917. doi: 10.7205/milmed.172.9.912. [DOI] [PubMed] [Google Scholar]
  60. Willett JB, Singer JD. Investigating onset, cessation, relapse, and recovery: why you should, and how you can, use discrete-time survival analysis to examine event occurrence. Journal of Consulting and Clinical Psychology. 1993;61(6):952–965. doi: 10.1037//0022-006x.61.6.952. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

01

RESOURCES