Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Nov 9.
Published in final edited form as: Psychol Med. 2017 Apr 4;47(13):2275–2287. doi: 10.1017/S003329171700071X

Using self-report surveys at the beginning of service to develop multi-outcome risk models for new soldiers in the U.S. Army

Anthony J Rosellini 1, Murray B Stein 2,3, David M Benedek 4, Paul D Bliese 5, Wai Tat Chiu 1, Irving Hwang 1, John Monahan 6, Matthew K Nock 7, Maria V Petukhova 1, Nancy A Sampson 1, Amy E Street 8,9, Alan M Zaslavsky 1, Robert J Ursano 4, Ronald C Kessler 1,*, On behalf of the Army STARRS Collaborators
PMCID: PMC5679702  NIHMSID: NIHMS917118  PMID: 28374665

Abstract

Background

The U.S. Army uses universal preventives interventions for several negative outcomes (e.g., suicide, violence, sexual assault) with especially high risks in the early years of service. More intensive interventions exist, but would be cost-effective only if targeted at high-risk soldiers. We report results of efforts to develop models for such targeting from self-report surveys administered at the beginning of Army service.

Methods

21,832 new soldiers completed a self-administered questionnaire (SAQ) in 2011–2012 and consented to link administrative data to SAQ responses. Penalized regression models were developed for 12 administratively-recorded outcomes occurring by December 2013: suicide attempt, mental hospitalization, positive drug test, traumatic brain injury (TBI), other severe injury, several types of violence perpetration and victimization, demotion, and attrition.

Results

The best-performing models were for TBI (AUC=0.80), major physical violence perpetration (AUC=0.78), sexual assault perpetration (AUC=0.78), and suicide attempt (AUC=0.74). Although predicted risk scores were significantly correlated across outcomes, prediction was not improved by including risk scores for other outcomes in models. Of particular note: 40.5% of suicide attempts occurred among the 10% of new soldiers with highest predicted risk, 57.2% of male sexual assault perpetrations among the 15% with highest predicted risk, and 35.5% of female sexual assault victimizations among the 10% with highest predicted risk.

Conclusions

Data collected at the beginning of service in self-report surveys could be used to develop risk models that define small proportions of new soldiers accounting for high proportions of negative outcomes over the first few years of service.

Keywords: Army, military, predictive modeling, risk assessment, mental health, violence, disciplinary problems

INTRODUCTION

The U.S. Army and Department of Defense (DoD) have implemented prevention programs for several negative outcomes (Department of the Army, 2015b), including suicide (Department of the Army, 2015a), workplace violence (Department of Defense, 2014b), and sexual assault (Department of Defense, 2014a), that have high prevalence in the early years of the Army career (Kaufman et al. 2000; Department of the US Army, 2010, 2012). These preventive interventions are mostly universal; that is, all personnel are required to participate and the interventions are relatively non-intensive. More intensive interventions exist and could be implemented (Vungkhanching et al. 2007; Parkkari et al. 2011; Shea et al. 2013; Rudd et al. 2015; Senn et al. 2015) but would be cost-effective only if targeted at high-risk personnel (Foster & Jones, 2006; Golubnitschaja & Costigliola, 2012). This targeting would require valid risk prediction tools. Recent studies have shown that Army and DoD administrative data can be used to develop such tools to predict negative soldier outcomes such as suicide (Kessler et al. 2015), violent crime perpetration (Rosellini et al. 2016), and sexual assault victimization (Street et al. 2016), but these models are limited by the fact that administrative data only become available over the course of time and are unavailable when preventive interventions might most logically be implemented at the beginning of service. An alternative would be to implement a risk factor survey at the beginning of service to target new recruits for preventive interventions.. The current report presents the results of an attempt to develop risk models for a number of high-priority negative outcomes using data collected in such a survey of new U.S. Army soldiers subsequently followed over the first two years of service. If successful, the logic of this approach might be generalizable to a wide range of other workplace settings.

METHODS

Sample

The survey was the New Soldier Survey (NSS) of the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS) (Ursano et al. 2014).The NSS was implemented April 2011-November 2012 in a representative sample of new U.S. Army soldiers prior to beginning Basic Combat Training (BCT) at Fort Benning, GA, Fort Jackson, SC, and Fort Leonard Wood, MO. Recruitment began by selecting weekly samples of 200–300 new soldiers at each installation to attend an informed consent presentation within 48 hours of reporting for BCT. The presentation explained study purposes, confidentiality, voluntary participation, and answered all attendee questions before seeking written informed consent for a self-administered computerized questionnaire (SAQ) and neurocognitive tests and to link these data to the soldier’s administrative records. These study recruitment and consent procedures were approved by the Human Subjects Committees of all Army STARRS collaborating organizations. The 21,832 NSS respondents considered here represent all Regular Army soldiers who completed the SAQ and agreed to administrative data linkage (77.1% response rate) (Rosellini et al. 2015). Data were doubly-weighted to adjust for differences in survey responses among the respondents who did versus did not agree to administrative record linkage and differences in administrative data profiles between the latter subsample and the population of all new soldiers.

Outcomes

Outcome data were abstracted from 14 administrative databases through December 2013 (13–33 follow-up months after NSS completion) to operationalize 12 high-priority outcomes involving mental-physical disorders (Canham-Chervak et al. 2010; Department of the US Army, 2010; Institute of Medicine, 2010), violent crime perpetration-victimization (Institute of Medicine, 2010; Department of the US Army, 2012), and career problems (Kubisiak et al. 2009; Kapp, 2013). Dichotomous dependent variables were defined for first occurrence of each of the following outcomes:

Mental and physical disorders

Suicide attempt was defined based on the DoD Suicide Event Reporting system and ICD-9-CM treatment codes. A positive test for illicit drug use was defined based on the Army substance abuse tracking database. Three health outcomes were defined from electronic medical records: mental hospitalization, traumatic brain injury (TBI); and any other severe injury exclusive of TBI (e.g., blindness, deafness, amputation, severe burns, paralysis), all based on International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes.

Violent crime

DoD criminal justice databases were used to define three measures of violent crime perpetration (major physical, sexual assault, and minor violence [e.g., harassment]) and two of victimization (minor violence; sexual assault) coded according to the Bureau of Justice Statistics National Corrections Reporting Program classification system (U.S. Department of Justice, 2011) and of sufficient frequency to develop prediction models. The perpetration outcomes were defined from records of “founded” offenses (i.e., where the Army found sufficient evidence to warrant full investigation). The victimization outcomes were defined using any officially reported victimization regardless of evidence.

Army career problems

Premature attrition from service due to career or personal problems (e.g., a character or behavior disorder; disability) was defined using an Army personnel database that tracks reasons for separating from service. Demotion was defined using information in the Army master personnel file.

Independent variables

Our goal was to optimize classification of soldiers who subsequently had the outcomes rather than to test specific hypotheses. As a result, we considered all potential predictors for which there was any evidence in the literature; e.g., risk-protective factors for suicidality (Nock et al. 2013; Afifi et al. 2016), mental hospitalization (Iribarren et al. 2000; Rytila-Manninen et al. 2014), substance problems (Kilpatrick et al. 2000; Kirst et al. 2014), TBI (Cassidy et al. 2004; Elmasry et al. 2017), other severe physical injuries (Bulzacchelli et al. 2014; Theodoroff et al. 2015), violence perpetration (Dahlberg, 1998; Elbogen et al. 2010), violence victimization (Suris & Lind, 2008; Turchik & Wilson, 2010), and career problems (Knapik et al. 2004; Booth-Kewley et al. 2010). In total, 727 independent variables were operationalized from the SAQ in addition to 8 performance-based neurocognitive test measures assessed in conjunction with the SAQ and 37 basic administrative variables recorded for all new soldiers at the beginning of service (772 total variables). (Appendix Table 1 summarizes all independent variables.)

Appendix Table 1.

Overview of the 772 independent variables used to predict the outcomes, organized by broad conceptual category

I. Self-report (727 predictors)1
 Socio-demographic (114 variables) Standard survey questions were used to define the majority of the socio-demographic variables. Variables included: sex; age at interview; race-ethnicity; religious afflation(s); several aspects of religiosity (e.g., religiosity/spirituality; religious participation; fundamentalism); educational attainment; marital and non-marital relationship status; length of marriage or divorce; number of dependents (e.g., total number; number of children; ages of children); number of older/younger siblings; and immigration status. These variables were primarily dichotomous (nested and non-nested dummies), though a few were continuous (e.g., age). In addition, the NSS included 11 questions that asked about reasons why the soldier decided to enlist. These questions were adapted from a prior survey of new recruits.3 The 11 items were subjected to an exploratory factor analysis, which resulted in defining two continuous scales: (i) enlisting for reasons related to patriotism and personal growth (e.g. enlisting to “serve your country” and “do something you can be proud of” ), and (ii) enlisting to escape from home (e.g., enlisting to “be away from home” ).
 Mental disorders (233 variables) Most of the lifetime DSM-IV mental disorder constructs were assessed using the self-report computerized version of the Composite International Diagnostic Interview screening scales (CIDI-SC),4 including: major depressive episode, bipolar I-II or subthreshold bipolar disorder, generalized anxiety disorder, panic disorder, intermittent explosive disorder, conduct disorder, oppositional defiant disorder, substance use disorder, and attention-deficit/hyperactivity disorder (ADHD in the past 6 months, not lifetime). Lifetime posttraumatic stress disorder (PTSD) was assessed using a screening version of the PTSD Checklist.5 Lifetime insomnia was assessed using an adapted version of the American Insomnia Survey.6 In addition to defining dichotomous variables representing the presence-absence of each of these disorders, we also created continuous variables representing the severity of symptoms during a lifetime episode. Lifetime social phobia, agoraphobia, specific phobia, and obsessive-compulsive disorder were assessed using single-item screeners adapted from the Family History Screen.7 Continuous and nested dichotomous (e.g., 1+, 2+, 3+ disorders) variables were defined to represent the total number of lifetime disorders. Disorder persistence (number of years with the disorder) was assessed for all disorders using questions adapted from the CIDI.4 Nested dichotomous variables were also defined for the persistence of each disorder that was assessed (e.g., 1+ years with the disorder, 2+ years with the disorder). Frequency of use of nine substances (e.g., alcohol, marijuana, prescription medications) during the peak period of use was also assessed using questions adapted from the CIDI.4 Nested dichotomous variables were also operationalized for frequency of use (e.g., less than once a month or more; 1–3 days per month or more; 1–3 days per week or more). Lifetime treatment or counseling (for psychological problems) with a mental health professional or any other type of provider (e.g., medical doctor, spiritual leader) was assessed using questions adapted from the Land Combat Study.8 We operationalized variables representing the present-absence of lifetime treatment as well as the nested dichotomous for number of years with treatment (e.g., 1+ years, 2+ years, 3+ years).
 Stressors (222 variables) The stress variables were operationalized using questions that assessed 12-month and lifetime events and chronic strains occurring both within and outside of the family, lifetime traumatic events, and adversity experienced during childhood. Questions assessing 13 stressful events in the 12-months prior to enlistment within social (e.g., divorce, break-up) and non-social domains (e.g., car accident, police trouble) were adapted from the Life Events Questionnaire9 and DoD Survey of Health Related Behaviors.10 We defined dichotomous variables for each stressful event as well as a composite (continuous) variable of total number of stressful events in the past 12 months. Extent of chronic role strain in eight domains (e.g., finances, health, overall stress) over the 12-months prior to enlistment were assessed using questions adapted from the National Comorbidity Survey – Replication (NCS-R11). We defined nested dichotomous variables within each domain (e.g., severe or very severe strains) as well as a composite (continuous) variable of total extent chronic strain across domains. Questions assessing the number of times each of 15 traumatic events (e.g., physical assault, sexual assault, suicide of close friend) occurred over the soldiers’ lifetime were adapted from the CIDI4 and used to define categorical (e.g., 0=never; 1=1 time; 2=2–4 times; 3=5–9 times; 4=10 or more times) and nested dichotomous (e.g., being physically assaulted 1+, 2+, 3+ times) independent variables, as well as a composite (continuous) variable for total number of lifetime traumas. Questions asking about lifetime number of head injuries (e.g., perforated eardrum; losing consciousness) were developed by the STARRS study team and used to operationalize nested dichotomous head trauma variables (1+ head injury, 2+ head injuries). Questions asking about the occurrence-frequency of childhood adversities were adapted from the Family History Screen,7 CIDI,4 Adverse Childhood Experiences Survey,12 and Childhood Trauma Questionnaire13 to assess parent-family psychopathology (e.g., anxiety, mood, substance use disorder), maladaptive family functioning (e.g., emotional, physical, or sexual abuse or neglect at home; having a parent in prison) and other family adversities (e.g., death of a parent or separation from a parent), positive family functioning (e.g., feeling cared for by family), and non-family adversities (e.g., victim of bullying; sexual abuse outside the family). We operationalized ordinal and nested dichotomous independent variables from these questions, the only exceptions being adversities that would be very unlikely to occur more than once (only presence-absence dichotomies were defined, e.g., for having a parent commit suicide; parental divorce). We also operationalized 6 composite (continuous) scales based on the questions assessing: (i) parental psychopathology (i.e., total number of mental disorders between both parents); (ii) total familial abuse-neglect (i.e., total frequency of emotional, physical, and sexual abuse-neglect in the family);(iii) total number of maladaptive family functioning events; (iv) total number of types of different maladaptive family function events; (v) total number of sexual assaults occurring inside and outside of the family; and (vi) the total number of positive family experience (adaptive family functioning).
 Personality (55 variables) The survey included 91 questions adapted from previously validated self-report personality questionnaires,15–29 intended to assess a total of 28 constructs. Four of the items were used to define dichotomous variables representing four attachment style (secure; dismissive; fearful; preoccupied)16 as well as nested dummies representing the degree of four attachment styles (e.g., someone or very characteristic; very characteristic). The remaining 87 items were used to develop 24 “rational” scales (continuous) based on 24 personality traits of interest. The same set of items was included in both the NSS and the Pre-Post Deployment Study (PPDS) baseline survey, allowing us to define and compare scales across independent samples. The validity of the rational scales was evaluated in the total NSS, PPDS, and combined samples by confirming their unidimensional structures using exploratory and confirmatory factor analysis. Solutions were evaluated based on scree plot slopes,30 parallel analysis,31 and goodness of model fit (e.g., root mean squared error of approximation).32,33 Four of the 87 items did not have salient loadings onto their rationale scales and were thus excluded from scale generation. Each rational scale was otherwise determined to be unidimensional with all items loading .40≤ onto a single substantively meaningful factor. The names of the rationale scales and example items are presented in eTable 7. Given potential overlap among the rational scales (e.g., neuroticism and emotional reactivity) and our explicit interest in higher-order traits, exploratory factor analyses of the rational scale scores (standardized) were then conducted in an attempt to identify a smaller number of meaningful and reliable factors that accounted for shared variance among the 24 scales. Using the same model evaluation procedures described above, we identified six empirically-interpretable second-order factors (standardized scales) using 20 of the rationale scales: negative affectivity34,35; thoughtfulness36,37; fearlessness38,39; self-assertion/expansion (“beta”40,41); social/emotional independence42,43; and negative cognitions. 44,45 See eTable 8 for the factor loadings for the second-order scales.
 Social networks (64 variables) Several questions were adapted from the National Comorbidity Survey – Adolescent Supplement46,47 to assess social networks during adolescence, including popularity with peers and involvement in sports and school activities. We created nested dichotomous variables based on these questions (somewhat involved or very involved; very involved with peers) and also used the questions to define a composite (continuous) total peer involvement variable. Army STARRS also developed questions to assess size of affiliative network (e.g., number of people who the soldier had to spend time with, number of people the soldier felt close to, number of people the soldier felt cared for them, number of family or friends their could rely on during times of need). Nested dichotomous independent variables were defined based on these questions (e.g,. 1+ person, 2+ people). Questions were also adapted from the NCS-R11,48 to assess number of sexual partners in the year prior to enlistment. Nested dichotomous independent variables were defined using these questions (e.g., 1+ partners, 2+ partners)
 Self-harm (39 variables) Questions assessing lifetime history of suicidal and self-harm behaviors were adapted from the Columbia-Suicide Severity Rating Scale.14 Dichotomous (non-nested and nested) and categorical (e.g., 0=1–2 lifetime self-harm behaviors; 1=3–5 behaviors; 2=6–10 behaviors) variables were created to operationalize lifetime presence-frequency of (i) suicidal ideation, plans, and attempts (presence of any of the three; age at onset; number of days with ideation during worst week of suicidality; duration of the days with ideation during worst week; difficultly controlling ideation during worst week), (ii) dangerous activities because of suicidality (e.g., reckless driving), and (iii) non-suicidal self-injury (e.g., cutting or burning oneself; age at onset; number of lifetime self-harm behaviors).
II. Neurocognitive (8 variables) Seven neurocognitive tests were used to assess seven neurocognitive constructs of interest: mental flexibility, attention, working memory, impulse control, facial memory, emotion identification, and bias toward negative emotions. See eTable 9 for detailed descriptions of the seven neurocognitive test that were used. Each test was scored based on two dimensions, accuracy and speed, which were averaged to define continuous “efficiency” scores. The seven standardized efficiency scores were included among the independent variables. We also included a general (composite) efficiency score variable based on results from a previously reported bifactor confirmatory factor model in the New Soldier Study.50
III. Administrative (37 variables) Administrative data available for all soldiers at the time of accession were used to operationalize a select number of military-specific independent variables, including: Armed Forces Qualifications Test score (i.e., used by the Army to determine intelligence-aptitude of enlistees); Physical Profile scores (i.e., PULHES: physical capacity, upper extremities, lower extremities, hearing, eyes/vision, psychiatric); having a medical failure at accession; enlistment waiver at accession; positive drug test at accession; and enlisted military occupational specialty (MOS). AFQT was categorically (i.e., 0=0–42nd percentile; 1=43–56th percentile; 2=57–74th percentile; 3=75th–100th percentile) and continuously coded, all other variables were non-nested dichotomous variables. MOS was defined both using three broad occupational classes (combat arms, combat support, combat service support) as well 21 specific classes (e.g., infantry, cannon crewmember, cavalry scout, combat engineer, other “direct” combat arms, see elsewhere for additional details of this MOS classification scheme49).
Open in a new tab
1

The entire self-administered questionnaire can be accessed online at http://starrs-ls.org/#/page/instruments

The SAQ variables were in 6 categories that included: socio-demographics (e.g., age; sex; race-ethnicity), self-reported lifetime history of DSM-IV mental disorders (ADHD, bipolar disorder, conduct disorder, generalized anxiety disorder, major depressive disorder, oppositional-defiant disorder panic disorder, PTSD, substance abuse-dependence), exposure to stressors (childhood adversities, other lifetime traumatic stressors; past-year stressful life events and difficulties), personality (e.g., neuroticism; impulsivity; secure attachment), social networks (e.g., number of friends, number of sexual partners), and lifetime suicidality/non-suicidal self-injury (referred to henceforth as “self-harm”).

The neurocognitive variables, described in more detail elsewhere (Moore et al. in press), assessed seven constructs: mental flexibility, attention, working memory, impulse control, facial memory, emotion identification, and bias toward negative emotions. A standardized efficiency score (the average of test accuracy and speed) was defined for each neurocognitive construct along with a composite overall efficiency score across the seven constructs. The administrative variables included Armed Forces Qualification Test (AFQT) scores, physical profile system (PULHES) scores, enlistment military occupational specialty classifications, and a series of indicators of enlistment waivers.

Analysis methods

Analysis was carried out remotely by Harvard Medical School analysts on the secure University of Michigan Army STARRS Data Coordination Center server. Given that respondents differed in number of months of follow-up, we calculated survival curves for each outcome based on observed outcome distributions to estimate number of respondents who would have each outcome if they were all followed 24 months after enlistment (24-month morbid risk) using the actuarial method (Halli & Rao, 1992) implemented in SAS Proc LIFETEST (SAS Institute Inc., 2010). We projected morbid risk to 24 months despite our data going to 33 months because the number of soldiers followed beyond 24 months was too small for projection.

Discrete-time survival analysis with person-month the unit of analysis and a logistic link function (Willett & Singer, 1993) was used to develop a separate prediction model for first occurrence of each outcome. As noted above, our goal was to maximize classification rather than to test hypotheses about specific predictors, leading us to consider all potential predictors in the models. The major danger in doing this was over-fitting (Ritchie, 2005; Upstill-Goddard et al. 2013). We addressed this problem by using the elastic net penalized regression method (Zou & Hastie, 2005) implemented in the R-package glmnet (Friedman et al. 2010) to select an optimal subset of predictors for each final model. Penalized regression methods are designed either to use shrinkage to include multiple highly correlated predictors in a single model, to select the most stable single predictor in each highly correlated set to represent all predictors in the set, or to use some combination of both approaches in the service of maximizing out-of-sample classification accuracy at the expense of coefficient accuracy. Given that performance-based neurocognitive test data collection was time-consuming (20 minute administration time) and required special software, elastic net was implemented both with and without the neurocognitive measures to evaluate their incremental importance.

Prior to using elastic net, univariate associations of each potential predictor were estimated with each outcome controlling BCT site and time of data collection in SAS proc logistic (SAS Institute Inc., 2010). Functional forms of significant non-dichotomous predictors were transformed to capture simple nonlinearities. The elastic net analysis was then limited to significant univariate predictors. Once final elastic net models were estimated, individual-level predicted probabilities were calculated for each outcome and area under the receiver operating characteristic curve (AUC) was generated from these predicted probabilities to evaluate overall model fit. The sample was then divided into 20 groups of equal size (ventiles) for each outcome based on predicted probabilities of the outcome. When concentration of risk (COR; the observed proportion of realizations of a given outcome in a given ventile) was at least 15% among soldiers in the top-risk ventile (i.e., three times the expected value), we examined model coefficients and inspected COR across all 20 ventiles of predicted risk of that outcome.

RESULTS

Morbid risk and correlations among outcomes

The most common outcome was attrition, with a 24-month morbid risk of 189.8/1,000 soldiers. (Table 1) Demotion had the next highest morbid risk (60.8/1,000 soldiers), followed by sexual assault victimization (37.7/1,000 female soldiers), mental hospitalization (34.0/1,000 soldiers), severe injury (29.9/1,000 soldiers), positive drug test (18.9/1,000 soldiers), and minor violence perpetration (11.2/1,000 soldiers). All other outcomes had 24-month morbid risk less than 10.0/1,000 soldiers. Several outcomes were strongly inter-correlated (Table 2), with tetrachoric correlations of .84 for suicide attempt with mental hospitalization, .48-.76 for major physical perpetration with the other two perpetration outcomes, .54 for minor violence perpetration with minor violence victimization, and .55-.70 for positive drug test with attrition and demotion.

Table 1.

24-month morbid risk per 1,000 soldiers and incidence per 1,000 person-years of adverse outcomes in the New Soldiers Study (n=21,832)1

24-month morbid risk/1,000 soldiers2
Incidence/1,000 person-years3
SE
(n)
I. Mental-physical disorders
 Suicide attempt 8.2 5.1 0.4 (169)
 Mental hospitalization 34.0 21.8 0.8 (739)
 Positive drug test 18.9 11.9 0.7 (407)
 Traumatic brain injury 2.6 1.8 0.3 (62)
 Other severe injury 29.9 19.1 0.9 (658)
II. Violence
 Major physical perpetration (Men) 3.8 2.5 0.3 (71)
 Minor violence perpetration 11.2 7.0 0.5 (234)
 Sexual assault perpetration (Men) 5.2 3.1 0.4 (88)
 Minor violence victimization 7.3 4.6 0.4 (162)
 Sexual assault victimization (Women) 37.7 25.1 2.6 (118)
III. Army career
 Attrition 189.8 119.0 2.9 (4,285)
 Demotion 60.8 39.2 1.2 (1,337)
Open in a new tab

Abbreviations: SE, standard error; n, number of observed cases of the outcome in the sample.

1

Although the same sample of new soldiers was used for all outcomes, the number of person-months varied across outcomes because we predicted first occurrence of each outcome and each sample was censored separately either after the month when the outcome first occurred, termination of Regular Army service, or December 2013, whichever came first. The range of person-months was between 52,842 (to predict sexual assault victimization among women) and 420,706 (to predict attrition).

2

Morbid risk was estimated as the number of new soldiers predicted to have each outcome within 24 months of the beginning of service based on an actuarial projection (Halli & Rao, 1992). The projection was made to 24 months even though we had data for up to 33 months for some respondents because the number of respondents became too small for estimation beyond 24 months.

3

Incidence was estimated from the person-month file as the observed proportion of person-months with a realization of the outcome multiplied by 12,000 (month-to-year conversion x 1,000). There is no necessary relationship between incidence and morbid risk, as the former is based on a dataset that extends for between 13 and 33 months and depends on the timing of occurrences, whereas the former is projected only to 24 months and would have the same value whether outcomes occurred early or late in that time period.

Table 2.

Tetrachoric correlation matrix for all 12 observed outcomes in the total sample (n=21,832)1

a B C d E F g h i J k L
I. Violence
 a. Major physical perpetration (Men) --
 b. Minor violence perpetration 0.76 --
 c. Sexual assault perpetration (Men) 0.48 0.26 --
 d. Minor violence victimization 0.41 0.54 0.15 --
 e. Sexual assault victimization (Women) --2 −0.15 --2 0.48 --
II. Mental-physical health
 f. Traumatic brain injury 0.08 −0.05 0.06 0.07 0.06 --
 g. Mental hospitalization 0.29 0.27 0.15 0.20 0.18 0.24 --
 h. Suicide attempt 0.27 0.23 0.17 0.15 0.21 0.10 0.84 --
 i. Positive drug test 0.36 0.18 0.16 0.17 0.23 0.16 0.31 0.22 --
 j. Severe injury −0.01 0.00 0.08 −0.14 −0.01 0.18 0.18 0.17 −0.05 --
III. Army career
 k. Attrition 0.21 0.16 0.15 −0.10 0.24 −0.09 0.40 0.42 0.55 −0.07 --
 l. Demotion 0.34 0.40 0.33 0.20 0.22 0.17 0.29 0.23 0.70 0.07 0.39 --
Open in a new tab
1

Correlations are estimated at the person-level ignoring duration of the follow-up period.

2

These correlations could not be estimated because these involve opposite-sex sex-specific outcomes

Model accuracy

The number of predictors selected by elastic net was 3–29 (Median=14) across outcomes. AUC for the models ranged between .62 (mental hospitalization, severe injury) and .80 (TBI). (Table 3) Focusing on the nine outcomes with top-ventile COR above the minimum pre-specified level of 15%, 32.1–38.2% of new soldiers with three outcomes (major physical and sexual assault perpetration, TBI) and 21.6–29.8% of those with four other outcomes (minor violence perpetration, sexual assault victimization, suicide attempt, positive drug test) were among the 5% in the highest predicted risk ventiles for those outcomes. (Figure 1) 40.5–46.5% of new soldiers with each of four outcomes (major physical and sexual assault perpetration, suicide attempt, TBI) were among the 10% in the two highest ventiles for those outcomes. And 55.8–57.2% of new soldiers with two outcomes (sexual assault perpetration, TBI) were among the 15% in the three highest ventiles for those outcomes. Morbid risk among new soldiers in the highest-risk ventile ranged widely due to the variations in overall morbid risk and COR, from a high of 501.1/1,000 soldiers for attrition to a low of 19.9/1,000 for TBI.

Table 3.

Performance of the final discrete-time survival model for each outcome1

AUC Concentration of risk in ventiles predicted to have highest risk2 24-month morbid risk/1,000 soldiers in ventiles predicted to have highest risk3
Top 5% Top 10% Top 15% Top 5% Top 10% Top 15%
I. Mental-physical disorders
 Suicide attempt 0.74 29.8 40.5 48.9 48.9 33.2 26.7
 Mental hospitalization 0.62 15.2 23.8 30.0 103.4 80.9 68.0
 Positive drug test 0.71 21.6 29.7 41.4 81.6 56.1 52.2
 Traumatic brain injury 0.80 38.2 46.5 55.8 19.9 12.1 9.7
 Other severe injury 0.62 11.2 34.4 34.4 67.0 102.9 68.6
II. Violence
 Major physical perpetration (Men) 0.78 34.0 45.8 47.7 25.8 17.4 12.1
 Minor violence perpetration 0.76 24.0 35.9 45.8 53.8 40.2 34.3
 Sexual assault perpetration (Men) 0.78 32.1 44.6 57.2 33.4 23.2 19.8
 Minor violence victimization 0.68 16.5 26.0 38.1 24.1 19.0 18.5
 Sexual assault victimization (Women) 0.71 23.1 35.5 40.3 174.2 133.8 101.3
III. Army career
 Attrition 0.65 13.2 21.6 28.4 501.1 410.0 359.4
 Demotion 0.65 11.2 19.8 27.6 136.2 120.4 111.9
Open in a new tab

Abbreviations: AUC, area under the receiver operating characteristic curve.

1

Each ventile represents 5% of the soldiers in the sample ranked in terms of their predicted risk of each outcome

2

Concentration of risk refers to the percent of all observed occurrences of the outcome in a ventile or ventiles of the predicted risk distribution.

3

Morbid risk was defined as the number of new soldiers within a ventile of predicted risk subsequently predicted to have each outcome within 24 months of the beginning of service based on the actuarial method. The projection was made to 24 months even though we had data for up to 33 months for some respondents because the number of respondents became too small for estimation beyond 24 months.

Figure 1. Proportion of outcome obsevervations within each ventile of predicted risk derived from final models1.

Figure 1

Open in a new tab

1Ventiles are 20 groups created by dividing the total sample into equally sized groups defined by rank order of predicted risk from the final models. Only the nine final models that had a concentration of risk in the top-ventile of risk at least three times the expected value are presented here.

Model predictors

As noted above in the section on analysis methods, penalized regression maximizes classification accuracy at the expense of coefficient accuracy, making it important to focus more on model performance than on the specific predictors that entered the models. It is notetheless noteworthy that 40.7% of the 772 potential predictors had significant (.05-level, 2-sides test) univariate associations with mental hospitalization, followed by 25.9–26.2% with suicide attempt and positive drug test, 16.7–17.9% with minor violence perpetration and sexual assault victimization, and no more than chance (4.8–6.3%) with the other outcomes. (Table 4) As the potential predictors were unequally distributed across classes, we focused on observed/expected (O/E) predictor ratios in final models. Personality measures were proportionally over-represented in the final models for seven outcomes, socio-demographics for six, stressors and administrative variables for three, mental disorders for two, and the other classes of predictors for 0–1 outcomes. (Odds ratios and 95% confidence intervals of predictors in final models are available on request.)

Table 4.

Proportion of significant univariate predictors and final-model predictors across the eight predictor categories1

Outcome Socio-demographic Mental disorders Stressors Personality Social Networks Self-harm Neurocognitive Administrative Total
%U2 O/E3 %U2 O/E3 %U2 O/E3 %U2 O/E3 %U2 O/E3 %U2 O/E3 %U2 O/E3 %U2 O/E3 %4 (n)5
I. Mental-or physical disorders
 Suicide attempt 10.5 1.2 34.3 1.2 18.9 0.6 58.2 1.8 29.7 0.5 15.4 1.7 62.5 -- 10.8 0.9 25.9 (23)
 Mental hospitalization 13.2 -- 61.4 0.9 33.3 1.0 70.9 2.6 42.2 -- 28.2 5.4 25.0 -- 8.1 -- 40.7 (11)
 Positive drug test 21.1 1.9 38.2 0.9 21.6 0.6 29.1 1.3 28.1 2.2 5.1 -- -- -- 13.5 -- 26.2 (22)
 Traumatic brain injury 5.3 0.9 3.9 0.4 12.6 1.6 3.6 0.9 1.6 -- -- -- 25.0 12.9 2.7 1.4 6.3 (15)
II. Violence
 Major physical perpetration (Men) 7.0 0.7 3.4 1.0 5.0 0.7 3.6 1.4 3.1 -- -- -- 50.0 -- 13.5 6.2 5.2 (10)
 Minor violence perpetration 7.0 1.1 26.2 2.0 10.4 0.2 18.2 0.8 23.4 0.7 2.6 -- 87.5 -- 10.8 1.2 16.7 (18)
 Sexual assault perpetration (Men) 7.0 2.1 1.3 0.3 5.0 0.8 14.5 3.2 3.1 0.9 2.6 1.5 37.5 -- 2.7 -- 4.8 (13)
 Minor violence victimization 1.8 1.2 5.6 0.9 5.4 1.3 10.9 2.6 4.7 -- 2.6 -- 25.0 -- -- -- 5.1 (11)
 Sexual assault victimization (Women) 7.9 1.1 16.7 0.6 26.6 2.0 49.1 1.2 1.6 -- 7.7 -- -- -- -- -- 17.9 (12)
% of all potential predictors 14.8 30.2 28.8 7.1 8.3 5.1 1.0 4.8
(n) potential predictors (114) (233) (222) (55) (64) (39) (8) (37) (772)
Open in a new tab

Abbreviations: Neurocog, neurocognitive; Admin, adminstrative Army career variables; U, univariate; O/E, observed number of final model predictors divided by expected number of final model predictors; FM, final model; P, proportion; N, number.

1

Results are shown for the nine outcomes that achieved a concentration of risk in the top-ventile of risk at least three times the expected value (i.e., 15.0%≤; see Table 3). Empty cells (--) indicate predictor categories that had no significant univariate associations or predictor categories that were not represented in a final model. Coefficients (odds ratios) for the final model predictors are available on request..

2

These columns report the percent of predictors within each predictor category with significant univariate associations with outcome (0.05 level, two-sided tests).

3

This ratio refers to the observed representation of the column predictor category divided by the expected representation of the column predictor category. For example, there was 1 socio-demographic variable selected among the 10 total final model variables for major physical perpetration - 10% representation. However 114 of all predictors were socio-demographic variables. – 14.8%. This ratio (10.0/14.8) is equal to 0.7.

4

This column reports the total proportion of all predictors with significant univariate associations with each outcome.

5

This column reports the total number of final model predictors selected by elastic net.

Several predictors selected by elastic net emerged in two or more final models. The common socio-demographics associated with elevated risk included minority status (predicting all violence perpetration outcomes, TBI, positive drug test), female sex (minor violence victimization; suicide attempt), low education (minor violence perpetration, positive drug test), and high religiosity (sexual assault perpetration, positive drug test). The most important mental disorder predictors associated with increased outcome risk were anger attacks (predicting major physical and minor violence perpetration, positive drug test), substance use disorders (minor violence perpetration, mental hospitalization, positive drug test), insomnia (minor violence perpetration and victimization, positive drug test), childhood behavioral disorders (minor violence perpetration, sexual assault perpetration and victimization, TBI, positive drug test), anxiety disorders (sexual assault perpetration, TBI, mental hospitalization, suicide attempt), total number of lifetime disorders (major physical and minor violence perpetration and victimization, mental hospitalization, suicide attempt), and lifetime treatment of mental disorders (major physical perpetration, suicide attempt).

The most important stressors included various chronic strains that occurred in the year prior to enlistment (predicting major physical perpetration, minor violence victimization, TBI), childhood physical abuse and physical assault victimization (sexual assault perpetration, TBI, positive drug test), family history of mental illness (sexual assault perpetration and victimization, TBI, mental hospitalization, suicide attempt), living in a foster home as a child (minor violence and sexual assault victimization), and number of lifetime stressors (all mental-physical disorder outcomes). Although a broad range of personality variables entered the different final models, only two were included in 2+ models: extroverted personality (predicting elevated risk in all three perpetration models); and non-secure attachment styles (predicting elevated risk of sexual assault perpetration, minor violence victimization, TBI, mental hospitalization, suicide attempt).

The remaining four groups of predictors were much less prominent in the final models. The only consistent predictor involving social networks was information about number of sexual partners (predicting elevated risk of minor violence perpetration, sexual assault perpetration, and positive drug tests). Indicators of self-harm were associated with elevated risk of sexual assault perpetration, mental hospitalization, and suicide attempt. The only administrative variable entering the models for multiple outcomes was the enlistment AFQT score, which inversely predicted risk of major physical and minor violence perpetration, TBI, and suicide attempts. Only one neurocognitive measure entered any final model: composite test efficiency predicted increased risk of TBI and improved top-ventile COR by 8.0% (from 38.2% to 46.2%).

Cross-outcome risk

Not surprisingly given that some outcomes were significantly inter-correlated, some composite predicted risk scores based on the final models were also significantly inter-correlated (detailed results available on request), raising a question whether predicted risk scores of Outcome A ever incrementally predicted Outcome B over the predict risk score for Outcome B. We explored this possibility by estimating models where each outcome was regressed on the predicted risk score for that outcome in addition to the predicted risk scores for the other outcomes. Only a handful of cross-outcome predictions were statistically significant and none improved COR in high-risk ventiles (detailed results available on request).

DISCUSSION

We found that self-report questionnaire data collected at the beginning of service can be used to develop risk models with high top-ventile concentrations of risk for a number of subsequent negative soldier outcomes in the early years of service. Of particular note: 40.5% of suicide attempts occurred among the 10% of new soldiers with highest predicted risk of that outcome, 34.0% of male major physical perpetrators were among the 5% with highest predicted risk of that outcome, 57.2% of male sexual assault perpetrators were among the 15% with highest predicted risk of that outcome, and 35.5% of female sexual assault victims were among the 10% with highest predicted risk of that outcome.

Caution is needed in interpreting the importance of the specific predictors in our final models because penalized regression methods maximize overall model performance at the expense of individual coefficient accuracy. Nonetheless, three observations are noteworthy about these predictors.

First, the vast majority of the signs of the associations between predictors and outcomes are consistent with those in prior studies of military (Suris & Lind, 2008; Elbogen et al. 2010; Turchik & Wilson, 2010; Nock et al. 2013; Afifi et al. 2016) and civilians (Dahlberg, 1998; Kilpatrick et al. 2000; Kirst et al. 2014; Rytila-Manninen et al. 2014). The major exception is as that religiosity is positively associated with risk of sexual assault perpetration and positive drug tests. Religiosity is usually (Miller et al. 2000; Nonnemaker et al. 2003; Salas-Wright et al. 2012) found to be protective against these outcomes. However, at least three previous studies found religiosity to be associated with increased risk of similar outcomes (Jeffords, 1984; Schensul & Burkholder, 2005; Herman-Stahl et al. 2007). An understanding of why this might be the case would require more focused investigation.

Second, personality is the only predictor category with an O/E ratio consistently greater than 1.0 across outcomes. Few previous studies have examined personality predictors of negative soldier outcomes (MacManus et al. 2012a; MacManus et al. 2012b). Given our results, though, additional research might be warranted to compare predictive validity of the personality measures in the NSS with personality measures collected independently by the Army (Drasgow et al. 2012) as pre-accession screens for negative outcomes (Niebuhr et al. 2013).

Third, despite some evidence that objective measures of psychological characteristics have incremental validity over self-reports (Fuentes et al. 2006; Back et al. 2009; Huntjens et al. 2014), the performance-based neurocognitive test variables considered here did not figure prominently in our final models despite having high proportions of significant univariate associations. This raises a question about the value of including these tests in future surveys of new soldiers.

Three study limitations are noteworthy. First, the NSS was described to new soldiers as an independent survey in which individual-level responses would not be shared with Army leaders. If model results of the sort reported here are used to target preventive interventions in future cohorts of new soldiers, results would have to be shared with Army leaders and respondents would have to be made aware of this fact before participating. This shift in auspices might alter reports in ways that reduce model performance. The STARRS results are nonetheless valuable in showing that respondent self-reports unencumbered by concerns about disclosure are capable of predicting the outcomes considered here with good accuracy. Based on these results, future studies might experiment with different approaches to motivate honest reporting in the context of respondents being told that results will be used to select soldiers for preventive interventions.

Second, our model results might not generalize beyond the 13–33 months of follow-up considered here and perhaps not beyond the 24 months for which we have a substantial sample. Replication of the current analyses over longer follow-up periods is needed to investigate this issue. Importantly, administrative data become richer over time, making it of interest in future long-term studies to examine the joint associations of baseline self-report measures and ongoing administrative measures in predicting the outcomes considered here.

Third, our use of administratively-recorded outcomes means that we excluded outcomes not reported to authorities (e.g., unreported sexual assault victimizations) and not detected by authorities (e.g., crime perpetrators who eluded authorities). Concern about this limitation is dampened, though, by two considerations. One is that administratively-recorded cases are often more severe than unrecorded cases. This is true, for example, of suicide attempts, where 100% of the severe attempts (i.e., those requiring hospitalization) need medical attention and are recorded in administrative records, even though some unknown number of less severe attempts are not known to authorities. The other is that administratively-recorded prevalence of these outcomes is high enough that prevention only of those cases would be of considerable value. Such interventions would also be expected to prevent some unreported cases, leading to conservative estimates of intervention cost-effectiveness based on administrative outcomes alone.

CONCLUSIONS

Within the context of these limitations, our results show that small subsets of new soldiers are responsible for high proportions of many of the negative outcomes considered here and that these high-risk new soldiers could be pinpointed at the beginning of their Army careers with models based on self-report data and basic administrative variables. These results argue strongly for the potential value of using self-report surveys with new soldiers to target preventive interventions if the issue of confidentiality could be addressed successfully. Of course, the ultimate value of prediction models based on such surveys depends on the broadly-defined costs (both direct costs and competing risks) of these outcomes to the Army and the individual soldier and the effectiveness of such interventions in terms of NNT (i.e., the number of high-risk new soldiers who would need to be treated to prevent one instance of a focal outcome). However, as noted in the introduction, a number of interventions exist that are very promising both in terms of costs and effectiveness when considered for targeted implementation, making the results reported here of considerable value.

Acknowledgments

Financial Support: Army STARRS was sponsored by the Department of the Army and funded under cooperative agreement (R.J.U. & M.B.S., cooperative agreement number U01MH087981) (2009–2015) with the U.S. Department of Health and Human Services, National Institutes of Health, National Institute of Mental Health (NIH/NIMH); Subsequently, STARRS-LS was sponsored and funded by the Department of Defense (R.J.U. & M.B.S., USUHS grant number HU0001-15-2-0004); This research was also funded by the Department of Defense, Office of the Assistant Secretary for Defense for Health Affairs, Defense Health Program (OASD/HA), awarded and administered by the U.S. Army Medical Research & Materiel Command (USAMRMC), at Fort Detrick, MD, (R.C.K., under contract numbers W81XWH-12-2-0113, W81XWH-13-1-0037); and Dr. Street’s effort was supported with resources and the use of facilities at VA Boston Healthcare System, Boston, MA.

Footnotes

Ethical Standards:

The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.

Group Information: The Army STARRS Team consists of Co-Principal Investigators: Robert J. Ursano, MD (Uniformed Services University of the Health Sciences) and Murray B. Stein, MD, MPH (University of California San Diego and VA San Diego Healthcare System). Site Principal Investigators: Steven Heeringa, PhD (University of Michigan) and Ronald C. Kessler, PhD (Harvard Medical School). National Institute of Mental Health (NIMH) collaborating scientists: Lisa J. Colpe, PhD, MPH and Michael Schoenbaum, PhD. Army liaisons/consultants: COL Steven Cersovsky, MD, MPH (USAPHC (Provisional)) and Kenneth Cox, MD, MPH (USAPHC (Provisional)). Other team members: Pablo A. Aliaga, MA (Uniformed Services University of the Health Sciences); COL David M. Benedek, MD (Uniformed Services University of the Health Sciences); K. Nikki Benevides, MA (Uniformed Services University of the Health Sciences); Paul D. Bliese, PhD (University of South Carolina); Susan Borja, PhD (NIMH); Evelyn J. Bromet, PhD (Stony Brook University School of Medicine); Gregory G. Brown, PhD (University of California San Diego); Laura Campbell-Sills, PhD (University of California San Diego); Catherine L. Dempsey, PhD, MPH (Uniformed Services University of the Health Sciences); Carol S. Fullerton, PhD (Uniformed Services University of the Health Sciences); Nancy Gebler, MA (University of Michigan); Robert K. Gifford, PhD (Uniformed Services University of the Health Sciences); Stephen E. Gilman, ScD (Harvard School of Public Health); Marjan G. Holloway, PhD (Uniformed Services University of the Health Sciences); Paul E. Hurwitz, MPH (Uniformed Services University of the Health Sciences); Sonia Jain, PhD (University of California San Diego); Tzu-Cheg Kao, PhD (Uniformed Services University of the Health Sciences); Karestan C. Koenen, PhD (Columbia University); Lisa Lewandowski- Romps, PhD (University of Michigan); Holly Herberman Mash, PhD (Uniformed Services University of the Health Sciences); James E. McCarroll, PhD, MPH (Uniformed Services University of the Health Sciences); James A. Naifeh, PhD (Uniformed Services University of the Health Sciences); Tsz Hin Hinz Ng, MPH (Uniformed Services University of the Health Sciences); Matthew K. Nock, PhD (Harvard University); Rema Raman, PhD (University of California San Diego); Holly J. Ramsawh, PhD (Uniformed Services University of the Health Sciences); Anthony Joseph Rosellini, PhD (Harvard Medical School); Nancy A. Sampson, BA (Harvard Medical School); CDR Patcho Santiago, MD, MPH (Uniformed Services University of the Health Sciences); Michaelle Scanlon, MBA (NIMH); Jordan W. Smoller, MD, ScD (Harvard Medical School); Amy Street, PhD (Boston University School of Medicine); Michael L. Thomas, PhD (University of California San Diego); Leming Wang, MS (Uniformed Services University of the Health Sciences); Christina L. Wassel, PhD (University of Vermont); Simon Wessely, FMedSci (King’s College London); Christina L. Wryter, BA (Uniformed Services University of the Health Sciences); Hongyan Wu, MPH (Uniformed Services University of the Health Sciences); LTC Gary H. Wynn, MD (Uniformed Services University of the Health Sciences); and Alan M. Zaslavsky, PhD (Harvard Medical School).

Disclaimer: The contents are solely the responsibility of the authors and do not necessarily represent the views of the Department of Health and Human Services, NIMH, or the Department of the Army, or the Department of Defense. No official endorsement should be made.

Conflict of Interest: Dr. Stein reports being paid as a consultant by Actelion Pharmaceuticals, Dart NeuroScience, Janssen Pharmaceuticals, Resilience Therapeutics, and Oxeia Biopharmaceuticals; and for editorial work for UpToDate and the journal Biological Psychiatry. Dr. Monahan is co-owner of Classification of Violence Risk (COVR), Inc. In the past 3 years, Dr. Kessler received support for his epidemiological studies from Sanofi Aventis; was a consultant for Johnson & Johnson Wellness and Prevention, Shire, Takeda; and served on an advisory board for the Johnson & Johnson Services Inc. Lake Nona Life Project. Kessler is a co-owner of DataStat, Inc., a market research firm that carries out healthcare research. The remaining authors declare no conflict of interest.

References

  1. Afifi TO, Taillieu T, Zamorski MA, Turner S, Cheung K, Sareen J. Association of child abuse exposure with suicidal ideation, suicide plans, and suicide attempts in military personnel and the general population in Canada. JAMA Psychiatry. 2016;73:229–38. doi: 10.1001/jamapsychiatry.2015.2732. [DOI] [PubMed] [Google Scholar]
  2. Back MD, Schmukle SC, Egloff B. Predicting actual behavior from the explicit and implicit self-concept of personality. Journal of Personality and Social Psychology. 2009;97:533–48. doi: 10.1037/a0016229. [DOI] [PubMed] [Google Scholar]
  3. Booth-Kewley S, Highfill-McRoy RM, Larson GE, Garland CF. Psychosocial predictors of military misconduct. The Journal of Nervous and Mental Disease. 2010;198:91–8. doi: 10.1097/NMD.0b013e3181cc45e9. [DOI] [PubMed] [Google Scholar]
  4. Bulzacchelli MT, Sulsky SI, Rodriguez-Monguio R, Karlsson LH, Hill MO. Injury during U.S. Army basic combat training: a systematic review of risk factor studies. American Journal of Preventive Medicine. 2014;47:813–22. doi: 10.1016/j.amepre.2014.08.008. [DOI] [PubMed] [Google Scholar]
  5. Canham-Chervak M, Hooper TI, Brennan FH, Jr, Craig SC, Girasek DC, Schaefer RA, Barbour G, Yew KS, Jones BH. A systematic process to prioritize prevention activities sustaining progress toward the reduction of military injuries. American Journal of Preventive Medicine. 2010;38(1 Suppl):S11–8. doi: 10.1016/j.amepre.2009.10.003. [DOI] [PubMed] [Google Scholar]
  6. Cassidy JD, Carroll LJ, Peloso PM, Borg J, von Holst H, Holm L, Kraus J, Coronado VG. Incidence, risk factors and prevention of mild traumatic brain injury: results of the WHO Collaborating Centre Task Force on Mild Traumatic Brain Injury. Journal of Rehabilitation Medicine. 2004;36(43 Suppl):28–60. doi: 10.1080/16501960410023732. [DOI] [PubMed] [Google Scholar]
  7. Dahlberg LL. Youth violence in the United States. major trends, risk factors, and prevention approaches. American Journal of Preventive Medicine. 1998;14:259–72. doi: 10.1016/s0749-3797(98)00009-9. [DOI] [PubMed] [Google Scholar]
  8. Department of Defense. Department of Defense 2014–2016 Sexual Assault Prevention Strategy. Department of Defense; Arlington, Virginia: 2014a. [Accessed 1 February 2016]. ( http://sapr.mil/public/docs/prevention/DoD_SAPR_Prevention_Strategy_2014-2016.pdf) [Google Scholar]
  9. Department of Defense. DoD Workplace Violence Prevention and Response Policy. Department of Defense; Arlington, Virginia: 2014b. [Accessed February 1, 2016]. ( http://www.dtic.mil/whs/directives/corres/pdf/143806p.pdf) [Google Scholar]
  10. Department of the Army. AR-600-24 Health Promotion, Risk Reduction, and Suicide Prevention. Department of the Army; Washington, DC: 2015a. [Accessed 1 February 2016]. ( http://www.lewis-mcchord.army.mil/dhr/asap/Doc/14%20APR%2015%20p600_24.pdf) [Google Scholar]
  11. Department of the Army. AR-600-63 Army Health Promotion. Department of the Army; Washington, DC: 2015b. [Accessed 1 February 2016]. ( https://www.army.mil/e2/downloads/rv7/r2/policydocs/r600_63.pdf) [Google Scholar]
  12. Department of the US Army. Army Health Promotion, Risk Reduction, and Suicide Prevention: Report 2010. US Army; Washington, DC: 2010. [Google Scholar]
  13. Department of the US Army. Army 2020: Generating Health & Discipline in the Force ahead of the Strategic Reset. US Army; Washington, DC: 2012. [Google Scholar]
  14. Drasgow F, Stark S, Chernyshenko OS, Nye CD, Hulin CL, White LA. Technical Report 1311 - Development of the Tailored Adaptive Personality Assessment System (TAPAS) to Support Army Selection and Classification Decisions. U.S. Army Research Institute for the Behavioral and Social Sciences; Fort Belvoir, Virginia: 2012. [Accessed 1 February 2016]. ( http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA564422) [Google Scholar]
  15. Elbogen EB, Fuller S, Johnson SC, Brooks S, Kinneer P, Calhoun PS, Beckham JC. Improving risk assessment of violence among military veterans: an evidence-based approach for clinical decision-making. Clinical Psychology Review. 2010;30:595–607. doi: 10.1016/j.cpr.2010.03.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Elmasry H, Boivin MR, Feng X, Packnett ER, Cowan DN. Preenlistment and early service risk factors for traumatic brain injury in the Army and Marine Corps: FY 2002–2010. The Journal of Head Trauma Rehabilitation. 2017;32:E1–E7. doi: 10.1097/HTR.0000000000000229. [DOI] [PubMed] [Google Scholar]
  17. Foster EM, Jones D. Can a costly intervention be cost-effective?: an analysis of violence prevention. Archives of General Psychiatry. 2006;63:1284–91. doi: 10.1001/archpsyc.63.11.1284. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Friedman J, Hastie T, Tibshirani R. Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software. 2010;33:1–22. [PMC free article] [PubMed] [Google Scholar]
  19. Fuentes D, Tavares H, Artes R, Gorenstein C. Self-reported and neuropsychological measures of impulsivity in pathological gambling. Journal of the International Neuropsychological Society : JINS. 2006;12:907–12. doi: 10.1017/S1355617706061091. [DOI] [PubMed] [Google Scholar]
  20. Golubnitschaja O, Costigliola V. General report & recommendations in predictive, preventive and personalised medicine 2012: white paper of the European Association for Predictive, Preventive and Personalised Medicine. The EPMA Journal. 2012;3:14. doi: 10.1186/1878-5085-3-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Halli SS, Rao KV. Advanced Techniques of Population Analysis. Plenum Press; New York, NY: 1992. [Google Scholar]
  22. Herman-Stahl MA, Krebs CP, Kroutil LA, Heller DC. Risk and protective factors for methamphetamine use and nonmedical use of prescription stimulants among young adults aged 18 to 25. Addictive Behaviors. 2007;32:1003–1015. doi: 10.1016/j.addbeh.2006.07.010. [DOI] [PubMed] [Google Scholar]
  23. Huntjens RJ, Rijkeboer MM, Krakau A, de Jong PJ. Implicit versus explicit measures of self-concept of self-control and their differential predictive power for spontaneous trait-relevant behaviors. Journal of Behavior Therapy and Experimental Psychiatry. 2014;45:1–7. doi: 10.1016/j.jbtep.2013.07.001. [DOI] [PubMed] [Google Scholar]
  24. Institute of Medicine. Returning Home fom Iraq and Afghanistan: Preliminary Assessment of Readjustment Needs of Veterans, Service Members, and their Families. The National Academies Press; Washington, DC: 2010. [PubMed] [Google Scholar]
  25. Iribarren C, Sidney S, Jacobs DR, Jr, Weisner C. Hospitalization for suicide attempt and completed suicide: epidemiological features in a managed care population. Social Psychiatry and Psychiatric Epidemiology. 2000;35:288–96. doi: 10.1007/s001270050241. [DOI] [PubMed] [Google Scholar]
  26. Jeffords CR. The impact of sex-role and religious attitudes upon forced marital intercourse norms. Sex Roles. 1984;11:543–552. [Google Scholar]
  27. Kapp L. Recruiting and Retention: An Overview of FY2011 and FY2012 Results for Active and Reserve Component Enlisted Personnel. Congressional Research Service; Washington, D.C: 2013. [Accessed 1 February 2016]. ( https://www.fas.org/sgp/crs/natsec/RL32965.pdf) [Google Scholar]
  28. Kaufman KR, Brodine S, Shaffer R. Military training-related injuries: surveillance, research, and prevention. American Journal of Preventive Medicine. 2000;18:54–63. doi: 10.1016/s0749-3797(00)00114-8. [DOI] [PubMed] [Google Scholar]
  29. Kessler RC, Warner CH, Ivany C, Petukhova MV, Rose S, Bromet EJ, Brown M, 3rd, Cai T, Colpe LJ, Cox KL, Fullerton CS, Gilman SE, Gruber MJ, Heeringa SG, Lewandowski-Romps L, Li J, Millikan-Bell AM, Naifeh JA, Nock MK, Rosellini AJ, Sampson NA, Schoenbaum M, Stein MB, Wessely S, Zaslavsky AM, Ursano RJ. Predicting suicides after psychiatric hospitalization in US Army soldiers: the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS) JAMA Psychiatry. 2015;72:49–57. doi: 10.1001/jamapsychiatry.2014.1754. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Kilpatrick DG, Acierno R, Saunders B, Resnick HS, Best CL, Schnurr PP. Risk factors for adolescent substance abuse and dependence: data from a national sample. Journal of Consulting and Clinical Psychology. 2000;68:19–30. doi: 10.1037//0022-006x.68.1.19. [DOI] [PubMed] [Google Scholar]
  31. Kirst M, Mecredy G, Borland T, Chaiton M. Predictors of substance use among young adults transitioning away from high school: a narrative review. Substance Use & Misuse. 2014;49:1795–807. doi: 10.3109/10826084.2014.933240. [DOI] [PubMed] [Google Scholar]
  32. Knapik JJ, Jones BH, Hauret K, Darakjy S, Piskator E. A review of the literature on attrition from the military services: risk factors for attrition and strategies to reduce attrition: USACHPPM REPORT NO. 12-HF-01Q3A-04. U.S. Army Center for Health Promotion and Preventative Medicine; Aberdeen Proving Ground, MD: 2004. [Accessed 1 February 2016]. ( http://www.dtic.mil/dtic/tr/fulltext/u2/a427744.pdf) [Google Scholar]
  33. Kubisiak UC, Lentz E, Horgen KE, Bryant RH, Connell PW, Tuttle MD, Borman WC, Young MC, Morath R. Review of interventions for reducing enlisted attrition in the U.S. military: an update. United Sates Army Research Institute for the Behavioral and Social Sciences; Arlington, VA: 2009. [Accessed 1 February 2016]. ARI Research Note 2009–13. ( http://www.dtic.mil/dtic/tr/fulltext/u2/a508188.pdf) [Google Scholar]
  34. MacManus D, Dean K, Al Bakir M, Iversen AC, Hull L, Fahy T, Wessely S, Fear NT. Violent behaviour in U.K. military personnel returning home after deployment. Psychological Medicine. 2012a;42:1663–73. doi: 10.1017/S0033291711002327. [DOI] [PubMed] [Google Scholar]
  35. MacManus D, Dean K, Iversen AC, Hull L, Jones N, Fahy T, Wessely S, Fear NT. Impact of pre-enlistment antisocial behaviour on behavioural outcomes among U.K. military personnel. Social Psychiatry and Psychiatric Epidemiology. 2012b;47:1353–8. doi: 10.1007/s00127-011-0443-z. [DOI] [PubMed] [Google Scholar]
  36. Miller L, Davies M, Greenwald S. Religiosity and substance use and abuse among adolescents in the National Comorbidity Survey. Journal of the American Academy of Child & Adolescent Psychiatry. 2000;39:1190–1197. doi: 10.1097/00004583-200009000-00020. [DOI] [PubMed] [Google Scholar]
  37. Moore TM, Gur RC, Thomas ML, Brown GG, Nock MK, Savitt AP, Keilp JG, Heeringa S, Ursano RJ, Stein MB. Development, administration, and structural validity of a brief, computerized neurocognitive battery. Assessment. 2017 doi: 10.1177/1073191116689820. Published online 30 January 2017. doi: http://dx.doi.org/10.1177%2F1073191116689820. [DOI] [PMC free article] [PubMed]
  38. Niebuhr DW, Gubata ME, Oetting AA, Weber NS, Feng X, Cowan DN. Personality Assessment Questionnaire as a pre-accession screen for risk of mental disorders and early attrition in U. S. Army recruits. Psychological Services. 2013;10:378–85. doi: 10.1037/a0032783. [DOI] [PubMed] [Google Scholar]
  39. Nock MK, Deming CA, Fullerton CS, Gilman SE, Goldenberg M, Kessler RC, McCarroll JE, McLaughlin KA, Peterson C, Schoenbaum M, Stanley B, Ursano RJ. Suicide among soldiers: a review of psychosocial risk and protective factors. Psychiatry. 2013;76:97–125. doi: 10.1521/psyc.2013.76.2.97. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Nonnemaker J, McNeely C, Blum R. Public and private domains of religiosity and adolescent health risk behaviors: evidence from the National Longitudinal Study of Adolescent Health. Social Science & Medicine. 2003;57:2049–54. doi: 10.1016/s0277-9536(03)00096-0. [DOI] [PubMed] [Google Scholar]
  41. Parkkari J, Taanila H, Suni J, Mattila VM, Ohrankammen O, Vuorinen P, Kannus P, Pihlajamaki H. Neuromuscular training with injury prevention counselling to decrease the risk of acute musculoskeletal injury in young men during military service: a population-based, randomised study. BMC Medicine. 2011;9:35. doi: 10.1186/1741-7015-9-35. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Ritchie MD. Bioinformatics approaches for detecting gene-gene and gene-environment interactions in studies of human disease. Neurosurgical Focus. 2005;19:1–4. doi: 10.3171/foc.2005.19.4.3. [DOI] [PubMed] [Google Scholar]
  43. Rosellini AJ, Heeringa SG, Stein MB, Ursano RJ, Chiu WT, Colpe LJ, Fullerton CS, Gilman SE, Hwang I, Naifeh JA, Nock MK, Petukhova M, Sampson NA, Schoenbaum M, Zaslavsky AM, Kessler RC. Lifetime prevalence of DSM-IV mental disorders among new soldiers in the U.S. Army: results from the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS) Depression and Anxiety. 2015;32:13–24. doi: 10.1002/da.22316. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Rosellini AJ, Monahan J, Street AE, Heeringa SG, Hill ED, Petukhova M, Reis BY, Sampson NA, Bliese P, Schoenbaum M, Stein MB, Ursano RJ, Kessler RC. Predicting non-familial major physical violent crime perpetration in the US Army from administrative data. Psychological Medicine. 2016;46:303–16. doi: 10.1017/S0033291715001774. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Rudd MD, Bryan CJ, Wertenberger EG, Peterson AL, Young-McCaughan S, Mintz J, Williams SR, Arne KA, Breitbach J, Delano K, Wilkinson E, Bruce TO. Brief cognitive-behavioral therapy effects on post-treatment suicide attempts in a military sample: results of a randomized clinical trial with 2-year follow-up. The American Journal of Psychiatry. 2015;172:441–9. doi: 10.1176/appi.ajp.2014.14070843. [DOI] [PubMed] [Google Scholar]
  46. Rytila-Manninen M, Lindberg N, Haravuori H, Kettunen K, Marttunen M, Joukamaa M, Frojd S. Adverse childhood experiences as risk factors for serious mental disorders and inpatient hospitalization among adolescents. Child Abuse & Neglect. 2014;38:2021–32. doi: 10.1016/j.chiabu.2014.10.008. [DOI] [PubMed] [Google Scholar]
  47. Salas-Wright CP, Vaughn MG, Hodge DR, Perron BE. Religiosity profiles of American youth in relation to substance use, violence, and delinquency. Journal of Youth and Adolescence. 2012;41:1560–1575. doi: 10.1007/s10964-012-9761-z. [DOI] [PubMed] [Google Scholar]
  48. SAS Institute Inc. SAS/STATR Software. SAS Institute Inc; Cary, NC: 2010. [Google Scholar]
  49. Schensul JJ, Burkholder GJ. Vulnerability, social networks, sites, and selling as predictors of drug use among urban African American and Puerto Rican emerging adults. Journal of Drug Issues. 2005;35:379–408. [Google Scholar]
  50. Senn CY, Eliasziw M, Barata PC, Thurston WE, Newby-Clark IR, Radtke HL, Hobden KL. Efficacy of a sexual assault resistance program for university women. The New England Journal of Medicine. 2015;372:2326–35. doi: 10.1056/NEJMsa1411131. [DOI] [PubMed] [Google Scholar]
  51. Shea MT, Lambert J, Reddy MK. A randomized pilot study of anger treatment for Iraq and Afghanistan veterans. Behaviour Research and Therapy. 2013;51:607–13. doi: 10.1016/j.brat.2013.05.013. [DOI] [PubMed] [Google Scholar]
  52. Street AE, Rosellini AJ, Ursano RJ, Heeringa SG, Hill ED, Monahan J, Naifeh JA, Petukhova MV, Reis BY, Sampson NA, Bliese PD, Stein MB, Zaslavsky AM, Kessler RC. Developing a risk model to target high-risk preventive interventions for sexual assault victimization among female US Army soldiers. Clinical Psychological Science. 2016;4:939–956. doi: 10.1177/2167702616639532. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Suris A, Lind L. Military sexual trauma: a review of prevalence and associated health consequences in veterans. Trauma, Violence & Abuse. 2008;9:250–69. doi: 10.1177/1524838008324419. [DOI] [PubMed] [Google Scholar]
  54. Theodoroff SM, Lewis MS, Folmer RL, Henry JA, Carlson KF. Hearing impairment and tinnitus: prevalence, risk factors, and outcomes in US service members and veterans deployed to the Iraq and Afghanistan wars. Epidemiologic Reviews. 2015;37:71–85. doi: 10.1093/epirev/mxu005. [DOI] [PubMed] [Google Scholar]
  55. Turchik JA, Wilson SM. Sexual assault in the U.S. military: A review of the literature and recommendations for the future. Aggression and Violent Behavior. 2010;15:267–277. [Google Scholar]
  56. U.S. Department of Justice. National Corrections Reporting Program, 2009 (ICPSR 30799) National Achive of Criminal Justice Data; Ann Arbor, MI: 2011. [Accessed 1 February 2016]. ( http://www.icpsr.umich.edu/icpsrweb/NACJD/studies/30799?archive=NACJD&permit%5B0%5D=AVAILABLE&q=30799&x=0&y=0) [Google Scholar]
  57. Upstill-Goddard R, Eccles D, Fliege J, Collins A. Machine learning approaches for the discovery of gene-gene interactions in disease data. Briefings in Bioinformatics. 2013;14:251–60. doi: 10.1093/bib/bbs024. [DOI] [PubMed] [Google Scholar]
  58. Ursano RJ, Colpe LJ, Heeringa SG, Kessler RC, Schoenbaum M, Stein MB. The Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS) Psychiatry. 2014;77:107–19. doi: 10.1521/psyc.2014.77.2.107. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Vungkhanching M, Heinemann AW, Langley MJ, Ridgely M, Kramer KM. Feasibility of a skills-based substance abuse prevention program following traumatic brain injury. The Journal of Head Trauma Rehabilitation. 2007;22:167–76. doi: 10.1097/01.HTR.0000271117.19652.98. [DOI] [PubMed] [Google Scholar]
  60. Willett JB, Singer JD. Investigating onset, cessation, relapse, and recovery: why you should, and how you can, use discrete-time survival analysis to examine event occurrence. Journal of Consulting Clinical Psychology. 1993;61:952–965. doi: 10.1037//0022-006x.61.6.952. [DOI] [PubMed] [Google Scholar]
  61. Zou H, Hastie T. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B. 2005;67:301–320. [Google Scholar]

RESOURCES