Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Jun 1.
Published in final edited form as: Suicide Life Threat Behav. 2016 Jul 22;47(3):257–265. doi: 10.1111/sltb.12280

An Examination of Potential Misclassification of Army Suicides: Results from the Army Study to Assess Risk and Resilience in Servicemembers

Kenneth L Cox 1, Matthew K Nock 2, Quinn M Biggs 3, Jennifer Bornemann 3, CAPT Lisa Colpe 4, Catherine L Dempsey 3, Steven G Heeringa 5, James E McCarroll 3, Tsz Hin Ng 3, Michael Schoenbaum 4, Robert J Ursano 3, Bailey G Zhang 3, David M Benedek 3, On behalf of the Army STARRS Collaborators
PMCID: PMC5253335  NIHMSID: NIHMS792488  PMID: 27447096

Abstract

Debate continues about the accuracy of military suicide reporting due to concerns that some suicides may be classified as accidents to minimize stigma and ensure survivor benefits. We systematically reviewed records for 998 active duty Army deaths (510 suicides; 488 accident, homicide and undetermined deaths; 2005-2009) and, using research criteria, reclassified 8.2% of the non-suicide cases to definite suicide (1), suicide probable (4), or suicide possible (35). The reclassification rate to definite suicide was only 0.2% (1/488). This low rate suggests that flagrant misclassification of Army deaths is uncommon and surveillance reports likely reflect the “true” population of Army suicides.

Keywords: psychological autopsy, suicide, military

Introduction

Suicide is now the second-leading manner of death among US military servicemembers, exceeded only by accident deaths (including traffic fatalities) (Corr, 2014). The increase in military suicides has been especially prominent in the US Army over the past decade (Nock et al., 2013). Historically, the US Army had enjoyed a considerably lower suicide rate than that in the general US population, but in 2008 the US Army rate surpassed the US civilian rate, and continued to climb. Seeking ways to reverse this trend, the US Army funded the Army Study to Assess Risk and Resilience among Servicemembers (Army STARRS; www.armystarrs.org), a multi-component epidemiological and neurobiological study of risk and resilience factors for suicidality and its psychopathological correlates among Army personnel. The details of the Army STARRS are described elsewhere (Kessler et al., 2013; Ursano et al., 2014).

Past studies found suicides were underreported or misclassified in various administrative data sources. One study reviewed military deaths from 1998-1999 and found 21 percent more suicides than had been officially reported (Carr, Hoge, Gardner, & Potter, 2004). Accurate and complete identification of suicide cases is essential to generate more complete knowledge about suicidal behaviors, inform policy, and allow the Army to implement effective prevention and treatment programs.

This paper describes a component of Army STARRS that used criminal investigation reports as a proxy for a psychological autopsy. Psychological autopsies are one systematic method for determining and characterizing the psychological and contextual circumstances of a suicide. Information is obtained from a wide variety of sources including medical records and interviews with people who knew the decedent (Cavanagh, Carson, Sharpe, & Lawrie, 2003; Conner et al., 2011). Such detailed information makes it easier to determine whether the individual intended to die, and facilitates the correct classification of deaths.

This study was primarily descriptive in nature and did not test any a priori hypotheses. The main goals were to: (1) validate Army criminal investigation reports of soldier deaths classified as suicide, and (2) apply research criteria to soldier deaths originally classified as accident, undetermined, or homicide to see if any of these deaths might have been suicides. This manuscript reports on differences between the manners of death assigned by Army criminal investigators with those determined by researchers, describes characteristics of those deaths classified as definite suicides by researchers, and discusses any possible differential effects on the overall Army suicide rate.

Methods

Sample

The sample consisted of all closed, final US Army Criminal Investigation Command (USACIDC, hereafter CID) Reports of Investigation (ROIs) for soldier deaths in the period of 2005-2009 where the assigned manner of death was suicide, accident (including traffic fatalities), homicide, or undetermined. Cases classified as natural death were not included in the sample. Open cases, i.e., where a final determination had yet to be made, were not reviewed, although virtually all cases for this historical time period were closed at the time of the data abstraction. As this sub-study of Army STARRS consisted of a record review of deceased individuals, it was determined to be an exempt protocol by the Uniformed Services University (USU Institutional Review Board (IRB)) –the IRB of record for Army STARRS.

Data Source

CID special agents investigate most soldier deaths in order to determine whether there was any criminal element. Some deaths are outside of CID's jurisdiction, (e.g., Army National Guard or US Army Reserve soldiers who die while not on active duty). Infrequently, some deaths go uninvestigated due to resource shortfalls or other issues. Suspected suicide cases are initially classified as undetermined. A final determination of suicide is made only after taking into account all available evidence (interviews, documents, forensic lab results, autopsy, medical examiner or coroner opinion, etc.). Generally, the CID classification matches the medical opinion, i.e., the manner of death reported on the death certificate. When the results of a CID death investigation differ from the medical opinion, CID policy requires the matter be referred to a formal Manner of Death Review Board (MDRB) for resolution. MDRBs are uncommon, e.g., there were none in the period of December 10, 2013, through December 9, 2014 (Department of Defense, 2015). In the absence of a medical opinion, the special agent makes a best judgment determination based on all available information.

While not duplicating all aspects of a formal psychological autopsy, military investigating agents often interview a wide range of informants including close relatives, friends, co-workers, supervisors, First Sergeants, commanders, and eye witnesses. The content of these interviews, both questions and answers, is captured verbatim in a Report of Investigation (ROI). The ROIs often contain demographic, family, health, autopsy, toxicology, and other relevant life-event information, much of which is not available in any other single military administrative data system. ROIs, maintained at the CID Crime Records Center, formed the primary data source for this study.

Data Collection

A team of four study-trained researchers with at least a master's level education reviewed all available 2005-2009 ROIs (998) involving suicides (510), accident deaths including traffic fatalities (426), homicides (14), and undetermined cases (48). A few of the ROIs involved multiple deaths, e.g., a military vehicle rolled over into an irrigation canal and three occupants drowned. In these cases, the researchers abstracted personal information for only the most relevant fatality, e.g., the driver. Demographic and military history information not available from the ROI was obtained from the Defense Manpower Data Center and the Department of Defense Suicide Event Report (DoDSER) system. Informant disagreement in the CID case files was not uncommon, e.g., one individual reported a history of a suicide attempt while others did not. Given that individuals do not confide in everyone, the reviewers did not use a preponderance of evidence approach and instead recorded any positive response as well as the number and type of individuals who made the same report, e.g., spouse, friend, co-worker, etc.

Data were entered into Blaise® data collection software. A copy of the full data collection instrument is available at www.armystarrs.org. Abstracted information included demographics, military history (rank, occupation, duty status), death specifics (method used if suicide, autopsy findings, toxicology results), past suicidal behavior, emotional and physical problems, risk factors and life stressors in the 12 months preceding death, intervention and treatment history, and a modified version of the Suicide Intent Scale (SIS) (Beck, Beck, & Kovacs, 1975). The original SIS questions were designed to collect information that would help determine whether the survivor of a nonfatal suicide attempt had truly intended to die. Consequently, not all questions in the SIS were suitable for post-mortem data collection and the wording of the retained questions was revised. The modified SIS helped reviewers systematically organize information related to intent, but was not used in any quantitative fashion.

Case Classification

The reviewers piloted the data collection instrument using CID 2004 records. This allowed the reviewers to learn the organization of the physical records and improve the instrument. Additionally, the reviewers developed descriptive vignettes to help define two special classifications, suicide probable and suicide possible. Cases classified as suicide probable were those in which there was clear evidence of suicidal intent leading up to the soldier's death, but insufficient evidence to say with 100% certainty that it was a suicide (e.g., statements about life being over leading up to the subject approaching police officers while wielding a gun in an apparent “suicide by cop”). Cases classified as suicide possible were those in which there was some evidence of suicidal intent leading up to the death, but the evidence was equivocal (e.g., presence of depression and statements about wanting to be dead in the weeks before an overdose of oxycodone). Generally, the reviewers found it difficult to identify consistent and reproducible criteria to use for suicide probable cases, resulting in more reclassifications to suicide possible. The distinction between these two additional categories was necessarily subjective, so all suicide probable or possible cases were reviewed by at least two reviewers, as described below.

After reviewing all available material, the reviewer assessed whether there was evidence of intent to die, and then assigned a manner of death to the case (natural causes, suicide, suicide probable, suicide possible, homicide, accident, or undetermined). The most compelling component of the modified SIS that influenced the reviewers to classify a case as a definite suicide was the presence of a suicide note. Though there have been reports of faked suicide notes, (Ho, Yip, Chiu, & Halliday, 1998; Pestian, Nasrallah, Matykiewicz, Bennett, & Leenaars, 2010) all of the suicide notes in the CID case files appeared genuine.

The primary reviews were not blinded, i.e., the reviewers knew the CID classification. Due to the nature of the CID physical records, blinding would have been extremely cumbersome and expensive. To promote consistent abstraction of clinical information and case classification, a subset of cases underwent a second review by another data coder. This allowed the researchers to assess inter-rater reliability (IRR) among the four primary data coders. There were second reviews of a random selection of approximately 25% of suicide cases and 25% of accident cases. All (100%) of the ambiguous cases underwent secondary review. The ambiguous cases included all undetermined cases, and all cases involving reclassification of a CID non-suicide case to suicide, suicide probable, or suicide possible. Secondary reviews were not blinded. The second reviewer was aware of both CID's original classification and the decision of the primary reviewer. The secondary reviews consisted of a full review of the ROI, but data abstraction was limited to basic identifying information and six primary domains (prior suicidality, emotional and physical health problems, life events, treatment and intervention for physical or emotional problems, suicidal intent, and the assigned manner of death). These domains contained questions that often required the data coder to make a judgment, e.g., was there evidence of depression? The primary and secondary reviewers compared their results for a given case and resolved any differences as described in the next section. Each reviewer's summary of evidence related to intent was a key part of the reconciliation process. When the two reviewers could not agree, an uncommon situation, one of the other two data coders was asked to render an opinion. Majority rule determined the “best response” for individual item responses other than the manner of death. When STARRS coders did not agree on the manner of death, decision rules favored identifying the case as having been misclassified. For example, if the CID classification was “accident” and one STARRS coder determined the manner of death to be “accident” while a second STARRS coder identified the case as “suicide possible,” the case was reclassified by the research team as “suicide possible” (additional details about the decision rules are available upon request and at www.armystarrs.org).

Results

Classification of Manner of Death

Trained researchers reviewed 998 death investigations by the CID. Of 510 cases classified as suicide by the CID, the researchers assessed the evidence as less than conclusive of definite suicide in 23 cases. In 5 instances reviewers felt suicide was probable with the other 18 cases classified as suicide possible. Thus the down-classification rate was 4.5% (23/510). Of the 488 cases the CID determined to be non-suicides, reviewers reclassified 35 cases to possible suicide, 4 cases to probable suicide, and 1 case to definite suicide. The overall up-classification rate for non-suicide cases was 8.2% (40/488). This resulted in a total of 488 definite suicide cases (510-23+1), as classified by the reviewers.

The researchers reclassified 26 of the 426 CID accident cases to either suicide (n=1), or suicide possible (n=25). It was often difficult to determine intent in cases of drug overdoses, especially in the absence of a suicide note.

Out of 14 CID cases of homicide, the researchers reclassified four as suicide probable (n=2) or suicide possible (n=2). An additional case was reclassified as an accident. In several of these cases there was sufficient evidence to suggest the individual acted deliberately in an effort to provoke a lethal response from law enforcement personnel (i.e., “suicide by cop”). Whereas justifiable homicide was the correct classification from a legal standpoint, one could consider these cases as a form of suicide.

The 48 CID undetermined cases presented the greatest challenge. A medical examiner or coroner must classify the case as undetermined “when the information pointing to one manner of death is no more compelling than one or more other competing manners of death when all available information is considered.” (Centers for Disease Control, 2003). Neither they nor the CID special agents have the options of suicide probable or suicide possible. The researchers, unconstrained by these guidelines, reclassified 16 undetermined causes as follows: suicide probable (n=2), suicide possible (n=8); accident (n=6). Many of the undetermined cases involved overdoses. In the absence of a suicide note it was difficult to delineate between an accidental overdose and intentional self-harm by a known recreational drug user.

The overall kappa (κ) statistic (Landis & Koch, 1977) for the assigned manner of death was 0.95.

Temporal Patterns of the Classification of Manner of Death

Classification patterns can vary over time due to new guidelines, increased emphasis on compliance, more exacting quality control measures, closer scrutiny, or other causes. Although no strong time-varying trends emerged from this study, there was a modest increase in the down-classification of suicide to suicide probable or possible for 2005-2009. Similarly, the up-classification of undetermined cases to any type of suicide decreased modestly over time. There is one seemingly anomalous year (2008) where the researchers did not reclassify any accident or undetermined cases to any type of suicide.

Discussion

Suicides will likely remain underrepresented in administrative databases. This is not surprising given how difficult it can be to determine the manner of death. Past studies suggested deliberate misclassification occurred in order to ensure benefits to survivors, avoid the perception of dishonor, or for other reasons may have increased misclassification rates (Bohnert et al., 2013; Carr et al., 2004; Claassen et al., 2010; Eaton, Messer, Garvey Wilson, & Hoge, 2006; Palmer et al., 2014; Tøllefsen, Hem, & Ekeberg, 2012). There would seem to be less need now for such deliberate misclassification given various changes over the past decade in business practices (e.g., Servicemembers Group Life Insurance is paid to survivors regardless of the manner of death), societal views, and other factors.

This study found misclassification rates lower than those previously reported in 1998-1999 military populations (Carr et al., 2004) suggesting that misclassification of military suicides may have substantially diminished in recent years. However, the earlier study used data from the Armed Forces Medical Examiner System (AFMES) and DoD casualty reports, not criminal investigation data. Also, the earlier study included all service member deaths whereas this study involved only deaths of US Army soldiers.

Differences in the classification of cases should not be construed as errors by either CID or the involved medical examiners and coroners. The researchers used a unique classification system reflecting clinical and academic perspectives. CID and the medical examiners are bound by different rules and constraints. The option of down-classifying suicide cases to suicide probable or possible is not available to medical examiners or law enforcement professionals. The up-classification of other manners of death to possible or probable suicide is of interest as it helps evaluate the possibility of underestimating suicides due to existing laws, policies, and practices pertaining to civilian law enforcement, Army CID investigations, and medical examiner activities (civilian and military).

Although no strong time-varying trends emerged from this study, there was a slight increase in the down-classification of suicide to suicide probable or possible for 2005-2009. This could indicate that over time CID has become more likely to classify difficult or borderline cases as suicide. Similarly, the up-classification of undetermined cases to any type of suicide decreased slightly over time. This, too, could reflect an increased tendency to classify some of these difficult cases as suicide.

The primary strength of this study was the large number of suicides and other deaths available for review. Additionally, most CID case files often contained large amounts of detailed, relevant information. There were some challenges in using law enforcement records as a proxy for a psychological autopsy. Law enforcement interviews seek factual information to decide whether a crime has occurred and do not always produce an exhaustive list of personal risk or protective factors and precipitating stressors. Questions were more likely to address previous suicidal thoughts, the victim's intent and personal actions at the time of death, substance misuse, and access to mental health clinical services. Information about social support, resilience, coping strategies, or specific mental health diagnoses was less common. To make things more difficult, it was not uncommon for informants to contradict each other. Another limitation to consider is that the reviewers were not blinded to either CID's or the paired reviewer's classification of the case. Lastly, this study involved only US Army Soldiers and the results may not reflect patterns among other branches of the US Armed Forces or the general US civilian population.

Conclusions

This study found a relatively low rate of potential misclassification of definite suicides in USACID reports of investigation. The results suggest that misclassification of suicides may have diminished in recent years. A number of factors may have contributed to a decline in misclassification, and further study of the degree to which these factors influenced classification has implications for the training of law enforcement personnel, medical examiners and coroners. Most importantly, the identified reclassification rate would have had a negligible effect on the overall Army suicide rate and certainly could not account for the increasing Army suicide rate from 2005-2009. The low rate of misclassification provides reassurance that manner of death classification by medical examiners and military criminal investigators likely reflect the “true” population of military suicides. Despite some variation in quality and completeness, information contained within military criminal investigation records revealed consistencies with other psychological autopsies that suggest that such records can be a useful and cost-efficient proxy for a formal psychological autopsy.

Table 1. Army STARRS Reclassification of Manner of Death in CID ROIs (N = 998).

Reclassification Rate per 100
Type of Reclassification 2005 2006 2007 2008 2009 Overall
(2005—09)
Suicide to Suicide Probable or Suicide Possible 3.8 2.4 3.9 5.6 5.4 4.5
Suicide to Accident or Traffic 0.0 0.0 0.0 0.0 0.0 0.0
Accident or Traffic to any Suicide1 6.3 4.1 6.8 0.0 10.7 6.1
Homicide to any Suicide 33.3 0.0 0.0 40.0 20.0 28.6
Homicide to Accident 0.0 0.0 0.0 0.0 20.0 7.1
Undetermined to any Suicide 37.5 37.5 27.3 0.0 10.0 20.8
Undetermined to Accident or Traffic 0.0 0.0 27.3 18.2 10.0 12.5
Any non-Suicide2 to any Suicide 10.0 7.3 9.0 2.2 11.0 8.2
Open in a new tab
1

Any Suicide” includes Suicide, Suicide Probable, and Suicide Possible

2

Non-Suicide” cases include Accident or Traffic, Homicide, and Undetermined

Acknowledgments

On behalf of the Army STARRS collaborators

Funding support: Army STARRS was sponsored by the Department of the Army and funded under cooperative agreement number U01MH087981 with the U.S. Department of Health and Human Services, National Institutes of Health, National Institute of Mental Health (NIH/NIMH).

The contents are solely the responsibility of the authors and do not necessarily represent the views of the Department of Health and Human Services, NIMH, the Veterans Administration, Department of the Army, or the Department of Defense.

Footnotes

Role of the sponsors: Although a draft of this manuscript was submitted to the Department of Defense, the Department of the Army, and NIMH for review and comment prior to submission, this was with the understanding that comments would be no more than advisory.

Additional contributions: The Army STARRS Team consists of Co-Principal Investigators: Robert J. Ursano, MD (Uniformed Services University of the Health Sciences) and Murray B. Stein, MD, MPH (University of California San Diego and VA San Diego Healthcare System)

Site Principal Investigators: Steven Heeringa, PhD (University of Michigan) and Ronald C. Kessler, PhD (Harvard Medical School)

National Institute of Mental Health (NIMH) collaborating scientists: Lisa J. Colpe, PhD, MPH and Michael Schoenbaum, PhD

Army liaisons/consultants: COL Steven Cersovsky, MD, MPH (USAPHC (Provisional)) and Kenneth Cox, MD, MPH (USAPHC (Provisional))

Other team members: Pablo A. Aliaga, MA (Uniformed Services University of the Health Sciences); COL David M. Benedek, MD (Uniformed Services University of the Health Sciences); K. Nikki Benevides, MA (Uniformed Services University of the Health Sciences); Paul D. Bliese, PhD (University of South Carolina); Susan Borja, PhD (NIMH); Evelyn J. Bromet, PhD (Stony Brook University School of Medicine); Gregory G. Brown, PhD (University of California San Diego); Laura Campbell-Sills, PhD (University of California San Diego); Catherine L. Dempsey, PhD, MPH (Uniformed Services University of the Health Sciences); Carol S. Fullerton, PhD (Uniformed Services University of the Health Sciences); Nancy Gebler, MA (University of Michigan); Robert K. Gifford, PhD (Uniformed Services University of the Health Sciences); Stephen E. Gilman, ScD (Harvard School of Public Health); Marjan G. Holloway, PhD (Uniformed Services University of the Health Sciences); Paul E. Hurwitz, MPH (Uniformed Services University of the Health Sciences); Sonia Jain, PhD (University of California San Diego); Tzu-Cheg Kao, PhD (Uniformed Services University of the Health Sciences); Karestan C. Koenen, PhD (Columbia University); Lisa Lewandowski-Romps, PhD (University of Michigan); Holly Herberman Mash, PhD (Uniformed Services University of the Health Sciences); James E. McCarroll, PhD, MPH (Uniformed Services University of the Health Sciences); James A. Naifeh, PhD (Uniformed Services University of the Health Sciences); Tsz Hin Hinz Ng, MPH (Uniformed Services University of the Health Sciences); Matthew K. Nock, PhD (Harvard University); Rema Raman, PhD (University of California San Diego); Holly J. Ramsawh, PhD (Uniformed Services University of the Health Sciences); Anthony Joseph Rosellini, PhD (Harvard Medical School); Nancy A. Sampson, BA (Harvard Medical School); LCDR Patcho Santiago, MD, MPH (Uniformed Services University of the Health Sciences); Michaelle Scanlon, MBA (NIMH); Jordan W. Smoller, MD, ScD (Harvard Medical School); Amy Street, PhD (Boston University School of Medicine); Michael L. Thomas, PhD (University of California San Diego); Leming Wang, MS (Uniformed Services University of the Health Sciences); Christina L. Wassel, PhD (University of Vermont); Simon Wessely, FMedSci (King's College London); Christina L. Wryter, BA (Uniformed Services University of the Health Sciences); Hongyan Wu, MPH (Uniformed Services University of the Health Sciences); LTC Gary H. Wynn, MD (Uniformed Services University of the Health Sciences); and Alan M. Zaslavsky, PhD (Harvard Medical School).

References

  1. Beck AT, Beck R, Kovacs M. Classification of suicidal behaviors: I. Quantifying intent and medical lethality. The American Journal of Psychiatry. 1975 doi: 10.1176/ajp.132.3.285. [DOI] [PubMed] [Google Scholar]
  2. Bohnert AS, McCarthy JF, Ignacio RV, Ilgen MA, Eisenberg A, Blow FC. Misclassification of suicide deaths: Examining the psychiatric history of overdose decedents. Injury Prevention. 2013;19(5):326–330. doi: 10.1136/injuryprev-2012-040631. [DOI] [PubMed] [Google Scholar]
  3. Carr JR, Hoge CW, Gardner J, Potter R. Suicide surveillance in the U.S. Military--reporting and classification biases in rate calculations. Suicide and Life-Threatening Behavior. 2004;34(3):233–241. doi: 10.1521/suli.34.3.233.42785. [DOI] [PubMed] [Google Scholar]
  4. Cavanagh JT, Carson AJ, Sharpe M, Lawrie SM. Psychological autopsy studies of suicide: A systematic review. Psychological Medicine. 2003;33(3):395–405. doi: 10.1017/s0033291702006943. [DOI] [PubMed] [Google Scholar]
  5. Centers for Disease Control. Medical examiners' and coroners' handbook on death registration and fetal death reporting. Hyattsville, MD: CDC; 2003. [Google Scholar]
  6. Claassen CA, Yip PS, Corcoran P, Bossarte RM, Lawrence BA, Currier GW. National suicide rates a century after Durkheim: Do we know enough to estimate error? Suicide and Life-Threatening Behavior. 2010;40(3):193–223. doi: 10.1521/suli.2010.40.3.193. [DOI] [PubMed] [Google Scholar]
  7. Conner KR, Beautrais AL, Brent DA, Conwell Y, Phillips MR, Schneider B. The next generation of psychological autopsy studies. Suicide and Life-Threatening Behavior. 2011;41(6):594–613. doi: 10.1111/j.1943-278X.2011.00057.x. [DOI] [PubMed] [Google Scholar]
  8. Corr W. Suicides and suicide attempts among active component members of the US Armed Forces, 2010-2012; methods of self-harm vary by major geographic region of assignment. Medical Surveillance Monthly Report. 2014;21(10):2–5. [PubMed] [Google Scholar]
  9. Department of Defense. DoDIG Investigative Policy and Oversight Report. Department of Defense; 2015. Evaluation of the Military Criminal Investigative Organizations' Violent Crime Investigative Compliance Oversight Management and Inspection Programs. [Google Scholar]
  10. Eaton KM, Messer SC, Garvey Wilson AL, Hoge CW. Strengthening the Validity of Population-Based Suicide Rate Comparisons: An Illustration Using US Military and Civilian Data. Suicide and Life-Threatening Behavior. 2006;36(2):182–191. doi: 10.1521/suli.2006.36.2.182. [DOI] [PubMed] [Google Scholar]
  11. Helmkamp JC. Occupation and suicide among males in the US Armed Forces. Annals of Epidemiology. 1996;6(1):83–88. doi: 10.1016/1047-2797(95)00121-2. [DOI] [PubMed] [Google Scholar]
  12. Ho T, Yip PS, Chiu C, Halliday P. Suicide notes: what do they tell us? Acta Psychiatrica Scandinavica. 1998;98(6):467–473. doi: 10.1111/j.1600-0447.1998.tb10121.x. [DOI] [PubMed] [Google Scholar]
  13. Kessler RC, Colpe LJ, Fullerton CS, Gebler N, Naifeh JA, Nock MK, Heeringa SG. Design of the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS) International Journal of Methods in Psychiatric Research. 2013;22(4):267–275. doi: 10.1002/mpr.1401. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Kessler RC, Stein MB, Bliese PD, Bromet EJ, Chiu WT, Cox KL, Ursano RJ. Occupational Differences in US Army Suicide Rates. Psychological Medicine. 2015;45(15):3293–3304. doi: 10.1017/s0033291715001294. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977:159–174. [PubMed] [Google Scholar]
  16. Nock MK, Deming CA, Fullerton CS, Gilman SE, Goldenberg M, Kessler RC, Schoenbaum M. Suicide among soldiers: A review of psychosocial risk and protective factors. Psychiatry. 2013;76(2):97–125. doi: 10.1521/psyc.2013.76.2.97. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Palmer BS, Bennewith O, Simkin S, Cooper J, Hawton K, Kapur N, Gunnell D. Factors influencing coroners' verdicts: An analysis of verdicts given in 12 coroners' districts to researcher-defined suicides in England in 2005. Journal of Public Health. 2014 doi: 10.1093/pubmed/fdu024. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Pestian J, Nasrallah H, Matykiewicz P, Bennett A, Leenaars A. Suicide Note Classification Using Natural Language Processing: A Content Analysis. Biomedical Informatics Insights. 2010:19–28. doi: 10.4137/BII.S4706. 3(2203-BII-Suicide-Note-Classification-Using-Natural-Language-Processing:-A-Conte.pdf) [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. SAS Institute. SAS® 9.3 Software. Cary, NC: SAS Institute Inc; 2011. [Google Scholar]
  20. Tøllefsen IM, Hem E, Ekeberg Ø. The reliability of suicide statistics: A systematic review. BMC Psychiatry. 2012;12(1):1. doi: 10.1186/1471-244X-12-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Trofimovich L, Reger MA, Luxton DD, Oetjen-Gerdes LA. Suicide risk by military occupation in the DoD active component population. Suicide and Life-Threatening Behavior. 2013;43(3):274–278. doi: 10.1111/sltb.12013. [DOI] [PubMed] [Google Scholar]
  22. Ursano RJ, Colpe LJ, Heeringa SG, Kessler RC, Schoenbaum M, Stein MB. The Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS) Psychiatry. 2014;77(2):107–119. doi: 10.1521/psyc.2014.77.2.107. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES