The Effects of Misclassification Biases on Veteran Suicide Rate Estimates

Abstract

Objectives. We assessed the impact that possible veteran suicide misclassification biases (i.e., inaccuracy in ascertainment of veteran status on the death certificate and misclassification of suicide as other manner of death) have on veteran suicide rate estimates.

Methods. We obtained suicide mortality data from the 2003–2010 National Violent Death Reporting System and the 2003–2010 Department of Defense Casualty Analysis System. We derived population estimates from the 2003–2010 American Community Survey and 2003–2010 Department of Veterans Affairs data. We computed veteran and nonveteran suicide rates.

Results. The results showed that suicide rates were minimally affected by the adjustment for the misclassification of current military personnel suicides as veterans. Moreover, combining suicides and deaths by injury of undetermined intent did not alter the conclusions.

Conclusions. The National Violent Death Reporting System is a valid surveillance system for veteran suicide. However, more than half of younger (< 25 years) male and female suicides, labeled as veterans, were likely to have been current military personnel at the time of their death and misclassified on the death certificate.

Rapidly rising suicide rates among current and former US military personnel have generated nationwide concern. There is increasing evidence that veterans of both genders are at greater risk for suicide than their nonveteran counterparts.^1–5 Moreover, veterans who receive care through the Veterans Health Administration (VHA) have higher rates of suicide than non–VHA users^6–8 and nonveterans.⁹ Furthermore, younger veterans (< 35 years) have the highest risk of suicide.^1,4,10,11 But some studies have found no connection between military service and suicide.^12–14

These conflicting findings have been attributed in part to misclassification biases. Previous research^15,16 and the Blue Ribbon Work Group’s report¹⁷ have noted that suicide mortality estimates are often compromised by 2 main sources of misclassification: (1) inaccuracy in ascertainment of veteran status on the death certificate and (2) misclassification of suicides as other manners of death. The first bias refers to the designation of veteran status on the death certificate: “ever served in the armed forces.” The argument is that the item does not distinguish between military personnel currently serving on active duty, in the National Guard, or in the Reserves, and veterans who have separated from the service. However, recent evidence shows high correlation between death certificate and self-reported veteran status (agreement rate = 95%¹⁸), proxy-derived information (κ = 0.91⁴), and VHA records (sensitivity = 95%; specificity = 91%¹⁹). Nonetheless, the validity of the designation of veteran status on the death certificate may differ by age.²⁰ It is likely that the death certificate item would be accurate for older veterans but not necessarily for younger decedents. As a consequence, suicide rate estimates derived for younger veterans are likely to be more affected by this bias resulting in overestimation. This concern is particularly important because the relative risk for suicide is highest among younger veterans.

The second bias refers to self-inflicted death misclassified as other manners of death such as injuries of undetermined intent. Previous studies have shown evidence that veterans’ deaths relative to nonveterans’ are less likely to be classified as injuries of undetermined intent.^5,17 Deceased veterans may be more likely than nonveterans to be classified as suicides for several reasons, including higher use of firearms (more likely to be classified as suicide than another method), greater availability of accurate information for death certificates among VHA users, and implicit societal beliefs that veterans are at elevated risk for suicide. In this case, misclassification bias (chiefly for nonveterans) may account for the apparently higher suicide rates found in the veteran population.

Few, if any, studies have attempted to test for the effect of these misclassification biases on suicide rate estimation in the veteran and nonveteran populations. Yet this information is essential for surveillance efforts and suicide prevention among veterans. Therefore, the aims of this study were (1) to assess the effect of excluding current military personnel (listed as veterans on the death certificate) from veteran suicide rate estimates by age and gender and (2) to test the effect of including deaths by injury of undetermined intent as suicides on suicide rate estimates.

METHODS

We performed analyses with data from the 2003–2010 National Violent Death Reporting System (NVDRS), the 2003–2010 Department of Defense Casualty Analysis System (DCAS),²¹ the 2003–2010 American Community Survey (ACS),²² and the 2003–2010 Department of Veterans Affairs (VA) veteran population estimates.²³ The NVDRS is a state-based surveillance system that provides detailed accounts of violent deaths that occur in the United States. As of 2010, 18 states (Alaska, Colorado, Georgia, Kentucky, Maryland, Massachusetts, Michigan, New Jersey, New Mexico, North Carolina, Ohio, Oklahoma, Oregon, Rhode Island, South Carolina, Utah, Virginia, and Wisconsin) contributed data to the NVDRS. The data derived from coroner or medical examiner records, police reports, death certificates, toxicology labs, crime labs, and Alcohol, Tobacco, Firearms, and Explosives gun trace reports. We ascertained veteran status with information obtained from death certificates. The NVDRS provided data on adult (aged 18 years and older) suicides and undetermined deaths (i.e., deaths that could have been either deliberate or accidental self-destruction). The DCAS is a surveillance system containing information on US service members (active duty, Reserves, and National Guard) who have been classified as deceased. Population estimates for those aged 18 years and older residing in the NVDRS states were based on the 2003–2010 ACS for nonveterans and on the VA population data for veterans. Estimates for numbers of currently serving military personnel in the NVDRS states were obtained from the Defense Manpower Data Center.²⁴

We computed suicide rates with adjustment for the 2 misclassification biases. First, we assumed that all current military personnel suicides were wrongly coded as veterans on the death certificate. Second, we assumed that all undetermined deaths were misclassified suicides.

The method used by Gibbons et al.¹ was to adjust the estimates for the exclusion of current military personnel as veterans on death certificate. In this approach, current military personnel suicides were removed from the NVDRS veteran suicides. It should be noted that Miller et al.²⁰ criticized the approach of Gibbons et al.,¹ suggesting improper adjustment for age. Specifically, Miller et al.²⁰ noted that the age distribution used by Gibbons et al.¹ was based on military enrollment rather than on military suicide consequently overestimating the veteran suicide rate among the youngest age group.

In the present study, we extended the method used by Gibbons et al.¹ and estimated the numbers of current military personnel suicides by age and gender for each year of the NVDRS. According to the Department of Defense Suicide Event Reporting,²⁵ 47% of the service member suicides were among military personnel aged 18 to 24 years, 36% among those aged 25 to 34 years, 16% were among those aged 35 to 44 years, and an estimate of 0.7% among those aged 45 to 64 years. Few (if any) older adults died by suicide while in active duty; thus, we excluded no suicides from this age group. Also, women accounted for 5% of all military suicides. We used these proportions to compute the estimated numbers of current military personnel suicides in the NVDRS states. Table 1 footnotes show the steps used to compute the estimated numbers of current military suicides by age and gender.

TABLE 1—

Estimation of Current Military Suicides in the National Violent Death Reporting System, 2003–2010

	Total				Men					Women
Year	All Current Military Suicides,a No.	All Current Military Suicide Rates/100 000	Proportion of Current Military in the NVDRS Statesb	Current Military Suicides in NVDRS States,c No.	All Suicidesd	Aged 18–24 Yearse	Aged 25–34 Yearse	Aged 35–44 Yearse	Aged 45–64 Yearse	All Suicidesd	Aged 18–24 Yearse	Aged 25–34 Yearse	Aged 35–44 Yearse	Aged ≥ 45 Yearse
2003f	190	11.0	0.17	33	31	14.7	11.1	4.9	2.1	2	0.8	0.6	0.3	0.1
2004f	197	11.5	0.37	72	68	32.5	24.6	10.9	4.7	4	1.7	1.3	0.6	0.2
2005	182	10.9	0.42	77	73	34.7	26.2	11.6	5.0	4	1.8	1.4	0.6	0.3
2006	213	13.2	0.42	89	85	40.3	30.5	13.5	5.8	4	2.1	1.6	0.7	0.3
2007	211	13.1	0.41	86	82	38.7	29.3	13.0	5.5	4	2.0	1.5	0.7	0.3
2008	259	15.4	0.41	107	102	48.3	36.6	16.2	6.9	5	2.5	1.9	0.9	0.4
2009	302	18.4	0.41	125	118	56.2	42.5	18.9	8.0	6	3.0	2.2	1.0	0.4
2010g	289	17.1	0.42	122	116	55.1	41.7	18.5	7.9	6	2.9	2.2	1.0	0.4
Total	1843			711	675	320	243	108	46	36	17	13	6	2

Note. NVDRS = National Violent Death Reporting System.

^aCurrent military (active duty, National Guard, and Reserves) suicide cases derived from Defense Casualty Analysis System.²¹

^bProportion of current military personnel in the NVDRS states out of the total US military personnel population derives from Defense Manpower Data Center distribution of personnel data.²⁴

^cNumber of suicides multiplied by the proportion of military personnel in the states participating in the NVDRS. The proportion is lower in the first 2 years because fewer states participated in the NVDRS.

^dNumber of suicides for each gender (95% for men and 5% for women).

^eNumber of suicides for each age group (18–24 = 47%; 25–34 = 36%; 35–44 = 16%; 45–64 = 0.7%). Of note, there were too few female veteran suicides aged ≥ 65 years to provide accurate estimates; thus, we combined those aged 45–64 years with those aged ≥ 65 years.

^fSeven and 13 states participated in the NVDRS during 2003 and 2004, respectively.

^gTwo states were added in 2010 to the 16 participating since 2005 in the NVDRS.

We then computed veteran suicide rates with the following 4 equations. We made 2 different adjustments to the numerator. First, we removed the estimated number of current military personnel suicides from the number of veteran suicides. Second, we added undetermined deaths to the number of veteran suicides.

We computed nonveteran suicide rates by using equations 5 to 8. We made adjustments to the numerator and the denominator. Regarding the numerator, we added the number of current military personnel suicides to the number of nonveteran suicides; next, we also added the number of nonveteran undetermined deaths to nonveteran suicides. The denominator includes civilians and current military personnel. We subtracted VA population estimates from the total ACS population because it is likely that the VA estimates more closely represent the true number of veterans than the “former military personnel” defined in ACS.

We also constructed Poisson log–linear regression models of unadjusted frequencies to examine age, military status (veteran, current military personnel, or civilian) and death status (suicide, undetermined death, or alive) and their interactions. We included main effects for age, military service, and death status plus all 2-way and 3-way interactions in gender-specific models.

RESULTS

The estimated current military personnel suicides misclassified as veteran on the death certificate accounted for 58% and 24% among men aged 18 to 24 years and 25 to 34 years and 50% and 19% among women aged 18 to 24 years and 25 to 34 years, respectively. Among other age groups, we estimated no more than 5% of the NVDRS suicides to be current military personnel at the time of death. Despite the reduction in the number of younger veteran suicides, the results revealed that male and female veterans of all ages had higher suicide rates than nonveterans (Table 2).

TABLE 2—

Suicide Rates by Veteran Status, Age, and Gender: National Violent Death Reporting System, 2003–2010

	Age Group, y
Veteran Status	18–24	25–34	35–44	45–64	≥ 65
Men
Veteran
Rate 1	108.7	33.8	29.8	27.1	29.2
Rate 2	49.6	25.9	28.3	26.7	29.2
Rate 3	119.0	37.9	34.9	31.5	30.4
Rate 4	59.9	30.0	33.4	31.1	30.4
Nonveteran
Rate a	18.3	19.6	21.4	23.2	24.1
Rate b	19.3	20.2	21.6	23.3	24.1
Rate c	21.4	23.7	26.7	27.9	25.5
Rate d	22.4	24.4	26.9	28.0	25.5
Womena
Veteran
Rate 1	25.6	9.3	12.0	9.9
Rate 2	13.8	7.6	11.6	9.8
Rate 3	27.9	11.0	14.2	13.1
Rate 4	16.1	9.3	13.8	12.9
Nonveteran
Rate a	3.8	4.8	6.9	6.1
Rate b	3.9	4.8	6.9	6.1
Rate c	5.0	6.7	9.8	8.3
Rate d	5.0	6.8	9.9	8.3

Note. Rate 1 = (veteran suicides/VA population estimate) * 100 000; rate 2 = [(veteran suicides - active-duty suicides)/VA population estimate] * 100 000; rate 3 = [(veteran suicides + veteran undetermined deaths)/VA population estimate] * 100 000; rate 4 = [(veteran suicides - active duty suicides + veteran undetermined deaths)/VA population estimate] * 100 000; rate a = [nonveteran suicides/(total population - VA population estimates)] * 100 000; rate b = [(nonveteran suicides + active duty suicides)/(total population - VA population estimates)] * 100 000; rate c = [(nonveteran suicides + nonveteran undetermined deaths)/(total population - VA population estimates)] * 100 000; rate d = [(nonveteran suicides + active duty suicides + nonveteran undetermined deaths)/(total population - VA population estimates)] * 100 000.

^aThere were too few female veteran suicides aged ≥ 65 years to provide accurate estimates; thus, we combined those aged 45–64 years with those aged ≥ 65 years.

Results from the Centers for Disease Control and Prevention’s Web-based Injury Statistics and Reporting System for states in NVDRS 2005–2010²⁶ showed that the proportion of undetermined deaths among all violent deaths was lower among male and female veterans (8.3% and 12.4%, respectively) than their nonveteran counterparts (14.2% and 22.4%, respectively). Nonetheless, adding the undetermined deaths to the suicide cases resulted in higher suicide rates for both veterans and nonveterans regardless of gender and regardless of the adjustment for current military personnel suicides (Table 2).

Chi-square test results showed that all veteran suicide rates were statistically significantly higher than those for nonveterans (P < .001; data not shown) despite the adjustments. Results from the Poisson log–linear models showed that for men, the 3-way interaction terms for veterans aged younger than 65 years were positive and significant (P < .001; data not shown). Among women, all 3-way interactions for veterans of all ages were significant (P < .01; data not shown). In other words, there was variation among age groups in the relative risk of suicide for veterans versus their nonveteran counterparts.

DISCUSSION

We assessed the potential effect of 2 misclassification biases on the rates of suicide among veterans. The results showed that the effect on suicide rates was negligible and did not alter the conclusions drawn from the findings. The results are consistent with earlier studies by Kaplan et al.⁴ and Bahraini et al.¹⁹ who showed that the designation of veteran status on the death certificate was reliable. In addition, the findings support the results in the study by Gibbons et al.¹ that rates of veteran suicides are elevated even after current military suicides are removed.

Consistent with the Blue Ribbon Panel report,¹⁷ nonveteran deaths were more likely to be classified as injuries of undetermined intent than among veterans. Nevertheless, the addition of undetermined deaths to suicides had little effect on the differences between veteran and nonveteran suicide risk estimates (especially for men).

This study has some inherent limitations. First, not all US states participate in the NVDRS. Second, the NVDRS does not include data on all unintentional deaths and, thus, the study does not account for suicides misclassified as all other manners of death (e.g., fatal vehicular injuries). Third, the adjustments (involving additions or subtractions to numerators and denominators) imply additive models that are not easily analyzed via Poisson (log–linear) regression.

Nonetheless, the results showed that, overall, male and female veterans of all ages had higher suicide rates than their nonveteran counterparts. The findings suggest that the misclassification biases associated with the designation of veteran status found in the NVDRS data have little impact on veteran suicide estimates and also imply that veteran status on the death certificate is a reliable source of information and appropriate for further research in this area.

The present study offers encouraging news for researchers who rely on death certificate information for suicide risk estimation among veterans. On the basis of this study’s results, we recommend that other researchers use VA-derived veteran population estimates to compute suicide rates while recognizing that as many as 58% of younger male and 50% of younger female suicides that were labeled as veteran suicides were likely to have been current military personnel at the time their death. Policymakers can also be reassured that the use of suicide estimates derived from the NVDRS are reliable. Although the biases are present, their negligible effect should not deter the development of suicide prevention programs aimed at reducing self-harm among veterans. In conclusion, the NVDRS can be considered a valid surveillance system for suicide in the veteran population.