Use of the Air Force Post-Deployment Health Reassessment for the Identification of Depression and Posttraumatic Stress Disorder: Public Health Implications for Suicide Prevention

Abstract

Objectives. Military members are required to complete the Post-Deployment Health Assessment on return from deployment and the Post-Deployment Health Reassessment (PHDRA) 90 to 180 days later, and we assessed the PDHRA’s sensitivity and specificity in identifying posttraumatic stress disorder (PTSD) and depression after a military deployment among US Air Force personnel.

Methods. We computed the PDHRA’s sensitivity and specificity for depression and PTSD and developed a structural model to suggest possible improvements to it.

Results. For depression, sensitivity and specificity were 0.704 and 0.651, respectively; for PTSD, they were 0.774 and 0.650, respectively. Several variables produced significant direct effects on depression and trauma, suggesting that modifications could increase its sensitivity and specificity.

Conclusions. The PDHRA was moderately effective in identifying airmen with depression and PTSD. It identified behavioral health concerns in many airmen who did not develop a diagnostic mental health condition. Its low level of specificity may result in reduced barriers to care and increased support services, key components of a public health approach to suicide prevention, for airmen experiencing subacute levels of distress after deployment, which may, in part, account for lower suicide rates among airmen after deployment.

The conflicts in Iraq and Afghanistan represent the longest wartime engagement in US military history.¹ Their impact on military members is only beginning to be understood, and the effects are likely to reverberate for decades. Because of the protracted nature of these conflicts, military members and veterans may have increased mental health needs.² As of 2009, 1.6 million US military members had deployed in support of Operation Enduring Freedom (Afghanistan) and Operation Iraqi Freedom. Of these, an estimated 300 000 have returned with a mental health condition, such as depression or posttraumatic stress disorder (PTSD).³ Exposure to violent combat is often a precursor to emotional dysfunction, most notably an increased risk of PTSD and depression,^4–6 that may lead to suicidal behavior, including suicide attempts and ideation. The relatively high rates of depression and PTSD and the marked increase in military suicide rates from 2005 to 2009 have made mental health issues the source of significant concern for the military.^6–8

In response to the physical and emotional hazards of deployment and the increasing frequency of suicides among military members that some believe are a consequence of prolonged and repeated deployments,^6,7,9,10 the US Department of Defense established a robust program to screen and track deployment-related physical and psychiatric illnesses.^11,12 Thus, all military members are currently required to complete the Post-Deployment Health Assessment, which is part of a broader military health monitoring system, immediately on their return from deployment. A nearly identical screening tool, the Post-Deployment Health Reassessment (PDHRA), is administered 90 to 180 days later.¹³ Additional screening may occur at the discretion of medical providers or military members’ commanders.¹⁴

The PDHRA has been used since 2005 to assess the health of military members in the months after a deployment.¹⁴ It was augmented in 2008 by broadening questions about traumatic brain injury (TBI) and alcohol misuse.¹⁵

Although significant resources have been dedicated to identifying postdeployment health and mental health issues, the efficacy of the screening and assessment instruments has not been established.^16,17 Postdeployment assessments were developed by consensus in professional working groups and rapidly deployed in response to a congressional mandate.¹⁶ No scaling or testing of the assessments was or has been conducted before or since implementation;¹⁶ therefore, their reliability and validity have not been established. Specifically, whether the PDHRA is an effective tool for identifying military members at risk for developing mental health concerns after a deployment is not known.

To address this gap, we evaluated the PDHRA’s effectiveness in identifying military members at risk for depression and PTSD and identified ways to improve its sensitivity and specificity. In addition, we assessed the relationship between deployment and other factors associated with depression and PTSD to further understand factors that might increase the risk of negative outcomes, including suicide, after a deployment.

METHODS

We used a comprehensive population sampling strategy. All active, reserve, and National Guard airmen (n = 58 242) who completed the PDHRA between January 1, 2008, and December 31, 2008, were included in this study. Because of the study’s large sample size, we used a rigorous standard for statistical significance, P < .001, to avoid capitalization on chance. Study participants were aged 17 years or older because of Department of Defense regulations.¹⁸ Study participants included all pay grades from Airman Basic (E-1) to Major General (O-8).

The PDHRA is a Web-based, 3-page, self-report questionnaire that includes questions on demographic characteristics, general health, physical symptoms, and environmental exposures and mental health items that may be deployment related.¹³ It is the primary tool used by the military medical system to identify individuals who have physical or behavioral health concerns after a deployment. The PDHRA is also the last in a series of formal screenings that the Department of Defense uses to identify service members who are experiencing distress after a deployment.¹³ When completed, the PDHRA becomes part of the military member’s medical record¹⁴ and is integrated into the Defense Medical Surveillance System database.¹³ Consequently, the PDHRA’s effectiveness in identifying service members who may be at elevated risk for PTSD or depression is central to maintaining a healthy military population and decreasing the risk for suicide.

PDHRA managers are responsible for triaging all positive PDHRAs at their military base. The criteria for a positive PDHRA are defined in a comprehensive algorithm available in the Post-Deployment Health Reassessment User’s Guide.¹⁴ Contact with airmen whose responses on the PDHRA indicate areas of concern may occur by telephone or immediate referral to a medical provider. The PDHRA manager or other medical provider typically calls the military member to discuss the results of his or her PDHRA and explores the need for follow-up. An airman’s primary care physician and the physician’s support staff have access to the airman’s PDHRA results, which provide clinically relevant information for use in determining treatment needs. In addition to the PDHRA’s primary questions, specific question sets were developed to assess for certain behavioral health disorders, such as PTSD and depression. Thus, airmen who screen positive for behavioral health concerns are offered the opportunity to complete the PTSD Checklist–Military Version (PCL-M), which assesses for trauma symptoms, and the Patient Health Questionnaire–9 (PHQ-9), which assesses for depression.¹⁴ The results of the PCL-M and PHQ-9 are also available to medical personnel and used to evaluate specific mental health concerns.

The data used in the current analysis were drawn from the M-2 military public health database of PDHRA results, housed at Brooks City-Base, San Antonio, Texas. The M-2 is a comprehensive database that includes all diagnostic data for all Air Force members. A data request was submitted for diagnostic data on all airmen. The primary dependent variables for this study were the International Classification of Diseases, Ninth Revision (ICD-9)¹⁹ codes that identified a diagnosis of depression or PTSD and acute stress disorder. We used the ICD-9 codes for PTSD and acute stress disorder to create the endogenous variable trauma diagnosis, and we used the ICD-9 codes for dysthymic disorder, major depressive disorder, depressive episode, and depression not otherwise specified to create the endogenous variable depression diagnosis (Figure 1). All trauma-related or depressive diagnoses were assigned by a physician, a licensed psychologist, or a licensed clinical social worker after an evaluation that determined that the airman met the criteria for these conditions.

FIGURE 1—

Post-Deployment Health Reassessment structural model for 58 242 airmen taken from the M-2 military public health database: Brooks City-Base, San Antonio, TX, January 1–December 31, 2008.

Note. PCL-M = PTSD Checklist–Military Version; PHQ-9 = Patient Health Questionnaire–9; TBI = traumatic brain injury.

*Indicates P < .001

We developed structural models using a 3-step process. All PDHRA questions related to behavioral health concerns according to the user’s manual were entered into a theoretically derived confirmatory factor analysis. Factors were then modeled on the basis of their hypothesized relationship to the endogenous variables (Figure 1). We then used the Wald χ² test and Lagrange multiplier test to identify the best-fitting structural model.²⁰ We conducted all statistical analyses using Mplus Version 5 software.²¹

RESULTS

Most of the sample were enlisted airmen (n = 48 290), and the largest group represented was staff sergeants (E-5; n = 15 139). The officer corps made up about 17% of the sample (n = 9817), compared with 18.6% of the total Air Force.²⁰ Although women make up 19% of the total Air Force,²² 15% of this study’s sample (n = 8859) were female.

The average respondent in this study had deployed twice (mean = 1.98; SD = 1.76), and many (17.8%; n = 10 344) had not deployed to a combat zone but were sent to more forward locations, such as Germany, to assist with combat operations. Many airmen in this study were exposed to direct combat. For example, many (13.4%; n = 7823) reported exposure to an explosion or blast during their deployment, whereas only 3% (n = 1757) reported experiencing a vehicular crash.

Although 16.5% of the sample noted 1 or more traumatic combat experiences, participants had a very low prevalence of diagnosed PTSD (0.3%; n = 160) or depression (0.6%; n = 338). PTSD and depression were significantly correlated in this sample (r = 0.346; P < .001), but only 0.1% (n = 81) of participants were diagnosed with comorbid PTSD and depression.

Psychometric Properties

The Cronbach’s α for the PDHRA question set used to screen for the PCL-M was within the acceptable range for nomothetic research (α = 0.76), and the Cronbach α for the question set used to screen for the PHQ-9 (α = 0.83) was high enough to serve as a guide for clinical decision-making.^23–25

The PHQ-9’s internal consistency was excellent (α = 0.98). Its mean for this sample (mean = 2.10; SD = 9.37) was within 1 standard deviation of the clinical range of 5, which suggests mild concerns, and 10, which suggests moderate concerns;²⁶ this score suggests that PDHRA depression screening items identify participants who may benefit most from early intervention.²⁷ Similarly, the PCL-M’s internal consistency was excellent (α = 0.99). The mean score of airmen completing the PCL-M was more than 3 standard deviations below the PCL-M’s clinical cutoff level of 50 (mean = 6.91; SD = 14.08);²⁸ this score suggests that PDHRA trauma-related screening items are too inclusive.

In this sample, depression diagnoses were significantly more common among individuals with PDHRA scores that were positive for behavioral health concerns than among those with PDHRA scores suggesting few behavioral health concerns (χ²_{1, n = 58 242} = 186.43; P < .001). One of 85 individuals with positive PDHRA scores eventually received a diagnosis of depression, compared with 1 of 378 individuals with negative PDHRA scores. This finding translates to airmen with positive PDHRA scores being more than 4 times as likely to be diagnosed with depression than airmen with PDHRA scores that do not indicate behavioral health concerns.

PTSD diagnoses were also significantly more common among individuals with PDHRA scores positive for behavioral health concerns than among those with negative PDHRA scores (χ²_{1, n = 58 242} = 108.81; P ≤ .001). Among airmen with positive PDHRA scores, 1 of 171 eventually received a diagnosis of PTSD, compared with only 1 of 922 airmen with negative PDHRA scores. Thus, airmen with PDHRA scores that indicated behavioral health concerns were more than 5 times as likely to be diagnosed with PTSD than were airmen with negative PDHRA scores.

Specificity and Sensitivity for Depression and Posttraumatic Stress Disorder

The 2 most common ways to assess the clinical value of a test are to determine its sensitivity and specificity.²⁹ Sensitivity is the proportion, for a given condition, of actualpositives that are correctly identified; specificity is the proportion of actual negatives that are correctly identified.^30,31 Specificity is related to the Type I error of a measure, with higher specificity suggesting a low Type I error rate, or fewer false positives. Sensitivity is related to the Type II error of a measure, with higher sensitivity suggesting a low Type II error rate, or fewer false negatives.^32–34

In this study, the PDHRA’s sensitivity for depression was 0.704 (n = 238), and its specificity was 0.651 (n = 37 713); its specificity for PTSD was 0.744 (n = 119), and its sensitivity was 0.650 (n = 37 772). The PDHRA’s lack of sensitivity for depression was 0.296 (n = 100), and its lack of specificity was 0.349 (n = 20 191); its lack of sensitivity for PTSD was 0.256 (n = 41), and its lack of specificity was 0.350 (n = 20 310).

Structural Equation Modeling Analysis

We analyzed a structural equation model to determine the contribution made by each PDHRA item to identify airmen at risk for experiencing trauma-related or depressive symptoms. The PDHRA structural model assessed the relationship between answers on the PDHRA and the later development of a diagnosis of depression or PTSD. The analysis suggested a good-fitting model (Figure 1). The χ² test of model fit was significant (χ²_{160, n = 58 242} = 6530.03; P ≤ .001). Other fit indices suggested a good- to excellent-fitting model: comparative fit index = 0.94; Tucker-Lewis index = 0.97; root-mean-square error of approximation = 0.03.²⁰ The latent variable depression produced a large, positive direct effect (b = 0.51; P ≤ .001) on the variable depression diagnosis, and the latent variable trauma produced a large direct effect (b = 0.55; P ≤ .001) on the variable trauma diagnosis.

The largest direct effects were produced by the latent variable support network conflict. Support network conflict produced a large, positive, statistically significant effect on depression (b = 1.48; P ≤ .001) and trauma (b = 1.45; P ≤ .001). Other variables produced statistically significant direct effects on depression and trauma: Being shot (b = 0.77; P ≤ .001) and being exposed to a blast (b = 0.68; P ≤ .001) had a large, positive direct effect on trauma. Being involved in a vehicle crash also had a moderate, positive direct effect on depression (b = 0.38; P ≤ .001) and trauma (b = 0.40; P ≤ .001). TBI symptoms had a small, positive direct effect on depression (b = 0.12; P ≤ .001) and trauma (b = 0.17; P ≤ .001). We found no statistically significant relationship between the number of deployments and depression (b = −0.01; P = .017) or trauma (b = −0.004; P = .479).

The structural model accounted for 22.2% (r² = 0.222) of the variance in depression diagnosis and 26.2% (r² = 0.262) of the variance in trauma diagnosis. It accounted for 64.5% (r² = 0.645) of the variance in the latent variable depression, and 61.3% (r² = 0.613) of the variance in the latent variable trauma.

DISCUSSION

We should consider several limitations. First, the PDHRA is based on self-report and may therefore be subject to the usual biases associated with these types of data. Second, the data used in this study were retrospective; if the PDHRA’s psychometric characteristics are improved, the relationships modeled in this study may not be replicable. Suggestions for enhancing the PDHRA are discussed later in this section.

The PDHRA appears to be a moderately effective clinical screening tool with this sample of airmen. Individuals whose PDHRA scores were positive for behavioral health concerns were significantly more likely to develop a diagnosis of PTSD or depression than individuals whose PDHRA scores did not indicate behavioral health concerns. The PDHRA’s sensitivity for both depression and PTSD did not reach the 0.85 threshold set forth in this study (depression = 0.71; PTSD = 0.74),³⁵ which was surprising given that it was designed to be overly inclusive of behavioral health concerns. Airmen who were diagnosed with depression or PTSD but for whom the PDHRA did not indicate behavioral health concerns may not have been directed or referred for services to address their mental health needs. Although the PDHRA’s sensitivity is inadequate for diagnostic purposes, it seems to provide medical and mental health professionals with a screening tool that is useful for identifying most individuals who are at risk for depressive and trauma-related disorders after a deployment.

The PDHRA’s lower than expected sensitivity may have resulted partially because some individuals were diagnosed with depression or PTSD that was the result of factors that occurred after their PDHRA screening and had little connection to their deployment. If this pattern was present in a very small segment of the overall sample, it would account for most of the false negative results. Future research controlling for life stressors that occur after PDHRA completion would more accurately describe its sensitivity.

Because the PDHRA was designed to be an overly inclusive screening tool,¹⁶ its low specificity was not surprising. However, its poor specificity may have resulted in many airmen who did not have acute levels of psychiatric distress being provided supportive services by medical and mental health providers. This process may have promoted help-seeking behaviors and facilitated the delivery of preventive medical interventions.^36,37 Recent research with veterans of Operation Iraqi Freedom and Operation Enduring Freedom has found high utilization of mental health services.¹¹ In short, after a deployment, those who need care appear to be receiving it, which may contrast with the experience of military members who have non–deployment-related depression or trauma. Prior studies have documented significant stigma associated with seeking care for mental health issues within military cohorts.³⁶

In addition, mental health screening during the military accession process and during annual physical health assessments is much less robust than postdeployment,^38,39 which may help explain the elevated risk of suicide among airmen who have not deployed.^40,41 Although the postdeployment screening process may be protective, our findings suggest several areas in which the PDHRA’s accuracy could be improved.

Support Networks

The latent variable support network conflict produced by far the largest effects on both depression and trauma in both PDHRA measurement models. The importance of support networks in enhancing resilience after traumatic exposure in combat veterans is well established in the literature.^42,43 On the PDHRA, support network conflict encompasses family, social, and occupational support networks. Social support, family functioning, and occupational satisfaction are each important components of a support network. Combat veterans who have high levels of social support are at less risk for depression, trauma-related problems, and suicide after a combat experience than peers who have inadequate social resources.^42,44 Closeness to family members also enhances military members’ resilience after traumatic combat-related experiences,⁴⁴ as do homecoming ceremonies.⁶ Because the PDHRA measures these 3 separate aspects of a support network with only 2 questions, it may undervalue and under-assess the role of support networks. The inclusion of a more robust assessment of support network functioning may provide critical clinical information about the presence or lack of the resiliency-enhancing systems available to military members after a deployment. A small set of screening questions for each of the constructs of social support, family functioning, and occupational satisfaction could be included in the PDHRA with minimal impact on the its length. For example, adding a question that directly asks whether the respondent had an important relationship end or change significantly during or since the deployment may be of value. These screening questions could then be supplemented with standardized measures of these constructs, similar to the use of the PCL-M and PHQ-9.

Gender

Previous research has identified a higher prevalence of depression and PTSD among women.^46,47 Being female was associated with marginally higher levels of trauma and depression. For military health care providers to be aware of the additional risk female airmen face during and after a deployment may be of value. In its present form, the PDHRA does not include gender as a behavioral health risk factor.¹⁴ Developing gender-specific thresholds for various behavioral health items on the PDHRA may improve its sensitivity. The PDHRA does not include gender-specific cutoffs for variables measuring alcohol-related problems.¹⁴ Expanding the use of these cutoffs to other constructs may be of value. Modifications to the PDHRA may help clarify gender-related risks associated with trauma and depression. Developing gender-specific clinical thresholds for depression and trauma screening questions may improve its sensitivity.

Traumatic Brain Injury Symptoms

The TBI symptoms factor produced significant effects on both trauma and depression. Another potential limitation is that the PDHRA does not currently include TBI symptoms items to identify behavioral health concerns. The variables that make up the TBI symptoms factor are used in the PDHRA to identify physical concerns. Medical providers must be informed that the TBI symptoms items may be indicative of both physical concerns and behavioral health concerns.

Previous research has suggested that military members may be at greater risk for injury after a deployment,⁴⁸ and increasing suicide rates among service members have recently been speculated to be a consequence of repeated deployments. However, current research has demonstrated a strong, negative correlation between the number of deployments and suicide among airmen (r = −0.994; P = .006).^41,49 These findings may, in part, be explained by the healthy warrior effect. Previous research identified a disproportionate loss of psychologically unfit personnel during early military training.^50–52 Consequently, military members who are able to serve long enough to deploy 1 or more times appear to have a certain level of resilience that increases over time. Noteworthy is the work of psychopathologist Henry A. Murray, who found in a 1948 study of returning veterans that combat experience did not always result in negative outcomes. Indeed, veterans with considerable trauma exposure were among the strongest and most resilient.⁵³

Our findings suggest an additional hypothesis for the low suicide rate among military deployers. The PDHRA process provides airmen who have only minor behavioral health concerns and do not have an acute level of psychiatric distress with supportive services from medical and mental health providers. This process may remove barriers to care for airmen experiencing more common precipitants of suicide among military members such as relationship problems, legal problems, or financial hardship⁹ and may prevent minor symptoms and issues that are unrelated or indirectly related to combat from escalating.^11,37

In summary, early detection and treatment of an array of common precipitants of suicide, such as addressing relationship, financial, and legal problems, are key components of a public health model of suicide prevention.⁵⁴ Although war is usually regarded as a serious stressor for military troops, some individuals may perceive it in a very positive and challenging way.⁵⁵ Individuals who have more support safety nets are not only protected from the potentially damaging impact of stressful events but also appear to experience fewer negative events.⁵⁶ This study lends important support to this observation. Moreover, early access to care for mental health and psychosocial problems, which may be facilitated by the PDHRA process, appears to play an important part in mitigating psychosocial risk factors for suicide. Taken together, our findings contribute to the overall observation of the expected effects of a public health–focused intervention, albeit brought about through a clinically focused assessment. To our knowledge, this study is the first demonstration of a screening and assessment instrument, the PDHRA, used in a military population that, because of its low specificity, meets one of the key tenets of a public health approach to suicide prevention. It underscores the need, from a population-based point of view, to assess for the presence of a mental health concern rather than the severity of the diagnosis. Most important, the observation of an inverse relationship between deployment and suicide among airmen is critical, given the abundance of media attention paid to this topic. We provide the first evidence of how an instrument designed to be a clinical triage tool resulted in effects that would be desirable in any program using a public health approach to suicide prevention. As such, it may provide future guidance in developing interventions and programs for reducing deaths from suicide in both military and veteran populations.