Development and cross-validation of a veterans mental health risk factor screen

Eve B Carlson; Patrick A Palmieri; Dawne Vogt; Kathryn Macia; Steven E Lindley

doi:10.1371/journal.pone.0272599

. 2023 Feb 8;18(2):e0272599. doi: 10.1371/journal.pone.0272599

Development and cross-validation of a veterans mental health risk factor screen

Eve B Carlson ^1,^2,^*, Patrick A Palmieri ³, Dawne Vogt ^4,⁵, Kathryn Macia ^1,², Steven E Lindley ^2,⁶

Editor: Chong Chen⁷

PMCID: PMC9907813 PMID: 36753482

Abstract

Background

VA primary care patients are routinely screened for current symptoms of PTSD, depression, and alcohol disorders, but many who screen positive do not engage in care. In addition to stigma about mental disorders and a high value on autonomy, some veterans may not seek care because of uncertainty about whether they need treatment to recover. A screen for mental health risk could provide an alternative motivation for patients to engage in care.

Method

Data from samples of veterans and traumatic injury survivors were analyzed to identify mental health risk factors that are characteristics of individuals or stressors or of post-trauma, post-deployment, or post-military service resources, experiences, or responses. Twelve risk factors were strongly related to PTSD (r > .50): current PTSD, depression, dissociation, negative thinking, and emotional lability symptoms, life stress, relationship stress, social constraints, and deployment experiences of a difficult environment, concerns about life and family, perceived threat, and moral injury. Items assessing each of these risk factors were selected and their validity to prospectively predict PTSD and/or depression 6 months later was assessed in a new sample of 232 VA primary care patients.

Results

Twelve items assessing dissociation, emotional lability, life stress, and moral injury correctly classified 86% of those who later had elevated PTSD and/or depression symptoms (sensitivity) and 75% of those whose later symptoms were not elevated (specificity). Performance was also very good for 110 veterans who identified as members of ethnic/racial minorities.

Conclusions

Mental health status was prospectively predicted in VA primary care patients with high accuracy using a screen that is brief, easy to administer, score, and interpret, and fits well into VA’s integrated primary care. When care is readily accessible, appealing to veterans, and not perceived as stigmatizing, information about mental health risk may result in higher rates of engagement than information about current mental disorder status.

Introduction

Military personnel who are deployed to a warzone experience high levels of stress during their service [1], and most are exposed to potentially traumatic stressors [2]. While many who have symptoms of PTSD appear to recover in the months following return from deployment [3], chronic Posttraumatic Stress Disorder (PTSD) and depression trouble a substantial number of veterans [4]. Prevalence rates for PTSD in military personnel and veterans who were deployed to Iraq or Afghanistan have been estimated to range from 5 to 20% [5], and rates of depression have been estimated to range from 5 to 37% [6]. In addition, from 1997 to 2005, the number of veterans with a diagnosis of PTSD receiving Veterans Administration (VA) specialty mental health services doubled and 87% were veterans who served in the Gulf War or earlier [7]. Given that delays in initiating treatment contributes to the chronicity of mental health problems in veterans [8], health care systems in both the Department of Defense (DoD) and VA have implemented screening programs aimed at early detection and treatment of mental health problems.

Diagnostic screening vs risk screening methods

Although both the military and VA screen for common mental health conditions in primary care [9, 10], only a minority of veterans who screen positive receive adequate mental health care [11, 12]. In one study of veterans screened in a primary care system that integrated behavioral health care with primary medical care and employed a proactive referral system, less than half of veterans who screened positive for depression and/or PTSD attended mental health intake sessions or received any psychotherapy in the following 18 months [13]. Poor predictive performance of mental health screens and disconnection between perceived needs and the focus of screening may both be contributing to this lack of engagement in mental health care.

A systematic review of research on post-deployment mental health screening that uses a diagnostic model has found high variability in sensitivity of screens (.46 to .83) and low to moderate treatment initiation rates following screening [10]. This means that positive diagnostic screens are not consistently accurate in identifying those in need of care. Many with positive screens may recover on their own without mental health care. About 50% of those who screen positive for PTSD, major depression, alcohol misuse, or other mental health problems when they return from deployment to a war zone appear to recover without treatment within three to six months [3]. Thus, depending on the timing and circumstances of screening, positive diagnostic screening for current mental health problems may not be a good indicator of risk for persisting mental health problems.

Another contributor to veterans disinclination to accept mental health services may be that information given to them about their current status in of a mental illness diagnosis does not seem relevant or actionable. Among many systemic and individual factors that can be barriers to care [14–20], veterans’ reluctance to engage in mental health care may be the greatest. Qualitative and quantitative research has found that many veterans expect their problems to resolve without treatment, have a strong desire to solve problems without help, and hold beliefs that discourage engagement in care [21, 22]. Specifically, many have general attitudes of mistrust of others and negative attitudes about mental illness diagnostic labels, about standard models of mental health care offered by VA and about those who develop mental health problems and those who seek care [12, 16, 22, 23]. Information about current symptoms of a mental illness diagnosis may not be sufficient to overcome these barriers and persuade some veterans that there is a pressing need to engage in mental health care.

Screening veterans for risk of future mental health problems may address predictive accuracy and relevance problems associated with diagnostic screening. Information about risk for later mental health problems may seem more relevant and actionable to veterans than current diagnostic information. Those at risk could be told that veterans with risk screen scores like theirs did not recover on their own. Research has shown that risk information about outcomes of concern to patients can play a critical role in determining health-related behaviors. For example, in a large RCT of primary care patients, participants who completed a screen for cancer risk and received generic risk information or information about their personal colorectal cancer risk subsequently completed colorectal cancer medical tests at three times the rate of those who did not complete a risk screen and received no risk information [24]. Our success in developing a multi-risk mental health screening tool for hospital patients that performed well to predict later mental health problems [25] encouraged us to pursue a similar approach for veterans who were primary care patients.

Theory and research on risk factors for posttraumatic and postdeployment mental health problems in veterans

This study was based on theoretical frameworks for the impact of traumatic stress [26, 27] and deployment stress [28] and research on risk factors for PTSD and depression following traumatic stress [29]. The etiologies of post-traumatic and post-deployment mental health problems are complicated because of overlap in exposure to trauma and deployment and because a large number of factors affect people’s responses to both types of experiences. These factors include characteristics of individuals (such as genetic or biological tendencies, developmental level and experiences, past trauma exposure, life stress, and gender), characteristics of stressors (such as severity, intentional or accidental nature, and duration) and time-of-trauma factors (such as social support), and posttraumatic life experiences, responses, and resources (such as social support, availability and use of medical and psychological treatment, and posttraumatic life stress) [26, 27, 29]. These factors may influence responses to potentially traumatic stressors by affecting perceptions of stressful events, expectations about the impact of events, the capacity to cope with strong emotions related to events, or some combination of these.

Military personnel and veterans are subject to the same risk factors as those exposed to other types of traumatic stressors, and also to risks associated with serving in a war zone or deployment to other assignments inside or outside the country. Most research on veterans has focused on combat-related experiences, but other aspects of deployment may also influence health and mental health outcomes [28, 30]. These include perceived threat, aftermath of battle, difficult living and working environment, sense of preparedness, exposure to toxic agents, concerns about life and family disruptions, sexual or general harassment, social support during and after deployment, and life stress following deployment. Moral injury has also been identified as deleterious to mental health in veterans [31–35]. Moral injury has been defined as psychological damage caused by transgressions of deeply held moral or ethical beliefs [33]. More recently, moral injury has been defined as disturbances in virtues, character, and identity that follow a moral failure event, and it has been posited that the pathways of influence of moral injury and traumatic stress are independent [36]. Because traumatic stress and moral injury events often co-occur, the experience of both may be especially damaging. To date, moral injury research has focused on veterans who were treatment-seeking, so little is known about the incidence of moral injury in other populations or its value as a prospective predictor of later mental health problems.

The quality of relationships with partners, family members, friends, and co-workers during and after military deployment can also influence mental health. Military personnel deployed in recent years often served longer in war zones and have been deployed for multiple tours more often than prior military cohorts [11]. These extended periods of time apart and experiences of war zone stress and traumatic stress make the transition from deployments to the homefront stressful. Veterans of Iraq and Afghanistan have reported high levels of stress in marital, family, and work relationships during and after military service [37, 38], but there has been little examination of the predictive value of reports of stress in relationships related to deployment or transition to civilian life and later mental health.

Development and cross-validation of a veterans mental health risk screen in current project

In the current project, we sought to create a screen for risk of later mental health problems in veterans seeking primary care in a VA clinic and to perform cross-validation by examining the predictive performance of selected risk factor items in a new sample [39]. We focused on risk factors that are thought to play causal roles in the development of mental health problems in order to identify factors that are clinically relevant and modifiable. In the Screen Development Study, we conducted secondary analyses of data from prior research to identify which risk factors are most strongly related to PTSD and/or depression and to select small subsets of items to measure these risk factors. In the Cross-Validation Study, we collected data in a new sample of VA primary care patients (VA Primary Care II) on the selected risk factor items and on PTSD and/or depression six months later, and we calculated the performance of the risk screen results to prospectively predict these mental health problems [40, 41].

Screen development study: Introduction

We sought to identify a set of risk factors that are strongly related to PTSD and depression by conducting secondary analyses of data from published studies of risk factors and mental health problems and new data on a novel risk factor and mental health status in VA primary care patients. For each of the most strongly-related risk factors, we then identified subsets of items to measure the risk factor by examining which items were most strongly related to total scores on the full risk measures, conducting factor analyses to determine which items represented the underlying constructs well, and conducting forward regressions to determine which items collectively accounted for at least 90% of the variance in total scores.

We chose datasets that were diverse in terms of time since exposure to trauma or deployment because veterans who seek VA primary care are similarly diverse in regard to the time that has passed since their exposure to deployment or traumatic stress. When available, we included datasets that included prospectively collected data on variables that were strongly related to later mental health outcomes. To examine risk factors that would relate to long-term mental health sequelae of traumatic and deployment stress in a broad sample of veterans that was not limited to those seeking mental health care, we included data from a diverse and nationally representative group of Gulf War veterans who had been deployed ten years earlier [28]. To examine risk factors that would relate to short-term mental health sequelae of traumatic and deployment stress in veterans who were not all seeking health or mental health care, we included a dataset that was comprised of military personnel who served in Iraq or Afghanistan and were assessed upon return from deployment and three to six months later [42]. To examine risk factors related to mental health sequelae of traumatic stress that were not available in a military or veteran sample, we included a dataset that was comprised of patients and family members exposed to traumatic injury who were assessed within two weeks of trauma and two months later [25]. To examine the associations between symptoms and relationships related to deployment and transition to civilian life following deployment, we collected new data in a sample of veterans seeking VA primary care services. We focused on symptoms of PTSD, which are common among veterans receiving VA mental health care [43].

Screen development study: Methods

The Administrative Panel on Human Subjects in Medical Research of Stanford University, the designated IRB for VA Palo Alto Health Care System research, approved this study as protocol 17192. For secondary datasets (Gulf War Veterans, Iraq and Afghanistan Military Personnel, Residential Veterans, VA Residential Veterans VA Homeless Domiciliary Residents, and Traumatic Injury Survivors) deidentified data were studied and consent was obtained from participants at the time the original research was conducted. For new data collected for the VA Primary Care I and VA Primary Care II samples, oral informed consent was obtained by participants.

Gulf War veterans participants, procedures and measures

Data from Gulf War veterans were collected as part of a large national study that had the primary aim to develop a measure of deployment risk and resilience [44]. Potential participants were identified through records held by the Defense Manpower Data Center (DMDC), a central repository for Department of Defense personnel data and the VA Gulf War Health Registry. These veterans were deployed to the 1990–1991 Gulf War conflict (Gulf War I), and the data used were collected approximately ten years after deployment. Complete data were obtained from 317 (64%) of 495 who were sent questionnaires by mail. A detailed description of the study methods is provided in King (2006). Risk and resilience factors were assessed using the Deployment Risk and Resilience Inventory (DRRI) [44]. The DRRI is an collection of 14 scales for assessing predeployment, deployment, and postdeployment psychosocial factors that put veterans at risk for poor post-deployment health and adjustment. Posttraumatic stress symptoms were assessed with the military PTSD Checklist (PCL-M) [45].

Iraq and Afghanistan (IA) military personnel participants, procedures and measures

Data from military personnel recently returned from deployment to Iraq or Afghanistan were collected as part of a study of mental and physical health status and health services use [42]. Potential participants were identified through the DMDC. A sample of 2,000 / (OEF/OIF) personnel who had returned from deployment to Iraq (Operation Iraqi Freedom) or Afghanistan (Operation Enduring Freedom) between 3 and 12 months prior was stratified by service component (50% Active, 25% National Guard, and 25% other Reserve) and gender (>50% women within each service). Of 1,833 eligible individuals, 1,043 (57%) received the initial survey, and 596 (57%) returned completed initial (T1) surveys in the time frame allowed. Survey respondents came from every state except Wyoming and from the District of Columbia, Puerto Rico, and the US Virgin Islands. Follow-up data (T2) were collected from 512 veterans (86% of the 596 who completed the T1 survey) between six and nine months following receipt of the T1 survey [46]. Risk and resilience factors were assessed at T1 with the DRRI [44] (described above). Symptoms at T1 and T2 were assessed with the PCL-M (described above) and the Behavior and Symptom Identification Scale (BASIS-24) [47]. The BASIS-24 uses 24 items to assess six symptom domains: depression/functioning, interpersonal relationships, psychotic symptoms, alcohol/drug use, emotional lability, and self-harm.

Traumatic injury survivors participants, procedures and measures

Data from traumatic injury survivors were collected as part of research on psychological responses in patients hospitalized after severe, sudden injury and family members of severely injured patients [25]. Most risk factors, including early symptoms, were assessed upon enrollment between one and fourteen days after injury in 147 participants (54% patients and 46% family members). Social support and social constraints were assessed between eight and twenty-one days after injury in 64 participants. PTSD symptoms assessed two months post-injury were available for 129 participants. There were no significant differences between patients and family members on any risk factor or any outcome. PTSD was assessed with the Screen for Posttraumatic Stress Symptoms (SPTSS) [48, 49]. In psychiatric inpatients, scores on the SPTSS showed strong internal consistency (α = 0.91) and good concurrent validity when correlated with other measures of PTSD [48]. In a study of recent combat veterans with a 15% rate of PTSD, scores on the SPTSS had a sensitivity of 0.89 and specificity of 0.89 to predict PTSD diagnosis from the SCID PTSD module [49].

VA residential veterans participants, procedures and measures

Data were collected from 240 Vietnam War veterans participating in a U.S. Veterans Health Administration residential treatment program for chronic and severe PTSD related to combat or other trauma exposure during military service. Data were collected at the time of admission as part of research on the psychometric properties of a measure of dissociation [50]. Data included in this study were scores on the PCL-M (described above) and the Dissociative Symptoms Scale (DSS). The DSS assesses dissociative symptoms, which are distortions in perceptions, attention, and memory that are strongly related to exposure to traumatic stress and to PTSD symptoms [50]. Scores on the DSS in this sample showed strong reliability and validity in Carlson et al. [50].

VA homeless domiciliary residents participants, procedures and measures

Data were collected from 115 veterans participating in a U.S. Veterans Health Administration residential program for homeless veterans. Data were collected as part of research on military and civilian trauma exposure and posttraumatic symptoms [51]. Data included in this study were scores on the PCL-M and the DSS (both described above). Findings of strong reliability and validity for scores on the DSS in this sample are reported in Carlson et al. [50].

VA primary care (I) participants, procedures, and measures

Data were collected from 131 VA primary care patients at a VA medical center. In order to limit the potential for errors in reports of past experiences due to the passage of time, inclusion was limited to veterans who were within five years of military discharge. Appointments were identified in the electronic medical records for primary care, laboratory services, audiology, dental, optometry, orthopedics, gastro-intestinal, and hand and upper extremity. While in the waiting area, veterans who did not opt out were invited to participate and provided informed consent. Data reported here included demographics, employment, and military service.

Stress in relationships related to deployment experiences or transition to civilian life was assessed in the 118 veterans who had been deployed during military service using a measure of stress in relationships called the Deployment Transitions Stress Scale (DTSS) that was developed to quantify stress in marital/partner and family relationships related to deployment and return from deployment and about changes in relationships with friends and in the work place since deployment. The DTSS was based on the Transitioning Families Questionnaire, an unpublished clinical interview authored by Erika Curran that was used to collect relationship information from VA residential PTSD treatment program patients and their spouses. The DTSS includes items focused on stress in the relationship with a spouse or partner, family members, friends, and work. Example items include “Compared to before deployment, how many disagreements do you and your partner have now? (fewer, about the same, somewhat more, a lot more); “Since returning, how hard is it to talk or answer questions about your war experiences with your children or other family members?” (not at all hard, a little bit hard, somewhat hard, very hard); “How much have you felt alienated from your friends since returning from deployment?” (not at all, a little bit, somewhat, very much); “How hard has it been for you to get along with coworkers in a new workplace since returning from deployment?” (not at all, a little bit, somewhat, very hard). Data on Relationship Stress: Spouse/Partner were available for the 32 veterans who were still in relationships with the same spouse/partner as at the time of deployment. Data on Relationship Stress: Friends were available for N = 104 of 118 who had deployed during military service. Cronbach’s alpha value for the total DTSS was 0.93 and values for the subscales were 0.76 (spouse/partner), 0.64 (family); 0.64 (friends), and 0.82 (work).

The SPTSS (described above) was used to assess DSM-IV PTSD symptoms.

Data analysis

We used common methods for shortening scales [52–54] to select subsets of items with the goal of creating very brief risk factor measures that could predict mental health outcomes. Creating short forms of hypothetical constructs that produce scores with strong psychometric properties typically involves shortening measures of 30 or more items to about 12 items [53]. However, a screen made up of a collection of typical short forms would be too long to use for screening in health care settings. Further, since some of the most predictive risk factors (e.g., PTSD symptoms) are multifactorial [55], combining short forms that assessed all of the underlying factors of these multi-factor risks would likewise produce a multi-risk screen that was too long for practical use. For these reasons, we did not seek to create brief measures of the entire content domains of risks or produce scores with psychometric properties strong enough to serve as independent measures of the constructs. Instead, we sought to identify a set of risks that could be assessed with relatively few items and demonstrate good predictive validity of future mental health status.

Some risk factor variables we studied are hypothetical constructs (e.g., social support), whereas others are composite measures made up of items that represent discrete, possibly uncorrelated experiences (e.g., combat experiences) [56]. To select items for brief measures of risk factors that are hypothetical constructs, we examined item-total correlations and favored items that were most highly correlated with total scores [52]. We also conducted factor analyses to examine whether risk measures included only one or multiple factors, and eliminated lower-loading items by factor. To shorten measures of risk factors that are composite variables, we relied primarily on removing items that related less strongly to total scores. For all risk factors (constructs and sets of experiences), in order to maximize criterion-related validity with respect to the original full scales, we conducted forward regressions with items as predictors of the score for the full risk factor measure and retained enough items to account for at least 90% of the variance in the full measure score. We used data from the IA Military sample to select items for brief measures of some deployment-related variables rather than data from the Gulf War sample because the former was a much larger sample and the data were collected prospectively and over a comparable time frame between risks and PTSD symptoms to be predicted.

Screen development study: Results and discussion

The number of participants, demographic characteristics, and service information for the samples studied are shown in Table 1. In the VA Primary Care (I) sample, the mean time since return from most recent deployment was 27 months (sd = 20.5). Table 2 shows the risk factors and symptoms assessed in each study, the measures used to assess them, and correlations between risk factors and outcomes in each dataset.

Table 1. Characteristics of the samples.

	Gulf War (N = 317)	IA^a Military (N = 512)	Injury (N = 129)	VA PTSD Resident (N = 240)	VA Homeless (N = 115)	VA Primary Care I (N = 131)	VA Primary Care II^b (N = 232)
Gender: Male	74%	40%	42%	89%	96%	86%	85%
Age: Mean (SD)	44 (9.0)	34 (9.2)	44 (14.1)	52 (5.6)	45 (6.3)	33 (9.5)	44 (16.2)
Hispanic Ethnicity	14%	13%	18%	13%	4%	17%	23%
Race
White	74%	76%	74%	57%	46%	66%	62%
Black	15%	17%	4%	22%	47%	7%	13%
Other or multiple	11%	7%	22%	8%	3%	27%	35%
Marital Status
Single	7%	21%	20%	7%	35%	46%	35%
Married/Partner	74%	67%	56%	33%	4%	26%	40%
Separated/Divorced	18%	12%	24%	56%	57%	28%	24%
Education
High school/GED	7%	18%	20%	-	-	15%	-
Some college/voc ed	52%	51%	31%	-	-	55%	-
Bachelors or more	41%	31%	47%	-	-	30%	-
Employment	-	83%	-	-	-	60%	-
Service Type
Active	26%	80%	-	-	-	70%	88%
National Guard/Reserve	74%	20%	-	-	-	30%	12%
Deployed	100%	100%	-	-	-	85%	81%

Open in a new tab

Notes

^aIA = Iraq and Afghanistan

^bVA Primary Care II sample was collected as part of the Cross-Validation Study.

Table 2. Risk factors assessed, measures used, and correlations between risk factors and PTSD symptoms in six samples.

Risk Factor	Measure	Gulf War Vets (N = 317)	IA Military Post-Deploy (N = 512)	Traumatic Injury (N = 129)	VA PTSD Residential (N = 240)	VA Homeless Domiciliary (N = 115)	VA Primary Care I (N = 104)
Pre-trauma Individual Characteristics
Education	-	-.31	-.25
Prior Stressors	DRRI-A		.37
Past High Magnitude Stressors	THS			.35
Childhood Family Environment	DRRI-B	-.06^a	-.25
Childhood Home Life	-			-.34
Preparedness for Deployment	DRRI-C	-.23	-.34
Perceived Life Stress (Pre-trauma)	PSS			.46
Stressor Characteristics and Time-of-Trauma Factors
Difficult Living and Working Environment	DRRI-D	.54^a	.48
Concerns about Life and Family Disruptions	DRRI-E	.46^a	.40
Deployment Social Support	DRRI-F	-.34^a	-.38
General Harassment	DRRI-G1	.44^a	.40
Sexual Harassment	DRRI-G2	.29^a	.32
Perceived Threat	DRRI-H	.53^a	.53
Combat Experiences	DRRI-I	.40^a	.45
Aftermath of Battle	DRRI-J	.43^a	.44
Nuclear, Biological, & Chemical Exposure	DRRI-K	.45^a	.35
Length of time in the military		-.14^a
Applied for service disability		.43^a
Post-Trauma Experiences, Responses and Resources
Post-deployment Social Support	DRRI-L	-.22^a	-.53
Post-deployment Stressors	DRRI-M	.41^a	.33
Relationship Stress: Spouse	DRRS						.80^a^, ^b
Relationship Stress: Friends	DRRS						.73^a^, ^c
Perceived Life Stress (Post-trauma)	PSS			.70
Social Constraints	SCS			.59^d
Social Support	SSS			-.42^d
General Self-Efficacy	GSES		-.39
PTSD Symptoms	PCL-M		.79
PTSD Symptoms	SPTSS			.62
Dissociation	DSS			.52^a	.46^a	.73^a
Negative Thinking	PTCI			.57
Depression	BDI-SF			.50
Depression/Anxiety score	BASIS-R		.61
Psychosis (Interpersonal Threat)	BASIS-R		.55
Emotional Lability	BASIS-R		.52
Interpersonal Problems	BASIS-R		.45
Substance Abuse	BASIS-R		.31
Days in past month with medical problems			.40
Alcohol Use	AUDIT	.21

Open in a new tab

Notes

^aRisk factor and outcome data collected at same time point.

^bData on Relationship Stress: Spouse/Partner were available for N = 32 who were still in relationships with the same spouse/partner as at the time of deployment.

^cData on Relationship Stress: Friends was available for N = 104 of 118 who had deployed during military service.

^dData on post-trauma Social Constraints and Social Support were available for N = 64 participants. AUDIT = Alcohol Use Disorders Test, BASIS-R = Behavior Symptom Identification Scale–Revised, BDI-SF = Beck Depression Inventory–Short Form, DRRI = Deployment Risk and Resilience Inventory, DSS = Dissociative Symptoms Scale, DTSS = Deployment and Transitions Stress Scale, GSES = General Self-Efficacy Scale, IA = Iraq and Afghanistan, PCL-M = Posttraumatic Stress Checklist–Military, PSS = Perceived Stress Scale, PTCI = Posttraumatic Cognitions Inventory, SCS = Social Constraints Scale, SSS = Social Support Survey, SPTSS = Screen for Posttraumatic Stress Symptoms, THS = Trauma History Screen

The number of items selected for each risk factor and the correlation between total scores on brief and full measures of constructs in the Screen Development Study are shown in Table 3. These correlations were generally quite high with most over r = .85, but were not as high as r = .95 suggested for creation of short forms [57].

Table 3. Number of Items in brief measures and correlations between brief and full measures.

Risk Factor	Number of Items	IA Military Post-Deploy (N = 512)	VA PTSD Resident & Homeless (N = 355)	Traumatic Injury (N = 129)	VA Primary Care-I (N = 131)
PTSD Symptoms	5				.95
Deployment & Transition Stress: Spouse/Partner	2				.86^a
Deployment & Transition Stress: Friends	1				.80
Depression	3	.94
Social Constraints	2			.90
Negative Thinking	4			.91
Interpersonal Threat	1	.86
Deployment Environment	4	.86
Deployment Concerns	3	.88
Dissociation	5		.96
Emotional Lability	1	.87
Life & Family Concerns	5	.85
Perceived Life Stress	2			.92

Open in a new tab

Notes

^aData on Deployment and Relationship Stress: Spouse/Partner available for N = 32 veterans.

^bData for VA PTSD Resident and Homeless samples were combined for this analysis because the veterans in the samples are similar and using a larger sample is better to select items for the brief measure of dissociation. IA = Iraq and Afghanistan.

Cross-validation study: Introduction

In the Cross-validation Study, we recruited a new sample of VA primary care patients (VA Primary Care II) to study the predictive performance of risk factor items selected in the Screen Development Study on symptoms of PTSD and depression six months later. We included symptoms of depression because they are so common among veterans receiving VA mental health care [43]. We used elevation in symptoms rather than diagnosis as the criterion, because diagnosis of mental disorder is highly stigmatizing [58], and effective implementation of a risk screen would require a more patient-centered approach. Use of self-report measures rather than diagnostic interviews also results in study samples that more closely reflect the intended population because a more representative sample can be enrolled and a higher proportion are retained than if a long diagnostic interview were required.

Given that predicting future mental health status is the goal of the screen, we examined the predictive or prospective validity of multiple risk factors to predict later mental health [52]. We collected data on the risk factors listed in Table 3 and on moral injury from VA primary care patients and data on symptoms of PTSD and depression six months later. We conducted analyses to: 1. Confirm that, in the new sample, the risk factors assessed by brief measures related strongly to PTSD and depression symptoms six months later. 2. Determine the subset of risk factors that most strongly predicted PTSD and depression status six months later. 3. Examine the performance of that set of risk factors to prospectively predict later elevated PTSD and/or depression symptoms. 4. Examine the predictive performance of the subset of risk factors in the subsample of veterans who identify as belonging to an ethnic or racial minority identity.

Cross-validation study: Methods

Participants and procedures

Patients were recruited for the cross-validation study using the same methods and procedures as described above in the VA Primary Care I sample. At the time of enrollment, participants completed risk factor measures. Six months following enrollment, participants completed mental health measures and reported on major life stressors since enrollment.

Measures

The risk factors identified in the Screen Development Study (shown in Table 3) were assessed with the items selected in the Screen Development Study. To make them applicable to all veterans, simplify wording, and combine very similar constructs, some items were modified. For example, items originally focused on deployment were reworded to “During military service,” and items originally specifying spouses or partners were modified to apply to those who did not have partners or spouses (e.g., refer to “your relationships” instead of “partner or spouse”). For all items, a simple, uniform set of response options was used of “not at all”, “a little bit”, “some”, “a lot” with risk factor scores coded to have higher scores reflect less favorable experiences or higher levels of symptoms.

Moral injury was also assessed with four novel items developed in consultation with four experts on moral injury. All members of the expert panel had doctoral level training in psychiatry or psychology and lived experience of military service and had published and presented on the concept of moral injury. After discussion about the construct of moral injury, four items were developed to assess the construct: “been bothered about things that happened during military service that were just not right”; “felt bad about yourself because of things you saw or did during military service”; “felt guilty about things you did or didn’t do while in the military”; and “been bothered by killing people during military service”. When administered, items were preceded by “In the past month, how much have you…” and response options were “not at all”, “a little bit”, “some”, “a lot”.

The SPTSS (described above) was used to assess PTSD symptoms six months after enrollment. SPTSS scores of 20 or higher were used to categorize participants as having elevated PTSD symptoms. This was based on strong performance of scores on the SPTSS to predict PTSD diagnosis by the Structured Clinical Interview for DSM-IV (SCID) in a sample of veterans (N = 317; SE = .89; SP = .89) [49] and by performance of scores of 20 or more on the SPTSS to predict PTSD diagnosis on the Clinician-Administered PTSD Scale (CAPS) in a sample of adults assessed two months after traumatic injury of themselves or a close family member (N = 40, SE = .90, SP = .80) [40].

The Patient Health Questionnaire—9 (PHQ-9) was used to assess depression symptoms over the past two weeks. The PHQ-9 has 9 items with 4 response options that range from “not at all” to “nearly every day”. The PHQ-9 has strong internal consistency (α = .89), test-retest reliability (r = .84), and correlations with other measures of depression severity and excellent SE (.95) and SP (.84) to a depression diagnosis [59, 60]. PHQ-9 scores of 10 or more were used to categorize participants as having elevated depression based on Veterans Health Administration/ Department of Defense guidelines for assessment of depression in primary care [61].

To identify veterans who experienced new traumatic stressors between the time of enrollment and the follow-up assessment six months later, the follow-up assessment included the question “Since you completed the first part of this study (about 6 months ago), have you had any major life events or new major stressors?” and descriptions of the events/stressors were collected for those who answered “yes”. These descriptions were used to identify veterans who had been exposed to potentially traumatic stressors.

Data analysis

Potentially traumatic stressors and multiple major stressors experienced in the six months following enrollment were expected to increase mental health symptoms and affect accuracy of prediction for veterans expected to have no disorder at six months. Therefore, data were excluded from analyses for four participants whose screen results predicted no disorder at six months and who reported major stressors between enrollment and follow-up that were considered potentially traumatic. All analyses reported here were conducted on the 228 participants who completed the follow-up and reported no new major life events or stressors that were considered likely to be traumatic stressors.

The sum of standardized scores on the SPTSS and the PHQ-9 was used as an index of mental health that reflected symptoms of both PTSD and depression. To determine the subset of risk factors that most strongly predicted elevation in combined PTSD and depression symptoms six months later, we conducted a forward regression to predict the index of mental health. The statistical software analyzed the variance in the outcome (index of mental health six months after screening) associated with all risk factors listed in Table 3 and added risk factors to the model in a stepwise fashion. At each step, the risk factor was added that provides the best improvement to the model and contributes significant additional variance with set at p < .05. We conducted a Receiver Operating Characteristic (ROC) analysis to identify a cut score on the subset of risk factors that maximized SE to predict mental health status without letting SP fall below 0.70. We examined the performance of the cut score to predict elevated mental health symptoms six months later.

Cross-validation study: Results

A sample of 284 veterans were enrolled and completed the risk measures. Symptoms measures were completed six months later by 232 (82%) of those enrolled. The demographic characteristics and service information for the cross-validation sample (VA Primary Care II) are shown in Table 1. The median length of time enrolled for health care at the VA facility which was the study site was five months; 65% had enrolled within the past year, and 82% had enrolled within the past two years.

Table 4 shows the number of items in brief measures of each risk factor, Cronbach’s alpha values in the new sample for brief risk factors that are constructs, and correlations between risk factors measured at the time of enrollment and PTSD and depression symptoms assessed six months later. As noted above, we only included data from those who reported no new potentially traumatic stressors during the follow-up period. While PTSD and depression assessed six months after risk factors were highly related (r = .83), the risks most strongly related to each were not the same.

Table 4. Correlations in cross-validation study between scores for selected risk factor item sets at enrollment and PTSD and depression symptoms 6 month later.

Risk Factor	Number of Items	Cronbach’s alpha	r with PTSD (N = 228)	r with depression (N = 228)
PTSD Symptoms	5	.89	.73***	.64***
Relationship Stress	3	-	.62***	.56***
Depression	3	.83	.67***	.64***
Social Constraints	2	.79	.68***	.59***
Negative Thinking	4	.88	.73***	.66***
Interpersonal Threat	1	-	.56***	.54***
Difficult Living and Working Environment	4	-	.23***	.14*
Perceived Threat	3	-	.45***	.34***
Dissociation	5	.87	.77***	.66***
Emotional Lability	1	-	.67***	.64***
Concerns about Life and Family Disruptions	5	-	.38***	.37***
Perceived Life Stress	2	.82	.69***	.66***
Moral Injury	4	.84	.62***	.56***

Open in a new tab

Note

*** = p < .0001

* = p < .05

A forward regression to predict the combined index of PTSD and depression six months after enrollment yielded R = .80 (df = 4, 208; p < .001) with risk factors of dissociation, emotional lability, life stress, and moral injury in the model. Table 5 shows the classification performance of these four risk factors assessed by 12 items to predict elevations in the combined index of PTSD and depression, in PTSD, and in depression. In patients reporting an ethnic/racial minority identity (n = 110), performance of a screen with 5 risks (17 items) to predict elevated PTSD and/or depression correctly classified 76% with SE = .90, SP = .66.

Table 5. Classification performance of screen including four risk factors to predict elevations in a combined index of PTSD and depression, PTSD, and depression.

Risks Assessed (items per risk)	Criterion	Positive if ≥	% Accurately Classified	SE	SP
Dissociation (5) Emotional Lability (1) Life Stress (2) Moral Injury (4)	Elevation Combined PTSD and Depression	9	79%	.86	.75
	Elevation PTSD Symptoms	9	78%	.93	.70
	Elevation Depression Symptoms	9	76%	.86	.71

Open in a new tab

Cross-validation study: Discussion

This two-part study was designed to create a screen for risk of later mental health problems in veterans seeking primary care in VA. Analyses identified risk factors that were strongly related to PTSD symptoms in samples of veterans, military personnel recently returned from deployment, adults recently exposed to traumatic injury, veterans in residential treatment programs, and veteran primary care patients. Items were selected to assess the most predictive risk factors, and data on risks and mental health six months later were collected for a new sample of veterans. Twelve items assessing dissociation, emotional lability, life stress, and moral injury showed strong predictive validity with high sensitivity and specificity for later elevated levels of PTSD and depression. Sensitivity was also high for veterans who identified as belonging to one or more ethnic or racial groups, but specificity was lower.

Performance of this screen was comparable to that of similar mental health risk screens. Total scores for the 18 items assessing four risks on the Hospital Mental Health Risk Screen showed a sensitivity of .86 and a specificity of .72 [40]. Total scores for the 18 items assessing 4 risks on the Military Mental Health Risk Screen showed a sensitivity of .80 and a specificity of .86 [41]. Availability of this accurate screen for mental health risk is only the first step to increase engagement of at-risk veterans in mental health care. Further research is needed on the impact of mental health risk information in combination with readily accessible and appealing options to reduce risk.

A strength of the study is that data from five samples of military veterans were analyzed to select the most predictive risk factors to study. Recommended psychometric methods were used to select subsets of items to measure the risk factors, and the brief risk factor measures demonstrated strong internal consistency in a new sample. Inclusion of moral injury among the risk factors studied made it possible to determine that this difficult type of experience [31] makes a unique predictive contribution. While variables such as PTSD, depression, negative thinking and social constraints at the time of screening were more strongly related to PTSD six months later, those variables covaried with risk factors already in the model (dissociation, emotional lability, and life stress) whereas moral injury accounted for unique variance. It is also a major strength that the capacity of the risk factors to prospectively predict later mental health was studied in a new sample. Lastly, we were able to recruit a very diverse sample and to retain 82% of participants, which allowed evaluation of screen performance in patients identifying with one or more ethnic or racial minority identities.

A limitation of the study is that insufficient data were collected to select brief measure items using Item Response Theory methods, which are more advanced methods that indicate the amount of information about underlying constructs provided by items [62]. However, measures developed based on classical test theory are often very similar in quality to those developed based on IRT methods. Lastly, since many risk factors were strongly predictive of later PTSD and depression, many different sets of risk factors would accurately predict later mental health.

Conclusions

Mental health status was prospectively predicted in veterans new to VA primary care with high accuracy using a screen that is brief, easy to administer, score, and interpret, clinically feasible in a variety of settings, and fits well into VA’s primary care which integrates medical and mental health care [63]. These findings should be replicated to confirm the predictive validity of screen scores. Use of a predictive screen could foster prevention and early intervention research. Future research might investigate whether providing veterans with information about their mental health risk results in higher rates of engagement than providing information about their current mental health status. Risk information seems likely to have the greatest impact when care is readily accessible, appealing to veterans, and not perceived as stigmatizing.

Acknowledgments

The contents of this article do not represent the views of the US Department of Veterans Affairs or the US Government.

The authors wish to thank the following experts for their input during the design of this research: Captain Paul S. Hammer, M.D., Dr. Jonathan Shay, Colonel Carl Castro, Ph.D., Neil Greenberg, M.D., and William Nash, M.D.

Data Availability

All data files are available from the Dryad database here: https://datadryad.org/stash/dataset/doi:10.5061%2Fdryad.tb2rbp03g.

Funding Statement

E.B.C. and P.A.P. were funded with a Merit Grant by the Clinical Services Research and Development program of the Department of Veterans Affairs. https://www.research.va.gov/services/csrd/default.cfm The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

1.Vogt D., Smith B. N., King L. A., King D. W., Knight J., and Vasterling J. J., “Deployment Risk and Resilience Inventory-2 (DRRI-2): An updated tool for assessing psychosocial risk and resilience factors among service members and veterans,” Journal of Traumatic Stress, vol. 26, no. 6, pp. 710–717, Dec. 2013, doi: 10.1002/jts.21868 [DOI] [PubMed] [Google Scholar]
2.Hoge C. W., Castro C. A., Messer S. C., McGurk D., Cotting D. I., and Koffman R. L., “Combat duty in Iraq and Afghanistan, mental health problems, and barriers to care,” New England Journal of Medicine, vol. 351, no. 1, pp. 13–22, 2004, doi: 10.1056/NEJMoa040603 [DOI] [PubMed] [Google Scholar]
3.Milliken C. S., Auchterlonie J. L., and Hoge C. W., “Longitudinal assessment of mental health problems among active and reserve component soldiers returning from the Iraq war,” Journal of the American Medical Association, vol. 298, no. 18, pp. 2141–2148, Nov. 2007, doi: 10.1001/jama.298.18.2141 [DOI] [PubMed] [Google Scholar]
4.Ramchand R., Rudavsky R., Grant S., Tanielian T., and Jaycox L., “Prevalence of, risk factors for, and consequences of posttraumatic stress disorder and other mental health problems in military populations deployed to Iraq and Afghanistan,” Curr Psychiatry Rep, vol. 17, no. 5, p. 37, May 2015, doi: 10.1007/s11920-015-0575-z [DOI] [PubMed] [Google Scholar]
5.Ramchand R., Schell T. L., Karney B. R., Osilla K. C., Burns R. M., and Caldarone L. B., “Disparate prevalence estimates of PTSD among service members who served in Iraq and Afghanistan: Possible explanations,” J. Traum. Stress, vol. 23, no. 1, pp. 59–68, Feb. 2010, doi: 10.1002/jts.20486 [DOI] [PubMed] [Google Scholar]
6.Tanielian T. L., “Invisible wounds of war: Psychological and cognitive injuries, their consequences, and services to assist recovery,” RAND Center for Military Health Policy Research, Santa Monica, CA, 2008. [Google Scholar]
7.Rosenheck R. A. and Fontana A. F., “Recent trends In VA treatment of post-traumatic stress disorder and other mental disorders,” Health Affairs, vol. 26, pp. 1720–1727, 2007, doi: 10.1377/hlthaff.26.6.1720 [DOI] [PubMed] [Google Scholar]
8.Maguen S., Madden E., Neylan T. C., Cohen B. E., Bertenthal D., and Seal K. H., “Timing of Mental Health Treatment and PTSD Symptom Improvement Among Iraq and Afghanistan Veterans,” PS, vol. 65, no. 12, pp. 1414–1419, Dec. 2014, doi: 10.1176/appi.ps.201300453 [DOI] [PubMed] [Google Scholar]
9.V. Department of Veterans Affairs, VHA Handbook 1160.01: Uniform Mental Health Services in VA Medical Centers and Clinics. Washington, DC: Department of Veterans Veterans Health Administration, 2008. [Google Scholar]
10.Panaite V. et al. , “Post-deployment Mental Health Screening: A Systematic Review of Current Evidence and Future Directions,” Adm Policy Ment Health, vol. 45, no. 6, pp. 850–875, Nov. 2018, doi: 10.1007/s10488-018-0869-7 [DOI] [PubMed] [Google Scholar]
11.Institute of Medicine, “Treatment for posttraumatic stress disorder in military and veteran populations: Final assessment,” National Academies Press, Washington, DC, 2014. [PubMed] [Google Scholar]
12.Brown M. C., Creel A. H., Engel C. C., Herrell R. K., and Hoge C. W., “Factors Associated With Interest in Receiving Help for Mental Health Problems in Combat Veterans Returning From Deployment to Iraq,” J Nerv Ment Dis, vol. 199, no. 10, pp. 797–801, Oct. 2011. doi: 10.1097/NMD.0b013e31822fc9bf [DOI] [PubMed] [Google Scholar]
13.Lindley S. E., Cacciapaglia H., Noronha D., Carlson E. B., and Schatzberg A., “Monitoring mental health treatment acceptance and initial treatment adherence in OEF/OIF versus other era veterans,” Annals of the New York Academy of Sciences, vol. 1208, pp. 104–113, 2010, doi: 10.1111/j.1749-6632.2010.05692.x [DOI] [PubMed] [Google Scholar]
14.Lu M. W., Plagge J. M., Marsiglio M. C., and Dobscha S. K., “Clinician Documentation on Receipt of Trauma-Focused Evidence-Based Psychotherapies in a VA PTSD Clinic,” J Behav Health Serv Res, vol. 43, no. 1, pp. 71–87, Jan. 2016, doi: 10.1007/s11414-013-9372-9 [DOI] [PubMed] [Google Scholar]
15.Harpaz-Rotem I., Rosenheck R. A., Pietrzak R. H., and Southwick S. M., “Determinants of Prospective Engagement in Mental Health Treatment Among Symptomatic Iraq/Afghanistan Veterans,” Journal of Nervous & Mental Disease, vol. 202, no. 2, pp. 97–104, Feb. 2014, doi: 10.1097/NMD.0000000000000078 [DOI] [PubMed] [Google Scholar]
16.Ouimette P. et al. , “Perceived barriers to care among Veterans Health Administration patients with posttraumatic stress disorder,” Psychological Services, vol. 8, no. 3, pp. 212–223, 2011, doi: 10.1037/a0024360 [DOI] [Google Scholar]
17.Pietrzak R. H., Johnson D. C., Goldstein M. B., Malley J. C., and Southwick S. M., “Perceived Stigma and Barriers to Mental Health Care Utilization Among OEF-OIF Veterans,” PS, vol. 60, no. 8, pp. 1118–1122, Aug. 2009, doi: 10.1176/ps.2009.60.8.1118 [DOI] [PubMed] [Google Scholar]
18.Possemato K., Wray L. O., Johnson E., Webster B., and Beehler G. P., “Facilitators and Barriers to Seeking Mental Health Care Among Primary Care Veterans With Posttraumatic Stress Disorder,” JOURNAL OF TRAUMATIC STRESS, vol. 31, no. 5, pp. 742–752, Oct. 2018, doi: 10.1002/jts.22327 [DOI] [PubMed] [Google Scholar]
19.Sayer N. A. et al. , “A Qualitative Study of Determinants of PTSD Treatment Initiation in Veterans,” Psychiatry: Interpersonal and Biological Processes, vol. 72, no. 3, pp. 238–255, Sep. 2009, doi: 10.1521/psyc.2009.72.3.238 [DOI] [PubMed] [Google Scholar]
20.Vogt D., “Mental Health-Related Beliefs as a Barrier to Service Use for Military Personnel and Veterans: A Review,” PS, vol. 62, no. 2, pp. 135–142, Feb. 2011, doi: 10.1176/ps.62.2.pss6202_0135 [DOI] [PubMed] [Google Scholar]
21.Straits-Troster K., Gierisch J., Calhoun P. S., Strauss J. L., Voils C., and Kudler H., “Living in transition: Young veterans’ health and the postdeployment shift to family life,” in Treating young veterans: Promoting resilience through practice and advocacy, Diann Kelly, Sydney Barksdale, and David Gitelson, Eds.New York: Springer, 2011. [Google Scholar]
22.Vogt D., Di Leone B. A. L., Wang J. M., Sayer N. A., Pineles S. L., and Litz B. T., “Endorsed and Anticipated Stigma Inventory (EASI): A tool for assessing beliefs about mental illness and mental health treatment among military personnel and veterans.,” Psychological Services, vol. 11, no. 1, pp. 105–113, 2014, doi: 10.1037/a0032780 [DOI] [PubMed] [Google Scholar]
23.Arbisi P. A., Rusch L., Polusny M. A., Thuras P., and Erbes C. R., “Does cynicism play a role in failure to obtain needed care? Mental health service utilization among returning U.S. National Guard soldiers,” Psychological Assessment, vol. 25, no. 3, pp. 991–996, 2013, doi: 10.1037/a0032225 [DOI] [PubMed] [Google Scholar]
24.Skinner C. S. et al. , “Impact of Risk Assessment and Tailored versus Nontailored Risk Information on Colorectal Cancer Testing in Primary Care: A Randomized Controlled Trial,” Cancer Epidemiol Biomarkers Prev, vol. 24, no. 10, pp. 1523–1530, Oct. 2015, doi: 10.1158/1055-9965.EPI-15-0122 [DOI] [PMC free article] [PubMed] [Google Scholar]
25.Carlson E. B., Palmieri P. A., Field N. P., Dalenberg C. J., Macia K. S., and Spain D. A., “Contributions of risk and protective factors to prediction of psychological symptoms after traumatic experiences,” Comprehensive Psychiatry, vol. 69, pp. 106–115, Aug. 2016, doi: 10.1016/j.comppsych.2016.04.022 [DOI] [PMC free article] [PubMed] [Google Scholar]
26.Carlson E. B., Dalenberg C. J., and Muhtadie L., “The etiology of Posttraumatic Stress Disorder,” in The Encyclopedia of Psychological Trauma, Reyes G., Elhai J., and Ford J. D., Eds. New York: Wiley, 2008, pp. 257–264. [Google Scholar]
27.Carlson E. B. and Dalenberg C. J., “A conceptual framework for the impact of traumatic experiences,” Trauma, Violence, & Abuse, vol. 1, no. 1, pp. 4–28, 2000. [Google Scholar]
28.King L. A., King D. W., Bolton E. E., Knight J. K., and Vogt D., “Risk factors for mental, physical, and functional health in Gulf War veterans,” Journal of Rehabilitation Research and Development, vol. 45, pp. 395–408, 2008, doi: 10.1682/jrrd.2007.06.0081 [DOI] [PubMed] [Google Scholar]
29.Brewin C. R., Andrews B., and Valentine J. D., “Meta-analysis of risk factors for posttraumatic stress disorder in trauma-exposed adults,” J Consult Clin Psychol, vol. 68, no. 5, pp. 748–766, Oct. 2000, doi: 10.1037//0022-006x.68.5.748 [DOI] [PubMed] [Google Scholar]
30.Vogt D., Samper R. E., King D. W., King L. A., and Martin J. A., “Deployment stressors and posttraumatic stress symptomatology: Comparing active duty and National Guard/Reserve personnel from Gulf War I,” Journal of Traumatic Stress, vol. 21, pp. 66–74, Feb. 2008, doi: 10.1002/jts.20306 [DOI] [PubMed] [Google Scholar]
31.Nash W. P., “Commentary on the Special Issue on Moral Injury: Unpacking Two Models for Understanding Moral Injury,” Journal of Traumatic Stress, pp. 465–470, 2019. doi: 10.1002/jts.22409 [DOI] [PubMed] [Google Scholar]
32.Nash W. P. and Litz B. T., “Moral injury: A mechanism for war-related psychological trauma in military family members,” Clin Child Fam Psychol Rev, vol. 16, no. 4, pp. 365–375, Dec. 2013, doi: 10.1007/s10567-013-0146-y [DOI] [PubMed] [Google Scholar]
33.Litz B. T. et al. , “Moral injury and moral repair in war veterans: A preliminary model and intervention strategy,” Clinical Psychology Review, vol. 29, no. 8, pp. 695–706, Dec. 2009, doi: 10.1016/j.cpr.2009.07.003 [DOI] [PubMed] [Google Scholar]
34.Frankfurt S. B., Frazier P., and Engdahl B., “Indirect relations between transgressive acts and general combat exposure and moral Injury,” Military Medicine, vol. 182, no. 11, pp. e1950–e1956, Nov. 2017, doi: 10.7205/MILMED-D-17-00062 [DOI] [PubMed] [Google Scholar]
35.Clewell R. D., “Moral dimensions in treating combat veterans with posttraumatic stress disorder.,” Bulletin of the Menninger Clinic, vol. 51, no. 1, pp. 114–130, Jan. 1987. [PubMed] [Google Scholar]
36.Atuel H. R. et al. , “Understanding moral injury from a character domain perspective.,” Journal of Theoretical and Philosophical Psychology, vol. 41, no. 3, pp. 155–173, Aug. 2021, doi: 10.1037/teo0000161 [DOI] [Google Scholar]
37.Straits-Tröster K., Kudler H., and Calhoun P., “Executive summary report on focus groups: Clinical needs assessment of Operation Iraqi Freedom (OIF) and Operation Enduring Freedom (OEF) Veterans and Families,” 2007. [Google Scholar]
38.Vogt D. et al. , “Predeployment, deployment, and postdeployment risk factors for posttraumatic stress symptomatology in female and male OEF/OIF veterans.,” Journal of Abnormal Psychology, vol. 120, no. 4, pp. 819–831, Nov. 2011, doi: 10.1037/a0024457 [DOI] [PubMed] [Google Scholar]
39.Dawis R., “Scale construction and psychometric considerations,” in Handbook of Applied Multivariate Statistics and Mathematical Modeling, Tinsley H. E. and Brown S. D., Eds. New York: Academic Press, 2000. [Google Scholar]
40.Carlson E. B., Palmieri P. A., and Spain D. A., “Development and preliminary performance of a risk factor screen to predict posttraumatic psychological disorder after trauma exposure,” General Hospital Psychiatry, vol. 46, pp. 25–31, 2017, doi: 10.1016/j.genhosppsych.2016.12.011 [DOI] [PMC free article] [PubMed] [Google Scholar]
41.Palmieri P. A., Carlson E. B., Eisen S., Vogt D., Harrington E. F., and Reiland S. A., “Predicting chronic mental health problems in military personnel following deployment,” Philadelphia, PA, 2013. [Google Scholar]
42.Eisen S. V. et al. , “Mental and physical health status and alcohol and drug use following return from deployment to Iraq or Afghanistan,” American Journal of Public Health, vol. 102, no. SUPPL. 1, pp. S66–S73, 2012, doi: 10.2105/AJPH.2011.300609 [DOI] [PMC free article] [PubMed] [Google Scholar]
43.Trivedi R. B. et al. , “Prevalence, Comorbidity, and Prognosis of Mental Health Among US Veterans,” Am J Public Health, vol. 105, no. 12, pp. 2564–2569, Dec. 2015, doi: 10.2105/AJPH.2015.302836 [DOI] [PMC free article] [PubMed] [Google Scholar]
44.King L. A., King D. W., Vogt D., Knight J. K., and Samper R. E., “Deployment Risk and Resilience Inventory: A collection of measures for studying deployment-related experiences of military personnel and veterans,” Military Psychology, vol. 18, pp. 89–120, 2006, doi: 10.1207/s15327876mp1802_1 [DOI] [Google Scholar]
45.Weathers F. W., Litz B. T., Herman D. S., Huska J. A., and Keane T. M., “The PTSD Checklist (PCL): Reliability, validity, and diagnostic utility,” San Antonio, TX, 1993. [Google Scholar]
46.Schultz M., Glickman M. E., and Eisen S. V., “Predictors of decline in overall mental health, PTSD and alcohol use in OEF/OIF veterans,” Comprehensive Psychiatry, vol. 55, no. 7, pp. 1654–1664, Oct. 2014, doi: 10.1016/j.comppsych.2014.06.003 [DOI] [PubMed] [Google Scholar]
47.Eisen S. V., Gerena M., Ranganathan G., Esch D., and Idiculla T., “Reliability and validity of the BASIS-24© Mental Health Survey for Whites, African-Americans, and Latinos,” J Behav Health Serv Res, vol. 33, no. 3, pp. 304–323, Jun. 2006, doi: 10.1007/s11414-006-9025-3 [DOI] [PubMed] [Google Scholar]
48.Carlson E. B., “Psychometric study of a brief screen for PTSD: Assessing the impact of multiple traumatic events,” Assessment, vol. 8, no. 4, pp. 431–441, Dec. 2001, doi: 10.1177/107319110100800408 [DOI] [PubMed] [Google Scholar]
49.Caspi Y., Carlson E. B., and Klein E., “Validation of a screening instrument for posttraumatic stress disorder in a community sample of Bedouin men serving in the Israeli Defense Forces,” Journal of Traumatic Stress, vol. 20, pp. 529–539, 2007, doi: 10.1002/jts.20237 [DOI] [PubMed] [Google Scholar]
50.Carlson E. B., Waelde L. C., Palmieri P. A., Macia K. S., Smith S. R., and McDade-Montez E., “Development and validation of the Dissociative Symptoms Scale,” Assessment, vol. 25, no. 1, pp. 84–98, 2018, doi: 10.1177/1073191116645904 [DOI] [PubMed] [Google Scholar]
51.Carlson E. B., Garvert D. W., Macia K. S., Ruzek J. I., and Burling T. A., “Traumatic stressor exposure and post-traumatic symptoms in homeless veterans,” Military Medicine, vol. 178, pp. 970–973, 2013, doi: 10.7205/MILMED-D-13-00080 [DOI] [PubMed] [Google Scholar]
52.DeVellis R. F., Scale development: Theory and applications, 2nd ed. Thousand Oaks, CA: Sage, 2003. [Google Scholar]
53.Goetz C. et al. , “Item reduction based on rigorous methodological guidelines is necessary to maintain validity when shortening composite measurement scales,” Journal of Clinical Epidemiology, vol. 66, no. 7, pp. 710–718, Jul. 2013, doi: 10.1016/j.jclinepi.2012.12.015 [DOI] [PubMed] [Google Scholar]
54.Loose K., Hou Y., and Berinsky A. J., “Achieving Efficiency Without Losing Accuracy: Strategies for Scale Reduction with an Application to Risk Attitudes and Racial Resentment,” Social Science Quarterly, vol. 99, no. 2, 2018. [Google Scholar]
55.Elhai J. D. and Palmieri P. A., “The factor structure of posttraumatic stress disorder: A literature update, critique of methodology, and agenda for future research,” Journal of Anxiety Disorders, vol. 25, no. 6, pp. 849–854, Aug. 2011, doi: 10.1016/j.janxdis.2011.04.007 [DOI] [PubMed] [Google Scholar]
56.Vogt D., King D. W., and King L. A., “Focus groups in psychological assessment: Enhancing content validity by consulting members of the target population,” Psychological Assessment, vol. 16, no. 3, pp. 231–43, Sep. 2004. doi: 10.1037/1040-3590.16.3.231 [DOI] [PubMed] [Google Scholar]
57.Harel D. and Baron M., “Methods for shortening patient-reported outcome measures,” Statistical methods in medical research, vol. 28, pp. 2992–3011, 2019. doi: 10.1177/0962280218795187 [DOI] [PubMed] [Google Scholar]
58.Acosta J. D., Mental health stigma in the military. Santa Monica, CA: RAND, 2014. [Google Scholar]
59.Cameron I. M., Crawford J. R., Lawton K., and Reid I. C., “Psychometric comparison of PHQ-9 and HADS for measuring depression severity in primary care,” British Journal of General Practice, vol. 58, pp. 32–36, 2008, doi: 10.3399/bjgp08X263794 [DOI] [PMC free article] [PubMed] [Google Scholar]
60.Kroenke K. and Spitzer R. L., “The PHQ-9: A new depression diagnostic and severity measure,” Psychiatric Annals, vol. 32, pp. 1–7, 2002. [Google Scholar]
61.V. Department of Veterans Affairs, “VA/DoD Clinical Practice Guideline for the Management of Major Depressive Disorder,” Department of Veterans Veterans Health Administration, Washington, DC, 2016. [Google Scholar]
62.Edelen M. O. and Reeve B. B., “Applying item response theory (IRT) modeling to questionnaire development, evaluation, and refinement,” Qual Life Res, vol. 16, no. S1, pp. 5–18, Aug. 2007, doi: 10.1007/s11136-007-9198-0 [DOI] [PubMed] [Google Scholar]
63.Post E. P. and Van Stone W. W., “Veterans Health Administration Primary Care-Mental Health Integration Initiative,” North Carolina Medical Journal, vol. 69, no. 1, 2008. [PubMed] [Google Scholar]

PLoS One. doi: 10.1371/journal.pone.0272599.r001

Decision Letter 0

Renan C Castillo

18 Mar 2022

PONE-D-21-33139Development and cross-validation of a veterans mental health risk factor screenPLOS ONE

Dear Dr. Carlson,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by May 02 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Renan C. Castillo

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments (if provided):

I would like to commend you on an interesting and well executed study. The reviewers had only minor revisions for you and I agree with their recommendations. As you can see from their reviews, the major substantive item is the request for a better description/justification of the rationale for the tool and the development of these specific items. I would encourage the authors to pay particular attention to these items in their response.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for your work on this manuscript and the opportunity to review it. On the whole, I considered the content interesting and your presentation very detailed. I recognize this was a lot to present in a single paper and therefore quite a bit of effort, and in that light, I would like to offer a few thoughts about the manuscript overall.

I thought the amount of detail was at times overwhelming, particularly leading up to the sections of the manuscript that spoke directly to the new work being presented. Overall, I recognize the importance in laying out this additional background as it is the foundation for the Screen Development and Cross Validation Studies, but I that some of the landscape and prior studies might have been more easily digested with the presentation of some of this information in either a tabular format or perhaps with a figure that laid out the overall flow of the work. This wasn’t a fatal flaw in my opinion, but I did feel the need to be extra careful in my reading to properly follow along.

Narrowing my comments to specific items in your manuscript, there are several points where additional clarification might be beneficial for the reader.

1. I’m not sure that I fully understood the motivation for developing this particular new screening method. Lines 141-145 indicate that you modeled the process for developing the current tool after those used for two other successful screening tools. My best understanding of the need for a new screener was to have something appropriate to the primary care clinic setting. If that is the case, I would recommend noting that more explicitly and/or what the improvement being sought here is.

2. Line 395 was the first time it became very clear about what the “new” work was in terms of the development and presentation of new data. Because I read the manuscript first, I did not carry an impression of what to expect through the reading of the manuscript that I realized later was more clearly laid out in the abstract. I thought that giving a little more information about what comprised the second half of the paper earlier in the text would have been helpful. As an example, I was first puzzled why the VA Primary Care-II cohort was not included in Tables 2 and 3 but it had been included with Table 1. Only after getting toward the end did I fully appreciate that this was the new cohort being described.

3. Line 48: It might be nice to include the range here to support the “high variability” noted here.

4. Line 54: References cited could be reflected as 12-18 unless there is a typographical error here.

5. Line 108-110: This sentence might benefit by adding a reference.

6. Lines 235-237, Tables 1, 2 and 3: There are 3 different numbers associated with the VA Primary Care-I cohort: 114 (line 235), 131 (table 1) and 104 (tables 2 and 3). It is not clear what these differences are or why they are presented with different denominators.

7. Lines 318-319: It appears this sentence is inadvertently cut short.

8. Line 332: One iteration of the word “identity” should be “identify.”

9. Line 380: Is the phrase, “who reported major stressors...” supposed to be, “who reported no major stressors...”?

10. Line 381-383: This sentence is important to understanding why Table 4 does not include 232 patients, so it may be better to locate it closer to the tables and the mention of 232 patients (line 395).

11. Table 3: Does not include the Gulf War cohort. Is there a reason for this?

12. Table 3: For clarity, it might be useful to note that the VA PTSD Resident & Homeless cohort is the combination of two cohorts presented separately in Tables 1 and 2? Is there a reason these are combined here but not elsewhere?

13. Table 4: Dissociation is listed as having 4 items here but having 5 items in tables 3 and 5.

14. Table 5: The caption notes 5 risk factors but only lists 4 in the table.

15. There is a mix of citation styles used in the manuscript that should be unified.

16. Abbreviations that are not specified in the text and tables include:

a. PTSD: Line 29

b. VA: Line 33

c. VHA: Line 36

d. SPTSS: Line 214

e. IA (presumably Iraq/Afghanistan): Tables 1, 2, and 3

Thank you again for your work on this manuscript.

Reviewer #2: This is a well-done and important study. I have only a few comments for the authors to consider.

1) The authors make the case that 'moral injury' is a uniquely predictive risk factor for PTSD and Depression and developed 4 novel items to assess moral injury. Because these items were developed de novo for this study, I would recommend that the authors further substantiate their decision to include these items instead of the 'next best' predictive risk factor, e.g., depression or social constraints. I also suggest a bit more description of the construct and/or noting the 4 assessment items developed by the panel since this construct is less well-understood than the other risk factors.

2) The authors note that a limitation of the study is that the population in the cross-validation study is proportionately younger than the population served by the VA in general. It would be helpful for the reader to know to what degree the authors expect that this difference would impact the external validity of their results, i.e., how big of a deal is this? Does this have implications for future studies to confirm the risk factor assessment?

3) On line 319, there appears to be a word missing from the end of the sentence, "...a more patient centered ____". I think this is missing the word "model" or "approach".

4) This line (323-324) is a worded a little confusingly: "We examined the predictive or prospective validity of multiple risk factors as this is the form of criterion-related validity that is relevant when predicting future status is the goal".

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2023 Feb 8;18(2):e0272599. doi: 10.1371/journal.pone.0272599.r002

Author response to Decision Letter 0

17 May 2022

Additional Editor Comments:

***Thanks! Will do! Responses to Reviewer comments are marked by *** below.

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

________________________________________

3. Have the authors made all data underlying the findings in their manuscript fully available?

Reviewer #1: No

Reviewer #2: No

***We have changed our Data Availability Statement to address this issue:

Datasets for the Screen Development Study and the Cross-Validation Study used to calculate results reported in this publication will be shared via Dryad with identifying information removed.

________________________________________

***We appreciate that the amount of detail is a lot to process. Rather than adding one more table or figure, we thought it best to eliminate the paragraphs describing prior research and replace them with sentence at the end of the 4th paragraph of the introduction that mentions the development of multi-risk mental health risk screening tool for hospital patients that preceded this study. See lines 78-80 of the revised MS.***

Narrowing my comments to specific items in your manuscript, there are several points where additional clarification might be beneficial for the reader.

***Sorry we didn’t make this point more effectively! We have rewritten the 2nd to 5th paragraphs of the introduction (***lines 38-80) to make the rationale clearer. The revised paragraphs emphasize that current diagnostic screening methods may fail to engage veterans into care because positive screens for current symptoms in veterans are a poor indicator of risk for persisting mental health problems and because information about current symptoms for a mental illness diagnosis may not seem relevant or actionable to veterans. Accurate information about risk for persisting mental health problems, on the other hand, may seem more relevant and actionable.***

***Thanks for this suggestion. We have revised the last two sentences of the last paragraph of the introduction (lines 127-133) to clarify the study design and specify that the Cross-Validation Sample was the Primary Care II sample. We also clarified this in the notes for Table 1, on lines 318-319, and on line 409. ***

***We made the suggested edits to address all comments below. New line numbers are noted. ***

3. Line 48: It might be nice to include the range here to support the “high variability” noted here.

*** (now on line 47). ***

4. Line 54: References cited could be reflected as 12-18 unless there is a typographical error here.

*** (corrected on line 59) ***

5. Line 108-110: This sentence might benefit by adding a reference.

*** (A reference supporting the point that recent veterans served multiple and longer tours was added on lines 116.) ***

***Apologies for failing to explain these differences! We have clarified on line 231 that data were collected from 131 veterans and on line 239 that data on stress in relationships related to deployment were collected from the 118 veterans who had been deployed during military service. On lines 254-257, we state that data on Relationship Stress: Spouse/Partner were available for 32 veterans who were still in relationships with the same spouse/partner as at the time of deployment and that data on Relationship Stress: Friends were available for 104 of the 118 veterans who were deployed during military service. ***

7. Lines 318-319: It appears this sentence is inadvertently cut short.

***We added the missing word “approach” on line 324. ***

8. Line 332: One iteration of the word “identity” should be “identify.” *** (corrected on line 337) ***

9. Line 380 (in revised manuscript on line 391-393): Is the phrase, “who reported major stressors...” supposed to be, “who reported no major stressors...”?

***The statement is correct as written. Because new stressors during the follow-up period would be expected to cause symptoms, we EXCLUDED data from those who reported major stressors that were considered potentially traumatic between enrollment and followup. Stated another way, we only included data from those who reported no new potentially traumatic stressors during the follow-up period. ***

***We added mention of the included data on line 416-417 just before Table 4. ***

11. Table 3: Does not include the Gulf War cohort. Is there a reason for this?

***We added an explanation on lines 277-280. The IA Military sample was used to select items for brief measures of deployment-related variables rather than data from the Gulf War sample because the former was a much larger sample and the data were collected prospectively and over a comparable time frame between risks and PTSD symptoms to be predicted. ***

***We added a footnote to Table 3 to explain that the data for VA PTSD Resident and Homeless samples were combined for this analysis because the veterans are very similar and using a larger sample is better to select items for the brief measure of dissociation. ***

13. Table 4: Dissociation is listed as having 4 items here but having 5 items in tables 3 and 5.

***The number in table 4 should be 5. This has been corrected. We are impressed that you noticed this detail! ***

14. Table 5: The caption notes 5 risk factors but only lists 4 in the table.

***The table name has been corrected to state four risk factors. ***

15. There is a mix of citation styles used in the manuscript that should be unified.

***This has been corrected. ***

16. Abbreviations that are not specified in the text and tables include:

a. PTSD: Line 29 ***Spelled out on line 28***

b. VA: Line 33 ***Spelled out on line 32***

c. VHA: Line 36 ***Changed all instances of VHA to VA for consistency. ***

d. SPTSS: Line 214 ***Spelled out on line 209-210. ***

e. IA (presumably Iraq/Afghanistan): Tables 1, 2, and 3 ***Spelled out in Table footnotes. ***

Thank you again for your work on this manuscript.

Reviewer #2: This is a well-done and important study. I have only a few comments for the authors to consider.

***Thanks for this comment. Your comment made me realize that a better explanation was needed of how the forward regression was conducted. We didn’t choose which variables to enter into the regression, the software did! We have added this explanation on lines 398-401 of the Data Analysis section. Given the results, we agree that the paper needs more explanation of how moral injury is defined and how it was operationalized in the study. On lines 105-111, we have expanded the explanation of moral injury and added a new reference. On lines 358-363, we have included the items used to assess moral injury. On lines 451-454, we have provided a more explicit explanation of why moral injury contributed unique variance to the prediction of later PTSD. ***

***We gave this comment a great deal of thought and realized that our original point wasn’t sound. While younger veterans who served in combat zones are not representative of all veterans who receive health care at VA, they are representative of veterans who currently present for the first time to VA for health care. For this reason, we decided it was best to omit the original point. ***

3) On line 319, there appears to be a word missing from the end of the sentence, "...a more patient centered ____". I think this is missing the word "model" or "approach".

***Yes, the missing word was “approach”! This has been corrected. ***

***We reworded this sentence on line 328-329 to improve its clarity. ***

Attachment

Submitted filename: Response to Reviewers 5-17-22.doc

Click here for additional data file.^{(61.5KB, doc)}

PLoS One. doi: 10.1371/journal.pone.0272599.r003

Decision Letter 1

Chong Chen

25 Jul 2022

Development and cross-validation of a veterans mental health risk factor screen

PONE-D-21-33139R1

Dear Dr. Carlson,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Chong Chen

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Reviewer #1: The authors are to be commended for addressing the items laid out by the reviewers. I found the paper much easier to digest and stronger as a result of the effort made. Overall, I recommend it be accepted for publication.

Reviewer #2: This is a fantastic piece of scholarship. Thank you for your contribution to the literature and the opportunity to review this work.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Lauren E. Allen, DrPH

**********

PLoS One. doi: 10.1371/journal.pone.0272599.r004

Acceptance letter

Chong Chen

6 Oct 2022

PONE-D-21-33139R1

Development and cross-validation of a veterans mental health risk factor screen

Dear Dr. Carlson:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Chong Chen

Academic Editor

PLOS ONE

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Attachment

Submitted filename: Response to Reviewers 5-17-22.doc

Click here for additional data file.^{(61.5KB, doc)}

Data Availability Statement

All data files are available from the Dryad database here: https://datadryad.org/stash/dataset/doi:10.5061%2Fdryad.tb2rbp03g.

[pone.0272599.ref001] 1.Vogt D., Smith B. N., King L. A., King D. W., Knight J., and Vasterling J. J., “Deployment Risk and Resilience Inventory-2 (DRRI-2): An updated tool for assessing psychosocial risk and resilience factors among service members and veterans,” Journal of Traumatic Stress, vol. 26, no. 6, pp. 710–717, Dec. 2013, doi: 10.1002/jts.21868 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref002] 2.Hoge C. W., Castro C. A., Messer S. C., McGurk D., Cotting D. I., and Koffman R. L., “Combat duty in Iraq and Afghanistan, mental health problems, and barriers to care,” New England Journal of Medicine, vol. 351, no. 1, pp. 13–22, 2004, doi: 10.1056/NEJMoa040603 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref003] 3.Milliken C. S., Auchterlonie J. L., and Hoge C. W., “Longitudinal assessment of mental health problems among active and reserve component soldiers returning from the Iraq war,” Journal of the American Medical Association, vol. 298, no. 18, pp. 2141–2148, Nov. 2007, doi: 10.1001/jama.298.18.2141 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref004] 4.Ramchand R., Rudavsky R., Grant S., Tanielian T., and Jaycox L., “Prevalence of, risk factors for, and consequences of posttraumatic stress disorder and other mental health problems in military populations deployed to Iraq and Afghanistan,” Curr Psychiatry Rep, vol. 17, no. 5, p. 37, May 2015, doi: 10.1007/s11920-015-0575-z [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref005] 5.Ramchand R., Schell T. L., Karney B. R., Osilla K. C., Burns R. M., and Caldarone L. B., “Disparate prevalence estimates of PTSD among service members who served in Iraq and Afghanistan: Possible explanations,” J. Traum. Stress, vol. 23, no. 1, pp. 59–68, Feb. 2010, doi: 10.1002/jts.20486 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref006] 6.Tanielian T. L., “Invisible wounds of war: Psychological and cognitive injuries, their consequences, and services to assist recovery,” RAND Center for Military Health Policy Research, Santa Monica, CA, 2008. [Google Scholar]

[pone.0272599.ref007] 7.Rosenheck R. A. and Fontana A. F., “Recent trends In VA treatment of post-traumatic stress disorder and other mental disorders,” Health Affairs, vol. 26, pp. 1720–1727, 2007, doi: 10.1377/hlthaff.26.6.1720 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref008] 8.Maguen S., Madden E., Neylan T. C., Cohen B. E., Bertenthal D., and Seal K. H., “Timing of Mental Health Treatment and PTSD Symptom Improvement Among Iraq and Afghanistan Veterans,” PS, vol. 65, no. 12, pp. 1414–1419, Dec. 2014, doi: 10.1176/appi.ps.201300453 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref009] 9.V. Department of Veterans Affairs, VHA Handbook 1160.01: Uniform Mental Health Services in VA Medical Centers and Clinics. Washington, DC: Department of Veterans Veterans Health Administration, 2008. [Google Scholar]

[pone.0272599.ref010] 10.Panaite V. et al. , “Post-deployment Mental Health Screening: A Systematic Review of Current Evidence and Future Directions,” Adm Policy Ment Health, vol. 45, no. 6, pp. 850–875, Nov. 2018, doi: 10.1007/s10488-018-0869-7 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref011] 11.Institute of Medicine, “Treatment for posttraumatic stress disorder in military and veteran populations: Final assessment,” National Academies Press, Washington, DC, 2014. [PubMed] [Google Scholar]

[pone.0272599.ref012] 12.Brown M. C., Creel A. H., Engel C. C., Herrell R. K., and Hoge C. W., “Factors Associated With Interest in Receiving Help for Mental Health Problems in Combat Veterans Returning From Deployment to Iraq,” J Nerv Ment Dis, vol. 199, no. 10, pp. 797–801, Oct. 2011. doi: 10.1097/NMD.0b013e31822fc9bf [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref013] 13.Lindley S. E., Cacciapaglia H., Noronha D., Carlson E. B., and Schatzberg A., “Monitoring mental health treatment acceptance and initial treatment adherence in OEF/OIF versus other era veterans,” Annals of the New York Academy of Sciences, vol. 1208, pp. 104–113, 2010, doi: 10.1111/j.1749-6632.2010.05692.x [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref014] 14.Lu M. W., Plagge J. M., Marsiglio M. C., and Dobscha S. K., “Clinician Documentation on Receipt of Trauma-Focused Evidence-Based Psychotherapies in a VA PTSD Clinic,” J Behav Health Serv Res, vol. 43, no. 1, pp. 71–87, Jan. 2016, doi: 10.1007/s11414-013-9372-9 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref015] 15.Harpaz-Rotem I., Rosenheck R. A., Pietrzak R. H., and Southwick S. M., “Determinants of Prospective Engagement in Mental Health Treatment Among Symptomatic Iraq/Afghanistan Veterans,” Journal of Nervous & Mental Disease, vol. 202, no. 2, pp. 97–104, Feb. 2014, doi: 10.1097/NMD.0000000000000078 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref016] 16.Ouimette P. et al. , “Perceived barriers to care among Veterans Health Administration patients with posttraumatic stress disorder,” Psychological Services, vol. 8, no. 3, pp. 212–223, 2011, doi: 10.1037/a0024360 [DOI] [Google Scholar]

[pone.0272599.ref017] 17.Pietrzak R. H., Johnson D. C., Goldstein M. B., Malley J. C., and Southwick S. M., “Perceived Stigma and Barriers to Mental Health Care Utilization Among OEF-OIF Veterans,” PS, vol. 60, no. 8, pp. 1118–1122, Aug. 2009, doi: 10.1176/ps.2009.60.8.1118 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref018] 18.Possemato K., Wray L. O., Johnson E., Webster B., and Beehler G. P., “Facilitators and Barriers to Seeking Mental Health Care Among Primary Care Veterans With Posttraumatic Stress Disorder,” JOURNAL OF TRAUMATIC STRESS, vol. 31, no. 5, pp. 742–752, Oct. 2018, doi: 10.1002/jts.22327 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref019] 19.Sayer N. A. et al. , “A Qualitative Study of Determinants of PTSD Treatment Initiation in Veterans,” Psychiatry: Interpersonal and Biological Processes, vol. 72, no. 3, pp. 238–255, Sep. 2009, doi: 10.1521/psyc.2009.72.3.238 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref020] 20.Vogt D., “Mental Health-Related Beliefs as a Barrier to Service Use for Military Personnel and Veterans: A Review,” PS, vol. 62, no. 2, pp. 135–142, Feb. 2011, doi: 10.1176/ps.62.2.pss6202_0135 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref021] 21.Straits-Troster K., Gierisch J., Calhoun P. S., Strauss J. L., Voils C., and Kudler H., “Living in transition: Young veterans’ health and the postdeployment shift to family life,” in Treating young veterans: Promoting resilience through practice and advocacy, Diann Kelly, Sydney Barksdale, and David Gitelson, Eds.New York: Springer, 2011. [Google Scholar]

[pone.0272599.ref022] 22.Vogt D., Di Leone B. A. L., Wang J. M., Sayer N. A., Pineles S. L., and Litz B. T., “Endorsed and Anticipated Stigma Inventory (EASI): A tool for assessing beliefs about mental illness and mental health treatment among military personnel and veterans.,” Psychological Services, vol. 11, no. 1, pp. 105–113, 2014, doi: 10.1037/a0032780 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref023] 23.Arbisi P. A., Rusch L., Polusny M. A., Thuras P., and Erbes C. R., “Does cynicism play a role in failure to obtain needed care? Mental health service utilization among returning U.S. National Guard soldiers,” Psychological Assessment, vol. 25, no. 3, pp. 991–996, 2013, doi: 10.1037/a0032225 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref024] 24.Skinner C. S. et al. , “Impact of Risk Assessment and Tailored versus Nontailored Risk Information on Colorectal Cancer Testing in Primary Care: A Randomized Controlled Trial,” Cancer Epidemiol Biomarkers Prev, vol. 24, no. 10, pp. 1523–1530, Oct. 2015, doi: 10.1158/1055-9965.EPI-15-0122 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0272599.ref025] 25.Carlson E. B., Palmieri P. A., Field N. P., Dalenberg C. J., Macia K. S., and Spain D. A., “Contributions of risk and protective factors to prediction of psychological symptoms after traumatic experiences,” Comprehensive Psychiatry, vol. 69, pp. 106–115, Aug. 2016, doi: 10.1016/j.comppsych.2016.04.022 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0272599.ref026] 26.Carlson E. B., Dalenberg C. J., and Muhtadie L., “The etiology of Posttraumatic Stress Disorder,” in The Encyclopedia of Psychological Trauma, Reyes G., Elhai J., and Ford J. D., Eds. New York: Wiley, 2008, pp. 257–264. [Google Scholar]

[pone.0272599.ref027] 27.Carlson E. B. and Dalenberg C. J., “A conceptual framework for the impact of traumatic experiences,” Trauma, Violence, & Abuse, vol. 1, no. 1, pp. 4–28, 2000. [Google Scholar]

[pone.0272599.ref028] 28.King L. A., King D. W., Bolton E. E., Knight J. K., and Vogt D., “Risk factors for mental, physical, and functional health in Gulf War veterans,” Journal of Rehabilitation Research and Development, vol. 45, pp. 395–408, 2008, doi: 10.1682/jrrd.2007.06.0081 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref029] 29.Brewin C. R., Andrews B., and Valentine J. D., “Meta-analysis of risk factors for posttraumatic stress disorder in trauma-exposed adults,” J Consult Clin Psychol, vol. 68, no. 5, pp. 748–766, Oct. 2000, doi: 10.1037//0022-006x.68.5.748 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref030] 30.Vogt D., Samper R. E., King D. W., King L. A., and Martin J. A., “Deployment stressors and posttraumatic stress symptomatology: Comparing active duty and National Guard/Reserve personnel from Gulf War I,” Journal of Traumatic Stress, vol. 21, pp. 66–74, Feb. 2008, doi: 10.1002/jts.20306 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref031] 31.Nash W. P., “Commentary on the Special Issue on Moral Injury: Unpacking Two Models for Understanding Moral Injury,” Journal of Traumatic Stress, pp. 465–470, 2019. doi: 10.1002/jts.22409 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref032] 32.Nash W. P. and Litz B. T., “Moral injury: A mechanism for war-related psychological trauma in military family members,” Clin Child Fam Psychol Rev, vol. 16, no. 4, pp. 365–375, Dec. 2013, doi: 10.1007/s10567-013-0146-y [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref033] 33.Litz B. T. et al. , “Moral injury and moral repair in war veterans: A preliminary model and intervention strategy,” Clinical Psychology Review, vol. 29, no. 8, pp. 695–706, Dec. 2009, doi: 10.1016/j.cpr.2009.07.003 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref034] 34.Frankfurt S. B., Frazier P., and Engdahl B., “Indirect relations between transgressive acts and general combat exposure and moral Injury,” Military Medicine, vol. 182, no. 11, pp. e1950–e1956, Nov. 2017, doi: 10.7205/MILMED-D-17-00062 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref035] 35.Clewell R. D., “Moral dimensions in treating combat veterans with posttraumatic stress disorder.,” Bulletin of the Menninger Clinic, vol. 51, no. 1, pp. 114–130, Jan. 1987. [PubMed] [Google Scholar]

[pone.0272599.ref036] 36.Atuel H. R. et al. , “Understanding moral injury from a character domain perspective.,” Journal of Theoretical and Philosophical Psychology, vol. 41, no. 3, pp. 155–173, Aug. 2021, doi: 10.1037/teo0000161 [DOI] [Google Scholar]

[pone.0272599.ref037] 37.Straits-Tröster K., Kudler H., and Calhoun P., “Executive summary report on focus groups: Clinical needs assessment of Operation Iraqi Freedom (OIF) and Operation Enduring Freedom (OEF) Veterans and Families,” 2007. [Google Scholar]

[pone.0272599.ref038] 38.Vogt D. et al. , “Predeployment, deployment, and postdeployment risk factors for posttraumatic stress symptomatology in female and male OEF/OIF veterans.,” Journal of Abnormal Psychology, vol. 120, no. 4, pp. 819–831, Nov. 2011, doi: 10.1037/a0024457 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref039] 39.Dawis R., “Scale construction and psychometric considerations,” in Handbook of Applied Multivariate Statistics and Mathematical Modeling, Tinsley H. E. and Brown S. D., Eds. New York: Academic Press, 2000. [Google Scholar]

[pone.0272599.ref040] 40.Carlson E. B., Palmieri P. A., and Spain D. A., “Development and preliminary performance of a risk factor screen to predict posttraumatic psychological disorder after trauma exposure,” General Hospital Psychiatry, vol. 46, pp. 25–31, 2017, doi: 10.1016/j.genhosppsych.2016.12.011 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0272599.ref041] 41.Palmieri P. A., Carlson E. B., Eisen S., Vogt D., Harrington E. F., and Reiland S. A., “Predicting chronic mental health problems in military personnel following deployment,” Philadelphia, PA, 2013. [Google Scholar]

[pone.0272599.ref042] 42.Eisen S. V. et al. , “Mental and physical health status and alcohol and drug use following return from deployment to Iraq or Afghanistan,” American Journal of Public Health, vol. 102, no. SUPPL. 1, pp. S66–S73, 2012, doi: 10.2105/AJPH.2011.300609 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0272599.ref043] 43.Trivedi R. B. et al. , “Prevalence, Comorbidity, and Prognosis of Mental Health Among US Veterans,” Am J Public Health, vol. 105, no. 12, pp. 2564–2569, Dec. 2015, doi: 10.2105/AJPH.2015.302836 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0272599.ref044] 44.King L. A., King D. W., Vogt D., Knight J. K., and Samper R. E., “Deployment Risk and Resilience Inventory: A collection of measures for studying deployment-related experiences of military personnel and veterans,” Military Psychology, vol. 18, pp. 89–120, 2006, doi: 10.1207/s15327876mp1802_1 [DOI] [Google Scholar]

[pone.0272599.ref045] 45.Weathers F. W., Litz B. T., Herman D. S., Huska J. A., and Keane T. M., “The PTSD Checklist (PCL): Reliability, validity, and diagnostic utility,” San Antonio, TX, 1993. [Google Scholar]

[pone.0272599.ref046] 46.Schultz M., Glickman M. E., and Eisen S. V., “Predictors of decline in overall mental health, PTSD and alcohol use in OEF/OIF veterans,” Comprehensive Psychiatry, vol. 55, no. 7, pp. 1654–1664, Oct. 2014, doi: 10.1016/j.comppsych.2014.06.003 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref047] 47.Eisen S. V., Gerena M., Ranganathan G., Esch D., and Idiculla T., “Reliability and validity of the BASIS-24© Mental Health Survey for Whites, African-Americans, and Latinos,” J Behav Health Serv Res, vol. 33, no. 3, pp. 304–323, Jun. 2006, doi: 10.1007/s11414-006-9025-3 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref048] 48.Carlson E. B., “Psychometric study of a brief screen for PTSD: Assessing the impact of multiple traumatic events,” Assessment, vol. 8, no. 4, pp. 431–441, Dec. 2001, doi: 10.1177/107319110100800408 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref049] 49.Caspi Y., Carlson E. B., and Klein E., “Validation of a screening instrument for posttraumatic stress disorder in a community sample of Bedouin men serving in the Israeli Defense Forces,” Journal of Traumatic Stress, vol. 20, pp. 529–539, 2007, doi: 10.1002/jts.20237 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref050] 50.Carlson E. B., Waelde L. C., Palmieri P. A., Macia K. S., Smith S. R., and McDade-Montez E., “Development and validation of the Dissociative Symptoms Scale,” Assessment, vol. 25, no. 1, pp. 84–98, 2018, doi: 10.1177/1073191116645904 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref051] 51.Carlson E. B., Garvert D. W., Macia K. S., Ruzek J. I., and Burling T. A., “Traumatic stressor exposure and post-traumatic symptoms in homeless veterans,” Military Medicine, vol. 178, pp. 970–973, 2013, doi: 10.7205/MILMED-D-13-00080 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref052] 52.DeVellis R. F., Scale development: Theory and applications, 2nd ed. Thousand Oaks, CA: Sage, 2003. [Google Scholar]

[pone.0272599.ref053] 53.Goetz C. et al. , “Item reduction based on rigorous methodological guidelines is necessary to maintain validity when shortening composite measurement scales,” Journal of Clinical Epidemiology, vol. 66, no. 7, pp. 710–718, Jul. 2013, doi: 10.1016/j.jclinepi.2012.12.015 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref054] 54.Loose K., Hou Y., and Berinsky A. J., “Achieving Efficiency Without Losing Accuracy: Strategies for Scale Reduction with an Application to Risk Attitudes and Racial Resentment,” Social Science Quarterly, vol. 99, no. 2, 2018. [Google Scholar]

[pone.0272599.ref055] 55.Elhai J. D. and Palmieri P. A., “The factor structure of posttraumatic stress disorder: A literature update, critique of methodology, and agenda for future research,” Journal of Anxiety Disorders, vol. 25, no. 6, pp. 849–854, Aug. 2011, doi: 10.1016/j.janxdis.2011.04.007 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref056] 56.Vogt D., King D. W., and King L. A., “Focus groups in psychological assessment: Enhancing content validity by consulting members of the target population,” Psychological Assessment, vol. 16, no. 3, pp. 231–43, Sep. 2004. doi: 10.1037/1040-3590.16.3.231 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref057] 57.Harel D. and Baron M., “Methods for shortening patient-reported outcome measures,” Statistical methods in medical research, vol. 28, pp. 2992–3011, 2019. doi: 10.1177/0962280218795187 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref058] 58.Acosta J. D., Mental health stigma in the military. Santa Monica, CA: RAND, 2014. [Google Scholar]

[pone.0272599.ref059] 59.Cameron I. M., Crawford J. R., Lawton K., and Reid I. C., “Psychometric comparison of PHQ-9 and HADS for measuring depression severity in primary care,” British Journal of General Practice, vol. 58, pp. 32–36, 2008, doi: 10.3399/bjgp08X263794 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0272599.ref060] 60.Kroenke K. and Spitzer R. L., “The PHQ-9: A new depression diagnostic and severity measure,” Psychiatric Annals, vol. 32, pp. 1–7, 2002. [Google Scholar]

[pone.0272599.ref061] 61.V. Department of Veterans Affairs, “VA/DoD Clinical Practice Guideline for the Management of Major Depressive Disorder,” Department of Veterans Veterans Health Administration, Washington, DC, 2016. [Google Scholar]

[pone.0272599.ref062] 62.Edelen M. O. and Reeve B. B., “Applying item response theory (IRT) modeling to questionnaire development, evaluation, and refinement,” Qual Life Res, vol. 16, no. S1, pp. 5–18, Aug. 2007, doi: 10.1007/s11136-007-9198-0 [DOI] [PubMed] [Google Scholar]

[pone.0272599.ref063] 63.Post E. P. and Van Stone W. W., “Veterans Health Administration Primary Care-Mental Health Integration Initiative,” North Carolina Medical Journal, vol. 69, no. 1, 2008. [PubMed] [Google Scholar]

PERMALINK

Development and cross-validation of a veterans mental health risk factor screen

Eve B Carlson

Patrick A Palmieri

Dawne Vogt

Kathryn Macia

Steven E Lindley

Roles

Abstract

Background

Method

Results

Conclusions

Introduction

Diagnostic screening vs risk screening methods

Theory and research on risk factors for posttraumatic and postdeployment mental health problems in veterans

Development and cross-validation of a veterans mental health risk screen in current project

Screen development study: Introduction

Screen development study: Methods

Gulf War veterans participants, procedures and measures

Iraq and Afghanistan (IA) military personnel participants, procedures and measures

Traumatic injury survivors participants, procedures and measures

VA residential veterans participants, procedures and measures

VA homeless domiciliary residents participants, procedures and measures

VA primary care (I) participants, procedures, and measures

Data analysis

Screen development study: Results and discussion

Table 1. Characteristics of the samples.

Table 2. Risk factors assessed, measures used, and correlations between risk factors and PTSD symptoms in six samples.

Table 3. Number of Items in brief measures and correlations between brief and full measures.

Cross-validation study: Introduction

Cross-validation study: Methods

Participants and procedures

Measures

Data analysis

Cross-validation study: Results

Table 4. Correlations in cross-validation study between scores for selected risk factor item sets at enrollment and PTSD and depression symptoms 6 month later.

Table 5. Classification performance of screen including four risk factors to predict elevations in a combined index of PTSD and depression, PTSD, and depression.

Cross-validation study: Discussion

Conclusions

Acknowledgments

Data Availability

Funding Statement

References

Decision Letter 0

Renan C Castillo

Roles

Author response to Decision Letter 0

Decision Letter 1

Chong Chen

Roles

Acceptance letter

Chong Chen

Roles

Associated Data

Supplementary Materials

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases