Analysis of a national response to a White House directive for ending veteran suicide

Abstract

Objective

Analyze responses to a national request for information (RFI) to uncover gaps in policy, practice, and understanding of veteran suicide to inform federal research strategy.

Data source

An RFI with 21 open‐ended questions generated from Presidential Executive Order #1386, administered nationally from July 3 to August 5, 2019.

Study design

Semi‐structured, open‐ended responses analyzed using a collaborative qualitative and text‐mining data process.

Data extraction methods

We aligned traditional qualitative methods with natural language processing (NLP) text‐mining techniques to analyze 9040 open‐ended question responses from 722 respondents to provide results within 3 months. Narrative inquiry and the medical explanatory model guided the data extraction and analytic process.

Results

Five major themes were identified: risk factors, risk assessment, prevention and intervention, barriers to care, and data/research. Individuals and organizations mentioned different concepts within the same themes. In responses about risk factors, individuals frequently mentioned generic terms like “illness” while organizations mentioned specific terms like “traumatic brain injury.” Organizations and individuals described unique barriers to care and emphasized ways to integrate data and research to improve points of care. Organizations often identified lack of funding as barriers while individuals often identified key moments for prevention such as military transitions and ensuring care providers have military cultural understanding.

Conclusions

This study provides an example of a rapid, adaptive analysis of a large body of qualitative, public response data about veteran suicide to support a federal strategy for an important public health topic. Combining qualitative and text‐mining methods allowed a representation of voices and perspectives including the lived experiences of individuals who described stories of military transition, treatments that worked or did not, and the perspective of organizations treating veterans for suicide. The results supported the development of a national strategy to reduce suicide risks for veterans as well as civilians.

What is known on this topic

• Suicide is a leading cause of death in the United States, and veterans are at high risk for suicide.

• Veterans not receiving care within the veteran affairs (VA) health care system are at a higher risk of suicide than the general population.

• Veterans have unique risk factors for suicide including combat exposure and deployment conditions that can exacerbate physical and mental stress.

What this study adds

• Findings from the analyses identified risk factors, assessments, and prevention strategies that are currently known, but also identified unique barriers to care, and ways in which data and research could be better aligned to improve outcomes.

• Specifically, veterans need a greater connection to the community from the point of military service transition, and health care professionals who understand and/or have experience with military culture.

• The findings emphasize that metrics for evidence‐based risk prevention and treatment methods need to incorporate veteran perspectives to ensure suicide prevention efforts address veteran needs and concerns.

1. INTRODUCTION

The objective of this study was to analyze a large body of text responses collected as a federal request for information (RFI) to uncover gaps in policy and practice across the United States for addressing and preventing veteran suicide. RFIs allow government, human service providers, and other stakeholders to identify relevant community challenges to support designing policy and program responses to address those challenges. This study aimed to identify themes and uncover what is yet not understood about veteran suicide from data collected from community members throughout the country who responded to the RFI and had personal or professional experience with veteran health and preventing suicide. The study was conducted as part of the response to Presidential Executive Order #13861, to address the rising, pressing need of finding and implementing effective interventions to reduce veteran suicide. ¹ The Veteran Affairs Clinical Science Research and Development Service (VA Research) used the RFI and the results of this study to uncover community needs that might be overlooked in current suicide prevention and intervention efforts.

Crafting health policy that is evidence‐based and grounded in the needs of community stakeholders is a critical, yet daunting task. Engaging the input of individuals with personal experiences in specific areas of health has been shown to improve clinical practices if those experiences are collected and recognized, ² and to improve the implementation of policies that support improved outcomes. ³ Stakeholder research for policy makers typically requires a combined effort of congressional support for the work, researchers to conduct the effort, and policy writers to take the results and put them into meaningful action. This article is an example of just such an effort.

1.1. Veteran suicide

Suicide is the 10th leading cause of death in the United States and has been identified as a major national public health issue due to remarkable increases over the past two decades. ⁴ , ⁵ This holds true for U.S. military service members and veterans as well as the general population. Rates of military suicide were significantly lower than the general population prior to 9/11 conflicts, while rates of veteran suicide were comparable to the overall U.S. population until after the post 9/11 conflicts when rates of suicide among both active duty service members and veterans began to increase dramatically after 2005. ⁶ , ⁷ , ⁸ Veteran death rates by suicide increased from 15.9 deaths per day in 2005 to an average of 20 per day in 2019. ⁹ Currently, the suicide rate for both men and women veterans is 1.5× higher than the overall U.S. population. ⁹ After controlling for age and sex, women veteran suicide is more than twice the rate of suicide in the average U.S. population. ⁹

For veterans and military personnel with a history of deployment, existing research suggests that unique factors such as activities carried out during active duty, combat exposure, combat environmental stressors such as sleep deprivation, and long deployment periods may contribute to suicide risk. ⁶ , ¹⁰ Some research suggests that veterans experiencing the effects of combat or deployment stressors and traumas may experience a sense of survivor guilt, moral injury, or becoming a burden on families and community that puts them at higher risk for suicide. ⁶ Yet, suicide risk is high even for non‐deployed military personnel, especially as they reenter civilian life and individual identities shift from military personnel to becoming a veteran. ¹¹ Positively, some intervention models have been proven effective in preventing veteran suicide, and veterans enrolled in VA health care have reduced rates of suicide compared with veterans who are not service‐connected. ¹² However, treatment options are not uniformly available across health care systems and may vary dramatically from location to location. As such, identifying needs of veterans at elevated risk for suicide has become an important research and policy task to ensure the ability to address these and other, unknown gaps.

2. METHODS

2.1. Data source and sample

On July 3, 2019, the White House Office of Science and Technology Policy released a formal RFI comprised of 21 open‐ended questions to inform the National Research Strategy for the President's Roadmap to Empower Veterans and End the National Tragedy of Suicide (PREVENTS). The RFI was closed on August 5, 2019. The RFI was published as Federal Register Document Number 2019‐14138; no IRB review or consent process was required. No identifying information about respondents was collected. Responses could be submitted online or by email. Respondents were asked to indicate if they responded on behalf of an “organization” or as a member of the public, which was coded within the data as “individual” responders. A total of 722 unique respondents provided answers to the RFI. Of these, 608 were individuals and 114 organizations. Of the 722 respondents, only 187 completed all 21 questions. As the nature of the RFI was only open‐ended textual responses, missing data were not flagged, but rather, every single response was counted as a “text response.” When accounting for each question answered by all respondents, the combined total corpus of qualitative data was a full 9040 separate text responses. See Table 1 for the major categories captured by the RFI.

TABLE 1.

PREVENTS request for information (RFI) major question categories

PREVENTS request for information question categories
A. How can we improve our ability to identify individual veterans and groups of veterans at greater risk of suicide?
B. How can we develop and improve individual interventions that increase overall veteran quality of life and decrease the veteran suicide rate?
C. How can we develop strategies to better ensure the latest research discoveries are translated into practical applications and implemented quickly?
D. How best to establish relevant data‐sharing protocols across Federal partners that align with community partners?
E. How should we draw upon technology to capture and use health data from nonclinical settings to advance behavioral and mental health research to the extent practicable?
F. How can we improve coordination among research efforts, prevent unnecessarily duplicative efforts, identify barriers to or gaps in research, and facilitate opportunities for improved consolidation, integration, and alignment?
G. How can we develop a public–private partnership model to foster collaborative, innovative, and effective research that accelerates these efforts?
H. Please provide any additional information not addressed by previous questions that is crucial to the creation, implementation, and success of a national research strategy to improve the coordination, monitoring, benchmarking, and execution of public‐ and private‐sector research related to the factors that contribute to service member and veteran suicide.

Abbreviation: PREVENTS, The President's Roadmap to Empower Veterans and End the National Tragedy of Suicide.

Source: Federal Register Document Number 2019‐14138.

2.2. Data extraction methods

Given the urgency of starting on policy deliberation, VA required an analysis to be completed within 3 months. A small unit of five researchers working within the Veteran Administration health system were selected to analyze the open‐ended data. The analytic team included two VA research scientists with clinical expertise in veteran health, mental health, and comorbid conditions contributing to risk for veteran suicide, two experts in machine learning and natural language processing (NLP) methods used to analyze medical data, and a medical anthropologist with expertise in rapid evaluation and mixed‐methods research to address policy (see Table 2). The volume of responses combined with the analytic timeline provided an opportunity to innovate a rapid qualitative analytic approach.

TABLE 2.

Characteristics of the research team

Qualitative researchers	Computational researchers
Lead researcher with clinical expertise—Military Registered Nurse (RN), Doctor of Philosophy (PhD) in Psychology, Veteran Affairs (VA) Career Researcher	Lead researcher with clinical expertise—Medical doctor (MD) infectious disease, Veteran Affairs (VA) career researcher
Lead qualitative analyst with a master's degree in Medical Anthropology	Computational analysts with clinical research expertise—Veteran Health Administration (VHA) data, chart abstraction
Military and veteran physical/mental health expertise	Natural language processing (NLP) and machine learning methods expertise
Mixed‐methods expertise	Health crisis response planning expertise
Embedded in the field of veteran suicide risk prevention	Embedded in the field of veteran health

The analytic team received the data in September 2019 as a corpus of text files. The team devised a technology‐assisted qualitative process to analyze the corpus of text data rapidly. The goal was to capture the unique perspectives from all respondents with the depth of qualitative methods to provide a robust, “thick description” ¹³ of all the data collected, yet at scale and speed. While rapid qualitative methodologies in health care and health systems are emerging, such as matrix analyses or rapid ethnography, these methods begin with developing data collection tools such as surveys with the analysis in mind. ¹⁴ The RFI was developed as an open‐source tool to solicit a broad range of responses. As such, the analytic plan harnessed computerized text‐mining methods in tandem with qualitative review to ensure a representative analysis of the dataset.

Prior work has shown the utility of leveraging computerized text‐mining methods such as NLP to analyze free‐text comments regarding health care. ¹⁵ , ¹⁶ , ¹⁷ , ¹⁸ , ¹⁹ , ²⁰ One of these studies observed in an analysis of over 105,000 comments about patient satisfaction that traditional approaches may require reading each text comment that would be costly in resources and time, whereas automated or semiautomated methods can scale to the volume of responses. ¹⁹ Several of these prior studies validated these automated methods with manually established reference datasets to measure their performance for reliability and validity. ¹⁵ , ¹⁶ , ¹⁹ , ²⁰ Related to suicide prevention, Cook et al ²¹ used NLP methods to predict suicidal ideation and predictors of risk for recently released patients asked to complete follow‐up questionnaires. The methods were used on open‐ended items such as “how do you feel today?”

The computerized data extraction methods were informed by this prior work using NLP to analyze large bodies of text and incorporated the traditional qualitative theoretical base of narrative inquiry ²² and the medical explanatory model ² to guide the computerized data extraction and qualitative review. These models use, to the highest degree possible, an individual person's way of describing a phenomenon as the ground point for theory and analysis, rather than relying on medical terms, descriptions, or available theory. Narrative inquiry and the medical explanatory model posit that unique and differing opinions or experiences offered by individuals are just as relevant as dominant themes identified by commonly expressed experiences. As such, data extraction would focus on finding broad‐scale themes in texts as well as less common words, terminologies, or themes.

As the dataset was so large, the team first used a set of a priori codes to extract data based upon the major categories of the RFI questions to count the frequency of terms and detect related topics within the text. ²³ , ²⁴ , ²⁵ , ²⁶ The computerized data extraction allowed the analytic team a broad overview of the character of the data, extract terms representing relevant themes, compare responses by group (organization or individual), and search for terms that were frequently or less frequently used. The qualitative analyst worked in tandem with the computational analysts to review terms and themes and how they differed across respondent type. Data were jointly coded by the qualitative and NLP analysts using the annotation tool eHost ²⁶ to review word snippets and terms and to refine topics. To enhance the trustworthiness of the data coding and analytic method, the team lead experts in military and veteran culture and health care reviewed text samples and helped identify relevant terms, such as military acronyms, and confirm themes identified in the data.

Initial coding and text extraction processes screened for terms related to risk factors, risk assessment, and prevention and intervention approaches, which were major themes the RFI attempted to account for. However, snippets of text associated with terms within these categories indicated other concepts were present in the data. To extract specific text phrases and terms to identify more unified concepts, the team applied terms from the Unified Medical Language System (UMLS) used in medical note‐taking to inform the text‐mining process for the a priori themes and identify potential new themes. ²⁷ In addition, unique snippets of texts, such as expletives or acronyms, were extracted for in‐depth review by the medical anthropologist and veteran health care experts to uncover rarer topics in the data and ensure all potential data points addressing risks for suicide were captured. Text‐mining techniques to extract combinations of semantic and syntactic patterns were used to distinguish and confirm how words and phrases were coded and grouped into themes (see Table 3). This process was helpful in distinguishing semantic and syntactic differences between organizations and individuals as well. Using these data, the research team was able to infer whether some of the respondents may have been military‐connected, a veteran or family member of a veteran, and which organizations appeared to have experience working on suicide prevention and treatment.

TABLE 3.

Examples of semantic and syntactic patterns identified within themes

Pattern	Theme	Example
Semantic patterns
“Drug addiction”	“RISK_FACTOR”	“Drug addiction services need to be offered”
“Suicide ideation”	“SUICIDE”	“No one will know except family and friends if a person is experiencing suicide ideation”
“Family” “Family support”	“COMMUNITY”	“It impacts the entire family and research on family support, education and treatment is as important as the individual treatment”
Syntactic patterns
“Risk factors such as <NOUN PHRASE>”	“RISK_FACTOR”	“Risk factors such as employment, housing, and health”
“Fear of <VERB>| < NOUN>”	“BARRIER”	“Fear of losing his benefits”
“Lack of <VERB>”	“BARRIER”	“Lack of resources on the front line”

Source: Federal Register Document Number 2019‐14138.

The process of large‐scale information extraction coupled with ongoing qualitative coding and expert review enabled reviewing a large body of qualitative data at a rapid scale, while still capturing the opinions, experiences, and emotions of unique respondents. This approach allowed the study team to quickly identify topics and categorize findings into themes, articulating differences within themes by respondent type, to support the development of policies geared toward organizational and personal support.

3. RESULTS

3.1. Principal findings

The findings were grouped into five major themes: risk factors, risk assessment, prevention and intervention, barriers to care, and data and research. Findings within risk factors identified emotional, physical, and behavioral health factors and the importance of the social environment on risk. Within risk assessment, respondents emphasized expanding the understanding of how symptoms from health conditions may overlap, and expanding the scope of what is considered “trauma.” The first two themes, risk factors and risk assessment, returned terms associated with the fluid vulnerability theory for suicide risk factors. ²⁸ , ²⁹ , ³⁰ , ³¹ That framework was subsequently used to organize texts within these categories that supported the notion that suicidal ideation fluctuates, with periods of lesser and greater risk that are affected by environmental triggers, cognition, behavior, and emotion. Texts within the prevention and intervention theme used terms and concepts associated with the evidence‐based crisis response planning model (CRP). ³² , ³³ Although other suicide prevention models exist, the use of terms and descriptions in the responses to the RFI may indicate respondents have had direct experience with this model of care as it has been evaluated as an effective method in reducing suicidality in U.S. soldiers. ³² Prevention and intervention was a rich theme with multiple subthemes identified. Major areas of concern related to this topic were the availability of treatments, how clinical care is administered, and how the transition from military to civilian life needs to be incorporated early into clinical care for all returning military service personnel, and a need to identify and promote protective factors such as social connection, employment, and the involvement of caregivers and peers in a person's care. In the fourth theme, barriers to care, many respondents mentioned issues with scheduling, access to certain services, and funding for those services. A need for better data and research, exploring both electronic health record and other data sources, emerged as the final major theme. Respondents recommended the need to research how co‐occurring conditions, military transitions, and social connection contribute to, reduce, or exacerbate suicidality. Respondents expressed need for better data integration across providers and from the military to VA and community‐based care. Many respondents recommended assessing the effectiveness of current intervention programs, including evidence‐based regimes, as treatment options may not work for every individual patient.

3.2. Systematic differences

Within the analysis of the themes, systematic differences were identified between organizations and individuals. To quantify the difference in vocabulary between the two groups, we ranked phrases according to the conditional probability that a question response containing a phrase was written by an organizational or individual respondent. ³⁴ This identified certain terms that were more strongly associated with one of the groups and illustrated general differences between the two groups. The highest probability terms for both groups are shown in Table 4. For example, individuals used more informal words, including expletives such as “shit” when describing a situation, or prefacing an explanation with “the problem is”. Organizations, on the other hand, were more likely to use academic or clinical language, including terms such as “predictive models” and “traumatic brain injury”. Once some differences were identified quantitatively, question responses were then reviewed to study higher‐level differences between the two groups.

TABLE 4.

Phrases and associated themes with the highest conditional probabilities by group

Organizations			Individuals
Text	Theme	Conditional probability	Text	Theme	Conditional probability
Post‐traumatic	Risk factor	97.0	Background checks	Prevention	100.0
Predictive models	Data	96.0	Medical marijuana	Prevention	100.0
Traumatic brain injury	Risk factor	93.8	Church	Spirituality	97.7
Post‐traumatic stress	Risk factor	92.0	Doctor	Clinical	97.1
Military veterans	Military	87.5	Marijuana	Prevention	97.0
Data collection	Data	87.0	Shit	Expletive	96.7
Biomarkers	Data	83.3	Pain management	Prevention	96.0
Nonclinical data	Data	80.0	Nurse	Clinical	95.2
Partners	Partnership	75.9	Therapist	Prevention	94.4
Databases	Data	72.7	Federal funding	Research	92.9
Predictive analytics	Data	71.4	Leaving the military	Transition	91.3
Clinical	Clinical	69.2	Illness	Risk factor	90.5
Outcomes	Research	68.9	Traumas	Risk factor	90.0
Brain injury	Risk factor	68.6	The problem is	Barrier	90.0
Mental health treatment	Prevention	66.7	Psychiatrist	Clinical	89.7

Source: Federal Register Document Number 2019‐14138.

Responses within prevention and intervention tended to diverge by respondent type more than, for example, risk factors. Both organizational and individual respondents suggested addressing “reasons to live” instead of solely on reasons to die. In a similar sentiment, both respondent types requested more holistic approaches to suicide prevention, for example, focusing on addressing physical health, employment, finances, trauma, and social connection. A more holistic strategy was advocated by both organizations and individuals, but organizations focused on treating medical conditions while individuals mentioned complementary and integrated health approaches. Many individuals appeared to resent a perceived overreliance by VA or non‐VA therapists and doctors on pharmacological treatment and rejection of complementary and integrative health approaches. Individual respondents also evinced a willingness to experiment with new therapies. Within this category, medical marijuana was mentioned and more often by individuals. In the case of therapies involving animals, there were multiple advocates from both groups. Differences between the groups extended to interventions facilitating military transition: organizations focused on clinical programs while individuals identified military or peer‐to‐peer programs.

Within the theme barriers to care, organizations and individuals cited a need for clinician training, but their perspectives were different. Individuals asked that clinicians have experience with military and veteran culture. Some individuals were concerned with clinician burnout dealing with suicide prevention and recommended help for clinicians. Organizations focused on training clinicians to recognize warning signs and follow best practices. Funding was mentioned by both organizations and individuals, yet individuals described difficulties with insurance claims and disability, while organizations emphasized needs for operational, research, or other types of funding.

Respondents were united in expressing the need for better assessment, prevention, and treatment, but they were divided between individual and organizational perspectives on what is important and how to accomplish it. Organizations focused on finding and implementing “best practices” and invoked theoretical approaches, while individuals asked for empirical evidence that programs help veterans and emphasized the need for individualized approaches. See Table 5 for a summary of findings within each theme. Two major thematic differences were identified.

TABLE 5.

Major themes and subthemes identified

Theme	Subthemes
Risk factors	Emotional, physical, and behavioral health risk factor themes
	The term “condition” was used commonly by both organizations and individuals to describe multiple factors such as financial or socioeconomic conditions, health conditions, and mental health conditions
	Both organizations and individuals cited “pain” and the treatment, or lack thereof, for pain as a major contributing risk factor for suicide
	Both organizations and individuals identified multiple forms of trauma including pre‐military, military, and post‐military trauma factors
	Social and environmental risk factor themes
	Both organizations and individuals cited “isolation” as a key risk factor
	Both organization and individuals identified socioeconomics such as employability, job loss, homelessness, and income inequality as risk factors
	Social support and other social determinants of health such as relationship issues, divorce, or loss of a family member or fellow soldier were closely related to isolation and socioeconomics as risk factors
	Text analyses indicated “lethal means” were closely related to other social risk factors, and individuals cited more concerns with access to firearms and lethal means
	Individuals identified risk factors that organizations overlooked, such as failings in care organizations to recognize and understand military culture, training, or combat experiences
Risk assessment	Organizations identified a need for better data collection, including expanding the scope of what is considered trauma and measuring social determinants of health in addition to mental health screenings
	Organizations identified many issues with self‐administered assessments in health and mental health and how overlapping symptoms can be misreported
	Individuals highlighted factors missed by most assessment methods including social and family factors, various forms of trauma, moral injury, or health factors such as hormonal imbalances
	Individuals often described a fear of risk assessments and potential misdiagnosis
	Individuals suggested examining other factors to identify those at risk and how to help them such as access to care, social media, and Internet use patterns, and assessing whether current treatment methods are making things better or worse for patients
Prevention and intervention	Clinical care delivery and treatments
	Organizations provided examples for the need for and use of comprehensive approaches to treatment and care
	Individuals indicated an interest in complementary and integrative health approaches
	Both organizations and individuals identified therapy animals as an important area of treatment specialty
	Both organizations and individuals described challenges with current pharmacological approaches
	Both organizations and individuals described non‐legal pharmacological approaches including medical marijuana
	Organizations emphasized the need for more research for medical marijuana
	Individuals indicated an interest in medical marijuana as well as other non‐legal pharmacological substances, e.g., psychedelics to reduce challenging symptoms
	Transition from military to civilian life
	Both organizations and individuals emphasized the need for better support of the transition from active duty to veteran status
	Organizations identified Veterans Affairs or other clinical programs as key post‐service support
	Individuals identified military or peer‐to‐peer programs as key post‐service support
	Individuals indicated peer‐to‐peer connectedness is vital for successful transition
	Protective factors identified
	Organizations identified the importance of resilience tied to community integration, education level, and reduced access to lethal means
	Individuals emphasized connectedness, resilience, and access to specific resources such as employment programs, service animals, or other programs supporting connectivity to civilian life to reduce risk factors
	Both organizations and individuals indicated caregivers and peers are protective factors that can support veterans
Barriers to care	Clinical barriers
	Both organizations and individuals indicated negative consequences associated with receiving care as a major barrier
	Individuals indicated concern with clinician burnout in suicide prevention efforts
	Individuals indicated a reluctance to be assessed by clinicians with little first‐hand knowledge of military or veteran culture. They emphasized a need for education and training for clinicians as well as all persons involved in veteran support
	Some organizations also indicated an awareness that a lack of education/training may be a barrier to providing successful intervention when it matters most
	Access to care
	Organizations emphasized resources, analytic tools, and staffing concerns to address suicide specifically
	Individuals indicated access issues such as availability of services including immediate and long‐term support, and access for underserved groups (e.g., women) or services (same‐day mental health)
	Both organizations and individuals identified pain management as a barrier
	Health care economics
	Both organizations and individuals identified a lack of funding for services
	Individuals identified barriers with VA disability and insurance claims
Data and research	Both organizations and individuals described the need for further data collection efforts
	Organizations emphasized a need to further analyze existing data
	Both organizations and individuals described how coordinating efforts between systems collecting or analyzing data might improve collaboration to reduce suicide
	Both organizations and individuals identified research needs for multiple, co‐occurring conditions that have overlapping symptoms such as post‐traumatic stress disorder, traumatic brain injury, anxiety, and depression
	Both organizations and individuals cited a need for research related to military transitions and the connections to suicide risk
	Both organizations and individuals recommend assessing the efficacy of current veteran intervention programs

Source: Federal Register Document Number 2019‐14138.

3.2.1. Overarching theme 1

Individuals were more likely to express that their concerns and desires for help were not always being recognized, captured, communicated, or appropriately acted upon by health care personnel and other institutional representatives. For example, while organizations felt that assessments were not answered appropriately, individuals felt that they were not being asked the right questions or provided opportunities to express various forms of need contributing to problems. Expletives in the data were one way it was possible to identify specific instances of veterans feeling misunderstood or overlooked by administrators or clinicians.

3.2.2. Overarching theme 2

Individuals expressed the need for veterans to have greater connection with the broader community, not just clinical relationships to address health or mental health issues. While many organizations indicated awareness that more provider training was needed to institute evidence‐based practices, individual respondents were more likely to note the lack of clinicians with military knowledge or experience. Individuals expressed concern that they needed to connect with others who had similar experiences and who could understand them. Many individuals expressed that these kinds of connections extend far beyond health care providers, extending to communities, caregivers, and peers. Individuals' recommendations for therapies also tended to be broader, including recreational therapies and vocational training, and support to connect veterans to jobs and social networks outside of the military.

4. DISCUSSION

4.1. Policy recommendations based on findings

The findings from this study pointed toward key areas where a national policy strategy could help unify and harmonize suicide treatment and prevention efforts. Many recommendations were proffered by respondents, and these were at times incongruous with one another. Clashing perspectives were apparent between organization and individual responses. However, key opportunities for policy support stood out. There appeared to be a strong need for rapid evaluation of assessment, prevention, and treatment efforts, as well as for a strategy to communicate these results to veterans. Traditional metrics for evidence‐based treatments may have an organizational bias; therefore, outcome metrics need to include a stronger veteran voice, perhaps by incorporating veteran engagement in multi‐stakeholder panels and the development of research and evaluation planning. In addition, complementary approaches for clinical care and other types of support that connect veterans to others need to be explored and made more widely available.

4.2. Policy outcomes

VA research used the findings and the policy opportunities identified from the RFI analysis to contribute to the eventual development of the PREVENTS Roadmap (Roadmap) with recommendations and implementation steps for suicide prevention. ³⁵ From the beginning of data collection through to the receipt of the analytic findings and over the course of the following months, multiple experts and federal agencies convened to develop the Roadmap. The PREVENTS campaign was launched just as the COVID‐19 pandemic was hitting the United States and text messages to a national emergency distress hotline run by the Substance Abuse and Mental Health Services Administration (SAMHSA) increased 1000% over the same time in 2019. ³⁵ As such, when the PREVENTS Roadmap was published on June 17, 2020, it outlined recommendations for policy, data collection, monitoring, research, partnerships across agencies, and individual support strategies not just for veterans, but for all persons who might be at risk for suicide including outlining strategies state policy makers could use to support organizations and health services for suicide risk prevention. The report includes a supplement based on this research, which outlines findings, policy opportunities, and potential strategies to implement at local, state, and federal levels. ³⁶ Implementation of the Roadmap and the VA's strategic plan for research is currently in process.

4.3. Continuing opportunities

The constraints of time and structure of the data provided an opportunity to innovate a collaborative rapid qualitative methodology combined with computer‐assisted text‐mining processes. This method has been recognized by VA Health Service Research and Development (HSR&D) as “QUICk: A Qualitative Interdisciplinary Collaboration” and has been recognized with a VA invention disclosure (VA ID 2020‐530). The process outlined here could be useful for other researchers to address similar public, open federal requests for information or help analyze other types of qualitative data gathered to fulfill state or federal mandates for research that must (1) include the public perspective and (2) be conducted in a timely manner. This specific analytic approach could support other state and/or federal agency research strategies as policy makers seek ways to gather and analyze data from communities quickly to inform policies, programs, and develop opportunities to support public needs.

4.4. Limitations

This research was limited by the design of the RFI in which the time required to analyze the large corpus of data was within a very compressed schedule. The RFI was designed to solicit a wide range of responses among a category of themes such as risk factors and prevention methods. Open, citizen‐science collection methods such as open‐ended surveys distributed to a wide audience with no participation restraint tend to be more grounded within community. ³⁷ This type of data collection allows anyone with any experience related to a topic to provide insight and knowledge, thus furthering and/or expanding concepts that researchers might miss otherwise. ³⁷ However, this kind of wide dissemination often precludes gathering identifying data such as age or gender of respondents. To analyze the data for this study, the analytic team had to use data extraction techniques to infer when an organization was working on suicide prevention and treatment, and when an individual respondent might be a veteran, a family member, spouse, or caregiver of a veteran. The RFI was only open for a month and the results needed to be analyzed and organized in such a way as to inform policy development a few months later. These constraints were met with an innovative analytic approach combining qualitative and computational methods to uncover themes and generate findings.

Another limitation of this study is the complexity of the policy making process. The analytic team could provide findings to the PREVENTS task force, who could provide recommendations to Congress. However, sustained policy change is complicated by factors beyond foundational research to support policy programming. Successful, long‐term policy change requires foundational political support that aligns from administration to administration. Here we present how these analyses were used to support the PREVENTS campaign to reduce veteran suicide, the long‐term policy outcomes of which are yet to be determined.

5. CONCLUSION

This study provides a rich example of a rapid, adaptive analysis of a large body of qualitative, public response data used to inform public health policy and mobilize government efforts to address veteran suicide. Combining qualitative and text‐mining methods allowed a high‐level representation of voices and perspectives, from individual perspectives describing the lived experiences of military life, transitioning to veteran status, and personal risks for suicide to the organizational perspective of people working for agencies to prevent veteran suicide. The findings were synthesized into a federal report and public health campaign. What began as a federal effort has been assumed as a policy campaign within the VA that has been promoting efforts across the nation to reduce veteran suicide as well as suicide in general. As of the writing of this article, the governors of 42 states and territories have submitted state proclamations to support the PREVENTS campaign and follow strategies outlined within the report. This research effort exemplifies how health scientists can collaborate with policy makers to incorporate individual and community needs to generate strategies, policies, and practices that directly support people who will be impacted.

Kalvesmaki AF, Chapman AB, Peterson KS, Pugh MJ, Jones M, Gleason TC. Analysis of a national response to a White House directive for ending veteran suicide. Health Serv Res. 2022;57(Suppl. 1):32‐41. doi: 10.1111/1475-6773.13931