Development of a veteran engagement toolkit for researchers

Abstract

Aim:

Engaging patients in the planning, implementation and dissemination of research can increase the credibility and relevance of results and lead to higher quality, more patient-centered care. Veterans have unique experiences and healthcare needs, making their input on research related to their care particularly important. However, existing veteran engagement resources primarily focus on veterans who receive care through the Veterans Health Administration (VA). This excludes those veterans – the majority – who do not use the VA for healthcare.

Methods:

To address this gap, we developed a veteran engagement toolkit that was informed by the work of both VA and non-VA researchers across the USA.

Results:

The resulting toolkit provides guidance on essential engagement activities relevant to researchers in a variety of settings.

Conclusion:

Investigators wishing to engage the veteran community may benefit from the experience and lessons summarized in this veteran-informed toolkit, in addition to resources directed at informing community engagement more generally.

Recently, researchers have acknowledged the importance of incorporating patient perspectives, values, and preferences into clinical and health services research [1–6]. In 2001, the National Academy of Medicine published the seminal document ‘Crossing the Quality Chasm’, which explicitly called for the inclusion of patient perspectives in efforts to address shortcomings in healthcare [7]. Since then, the field has gained momentum with the creation of the Patient-Centered Outcomes Research Institute (PCORI), which was established to provide funding for comparative effectiveness research [8]. The goal of comparative effectiveness research is to improve the evidence base that is needed for patients, clinicians, and others to make informed health decisions by comparing the effectiveness of different prevention, diagnostic or treatment approaches [8,9]. Implicit in this goal is the importance of engaging stakeholders in the research process to ensure research findings are relevant to patients and valuable to healthcare decision-makers [9]. PCORI has operationalized this by requiring that investigators demonstrate they are meaningfully engaging relevant stakeholders – particularly patients – throughout the research process [3,5,10,11].

The Veterans Health Administration (VA) Health Services Research and Development (HSR&D) service supports research to improve healthcare delivery to veterans. In 2015, HSR&D developed a veteran engagement workgroup, composed of VA health services researchers and veterans with engagement experience. The workgroup was tasked with identifying ways veteran engagement could be incorporated into HSR&D research. The workgroup recommended veteran engagement be operationalized through the development of veteran engagement boards, trainings, and changes to the HSR&D request-for-application guidelines and review process.

Workgroup members, in partnership with HSR&D leadership, have made notable progress on these priorities. Several workgroup members have led cyberseminars on topics related to veteran engagement. These cyberseminars were recorded and are posted publicly on the HSR&D website [12]. In addition, HSR&D now includes veteran engagement in its current methods-oriented priority areas for funding [13]. In 2017, HSR&D also allocated funding for the year-long ‘Strengthening Excellence in Research through Veteran Engagement’ (SERVE) project. The purpose of this project was to identify veteran engagement best practices within the VA and develop resources to assist VA research teams in engaging veterans. Deliverables of this project include a best practices toolkit and a process for offering expert consultations to VA researchers on veteran engagement.

HSR&D has made significant progress promoting veteran engagement through the development of resources for researchers who wish to engage veterans in their studies. However, these resources are primarily targeted toward researchers within the VA. Because only about 30% of veterans receive their healthcare through the VA, it is important that researchers outside the VA also have the skills and resources needed to effectively engage veterans [14].

We sought to address this gap by developing a veteran engagement toolkit that includes general guidance on how to engage veterans that would be relevant to both VA and non-VA researchers. This toolkit was developed as part of a grant from the University of Wisconsin (UW) Institute for Clinical and Translational Research (ICTR). The William S Middleton Memorial Veterans Hospital (Madison VA) obtained the funding in 2016. The Madison VA is affiliated with UW-Madison and has a research program that includes health services research. The goal of the funding is to develop infrastructure and resources to enable expanded engagement of veterans at Madison VA, UW, and in the broader research community. For this reason, the toolkit contains resources specific to Madison VA and UW researchers in addition to more general information that is relevant to any researcher wishing to engage veterans. In the following sections, we describe the contents of the toolkit and the process used to develop the toolkit.

Toolkit development process

Because the target audience for our toolkit was researchers, we felt it was important to learn more about strategies researchers were already using to engage veterans. To do this, we conducted informal interviews with researchers engaging veterans in both VA and non-VA projects. We used several techniques to identify researchers. First, we invited researchers from the veteran engagement workgroup to participate. Second, we searched the PCORI website for projects that appeared to involve veterans as stakeholders. Third, we completed an informal internet search on research involving veteran engagement. Finally, we asked researchers we interviewed to suggest other researchers we might speak to who also had projects involving veteran engagement. 16 of 26 researchers we contacted completed an interview (Table 1). The interviews were conducted by phone and ranged from 20 to 60 min in length.

Table 1. . Recruitment of researchers.

Method for finding researchers	Number approached	Number participating
Member of the VA HSR&D chartered veterans engagement workgroup	14	8
Investigators with PCORI funding for projects that had ’veteran’ in title	8	4
Individuals identified via internet search for veteran related research	1	1
Snowball sampling from experts involved in first three steps	3	3

HSR&D: Health Services Research and Development; PCORI: Patient-Centered Outcomes Research Institute; VA: Veterans Health Administration.

The unstructured interviews began by asking the researchers to describe their engagement activities with veterans. We prepared probes to ensure that key aspects of their engagement process were discussed. These key aspects were identified based on our prior experience and existing literature on engagement [4,5,8,15–20]. They included recruitment strategy, screening process, compensation, orientation activities and evaluation of engagement quality. Finally, we asked researchers about any resources they found helpful as they planned their engagement.

The lead author took contemporaneous notes to capture the responses, which we compiled after the interviews were completed. We summarized the engagement models used by the researchers and what we learned about the key aspects of the engagement process. These summaries formed the basis of the initial toolkit draft. Table 2 provides an overview of researcher characteristics and the nature of veteran engagement activities employed.

Table 2. . Researchers characteristics.

Investigator institutional affiliation
Academic only	2
VA only	6
Dual academic and VA affiliation	8
Engagement model (some investigators used multiple models)
Center-based VEG	6
Ad hoc or informal engagement:  • Input on project development  • Participation in study implementation  • Instrument design  • Results dissemination	7
Project-specific veteran engagement board	3
Miscellaneous engagement	2
Research topics for which veterans were engaged
Chronic disease	1
Opioid use	1
Post-traumatic stress disorder	4
Suicide prevention	1
Traumatic brain injury	1
Multiple research topics	8

VA: Veterans Health Administration; VEG: Veteran engagement group.

We developed a draft toolkit then solicited feedback from seven paid veteran consultants. We chose to have the veterans provide feedback on a draft of the toolkit rather than participate in developing that draft based on our initial conversations with them. The veterans felt that it would be easier for them to provide feedback on a toolkit that was developed by the authors. We asked the veteran consultants to comment on the draft toolkit and some online resources we were considering for inclusion. After they completed their review, the lead author met with them in person or by phone to get their input. All veterans indicated that the material in the toolkit seemed appropriate and useful. They also provided suggestions on what terminology and acronyms were important to define and identified additional resources to include.

We recruited the veteran consultants through our existing networks. Five had worked with one of the authors (N Safdar or J Whittle); two had presented at a workshop on military and veteran culture for healthcare professionals that the lead author attended. Era of service for the veterans included the Vietnam War (2), the Cold War (1), and post-9/11 (4). Most of the veterans served in the Army except one who served in the Navy and one who served in the Marine Corps.

After incorporating the veterans’ input, we sought additional feedback from UW ICTR researchers and the HSR&D veteran engagement workgroup via email. This led to only minor additional revisions.

Engagement models

The researchers used a variety of models for veteran engagement which are summarized in Table 2. The use of ongoing, center-based veteran engagement groups (VEGs) was most common among researchers from HSR&D-funded Centers of Innovation. In this model, a group of veterans convenes regularly to provide feedback on different projects from researchers within one research center. Each meeting focuses on a different researcher’s project. Veterans generally had to go through an application process. Often training or orientation materials were provided to both veterans and investigators. Investigators were instructed on how to present their research in an accessible way and clearly communicate the feedback they sought from the veterans. As of May 2017, six of the 19 Centers of Innovation had VEGs established and four were in the process of developing a VEG (two research centers did not respond to the inquiry; personal communication, SERVE team, 22 February 2018).

One workgroup member from a HSR&D center described a variation of this model, based on the community-engaged studio model developed by the Vanderbilt Institute for Clinical and Translational Research [21]. In this model, a navigator recruits and orients stakeholders with a variety of different backgrounds and experiences with health conditions. Researchers contact the navigator with specific engagement needs. The navigator then convenes a ‘studio’ made up of a subset of the stakeholders that have experience relevant to the research topic in which feedback is provided to the researcher.

Researchers also described the development of veteran engagement groups specific to their research. Veterans in these groups had experience with the health condition or topic being studied. The veterans provided feedback throughout the course of the project. Researchers also described engaging a small number of veterans more informally to participate in identifying study topics, developing a grant proposal, planning the project and disseminating results. These engagement strategies were most common in researchers with PCORI funding.

Another model of engagement is to incorporate veterans into the research team. One author (J Whittle) and his colleagues train veterans as peer specialists who conduct study activities such as recruiting participants or implementing the intervention. The researchers recruited peer specialists from veterans organizations, including the Veterans of Foreign Wars and the American Legion. They acted as liaisons between the researchers and other veterans, helping the researchers to clearly communicate to veterans and bringing any concerns of the veterans back to the researchers [22].

Toolkit contents

We divided the toolkit into three main sections that incorporate the feedback and resources obtained from the researchers and veteran consultants. These sections are summarized in Table 3 and described in further detail below.

Table 3. . Toolkit structure.

Preparing to engage veterans

• Overview of military structure, culture and terminology

• Veteran Engagement Trainings and Resources:  ○ VA HSR&D Workgroup and Cyberseminars

• General Stakeholder Engagement Trainings and Resources:  ○ PCORI website  ○ UW ICTR Website

Recruiting veteran stakeholders

• Strategies (flyers, networking, presentations)

• Recruitment venues

• Developing recruitment materials

• Screening and interviewing potential veteran stakeholders

Developing a veteran engagement plan

• Engagement at each stage of the research process

• Compensation

• Complying with the Federal Advisory Committee Act

• Orientations for veterans and researchers

• Evaluation

ICTR: Institute for Clinical and Translational Research; PCORI: Patient-Centered Outcomes Research Institute; UW: University of Wisconsin; VA: Veterans Health Administration.

Preparing to engage veterans

The first section focuses on ways researchers can prepare themselves to engage veterans. Although the researchers we interviewed did not raise the issue, we identified the need for non-VA researchers to familiarize themselves with the military in order to effectively engage veterans. The lead author attended a workshop on military and veteran culture for healthcare professionals which was led by a UW-Madison campus veteran group. Workshop presenters stressed how important it was for health professionals to have a basic understanding of the military to effectively care for service members and veterans. It seemed reasonable that this understanding would also be valuable for researchers who want to engage veterans. Veterans have unique experiences associated with their service that have likely had a significant impact on their lives. Many veterans continue to deal with the physical or emotional effects of their service long after they leave the military. These effects include amputations, burns, respiratory issues, post-traumatic stress disorder, traumatic brain injury and Gulf War Syndrome. These effects are important for researchers to understand in order to ensure they are engaging with veterans in a way that is safe and respectful. Our veteran consultants agreed that including an overview of the military was important. We therefore assembled a variety of online resources for researchers without a military background.

The section also includes information on existing engagement trainings and resources. Many of the resources focus specifically on veteran engagement, but we also included more general stakeholder engagement resources, such as the resources provided on the PCORI website [23].

Recruiting veteran stakeholders

The second section provides strategies for recruiting veterans for engagement. Researchers recruited veterans to engage in their projects in a variety of ways. Some asked veterans, physicians or other people they already knew to recommend veterans who might be a good fit to engage in their projects. Distributing flyers and presenting at VA voluntary services committee meetings were also common strategies. Although time consuming, several researchers found presenting at veteran events to be worth the effort. The researchers suggested potential recruitment venues outside of VA facilities, such as veterans organizations and campus veteran centers.

We also provided tips for developing recruitment materials and screening potential veteran stakeholders. To ensure veterans had the skills and interest necessary to be effective stakeholders, some researchers asked veterans to complete a statement of interest, an application form and/or interviews. Characteristics of veterans sought by researchers included knowledge of the VA, strong communication and listening skills, experience on other advisory boards, and an ability to commit to the engagement activities. Some researchers emphasized recruiting veterans with diverse backgrounds for an engagement group, while others emphasized the need for veterans with specific backgrounds when researching specific clinical conditions.

Developing a veteran engagement plan

The final section focuses on developing a strong veteran engagement plan. It provides guidance on a number of logistic considerations including whether and how much to pay veterans for engagement. Few researchers relied on volunteers and the amount of compensation the researchers provided to veterans for engagement activities varied widely. Some paid veterans an hourly rate ranging between US$25 and US$50. Others paid veterans a set amount or gave them a gift card for each engagement meeting. Most researchers reported that they determined the amount based on what seemed fair. Some researchers consulted colleagues who had previously engaged stakeholders or PCORI resources [24] to determine the appropriate amount.

VA researchers pointed out that ongoing veteran engagement boards within the VA needed to comply with the Federal Advisory Committee Act (FACA). Groups that are subject to FACA are committees of people who are not government employees but provide advice to federal agencies. These groups are required to meet certain standards in terms of meeting structure, membership representation and public access to meeting content [25]. The researchers with VEGs took steps to ensure they were not subject to FACA, since the required structure would likely reduce participation. Some had veteran stakeholders become employees of the VA by obtaining Without Compensation Employee status. Others established a memorandum of understanding with their VA institution to document that veteran stakeholders were providing their individual viewpoints only and were not functioning as an advisory committee as defined by FACA.

Many researchers provided orientations to both veterans and research team members early in the engagement process. They found orientation to be helpful in promoting meaningful engagement by aligning researcher and stakeholder expectations. Other considerations that researchers cited as important were making sure meetings were accessible and having a skilled meeting facilitator.

Finally, researchers noted the value of assessing the quality of engagement so that changes can be made to engage veterans more effectively. Aspects perceived as worthy of evaluation included the level of transparency, quality of communication, access to needed information and clarity of project roles. Researchers typically evaluated their engagement processes by having the veterans complete a survey at the end of engagement meetings.

Disseminating the toolkit

The toolkit is available on the HIPxChange website, which is maintained by the UW Health Innovation Program. The site hosts toolkits on a variety of different health topics. We worked with a HIPxChange staff member to create web-based and PDF versions of the toolkit. Both versions can be accessed at the following web address [26]. A PDF version of the toolkit is also posted on the VA Quality Enhancement Resources Initiative website [27].

Discussion

We developed a veteran engagement toolkit relevant to researchers both within and outside the VA. We hope the toolkit will facilitate an increase in veteran engagement in research. Many toolkits on engaging patients and other stakeholders in research are available to guide researchers [21,28–32]. However, this is the first toolkit we are aware of that focuses specifically on engaging veterans. Although two other veteran engagement toolkits are currently being developed, each has a narrower focus. Rather than being duplicative, our toolkit will be complementary to these toolkits.

The SERVE project’s best practices toolkit will be finalized in 2018. This toolkit focuses on the development of center-based VEGs. This model works well for the Centers of Innovation that often have identified a specific health condition or issue upon which many of their researchers focus. However, this model of engagement may not work as well at research centers that study a variety of health conditions. It is sometimes important that the veterans have lived experience with the health condition being studied for them to be meaningfully engaged. Additionally, VA research centers that are not HSR&D-funded Centers of Innovation may not have the resources necessary to develop an ongoing VEG. Our toolkit will be useful to researchers in any VA setting, regardless of the engagement model employed by the center.

The second veteran engagement toolkit is being developed as part of a PCORI engagement project [33]. The toolkit will focus on a specific model of engagement – the development of Veterans’ Action League (VAL) units. VAL units are engagement groups led by a veteran leader and an academic researcher that meet monthly to participate in engagement activities. VAL units have been established in six states, and together form a national platform to facilitate the engagement of veterans and other stakeholders in research. Again, the broad engagement strategies outlined in our toolkit are applicable to this model of engagement so our toolkit could be used in addition to the VAL toolkit.

The process we used to develop our toolkit has limitations. We interviewed researchers who had previous experience engaging veterans in health services research. Therefore, some of their engagement experiences may not be relevant to other types of researchers. We also did not talk to researchers who had not engaged veterans, nor evaluate the relative merit of different engagement strategies. Doing this would likely strengthen the toolkit but was outside the scope of the current project.

Conclusion

Our toolkit is a novel, veteran-informed resource which will complement existing resources on stakeholder engagement. It provides guidance that is uniquely relevant to both VA and non-VA researchers wishing to engage veterans in their research.

Future perspective

Future work is needed to systematically evaluate different strategies to determine their relative merit. Further evaluation is also necessary to determine whether engagement efforts are leading to the desired benefits for veterans – relevant and meaningful research findings that lead to improved health outcomes. This is important to ensure that the costs of engagement to veterans are commensurate to the benefits they derive from engagement. Assuming engagement does lead to the desired benefits, efforts should be made to expand veteran engagement to different areas of research. Veteran engagement efforts have primarily centered around health services research. However, more work needs to be done to identify ways veterans can be engaged in other types of research. We anticipate that this work will make it easier for researchers to engage veterans in the future. Now is the opportune time to facilitate the development of a community of devoted researchers who effectively incorporate the veteran perspective into their research.

Summary points.

• Researchers and funding agencies increasingly recognize the value of incorporating patient perspectives, values, and preferences into clinical and health services research.

• The Veterans Health Administration (VA) Health Services Research and Development service has taken steps to promote veteran engagement in VA research.

Toolkit development process

• We interviewed VA and non-VA researchers with experience engaging veterans, reviewed existing patient engagement resources, and solicited the input of veterans to inform the development of a veteran engagement toolkit.

Engagement models used by researchers

• The interviews revealed a number of different engagement models being used including ongoing engagement groups and the incorporation of veterans into the research team.

Toolkit contents

• The toolkit is organized in three main sections: preparing to engage veterans, recruiting veterans for engagement and developing a veteran engagement plan.

Disseminating the toolkit

• The toolkit is posted online at: [26] and [27].

Discussion

• This toolkit focuses specifically on engaging veterans in the research process.

• Two other veteran engagement toolkits are currently being developed.

• We believe the generally accepted engagement principles provided in our toolkit will supplement these other toolkits.