A Web-Based Self-Management Program for Recent Combat Veterans With PTSD and Substance Misuse: Program Development and Veteran Feedback

Abstract

Combat veterans from the wars in Iraq and Afghanistan commonly experience posttraumatic stress disorder (PTSD) and substance use problems. In addition, these veterans often report significant barriers to receiving evidence-based mental health and substance use care, such as individual beliefs that treatment will be unhelpful, inconvenient, or that they should be able to handle their problems on their own. To increase access to treatment for this underserved population, a Web-based patient self-management program that teaches cognitive-behavioral therapy (CBT) skills to manage PTSD symptoms and substance misuse was developed. This paper describes and provides results from an iterative, multistage process for developing the Web-based program and seeks to inform clinicians in the field about the preferences of veterans for using a Web-based CBT program. Systematic feedback was gathered from (a) three expert clinicians in the field, (b) focus groups of combat veterans (n = 18), and (c) individual feedback sessions with combat veterans (n = 34). Clinician feedback led to the incorporation of motivational strategies to increase participant engagement and an optional module that guides written trauma exposure work. Focus group feedback guided the research team to frame the program in a strength-based approach and allows for maximum flexibility, adaptability, interactivity, and privacy for veterans. In individual feedback sessions, veterans generally found the program likable, easy to use, and relevant to their experiences; critiques of the program led to revised content meant to increase clarity and participant interest. Our findings provide specific guidance for clinicians who are interested in developing or providing technology-based treatment, including the need to gather feedback from an intervention's target audience when adapting a psychotherapeutic intervention and that the treatment must be highly interactive and private to engage clients.

Many of the over 1.65 million individuals who have served in military efforts in or near Afghanistan as part of Operation Enduring Freedom (OEF) or in or near Iraq as part of Operation Iraqi Freedom (OIF) and Operation New Dawn (OND) have experienced significant stressors, which may contribute to an array of mental health and substance use problems. Of the OEF/OIF/OND veterans seeking services at Veterans’ Administration (VA) medical centers, over two thirds (69%) have screened positive for a mental health problem (Seal et al., 2010). The most prevalent diagnosis is posttraumatic stress disorder (PTSD), with rates ranging from 14 to 22% (Seal et al., 2009; Tanielian & Jaycox, 2008). Hazardous alcohol use is also highly prevalent among OEF/OIF/OND veterans (Cohen et al., 2009). Studies of OEF/OIF/OND veterans in VA primary care found that 37–39% screen positive for PTSD, 23–27% for hazardous alcohol use, and 16% for both (Jakupcak, Luterek, Hunt, Conybeare, & McFall, 2008; McDevitt-Murphy et al., 2010). Even more alarming, a national study found that 63% of recent combat veterans with an alcohol use disorder also have PTSD (Seal et al., 2010). Clearly, many OEF/OIF/OND veterans have significant mental health concerns that need to be addressed.

PTSD and substance use are hypothesized to be related to each other in a number of ways. Individuals often attempt to "self-medicate" their PTSD symptoms with substance use (Khantzian, 1997). In addition, substance use may place individuals at higher risk for exposure to trauma, inhibit natural processing and resolution of trauma-related distress, and heighten physiologic arousal and exacerbate PTSD symptoms (Jacobson, Southwick, & Kosten, 2001). Given the complex relationship between these disorders, and that individuals often experience PTSD/substance use comorbidity as one disorder, providing an integrated treatment program is optimal. Early research has found that integrated treatments are well tolerated by patients and can lead to significant reductions in both PTSD and substance use symptoms (McCauley, Killeen, Gros, Brady, & Back, 2012). While research on integrated programs is limited, there is promising preliminary evidence that cognitive-behavioral therapy (CBT) approaches may produce favorable outcomes in individuals with comorbid PTSD and substance use (e.g., Hien et al., 2009). For instance, a CBT program that relies heavily on cognitive restructuring of maladaptive thoughts has been successfully adapted for treating individuals with PTSD and substance use (McGovern, Lambert-Harris, Alterman, Xie, & Meier, 2011).

Many OEF/OIF/OND veterans, especially those with PTSD symptoms, are reluctant to seek mental health treatment (Hoge et al., 2004). Treatment-seeking barriers include individual beliefs that treatment will not be helpful or will induce too much distress, concerns about the stigma of mental health treatment, practical barriers (e.g., transportation), and avoidance symptoms characteristic of PTSD (Hoge et al., 2004; Pietrzak, Johnson, Goldstein, Malley, & Southwick, 2009; Stecker, McGovern, & Herr, 2012). However, individuals with PTSD often seek treatment for physical complaints in a primary care setting, presenting a potential opportunity to provide services to these individuals (Zlotnick et al., 2004). One VA initiative to address barriers to care has been to integrate mental health providers in the primary care setting to deliver brief assessments and interventions (Veterans Health Administration, 2008). Brief primary care-based treatment can help engage veterans in mental health care, but full-length (8–12 session) interventions do not fit well in the fast-paced primary care environment (Gunn & Blount, 2009).

An important way that brief primary care interventions can be extended in scope and intensity is through the use of technology-based resources. Research supports that rural and urban-dwelling individuals with PTSD, military personnel, and individuals who abuse substances are often highly receptive to technology-based treatment (Grubaugh, Kain, Elhai, Patrick, & Frueh, 2008; Marsch, 2011; Wilson, Onorati, Mishkind, Reger, & Gahm, 2008). Incorporating the use of technology may be especially important for individuals with PTSD, because they are more likely to choose to live in remote, rural areas to reduce overstimulation, hyperarousal, and interpersonal conflict (Moreland, Frueh, Pierce, & Miyahira, 2003). Technology-based treatment may also address many common barriers to care (e.g., inconvenience, trauma-related avoidance, stigma) by allowing veterans to receive treatment in the privacy of their own homes, at a time that is convenient to them. Computerized programs specifically focused on using CBT strategies to address PTSD (Klein et al., 2010; Roy-Burne et al., 2010) or substance use (Bickel, Marsch, Buckhalter, & Badger, 2008; Carroll et al., 2009) have been found to be efficacious, while those that simultaneously address PTSD and alcohol use are currently being tested (Brief, Rubin, Enggasser, Roy, & Keane, 2011).

To address the public health concern of elevated rates of PTSD and substance use in OEF/OIF/OND veterans, a research project is being conducting to develop and evaluate an innovative, Web-based psychosocial intervention for the treatment of PTSD symptoms and problematic substance use. This program enables a complex intervention to be delivered with high fidelity, at a low cost, without increasing demands on providers’ time or training needs. Program content was derived from efficacious CBT treatments for substance use disorders (e.g., Carroll, 1998) and PTSD (e.g., Mueser et al., 2008) and utilizes an interactive delivery system that incorporates informational technologies shown to be effective in promoting knowledge and skills. Development of this program progressed in several iterative stages, including gathering systematic feedback from expert clinicians, conducting focus groups of OEF/OIF/OND veterans, and obtaining individual feedback sessions with OEF/OIF/OND veterans. The research team incorporated feedback prior to launching a final version of the program. A randomized clinical trial (RCT) is now under way comparing primary care treatment as usual to the Web-based program plus primary care treatment as usual.

This paper describes the development of our Web-based program, called "Coming Home and Moving Forward," including the technical and content development, and quantitative and qualitative data collected from clinician feedback, veteran focus groups, and individual feedback sessions with veterans. We also detail how recommended changes were processed by the research team and used to revise program content. A description of the launched Coming Home and Moving Forward program, including program layout, interactive features, and the content of the 24 modules also is described. This paper is meant to guide CBT practitioners on how to develop and evaluate technology-based treatments that address a critical public health need. In describing this process, we hope to provide an example of how evidence-based interventions may be adapted for delivery via technology. The current program was developed in the context of rigorous research and ethical standards of practice and therefore may serve as a guide for future development and adaptation efforts. The iterative process described within can be applied to the adaptation and/or development of any number of interventions, including those targeted toward different populations and that use a variety of therapeutic strategies and technologies such as mobile phones, tablets, and social networks.

Development of the Web-Based Intervention

Data Security

The Web-based intervention was designed to allow for optimal data security and information privacy. All identifiable and protected health information needed for the research study was collected in person and not via the Web-based intervention. All information collected via the Web was coded with a participant ID and saved on a server that is certified to be compliant with the Federal Information and Security Management Act of 2002. This server was located at a university affiliate and not within the VA. The data security plan was approved by the local VA Information Security Officers.

Learning Technologies

Technologies to enhance participant learning, including computer-assisted instruction (CAI; Binder, 1993), interactive exercises (Rafaeli & Ariel, 2007), and modularity were employed in development. Fluency-based CAI is an educational technology, based on the precision teaching/overlearning literature, which requires the learner to develop a predetermined level of accuracy and speed in responding. A CAI engine (HealthSim, 1997) incorporates fluency-building methodologies by presenting the user with a series of questions and then providing immediate feedback on all responses. Depending on a user’s response to a question, both the read and response time for that question are adjusted for the next presentation. Users must meet a preset criterion (e.g., answer X questions correctly within X time) to complete the “quiz.” In addition, interactive exercises were employed to enhance learning and personalize content for participants. Specifically, veterans are asked to type in personalized responses to complete different interactive worksheets and/or exercises throughout the program. Finally, content was presented in a modular format to enhance learning. Modularity refers to the user's ability to access different portions of a program and to move from one portion to another (Dede & Fontana, 1995). Text in the program was presented in bulleted format and accompanied by optional audio that expanded on the text to accommodate individuals with reading difficulties and/or individuals who may prefer to both read and listen to all program content.

Program Orientation

The content of the program is based on a CBT orientation. This approach focuses on acquisition of self-management skills to help clients develop and strengthen their ability to self-regulate and cope with anxious, trauma-related thoughts that have contributed to maladaptive behaviors including substance use. Self-management strategies encourage and reinforce clients to play an active role in learning and applying skills to manage PTSD and substance use problems. Additionally, the content encourages and guides clients to practice and rehearse skills learned during the acquisition phase of the CBT intervention in a variety of different contexts.

Overview of Iterative Development Process

We used an iterative approach to develop our program, using the principles outlined in Aronson, Marsch, and Acosta (2012). This approach included creating detailed outlines of all planned content, drafting the script for the computer modules, and developing a detailed plan for all of the program’s interactive and tracking features. At each stage, multiple versions of draft content were edited/commented on by the research team, consisting of VA and non-VA researchers with expertise in PTSD, substance use, intervention development, and Web-based technology. Systematic feedback from experts and veterans was then sought in three distinct phases: (a) clinician feedback, (b) veteran focus groups, and (c) individual veteran feedback sessions. The methods and results of each phase and the modifications made to the program based on feedback from that phase are detailed next.

Expert Feedback

Methods

The study team obtained input from experts within our team and expert consultants external to the team on detailed outlines for each of the planned intervention sections, as well as various scripted content and features. Our external experts consisted of two VA clinicians with extensive experience treating PTSD and substance use in OEF/OIF/OND veterans. Additional feedback on the overall concept and structure of the intervention also was provided by an addiction psychiatrist with experience treating comorbid substance use and mental health problems. VA consultants provided independent feedback on their impressions of the intervention as a whole, on each detailed program section outline, and specific interactive program features by completing a questionnaire about each program module and feature. Consultant feedback was consolidated into a table that included major and minor suggested changes. This table was circulated among the study team members, and each suggestion was discussed. The study team voted on which changes should be made and then discussed the best way to implement the changes.

Results and Program Modifications

Overall, the consultants were positive about the planned intervention content and agreed with our planned CBT approach. They felt that recent veterans would be comfortable using a Web-based program. The two VA experts cautioned about veterans’ potential reluctance to “buy into” the program and/or use the program consistently. They suggested using a motivational approach early on to try and engage veterans. In response to this feedback, a “Readiness to Change” module was developed.

In many cases, consultant feedback consisted of very specific strategies to execute the content that was in our intervention outlines, or specific examples to use that might help make the intervention more credible for veterans. Feedback of this type was readily incorporated into the modules. On some occasions, the consultant feedback consisted of suggestions for the addition of new content into the intervention. For example, it was suggested that the intervention cover additional topics such as women’s issues, parenting concerns, and spirituality. Many of these suggestions, while sound, were deemed to be beyond the scope of the intervention and therefore were not incorporated. Others were incorporated in existing sections, such as using examples of parenting concerns in the “Communicating With Others” module.

One expert suggested the addition of an optional written exposure module, noting that in her experience this intervention was effective in remediating PTSD symptoms and that many veterans were willing to try it. Furthermore, participants would already be learning specific skills, such as relaxation strategies, in the program that would prepare them for exposure work. This advice led to the decision to develop an optional guided written exposure module, allowing users to access it when they had completed the core (mandatory) intervention content. Experts provided concrete suggestions on how to decrease risk to veterans while using the intervention, especially around emotionally charged aspects such as trauma exposure.

Veteran Focus Groups

Methods

Following the expert feedback stage, we conducted four 90-minute focus groups with OEF/OIF/OND veterans (n = 18) who had screened positive in their primary care clinic for hazardous alcohol/substance use or PTSD symptoms. We overrecruited women and individuals of ethnic/racial minority status to ensure we received feedback from a diverse sample. We accomplished this goal in that our participants were more diverse than the clinic populations they were recruited from: four (22%) participants were women and three (17%) were of ethnic/racial minority status. Following informed consent, focus group participants were presented with an overview of the planned intervention and then asked a series of questions to assess their impressions; whether or not the planned content was appropriate; anticipated barriers to using the program; and the extent to which the language, graphics, presentation style, and structure were (a) understandable, (b) engaging, and (c) relevant to their experience. Discussion on each topic was initiated by open-ended questions posed by one of the two group facilitators. This format was flexible, allowing participants to expand on topics, comment on points made by others, and introduce related topics. Focus group participants were compensated $50 for their participation.

All data obtained from focus groups were qualitative in nature and were audiotaped, then transcribed. The project director then created detailed summaries of all transcriptions and identified several themes consistent throughout the groups. This information was reviewed by the entire study team, and a series of group discussions were held to determine what changes needed to be made to the planned intervention. In determining if the intervention should be revised based on specific focus group feedback, the study team considered the scope of the intervention, feasibility of implementing the change, and whether it was anticipated that the change would add to the existing intervention's accessibility or effectiveness. No formal qualitative data analytic approach was used; however, themes that were repeated throughout the focus groups and/or endorsed by the majority of participants were weighted heavily in informing module development.

Results

Veterans suggested that the program would be most appropriate for people after they have been back from deployment for several months, because that is when they started to experience reintegration problems. In addition, three central, recurrent themes were noted throughout the focus groups: (a) use of technology and privacy, (b) taking a strength-based approach, and (c) desired module content. Regarding the first theme, veterans largely supported the use of the Web-based technology, because they thought the program gave them anonymity and privacy, that it could be an effective way to reach people who needed help, and that it could be a "stepping stone" to get people into face-to-face treatment. They also liked that it allowed users to go at their own pace. However, veterans expressed concerns about protecting the privacy of their personal information that they would input into the program for interactive exercises. Many participants stated that they would not answer questions honestly unless they knew their responses/interaction with the intervention was kept private.

The second theme centered on taking a strength-based approach. Participants reported concerns about being “labeled” with PTSD, alcoholism, and/or as drug users, and instead suggested framing problems as issues of "reintegration" and "readjustment." They were particularly sensitive to references to illicit/nonprescribed drug use, as they saw alcohol use as less stigmatizing, and therefore suggested that the program avoid clumping all substance use together. Participants expressed preferences for alcohol-related interventions to use risk reduction rather than abstinence strategies. Participants were given a list of possible names for the program and preferred names that did not remind them of the military or use diagnostic labels, but instead focused on making positive changes in life.

The third theme that emerged from the focus groups was specific concerns and suggestions for the intervention. There was discussion that some of the modules should be optional (not mandatory), as participants felt that not all of the modules were pertinent to them. In a majority of the focus groups, participants felt the planned Web-based program should incorporate content on common problems such as anger and communication skills. Participants also gave suggestions about building in options to communicate with others through the program such as their primary care team and veterans via chat rooms. Participants had mixed feedback on desired module length with some feeling that 20–30 minutes was appropriate and others stating that it was too long. Regardless, participants thought that modules must be interactive, simple, and user-friendly to engage them in the program. They also felt that e-mail/text reminders, even the possibility of a phone application, would be helpful for them to remember to complete modules. In addition to reminders, participants felt that an incentive or reward for completion may help to reinforce program participation. Participants liked the idea of being able to track their progress with individualized graphs embedded in the program. Some participants also felt that there should be access to additional resources and an on-call clinician during the program.

Program Modifications

Many of the recommendations made during the focus groups were incorporated into the program development. These included privacy safeguards, use of strength-based, nonlabeling language (including the program name), a focus on reducing "problematic" substance use rather than abstaining from all substances, the ability for participants to choose from a range of optional modules after completing initial "core" modules, greater emphasis on interactive features, and a comprehensive listing of VA and community-based resources to help veterans address a variety of reintegration concerns. Some of the comments could not be incorporated because they were beyond the scope of our study (e.g., changing the current intervention to be delivered via mobile phone). We choose not to incentivize module completion because this would not allow completion rates to be generalized to clinical settings in the future. At this stage in development, a document detailing the functional specifications for the technical development team, specifying introductory program features (e.g., home page), the interactive functionality of each modules, common and specific module elements (e.g., slides, graphics, navigational flow, summary tables), and data storage was created. From this our developers constructed a fully functioning program, referred to as the beta intervention.

Individual Veteran Feedback Sessions

Methods

As part of the iterative development process, a series of feedback sessions were conducted to systematically obtain feedback from two to four individual participants on each program module in our beta intervention. A total of 34 OEF/OIF/OND veterans who had screened positive in their primary care clinic for hazardous alcohol/substance use and PTSD symptoms were recruited for participation. Participants from prior focus groups were not eligible to participate in feedback sessions. Once again, a diverse sample was recruited that included four women and nine individuals who identified as ethnic/racial minorities. Veterans were asked to participate in an individual 60- to 90-minute feedback session where they were assigned three different program modules to review. Participants were first asked to complete five to six multiple-choice questions, assessing their baseline level of knowledge about the module topic, then asked to complete each module, while being monitored by a researcher who recorded time spent on each module. Following module completion, they were asked to respond how much they (a) liked the module, (b) found it acceptable, (c) found it easy to use, (d) thought it was relevant to their experiences, (e) thought it promoted learning of new information, (f) expected it would affect their behavior, (g) thought it was interesting, (h) found it to be threatening, and (i) how the modules compared with any related materials they had previously received regarding substance abuse and PTSD on a 100-point visual analogue scale (VAS) ranging from not at all to very much. Free text response areas also were available to obtain qualitative data on what they liked “best” and “least” about the module and what changes they suggest. Participants then were readministered the same knowledge assessment they completed before accessing the module to assess knowledge increase.

Results and Program Modifications

Table 1 displays the title and key features of each of the 24 modules, whether they were planned to be core (mandatory) or optional modules, the number of participants who reviewed each module, and mean VAS scores on each dimension surveyed. Mean scores of the modules in general were positive (M = 75 of 100, range = 40.90–94.60). Of the 24 modules tested, 6 received aggregate ratings below our “action” criteria (M = 60), indicating the content or presentation needed reconsideration. In reviewing feedback, we found that several of the participants who viewed these modules rated everything they viewed very poorly, regardless of differing content. Three of the six poorly rated modules contained core skills essential to the intervention: “Identifying Automatic Thoughts,” “Evaluating Automatic Thoughts,” and “Challenging Automatic Thoughts.” Given closely related content of these modules, changes were made to the presentation/content of the first one (i.e., “Identifying Automatic Thoughts”) in hopes that it also would positively affect the reception of the subsequent modules. Consistent with our iterative process, we then conducted four additional feedback sessions for each of these three modules. These additional data were more positive than those collected in the original feedback sessions, and the average of the data across all eight participants (who viewed the “automatic thoughts” modules) fell above our “action” criteria (M range = 65.4–72.7). The remaining three modules that scored in the actionable range (“Finding Patterns in Thoughts and Behaviors,” Managing Anger,” and “Communicating With Others”) were considered on a case-by-case basis. “Finding Patterns in Thoughts and Behaviors” was retained as a mandatory module because it addresses core skills needed in subsequent modules, and the other two modules were made optional. Across all modules, knowledge test scores were found to have improved significantly after viewing module content: average score pretest = 68%, posttests = 91%, t(df = 78) = –7.98, p < .001.

Table 1.

Description and Veterans’ Ratings of Final Program Modules

Module Name	Key Features	Core or Optional	# Participants	Interesting Mean (SD)	Useful Information Mean (SD)	New Information Mean (SD)	Ease of Use Mean (SD)	Understood Mean (SD)	Likeability Mean (SD)	Mean VAS / Mean SD
Introduction	Provides intervention rationale and goals; emphasizes the importance of appraising thoughts and feelings about drug/alcohol use and trauma symptoms and engaging in various cognitive, physical, and social activities	C	4	64.3 (5.8)	64.3 (20.5)	73.0 (23.9)	97.5 (3.8)	96.0 (6.7)	67.8 (24.7)	77.1 (13.0)
Connecting Trauma- Related Problems and Alcohol/ Drug Use	Seeks to describe how substances are often used to self-medicate trauma symptoms, reduce fear and shame about trauma-related symptoms; “normalize” veterans’ experience; and provide hope for recovery based on the learning of strategies	C	4	73.8 (30.9)	68.0 (13.6)	38.0 (17.3)	96.0 (6.7)	95.8 (8.5)	46.3 (26.9)	69.6 (9.7)
Relaxation	Teach veterans how to use deep-breathing relaxation	C	4	70.0 (29.8)	79.8 (23.0)	75.3 (24.8)	96.8 (5.3)	98.3 (2.4)	74.5 (24.3)	82.4 (17.7)
Emotional Centering	Teach a grounding exercise that will enable veterans to gain distance from their emotional pain and substance cravings	C	4	51.0 (40.9)	55.5 (41.0)	49.3 (25.1)	88.3 (23.5)	91.3 (17.5)	54.8 (40.8)	65.0 (25.3)
Patterns in Thoughts and Behaviors	Seeks to help veterans recognize their own thought patterns and the behaviors associated with them	C	3	16.7 (17.2)	37.7 (32.3)	24.7 (24.5)	93.3 (11.5)	87.7 (10.8)	32.7 (28.4)	48.8 (13.1)
What Is Functional Analysis?	Explains how functional analysis can be used to identify various risk factors and consequences of recent substance use	C	4	56.8 (27.2)	59.5 (31.4)	28.5 (13.4)	98.8 (2.5)	98.8 (2.5)	76.0 (24.5)	69.7 (15.9)
Functional Analysis of Substances	Interactive exercises that walks veterans through the process of a personalized functional analysis	C	4	75.3 (28.0)	76.3 (27.5)	24.3 (26.0)	99.0 (1.2)	100 (0)	73.8 (30.4)	74.8 (17.2)
Self- Management Planning	Interactive exercises teaching strategies to manage or avoid high-risk situations for drug use	C	4	62.0 (26.5)	73.5 (36.8)	49.0 (25.7)	96.5 (7.0)	98.8 (2.5)	63.5 (30.5)	73.9 (17.1)
Identifying Automatic Thoughts	Introduces the concept of traumatic thoughts (e.g., guilty thoughts and exaggerated thoughts about danger) and how certain emotions and environmental events can elicit thoughts that can have a damaging influence on emotions and behavior	C	8	57.9 (31.0)	68.6 (29.9)	42.5 (31.1)	80.6 (26.5)	86.9 (19.2)	55.6 (38.2)	65.4 (24.4)
Evaluating Automatic Thoughts	Interactive exercises ask veterans to evaluate traumatic thoughts to determine whether they are accurate; examples include pervasive beliefs about the world as an unsafe place and that people cannot be trusted	C	8	57.8 (35.7)	71.5 (26.7)	72.8 (26.6)	90.6 (14.1)	82.3 (16.8)	61.1 (35.3)	72.7 (23.4)
Challenging Automatic Thoughts	Interactive exercises ask veterans to challenge distorted thinking by developing facts that dispute a distorted thought, and constructing realistic alternative responses	C	7	58.6 (37.7)	64.7 (37.0)	68.7 (25.7)	92.7 (11.8)	83.7 (24.4)	56.9 (43.4)	70.9 (27.1)
Problem Solving	Teaches steps for effective problem solving, such as talking about the trauma and associated feelings with friends or family	O	3	62.7 (32.3)	61.7 (33.5)	61.7 (37.0)	93.3 (11.5)	94.3 (9.8)	72.7 (31.6)	74.4 (22.5)
Solving Real Problems	Interactive exercises to identify, analyze, and solve problems; examples include obtaining emotional support and reassurance to help relieve irrational feelings of guilt	O	3	82.7 (15.3)	92.7 (6.4)	55.7 (38.7)	93.3 (11.5)	99.3 (1.2)	100 (0)	87.3 (7.8)
Your Health and Wellness	Targets the importance of a healthy mind and body, and how to continually keep oneself healthy	O	2	79.5 (29.0)	87.0 (18.4)	84.0 (22.6)	100 (0)	100 (0)	94.5 (7.8)	90.8 (13.0)
Readiness to Change	Interactive exercise that asks veterans to assess their current readiness to change, and where they would like to be	C	3	64.0 (12.5)	71.0 (15.9)	81.7 (15.9)	94.3 (9.8)	88.7 (13.3)	72.0 (24.0)	78.6 (12.3)
Managing Anger	Interactive exercises asks veterans to identify angry thoughts and behaviors and how anger may be affecting their life; teaches the difference between constructive and destructive expressions of anger	O*	2	4.5 (5.0)	18.0 (24.0)	6.5 (2.1)	100 (0)	100 (0)	16.5 (21.9)	40.9 (8.8)
Communicating With Others	Gives step-by-step instructions for practicing effective communication (e.g., expressing emotions, asking for help, assertive behavior); explains how behavioral avoidance and emotional numbing can impair communication	O*	2	18.0 (24.0)	35.0 (21.2)	33.5 (2.1)	100 (0)	100 (0)	14.0 (12.7)	50.1 (5.8)
Overcoming Trauma With Exposure	Step-by-step instructions on how to complete writing exercises to confront feared trauma- related situations, memories, and images	O	4	66.0 (11.2)	81.5 (11.6)	85.8 (12.1)	100 (0)	87.0 (15.2)	80.3 (25.3)	83.4 (8.1)
Education About Pain	Describes the nature of chronic pain and common pain myths; educates veterans to assess their own level of chronic pain	O	3	88.7 (15.5)	100 (0)	79.7 (26.3)	100 (0)	100 (0)	99.3 (1.2)	94.6 (4.8)
Managing Chronic Pain	Gives step-by step instructions on pacing activities as a method to reduce pain and draws upon previously learned cognitive lessons to manage pain-related thoughts	O	3	88.7 (10.3)	90.7 (12.1)	83.7 (24.1)	100 (0)	100 (0)	90.7 (12.1)	92.3 (7.7)
Preventing Stress From Ruining Relationships	Provides information about the impact of one’s stress on other people and how to create a supportive environment	O	4	64.8 (15.8)	74.0 (18.2)	68.0 (24.3)	95.0 (10.0)	94.5 (8.0)	67.3 (24.4)	77.3 (12.1)
Insomnia	Teaches veterans to use a sleep diary to monitor sleep habits and to improve their hygiene	O	3	60.7 (30.4)	71.7 (24.8)	61.7 (18.6)	100 (0)	98.3 (2.9)	73.7 (38.9)	77.7 (18.4)
Time Management	Teaches veterans how to plan and schedule events to engage in activities using a time management worksheet	O	3	59.0 (29.5)	78.7 (28.0)	61.0 (38.6)	96.3 (6.4)	99.3 (1.2)	70.0 (45.9)	77.4 (22.8)
Tobacco Use	Interactive exercises to reduce or stop smoking including using a smoking diary and coping with negative thinking and emotions	O	3	60.7 (51.7)	65.0 (52.0)	83.7 (14.8)	100 (0)	97.3 (4.6)	71.7 (42.3)	79.7 (25.8)

Note.

^*Originally intended to be core modules but changed to optional based on participant feedback.

During their feedback sessions, veterans also had the opportunity to comment on what they "liked best," "liked least," and suggest ideas for improvement in a free-response format. Overall, there was a similar amount of positive and negative feedback across modules. Common positive themes included participants reporting that the program was useful, simple, informative, and offered a unique way to address problems. Common negative themes included concerns about having to click too much to get through modules, confusion of why quiz questions were presented more than once, and a minority of participants felt like they could not relate to some of the content about automatic thoughts or drugs/alcohol. To illustrate typical participant feedback, a few qualitative examples are described here. During review of “Overcoming Trauma Through Exposure,” a participant reported, “I liked that the program showed me how to confront ideas and images that I have of past events that took place.” “Regarding Education About Chronic Pain,” a participant said, “The module showed that many people experience pain, and often no cause for it is known. It reassured me that the pain is not made up.” During review of “Managing Chronic Pain,” a participant said, “It seems to progress a little slow. Some individuals may get bored.” Regarding “Challenging Automatic Thoughts” a participant reported, “This wouldn't help me because my anxiety symptoms have not appeared in conjunction with other automatic thoughts.”

Final Moving Forward Program

Based on data from feedback sessions on the beta program, a gold program was created. The final program contains 24 modules housed within a home page (Figure 1). Modules are self-paced, with each module taking approximately 15–25 minutes to complete. Modules include interactive exercises and veteran stories that illustrate common symptoms and coping strategies. Also included in the program are recurring queries assessing problematic substance use and trauma symptoms, graphs showing progress made in these areas (Figure 1), a workbook containing key points and fillable copies of exercises from each program module, and a resources page. The modules teach a variety of cognitive-behavioral skills via interactive exercises, including identifying, evaluating, and challenging automatic thoughts (Figure 2) and doing a functional analysis of problematic alcohol or drug use (Figure 3). Participants access the program using a unique login and password. The program also has a back-end system where usage data is tracked and stored on a secured server for later analyses.

Figure 1.

Screen shot of home page.

Figure 2.

Screen shot of sample thought record.

Figure 3.

Screen shot of a sample functional analysis of alcohol.

Based on veteran focus group feedback, the gold program titled “Coming Home and Moving Forward” is currently being tested in an RCT. Veterans (target N = 164) are being recruited from primary care clinics in four VA facilities. To be included they must be an OEF/OIF/OND veteran, report hazardous alcohol on the Alcohol Use Disorders Identification Test (AUDIT; score ≥ 8 for men, ≥ 7 for women; Babor, Biddle-Higgins, Saunders, & Monteiro, 2001) or substance use on the Drug Abuse Screening Test (DAST; score ≥ 2; Skinner, 1982) and report symptoms consistent with at least subthreshold PTSD (a military-related traumatic life event plus, at least, one reexperiencing symptom, three avoidance symptoms, or two hyperarousal symptoms) on the Clinician-Administered PTSD Scale (Blake et al., 1995). Participants who are currently receiving other treatments for PTSD or substance use, in need of inpatient substance detoxification, have a plan to harm themselves, or have documentation of a psychotic disorder or moderate to severe traumatic brain injury in their VA medical chart are excluded. To date, 106 eligible veterans have been enrolled. Full trial information can be found in the Clinical Trials Registry (www.clinicaltrials.gov) under NCT01710943.

Discussion

In response to the significant mental health and substance use needs of many OEF/OIF/OND veterans and the barriers (e.g., stigma, lack of convenience) to receiving evidence-based psychotherapy, a Web-based patient self-management program that teaches cognitive and behavioral skills to manage PTSD symptoms and problematic substance use was developed. The iterative development process incorporating feedback from a diverse research team, expert clinicians in the field, and OEF/OIF/OND veterans (the targeted “end users” of the program) proved invaluable in tailoring the intervention to the target audience of OEF/OIF/OND veterans and ensuring that the content and style of delivery would be acceptable and potentially effective for this population.

Our expert clinicians helped us to identify which therapeutic skills may be most effective, strategies to engage veterans in the treatment process, and allowed us to carefully consider risks and benefits of different tools for this population. Having nonresearcher experts weigh in on the intervention provided invaluable feedback on the safe and appropriate use of this intervention. In addition, feedback from the clinicians helped us to consider the contexts in which veterans may find the intervention useful. Results from veteran focus groups helped to make the intervention more contextually anchored in veterans' experiences and objectives for treatment. In addition, focus group feedback led us to adopt a strength-based approach to the content, and employ a delivery system via the Web-based program that allowed for maximum flexibility, adaptability, interactivity, and privacy for the veteran. Our intervention now addresses the stressful aspects of reintegration and teaches veterans strategies to manage problematic substance use, rather than abstinence-based strategies. In individual veteran feedback sessions, results indicated that veterans generally found the session topics likable, easy to use and relevant to their experiences, providing support for the research teams' hypotheses that Web-based technology would offer anonymity, privacy, and opportunity for self-pacing; features that might not be available through traditional encounters. Feedback from veterans during both initial focus groups and more specific feedback sessions helped to increase the authenticity of the intervention (i.e., how much the content “rang true” for veterans), which veterans noted as being an important factor in their engagement. Overall, our results support the importance of including an intervention's target audience when adapting a psychotherapeutic intervention to be delivered using a technology format. Ensuring representation from males and females, as well as individuals from diverse ethnic and racial backgrounds, enabled the team to gather relevant and useful feedback on the development process.

The feedback from combat veterans who screened positive for substance use and PTSD can be applied to guide clinicians wanting to develop, adapt, and/or provide technology-based treatment programs. Programs that avoid diagnostic labels and instead focus on improving symptoms that clients deem to be problematic are likely to be well received. Treatments that allow clients to progress at their own pace, have rigorous privacy safeguards, and can be used as a gateway to access additional treatment services may lead to high client engagement and utilization. Also, programs that allow patients to track their progress and symptom change using visual displays may help build client momentum. Several desired program features that were not built out in the current program should also be considered by clinicians when offering technology-based programs. These include enabling the program to be used as a communication device with current treatment providers and client peers, and making the program available on multiple platforms, including smartphones and tablets. The RCT currently under way will be able to assess whether the iterative developmental process described here results in a program that effectively increases access and extends the scope of mental health and substance use treatment to a population with significant mental health concerns that are commonly undertreated.

The methods used to develop our intervention can be applied to a variety of treatment development contexts because they are client centered and allow stakeholder (expert and client) input to refine the final intervention product. The process of (a) developing a treatment within a research team, (b) seeking expert input, (c) eliciting both group and individual feedback from the target audience/end user, and then (d) revising the treatment based on this feedback can be used regardless of type of treatment or presenting concern of the end user. For instance, this treatment development approach can be applied equally well to women with postpartum depression or minorities who want to quit smoking because it allows every unique sample to provide individualized feedback. The methods outlined here are particularly relevant to technology-based interventions because they provide a process for the translation of in-person interventions to new platforms (e.g., Web or mobile application) that includes gathering systematic feedback from multiple stakeholders on both the intervention content and the technology-based delivery system. Several of the specific strategies we employed, such as CAI and interactive exercises, are especially applicable to interventions that seek to teach specific behavioral skills. While we describe a self-management intervention here, this treatment development approach could also be used to develop clinician-guided interventions, if the feedback process was modified to include the clinician end users. A similar process of gathering stakeholder feedback can also be useful in later stages of research when effective interventions need to be adapted for implementation in real-world clinics (Curran, Mukherjee, Allee, & Owen, 2008). Should this Web-based intervention effectively reduce PTSD symptoms and substance use, our research team plans to conduct future research on how best to implement this in VA primary care clinics.

Highlights.

• Eliciting target audience feedback when adapting an intervention is optimal.

• Strength-based technology treatments are likely to be well received.

• Programs that allow self-pacing may lead to high client utilization.

• Rigorous privacy standards are necessary for client engagement.

• Enabling clients to track their progress may help build momentum.