Abstract
Implementation of evidence-based treatments (EBT) is necessary to address posttraumatic stress disorder (PTSD) in Operation Enduring Freedom/Operation Iraqi Freedom (OEF/OIF) military service personnel. Because virtual reality (VR) offers a promising tool for delivery of one type of EBT—exposure therapy—this study explored veterans' perceptions of VR as an assessment tool and treatment adjunct. We conducted semi-structured interviews with 14 OEF/OIF veterans being treated for PTSD after viewing two 3 minute VR scenarios as part of a larger research study. Veterans reported a capacity for immersion in VR in both combat and civilian environments, characterized by self-reported physiological reactivity, thoughts/behaviors similar to those experienced in Iraq, and triggered memories. Although participants were generally positive about VR, they expressed concerns about the possibility of negative reactions after viewing VR. Findings are discussed in the context of further development of VR aided interventions in veteran healthcare systems.
Introduction
Returning Operation Enduring Freedom/Operation Iraqi Freedom (OEF/OIF) military service personnel experience high rates of posttraumatic stress disorder (PTSD) and related mental health problems,1–5 creating a need for innovative strategies to screen, systematically monitor treatment progress, and reduce symptoms. Although exposure therapy has the strongest empirical evidence for PTSD treatment,6 resulting in large scale dissemination initiatives by the Veterans Healthcare System, fewer than 10% of clinicians have volunteered for training.7 Evidence suggests that clinicians do not fully accept or use exposure therapy because of concerns about effectiveness and safety in community8 and veterans' healthcare settings.9,10 In addition, studies indicate that some individuals may not respond to this type of intervention due to difficulty activating the trauma memory or poor emotional engagement in the treatment process.11
Virtual reality (VR) environments represent a unique method to immerse veterans in simulated military and civilian scenarios while monitoring psychophysiological reactivity12 as part of an exposure therapy. The technology integrates computer graphics via a headgear device that displays the VR environments directly to the veteran rather than on a computer screen, and visual and auditory stimuli that can be modified during the intervention. Psychophysiological data can be simultaneously monitored during VR and stored using specialized software. Because of the strong association between PTSD and increased heart rate, cardiac acceleration, skin conductance, and more pronounced eye-blink responses,13–24 VR may also be an effective tool to assess and monitor outcomes for PTSD. In one of the few published studies on the physiological effect of VR, Wood et al.25 found increased arousal, as evidenced by high skin conductance, peripheral temperature, and heart rate, in an OIF veteran with PTSD while viewing Virtual Baghdad© (www.vrphobia.com/), a VR environment modeled on Iraqi combat and civilian environments developed by the Virtual Reality Medical Center in San Diego, CA. Case studies of six Navy personnel deployed to Iraq documented similar results.26
When used as an adjunct to exposure therapy, VR has also shown promise for reducing symptoms of PTSD in Vietnam and OIF veterans.12,27–32 Basic components of treatment include exposure to visual and auditory stimuli resembling the original trauma setting followed by a recounting of memories specific to the individual during the event(s). Case studies have also documented the feasibility of using VR in clinical settings and patients' general acceptance of VR.25,26,30 In a larger study of OIF/OEF veterans without PTSD, 86% and 82% rated the overall realism of a VR combat and civilian environment respectively as adequate to excellent.11 Moreover, in a survey of 352 military service personnel, 58% reported some willingness to use a VR based therapy.33
Despite general acceptance of VR by research participants, no study to date has systematically explored veterans' perceptions. Although multiple barriers and facilitators exist for implementing evidence-based practices into routine care,34,35 patient preferences are extremely critical and may constitute one of the most important factors in the dissemination of new innovations. Therefore, the purpose of the present study was to explore the acceptability, appropriateness, and effectiveness of using VR to assess for PTSD related symptoms and serve as a therapeutic tool in PTSD treatment based on interviews from OIF/OEF veterans in treatment for PTSD.
Methods
Participants
Participants consisted of 14 OEF/OIF veterans recruited through inpatient/residential and outpatient settings through the Central Arkansas Veterans Healthcare System in North Little Rock. Mean±SD age was 32.9±7.8 years. Twelve were male; two were female. Half was Caucasian; four participants (28.6%) were African American, two (14.3%) were Native American, and one (7.1%) identified himself as Caucasian of Hispanic ethnicity. Most were married or cohabitating (64%). All had received at least a high school diploma or equivalent.
Measures
The Clinician-Administered PTSD Scale (CAPS) is a 34-item structured interview assessing PTSD symptoms with excellent psychometric properties.36 Aggregate CAPS scores are derived by summing intensity and frequency ratings. Participants reported their Subjective Units of Distress (SUDS)37 on a scale of 0 to 100 before and after viewing the VR combat and civilian scenarios. After viewing the VR environments, participants also completed a 7-item Presence Questionnaire modified from Witmer and Singer.38 The 7-point Likert-type scale assesses the level of absorption participants experience during VR by asking, for example, “How much did the visual aspects of the environment engage you?” Maximal presence occurs when the user feels immersed in the computer generated environment, feels capable of interaction with the environment, and has an interest in the environment or task portrayed.
Procedures
Participants were recruited from a larger parent study examining psychophysiological reactivity in veterans with VR from 2008 to 2010. For the purposes of the current study, only participants in treatment were included. Initial contact with potential participants for the larger parent study occurred in person or by phone. After the parent study was briefly described and the individual expressed interest, we conducted the informed consent and eligibility process. Once participants for the parent study completed their baseline assessment, including experiencing the VR environments with simultaneous psychophysiological monitoring, every third participant was asked if he or she would participate in a qualitative interview about the VR scenarios. If the participant consented, we randomized him or her to one of two qualitative interviews focusing on the use of VR in conjunction with psychophysiological measures for PTSD screening or the use of VR as an adjunct for exposure-based treatment. The CAVHS Institutional Review Board and Research and Development Committee approved the study. Participants received $10 for the qualitative interview.
The VR experience consisted of two 3 minute, low threat scenarios of combat and civilian environments with visual and auditory stimuli using software developed at the Virtual Reality Medical Centers, San Diego. The low threat combat environment consisted of an Iraqi marketplace through which the soldier progresses as if on a moving walkway, while the noise level, including distant bombing and helicopter sounds, increases and crowd activity intensifies. The low threat civilian environment involved walking down a city street that becomes progressively more crowded and noisier. The VR experience was counterbalanced; a rest period of 5 minutes between and after observing the VR allowed for psychophysiological measures to return to baseline. Because no one had experienced VR as a treatment intervention, participants also viewed a 7 minute DVD presentation previously aired on CNN (January 28, 2007) summarizing VR graded exposure therapy for PTSD, narrated by Dr. Sanjay Gupta. (The DVD was purchased from CNN, and additional information can be obtained from the second author.)
A research assistant experienced in qualitative methods conducted the interviews with supervision from the first author. The grand tour question for individuals asked about treatment (n=8) was “Given this explanation of VR and your exposure to it in the parent study, what would you think about using it as a treatment tool for PTSD?” The grand tour question for individuals asked about assessment (n=6) was “Given your exposure to VR, what would you think about using it as a screening or assessment tool for PTSD?” Subsequent probe questions covered areas such as whether the veteran had different responses to the civilian versus combat environments, aspects of VR they liked or disliked, and the advantages and disadvantages of using VR as a screening tool or intervention for PTSD. Participants also ranked VR on a 5-point scale ranging from very unacceptable (1) to very acceptable (5). The digitally recorded interviews lasted 30 to 45 minutes. The interviews were transcribed, checked for accuracy, and identifying information deleted. Digital recordings were destroyed once accuracy of the transcripts was confirmed.
The first author reviewed all transcripts to develop an initial coding scheme based on primary themes identified by veterans. After completing this phase, the first and second authors independently coded five transcripts to assess understanding of codes, discuss disagreements, and modify the coding manual. An additional three transcripts were independently coded and checked by the first and second authors. Once this phase was complete, the coders achieved a final interrater agreement of 80% on all coding segments. Discrepancies were reviewed until consensus was reached. The team also recorded all coding and thematic decisions to provide an audit trail of the analytic process.38,39
Results
Twenty-six veterans consented to the qualitative interview. We excluded participants not in treatment at the time they viewed the VR or those interviewed more than 1 month following their VR experience, resulting in 14 interviews. None had experience using VR in assessment or clinical interventions. The average number of years in the military for participants was 9.9 (SD=6.4). The mean CAPS36 score at baseline was 73.6 (SD=22.8); 12 of the 14 were above a cut-off of 50 on the instrument.
Five primary themes depicted in Table 1 were identified. Participants most frequently discussed their degree of immersion in the VR environment, although they did not specifically use the terms immersion or presence. When they entered the scenario, they reported rapid breathing, sweating, or heart racing, which did not dissipate during the 3 minutes of the VR experience, and selectively attending to certain aspects of the environment that might suggest the presence of an improvised explosive device (IED), such as cartons in the marketplace, empty vehicles, or cracks on the ground that reminded them of wires. During the interview, most participants described in detail their experiences during deployment, comparing the VR environment to specific memories.
Table 1.
Coding Category, Definition, and Examples of Participant Responses to Marketplace and Civilian Virtual Reality (VR) Scenarios
| Category | Definition | Examples |
|---|---|---|
| Immersion | The phenomenon of holding two concepts of time and place simultaneously. Characterized by behaving, thinking, or feeling as if the veteran is in Iraq rather than a research office; report of physiological reactivity such as heart racing, sweating, increased respiration; and discussions of memories triggered by the experience. This category is a continuum and can include the absence of immersion, mild to moderate immersion, or significant immersion. |
Iraqi marketplace • “It brought back triggers. Like, one of my real big triggers is the women with their head covers or whatever they're called … all I have to do is see it and I start having a panic attack” (CAPS >50; Post-VR Combat SUDS >50). • “With the Iraqi scene, I found myself doing what I used to do when I was out on patrol … scanning the ground, scanning the rooftops, scanning the windows, looking all over the place, and I actually started sweating a little bit …” (CAPS >50; Post-VR Combat SUDS >50). • “I know my heart rate went up, I could tell that, probably more than it did when I was actually there” (CAPS <50; Post-VR Combat SUDS <50). • “It's real calm so I really didn't get too anxious … No one got hurt … No one was really hiding or ducking or anything like that. I guess it wasn't too realistic enough to get nervous” (CAPS >50; Post-VR Combat SUDS <50). Civilian Street Scene • “And that's one big contrast from the civilian one, because I wasn't looking around that much. I knew I was in a safe place, you know. I didn't feel the need to, you know, scan everywhere” (CAPS >50; Post-VR Civilian SUDS <50). • “If someone came speeding up on us like that, you know, we would have been ready already and before they got that close they would have gotten ‘lit up’ [fired upon]” (CAPS >50; Post-VR Civilian SUDS >50). • “It's like at one point there was the fire engine going and people talking … It was like overload for me almost. I feel like I got rattled a little bit” (CAPS >50; Post-VR Civilian SUDS >50). |
| Content/technology | Positive or negative characteristics of the VR environment that were facilitators or barriers to immersion. This category may include suggestions for modifications in the VR environment, equipment, administration, graphics, or physiological monitoring. |
Iraqi marketplace • “The sounds in it and everything were dead-on, so I was impressed with that, really impressed” (CAPS >50; Post-VR Combat SUDS >50). • “To me, it felt pretty realistic, you know. I mean, I wish I would've had, you know, like a gun I could shoot back with” (CAPS >50; Post-VR Combat SUDS >50). • “It needs to be a little bit more horrific. It could have just been different, like when the shots and stuff went off, the people ducked down” (CAPS >50; Post-VR Combat SUDS <50). Civilian street scene • “The sounds were pretty vivid, you know, it's like you're actually walking down the street; you're hearing birds chirping, cars honking, and people talking and it was actually realistic conversations” (CAPS >50; Post-VR Civilian SUDS <50). |
| Impact of time | Comments that time has influenced or may influence the veteran's responses to the VR, e.g., before vs. after deployment, immediately postdeployment, or 6 months following deployment. |
Iraqi marketplace • “When you first get back, if I watched that when I first got back, it would probably be like, maybe like a joke … Because you come from that real intense … but, you know, like as time goes by, and then it might be that you know to help the effects out also” (CAPS >50; post-VR SUDS >50). • “As far as trying to get them to calm back down after getting back from over there, it probably won't do that because I've been back for almost 2.5 years and it got me fired back up” (CAPS >50; Post-VR Combat SUDS >50). |
| Postsafety concerns | Concerns expressed by the veteran pertaining to safety, emotional reactivity, and difficulty returning to baseline after the VR experience. |
Iraqi marketplace • “I'd say it took a couple of weeks to clearly get it out of my head and get my focus back on into the program that I'm in” (CAPS >50; Post-VR Combat SUDS >50). • “If you're treating this as an immunization you give now, and you send him outside, you just created an anaphylactic reaction in this guy's brain and you're not monitoring him” (CAPS <50; Post-VR Combat SUDS <50). • “ ’Cause, I mean, you could get somebody that experienced a lot more than I did, and whenever they go to get fired up, they end up getting somebody hurt” (CAPS >50; Post-VR Combat SUDS >50). • “Just the anxious feelings whenever you first get back at home, you just want to, you don't want to think about that. You just want to be around your family and on vacation” (CAPS <50; Post-VR Combat SUDS >50). • “I could probably see it making a soldier more active, maybe if they were suicidal” (CAPS <50; Post-VR Combat SUDS >50). |
| Veteran preferences/appropriateness | General comments about VR, such as “It was cool” or “This is not for me” as well as the type of veteran who might benefit from VR (e.g., younger veterans). |
Iraqi marketplace • “It would really depend on the situation and the person themselves and how they prefer to deal with their situation” (CAPS >50; Post-VR Combat SUDS >50). • “You'd have to be cautious on which one, like who you actually tried to screen with it. Like I said, somebody who's been through a lot could end up possibly getting somebody hurt” (CAPS >50; Post-VR Combat SUDS >50). • “I think it could be very helpful for the younger group” (CAPS <50; Post-VR Combat SUDS <50). |
| Treatment | Specific comments about VR as a treatment tool, including how it would be incorporated into the therapeutic relationship, benefits of using it as a therapeutic tool, etc. |
Iraqi marketplace • “I'd feel like it might bring your emotions to the surface a little bit more … And it might be easier for you to express ’em or maybe even talk about ’em” (CAPS >50; Post-VR Combat >50). • “I mean, so if they're trying to trigger something and then maybe help that Marine or that service member from that point deal with, you know, getting in touch with those feelings and deal with them on the spot, I can see that being beneficial” (CAPS >50; Post-VR Combat >50). • “Every time you did it then it'd just be less … you'd be more numb to it, I guess” (CAPS >50; Post-VR Combat SUDS <50). • “As you go through that same scenario over and over, it helps you to better cope with the intrusive thought or with the nightmare. It'll help you cope, but it won't cure it” (CAPS >50; Post-VR Combat SUDS >50). • “I think that would actually be a very effective tool” (CAPS >50; Post-VR Combat SUDS >50). |
| Assessment | Specific comments about VR as an assessment tool. |
Iraqi marketplace • “That would actually show if they do have a problem, what the anxiety or the stuff is that is related to PTSD. That would definitely bring it out. So, it could possibly be used as a diagnosis tool” (CAPS >50; Post-VR Combat SUDS >50). • “I think you'd be able to tell who have and who haven't been there” (CAPS >50; Post-VR Combat SUDS >50). • “This doctor can take this and go by statistics and look and see what various people that have been over there with various jobs … and see how they react according to their diagnosis of PTSD and what can be done to change things for the better” (CAPS <50; Post-VR Combat SUDS >50). |
Another key theme pertained to specific aspects of VR's visual and auditory technology and content that enhanced immersion. For example, the sounds of helicopters, fire fights, people talking or yelling, and Middle Eastern music contributed to the realism of the military scene. However, participants also recommended VR changes that might intensify feelings of presence, such as having access to and ability to fire a gun. Moreover, VR typically permits the participant to control movement, pace, and visual field. However, in the current study, participants progressed through the market place or urban street as if on a moving walkway to limit variability in the amount of time and type of visual cues to which they were exposed. Most found these limitations to be frustrating, and complained that such restrictions affected the extent to which they connected with the VR environment. They also recommended several modifications in content, including improved synchronization between visual and auditory stimuli, access to more diverse VR environments, and increased emotional reactions from virtual people in the Iraqi market when bombs exploded in the distance, helicopters flew overhead, or virtual Iraqi soldiers appeared.
Several participants expressed safety concerns about VR. Specifically, they were concerned that VR would activate anxiety, which veterans would not be able to moderate before leaving the assessment or treatment. Some reported they had difficulties returning to baseline after their experience with the VR. The least mentioned independent theme was the impact of time. Several participants reported that viewing the VR environment immediately after deployment might activate memories before veterans were prepared or had acclimated to being home. In addition, participants were concerned that experiencing a VR environment immediately postdeployment might cause additional stress to a soldier “when all he wants is to get back to his family.”
Different responses emerged when participants compared the civilian and military VR environments. The majority reported less immersion overall in the civilian environment. Participants minimally addressed the impact of time, safety concerns, and veteran preferences with regard to the civilian scene.
Table 2 shows participant responses to specific items on the Presence Questionnaire. In addition, participants' mean SUDS scores increased from baseline (28.4; SD=26.8) to post-VR (67.9; SD=30.6) on the military VR scenarios (t=4.3, p<0.0009). Baseline (M=17.0; SD=20.2) and post-VR mean scores (27.1; SD=26.1) in response to the civilian environments were not as high, and there was no significant difference between baseline and post-VR experience. Only three participants (CAPS scores=46, 52, and 52) reported minimal or no physiological response to either the marketplace or civilian VR environments. An additional participant denied feeling anxious or scared but admitted to feeling “angered” and “irritated” by the Iraqi scene, primarily due to the characteristics of the virtual people, which triggered his stress response. Quantitative rankings ranged from 2 to 5 on VR as an assessment tool (M=3.88; mode=4) and 3 to 5 on VR as a therapeutic tool (M=4.33; mode=4).
Table 2.
Mean (Standard Deviation) for Individual Items (Range=0–7) on the Presence Questionnaire by VR Environment Type
| Item | Military | Civilian |
|---|---|---|
| How natural did your interactions with the environment seem?a | 5.1 (1.5%) | 4.8 (1.4) |
| How much did the visual aspects of the environment involve you?b | 5.0 (1.4) | 4.9 (1.5) |
| How much did the auditory aspects of the environment involve you?b | 5.6 (1.4) | 5.2 (1.7) |
| How much did your experiences in the VR seem consistent with your real world experiences?b | 4.9 (1.5) | 5.6 (1.2) |
| How completely were you able to actively survey or search the environment using vision?b | 5.8 (0.8) | 5.8 (1.0) |
| How well could you identify sounds?b | 6.2 (0.9) | 6.1 (1.0) |
| How involved were you in the virtual environment experience?b | 5.9 (0.7) | 5.9 (1.0) |
Artificial=0; natural=7; blow=0; high=7.
Discussion
Dissemination and implementation of new technologies in healthcare have long been studied, beginning with the introduction of tetracycline in 1966 in four U.S. Midwestern cities40 and extending to interventions in mental healthcare in the past decade.41–44 Multiple barriers and facilitators have been identified, including characteristics of the provider, organization, consumer, and intervention itself.31,34 However, in the domain of mental health, few studies have focused on consumer perceptions of interventions prior to development or application of the intervention in clinical settings.
Veterans in treatment for PTSD were generally impressed by and responded to the realism of the VR environment. Veterans with a cut-off of 50 or more on the CAPS and more than 50 on the SUDS consistently reported feeling anxious, fearful, “on guard,” suspicious, and hypervigilant. According to Witmer and Singer,38 the necessary conditions for presence in a VR environment are involvement, which is focusing one's attention on the stimuli, and immersion, when the individual feels enveloped by an experience. These conditions are determined by four factors: (a) control, (b) sensory stimulation, (c) distraction, and (d) scene realism. As mentioned, we limited control to preserve the integrity of the research study. Most VR environments used for treatment purposes allow the participant to walk freely into the environment, turn as needed to scan the scene, and decrease or increase pace. The participants in this study requested more control over their movements, 180° vision, and access to a weapon or artillery, which purportedly would have enhanced their sense of immersion. The sensory stimulation of the VR environment included visual as well as auditory signals. Generally, the veterans were positive about this aspect of the VR environment, especially the auditory stimuli in the Iraqi marketplace. With regard to distractibility, the participants were most influenced by the headgear and wires for psychophysiological monitoring. However, this did not appear to alter significantly their ability to engage in the VR experience. Veterans' most frequently suggested modifications included customization of scenarios to “fit” the tour of duty, which are components of existing commercially available VR programs used in treatment applications; improvement in technology, such as visual and auditory synchronicity and quality; presence of other soldiers, since missions in the field were conducted with unit members; and more intensive portrayals of the battlefield, which we omitted for the purposes of this study due to concerns about overstimulating research participants.
The results also suggest that the use of VR will face challenges inherent in the adoption and assimilation of any new evidence-based practice into routine care. For example, participants expressed concerns about experiencing such an intervention without appropriate follow-up, which would occur if VR were used as a screening or assessment tool. They were more favorable toward VR as a treatment tool, primarily because they perceived its use in the context of a therapeutic modality in which ongoing monitoring of veterans' reactions would occur.
The findings are interesting in light of clinicians' responses to VR.10 Generally, clinicians viewed VR as unrealistic and doubted whether veterans would be capable of immersing themselves in the scenarios. Clinicians also wanted more empirical evidence to support the use of VR as an assessment or therapeutic tool, and expressed concerns about veterans' negative reactions to the scenes. Unlike clinicians, veterans did not mention concerns about the evidence supporting VR, preference for type of clinician administering VR, or the potential for VR to interfere with service connected disability benefits. In fact, several suggested using VR to determine whether a veteran had actually been exposed to traumatic combat during his or her tour of duty. Clinicians and veterans both expressed a need for close monitoring and follow-up of VR recipients due to the potential exacerbation of PTSD symptoms after experiencing a VR environment.
Several limitations of the current study may affect the generalizability of the results. Although the participants discussed an array of issues to consider in the use of VR, their views may not be representative of veterans in other settings. The sample size was small and was comprised solely of volunteers participating in a larger VR study who were in treatment at the time. It is unknown how other veterans would respond to this type of intervention in a clinical environment as part of their inpatient or outpatient care. Moreover, to control the VR experience across participants for comparison of psychophysiological reactivity in a research study, our VR environment prohibited range of movement, use of artillery, or customization of visual or auditory stimuli. Intensity was set at a low threat level, and each scenario lasted only 3 minutes compared to the 20–30 minutes veterans might experience during exposure therapy. It is unknown how veterans would respond to a VR environment that is customized, lasts longer, and/or has a higher threat level. In addition, participants watched a video on the use of VR in therapy, which may have biased their perceptions prior to the interview. There was also a variation in time between initially experiencing the VR and participating in an actual interview, which may have resulted in reported differences due to retrospective reporting. Finally, although research has demonstrated VR to be effective when used in the context of exposure therapy with a trained clinician, early dropout may be associated with safety concerns raised by the veterans.
Despite the limitations, this study illustrates veterans' perceptions of a promising but new intervention as well as challenges in large-scale dissemination. Importantly, participants experienced an increase in self-rated distress in both military and civilian VR environments, supporting the use of this technology to screen for PTSD and other anxiety responses. Moreover, it appears that most participants were favorably inclined toward VR as a treatment tool because of its capacity to evoke memories and trigger psychophysiological responses that could then be discussed in a therapeutic setting. Changes in VR technology might be considered before widespread dissemination, but the flaws were minor and minimally influenced veterans' preferences. Routine monitoring and follow-up are required to address perceived safety concerns.
Acknowledgments
This research was supported by a grant (065-VANIMH) from the Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development, and the National Institute of Mental Health. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States government. The authors would like to express their appreciation to the OIF/OEF veterans willing to share their perceptions of VR and personal stories with the research team.
Author Disclosure Statement
No competing financial interests exist.
References
- 1.Hoge CW. McGurk D. Thomas JL, et al. Mild traumatic brain injury in U.S. soldiers returning from Iraq. New England Journal of Medicine. 2008;358:453–63. doi: 10.1056/NEJMoa072972. [DOI] [PubMed] [Google Scholar]
- 2.Hoge CW. Castro CA. Messer SC, et al. Combat duty in Iraq and Afghanistan, mental health problems, and barriers to care. New England Journal of Medicine. 2004;351:13–22. doi: 10.1056/NEJMoa040603. [DOI] [PubMed] [Google Scholar]
- 3.Kolkow TT. Spira JL. Morse JS, et al. Post-traumatic stress disorder and depression in health care providers returning from deployment to Iraq and Afghanistan. Military Medicine. 2007;172:451–5. doi: 10.7205/milmed.172.5.451. [DOI] [PubMed] [Google Scholar]
- 4.Milliken CS. Auchterlonie JL. Hoge CW. Longitudinal assessment of mental health problems among active and reserve component soldiers returning from the Iraq war. Journal of the American Medical Association. 2007;298:2141–8. doi: 10.1001/jama.298.18.2141. [DOI] [PubMed] [Google Scholar]
- 5.Seal KH. Bertenthal D. Maguen S, et al. Getting beyond “Don't ask; don't tell”: an evaluation of US Veterans Administration postdeployment mental health screening of veterans returning from Iraq and Afghanistan. American Journal of Public Health. 2008;98:714–20. doi: 10.2105/AJPH.2007.115519. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Institute of Medicine. Treatment of posttraumatic stress disorder: an assessment of the evidence. Washington, DC: National Academies Press; 2007. [Google Scholar]
- 7.Karlin BE. Ruzek JI. Chard KM, et al. Dissemination of evidence-based psychological treatments for posttraumatic stress disorder in the Veterans Health Administration. Journal of Traumatic Stress. 2010;23:663–73. doi: 10.1002/jts.20588. [DOI] [PubMed] [Google Scholar]
- 8.Frueh BC. Cusack KJ. Grubaugh AL, et al. Clinician perspectives on cognitive behavioral treatment for PTSD among persons with severe mental illness. Psychiatric Services. 2006;57:1027–31. doi: 10.1176/ps.2006.57.7.1027. [DOI] [PubMed] [Google Scholar]
- 9.Cahill SP. Foa EB. Hembree EA, et al. Dissemination of exposure therapy in the treatment of posttraumatic stress disorder. Journal of Traumatic Stress. 2006;19:597–610. doi: 10.1002/jts.20173. [DOI] [PubMed] [Google Scholar]
- 10.Kramer TL. Pyne JM. Kimbrell TA, et al. Clinician perceptions of virtual reality to assess and treat returning veterans. Psychiatric Services. 2010;61:1157–61. doi: 10.1176/ps.2010.61.11.1153. [DOI] [PubMed] [Google Scholar]
- 11.Reger GM. Gahm GA. Rizzo AA, et al. Soldier evaluation of the virtual reality Iraq. Telemedicine & e-Health. 2009;15:101–4. doi: 10.1089/tmj.2008.0050. [DOI] [PubMed] [Google Scholar]
- 12.Rothbaum BO. Hodges L. Alarcon R, et al. Virtual reality exposure therapy for PTSD Vietnam veterans: a case study. Journal of Traumatic Stress. 1999;12:263–71. doi: 10.1023/A:1024772308758. [DOI] [PubMed] [Google Scholar]
- 13.Blanchard EB. Kolb LC. Pallmeyer TP, et al. A psychophysiological study of post traumatic stress disorder in Vietnam veterans. Psychiatric Quarterly. 1982;54:220–9. doi: 10.1007/BF01064817. [DOI] [PubMed] [Google Scholar]
- 14.Blanchard EB. Kolb LC. Prins A. Psychophysiological responses in the diagnosis of posttraumatic stress disorder in Vietnam veterans. Journal of Nervous & Mental Disease. 1991;179:97–101. doi: 10.1097/00005053-199102000-00006. [DOI] [PubMed] [Google Scholar]
- 15.Blanchard EB. Hickling EJ. Buckley TC, et al. Psychophysiology of posttraumatic stress disorder related to motor vehicle accidents: replication and extension. Journal of Consulting & Clinical Psychology. 1996;64:742–51. doi: 10.1037//0022-006x.64.4.742. [DOI] [PubMed] [Google Scholar]
- 16.Forneris CA. Butterfield MI. Bosworth HB. Physiological arousal among women veterans with and without posttraumatic stress disorder. Military Medicine. 2004;169:307–12. doi: 10.7205/milmed.169.4.307. [DOI] [PubMed] [Google Scholar]
- 17.Karl A. Malta LS. Alexander J, et al. Startle responses in motor vehicle accident survivors: a pilot study. Applied Psychophysiology & Biofeedback. 2004;29:223–31. doi: 10.1023/b:apbi.0000039060.57276.aa. [DOI] [PubMed] [Google Scholar]
- 18.Keane TM. Kolb LC. Kaloupek DG, et al. Utility of psychophysiological measurement in the diagnosis of posttraumatic stress disorder: results from a Department of Veterans Affairs cooperative study. Journal of Consulting & Clinical Psychology. 1998;66:914–23. doi: 10.1037//0022-006x.66.6.914. [DOI] [PubMed] [Google Scholar]
- 19.Malloy PF. Fairbank JA. Keane TM. Validation of a multimethod assessment of posttraumatic stress disorders in Vietnam veterans. Journal of Consulting & Clinical Psychology. 1983;51:488–94. doi: 10.1037//0022-006x.51.4.488. [DOI] [PubMed] [Google Scholar]
- 20.Metzger LJ. Orr SP. Berry NJ, et al. Physiologic reactivity to startling tones in women with posttraumatic stress disorder. Journal of Abnormal Psychology. 1999;108:347–52. doi: 10.1037//0021-843x.108.2.347. [DOI] [PubMed] [Google Scholar]
- 21.Pitman RK. Orr SP. Forgue DF, et al. Psychophysiologic assessment of posttraumatic stress disorder imagery in Vietnam combat veterans. Archives of General Psychiatry. 1987;44:970–5. doi: 10.1001/archpsyc.1987.01800230050009. [DOI] [PubMed] [Google Scholar]
- 22.Pitman RK. Lanes DM. Williston SK, et al. Psychophysiologic assessment of posttraumatic stress disorder in breast cancer patients. Psychosomatics. 2001;42:133–40. doi: 10.1176/appi.psy.42.2.133. [DOI] [PubMed] [Google Scholar]
- 23.Shalev AY. Peri T. Brandes D, et al. Auditory startle response in trauma survivors with posttraumatic stress disorder: a prospective study. American Journal of Psychiatry. 2000;157:255–61. doi: 10.1176/appi.ajp.157.2.255. [DOI] [PubMed] [Google Scholar]
- 24.Wolfe J. Chrestman KR. Ouimette PC, et al. Trauma-related psychophysiological reactivity in women exposed to war-zone stress. Journal of Clinical Psychology. 2000;56:1371–9. doi: 10.1002/1097-4679(200010)56:10<1371::AID-JCLP8>3.0.CO;2-X. [DOI] [PubMed] [Google Scholar]
- 25.Wood DP. Murphy J. Center K, et al. Combat-related post-traumatic stress disorder: a case report using virtual reality exposure therapy with physiological monitoring. Cyberpsychology & Behavior. 2007;10:309–15. doi: 10.1089/cpb.2006.9951. [DOI] [PubMed] [Google Scholar]
- 26.Wood DP. Murphy JA. Center KB, et al. Combat related post traumatic stress disorder: a multiple case report using virtual reality graded exposure therapy with physiological monitoring. Studies in Health Technology & Informatics. 2008;132:556–61. [PubMed] [Google Scholar]
- 27.Parsons TD. Rizzo AA. Affective outcomes of virtual reality exposure therapy for anxiety and specific phobias: a meta-analysis. Journal of Behavior Therapy & Experimental Psychiatry. 2008;39:250–61. doi: 10.1016/j.jbtep.2007.07.007. [DOI] [PubMed] [Google Scholar]
- 28.Powers MB. Emmelkamp PM. Virtual reality exposure therapy for anxiety disorders: a meta-analysis. Journal of Anxiety Disorders. 2008;22:561–9. doi: 10.1016/j.janxdis.2007.04.006. [DOI] [PubMed] [Google Scholar]
- 29.Reger GM. Holloway KM. Candy C, et al. Effectiveness of virtual reality exposure therapy for active duty soldiers in a military mental health clinic. Journal of Traumatic Stress. 2011;24:93–6. doi: 10.1002/jts.20574. [DOI] [PubMed] [Google Scholar]
- 30.Rizzo A. Reger G. Gahm G, et al. Virtual reality exposure therapy for combat related PTSD. In: Shiromani PJ, editor; Keane TM, editor; LeDoux JE, editor. Post-traumatic stress disorder: basic science and clinical practice. Totowa, NJ: Human Press; 2009. pp. 375–400. [Google Scholar]
- 31.Rothbaum BO. Hodges LF. Ready D, et al. Virtual reality exposure therapy for Vietnam veterans with posttraumatic stress disorder. Journal of Clinical Psychiatry. 2001;62:617–22. doi: 10.4088/jcp.v62n0808. [DOI] [PubMed] [Google Scholar]
- 32.Spira JL. Pyne JM. Virtual reality and other experimental therapies for combat-related PTSD. Primary Psychiatry. 2006;13:58–64. [Google Scholar]
- 33.Wilson JA. Onorati K. Mishkind M, et al. Soldier attitudes about technology-based approaches to mental health care. Cyberpsychology & Behavior. 2008;11:767–9. doi: 10.1089/cpb.2008.0071. [DOI] [PubMed] [Google Scholar]
- 34.Fixsen DL. Naoom SF. Blase KA, et al. Tampa, FL: The National Implementation Research Network, Louis de la Parte Florida Mental Health Institute, University of South Florida; 2005. Implementation research: a synthesis of the literature. [Google Scholar]
- 35.Schoenwald SK. Hoagwood K. Effectiveness, transportability, and dissemination of interventions: what matters when? Psychiatric Services. 2001;52:1190–7. doi: 10.1176/appi.ps.52.9.1190. [DOI] [PubMed] [Google Scholar]
- 36.Blake DD. Weathers FW. Nagy LM, et al. The development of a clinician-administered PTSD scale. Journal of Traumatic Stress. 1995;8:75–90. doi: 10.1007/BF02105408. [DOI] [PubMed] [Google Scholar]
- 37.Wolpe J. The practice of behavior therapy. New York: Pergamon Press; 1969. [Google Scholar]
- 38.Witmer BG. Singer MJ. Measuring presence in virtual environments: a presence questionnaire. Presence. 1998;7:225–40. [Google Scholar]
- 39.Rodgers BL. Cowles KV. The qualitative research audit trail: a complex collection of documentation. Research in Nursing & Health. 1993;16:219–26. doi: 10.1002/nur.4770160309. [DOI] [PubMed] [Google Scholar]
- 40.Coleman JS. Medical innovation: a diffusion study. Indianapolis, IN: The Bobbs-Merrill Co.; 1966. [Google Scholar]
- 41.Henderson JL. MacKay S. Peterson-Badali M. Closing the research–practice gap: factors affecting adoption and implementation of a children's mental health program. Journal of Clinical Child & Adolescent Psychology. 2006;35:2–12. doi: 10.1207/s15374424jccp3501_1. [DOI] [PubMed] [Google Scholar]
- 42.Roth D. Panzano PC. Crane-Ross D, et al. The Innovation Diffusion and Adoption Research Project (IDARP): oving from the diffusion of research results to promoting the adoption of evidence-based innovations in the Ohio mental health system. In: Roth D, editor. New research in mental health. Vol. 15. Columbus, OH: Ohio Department of Mental Health; 2002. pp. 149–56. [Google Scholar]
- 43.Schmidt F. Taylor TK. Putting empirically supported treatments into practice: lessons learned in a children's mental health center. Professional Psychology: Research & Practice. 2002;33:483–9. [Google Scholar]
- 44.Schoenwald SK. Sheidow AJ. Letourneau EJ, et al. Transportability of multisystemic therapy: evidence for multilevel influences. Mental Health Services Research. 2003;5:223–9. doi: 10.1023/a:1026229102151. [DOI] [PubMed] [Google Scholar]
