ABSTRACT
Combat exposure among military personnel results in increased risk of posttraumatic stress disorder (PTSD), major depression, substance use, and related health risks. PTSD symptoms require innovative approaches to promote effective coping postdeployment. PTSD's nature and scope requires an approach capable of integrating multiple health risks while reaching large populations. This article provides the rationale and approach to adapt and evaluate a Pro-Change computerized tailored intervention (CTI) targeted at behavioral sequelae (i.e., smoking, stress, and depression) for veterans with or at risk for PTSD. The three-phase approach includes: 1) focus groups to review and, subsequently, adapt content of the existing CTI programs; 2) usability testing; and 3) feasibility testing using a three-month pre–postdesign. Effective, theory-based, real-time, multiple behavior interventions targeting veterans' readiness to quit smoking, manage stress, and depression are warranted to provide potential health impact, opportunities for learning veteran-specific issues, and advance multiple health behavior change knowledge.
KEYWORDS: Computerized tailored interventions, Posttraumatic stress disorder, Transtheoretical model, Veterans
Combat-related posttraumatic stress disorder (PTSD) is a significant and long-lasting problem with up to 15% of veterans meeting current and 31% meeting lifetime PTSD diagnostic criteria (1). Examining the mental health effects in U.S. military personnel returning from current deployments to Iraq and Afghanistan is of increasing importance, particularly since research conducted following other military conflicts and Operation Enduring Freedom (OEF) or Operation Iraqi Freedom (OIF) has shown that deployment and exposure to combat result in increased risk of PTSD, major depression, substance abuse, functional impairment in social and employment settings, and the increased use of health care services (2–7).
The majority of OEF/OIF veterans have been involved in combat situations, and approximately 10–17% of veterans in combat infantry units have reported symptom levels consistent with a diagnosis of PTSD (8). Given that U.S. soldiers are currently deployed for 15-month rotations, exposure to potentially traumatic events is lengthy, and traumatic stress symptoms may be more likely (9). Shortly after redeployment, approximately 44% of service members reported clinically significant depressive and/or posttraumatic stress symptoms (9). Because PTSD symptoms seldom disappear completely, it is usually a continuing challenge for survivors of trauma to cope with PTSD symptoms and the problems they cause. Comorbid conditions, including depression, other anxiety disorders, and substance misuse, are common along with relationship difficulties, excessive anger, work problems, physical health, illness, and healthcare utilization (10, 11).
According to a 2007 report from the Defense Health Board Task Force on Mental Health, the military's mental health system does not have adequate resources, funding or personnel to support the psychological health of service members and their families (12). In response to anticipated need, the Veterans Affairs (VA) health care system has increased the number of psychologists since 2005 by 478, but at least 330 more are needed (13) indicating potential gaps in service provision. Such gaps in service come at a significant cost to the military. PTSD and depression in returning service members cost up to $6.2 billion in the 2 years following deployment (5). However, evidence-based treatment for PTSD and depression would pay for itself within 2 years; thus, there is a need to develop and evaluate effective interventions. The ultimate impact of successfully intervening on young veterans is that it has very real implications on recovery, relapse prevention, and quality of life. This will ultimately allow affected individuals to return to their task, occupations, and family life.
BEHAVIORAL HEALTH RISK FACTORS AND TREATMENT FOR PTSD
Research with returning OEF/OIF service members suggests that there is a new generation of veterans with high levels of PTSD and depression (2). Treatment options for PTSD include cognitive–behavioral therapy (CBT), group psychotherapy, and pharmacotherapy to ease depressive symptoms and promote sleep (13, 14); however, most efficacy trials using randomized controlled designs have focused on CBT (15). In general, CBT methods have been effective in producing significant reductions in PTSD symptoms in civilian populations, but the degree of remission has been somewhat less in veterans with chronic combat-related PTSD (16, 17).
It is, therefore, imperative to identify effective ways of increasing access to efficacious treatments for combat-related PTSD and associated comorbid behavioral health conditions. Moreover, given the rapid development of telemedicine programs within the military, it is vital that research address the effectiveness of this mode of service delivery for specialty services such as PTSD treatment. Research that investigates prospective, randomized evaluations of clinical and process outcomes for specialized PTSD interventions is imperative (18). This paper describes the development of a computerized, tailored intervention (CTI) that will target health behaviors associated with PTSD in veterans (specifically, smoking, depression and stress). Ultimately, the goal of the CTI program is to teach users healthy coping skills that will promote effective management of the psychological impact of traumatic events.
Smoking
PTSD is associated with a high prevalence of smoking, heavy cigarette consumption and low cessation rates (19), and for some, becomes a way of coping with chronic symptoms (20, 21). Smokers report a higher frequency of smoking in response to military memories (19, 22); however, stopping smoking is not associated with worsening PTSD or depression (23). Because PTSD is associated with elevated rates of nicotine use, it has an indirect impact on cardiovascular health (24). Further, compared to Vietnam veterans with PTSD who do not smoke, Vietnam veterans with PTSD who do smoke have reported higher levels of PTSD symptoms, trait anxiety, and depression (25).
Stress
The PTSD impact extends beyond trauma victims by disrupting their intimate relationships and families (26). In combating anger regulation problems, stress management interventions are critical to reduce the heightened physiological arousal, anxiety, depression, other comorbid problems, and maladaptive coping strategies accompanying PTSD (27, 28).
Depression
A recent study of combat troops following return from deployment to Afghanistan or Iraq found postwar rates of depression from 7.1% to 7.9% (29). More importantly, the majority of soldiers with PTSD or depression at 7 months did not meet criteria for either condition at 1 month (29). The RAND study (5) also found that symptoms of PTSD and depression can have a delayed onset—appearing months after exposure to stress.
COMPUTERIZED, TAILORED INTERVENTIONS (CTIs) FOR PTSD
PTSD interventions can vary considerably in content, timing, intensity, and delivery method; research on the efficacy of self-help protocols for behavior change has been promising. The proposed intervention will deliver empirically-based, tailored communications for smoking cessation, depression, and stress management on a personal computer. Compared to nontailored materials, tailored materials are better remembered, perceived as more relevant and credible, and are more effective in changing health behavior (30, 31). Advances in behavioral science, communications, and computer technology have contributed to the development of behavioral health interventions that effectively motivate behavior change (32, 33) with minimal or no clinician contact. Computer/Internet-based interventions yield equally effective treatment outcomes compared to self-help interventions delivered via other methods (34).
Computerized interventions have several potential advantages over noncomputerized protocols (32). First, computer-based interventions allow for personalization of recommendations including tailoring over time with minimal burden of superfluous material. Second, the application of precise user data (e.g., time burden on users, answers to knowledge questions) collected via interactive computerized interventions present a unique advantage relative to noncomputerized self-help methods such as bibliotherapy or videotape protocols. Third, where Internet is available, Internet-based interventions can reach a large population at a relatively low cost. Fourth, they can be accessed privately from individuals' homes and completed at each user's own pace. Finally, they can be easily adapted and updated to reflect emerging empirical findings to ensure the highest quality of care.
Web-based computer tailored interventions (CTIs) are particularly beneficial for intervening with some mental health issues because they offer anonymity (35, 36), reduce fear of stigma (35, 36), and increase self-disclosure (37, 38). CTIs can be more engaging, allowing participants to control their learning environment, move at their own pace, and allow access to sensitive information (39–41). They can also potentially increase retention rates by increasing convenience and allowing doses of interventions as needed (42). Additionally, advanced CTIs employ empirical databases consisting of data collected from thousands of participants and heuristics. These databases provide the basis for decision rules that guide the development of individualized interventions tailored according to behavior change theory variables (33, 43).
The Transtheoretical Model of Behavior Change (TTM), one of the leading behavior change theories (43), has been frequently employed as a framework for this type of tailoring. The TTM (44) is a comprehensive model of behavior change that integrates diverse psychological constructs (i.e., stage of change, decisional balance, process of change, and self-efficacy) to explain and predict how and when individuals change their health behaviors (for a full description of the TTM see Ref. (45)). Several clinical trials have documented the ability of TTM interventions to recruit, retain, and effect change across a number of health behaviors including smoking (46, 47), stress (48), depression prevention (49), and multiple health behaviors (50–52), and has also shown impacts on readiness to change, perceived treatment relevance, attendance at group treatment sessions, and attrition in Vietnam veterans with PTSD (53).
TTM-BASED TAILORED APPROACHES FOR TREATMENT-RESISTANT POPULATIONS
Resistance to treatment can be expressed in several major ways, e.g., not seeking treatment or dropping out of treatment. TTM has been found to predict and reduce both of these treatment-resistant behaviors (49, 54). In the active duty military, there are unique factors that contribute to resistance to seeking mental health care, particularly the concern about how a soldier will be perceived by peers and by leadership (55). Concern about stigma is disproportionately greatest among those most in need of help from mental health services (55). Soldiers often report more discomfort in discussing potential psychological problems than medical problems, especially when they are returning to their units. In addition, soldiers report a lesser likelihood of following through with a psychological referral than a medical referral (7). Moreover, war fighters may have legitimate incentives to minimize their distress such as hastening discharge, to accelerate return to their family, or to avoid compromising their military career or retirement (56).
Clinicians also acknowledge that thousands of OEF/OIF veterans are reluctant to seek help even those experiencing distressing psychiatric symptoms (17). For example, of those soldiers and Marines returning from Iraq who reported experiencing a mental health problem, only 38 to 45% indicated an interest in receiving help, and only 23 to 40% reported actually receiving professional help (7).
Premature termination or dropout from treatments for PTSD is typically in the 50% range. This occurrence is common across treatments for a broad range of mental health problems (57). In their metaanalysis of 125 studies on dropout, Wierzbicki and Pekarik (57) found poor ability to predict dropout with the only variables being minority status, lower education and addiction behaviors. In contrast, Brogan et al. (54) found that TTM variables were able to predict over 90% of premature terminators from therapy for a broad range of mental health problems with premature terminators being similar to people in precontemplation.
RATIONALE FOR A MULTIPLE BEHAVIOR APPROACH
Conventional wisdom on disease management has been that it is not possible to treat multiple behaviors simultaneously because it places too many demands on a person's inherent ability to change (58). In fact, one of the limitations of much of the published research is that it has been based primarily on an action paradigm limiting application to the majority of individuals. Using a TTM-based approach allows multiple behaviors to be addressed without overburdening participants. The TTM posits reduced resistance and greater behavior change occurence when interventions are tailored to the individual's stage of change, rather than “one size fits all.” The TTM provides a framework for intervening when individuals are not ready or ambivalent to change unhealthy behaviors or adhere to traditional treatments such as CBT used in the treatment of PTSD. Multiple behavior change interventions based on TTM for a common health objective, e.g., cancer prevention, diabetes self-management, and weight management, have been shown to have significant impact on entire populations (50, 51, 59–63). For example, smokers treated for two or three behaviors were as effective in being abstinent at long-term follow-up as those treated for only smoking (52).
TTM-based CTI's tend to generate much higher rates of participation (e.g., 65% to 85%) for problems like smoking, stress, and obesity than the 2% to 20% rates commonly found with action-oriented clinic-based treatments (46, 48, 60, 64). Further, participants who are traditionally at the greatest risk for dropping out in the precontemplation stage completed CTIs at the same high rate as those ready to take action (65). This project builds on this recent evidence that treating multiple behaviors with a TTM-based CTI is effective with each of the target behaviors without reducing the efficacy of treating one behavior at a time. It also builds on the covariation/coaction concepts that individuals taking effective action on one target behavior are much more likely to take effective action on a second behavior and that individuals are likely to take effective action on untreated behaviors that are related to the treated behaviors.
THE STR2IVE PROJECT (STRESS REDUCTION STRATEGIES TO IMPROVE VETERAN'S HEALTH)
Currently, there are no multiple behavior CTI programs for veterans. Utilizing a CTI approach can produce healthier coping strategies to reduce stress, depression, and smoking. It is anticipated that this effort will lead to a fully integrated, scalable, multibehavioral system that can be easily disseminated online to serve veterans and nonveterans with a number of negative health risks. The STR2IVE program has been fully conceptualized and is now being developed and tested. This paper addresses the concept and approach; thus, the purpose and methodology is presented below. This allows the detailing of rationale and methodology that are not otherwise possible in typical data-based publications.
The primary aims of STR2IVE are:
To adapt and test the feasibility of a multiple behavior TTM-based CTI designed for the general adult population to be appropriate for veterans with or at-risk for PTSD.
To demonstrate preliminary behavior change in each of the three behaviors targeted by the CTI—smoking, depression, and stress—as well as reductions in PTSD-related symptoms and improved quality of life.
Research will be completed in three phases. Phase 1 focus groups obtain feedback from combat veterans on three TTM-based CTI programs previously developed and validated by Pro-Change Behavior Systems, a research-based behavior change product development company. The focus group data will be used to guide veteran-specific adaptation or revisions. Phase 2 usability testing uses Morae® software, the Think Aloud Protocol (66, 67) and the Wizard of OZ method (68, 69). Phase 3 includes pre–post feasibility testing of the adapted system.
Participants
A total of 95 male and female veterans aged 18 or older, preferably with former military service in Iraq or Afghanistan, are being recruited to participate in three phases of this project: 1) focus groups (n = 30); 2) usability study (n = 15); and 3) a feasibility study (n = 50). Participants are being recruited from the veteran community residing in Hawai’i through posters and flyers at VA clinics, referrals from VA mental health providers, and targeted mailings to VA patients at risk (exhibiting associated psychological symptoms) for PTSD.
Inclusion criteria for all phases are: veterans aged 18 years or older, OEF/OIF service preferred, computer literacy at the beginner level, eighth grade English literacy level, and access to a computer with Internet connectivity. Exclusion criteria for all phases are: history of mania, schizophrenia, or other psychoses; special medical conditions that may prevent engagement with the CTI system such as history of significant head injury; and suicidal ideation.
Description of the CTI system
Initially, the participants are prompted to access any or all behavior modules within 7 days and as often as desired thereafter. Subsequently, participants will be required to access the system a minimum of once a month over a three-month period. Users can update their assessments every 30 days with access to their most recent report and the interactive e-Workbook during the interim. Participants who fail to return as often as recommended will receive proactive email prompts as reminders to revisit the program.
The entry point into the CTI system assesses readiness to change the key health behaviors addressed. The participant's homepage provides information and access to the three individualized modules that are available to the individual based on risk (being in a preaction stage). During each session for a behavior module, individuals are assessed on all relevant TTM variables (i.e., stage of change, decisional balance, processes and self-efficacy) in addition to relevant constructs for a particular behavior and receive feedback based on the assessment. The Stress Management module, for example, includes all appropriate constructs of the TTM as well as tailored feedback on positive coping strategies and behaviors. In the baseline session for each behavior (e.g., smoking), the CTI system compares a participant's responses to a large comparative sample of other individuals in that stage (normative comparisons) and provides individualized, real-time onscreen feedback on which principles of change they are underutilizing, overutilizing or appropriately utilizing to facilitate forward stage movement.
All subsequent follow-up sessions are based on normative comparisons and ipsative comparisons. Ipsative feedback, which involves access to a database of results of previous contacts, reinforces progress individuals have made since their last assessment. The CTI system for one behavior can generate over 150 unique feedback sessions at baseline and more than 20,000 unique sessions at follow-up; this ensures uniqueness of feedback each time they interact with the system. The CTI sessions employ statistical decision-making to guide individuals through the intricacies of each stage, encouraging the use of the most appropriate change processes. Without this kind of expert and individualized feedback, participants cannot assess how much progress they are making, what processes and principles they are applying most effectively and which ones they need to emphasize the most in order to change successfully. This level of feedback is particularly helpful to those who may not yet have progressed to the next stage, despite substantive gains in appropriate use of change processes. The personal feedback report provided at the end of the session typically is two to three full color pages. From the report, the user can link to an interactive e-Workbook, in addition to the other behavior modules. The e-Workbook contains interactive exercises that are designed to engage the participant in using one or more processes or principles of the TTM that are most appropriate for that stage. The e-Workbook also includes links to external resources, assessments, tracking tools (e.g., logs and diaries to record temptations to engage in unhealthy stress management behavior), testimonials (e.g., how people develop new habits), and activities (e.g., rate the benefits; calculate the cost of unhealthy stress management behavior, True/False quiz). Users may choose to go through a particular stage or the entire e-Workbook. Participants have unlimited access, which can be used to progress between expert system sessions.
Procedures
All procedures have been approved by the VA Pacific Islands Health Care System (VAPIHCS), the U.S. Army Medical Research and Materiel Command's Human Research Protection Office (HRPO), and participating IRBs.
Phase I: focus groups
Three focus groups will gather information on the acceptability of existing CTI program content for each behavior. Each group reviews a behavior change module in order to match the expectations and needs of the target population. Each focus group (8–10 veterans; about 1.5 h) will be audio recorded with a note-taker present. Prior to the focus group meetings, participants provide informed consent and complete stage of change and basic demographics measures.
CTI system content for the smoking cessation, stress management, and depression prevention behavioral modules are reviewed, adapted, and adjusted based on the theme-based focus group analyses. Participants will be asked to provide feedback about the graphics, text and layout based on screen shots of the online system. This will include probes about veteran-specific issues. The major focus of the revisions will address adaptation of language, tone, and content to be appropriate and relevant the veteran population.
Phase II: usability study
After the CTI program has been adapted and beta-tested, its acceptability and usability (70) will be examined as part of the system development process. Efforts will be made to include veteran participants with varying levels of experience using interactive web-based multimedia programs (70) and at various stages of change for the different behaviors. Usability testing provides a scientific assessment of user errors, misunderstandings of content, navigation problems, and subjective satisfaction. This feedback is invaluable to the system design process and will improve the acceptability and usability of the final system.
During the usability testing, a research assistant observes a participant as he or she navigates through the CTI system, resisting the temptation to offer help too soon so that usability issues will be revealed. Participants are asked to think and make comments aloud as they work through the various screens in the registration process, the introduction to the program, the assessment questions, the feedback messages as well as the integration of the different behavior modules. Their audio and video of screens visited and mouse movements are captured and recorded automatically for review and analysis. Participants are asked to interact with relevant sections of the integrated program including the e-Workbook and asked to comment on the options and content available. Participants are asked to provide qualitative and quantitative feedback on overall presentation and usability as well as quality of the program, navigation, ease of use, attractiveness, etc. Feedback from individual interviews is then used to modify the behavior modules before additional usability interviews are conducted. The comments from all interviews are used to modify the CTI system prototype prior to the feasibility study. At the conclusion of each usability interview, participants are asked to provide feedback on the module they interacted with using a measure adapted from existing acceptability measures (70–72) that has been used by Pro-Change in previous research.
Phase III: feasibility
A three-month feasibility test is designed to assess acceptability and viability of the CTI system and the behavioral modules by veterans in the community, particularly recent veterans returning from the Iraq or Afghanistan combat theaters. Participants will provide informed consent and complete baseline, one- and three-month online questionnaires (see entire content of the questionnaires in “Outcome measures and assessment instruments” below).
The project has been approved for online screening and informed consent, allowing the registration and enrollment process to be automated for the feasibility testing. Veterans interested in participating in the feasibility study and meeting the basic criteria listed in the recruitment material can visit the project webpage to complete the screening and consent process. After confirming that they are military veterans over the age of 18 and comfortable using a computer and the Internet, prospective participants must meet inclusionary criteria by confirming that they have had a military experience so traumatic that in the past month they have had nightmares, been unable to stop thinking about it, were constantly on guard, and/or felt numb or detached because of it. Following this, they are asked to self-report whether they have any of the exclusionary criteria. If they pass the online prescreening questions, they are asked to view and print the informed consent fact sheet, which lists information about the study and their rights and responsibilities as participants. Next, they complete the PCL-M and PHQ-8 questionnaires and are enrolled in the study if the scores fall within the allowable range for the study (exhibit mild to moderate PTSD >24 and <74 on the PCL-M and are not severely depressed <20 on the PHQ-8) and they choose to enroll.
Participants will be provided access for 3 months to the Internet-based CTI system addressing smoking cessation, effective stress management, and depression prevention. Participants who are nonsmokers will not be given access to the smoking cessation module. Participants are asked to participate in a minimum of three sessions for each module (at least once per month) and for at least two of the three modules. Enrollment includes the consenting process and collecting demographics and an email address. The email address will then be used to provide reminders to participants who have not accessed the system in 30 days.
Outcome measures and assessment instruments
Demographics questionnaire
Demographic data including race/ethnicity, age, gender, combat theater(s) served in, and total number of months in theater(s) will be collected.
PTSD Symptom Checklist (PCL-M) (73)
The PCL-M consists of 17 questions that map directly onto DSM-IV criteria for PTSD. Respondents are asked how often they have been bothered by each symptom in the past month on a five-point Likert scale (1 = “not at all” to 5 = “extremely”). All items are summed to obtain a total severity score. A score of 44 is considered PTSD-positive for the general population, while a score of 50 is considered PTSD-positive in military populations.
Combat Trauma Exposure Survey (CTES) (5)
The CTES is an 11-item, self-report survey that assesses the type of an individual's combat trauma experiences. It includes both direct (e.g., injury requiring hospitalization) and indirect trauma exposure (e.g., witnessing a traumatic event) adapted from Ref. (55) that requires only a yes or no response (5). The subset of 11 exposures used in the brief survey was found to be as predictive of PTSD as the full 24 items in veterans residing in NY (74).
The Perceived Stress Scale (PSS) (75)
The PSS is a 10-item questionnaire assessing the degree to which situations in one's life are appraised as stressful. Items are designed to tap how unpredictable, uncontrollable, and overloaded respondents find their lives to be.
Patient Health Questionnaire (PHQ-8) (76, 77)
The PHQ-8 is an eight-item version lacking the ninth question regarding suicidal ideation of the PHQ-9, a tool for assisting clinicians in diagnosing depression as well as selecting and monitoring treatment. The PHQ-8 is selected for this study as responses are collected independently online and there is no personal interaction with the participants so any response by clinicians would be delayed. Also, the suicidal ideation question is rarely endorsed and most often reflects passive rather than active thoughts of suicide (78). The PHQ-8 is based directly on the diagnostic criteria for major depressive disorder in the DSM-IV and includes items such as “Little interest or pleasure in doing things” and “Feeling tired or having little energy.” Respondents rate how often they have been “bothered by problems over the past 2 weeks” using a four-point Likert scale (0 = not at all, 3 = nearly every day). Scores in the range of 5–9 indicate minimal symptoms, 10–14 minor depression or dysthymia, 15–19 major depression (moderate), and greater than 20 severe major depression.
Quality of Life Scale ([QOLS]) (79, 80): (feasibility study)
The QOLS contains 16 items that represent five conceptual domains of quality of life. QOLS was developed with more consideration to cultural diversity and individual perspectives than other commonly used measures. It uses a unique seven-item Likert scale that allows responses regarding different aspects of life to range from “delightful” to “terrible”. The original 15-item QOLS satisfaction scale was found to be internally consistent with alpha from 0.82 to 0.92 and showed high test–retest reliability over 3 weeks (r = 0.78 to r = 0.84). Similar reliability was reported for the 16-item version used in this study (81).
Stage of Change for Depression (49)
This measure assesses readiness to engage in effective methods for preventing depression. The assessment includes a short description of depression prevention—using effective methods to keep depression from occurring, or if it does occur, to keep it as mild and brief as possible. Respondents are asked, “Do you effectively practice depression prevention in your daily life?” A single item response category places individuals in one of the five stages of change.
Stage of Change for Stress Management (48)
This measure assesses readiness to effectively manage their daily stress. The assessment includes a short description of stress management (stress management includes regular relaxation, physical activity, talking with others, and/or making time for social activities) and asks respondents, “Do you effectively practice stress management in your daily life?” A single item response category places individuals in one of the five stages of change.
Stage of Change for Smoking Cessation (82)
This measure assesses a readiness to quit smoking. Participants are asked if they have quit smoking. A single item response category places individuals in one of the five stages of change.
Data analysis
Primary aim 1
Primary aim 1 is to adapt and test the feasibility of a multiple behavior TTM-based CTI designed for the general adult population to be appropriate for veterans with or at-risk for PTSD. In order to determine successful outcomes for Aim 1, data analysis will be conducted from qualitative data obtained from the smoking, depression, and stress focus groups. Data from the focus groups will be coded and analyzed according to guidelines (83, 84). Analysis will include systematic steps for identifying basic concepts and comparing results with other groups in order to find common patterns (84).
Primary aim 2
Primary aim 2 is to demonstrate preliminary behavior change in each of the three behaviors targeted by the CTI—smoking, depression, and stress—as well as reductions in PTSD-related symptoms and improved quality of life. Data analysis for Aim 2 will be focused on the self-report measures described above to provide estimates of effect size for the intervention on PTSD symptoms (PCL-M), perceived stress (PSS), depression (PHQ-8), quality of life (QOLS), status on TTM variables including stage of change, and ratings of acceptability and satisfaction with intervention (ESIM, SUS). Given that this is a feasibility study, the analyses are not adequately powered to detect statistically significant differences across all three time points; therefore, differences in effect sizes (i.e., Cohen's d and odds ratios) and visual trends will be examined. More specifically, repeated measures Analysis of Variance (ANOVA) and logistic regression will be used to examine differences in continuous (e.g., decisional balance, confidence, number of cigarettes, and satisfaction) and categorical measures (e.g., progressing to Action or Maintenance and willing to recommend program to friends). Frequency of use and acceptability ratings will also be compared across the stages of change. Finally, how utilization relates to stage progress and changes in other TTM constructs will be examined. Measures such as number of log ins and interactions and satisfaction ratings will be compared using repeated measures ANOVAs.
Assessment of feasibility
The following criteria will be used to determine feasibility: a) completing the customization and testing of the baseline CTI; b) recruitment and delivery of the baseline intervention to 40–50 veterans; and c) determination of the acceptability of the intervention, represented by an overall rating of 4 (good) or better by 75% of the participants. The feasibility study analysis will include the completion of the beta tests, the delivery of the complete intervention to 40–50 participants, and the determination of the acceptability of the intervention represented by an overall rating of 4 (good) or better on the ESIM and SUS by 75% of the participants.
DISCUSSION AND CONSIDERATIONS
The largest integrated health care system in the United States is the network of facilities operated for veterans of military service by the Department of Veterans Affairs (VA). The proposed research is particularly relevant in lieu of the current and anticipated demands on the VA mental health system with the return of OEF/OIF. The anticipated impact of intervening on veterans with PTSD with our intervention is that it may have very real implications on recovery, relapse prevention, and quality of life. This project may also have direct and indirect impacts on patient care such as: 1) providing empirically based behavioral interventions, as additional resources, for health care providers who have increasingly limited time and resources; 2) providing support and intervention for individuals who have PTSD but are not yet ready to address these health risk behaviors by progressing them towards becoming ready; 3) providing support and relapse prevention tools for individuals who are successfully coping with PTSD, but may be at risk for relapse; and 4) improving the ability to reach individuals with PTSD at “teachable moments” through the Internet or disseminated technologies (e.g., computers, smart phones, and cell phones). In other words, individuals will have access to the intervention when they are ready to receive the message.
The proposed CTI system is also flexible enough that additional modules targeted at other health risk behaviors or coping strategies can be added. These modules may include anger management, sleep disorders, pain management, domestic violence, war memories, and social support, among others. Ideally, access to the system should be expanded through other VA healthcare systems in the U.S. and Guam and veterans who are at risk for chronic diseases caused by smoking, stress, and depression should be included.
Acknowledgements
This research was supported by the U.S. Department of the Army (Award No. W81XWH-09-2-0106.) The U.S. Army Medical Research Acquisition Activity, Fort Detrick, MD is the awarding and administering acquisition office. The content of this article does not necessarily reflect the position or the policy of the Government, and no official endorsement should be inferred. Editorial support was in part provided by University of Hawai'i at Mānoa School of Nursing and Dental Hygiene Center for ‘Ohana and Self-Management of Chronic Illnesses in Hawai‘i (5P20NR010671). We acknowledge the support of Sarah D. Miyahira, Ph.D., James L. Spira, Ph.D., Julia Whealin, Ph.D., Viil Lid, M.S., Julie Padula, Simay Gokbayrak, Michelle Kawasaki, M.A., Stacy Daly, M.A. and Mr. Clyde Hladky on the project.
Footnotes
Implications
For researchers: A process including qualitative content evaluation, usability and feasibility testing is recommended to adapt existing effective computer tailored interventions addressing multiple PTSD risk factors for veteran populations.
For practitioners: Although there may be common underlying principles addressing multiple behavior change targeting PTSD risk factors such as smoking, depression and stress, it is important to identify population-specific issues prior to intervening to maximize acceptability and effectiveness.
For policy-makers: Evidence of effectiveness in veterans should be required prior to implementing and disseminating PTSD prevention programs for this population.
Contributor Information
Patricia J. Jordan, Phone: +1-808-433-7346, Email: patricia.jordan@pacifichui.org.
Kerry E. Evers, Email: Kevers@prochange.com.
Katherine Y. M. Burke, Phone: +1-808956-2752, Email: kymburke@hawaii.edu.
Laurel A. King, Phone: +1-808433-0131, Email: laurel.king@va.gov.
Claudio R. Nigg, Phone: +1-808956-2862, Email: cnigg@hawaii.edu.
References
- 1.Cook JM, Elhai JD, Cassidy EL, et al. Assessment of trauma exposure and post-traumatic stress in long-term care veterans—Preliminary data on psychometrics and post-traumatic stress disorder prevalence. Mil Med. 2005;170(10):5. doi: 10.7205/milmed.170.10.862. [DOI] [PubMed] [Google Scholar]
- 2.Hoge CW, Auchterlonie JL, Milliken CS. Mental health problems, use of mental health services, and attrition from military service after returning from deployment to Iraq or Afghanistan. J Am Med Assoc. 2006;295(9):1023–1032. doi: 10.1001/jama.295.9.1023. [DOI] [PubMed] [Google Scholar]
- 3.Hoge CW, Terhakopian A, Castro CA, Messer SC, Engel CC. Association of posttraumatic stress disorder with somatic symptoms, health care visits, and absenteeism among Iraq war veterans. Am J Psychiatry. 2007;164(1):150–153. doi: 10.1176/appi.ajp.164.1.150. [DOI] [PubMed] [Google Scholar]
- 4.Seal KH, Bertenthal D, Miner CR, Sen S, Marmar C. Bringing the war back home: Mental health disorders among 103,788 US veterans returning from Iraq and Afghanistan seen at Department of Veterans Affairs facilities. Arch Intern Med. 2007;167(5):476–482. doi: 10.1001/archinte.167.5.476. [DOI] [PubMed] [Google Scholar]
- 5.Tanielian TL, Corporation R, eds. Invisible wounds of war: Psychological and cognitive injuries, their consequences, and services to assist recovery. Rand Corporation; 2008:453.
- 6.Davidson JRT. Recognition and treatment of posttraumatic stress disorder. J Am Med Assoc. 2001;286(5):584–588. doi: 10.1001/jama.286.5.584. [DOI] [PubMed] [Google Scholar]
- 7.Greene-Shortridge TM, Britt TW, Castro CA. The stigma of mental health problems in the military. Mil Med. 2007;172(2):157–161. doi: 10.7205/milmed.172.2.157. [DOI] [PubMed] [Google Scholar]
- 8.Gellis LA, Gehrman PR, Mavandadi S, Oslin DW. Predictors of sleep disturbances in Operation Iraqi Freedom/Operation Enduring Freedom veterans reporting a trauma. Mil Med. 2010;175(8):567–573. doi: 10.7205/milmed-d-09-00123. [DOI] [PubMed] [Google Scholar]
- 9.Lapierre CB, Schwegler AF, Labauve BJ. Posttraumatic stress and depression symptoms in soldiers returning from combat operations in Iraq and Afghanistan. J Trauma Stress. 2007;20(6):933–943. doi: 10.1002/jts.20278. [DOI] [PubMed] [Google Scholar]
- 10.Forbes D, Creamer MC, Phelps AJ, et al. Treating adults with acute stress disorder and post-traumatic stress disorder in general practice: A clinical update. Med J Aust. 2007;187(2):120–123. doi: 10.5694/j.1326-5377.2007.tb01158.x. [DOI] [PubMed] [Google Scholar]
- 11.Jankowski K. PTSD and physical health. National Center for PTSD Fact Sheet. 2007. Available at: http://www.ncptsd.va.gov/ncmain/ncdocs/fact_shts/fs_physical_health.html.
- 12.Department of Defense Task Force on Mental Health. An achievable vision: Report of the Department of Defense Task Force on Mental Health. Falls Church, VA; 2007:1-100.
- 13.Munsey C. Transforming military mental health. Monit Psychol. 2007;38(8):38–55. [Google Scholar]
- 14.Feldner MT, Monson CM, Friedman MJ. A critical analysis of approaches to targeted PTSD prevention: Current status and theoretically derived future directions. Behav Modif. 2007;31(1):80–116. doi: 10.1177/0145445506295057. [DOI] [PubMed] [Google Scholar]
- 15.Foa EB, Meadows EA. Psychosocial treatments for posttraumatic stress disorder: A critical review. Annu Rev Psychol. 1997;48:449–480. doi: 10.1146/annurev.psych.48.1.449. [DOI] [PubMed] [Google Scholar]
- 16.National Center for PTSD. Empirical evidence regarding behavioral treatments for PTSD (Fact Sheet). 2007.
- 17.Erbes C, Westermeyer J, Engdahl B, Johnsen E. Post-traumatic stress disorder and service utilization in a sample of service members from Iraq and Afghanistan. Mil Med. 2007;172(4):5. doi: 10.7205/milmed.172.4.359. [DOI] [PubMed] [Google Scholar]
- 18.Frueh BC. Procedural and methodological issues in telepsychiatry research and program development. Psychiatr Serv. 2000;51(12):1522–1527. doi: 10.1176/appi.ps.51.12.1522. [DOI] [PubMed] [Google Scholar]
- 19.Beckham JC, Kirby AC, Feldman ME, et al. Prevalence and correlates of heavy smoking in Vietnam veterans with chronic posttraumatic stress disorder. Addict Behav. 1997;22(5):637–647. doi: 10.1016/S0306-4603(96)00071-8. [DOI] [PubMed] [Google Scholar]
- 20.Zweben JE, Clark HW, Smith DE. Traumatic experiences and substance abuse: Mapping the territory. J Psychoactive Drugs. 1994;26(4):327–344. doi: 10.1080/02791072.1994.10472453. [DOI] [PubMed] [Google Scholar]
- 21.Green BL, Lindy JD, Grace MC, Leonard AC. Chronic posttraumatic stress disorder and diagnostic comorbidity in a disaster sample. J Nerv Ment Dis. 1992;180(12):760–766. doi: 10.1097/00005053-199212000-00004. [DOI] [PubMed] [Google Scholar]
- 22.Beckham JC, Moore SD, Feldman ME, et al. Health status, somatization, and severity of posttraumatic stress disorder in Vietnam combat veterans with posttraumatic stress disorder. Am J Psychiatry. 1998;155(11):1565–1569. doi: 10.1176/ajp.155.11.1565. [DOI] [PubMed] [Google Scholar]
- 23.McFall M, Atkins DC, Yoshimoto D, et al. Integrating tobacco cessation treatment into mental health care for patients with posttraumatic stress disorder. Am J Addict. 2006;15(5):336–344. doi: 10.1080/10550490600859892. [DOI] [PubMed] [Google Scholar]
- 24.Buckley TC, Mozley SL, Bedard MA, Dewulf A-C, Greif J. Preventive health behaviors, health-risk behaviors, physical morbidity, and health-related role functioning impairment in veterans with post-traumatic stress disorder. Mil Med. 2004;169(7):536–540. doi: 10.7205/milmed.169.7.536. [DOI] [PubMed] [Google Scholar]
- 25.Beckham JC, Roodman AA, Shipley RH, et al. Smoking in Vietnam combat veterans with post-traumatic stress disorder. J Trauma Stress. 1995;8(3):461–472. doi: 10.1002/jts.2490080308. [DOI] [PubMed] [Google Scholar]
- 26.Riggs DS, Byrne CA, Weathers FW, Litz BT. The quality of the intimate relationships of male Vietnam veterans: Problems associated with posttraumatic stress disorder. J Trauma Stress. 1998;11(1):87–101. doi: 10.1023/A:1024409200155. [DOI] [PubMed] [Google Scholar]
- 27.Taft C, Niles B. Assessment and treatment of anger in combat-related PTSD. In: National Center for PTSD and Walter Reed Army Medical Center, editor. Iraq war clinician guide. 2. White River Junction, VT: Department of Veterans Affairs/National Center for PTSD; 2004. [Google Scholar]
- 28.Sinha R. How does stress increase risk of drug abuse and relapse? Psychopharmacology. 2001;158(4):343–359. doi: 10.1007/s002130100917. [DOI] [PubMed] [Google Scholar]
- 29.Grieger TA, Cozza SJ, Ursano RJ, et al. Posttraumatic stress disorder and depression in battle-injured soldiers. Am J Psychiatry. 2006;163(10):1777–1783. doi: 10.1176/appi.ajp.163.10.1777. [DOI] [PubMed] [Google Scholar]
- 30.Kreuter MW, Strecher VJ, Glassman B. One size does not fit all: The case for tailoring print materials. Ann Behav Med. 1999;21(4):276–283. doi: 10.1007/BF02895958. [DOI] [PubMed] [Google Scholar]
- 31.Skinner CS, Campbell MK, Rimer BK, Curry S, Prochaska JO. How effective is tailored print communication? Ann Behav Med. 1999;21(4):290–298. doi: 10.1007/BF02895960. [DOI] [PubMed] [Google Scholar]
- 32.Ruggiero KJ, Resnick HS, Acierno R, et al. Internet-based intervention for mental health and substance use problems in disaster-affected populations: A pilot feasibility study. Behav Ther. 2006;37(2):190–205. doi: 10.1016/j.beth.2005.12.001. [DOI] [PubMed] [Google Scholar]
- 33.Velicer WF, Prochaska JO, Redding CA. Tailored communications for smoking cessation: Past successes and future directions. Drug Alcohol Rev. 2006;25(1):49–57. doi: 10.1080/09595230500459511. [DOI] [PubMed] [Google Scholar]
- 34.Hirai M, Clum GA. A meta-analytic study of self-help interventions for anxiety problems. Behav Ther. 2006;37(2):99–111. doi: 10.1016/j.beth.2005.05.002. [DOI] [PubMed] [Google Scholar]
- 35.Marlatt GA, Witkiewitz K. Harm reduction approaches to alcohol use: Health promotion, prevention, and treatment. Addict Behav. 27(6):867-886. [DOI] [PubMed]
- 36.Bewick BM, Trusler K, Barkham M, et al. The effectiveness of web-based interventions designed to decrease alcohol consumption—A systematic review. Prev Med. 2008;47(1):17–26. doi: 10.1016/j.ypmed.2008.01.005. [DOI] [PubMed] [Google Scholar]
- 37.Parks KA, Pardi AM, Bradizza CM. Collecting data on alcohol use and alcohol-related victimization: A comparison of telephone and web-based survey methods. J Stud Alcohol. 2006;67(2):318–323. doi: 10.15288/jsa.2006.67.318. [DOI] [PubMed] [Google Scholar]
- 38.Sobell LC, Cunningham JA, Sobell MB. Recovery from alcohol problems with and without treatment: Prevalence in two population surveys. Am J Public Health. 1996;86(7):966–972. doi: 10.2105/AJPH.86.7.966. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Fotheringham MJ, Owies D, Leslie E, Owen N. Interactive health communication in preventive medicine: Internet-based strategies in teaching and research. Am J Prev Med. 2000;19(2):113–120. doi: 10.1016/S0749-3797(00)00188-4. [DOI] [PubMed] [Google Scholar]
- 40.Atkinson NL, Gold RS. The promise and challenge of eHealth interventions. Am J Heal Behav. 26(6):494-503. [DOI] [PubMed]
- 41.Wantland DJ, Portillo CJ, Holzemer WL, Slaughter R, McGhee EM. The effectiveness of web-based vs. non-web-based interventions: A meta-analysis of behavioral change outcomes. J Med Internet Res. 2004;6(4):e40. doi: 10.2196/jmir.6.4.e40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Marcus BH, Nigg CR, Riebe D, Forsyth LH. Interactive communication strategies: Implications for population-based physical-activity promotion. Am J Prev Med. 2000;19(2):121–126. doi: 10.1016/S0749-3797(00)00186-0. [DOI] [PubMed] [Google Scholar]
- 43.Glanz K, Rimer B, Lewis F. Theory, research, and practice on health behavior and health education. In: Glanz K, Rimer B, Lewis F, editors. Health behavior and health education: Theory, research and practice. San Francisco, CA: Wiley; 2002. pp. 22–44. [Google Scholar]
- 44.Prochaska JO, DiClemente CC. Stages and processes of self-change of smoking: Toward an integrative model of change. J Consult Clin Psychol. 1983;51(3):390–395. doi: 10.1037/0022-006X.51.3.390. [DOI] [PubMed] [Google Scholar]
- 45.Prochaska JO, Velicer WF. The transtheoretical model of health behavior change. Am J Health Promot. 1997;12(1):38–48. doi: 10.4278/0890-1171-12.1.38. [DOI] [PubMed] [Google Scholar]
- 46.Prochaska JO, Velicer WF, Fava JL, et al. Counselor and stimulus control enhancements of a stage-matched expert system intervention for smokers in a managed care setting. Prev Med. 2001;32(1):23–32. doi: 10.1006/pmed.2000.0767. [DOI] [PubMed] [Google Scholar]
- 47.Prochaska JO, DiClemente CC, Velicer WF, Rossi JS. Standardized, individualized, interactive, and personalized self-help programs for smoking cessation. Health Psychol. 1993;12(5):399–405. doi: 10.1037/0278-6133.12.5.399. [DOI] [PubMed] [Google Scholar]
- 48.Evers KE, Prochaska JO, Johnson JL, et al. A randomized clinical trial of a population- and transtheoretical model-based stress-management intervention. Health Psychol. 2006;25(4):521–529. doi: 10.1037/0278-6133.25.4.521. [DOI] [PubMed] [Google Scholar]
- 49.Levesque D, Marter D Van, Schneider R, et al. (in press) Randomized trial of a transtheoretical model intervention for depression in primary care. Am J Heal Promot.
- 50.Prochaska JO, Velicer WF, Rossi JS, et al. Multiple risk expert systems interventions: Impact of simultaneous stage-matched expert system interventions for smoking, high-fat diet, and sun exposure in a population of parents. Health Psychol. 2004;23(5):503–516. doi: 10.1037/0278-6133.23.5.503. [DOI] [PubMed] [Google Scholar]
- 51.Prochaska JO, Velicer WF, Redding C, et al. Stage-based expert systems to guide a population of primary care patients to quit smoking, eat healthier, prevent skin cancer, and receive regular mammograms. Prev Med. 2005;41(2):406–416. doi: 10.1016/j.ypmed.2004.09.050. [DOI] [PubMed] [Google Scholar]
- 52.Prochaska JJ, Velicer WF, Prochaska JO, Delucchi K, Hall SM. Comparing intervention outcomes in smokers treated for single versus multiple behavioral risks. Health Psychol. 2006;25(3):380–388. doi: 10.1037/0278-6133.25.3.380. [DOI] [PubMed] [Google Scholar]
- 53.Murphy RT, Thompson KE, Murray M, Rainey Q, Uddo MM. Effect of a motivation enhancement intervention on veterans' engagement in PTSD treatment. Psychol Serv. 2009;6(4):264–278. doi: 10.1037/a0017577. [DOI] [Google Scholar]
- 54.Brogan MM, Prochaska JO, Prochaska JM. Predicting termination and continuation status in psychotherapy using the transtheoretical model. Psychother Theory Res Pract Train. 1999;36(2):105–113. doi: 10.1037/h0087773. [DOI] [Google Scholar]
- 55.Hoge CW, Castro CA, Messer SC, et al. Combat duty in Iraq and Afghanistan, mental health problems, and barriers to care. N Engl J Med. 2004;351(1):13–22. doi: 10.1056/NEJMoa040603. [DOI] [PubMed] [Google Scholar]
- 56.Ruzek J, Curran E, Friedman M, et al. Treatment of the returning Iraq war veteran (fact sheet). 2007.
- 57.Wierzbicki M, Pekarik G. A meta-analysis of psychotherapy dropout. Prof Psychol Res Pract. 1993;24(2):190–195. doi: 10.1037/0735-7028.24.2.190. [DOI] [Google Scholar]
- 58.Patterson R. Changing patient behavior: Improving outcomes in health and disease management. Jossey-Bass; 2001:238.
- 59.Prochaska JO, Velicer WF, Prochaska JM, Johnson JL. Size, consistency, and stability of stage effects for smoking cessation. Addict Behav. 2004;29(1):207–213. doi: 10.1016/S0306-4603(03)00086-8. [DOI] [PubMed] [Google Scholar]
- 60.Johnson SS, Paiva AL, Cummins CO, et al. Transtheoretical model-based multiple behavior intervention for weight management: Effectiveness on a population basis. Prev Med. 2008;46(3):238–246. doi: 10.1016/j.ypmed.2007.09.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Jones H, Edwards L, Greene G, et al. The efficacy of stages of change interventions in diabetes self-care and control. In: 61st Annual Conference of the American Diabetes Association. Philadelphia, PA; 2001.
- 62.Jones H, Edwards L, Vallis TM, et al. Changes in diabetes self-care behaviors make a difference in glycemic control: The Diabetes Stages of Change (DiSC) study. Diabetes Care. 2003;26(3):732–737. doi: 10.2337/diacare.26.3.732. [DOI] [PubMed] [Google Scholar]
- 63.Rossi J, ed. The healthy changes trial biomedical results. In L. Ruggiero (Chair), Helping people change: The impact of stage-based behavior change interventions in diabetes care. In: 28th Annual Conference of the American Association of Diabetes Educators. Louisville, KY; 2001.
- 64.Prochaska JO, Velicer WF, Fava JL, Rossi JS, Tsoh JY. Evaluating a population-based recruitment approach and a stage-based expert system intervention for smoking cessation. Addict Behav. 2001;26(4):583–602. doi: 10.1016/S0306-4603(00)00151-9. [DOI] [PubMed] [Google Scholar]
- 65.Prochaska JJ, Velicer WF, Nigg CR, Prochaska JO. Methods of quantifying change in multiple risk factor interventions. Prev Med. 2008;46(3):260–265. doi: 10.1016/j.ypmed.2007.07.035. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66.Lewis CH. Using the “thinking aloud” method in cognitive interface design (technical report IBM RC-9265). Yorktown Heights, NY; 1982.
- 67.Lewis CH, Rieman J. Task-centered user interface design: A practical introduction. 2011.
- 68.Kelley JF. An empirical methodology for writing user-friendly natural language computer applications. New York, New York, USA: ACM Press; 1983. pp. 193–196. [Google Scholar]
- 69.Kelley JF. An iterative design methodology for user-friendly natural language office information applications. ACM Trans Inf Syst. 1984;2(1):26–41. doi: 10.1145/357417.357420. [DOI] [Google Scholar]
- 70.National Cancer Institute. Making health communication programs work: A planner's guide (publication no. 89–1493). Bethesda, MD; 1989.
- 71.Cardinal BJ. Development and evaluation of stage-matched written materials about lifestyle and structured physical activity. Percept Mot Skills. 1995;80(2):543–546. doi: 10.2466/pms.1995.80.2.543. [DOI] [PubMed] [Google Scholar]
- 72.Rimer BK, Orleans CT, Fleisher L, et al. Does tailoring matter? The impact of a tailored guide on ratings and short-term smoking-related outcomes for older smokers. Health Educ Res. 1994;9(1):69–84. doi: 10.1093/her/9.1.69. [DOI] [PubMed] [Google Scholar]
- 73.Weathers FW, Litz BT, Herman DS, Huska JA, Keane MT. The PTSD Checklist (PCL): Reliability, validity, and diagnostic utility. In: The 9th Annual Meeting of the International Society for Traumatic Stress Studies, October 24–27; 1993.
- 74.Schell T, Tanielian T, eds. A needs assessment of New York State Veterans. RAND Corporation; 2011. [PMC free article] [PubMed]
- 75.Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav. 1983;24:385–396. doi: 10.2307/2136404. [DOI] [PubMed] [Google Scholar]
- 76.Spitzer R, Kroenke K. The PHQ-9: A new depression diagnostic and severity measure. Psychiatr Ann. 2002;32(9):1–7. [Google Scholar]
- 77.Kroenke K, Spitzer RL, Williams JB. The PHQ-9: Validity of a brief depression severity measure. J Gen Intern Med. 2001;16(9):606–613. doi: 10.1046/j.1525-1497.2001.016009606.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 78.Kroenke K, Spitzer RL, Williams JBW, Löwe B. The patient health questionnaire somatic, anxiety, and depressive symptom scales: A systematic review. Gen Hosp Psychiatry. 32(4):345-359. [DOI] [PubMed]
- 79.Flanagan JC. A research approach to improving our quality of life. Am Psychol. 1978;33:138–147. doi: 10.1037/0003-066X.33.2.138. [DOI] [Google Scholar]
- 80.Flanagan JC. Measurement of quality of life: Current state of the art. Arch Phys Med Rehabil. 1982;63(2):56–59. [PubMed] [Google Scholar]
- 81.Burckhardt CS, Anderson KL. The quality of life scale (QOLS): Reliability, validity, and utilization. Health Qual Life Outcomes. 2003;23(1):60. doi: 10.1186/1477-7525-1-60. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 82.DiClemente CC, Prochaska JO, Fairhurst SK, et al. The process of smoking cessation: An analysis of precontemplation, contemplation, and preparation stages of change. J Consult Clin Psychol. 1991;59(2):295–304. doi: 10.1037/0022-006X.59.2.295. [DOI] [PubMed] [Google Scholar]
- 83.Miles M, Huberman AM. Qualitative data analysis: A sourcebook of new methods. Beverly Hills, CA: SAGE; 1984. p. 264. [Google Scholar]
- 84.Morgan DL, Krueger RA, King JA. Analyzing and reporting focus group results, volume 6. SAGE; 1998:139.
