Abstract
This pilot study tested the efficacy of the My Disaster Recovery (MDR) website to decrease negative affect and increase coping self-efficacy. Fifty-six survivors of Hurricane Ike were recruited from a larger study being conducted at the University of Texas Medical Branch at the first anniversary of the storm. Restricted randomization was used to assign participants to the MDR website, an information-only website, or a usual care condition. Group × time interactions indicated that MDR reduced participant worry more than the other conditions. A similar trend was also identified for depression. Both websites were accessed a small to moderate amount and participants reported mixed satisfaction for both websites. Although the effect sizes for worry and depression were in the moderate to large range, small sample size and timing of the intervention qualify the findings. These preliminary findings encourage further evaluation of MDR with a larger, demographically diverse sample and indicate that the MDR website might be helpful in reducing worry and depression.
Keywords: disaster, coping, internet intervention, posttraumatic stress, randomized controlled trial, coping self-efficacy
Over the past decade, the internet has quickly become a popular medium for delivering mental health interventions (Ritterband et al., 2003). However, few programs have been developed to assist populations after large-scale community experienced traumas (e.g., hurricane, terrorist attack) and no randomized control data is available on web-based interventions following a major disaster (Benight, Ruzek, & Waldrep, 2008). My Disaster Recovery (MDR) is a self-help, web-based empowerment program designed to assist disaster survivors and facilitate their recovery (Benight et al.). Based on social cognitive theory, MDR targets coping self-efficacy, which has been shown to play an important role in determining psychological outcomes after experiencing a disaster (Benight & Bandura, 2004; Luszczynska, Benight, & Cieslak, 2009).
The current pilot study's primary aim was to provide an initial test of the effectiveness of the MDR website following a major disaster. It was hypothesized that the MDR and information-only website groups would show greater improvement (reduced PTSD, depression, worry, stress level, and increased coping self-efficacy) than the usual care control group (who continued their standard physical and mental healthcare during the course of the study). It was also hypothesized that the MDR website group would show greater improvement than the information-only website group because the MDR website would be used and liked more due to its multi-media components and interactivity.
Method
Materials
My Disaster Recovery and information-only websites
MDR consists of six interactive modules: seeking professional help, relaxation, social support, unhelpful ways of coping, self-talk, and trauma triggers and memories (detailed information on the specific module content and structure is provided elsewhere in Benight et al., 2008). The website utilizes interactive experiences tailored to each user (e.g., self tests, video vignettes) to target coping self-efficacy pathways (i.e., mastery experiences, vicarious success modeling, verbal persuasion, and tools to monitor and regulate internal distress). The comparative information-only site presented all the same information as e-books without the interactive components.
Measures
The Trauma Screening Questionnaire (TSQ; Brewin et al., 2002) was used as a screening tool for participant selection. The TSQ is a 10-item, self-report screening assessment for PTSD with five items assessing re-experiencing symptoms and five items assessing arousal symptoms. The TSQ has shown good predictive validity (Blake et al., 1995). In the current study, the Cronbach's alpha coefficient was .72.
The Perceived Stress Scale (PSS; Cohen, Kamarck, & Mermelstein, 1983) was used as a screening tool and as a primary outcome measure. The PSS is a 10-item, self-report measure assessing stressful feelings and thoughts. The PSS asks how often in the past month participants have been feeling,with items such as “You felt that you were unable to control the important things in your life?” on a scale ranging from 0 (never) to 4 (very often) The PSS shows acceptable reliability and validity (Cohen & Williamson, 1988). In the current study, the Cronbach's alpha coefficient was .82.
The Coping Self-Efficacy Scale for Trauma (CSE) is a 20-item questionnaire that assesses self-reported belief in competency to manage demands associated with trauma recovery on a scale ranging from 1 (not at all capable) to 7 (I'm totally capable). Preliminary psychometric data from four different samples suggest a unidimensional construct explaining over 50% of the variance in each sample, factor loadings were all above .50, and Cronbach's alphas were all above .85 in each sample (C. C. Benight, personal communication, December 6, 2010). In the current study, the Cronbach's alpha coefficient was .97.
The Modified PTSD Symptoms Scale (MPSS; Falsetti, Resnick, Resick, & Kilpatrick, 1993) is a 17-item questionnaire that assesses the frequency and duration of PTSD symptoms in the past two weeks on a scale ranging from 0 (not at all) to 4 (five or more times a week). Respondents also indicate how severe the symptom was on a five-point scale ranging from A (not distressing at all) to E (extremely distressing). The MPSS has shown high internal consistency and good concurrent validity. The Cronbach's's alpha coefficient for the present study was .85.
The Center for Epidemiologic Studies Depression Scale (CES-D; Radloff, 1977) is a 20-item, self-report questionnaire measuring the frequency of depressive symptoms in the past week on a four-point scale ranging from rarely or none of the time (less than one day) to most or all of the time (five to seven days). The CES-D has shown high internal reliability and validity (Radloff). The Cronbach's alpha coefficient for the current study was .71.
The Penn State Worry Questionnaire (PSWQ; Meyer, Miller, Metzger, & Borkovec, 1990) is a 16-item questionnaire that measures uncontrollable worry in adults on a scale ranging from 1 (not at all typical) to 5 (very typical). The measure has acceptable internal consistency and validity (Zlomke, 2009). In the current study, the Cronbach's alpha coefficient was .94.
A seven-item demographic questionnaire (age, gender, race, ethnicity, marital status, education, and an item about whether they were receiving therapy or medication) was included in the Time 1 assessment.
Website-use and user satisfaction questionnaires were included in the follow-up assessment for participants assigned to the MDR and information-only groups. More information about the items included in these questionnaires and the scale ranges is provided in the results section.
Participants and procedure
Participants were recruited through the University of Texas Medical Branch from a larger study being conducted on stress and coping in Hurricane Ike survivors. Individuals from the stress and coping study who had access to the internet, were 21 years or older, and met distress criteria (i.e., scored five or greater on the TSQ [Brewin et al., 2002] or a score higher than 20 on the PSS [Cohen et al., 1983]) were invited to participate.
Data collection for the current study began 11 months after Hurricane Ike hit Galveston. A recruitment e-mail directed interested participants to an on-line survey where they were informed of the procedures for the study, consented to participate, and completed the 45-minute-long Time 1 assessment. Rolling assignment was used to allocate the 56 participants to one of the three conditions after they completed the Time 1 assessment. Given the small size and large number of females in our sample, restricted randomization was used to ensure balance in group size and gender. Neither researchers nor participants were blind to group assignment. Participants assigned to the usual care group were informed that they should continue with their usual standard physical and mental healthcare whereas participants in the MDR and information-only groups were told that they should work on their assigned website as often as they could over the next month. One reminder e-mail about the second assessment was sent halfway through the study. Each participant was allowed a minimum of 30 days to access the website, and all participants were prompted by e-mail to take the follow-up questionnaire. Participants assigned to the usual care group and information-only groups were allowed to access MDR after completing the second survey. Participants who provided contact information were compensated $20 each time they completed an assessment.
Participant flow through the Consolidated Standards of Reporting Trials (CONSORT; Altman et al., 2001; Moher, Schulz, & Altman, 2001) chart can be obtained by contacting the corresponding author (C. C. Benight). One person (less than 1%) in the usual care group failed to complete the follow-up survey compared to 22% of the participants in the MDR group and 32% of the participants in the information-only group. Further, three participants in the MDR group and two participants in the information-only group did not receive the allocated intervention because they failed to access the website despite completing the follow-up assessment. All participants were included in the final intent to treat analyses by carrying forward Time 1 scores. Table 1 shows that random assignment equalized most of the demographic variables among the groups, except the MDR group had more participants who were previously married and were more highly educated than the other groups. No significant differences were found between groups at Time 1 for the outcome measures.
Table 1.
Group Demographic Characteristics
| Usual care (n = 19) | MDR (n = 18) | Information-only (n = 19) | |
|---|---|---|---|
| Age | 45.37 (10.73) | 43.06 (11.51) | 40.53 (11.08) |
| Female | 16 (84%) | 15 (83%) | 17 (89%) |
| Caucasian | 18 (95%) | 16 (89%) | 17 (89%) |
| Single, never married | 3 (16%) | 4 (22%) | 5 (26%) |
| Married/living with partner | 13 (68%) | 8 (44%) | 12 (63%) |
| Separated, divorced, widowed | 3 (16%) | 6 (33%) | 1 (.50%) |
| Less than a Bachelors degree | 8 (42%) | 2 (11%) | 7 (37%) |
| Bachelors degree or more | 11 (58%) | 16 (89%) | 12 (63%) |
Note. SDs in parentheses.
Results
Table 2 provides descriptive and group × time interaction statistics for all outcome variables. A significant group × time interaction was found for worry (Wilks' Lambda = .90, F[2, 53] = 3.12, p = .05, partial ɳ2 =.11). Simple effects analysis for worry showed the MDR group significantly improved over time (p < .01; Time 1 M = 52.83, SD = 14.81; follow-up M = 47.94, SD = 15.47), whereas the information-only and usual care groups showed no change. A similar trend, although not significant, was found for depression (Wilks' Lambda = .92, F[2, 53] = 2.45, p = .10, partial ɳ2 = .09). Follow-up analyses showed that depression scores revealed significant improvement in the MDR group (p =.07; Time 1 M = 24.22, SD = 10.21; follow-up M = 17.72, SD = 11.74), whereas the information-only and usual care groups showed no change. There were no significant interactions for PTSD, perceived stress, or coping self-efficacy. Table 3 shows that no differences were found in website use and satisfaction between the MDR and information-only website groups. Participants in both groups reported that they spent a little to a moderate amount of time on the websites and responses for website satisfaction were mixed and tended to fall into the neutral range (MDR = 17.00 [SD = 4.40]; information-only = 17.90 [SD = 2.95]).
Table 2.
Descriptive Statistics and Group × Time Comparisons for Primary Outcome Variables
| Usual care (n = 19) | MDR (n = 18) | Information-only (n = 19) | Wilks' Lambda | Statistic | p | Partial ɳ2 | |
|---|---|---|---|---|---|---|---|
| CSE | .95 | F(2, 53) = 1.41 | .25 | .05 | |||
| Time 1 | 87.63 (21.74) | 104.11 (27.45) | 93.00 (26.80) | ||||
| Follow-up | 100.58 (22.52) | 109.44 (25.83) | 101.26 (28.35) | ||||
| MPSS | .92 | F(2, 53) = 2.23 | .12 | .08 | |||
| Time 1 | 28.26 (18.42) | 26.72 (17.39) | 29.84 (18.42) | ||||
| Follow-up | 21.58 (16.30) | 23.94 (16.74) | 29.26 (19.46) | ||||
| Perceived Stress | .94 | F(2, 53) = 1.61 | .21 | .06 | |||
| Time 1 | 22.00 (4.96) | 21.06 (5.36) | 23.42 (6.09) | ||||
| Follow-up | 21.79 (5.37) | 19.11 (7.14) | 23.58 (6.16) | ||||
| Worry | .90 | F(2, 53) = 3.12 | .05 | .11 | |||
| Time 1 | 51.37 (15.25) | 52.83 (14.81) | 59.68 (15.47) | ||||
| Follow-up | 52.68 (15.14) | 47.94 (15.47) | 58.00 (15.21) | ||||
| Depression | .92 | F(2, 53) = 2.45 | .10 | .09 | |||
| Time 1 | 22.37 (12.10) | 24.22 (10.21) | 24.58 (12.56) | ||||
| Follow-up | 21.05 (11.03) | 17.72 (11.74) | 22.58 (12.76) |
Note. SDs in parentheses.
Table 3.
Descriptive Statistics for Reported Website Use and Satisfaction
| Website Use | MDR (n = 11) | Information-only (n = 11) |
|---|---|---|
| Average number of log-on | 3.64 (2.06) | 3.64 (2.87) |
| Average amount of minutes spent per visit | 30.91 (17.72) | 28.64 (17.90) |
| How much accessed sitea | 2.64 (1.36) | 2.55 (0.93) |
| Time spent on modules/e-booksb | 2.38 (0.96) | 2.39 (1.02) |
| Website Satisfaction |
||
| User satisfaction (5 items)c | 17.00 (4.40) | 17.90 (2.95) |
| Perceived helpfulness of modules/e-booksb | 2.44 (1.16) | 2.20 (1.01) |
| Helpfulness of videos and vignettesb | 3.36 (1.36) |
Note. SDs in parentheses.
Six-point scale ranging from 1 (very brief) to 6 (very extensive).
Five-point scale ranging from 1 (not at all) to 5 (very much).
Five-point scale totaled across items ranging from 1 (strongly disagree) to 25 (strongly agree completely).
Discussion
Exploratory results from this pilot study show that the MDR website significantly reduced worry when compared to the information-only and usual care groups. Although not significant, a similar trend was identified for depression. The partial ɳ2 values for worry and depression indicate moderate to large effects (Cohen, 1973). No significant effects were found for PTSD symptoms, perceived stress, or coping self-efficacy. Differential types of distress at Time 1 are a likely explanation for these mixed findings. Our sample expressed minimal presence of PTSD symptoms (Falsetti et al., 1993) and the mean CSE score was at the midpoint of the scale at Time 1, whereas participants reported higher levels of worry (Radloff, 1977) and depression (Meyer et al., 1990). The aspects of the MDR website targeted at decreasing worry and possibly depressive symptoms may have been more relevant to participants' needs, whereas aspects of the site targeted at decreasing posttraumatic symptoms were not as relevant. Thus, the timing of the intervention post-disaster should be evaluated in future studies utilizing MDR (or other web-intervention systems) relative to different recovery outcomes (e.g., PTSD symptoms). Repeated assessment trials need to be conducted beginning during the acute recovery period and following participants over time as recovery demands change. Further, it seems future studies may want to consider that screening for worry might be more useful than screening for PTSD symptoms when assessing post-disaster distress after the acute phase of recovery has ended.
The differential drop out rate between conditions in our study is likely a result of the larger participant burden required for the intervention groups. The dropout rate for our MDR intervention group (33%, including the 3 participants who failed to receive intervention) appears reasonable when compared to other web-based studies for trauma, particularly when you consider these other programs involved more personal contact (18 –36%; Hirai & Clum, 2005; Lange et al., 2003). However, the attrition superimposed on an already small sample size limited the current study to detect only medium to larger effect sizes. Our attempt to counterbalance the differential drop out rate between groups with conservative, intent to treat analyses does not remedy the study's limitations. The findings of this study need replication and interpretation of the results must be qualified with the differences detected among the groups at Time 1 for education and marital status. It is conceivable that the MDR group's higher education and being married in some way influenced their reception to the MDR treatment, thus causing their greater improvement in depression and worry from Time 1 to follow-up. Future research should examine whether some groups of people may benefit more than others from self-directed, on-line interventions.
The limited ability to detect meaningful effects is further complicated by the findings that participants who did complete the study reported that they used the MDR and information-only websites a little to moderate amount. However, the only other published paper on a web-system for disaster recovery also found limited utilization (50% module utilization ranging from 36% – 64%; Ruggiero et al., 2006). It is unclear at this time how much a web intervention needs to be accessed by users in order for it to have an influence on recovery. Future studies need to examine the relationship between amount of time spent utilizing a web system and improvements in critical outcomes. Lastly, the present study is limited in its generalizability given the study sample was predominately female (85%) and Caucasian (91%). Future studies should evaluate the MDR website within a larger, demographically diverse sample.
In closing, despite the study's limitations, these exploratory findings provide the first RCT completed with a web intervention system following a major disaster. This information provides a useful platform for future studies that can test the potential benefit of the internet as an intervention vehicle in post-disaster environments.
Acknowledgements
This study was funded by National Institute of Mental Health grant R41MH082498-01. This material is based upon work supported in part by the Air Force Office of Sponsored Research under Award FA9550-04-1-0239.
Footnotes
Dr. Benight owns equity in BlueSun, Inc. which has an exclusive license from the University of Colorado. The terms of Dr. Benight's involvement in this project were approved by the University of Colorado at Colorado Springs in accordance with their conflict of interest policy.
References
- Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, Lang T. The revised CONSORT statement for reporting randomized trials; Explanation and elaboration. Annals of Internal Medicine. 2001;134:663–694. doi: 10.7326/0003-4819-134-8-200104170-00012. doi:10.1001/jama.285.15.1987. [DOI] [PubMed] [Google Scholar]
- Benight CC, Bandura A. Social cognitive theory of posttraumatic recovery: The role of perceived self-efficacy. Behavior Research and Therapy. 2004;42:1129–1148. doi: 10.1016/j.brat.2003.08.008. doi:10.1016/j.brat.2003.08.008. [DOI] [PubMed] [Google Scholar]
- Benight CC, Ruzek JI, Waldrep E. Internet interventions for traumatic stress: A review and theoretically-based example. Journal of Traumatic Stress. 2008;21:513–520. doi: 10.1002/jts.20371. doi:10.1002/jts.20371. [DOI] [PubMed] [Google Scholar]
- Blake DD, Weathers FW, Nagy LM, Kaloupek DG, Gusman FD, Charney DS, Keane TM. The development of a clinician-administered PTSD scale. Journal of Traumatic Stress. 1995;8:75–90. doi: 10.1007/BF02105408. doi:10.1007/BF02105408. [DOI] [PubMed] [Google Scholar]
- Brewin CR, Rose S, Andrews B, Green J, Tata P, McEvedy C, Foa EB. Brief screening instrument for post-traumatic stress disorder. The British Journal of Psychiatry. 2002;181:158–162. doi: 10.1017/s0007125000161896. Retrieved from http://bjp.rcpsych.org/ [DOI] [PubMed] [Google Scholar]
- Cohen J. Eta-squared and partial eta-squared in fixed factor ANOVA designs. Educational and Psychological Measurement. 1973;33:107–112. [Google Scholar]
- Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. Journal of Health and Social Behavior. 1983;24:385–396. doi:10.2307/2136404. [PubMed] [Google Scholar]
- Cohen S, Williamson GM. Perceived stress in a probability sample of the United States. In: Spacapan S, Oskamp S, editors. The Social Psychology of Health: Claremont Symposium on Applied Social Psychology. Sage; Newbury Park, CA: 1988. pp. 31–67. [Google Scholar]
- Falsetti SA, Resnick HS, Resick PA, Kilpatrick DG. The modified PTSD symptom scale: A brief self-report measure of posttraumatic stress disorder. The Behavior Therapist. 1993;16:161–162. Retrieved from http://www.elsevier.com/wps/find/homepage.cws_home. [Google Scholar]
- Hirai M, Clum GA. An internet-based self-change program for traumatic event related fear; distress, and maladaptive coping. Journal of Traumatic Stress. 2005;18:631–636. doi: 10.1002/jts.20071. doi: 10.1002/jts.20071. [DOI] [PubMed] [Google Scholar]
- Lange A, Rietdijk D, Hudcovicova M, Van de Ven J-P, Schrieken B, Emmelkamp PMG. Interapy: A controlled randomized trial of the standardized treatment of posttraumatic stress through the internet. Journal of Consulting and Clinical Psychology. 2003;71:901–909. doi: 10.1037/0022-006X.71.5.901. doi: 10.1037/0022-006X.71.5.901. [DOI] [PubMed] [Google Scholar]
- Luszczynska A, Benight CC, Cieslak R. Self-efficacy and health-related outcomes of collective trauma. European Psychologist. 2009;14:51–62. doi:10.1027/1016-9040.14.1.51. [Google Scholar]
- Meyer TJ, Miller ML, Metzger RL, Borkovec TD. Development and validation of the Penn State Worry Questionnaire. Behaviour Research and Therapy. 1990;28:487–495. doi: 10.1016/0005-7967(90)90135-6. doi:10.1016/0005-7967(90)90135-6. [DOI] [PubMed] [Google Scholar]
- Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. The Lancet. 2001;357:1191–1194. doi:10.1016/S0140-6736(00)04337-3. [PubMed] [Google Scholar]
- Radloff LS. The CES-D scale: A self-report depression scale for research in the general population. Applied Psychological Measurement. 1977;1:385–401. doi: 10.1177/014662167700100306. [Google Scholar]
- Ritterband LM, Gonder-Frederick LA, Cox DJ, Clifton AD, West RW, Borowitz SM. Internet intervention: In review, in use, and into the future. Professional Psychology: Research and Practice. 2003;34:527–534. doi: 10.1037/0735-7028.34.5.527. [Google Scholar]
- Ruggiero KJ, Resnick HS, Acierno R, Carpenter MJ, Kilpatrick DG, Coffey SF, Galea S. Internet-based intervention for mental health and substance use problems in disaster-affected populations: A pilot feasibility study. Behavior Therapy. 2006;37:190–205. doi: 10.1016/j.beth.2005.12.001. doi:10.1016/j.beth.2005.12.001. [DOI] [PubMed] [Google Scholar]
- Zlomke K. Psychometric properties of internet administered versions of Penn State Worry Questionnaire (PSWQ) and Depression, Anxiety, and Stress Scale (DASS) Computers in Human Behavior. 2009;25:841–843. doi:10.1016/j.chb.2008.06.003. [Google Scholar]