Bounce Back Now! Protocol of a Population-Based Randomized Controlled Trial to Examine the Efficacy of a Web-based Intervention with Disaster-Affected Families

Abstract

Disasters have far-reaching and potentially long-lasting effects on youth and families. Research has consistently shown a clear increase in the prevalence of several mental health disorders after disasters, including depression and posttraumatic stress disorder. Widely accessible evidence-based interventions are needed to address this unmet need for youth and families, who are underrepresented in disaster research. Rapid growth in Internet and Smartphone access, as well as several web based evaluation studies with various adult populations has shown that web-based interventions are likely to be feasible in this context and can improve clinical outcomes. Such interventions also are generally cost-effective, can be targeted or personalized, and can easily be integrated in a stepped care approach to screening and intervention delivery. This is a protocol paper that describes an innovative study design in which we evaluate a self-help web-based resource, Bounce Back Now, with a population-based sample of disaster affected adolescents and families. The paper includes description and justification for sampling selection and procedures, selection of assessment measures and methods, design of the intervention, and statistical evaluation of critical outcomes. Unique features of this study design include the use of address-based sampling to recruit a population-based sample of disaster-affected adolescents and parents, telephone and web-based assessments, and development and evaluation of a highly individualized web intervention for adolescents. Challenges related to large-scale evaluation of technology-delivered interventions with high-risk samples in time-sensitive research are discussed, as well as implications for future research and practice.

Disasters are a significant public health concern because they can affect the wellbeing of a substantial number of families simultaneously (Galea et al., 2002; North & Pfefferbaum, 2013). Disasters confront youth and families with a wide range of stressors, including threat of death or injury, loss of loved ones, limited access to basic needs, and financial strain. The prevalence of mental health disorders (e.g., PTSD, depression, substance abuse) increases in the post-disaster period (e.g., Del Gaizo et al., 2011; Garrison et al., 1995; La Greca et al., 1996; Norris et al., 2006; Oldham, 2013; Ruggiero et al., 2012). Widely accessible, evidence-based interventions are needed (Gillies et al., 2012). Tailored and automated interventions (i.e., do not require provider navigation) are of particular interest because they may address common access-to-care barriers.

Disaster mental health interventions for youth are a major priority. Pfefferbaum (2014) and Soltis & Ruggiero (2013) reviewed post-disaster intervention studies for youth. Most (n=32 of 48) evaluated behavioral and/or cognitive interventions. Thirty were conducted outside of the US (i.e., Giannopoulou et al., 2006; Shooshtary et al., 2008; Lesmana et al., 2009). Most were conducted with hurricane-affected samples (e.g., Jaycox et al., 2010; Taylor & Weems, 2011; Salloum & Overstreet, 2008, 2012). Many interventions were implemented several months or years after the disaster, and none was delivered with web-based or mobile technology.

Rapid growth in Internet and Smartphone access presents an opportunity to deliver evidence-based resources widely and cost-efficiently (Fox & Duggan, 2013; Madden et al., 2013). Research has supported the feasibility of web-based interventions (Ruggiero et al., 2006; Crutzen & De Nooijer, 2011), and web-based interventions have performed well in efficacy evaluations among adults (see Amstadter, Broman-Fulks, et al., 2009; Bickel et al., 2011; Portnoy et al., 2008). The few studies relating to child and adolescent mental health intervention programs have generally yielded encouraging findings, suggesting high potential for population-level impact (Calear & Christensen, 2010; Tait & Christensen, 2010).

The primary aim of this study was to evaluate the efficacy of Bounce Back Now (BBN), a web-based intervention for disaster-affected adolescents and parents. Address-based sampling was used to recruit adolescents and parents in Alabama and Joplin, MO; areas that experienced costly, deadly tornados in the spring of 2011 (NOAA, 2011). A secondary aim was to inform the knowledge of biologic underpinnings of PTSD and related phenotypes in youth via collection and analysis of adolescent salava samples. Heritability estimates are moderate for PTSD (see Amstadter, Nugent, & Koenen, 2009), but little is known about the molecular variation that accounts for genetic influence. Last, this study will inform the growing field of therapygenetics as the first to explore gene-by-treatment outcome interactions with a web-based intervention. A small number of PTSD treatment studies have found that individuals carrying risk variants of certain genes, such as the s′ allele of the 5-HTTLPR and the Met-66 allele of the BDNF polymorphism, were less responsive to therapy than individuals homozygous for the major alleles for these genes (Bryant et al., 2010, Felmingham et al., 2013).

Method

I. Study Overview

A population-based randomized controlled trial (RCT) was conducted to examine the initial efficacy of BBN. A sample of 2,000 disaster-affected families was recruited between September, 2011 and June, 2012. Structured baseline, 4-month, and 12-month interviews were conducted with an adolescent and a designated parent. When multiple eligible adolescents lived in the home, the adolescent participant was randomly selected using the most recent birthday method. Parents and adolescents were invited to access the Website after the baseline interview. Families were randomized to one of the 3 experimental conditions only after accessing the website: (a) Web intervention for disaster-affected youth (i.e., addressing only adolescent outcomes via the adolescent and parenting modules), (b) Web intervention for disaster-affected families (i.e., addressing both youth outcomes but also adult outcomes via self-help modules accessible to the parents), or (c) an assessment-only Web-comparison. Web use and knowledge change data were collected. Adolescents also were invited to contribute a saliva sample. DNA was isolated from the received samples, and genotyping was conducted for the purpose of identification of genetic variation that contributes to post-disaster mental health symptoms in youth. All follow-up interviews were completed by August of 2013. We describe below the methodological aspects of the project as well as the rationale for critical methodological decision points.

II. Setting

The current project focused recruitment on two regions of the United States sustaining particularly severe impact from this spree of tornadoes. On April 27, 2011, northern Alabama experienced a historic 39 tornados ranging from Enhanced Fujita (EF) scale categories 4 (winds 166–200 mph) to EF 5 (winds greater than 200mph). The tornadoes caused significant property damage, injury, and death. Over 14,000 homes were destroyed or deemed uninhabitable, 2,200 people were injured, and 240 individuals lost their lives as a result of tornadoes impacting northern Alabama (Addy & Ijaz, 2011; National Oceanic and Atmospheric Administration [NOAA], 2011, 2012, 2013; Wind Science and Engineering Center, 2004). On May 22, 2011, an EF 5 Tornado struck the city of Joplin, Missouri, leaving more than 150 dead and over 1,000 injured (Federal Emergency Management Administration [FEMA]; NOAA, 2012), as well as leaving a wake of significant destruction in the town, including almost 7,000 homes destroyed (FEMA, 2011b). Households within these two regions that were within 5 miles of these tornadoes were included in our recruitment sample.

Rationale for Selection of Disaster-Affected Areas

Disaster-affected children and adolescents are at-risk for a variety of mental health problems (Blaze & Schwalb, 2009; Boer et al., 2009; Comer et al., 2010) with the experience of PTSD and depressive symptoms being among the most prominent of outcomes (Belter, Dunn, & Jeney, 1991; Catani et al., 2008). Tornadoes occur more frequently than other disasters, and are unique in that they can strike with little warning, specific paths are largely unpredictable, patterns of damage are selective, and they have a devastating impact (Evans & Oehler-Stinnett, 2006; NOAA, 2013).

To date, there have only been a handful of studies examining mental health outcomes post-tornado (Polusny et al., 2011; Evans & Oehler-Stinnett, 2006; Houlihan et al., 2008). Optimally, Bounce Back Now is designed for the 1–6 month period after disasters. We chose to recruit families affected by the spring 2011 tornado outbreaks for purposes of this study due to the relative recency of the outbreak and numerous risk factors present in the areas of Joplin, MO, and several areas of Alabama.

III. Participants

The sample consisted of 2,000 adolescents aged 12–17 years and their 2,000 parents. Only one adolescent and one parent participated from each household. These families resided in the areas of Joplin, Missouri and Alabama affected by the tornados during the spring of 2011. All dyads had Internet access at home. Families were excluded if they resided in institutional settings, if they had neither a landline nor a cellphone, if their household did not have Internet access, or if they were not fluent in English. Non-English speaking participants were excluded because time and budget restrictions did not allow the interviews and intervention materials to be translated into additional languages.

Rationale for Targeting Adolescents and Their Families

Our goal was to develop a multi-module Web-based, self-paced, intervention for adolescents and their parents. We selected the target population to be adolescents aged 12–17 years because (a) data and resources for disaster-affected youth and families are especially limited; (b) a wider age range (e.g., 6–17 years) would have produced considerable complexity around developmental issues that would have necessitated the use of multiple intervention, measurement, and Web-design strategies for both youth and parents; (c) a narrower age range would have substantially raised costs associated with household sampling procedures (i.e., narrow eligibility windows increase the number of households that need to be called before an eligible family is reached); and (d) adolescence is a particularly developmentally sensitive period for mental health problems such as depression and substance abuse.

IV. Recruitment and Sampling Procedures

We used a targeted address-based sampling (ABS) frame based on latitude/longitude coordinates to reduce screening and maximize the incidence of adolescents most affected by the tornadoes. ABS is the sampling of addresses from a near universal listing of residential mail delivery locations. The sampling frame is a list of addresses. The choice of list depends upon the population of generalizability. The most complete frame of household addresses that has emerged for all 50 states and the District of Columbia and been made commercially available is the Computerized Delivery Sequence File (CDSF) developed and maintained by the United States Postal Service. This file is estimated to cover approximately 97% of the mailing addresses. That coverage varies by state and is somewhat lower in rural areas; it is considered the most comprehensive available. The CDSF is intended to specify the delivery sequence for postal mail carriers that deliver the mail on their individual routes in a prescribed and inviolable order of addresses. Licensed commercial vendors receive the latest versions of the CDSF and process it to make it suitable for sampling. Each vendor makes enhancements to improve the usability and accuracy of the information it contains. Vendors also can append ancillary information to each address from a variety of commercial databases including the telephone White Pages for purposes of targeting, creating stratified samples by demographic characteristics or geography, or for doing pseudo non-response analyses. Most commonly, these addresses are matched to obtain a landline telephone number for either initial survey contact or telephone follow-up efforts after an initial mail contact. Currently, there is no extant database for matching addresses to cellphone numbers.

A multi-step sampling procedure was employed. First, the paths of the selected tornadoes were plotted and the coordinates identified. Then, Census block IDs were assigned by plurality to increasing radii around each coordinate (e.g., 0.5 mile, 2 miles). These radii served as the strata from which over 200,000 addresses were selected. We intended to recruit via random selection but ultimately, in most areas, targeted all households (i.e., full census) in geographic areas affected by the tornadoes due to the size of the population in most (rural) areas and the nature of the disaster that necessitated confining the geographic areas for recruitment. Next, phone numbers were appended if a match could be made. Households for which phone numbers could be matched to addresses were called and screened for presence of children aged 12–17 years and Internet access. Addresses unable to be matched to a phone number were sent a letter and a screening questionnaire that assessed eligibility criteria to identify households with adolescents and Internet access, obtain contact information, and assess phone status and preferred contact method (cellphone, landline). Families mailed a screening questionnaire were offered $5 for returning the screener, regardless of their eligibility status for the larger study. Baseline telephone interviews then were conducted with eligible households from both samples.

The phone interviews were conducted using Computer-Assisted Telephone Interviewing (CATI) technology. This technology has many advantages: (a) CATI is better able to handle skip patterns in interview schedules; (b) CATI insures that all questions will be asked because interviewers cannot proceed in the program without entering participant responses (or refusals); (c) CATI interviews take less time than other methods and result in less respondent fatigue, increased adherence, and reduced termination likelihood; and (d) CATI also reduces the time for data reduction and cleaning. As described by Cantor and Lynch (2000), surveys using the CATI method appear to increase detection of sensitive incidents, such as sexual assault and other forms of crime. Interviewers were Abt SRBI employees who are well trained, highly skilled, and experienced in conducting CATI interviews.

After an eligible family agreed to participate, baseline interviews (described below) were conducted with the identified parent and (if necessary, randomly selected) adolescent. Toward the end of the baseline interview, after the Web component had been described, participants were given instructions for accessing the Website and were assigned unique userids. Parents and adolescents were assigned different userids that linked to one another as well as to the same intervention condition. All participants subsequently received a mailed follow-up letter that described the Web component of the study, provided details about how to access the Website, and again provided the userid to the participants. At the time of interview, both the interviewer and participants were blind to intervention condition. This was possible because random assignment to conditions did not occur until the time that participants accessed the Website and logged in with their userid. As noted, the three study conditions were as follows: (a) Web intervention for disaster-affected youth only, (b) Web intervention for disaster-affected families (youth plus adult self-help), or (c) an assessment-only Web-comparison.

Rationale for Use of Population-based (vs. Convenience) Samples

Population-based samples were preferable to convenience (e.g., treatment-seeking, student) samples for several reasons. First, recruitment of treatment seeking or other high-risk samples would have severely limited information regarding the community-level impact of the intervention. Our intervention was designed to serve adolescents with clinical and subclinical symptom levels, and therefore was assumed to be beneficial to a broader range of families than found in treatment-seeking samples. Second, as we learned in prior work (Ruggiero et al., 2006), recruitment of a sample with a clearly defined, population-based recruitment pool offers valuable and potentially generalizable estimates of use and benefit of the intervention. Many studies using convenience samples and unknown recruitment pools do not offer this benefit. Third, data from population-based samples, as compared to clinical samples, are likely to generalize well to disaster-affected communities, which typically have high demographic variability, variability in mental health and service utilization histories, and experience seeking health-relevant information online.

Rationale for Address-Based Sampling Methods

The concept of using the CDSF to generate a sampling frame was first examined by Iannacchione, Staab, and Redden (2003). Address-based sampling emerged as an alternative to manual enumeration to enfranchise individuals without landline phones, historically reached through random-digit dial (RDD) surveys. Landline use has decreased dramatically in recent years (Rainie, 2008). Over 31% of US homes used only cellphones in 2011 (Blumberg & Luke, 2011). Traditional RDD methodology does not capture these households and increases potential for non-coverage bias (Keeter, Kennedy, Clark, Tompson, & Mokrzycki, 2007). Further, RDD methodology is cost-prohibitive in recruitment of tornado-affected samples because—unlike hurricanes—tornadoes generally affect a small percentage of households within a large geographic area. Recruitment at the state, county, or zip code level therefore would have resulted in a high percentage of participants with minimal disaster exposure, resulting in a lower eligible-to-screened ratio. The potential for bias also may be reduced in ABS samples, relative to RDD samples, when interruption of landline phone service occurs following a disaster (Fleeman, Henderson, Boyle, & Ruggiero, 2012). Phone numbers, especially cellphones, do not indicate precise location (Fleeman et al., 2012) and delivering the surveys in-person would be prohibitively expensive. Therefore, an ABS frame based on latitude and longitude coordinates offered the best and most cost-conscious solution.

V. Intervention: Bounce Back Now

I. Intervention

a. Development Process

Our primary focus was to develop a developmentally appropriate, disaster-focused intervention for adolescents at risk for post-disaster mental health problems. As such, the site was conceived to include four key features. First, the site needed to have the potential to address a wide range of mental health problems to ensure high relevance to the population as a whole. To achieve this aim, we concluded that several evidence-informed modules would need to be developed to provide education and recommendations addressing post-disaster mental health and health-risk areas. Moreover, each module was designed to function as a stand-alone brief intervention.

Second, we felt it was essential to emphasize behavior change over knowledge change across all of the intervention modules. Knowledge change may be an important ingredient in the behavior change process for many individuals, but acquisition of knowledge is insufficient in cases for which skill levels and motivation to change are inhibiting factors. For this reason, in addition to targeting knowledge directly, we addressed and developed content to enhance motivation levels and attempted to enhance skill acquisition where appropriate. Motivation was assessed with behaviorally specific rating scales, and we integrated motivation-enhancement content into each of the modules. For example, if an adolescent indicated low motivation to follow through with a recommendation targeting posttraumatic stress symptoms, the module would respond to this by listing potential barriers to motivation and asking the user to select relevant barriers. Barriers that are selected by the user are then addressed with educational content that aimed to enhance motivation levels by suggesting potential solutions to barriers as appropriate.

Third, the goal of tailoring Web design to an adolescent population, especially one that spans such a breadth of developmental changes as is found in a 12–17 year age range, presented many unique challenges. We sought to find a balance between the provision of engaging and interesting ways to explain and teach prevention and intervention strategies for a wide range of learning styles while simultaneously catering to short attention spans, limited vocabulary, and reading skill level. To this end, content was concrete and clear, with frequent examples, illustrations, and knowledge checks with immediate feedback and opportunities for application. Brief video clips were used to improve engagement and variety of presentation formats. Further, we minimized redundancy and maximized creativity in an effort to retain the user’s attention. Popular, trendy colors were used as well as a wide range of creative design graphics (Loranger & Nielsen, 2005). Additionally, reading skills in this age range vary widely, necessitating the use of simple, straightforward text with a streamlined vocabulary and minimal use of clauses. We also incorporated “games,” “quizzes,” and roll-over content throughout the modules to heighten engagement, interactivity, and knowledge acquisition.

Fourth, we decided to use a multi-media approach, guided by the recognition that different individuals learn differently based on the method by which educational stimuli are presented. This decision was also informed by the Web-based training literature that suggests that the use of different media (e.g., text, graphics, animation, video, audio, quizzes) can improve instruction, reduce effort required to convey a message, heighten active engagement, and focus learners’ attention (Driscoll, 2002). This is particularly important for adolescents, who have a dramatically lower level of patience than adults when attempting to execute various functions on the Internet (Loranger & Nielsen, 2005). Adolescents attend more to a Website’s visual appearance than adults; but it is also noteworthy that adolescents tend to prefer a clean, easy-to-use design over a glitzy, complicated design. Website features such as online quizzes, games, printable forms, and other interactive tasks are of particular value when attempting to engage an adolescent (Nielsen Norman Group, 2007). We also conducted usability testing of components of the intervention with adolescents (Yuen et al., 2014).

b. Description of Bounce Back Now

BBN (www.bouncebacknow.org) included content targeting adolescents at risk for post-disaster mental health problems. In order to address a wide range of mental health problems to ensure high relevance to the population as a whole, we developed several evidence-informed “modules” to provide education and recommendations addressing post-disaster mental health and health-risk areas. BBN is composed of five stand-alone modules: PTSD, Depression (Mood), Smoking, Alcohol Use, and Parenting. Module screeners were brief (i.e., generally 3–5 items relating to hallmark symptoms of constructs targeted by the intervention). Users’ patterns of screener endorsement and the study condition to which participants were assigned dictated which modules would be directly accessible. Adolescents who satisfied screening criteria for a module (i.e., subclinical or clinical symptom levels) were given a recommendation to complete that module, whereas adolescents who did not screen into the module were told that the module may not be relevant to their needs and were invited, but not required, to exit the module. For example, adolescents who endorsed several depressive symptoms were advised to complete the Mood module, whereas adolescents who denied depressive symptoms were invited to exit the module and continue to the next screener. The Alcohol Use module did not have a screener because that module was considered to be suitable as a primary prevention approach. Notably, there was a considerable level of flexibility with which participants were permitted to use the system. That is, users did not have to complete modules in any particular order, were not required to complete one module before gaining access to another, and could spend as much or as little time using each module as they preferred.

BBN used separate sets of screeners for adolescents and parents. Parents did not have access to adolescent screener responses and vice versa. Rather, adolescents completed brief screeners based on their own symptoms and risk behavior, whereas parents completed screeners based on their perceptions of their adolescent’s symptoms and risk behavior, as well as screeners regarding their own mental health and health-risk status. It therefore is possible, and may be relatively common (cf. Achenbach, McConaughy, & Howell, 1987), for parents and adolescents to respond somewhat differently on screener items relating to adolescent symptoms and behavior, and parents and adolescents in these instances would receive somewhat different educational content as a result. Once on the site, participants were able to access the individual modules. Bounce Back Now is a multi-session intervention. However, users receive the majority of the educational material during the first session. Users are then encouraged to return to the site to monitor their symptoms, indicate barriers they had experienced in carrying out the various treatment strategies, and receive education designed to assist them in overcoming those barriers.

The PTSD module was designed to provide psychoeducation as well as evidence-based recommendations focusing on exposure, reduction of avoidance of traumatic cues, coping strategies and anxiety management (Deblinger et al., 1990; Pynoos, 1993; Ruggiero et al., 2001). The Depression module featured behavioral activation strategies, which have shown promise as easily understood, efficacious, parsimonious, and cost-effective approaches in treatment of depression (Hopkoet al., 2003; Jacobson et al., 1996; Ruggiero et al., 2007). The Smoking and Alcohol Use modules made use of combined brief motivational-enhancement and cognitive-behavioral strategies that have received promising support in the literature (Dennis et al., 2004; Waldron & Kaminer, 2004). These strategies include elements that specifically address the needs of adolescent populations such as teaching skills to: (a) refuse substance abuse offers from peers; (b) establish a positive family and peer network that is supportive of the youth abstaining from use; (c) develop a plan for positive, enjoyable activities to replace substance use-related activities, and (d) cope with stressful and/or high-risk situations (Sampl & Kadden, 2001). Finally, the Parenting module provides education on parent-adolescent communication patterns, disruption in social routines, and the relation between parent and adolescent reactions to stressors. Additionally, it provides strategies for how to facilitate appropriate discussion about recent stressors with the adolescent and education about if, when, why, and how to seek help from a health-care professional.

c. Description of Assessment Only Control content

All study conditions contained identical Web screeners and assessments. Only the educational content differed across groups. Moreover, even though a family may have been randomly assigned to, for example, condition (1), the intervention was not considered to have been “prescribed” to the participant if neither the parent nor adolescent “screened positive” within a module (i.e., reported sufficient symptoms or health-risk behavior to reach sub-clinical screening thresholds). Consequently, a meaningful percentage of families randomly assigned to one of the three experimental conditions did not screen into any modules and therefore did not actually receive intervention content—these individuals experienced a condition no different from participants in the attention-control condition (Web-based symptom assessment only).

VI. Assessment

A comprehensive approach to evaluation ensures that the intervention possesses key qualities to maximize benefit to users with minimal burden in terms of time, effort, and cost. The BBN project was guided by Kirkpatrick’s (1998) conceptual model of program evaluation to allow for evaluation of outcomes on several levels: (a) reaction, (b) learning, (c) behavior, and (d) results. The first level, reaction, relates to participant satisfaction and interest. The importance of a favorable reaction to the intervention is critical, as adolescents or parents who respond negatively are likely to discontinue use and/or have little motivation to benefit from intervention content. Reaction level outcomes in the current study included website access rates, as well as questions at follow-up regarding barriers to site access among those failing to access the intervention. We also conducted a qualitative evaluation of Bounce Back Now with 29 adolescents (Yuen et al., 2014). At the second level of evaluation, learning, several aspects of this domain are considered, such as knowledge gain, skill acquisition, and attitude or motivational changes. Learning is important to assess because, if the user does not experience behavior change, it is critical to identify the level(s) at which the intervention failed. In this instance, knowledge regarding key concepts and educational information contained in the BBN modules were assessed at the outset and upon completion of each module. The third level of evaluation, behavior, focuses on the extent to which symptom change and behavior change occurs. This level also presents an opportunity to examine symptom and behavior change in relation to knowledge gain. Finally, the fourth level of evaluation, results, entails a cost-benefit analysis of the intervention, such that the outcome of the intervention at the first three levels of evaluation is considered in light of the burden of the intervention. This final level of evaluation was beyond the scope of the current project; however, initial effectiveness information is currently being assessed.

Baseline Interview

The structured baseline interview assessed demographics; disaster characteristics; history of life stressors; family environment/relationship variables; history of formal and informal help seeking; past-month cigarette, alcohol, and marijuana use; and past-month symptoms of PTSD and depression. Our decisions in the selection of substance use and mental health assessment components were guided by several factors. First, selected measures were designed to assess the psychosocial constructs targeted by the Web intervention (see Table 1). Second, each of these measures is brief and has well-supported psychometric properties. Third, the mental health measures have all been used in clinical efficacy trials, and have demonstrated satisfactory sensitivity to treatment effects. Taken together, these measures provide a relatively inexpensive, brief, psychometrically sound method to detect behavior change as a function of intervention condition. The inclusion of widely used measures potentially offers the further advantage that comparisons may be made between effect sizes from this study and those emerging from other studies.

Table 1.

Study assessments

Domain of Assessment	Assessment Measure	Assessment Time Point (Baseline, Web-Based, Follow-up)
Exposure to traumatic life events (Impact of the Tornado)	Five questions assessing the impact of the tornado (i.e., Did the tornado cause damage to your house or property?)	Baseline only
Parent-adolescent conflict behavior	Conflict Behavior Questionnaire (CBQ; Robin & Foster, 1989)	Baseline and Follow-up
Cigarette smoking, drug, and alcohol use and abuse	CRAFFT (alcohol and substance abuse)	Baseline and Follow-up
Tobacco use	Six questions assessing adolescent tobacco use (i.e., Have you ever tried cigarette smoking, even one or two puffs?)	Baseline and Follow-up
Alcohol use	Four questions assessing adolescent alcohol use (i.e., During the past 12 months, did you drink beer, wine, liquor, or any alcoholic beverages every day, some days, or not at all?)	Baseline and Follow-up
Mental Health Functioning	UCLA-PTSD Index (Pynoos, et al 1998; Rodriguez et al, 1998)	Baseline and Follow-up
Mental Health Functioning	Center for Epidemiologic Studies-Depressed Mood Scale (CES-D; Radloff, 1987)	Baseline and Follow-up
Knowledge relating to intervention content	Seven questions to assessing knowledge of intervention content (i.e., The best way for someone to decrease their anxiety is to…;)	Follow-up only
Barriers/facilitators associated with Web participation	Twelve questions assessing barriers/facilitators associated with Web participation (i.e., Can you tell me the reasons why you chose not to visit and/or complete any activities on the website?)	Follow-up only
Internet use for health	8 questions to assess adolescent online use for health related information (i.e. Have you looked online for health information?)	Baseline and Follow-up
PTSD Diagnostic	NSA PTSD Module (Kilpatrick et al., 2003; Resnick, Kilpatrick, Dansky, Saunders, & Best, 1993)	Baseline and Follow-up
Depression Diagnostic	NSA Depression Module (Kilpatrick et al., 2003; Adams, Boscarino, & Galea, 2006)	Baseline and Follow-up
General Traumatic Event Screen	Five questions assessing exposure to potentially traumatic events	Baseline and Follow-up
Family Environment	“How easy is it for you and your family to think of things to do together as a family?”; “How close do family members usually feel to each other?”	Baseline only
Module Access and Completion	Assessment of user module access, use and completion	Web-Based only
Social Support	Social Support Scale (SSAS; Dubow & Ullman, 1989)	Baseline and Follow-up

Web-phase Assessment

At completion of the baseline interview, participants were oriented to the Web phase of the study, including the purpose, consent process, and randomization procedure. Similar to the protocol used in our prior work, participants were told that the purpose was to evaluate the helpfulness of education provided via the Web about a variety of health-related concerns that tornado affected adolescents and adults may have. Following the baseline interview, Web use statistics (e.g., participant frequency of access, module qualification), pre/post knowledge data, and satisfaction levels were recorded. In addition to those administered via telephone, some brief measures were given in the context of the Web intervention for purposes of tailoring module content.

Follow-up Interviews

Four- and 12-month follow-up telephone interviews were administered to assess symptom and substance use levels, retention of knowledge gains, changes in family environment and relationship variables, and barriers to accessing the Web intervention. Interviewers remained blind to intervention condition, although at the follow-up interview it was possible that participants could volunteer information about their experience that identifies features of a condition, or that interviewers could guess the condition based on knowledge levels of the participant. To minimize this possibility, knowledge questions and barriers to participation in the Web component of the study were assessed toward the end of the follow-up interviews.

Loss to Follow-Up

When following population-based samples, there is always a concern about follow-up rates and their potential impact on results. To maximize follow-up, we employed proven retention techniques. Participants were contacted at times that are preferable for them. All efforts were made to minimize the likelihood of refusal and to re-contact individuals instead. Additionally, families were reimbursed for participation. Households that completed the baseline interview were mailed an incentive. Households also had the opportunity to receive up to $25 for completing the Bounce Back Now online intervention. Youth who chose to return a saliva sample for the genetic component of the study also were reimbursed $50. Last, four and twelve months post baseline, households received $15 for each completed follow up telephone interview.

VII. Genetic Data Collection

We received additional funding in the form of an NIH OppNet grant (PIs: Drs. Kenneth J. Ruggiero and Ananda B. Amstadter), to add a genetic component (i.e., saliva collection from the adolescents for DNA isolation and assay) to the study protocol. During the baseline interview, families were asked permission for the study team to send them information regarding other components of the study in which they could choose to participate. Following the baseline interview, families were mailed a packet that included reimbursement for their baseline interview, a letter informing them about the optional genetic component of the study, a saliva collection kit that included detailed instructions for use, and a self-addressed postage paid envelope for mailing the saliva specimen to the laboratory. Kits were coded with the unique identifier, and thus, were de-identified for purposes of confidentiality. Laboratory staff receiving the samples was blind to the phenotypic status of participants, as well as blind to the study condition that the families were assigned to. For the 780 samples obtained from youth who chose to participate in this aspect of the study, DNA extraction was conducted via Gentra Puregene kits. Using these kits, the average yield is generally 110 μg of DNA from each sample, with a failure rate of about 3%. Genotyping was then completed via the Illumina Human Exome v1.1 DNA Analysis BeadChip, which provides coverage of putative functional exonic variants from over 12,000 individual exome and whole exome-sequences.

Rationale for Inclusion of a Genetic Component

We added a genetic component to this study to inform the knowledge of biologic underpinnings of PTSD and related phenotypes in youth. Genetically informed epidemiologic studies of trauma-exposed adolescents are sorely needed; despite moderate heritability estimates (~35%) for PTSD (for review see Amstadter et al., 2009), only modest progress has been made toward identification of the molecular variation that accounts for the genetic influence on this phenotype. Notably, there are no existing genetically informed studies of adolescent PTSD. Further, the nature of the sample lends itself nicely to a genetic study of PTSD in four ways. First, cases and controls are both trauma exposed (genetically similar individuals may fail to show symptoms of PTSD simply because they have not been exposed to a traumatic event). Second, given the random nature of disaster exposure, the design inherently minimizes confounding effects of gene-environment correlation (rGE), which refers to how genetic factors influence an individual’s proclivity to some forms of traumatic event exposure, as well as risk for PTSD, resulting in possible rGE errors in causal attribution; however, the ‘chance’ nature of disaster exposure means that concerns about rGE are less pronounced providing a fairly unique opportunity to study genetic associations in PTSD. Third, given the universal nature of the exposure, and given that disaster exposure occurs at the same time for all those exposed, some confounds that are typical for PTSD research are not introduced (e.g., differing time elapsed across participants in exposure and assessment). Fourth, although disasters affect a large group of individuals simultaneously, the extent of exposure varies, and can be quantified (e.g., Amstadter et al., 2009). Therefore, it is possible to examine if the degree of exposure moderates genetic effects. For these reasons, this supplemental projectwill afford a deeper understanding of the interplay between the environment and biology in the aftermath of a traumatic stressor.

Rationale for DNA Collection Method

The collection of a sufficient amount of high quality genomic DNA is critical for any study aiming to examine the molecular genetic underpinnings of a phenotype. New advances in genomic DNA collection methods have increased ease of collection, lowered costs of collection, and have made it more feasible for researchers to conduct genetically-informed studies. Given the invasiveness of obtaining a blood sample, and the need for participants to be seen by a phlebotomist for the sample collection, researchers have been increasingly using buccal cell collection for DNA isolation (Feigelson et al., 2001). There are quite a few ways in which buccal cells can be collected, including but not limited to, cytobrushes, mouthwash procedures, and saliva collection (e.g., Oragene kits; LeMarchand et al., 2001). Direct comparison of DNA yield is difficult across studies (e.g., the participant population differs, the specific parameters differ), but generally the collection methods from lowest to highest yield are: cytobrushes/cotton swabs, mouthwash kits, then saliva collection kits (Garcia-Closas et al., 2001; Rylander-Rudqvist, et al., 2006). We chose to employ Oragene kits (a small vial into which the participant spits; when the lid of the collection vial is closed, the saliva is mixed with DNA preserving and purifying chemicals) as our collection method to maximize the DNA yield, thereby allowing for genome-wide assays, as well as “left over” isolated DNA for future investigations.

Rationale for Genotyping Platform

The Illumina Human Exome v1.1 DNA Analysis BeadChip was chosen for genotyping. The exonic content consists of >240,000 markers. About 250,000 of the SNPs were identified in a variety of exome sequencing projects, including non-synonymous (93%), splice (4.5%), stop-codon altering single nucleotide polymorphisms (SNPs) (2.5%), as well as SNPs unique to the 1000 Genomes Project or under-represented populations, which was one of the primary reasons that the Illumina platform was chosen, as the BBN sample is diverse in nature. The markers include genome wide association study (GWAS) tag SNPs, randomly chosen synonymous markers, as well as ancestry informative markers (AIM) for European Americans, African Americans, Native Americans, and Hispanics. Markers also include HLA tag, fingerprint, microRNA target site, mitochondrial and Chromosome Y SNPs, and insertion deletion polymorphisms.

VIII. Planned Statistical Methodology

Our data analytic strategy entails a five-step process. First, we assessed potential differences between Census estimates vs. sample distributions of gender, age, and racial/ethnic characteristics, using appropriate tests (e.g., t-tests, chi-square). Sample weights were calculated to account for differences between these groups. Second, descriptive statistics such as frequency counts, percentages, means, medians, standard deviations, and other statistics were used to fully characterize the sample on different variables of interest at each point in time, including disaster characteristics such as exposure, loss, and damage (See Results). Weighted contingency tables with appropriate statistical tests were constructed to test for differences in key variables across the three-group experimental design using baseline interview data.

Third, descriptive statistics were calculated to summarize basic Web-use statistics, such as how many adolescents and parents in each group accessed the Web intervention, how and the extent to which Web-access rates differed across different locations, percentage of participants that endorsed sufficient symptoms to screen into a given module, and percentage of participants that fully completed each of the modules, satisfaction data, and pre-intervention and post-intervention knowledge performance.

Fourth, generalized linear models were conducted to examine baseline variables that were indicative of participation in the Web component. Participation was divided into two categories classified by prior research: use and completion (Eysenbach, 2005). Use was defined as going to a given module whereas completion was defined as reaching the final posttest at the end of the module. Proposed predictors of use and completion included demographics, degree of disaster exposure and other disaster characteristics, traumatic event histories, mental health symptoms and health-risk behavior, and other key variables.

The fifth step is to examine the efficacy of the Web intervention modules in the three study conditions. Such an approach is necessary to determine the effect of the intervention on cases. A case is defined as an individual who demonstrated sufficient mental health risk that would be addressed by the intervention. Analyses will be conducted with an intent-to-treat sample that will include all participants who “screened in” and accessed a given module. Missing follow-up data will be examined to determine the mechanism of missingness. If the mechanism of missingness is assumed to be Missing at Random (MAR), multiple imputation will be used to create 25 complete datasets (Schafer & Graham, 2002). Variables used to determine missing cases for the complete datasets include demographics, disaster exposure, mental health symptoms, physical health symptoms, and web participation. Outcome analyses will use mixed effect models to account for the nested structure of the data (measurements nested within participants). Time will be scaled in months such that the intercept corresponds to the baseline assessment point. Cross-level interactions will evaluate differences in changes between the baseline and 4-month follow-up interviews and from the 4–month and 12–month follow-up interviews. Intraclass correlation coefficients (ICCs) will be calculated to determine the proportion of variance in outcome that is attributed to disaster region. If a significant proportion of variance (> 0.05) is identified, then region will be added as an additional level of nesting. Secondary analyses will be conducted that isolate participants in the experimental vs. comparison conditions who complete a module to examine efficacy of the intervention “as delivered.” Furthermore, moderators of treatment response will be explored that include disaster exposure, family factors, and demographic factors.

Genetic Analyses

Given the modest sample return rate and resulting implications on statistical power, we plan to conduct gene-based tests (rather than examining individual SNPs), which will serve to reduce the overall number of tests, as well as increase our ability to detect significant effects by combining sets of SNPs that may individually account for a negligible proportion of variance in the constructs of interest. We also plan to assess patterns of significant gene findings for functional gene groupings or pathways that may be of relevance for further study in relation to PTSD and related phenotypes. We intend to perform the gene level statistical analysis of typed variants using regression kernel methods (e.g. SKAT, Wu et al., 2011; EREC, Lin & Tang, 2011). We chose these approaches because they are very flexible and generalize many previously proposed methods including collapsing based tests (Li & Leal, 2008) and C-alpha (Neale et al., 2011). Even more, these methods also accommodate covariates, heterogeneity of variants’ effect, and, by aggregating individual score statistics of SNPs in a gene, they accommodate both rare and common variants.

Results

Two thousand families with adolescents (12 to 17 years of age) were recruited for this study. The overall cooperation rate for this study, calculated according to the American Association for Public Opinion Research industry standards [i.e. [number screened]/[number screened + screen-outs + unknown eligibility)] was 61%. Cooperation for the genetic component of the study also was satisfactory; 780 of the samples were returned, and of those, 763 were genotyped as some samples were not viable. Overall, the genotyping call rate was satisfactory, with only 20 chromosomal failures out of the 763 genotyped subjects (see Figure 1).

Figure 1.

Genotyping call rate

The sample contained 2,000 adolescents (M age=14.5, SD=1.7). The gender distribution of the sample was nearly equal (Boys: n=981, 49.0%; Girls: n=1019, 51.0%). Self-reported race was White (70.5%), Black (25.6%), and other (3.9%). Approximately 3% of participants reported Hispanic ethnicity. Reports from the baseline interview revealed that over 90% of participants were present in the affected area when the tornado touched down. Close to 75% of caregivers reported concern about the safety or whereabouts of family members. Physical injury was uncommon (2.9%). Nearly one-tenth of families experienced displacement, with 40% of families reporting loss or damage to residence (40%), cars (19%), other household contents (18%), sentimental possessions (10%), and pets (4%). With regard to mental health status at baseline, 6.7% of adolescents met criteria for PTSD and 7.5% met criteria for major depressive episode (MDE) since tornado exposure (see Adams et al., 2014).

Loss to Follow-Up

There were few significant (α = .05) differences between individuals who completed follow-up interviews vs. individuals who were lost. Also, when there were differences, they were generally small in magnitude. Adolescents who completed follow-up interviews were slightly older at baseline than those who were lost to follow-up (14.6 years vs. 14.4 years), t (1994) = −2.92, p=.004. Retention for White participants (60.2%) was lower than retention for Black participants (67.7%) and those with other racial backgrounds (68.1%), χ² (2, n = 1656) = 8.00, p = .018. There was some variability in retention for different household income levels, χ² (6, n = 1818) = 16.95, p = .009. However, there was not a clear linear pattern to this finding; retention was 56%–59% in 4 of 7 income groups (< $10K, $10K-$20K, $40K-$60K, $60K-$80K), but was highest in the $20K-$40K income group (69.6%). Completers and non-completers did not differ with regard to gender, prior trauma history, and tornado exposure characteristics. Importantly, there also were no differences between families who completed follow-up interviews and those lost to follow-up on any of the adolescent or parent mental health or substance-related outcomes, including PTSD, depression, cigarette use, and substance abuse.

Discussion

The goal of this work was to apply an iterative, consumer-informed process to development and evaluation of a novel web-based intervention, Bounce Back Now. The intervention was designed to address common post-disaster mental health problems among adolescents and their families. Several aspects of the Bounce Back Now project represent innovative design decisions with respect to intervention design, trial setting, and sampling methods, as well as the conceptual model guiding our assessment and subsequent analysis plan. The strengths and challenges associated with these unique components are briefly highlighted.

Bounce Back Now was designed with an intentionally broad platform intended to appeal to adolescents with a range of possible post-disaster mental health needs: PTSD, depression, alcohol and other substances, and smoking. The availability of highly accessible resources for a wide range of possible symptom presentations may be particularly advantageous given the notable prevalence of overlap and comorbidity of these symptoms among adolescents in the post-disaster context (Adams et al., 2014a; Adams et al., 2014b; Catani et al., 2008). We found that adapting content from such a wide array of evidence-based behavioral interventions necessitated a multidisciplinary team of contributors representing clinical and empirical expertise in each domain. Further, we addressed the challenge of youth fatigue or boredom in a variety of ways, including creation of screening questions that allowed the intervention to direct youth to content that would be most appropriate and relevant to their needs. The team also designed the intervention to be self-paced.

The decision to include a parental component was driven by literature supporting the importance of parental monitoring and parent-child communication, as well as the impact of parents’ own mental health functioning, in mitigating risk for adolescent mental health and substance use problems (Felix, You, Vernberg, & Canino, 2013; Ryan, Jorm, & Lubman, 2010; Yap, Pilkington, Ryan, & Jorm, 2014). To the authors’ knowledge, this is the first internet accessible intervention to target the family in the post-disaster context. Our main concern associated with inclusion of adolescent- and parent-focused content within the same site was that parents and adolescents felt secure that their responses would be confidential. We chose to address this concern by providing separate, independent secure log-in credentials to parents and adolescents. This choice was based on the assumption that parents and, particularly, adolescents may be more likely to acknowledge mental health symptoms if they are completing the intervention on their own vs. completing an intervention together with a parent. This assumption warrants further investigation. There may be advantages to interventions that are accessed by parents and adolescents together. Future research should investigate the feasibility of alternative intervention designs.

Bounce Back Now was designed for potential application in the wake of an array of natural and man-made disasters. To the best of our knowledge, it is the first intervention to be examined with tornado-affected adolescents. The nature of tornado outbreaks meant that individuals in affected areas were more likely to have been present for the disaster in comparison to disasters with more advance warning (e.g., hurricanes). They were also considerably more likely to be concerned about their whereabouts of family members during the tornado than is traditionally seen in hurricane-affected samples (e.g., Ruggiero et al., 2012). The selection of families affected by the spring 2011 tornadoes also allowed us to provide these families access to the intervention within months of the disaster (4–13 months post-tornado). This allowed us to explore the utility of Bounce Back Now at a time when the content had relevance to the population. However, the brief, low-intensity nature of the intervention is designed more specifically for the 1–6 month post-disaster period. Future research is needed to test this intervention model in a way that allows greater control over the timing of delivery. Research in pediatric emergency departments may be suitable for this purpose. Partnerships with American Red Cross, Office of the Assistant Secretary for Preparedness and Response, and other disaster response agencies also are likely to advance the field by allowing targeted and more rapid delivery of these interventions. We are in the process of developing such partnerships, and plan to test Bounce Back Now and other promising intervention approaches with disaster survivors who are accessed via shelters and social media.

The selective patterns of tornado impact and damage created a substantial challenge with regard to participant sampling in discrete geographic regions. To address this challenge while maintaining the advantages of representativeness and enhanced generalizability, the team opted to apply an address-based sampling methodology to participant recruitment. Unless and until representative registries of cell phone-only households and their address are developed, address-based sampling offers a cost-efficient means of contacting these households for study recruitment. Future research beyond the post-disaster context would likely benefit from direct comparisons of the cost, representativeness, and cooperation rates associated with recruitment through address-based sampling versus targeted internet sampling through mechanisms like Facebook, Amazon’s Mechanical Turk, or web-enabled panels, which are not favored in rigorous designs but may provide cost-efficient alternatives for purposes of piloting new protocols in preparation for larger-scale research.

The importance of conceptual models and theories in providing a framework for the assessment of outcomes and conceptualization of findings in a manner meaningful for translation into practice has been widely recognized (Glanz & Bishop, 2010; Tabak, Khoong, Chambers, & Brownson, 2012). The current project was guided by Kirkpatrick’s (1998) conceptual model and incorporated assessment of outcomes on three levels of participants’ interaction with Bounce Back Now: (1) user reactions - including satisfaction, usability, completion of content; (2) learning–assessed by knowledge check questions throughout the intervention; and (3) behavior–assessed within the intervention via return-visit check-ins, as well symptom change in follow-up telephone interviews. Recognizing that individuals’ environment and genetics may serve as important moderators of intervention impact, the current project also assessed disaster impact, demographic characteristics, and prior traumatic experiences, and also collected DNA samples from adolescents and their parents. The collection of multi-level data will allow our team to speak beyond the efficacy of the intervention itself, and to comment on factors influencing risk for post-disaster psychopathology and factors (including the intervention) that enhance resilience in the face of such circumstances. This information will be particularly helpful as we move forward to evaluation of the fourth and final level of Kirkpatrick’s model: results, or the cost-effectiveness of full-scale deployment of the intervention. It is the hope of our team that this line of research will provide the field with the necessary information for future efforts to appropriately target, disseminate, and implement the efficacious components of technology-based disaster mental health interventions to those most likely to receive benefit in the aftermath of natural and man-made disasters.