Research Methods in Child Disaster Studies: A Review of Studies Generated by the September 11, 2001, Terrorist Attacks; the 2004 Indian Ocean Tsunami; and Hurricane Katrina

Abstract

Background

A comprehensive review of the design principles and methodological approaches that have been used to make inferences from the research on disasters in children is needed.

Objective

To identify the methodological approaches used to study children’s reactions to three recent major disasters—the September 11, 2001, attacks; the 2004 Indian Ocean Tsunami; and Hurricane Katrina.

Methods

This review was guided by a systematic literature search.

Results

A total of 165 unduplicated empirical reports were generated by the search and examined for this review. This included 83 references on September 11, 29 on the 2004 Tsunami, and 53 on Hurricane Katrina.

Conclusions

A diversity of methods has been brought to bear in understanding children’s reactions to disasters. While cross-sectional studies predominate, pre-event data for some investigations emerged from archival data and data from studies examining non-disaster topics. The nature and extent of the influence of risk and protective variables beyond disaster exposure are not fully understood due, in part, to limitations in the study designs used in the extant research. Advancing an understanding of the roles of exposure and various individual, family, and social factors depends upon the extent to which measures and assessment techniques are valid and reliable, as well as on data sources and data collection designs. Comprehensive assessments that extend beyond questionnaires and checklists to include interviews and cognitive and biological measures to elucidate the negative and positive effects of disasters on children also may improve the knowledge base.

Introduction

A sizeable body of research has revealed that for a substantial portion of youth, exposure to a disaster can have deleterious consequences for their physical and mental health (e.g., Becker-Blease et al. 2010; Fremont 2004; Joshi and O’Donnell 2003; Norris et al. 2002; Silverman and La Greca 2002). Effects are not limited to stress responses but involve a host of biological, psychological, behavioral, cognitive, and social consequences. The level of negative impact appears to be a function of the extent and type of exposure as well as a number of mediating and moderating factors. Circumscribed by three recent disasters—the September 11, 2001, attacks; the 2004 Indian Ocean Tsunami (referred to here as the 2004 Tsunami); and Hurricane Katrina—this review describes fundamental components of research investigations, design principles, and methodological approaches as a framework for making valid inferences from the research on disasters and children. We chose to review and compare studies on these three disasters because they were major events, they occurred recently but there has been sufficient time for publication of research findings, they provide diverse contexts of disaster experiences with both human-caused and natural etiology and occurred within this country and internationally, and they have generated considerable research to enhance our understanding of disaster effects. This report also identifies methodological advances that might improve the next generation of research to further knowledge development and ultimately aid disaster research policy and planning. Table 1 presents summary information on methodological issues for studies on each of the three disasters.

Table 1.

Research design features of child disaster mental health research studies

	September 11	2004 Tsunami	Hurricane Katrina	Total
	n = 68 (46.9 %)	n = 28 (19.3 %)	n = 49 (33.8 %)	n = 145 (100 %)
Study design
Cross-sectional	41 (60.3)	17 (60.7)	24 (49.0)	82 (56.6)
Longitudinal	8 (11.8)	7 (25.0)	10 (20.4)	25 (17.2)
Before-and-after	15 (22.1)	0	6 (12.2)	21 (14.5)
Intervention	4 (5.9)	4 (14.3)	9 (18.4)	17 (11.7)
Sampling method
Convenience/purposive	51 (75.0)	21 (75.0)	42 (85.7)	114 (78.6)
Random	17 (25.0)	7 (25.0)	5 (10.2)	29 (20.0)
Census	0	0	2 (4.1)	2 (1.4)
Sample size
< 100	20 (29.4)	7 (25.0)	17 (34.7)	44 (30.3)
100–500	28 (41.0)	15 (53.6)	25 (51.0)	68 (46.9)
501–1000	11 (16.2)	3 (10.7)	4 (8.2)	18 (12.4)
> 1000	7 (10.3)	3 (10.7)	3 (6.1)	13 (9.0)
Not reported	2 (2.9)	0	0	2 (1.4)
Timing of first assessment after disaster
0–6 months	40 (58.8)	12 (42.9)	16 (32.7)	68 (46.9)
7–12 months	9 (13.2)	9 (32.1)	14 (28.6)	32 (22.1)
13–18 months	3 (4.4)	1 (3.6)	7 (14.3)	11 (7.6)
19–36 months	7 (10.3)	2 (7.1)	8 (16.3)	17 (11.7)
> 36 months	2 (2.9)	1 (3.6)	2 (4.1)	5 (3.5)
Not reported	7 (10.3)	3 (10.7)	2 (4.1)	12 (8.3)
Informant
Child	59 (72.1)	27 (96.4)	42 (85.7)	118 (81.4)
Parent/caregiver	37 (54.4)	9 (32.1)	19 (38.8)	65 (44.8)
Teacher	6 (8.8)	1 (3.6)	1 (2.0)	8 (5.5)
Others	3 (4.4)	1 (3.6)	2 (4.1)	6 (4.1)
Multiple informants	24 (35.3)	8 (28.6)	13 (26.5)	45 (31.0)

Nature of the Review and Overview of the Studies

This review was guided by a systematic search of the following bibliographic databases: PubMed, Medline, PsycINFO, All EBM Review, EMBASE, ERIC, Social Work Abstracts, and PILOTS. The initial search was conducted in September 2011, followed by an updated search conducted in December 2012. In addition to the name of each specific disaster, other key word terms focused the search to address mental, emotional, and behavioral issues in children, adolescents, and families. The search was confined to English language sources. Titles and abstracts were reviewed to identify publications describing scientific studies. Throughout the search process, the reference sections of publications and review articles, as well as literature known to the authors that was not generated by the search, were used to locate additional materials.

The search generated a total of 83 unduplicated empirical reports on September 11, 29 on the 2004 Tsunami, and 53 on Hurricane Katrina resulting in a total of 165 empirical reports covering the three events. Most reports addressed unique data sets, but some data sets were used in multiple reports on the same children or were studies using similar data sets in which the sample size changed over continued data collection or across multiple waves in longitudinal studies. For example, Hensley and Varela (2008) published a cross-sectional study examining children 5–8 months after their exposure to Hurricane Katrina; a second report using the same data and data collected 17–18 months post incident constituted a longitudinal follow-up (Hensley-Maloney and Varela 2009). Among 30 publications using ten different research datasets, only the ten index studies were included in the computation of frequencies and proportions for presentation in Table 1. For example, among the papers describing the results of the assessment of the psychological impact of the September 11 attacks on New York City public school students (e.g. Applied Research and Consulting LLC 2002; Comer et al. 2010; Duarte et al. 2006, 2011; Hoven et al. 2004, 2005, 2006, 2009; Paasivirta et al. 2010; Rosen and Cohen 2010; Wu et al. 2006), only the 2004 study by Hoven et al. was used to calculate frequencies and proportions in Table 1. Reports presenting the baseline data of a longitudinal study and those reporting findings in the follow-up assessments for the same sample were all included. Thus, of the 165 studies comprising the database, a total of 145 field investigations were used to examine methodological issues with 68 on September 11, 28 on the 2004 Tsunami, and 49 on Hurricane Katrina. Research design features of these 145 investigations are summarized in Table 1.

Research Methodology

Decades of research have helped to explain the connection between independent disaster-related variables (e.g., exposure) and a multitude of emotional, behavioral, cognitive, and biological outcomes, while generating relatively little in the way of explanation or prediction. Many child disaster mental health studies have been guided by an exploratory approach, supplying a foundation for further formulation of a more quantitative, theory-driven, systematic approach to the examination of variables and their interrelationships. Hypothesis-generating research allows investigators to establish the existence of associative or possible causal relationships and to explore novel issues, such as the impact of the sociopolitical climate in which a disaster occurs.

With post-disaster environments in particular, the diversity inherent in the populations affected and in the events themselves creates challenges to the selection of research questions and obstacles with respect to methodology. The very nature of disasters generally precludes true experimental research, and the opportunity to infer causality diminishes as designs deviate further from experimental conditions. Likewise, the ability to manipulate critical variables (e.g., exposure to trauma) is limited as is control of potential intervening variables. As a result, post hoc statistical control (as opposed to experimental control) often is utilized, whereby researchers address possible confounds (e.g., stressful life events) and identify intervening variables to examine linkages between independent (e.g., exposure) and dependent variables (e.g., posttraumatic stress).

Research Approaches

Methodologists generally distinguish between two broad types of research approaches, namely, those that are experimental and allow causal conclusions and those that are quasi-experimental or correlational in nature (Kazdin 2003; Rosenthal and Rosnow 2008). True experiments represent the ideal combination of manipulation, control, and randomization, while quasi-experiments involve conditions that approximate experiments. The only true experimental studies in this review were intervention studies that randomly assigned participants to intervention or control conditions. Loeber and Farrington (1994) have argued that experimental interventions included in longitudinal studies of childhood psychopathology can reveal variables that buffer certain risk factors.

A total of 17 (11.7 %) intervention studies were reviewed for this report with four using September 11 samples (Brown et al. 2004, 2006; CATS Consortium 2010; Loumeau-May 2008), four using 2004 Tsunami samples (Berger and Gelkopf 2009; Catani et al. 2009; Leitch 2007; Vijayakumar et al. 2006), and nine using Hurricane Katrina samples (Cain et al. 2010; Cohen et al. 2009; Jaycox et al. 2010; Plummer et al. 2009; Salloum and Overstreet 2008, 2012; Scheeringa et al. 2011; Taylor and Weems 2011; Weems et al. 2009). Research on Hurricane Katrina had the highest percentage of studies (18.4 %) evaluating interventions which was significantly greater than the percentage of September 11 studies (5.8 %) (Fisher’s exact test: p = .04). Randomized control design contributes the strongest evidence for intervention studies. Random assignment to intervention or control conditions was used in nine intervention studies (Berger and Gelkopf 2009; Catani et al. 2009; Cohen et al. 2009; Jaycox et al. 2010; Salloum and Overstreet 2008, 2012; Scheeringa et al. 2011; Taylor and Weems 2011; Weems et al. 2009); two controlled trials did not randomly assign the participants to the intervention arms (Brown et al. 2006; Vijayakumar et al. 2006). While uncontrolled trials provide weaker evidence than studies using true experimental design, they often are used as a first step in investigating the effectiveness of interventions (U.S. Food and Drug Administration 1988; White and Ernst 2001). Six intervention studies in this review used a quasi-experimental design with no control group (Brown et al. 2004; Cain et al. 2010; CATS Consortium Assessment 2010; Leitch 2007; Loumeau-May 2008; Plummer et al. 2009). See Table 1.

Exemplifying a longitudinal approach with experimental design and highlighting the potential use of intervention studies to answer theoretical questions about disaster exposure, Weems et al. (2009) used both experimental and quasi-experimental components in their investigation of New Orleans children exposed to Hurricane Katrina. The study utilized a group randomized intervention design in which individuals were not randomly assigned to groups, but groups were randomly assigned to conditions (immediate treatment versus waitlist). This design provided a waitlist control group to evaluate experimentally the effect of the intervention on test anxiety symptoms. Researchers followed the entire sample to see if changes in test anxiety predicted changes in posttraumatic stress disorder (PTSD) symptoms. Benefits of the design were that all participants could receive the intervention, treatment effects could be rigorously evaluated, and quasi-experimental inferences could be made about other variables of interest.

Quantitative Versus Qualitative Studies

Arguably, every phenomenon in existence is capable of quantification, but qualitative information can be indispensable in certain circumstances if, for example, quantification is not viable or provides insufficient richness to describe the circumstances. In disaster research, the inclusion of qualitative data enables participants to provide detailed answers, contributing to an overall understanding of the effects of disasters. Following the September 11 attacks, Beauchesne et al. (2002) studied Boston children and their parents to examine their recollection of how they learned about the attacks and their emotional reactions. The qualitative results of this study indicated a variety of emotional reactions among children residing in a city from which two of the planes involved in the attacks departed, including fear about safety and the future and feelings of revenge, sadness, disappointment, and empathy. Some studies have used both quantitative and qualitative design (e.g., Burnham et al. 2008; Leitch 2007; Noppe et al. 2006; Salloum and Lewis 2010). For example, in a Hurricane Katrina study, children in New Orleans were asked open–ended questions about their coping strategies, and they also completed an eight-item exposure questionnaire (Salloum and Lewis 2010).

Focus groups can be used to obtain qualitative data through interactions among participants. The number of participants must be large enough to stimulate a group dynamic while small enough to engage all participants. Limitations associated with the focus group design include bias caused by social desirability, dynamics of any particular group, and the potential influence of the facilitator. A number of studies from the three disasters employed focus groups consisting of children or adolescents (e.g., Pfefferbaum et al. 2006a, 2007), parents (e.g., Hestyanti 2006; Klein et al. 2009), and social workers (e.g., Hestyanti 2006) to gather qualitative data.

Assessment Design

In addition to the two broad types of research approaches (experimental versus non- or quasi-experimental), other study design considerations, including the number and timing of assessments, are important to inferences that can be drawn. The timing of a disaster study depends on the research questions to be addressed and is influenced by the chaotic nature of disasters themselves, the availability of resources, and access to participants. The period for collecting data on children’s disaster reactions extends from pre disaster to the weeks, months, and years that follow. Across the three disasters, most studies were conducted within the first 12 months after the disaster, perhaps reflecting the concern that was raised by these mass events. Decreasingly fewer studies were conducted over the succeeding years such that there is a dearth of information about the long-term effects of, and recovery from, disasters in children. See Table 1.

Table 1 presents summary information on the number of studies from each disaster that used cross-sectional (involving only one assessment), longitudinal (involving multiple post-event, follow-up assessments), and before-and-after (involving pre-disaster assessment) assessment designs. Cross-sectional design was used in most of the reviewed studies (n = 82, 56.6 %). With cross-sectional design, data are collected at the same time, or within a short time frame, from all participants and represent findings on a cross-section of the population at that particular point in time. This design can generate hypotheses for future research but does not allow the attribution of cause and effect. Cross-sectional data are useful for many purposes including the advancement of public health and clinical goals through, for example, needs assessments and screening (Pfefferbaum et al. 2012a). In large-scale disasters, screening all members of an affected community to identify those with severe post-trauma reactions may not be feasible. Instead, targeting specific subgroups known to be at high risk for these outcomes is a more efficient method to identify individuals who may need services. Rosen and Cohen (2010) performed a secondary analysis of data from a sample of New York City public school children in Grades 4 through 12 screened for PTSD after the September 11 attacks. They found that fourth grade students and those who reported that a friend or family member was directly exposed to the attacks had a greater likelihood of screening positive for probable PTSD compared to other children or adolescents. In the context of limited resources, similar methods can be implemented to identify subgroups most in need of services.

In contrast to cross-sectional studies, longitudinal studies entailing multiple assessments across time allow the researcher to measure change and to draw inferences about the temporal sequence of identified variables (North and Pfefferbaum 2002). Statistical comparisons are made among the independent variables between or across groups (e.g., those with high and low levels of exposure) or within groups at different time points for the same sample (Kazdin 2003). In their 2002 analysis of the disaster mental health literature, Norris et al. found that two-thirds of the studies they reviewed used a single assessment (Norris et al. 2002). Thus, as indicated in Table 1, it is notable that a number of studies conducted since then have used longitudinal design to study children after the September 11 attacks (n = 8, 11.8 %), the 2004 Tsunami (n = 7, 25.0 %), and Hurricane Katrina (n = 10, 20.4 %).

Kessler et al. (2008, p. S12) used the term “panel” to describe a within-groups approach involving respondents assessed repeatedly over time. For example, in their September 11 World Trade Center studies, Pfeffer et al. (2007) assessed participants every 6 months over a 2-year period; Hafstad et al. (2010) assessed respondents three times after the 2004 Tsunami at 6, 10, and 29 months post incident; and Weems et al. (2010) assessed a Hurricane Katrina sample twice at 24 and 30 months post hurricane. Another approach involving between-groups analysis uses a new sample of respondents in each phase (“trend” design) (Kessler et al. 2008, p. S12). For example, Madkour et al. (2011) used the biennial New Orleans Youth Risk Behavior Survey data for 2003 (pre Katrina), 2005 (pre Katrina), and 2007 (post Katrina) to assess the prevalence of violence behavior in New Orleans high-school students.

For longitudinal studies, Kessler et al. (2008) underscored the importance of choosing the appropriate frequency and time interval between assessments to capture participants’ reactions. Frequent assessments may not be feasible, but if the time between assessments is lengthy, the investigator may miss reactions that have resolved before an assessment. The time interval referenced in an assessment can be chosen to address this concern by using continuous time tracking in which participants are asked to reference the entire time interval since the last assessment rather than the specific time point when data are collected (referred to as a snapshot design). Mixed designs involving both continuous tracking and snapshot also may be used (Kessler et al. 2008). To avoid problems with memory and to enhance the accuracy of data, Kessler et al. (2008) recommended short time intervals between assessments (1–6 months) to capture information that reflects the entire post-disaster period that could have been missed if information was collected for specific time points only. In this review, six of the eight longitudinal studies on the September 11 attacks (Garrett et al. 2007; Gil-Rivas et al. 2007; Holmes et al. 2007; Malin and Fowers 2004; Pfeffer et al. 2007; Stein et al. 2004) and three of the seven longitudinal 2004 Tsunami studies (Hafstad et al. 2010; John et al. 2007; Piyasil et al. 2007) used short time intervals between assessments (6 months or less). Average time intervals were longer (more than 6 months) in all of the longitudinal studies assessing Hurricane Katrina. This issue should be considered in future research.

Baseline (pre-disaster) information can be used in longitudinal studies to discern the unique effects of the disaster and other risk factors to predict subsequent onset of adverse outcomes. Otherwise, pre-existing risk (as opposed to disaster exposure) cannot be ruled out as an alternative explanation of findings. As such, before-and-after disaster studies provide one of the strongest non-experimental designs available to disaster researchers. Unfortunately, because of the unpredictability of disasters, planned research assessments made prior to the disaster are rare. As a result, investigators may take advantage of ongoing research to conduct a post-disaster assessment. Studies examining children after the September 11 attacks (n = 15, 22.1 %) and after Hurricane Katrina (n = 6, 12.2 %) used this approach. For example, a sample of adolescents who, as children in 1996, had participated in a study of a large school-based violence prevention program was used after September 11 to explore adolescent mental health symptoms, social attitudes, different types of exposure to the attacks, and exposure to previous trauma (Aber et al. 2004). Weems et al. (2007) used measures of trait anxiety and negative affect, collected 17 months prior to Hurricane Katrina, to predict symptoms of PTSD, generalized anxiety, and depression post Katrina while controlling for pre-Katrina PTSD symptoms.

Populations of Interest and Sampling

Random sampling is essential for making inferences from samples to populations; however, in the disaster context, using random samples not only is impractical but perhaps irrelevant for the goals of disaster research. Indeed, the study purpose in disaster research is more often the assessment of the relationship between dependent and independent variables, and not generalization of the results to a target population. Thus, the researcher is more concerned about replicability of results (i.e., whether other investigators implementing the same research protocol would observe similar findings) (Guttman 1977; Makel et al. 2012; Miller and Schwarz 2011). If a representative sample is not used, inferences cannot be made about the population from which the study participants were selected; rather, inferences should be limited to the study sample. Nonetheless, researchers should strive to avoid selection biases (e.g., self-selection bias). The selection of study participants should be independent of the variables of interest if the sampling process is to be disregarded in the analysis of data (Smith 1983). Given the nature of disasters and the questions often posed in disaster research, in many instances, determining the relationship among variables of interest in the study sample may be preferable to making inference to a population. Unfortunately, at present there is no reliable statistical method to estimate replicability and some question if one can be found (e.g., Miller and Schwarz 2011).

The choice of a population for a disaster study may be based on a number of factors. For example, researchers may select a study population based on some specific characteristic such as demographics (e.g., age, ethnicity), cultural heritage, geographic location, type and severity of disaster exposure, or source of participants (e.g., schools, daycare settings) (North and Norris 2006). Researchers seeking to assess directly-affected children, who constitute a high-risk group, may have a particularly difficult time accessing participants. Important issues associated with sampling from a population of children affected by disaster include the types of samples and sampling approaches as well as access to appropriate populations, recruitment strategies, and participation rates.

Sampling Methods

The sampling methodology affects the internal as well as the external validity (i.e., the generalizability of a study’s findings) of a disaster research study. Ideally, the sampling process yields a sample that is representative of the population, but this is not always feasible. Although utilizing a representative sample may provide more precise data about the occurrence and distribution of various outcomes, the choice of child disaster samples often is based on accessibility (Norris 2006). Norris (2006) identified five sampling strategies in disaster research—convenience, purposive, clinical, random, and census sampling. Clinical samples can be obtained with convenience, purposive, and random sampling. In the review conducted by Norris (2006), convenience samples were the most common, followed by census, random, purposive, and clinical samples. Summary information on sampling strategies in the studies reviewed for this report is presented in Table 1.

Convenience and Purposive Sampling

Convenience samples typically include participants who are selected because of their proximity and relatively easy accessibility without regard to how representative they are of the population of interest. Purposive samples include individuals exhibiting certain characteristics or individuals chosen as “information-rich cases” for exploring specific issues (Patton 2002, p. 243). The distinction between these two sampling methods may be academic, however, as the sample itself may have been conveniently accessed but intentionally selected by the investigator who, for example, examined samples of children chosen because of their specific exposure experiences. Pfeffer et al. (2007) selected a sample of children who lost a parent in the September 11 attacks through schools, police and fire departments, and organizations for bereaved children and a comparison group of non-bereaved children through schools. The bereaved children were compared with non-bereaved controls, recruited from local schools as children whose families lived or worked at or near the World Trade Center. Results revealed that the bereaved children were more likely to have anxiety disorders (including PTSD) and higher morning and afternoon baseline cortisol levels, indicating their increased risk for limbic–hypothalamic–pituitary–adrenal axis dysfunction and psychiatric symptomatology.

Random Sampling

Random samples consist of participants who are representative of a population under investigation and thus allow generalizations from the sample to that population. One-fifth (n = 29, 20.0 %) of the studies in this review used random samples. For example, Hoven et al. (2005) studied the reactions of children to the September 11 attacks by using a stratified sample (ground zero, high-risk area, and all other areas) of all schools in the New York City public school district. Substantiated by their sampling method, these researchers concluded that nearly 30 % of all public school children in New York City had one or more probable mental disorders following the disaster, regardless of their proximity to the attacks. Additionally, sampling from groups with different exposures permitted hypothesis testing about the effects of exposure. The study found that greater exposure, either through their personal experiences or through the experiences of a directly-exposed family member, was associated with higher prevalence of anxiety disorders (including PTSD) and depression.

Census Sampling

In some studies, when the purpose is to obtain information from all affected children, a census is used rather than sampling (Norris 2006). It may be feasible to assess a census in disasters that affect a confined geographic region with a relatively small number of survivors, but in mass disasters that affect large numbers of individuals and/or individuals across wide geographic areas, such as the disasters reviewed for this report, the census approach may be impossible. Few studies in this review used the census method, and they limited the sample to a specific subpopulation and/or area of study (e.g., Osofsky et al. 2009; Ward et al. 2008). For example, Osofsky et al. (2009) examined all 4th through 12th grade children in schools in four heavily-affected parishes in the New Orleans area after Hurricane Katrina. Ward et al. (2008) used Mississippi Department of Education school record data to compare students who were displaced from other Mississippi public schools, students who were displaced from schools other than Mississippi public schools, and non-displaced students for Grades pre-kindergarten through 12 in the state of Mississippi during the 2005–2006 school year.

Clinical Sampling

Few studies in this review used clinical samples (e.g., Olteanu et al. 2011; Otto et al. 2007; Rath et al. 2007; Wiguna et al. 2010) which may be drawn from clinical facilities. For example, Rath et al. (2007) studied individuals between birth and 24 years of age who consulted at health care facilities in New Orleans after Hurricane Katrina. Wiguna et al. (2010) enrolled children who presented for care in the aftermath of the 2004 Tsunami and included as subjects those for whom records were complete (including demographic data, answers to the Strengths and Difficulties Questionnaire, and psychiatric diagnoses). Employing a different approach, Otto et al. (2007) used a clinical sample of adults in treatment for panic disorder or major depression (with no history of agoraphobia or panic disorder) to assess the psychological impact of the September 11 attacks on their children who were considered at risk for anxiety disorders. Hence, the parents, rather than the children, were sampled from the clinical settings. Children of parents not treated for these conditions were used as controls.

Sampling Approaches for Distant Samples

Researchers have examined the remarkable losses caused by the September 11 attacks, the 2004 Tsunami, and Hurricane Katrina at a local level and have investigated the magnitude of the disasters’ effects on a national and global scale. Seeking information about widespread reactions to these disasters has challenged the field to identify ways of sampling distant populations. Irrespective of the sampling strategy employed, data from geographically distant samples may be obtained through a variety of approaches. For example, Henry et al. (2004) conducted in-home interviews with children and their parents from economically-disadvantaged neighborhoods in Chicago before and after the September 11 attacks. Due to the limited feasibility of conducting personal interviews nationwide, telephone and web-based surveys provided an alternative avenue for conducting research with children and their families. For example, Schuster et al. (2001) collected data from parents using random-digit dialing, while statistically controlling for any disparities between the composition of their obtained sample and the U.S. population. In another September 11 study, Schlenger et al. (2002) conducted a web-based survey using stratified random sampling to query a nationally representative sample of adults about their children. Gil-Rivas et al. (2007) also used a web-based survey to assess a nationally representative cross-sectional sample of adolescents and adults regarding their reactions to the attacks. These methods, enhanced by technological advances, allow for the consideration of samples that would not otherwise be reached through traditional means.

Variables of Interest

On the most fundamental level, disaster research provides descriptive information about the effects of disasters on children. Data gleaned from the past 30 years have revealed an abundance of individual, family, and social factors potentially linked to children’s adjustment following disaster. Unfortunately, because of the barriers impeding manipulation, control, and randomization, cause–effect relationships are not always revealed in disaster research. In most disaster studies, independent variables may be best described as correlates or covariates or, to a lesser extent, predictors, of dependent or criterion variables. Predictor variables may be used when the researcher is interested in the degree to which one variable predicts or helps to explain the behavior of another variable. The term “correlate” applies to an association between two or more variables, without any clear indication of one variable preceding the other (Kazdin 2003, p. 117). A variable that precedes and increases the likelihood of a negative outcome is referred to as a risk factor (Kazdin 2003); a variable is a protective factor when it decreases the likelihood of that negative outcome.

Often, variables that describe characteristics of the individual participants in a study serve as correlates and determine the grouping of participants for analysis. For example, utilizing types of exposure (e.g., geographic proximity, interpersonal) to the September 11 terrorist attacks on the World Trade Center as risk factors, Hoven et al. (2005) found that family exposure to the attacks correlated with probable mental disorder above and beyond the effects of children’s direct exposure. Correlates, predictors, and outcomes almost always reflect children’s own inherent characteristics and experience (e.g., demographics, disposition, psychological functioning), family factors (e.g., parental support, parental stress), and social factors (e.g., socioeconomic status, community support) associated with their pre-, peri-, and post-disaster environments. Many of these variables are well-known and studied in numerous contexts and for conditions not specific to disasters.

Importantly, before attempting to reveal the nature of relationships among variables, researchers must define the variables of interest. To test concepts adequately and allow for replication, variables are operationalized, or defined by the specific operations or assessments utilized in a study (Kazdin 2003). For example, exposure is a well-acknowledged predictor variable in disaster outcomes but, as further explicated below, exposure can be operationalized in a number of ways including, for example, geographic proximity and interpersonal relationship to disaster victims.

Intermediary Processes: Moderators and Mediators

As disaster research has developed, methodological approaches and measurement techniques have become more sophisticated, permitting researchers to address a broader range of interrelationships among variables. For negative disaster outcomes, the effects of multiple risk factors may be cumulative in that the presence of more risk factors presumably is related to a greater probability of negative outcome. However, because not all children, even those who might be considered at high-risk, exhibit a negative disaster reaction, it is important to examine factors that serve to ameliorate or modify the effects of risk factors. Resilience may be used to describe a positive outcome for an individual despite the presence of risk factors.

Protective factors or mechanisms are the specific attributes that promote resilience among individuals at high risk. Risk factors interact with protective factors to potentiate the negative outcome. In both cases, the result is called a moderating effect. Barron and Kenny (1986) defined a moderator variable as one that affects the strength of the relationship between the predictor and criterion variables. Moderator variables may be important for prevention and intervention efforts (La Greca et al. 2002). For example, Wilson et al. (2010) found that pre-September 11 parenting and child temperament interacted to predict children’s posttraumatic responses. While high negative emotion created risk for elevated posttraumatic stress outcomes for all children, maternal acceptance appeared to be protective for those whose temperament was characterized by low negative emotionality.

Promotive factors are those variables associated with more positive outcomes, regardless of a child’s level of risk (Masten 2007). Gil-Rivas et al. (2007) found that adolescents whose parents recommended particular coping strategies, such as positive reframing, acceptance, and emotional expression, reported less distress in response to the September 11 terrorist attacks. Reducing parent–child separation also is linked to better child outcomes (Weems and Overstreet 2008). Despite the disruption for the individuals, groups, and communities encompassing and protecting children, systemic support is important for promoting positive outcomes for children in the aftermath of disasters.

A moderator variable affects the strength of the relationship between two other variables, while a mediator explains or accounts for the observed relationship between them (Barron and Kenny 1986). For example, in their study of youth exposed to long-term armed conflict in Sri Lanka, Fernando et al. (2010) found support for their hypothesis that disaster-related daily stressors at least partly explained (i.e., mediated) the relationship between exposure to the 2004 Tsunami and disaster-related psychological symptoms. They suggested that failure to include daily stressors in explanatory models of disaster outcomes was likely to ignore important potential sources of variance and might exaggerate the predictive power of exposure.

The Role of Exposure as a Predictor

Central to the disaster experience and requisite for a diagnosis of PTSD, exposure is a key consideration in disaster research. As the field of disaster mental health has matured, researchers have grown to appreciate the importance of assessing various qualitative aspects of the child’s experience. Exposure has been addressed in two ways in the research. As discussed above, some investigators have dealt with exposure through purposive sample selection by enrolling children with specific exposure characteristics (e.g., bereaved children, children of first responders) as participants for study. Other investigators have used types or aspects of exposure as predictors to characterize and compare groups of children or to quantify exposure.

Researchers have drawn samples of children to examine the effects of certain specific aspects of exposure that create risk for children. For example, in their study of PTSD in children exposed to the 2004 Tsunami, Piyasil et al. (2007) compared children who survived direct hit by the Tsunami with those who were not in the waves themselves but were exposed to survivors. The availability of pre-event data determined the exposure categories examined by Ward et al. (2008) in their investigation of displaced students after Hurricane Katrina. Ultimately, these investigators compared students displaced from Mississippi public schools with non-displaced students, eliminating a group of children displaced from schools other than Mississippi public schools for whom they could not obtain pre-event data. Finding lower rates of school attendance and performance and higher rates of behavioral difficulties and school dropout in displaced students from other Mississippi public schools compared to non-displaced students, the authors acknowledged that these differences may have been associated with lower socio-economic status of non-displaced students compared to displaced students from Mississippi public schools.

Other researchers have used distinct experiences to quantify exposure. For example, studying children in New York City schools after September 11, Hoven et al. (2005) identified three forms of exposure including attendance in a school in the ground-zero area, direct exposure defined through a variety of personal encounters with the event, and family exposure defined as family member killed or injured in the attacks or witnessing the attacks and escaping unharmed. They then combined these exposures to distinguish levels of exposure, with severe exposure defined as two or more direct and/or one or more family exposures; moderate exposure defined as one direct and no family exposure; and mild exposure defined as neither direct nor family exposure. Media exposure was considered “high” if the child spent “a lot of time” watching event-related television coverage (p. 547). In a separate analysis of these data, Duarte et al. (2006) categorized children in first responder families according to the first responders’ professions (e.g., police officers, firefighters, emergency medical technicians) and those with no first responder in the family to study the prevalence of probable PTSD using hierarchical logistic regression techniques, controlling for exposures and demographics.

Categories of Exposure

While there is no universally-accepted categorization of exposures, various experiences have been described. Direct exposure occurs through physical presence, and interpersonal exposure derives from relationship with individuals who directly experienced the disaster. Indirect and distant trauma have been of interest, especially with mass disasters that create national threat, such as the September 11 attacks, and those that are widely covered in the media, such as Hurricane Katrina. These forms of exposure can be used in application to children residing in the disaster community who were not themselves directly exposed (indirect trauma) and those at greater distances outside the disaster community who were remotely affected (distant trauma). Children’s exposure may derive from contact with media coverage, often measured as the amount of time the child was exposed to coverage and/or the content of the coverage the child consumed. Media coverage also may constitute an additional conduit of connection with the disaster in children with other forms of exposure.

Direct Exposure Through Physical Presence

Direct exposure for those physically present at, or geographically near, the disaster site often is measured as proximity (e.g., physical distance from the disaster site), through personal experiences (e.g., suffered injury), through sensory experiences (e.g., what the child saw, heard, felt, or smelled), and/or by the child’s subjective experiences (e.g., perceived life threat). While researchers may have difficulty recruiting directly exposed children, the sheer magnitude of the three disasters reviewed in this analysis unfortunately resulted in direct exposures for substantial numbers of children. The September 11 sample of New York City public school children assessed by Hoven et al. (2005) included numerous children from ground zero schools and children who were exposed through personally witnessing the attacks, were hurt in the attacks, were in or near a cloud of dust and smoke, were evacuated to safety, and/or were extremely worried about the safety of a loved one.

Studies of the 2004 Tsunami examined children from rural communities in southern India who were being sheltered in rehabilitation camps or hostels or who were living with relatives (John et al. 2007); directly-exposed children from the Andaman and Nicobar Islands of India in the Bay of Bengal (Math et al. 2008); children in provisional refugee camps located in northern Sri Lanka (Catani et al. 2009); school children in Thailand, many of whom had been directly hit by the waves and were rescued from the flood (Piyasil et al. 2007); and Norwegian children, many of whom had experienced physical danger and/or had witnessed the injuries of others while in Tsunami-affected areas (Nygaard et al. 2010). Many children also were directly exposed to Hurricane Katrina. For example, the sample studied by Kilmer and Gil-Rivas (2010) included children who were exposed through personal injury; by being in buildings that were damaged; or by being in, or evacuated from, a community that was flooded.

Subjective Experiences

Perceived life threat and/or other cognitive appraisals, subjective distress, and emotional reactions at the time of the disaster often are included in measures of disaster exposure (e.g., Gould et al. 2004; Hafstad et al. 2010, 2011; Holmes et al. 2007; Jensen et al. 2009; Pfefferbaum et al. 2006a). In a meta-analysis of posttraumatic stress in youth exposed to disasters, perceived threat and distress at the time of the disaster were associated with increased posttraumatic stress (Furr et al. 2010). Gould et al. (2004) studied peri-event numbing reactions as well as interpersonal exposure in students in New York state high schools 4 months after the September 11 attacks, and Pfefferbaum et al. (2006b) studied the association of peri-event panic attacks with subsequent probable PTSD and probable sub-threshold PTSD in adolescents residing in the New York City Metropolitan area 6–9 months after the attacks. Studying perceived life threat and intrusive imagery in London school children who had viewed televised coverage of the September 11 attacks, Holmes et al. (2007) found an association of these variables with persistence of PTSD symptoms at 2 and 6 months after the event.

Secondary Adversities

Children also are susceptible to secondary adversities associated with disasters (e.g., disruption, economic loss, displacement), which may constitute an aspect of exposure. For example, Blaze and Shwalb (2009) found that both general psychological distress and posttraumatic stress were related to resource loss in displaced high school students from southeastern Louisiana after Hurricane Katrina. Seeking to examine secondary stressors associated with Hurricane Katrina, Overstreet et al. (2010) assessed an ethnically and socioeconomically diverse sample of adolescents using eight items to measure current hurricane-related disruptions including ongoing damage to their home, continued parental unemployment, attending a new school, and disruptions in family and peer networks. The results revealed that secondary stressors were related to overall PTSD symptoms and to each of the three PTSD symptom clusters (Overstreet et al. 2010).

Jensen et al. (2009) examined the effects of the 2004 Tsunami in a setting in which secondary adversities were thought to be minimal. These investigators assessed Norwegian children who had been exposed to the 2004 Tsunami but were quickly evacuated from the disaster region and returned to their home country. Despite severe exposures in which approximately two-thirds of the children were injured, experienced a life-threatening situation, or witnessed grotesque scenes, PTSD rates were low, which the authors attributed at least in part to the relative lack of secondary adversities these children and families experienced.

Interpersonal Exposure

Interpersonal exposure, which can constitute a qualifying exposure for a diagnosis of PTSD (American Psychiatric Association 2000) derives from relationships with exposed individuals. Interpersonal exposure can be measured as the nature (or closeness) of the relationship (e.g., family member, other relative, friend), the experiences of the related individual (e.g., death, injury), or the number of people the child knew who were involved. Chemtob et al. (2011) used three questions to examine interpersonal exposure to the September 11 World Trade Center attacks—family member who escaped but was not injured, family member who was injured but not killed, and knew someone who was killed—in their study of PTSD symptoms and suicidal ideation in New York City adolescents. These categories capture both the nature of the relationship and the characteristics of exposure of the related individual.

Children also are affected by the effects of a disaster on their caregivers, which can occur as a loss of important resources (e.g., parental job loss) or through their caregivers’ transmission or role modeling of fear reactions or impaired ability to nurture or provide care. New York City children’s posttraumatic stress reactions after September 11 were related to parental viewing of disturbing television images and parental crying in the presence of the children (Fairbrother et al. 2003). Maternal responses to the September 11 attacks influenced the reactions of children geographically distant from the attacks (Wilson et al. 2010) as well.

Indirect and Distant Trauma

Children who are not directly exposed to an event but reside in a community where a disaster strikes may react to the disaster’s effects on their environment. Other children residing in places geographically distant from the disaster epicenter may be exposed through relationships with people in closer proximity to the incident or through word of mouth or media coverage of the incident. The focus on these indirectly-exposed and remotely-affected children has increased with concern about children’s reactions to catastrophic events like the three disasters reviewed for this report. Thus, researchers may draw their sample from the population of children located outside the disaster area (i.e., geographically distant), particularly when a disaster generates national security concerns and/or widely-covered media attention. For example, to compare pre-and post-attack reactions among children and parents following the September 11 attacks, Henry et al. (2004) gathered information from a sample of families in Chicago already under study at the time. These investigators noted an absence of disaster-related changes in psychological symptoms in the children, but their parents reported increased monitoring of their children and the development of stronger beliefs about the importance of family following the attacks. In another study, a nationally-representative group of adults and their adolescent children served as a geographically-distant sample (Gil-Rivas et al. 2007). Despite their distance from the attack sites, adolescents’ acute reactions and prior mental health history, and their parents’ reactions and inabilities to help their children, were associated with posttraumatic stress symptoms (Gil-Rivas et al. 2007).

Contact with Media Coverage

Early studies addressing children’s contact with disaster media coverage generally used the amount of time children consumed media coverage as a measure of media consumption (e.g., Pfefferbaum et al. 2001, 2003). September 11 studies broadened that perspective. For example, Fairbrother et al. (2003) explored children’s reactions in relation to media content. Children with severe posttraumatic stress were more likely than those with less severe reactions to have viewed television coverage with images of buildings collapsing or people falling or jumping. Some studies have measured children’s subjective appraisals of their media consumption despite the fact that these appraisals may not correspond to quantitative doses or intensity of media contact. For example, the use of response options like “a lot” of media coverage (e.g., Lengua et al. 2005) may convey different meanings across individual children and contexts.

Thirty-one (45.6 %) studies of the September 11 attacks in this review examined the relationship of media contact with emotional and health behaviors and outcomes in children in New York City (e.g., Duarte et al. 2011; Hoven et al. 2005), Boston (e.g., Otto et al. 2007), California (e.g., Kennedy et al. 2004; Milam et al. 2005), Seattle (e.g., Lengua et al., 2005), a southeastern coastal city (e.g., Saylor et al. 2003), across the United States (e.g., Schuster et al. 2001), and as far away as London (e.g., Holmes et al. 2007) and India (e.g., Ray and Malhi 2005). These studies used a variety of assessment approaches including surveys or interviews delivered in person (e.g., Hoven et al. 2005), on the telephone (e.g., Schuster et al. 2001), through the postal mail (e.g., Levine et al. 2005; Whalen et al. 2004), and on the Internet (e.g., Gil-Rivas et al. 2007). No studies of the 2004 Tsunami or Hurricane Katrina examined media issues.

While most media studies to date have examined television coverage (e.g., Holmes et al. 2007; Hoven et al. 2005; Kennedy et al. 2004; Schuster et al. 2001), others have examined other media forms as well (e.g., Duarte et al. 2011; Ray and Malhi 2005; Saylor et al. 2003). For example, Duarte et al. (2011) studied the effect of Internet coverage (among other forms of media) after the September 11 attacks. These researchers found that reactions associated with media contact may vary with the type of media and other factors. In children without any direct or family exposure to the September 11 attacks, they concluded that probable PTSD was linked to media contact, while for children who were exposed directly or through their families, the direct or family exposures were the primary determinant of probable PTSD. Probable PTSD was reportedly more consistently associated with intense use of television, radio, and print media among those with no direct or family exposure than was intense use of media among those with direct or family exposure; this was not true for use of the Internet. In a sample of children from a southeastern coastal city, Saylor et al. (2003) reported different findings related to September 11 coverage. High PTSD symptom scores were associated with Internet use but not with other types media. Possible explanations mentioned by the authors included a greater number, compared to television, of graphic images of the attack that were available on the Internet and the easier and repetitive access to those materials by children. September 11 and Hurricane Katrina studies also have examined potentially confounding variables and employed more sophisticated research designs than earlier cross-sectional media studies. For example, temperament and pre-existing symptoms (Lengua et al. 2005) and pre-existing family difficulties and maternal stress (Kennedy et al. 2004), along with measures of media contact, have been identified as determinants of children’s outcomes.

Critically important, a cause–effect relationship should not be assumed from findings related to media coverage. Future studies on media coverage should use more sophisticated research designs to consider populations with varying forms and levels of disaster exposure and across the spectrums of development and culture; incorporate more comprehensive assessments than extant studies have employed; investigate various media forms, content, and context of media contact; and explore a range of predictors and outcomes.

Dose–Response Relationship

The extant research on disasters suggests that there is a dose–response relationship between exposure and outcome, with those who experience relatively more intense and severe exposures having more intense and negative reactions (Furr et al. 2010; Masten and Narayan 2012). Probably the most universally understandable index of exposure is physical presence or geographic closeness, quantified through various measures of proximity or intensity of the experience, including perceived threat and distress at the time of the incident (Furr et al. 2010). While distance or location may be objective, the contributions that simple proximity measurement brings to understanding the impact of disaster can vary greatly. In general, when children physically present at, or geographically near, the disaster site or from the disaster community were compared to those who were not exposed, or to those who were geographically distant from the disaster, those in closer proximity evidenced greater adverse outcomes. This is not always the case, however. In their Internet-based study with a nationally representative sample of adults throughout the United States, Schlenger et al. (2002) found that the proportion of families in which at least one child was reported to be “upset” was highest in the New York City area (60.7 %), but it was not statistically different from families in any other location throughout the nation (p. 586). It is likely that outside of the disaster community, or away from the disaster site if in a safe location, differences in geographic proximity are less important than other factors in predicting outcome.

The dose–response relationship can extend to measures of interpersonal exposure, the importance of which is reflected in its inclusion as a form of exposure for a diagnosis of PTSD (American Psychiatric Association 2000). Gould et al. (2004) found that many children knew someone who was injured, missing, or killed in the September 11 attacks, and that most of the people known by the children were not close family members or friends but were more likely known through distant relationships such as a fireman, family member of a friend, or acquaintance’s family member. Those who experienced the injury or death of a family member or friend were more likely than those who knew no one injured or killed to report experiencing PTSD symptoms but not more likely to meet criteria for a diagnosis of PTSD (Gould et al. 2004).

There may be a dose–response relationship with media coverage as well. For example, studies have examined subjective measures of the amount of contact. Lengua et al. (2005) used response options ranging from “hardly at all” to “a lot” (p. 635). Becker-Blease et al. (2008) asked parents how many times their child viewed televised news stories or pictures about each of three events—the September 11 attacks, child kidnapping, and the 2002 sniper shootings. Fairbrother et al. (2003) assessed children’s exposure to particular televised images of the September 11 World Trade Center attacks, specifically an airplane hitting the Towers, buildings collapsing, people running from smoke or debris, and people falling or jumping from the buildings.

Masten and Narayan (2012) noted current interest in “disaggregating dose to identify toxic experiences, understanding nonlinear effects, and delineating the processes that mediate observed dose effects” (p. 235). Results often are presented as means which may conceal considerable variation among those at similar levels of risk. Not all dose–response relationships are linear; they may exhibit nonlinear, threshold, or other patterns (Masten and Narayan 2012). Moreover, finding a disaster-response relationship may depend on the timing of the assessment and the informants (Furr et al. 2010; Masten and Narayan 2012). In their review, Masten and Narayan (2012) concluded that dose–response relationships may dissipate over time, and they found evidence that studies using child informants revealed clearer patterns of the dose–response relationship than those using parental report.

Outcomes

Early work examining children in the context of disasters focused primarily on posttraumatic stress reactions (Saylor and Deroma 2002), later expanding to address other outcomes such as depression, anxiety, grief and traumatic grief, and behavior problems (Balaban 2006; Saylor and Deroma 2002; Silverman and Ollendick 2005). More recent research has studied concerns such as academic performance (e.g., Weems et al. 2013), substance use (e.g., Chemtob et al. 2009; Hoven et al. 2005; Wu et al. 2006), suicidal ideation (e.g., Gould et al. 2004), general functioning (e.g., Berger and Gelkopf 2009; Brown et al. 2004; Chemtob et al. 2009; Wilson et al. 2010), coping (e.g., Hafstad et al. 2011; Salloum and Overstreet 2008; Vigil and Geary 2009), and posttraumatic growth (e.g., Hafstad et al. 2010, 2011; Kilmer and Gil-Rivas 2010; Milam et al. 2005).

Selection of outcome variables depends, in part, on the timing of the assessment. For example, depression may occur later in the post-disaster course than symptoms of Acute Stress Disorder or PTSD and may be a reaction to the enduring secondary adversities occasioned by damage and devastation of the community or demoralization and despair of its members. Moreover, it is incorrect to determine a diagnosis of PTSD from assessments conducted within the first month following a disaster because diagnostic criteria require the presence of symptoms for more than 1 month (American Psychiatric Association 2000).

The primary focus of disaster research has been on adverse outcomes, such as distress reactions and changes in behavior, which represent public health concerns, and on psychopathology which has implications for clinical services. The setting in which children are recruited reflects these distinctions. For example, children recruited from mental health settings are likely to suffer symptoms at pathological levels constituting diagnostic conditions, while non-pathological distress is to be expected in school-based and other community samples. Jaycox et al. (2010) delivered two separate interventions, Trauma-Focused Cognitive-Behavioral Therapy (TF-CBT) and Cognitive-Behavioral Intervention for Trauma in Schools (CBITS), to children 15 months post Hurricane Katrina and measured outcomes appropriate to the settings in which the children received the interventions. PTSD symptoms were assessed in children receiving both interventions and both groups of children showed improvement. PTSD diagnosis was measured only in children receiving TF-CBT, which was delivered in a local clinic. Symptoms of depression decreased post treatment in children who received CBITS, but those who received TF-CBT showed no significant improvement in depressive symptoms.

While reports of disaster studies tend to be framed in terms of negative effects, with investigators often appearing to assume that endorsement of any distress reaction is pathologic, the literature increasingly recognizes that statistically significant results may not be clinically or practically meaningful. Thus, changes over time and comparisons among groups must be interpreted with an eye toward the magnitude of change and the implication of the level of reaction.

Research is emerging that broadens traditional inquiries to include investigations of positive outcomes. In fact, a growing body of research focuses on the fact that many children overcome significant trauma and perhaps even thrive in the face of disasters. Saylor and Deroma (2002) emphasized the importance of assessing coping, stress appraisal, and social support. Researchers have begun to incorporate measures of the extent to which children’s functioning is restored despite transient symptoms (resilience) and to study positive changes following trauma (e.g., greater appreciation for life, increased sense of personal strength) including personal, interpersonal (e.g., improved relationships), and/or spiritual growth in disaster studies (e.g., Cryder et al. 2006).

There is increased recognition that outcomes derive from complex pathways that evolve over time, leading to the identification of outcome trajectories that permit a more detailed picture of individual variation. Bonanno et al. (2010) and (2011) described four prototypical trajectories that have emerged from research across a variety of adversities: chronicity, characterized by a sharp elevation in symptoms and in functional impairment leading to chronic dysfunction that may endure for years; resilience or healthy adjustment, indicated by transient symptoms and minimal impairment; recovery, characterized by initial elevations in symptoms and distress following a traumatic event that gradually decrease over months; and delayed distress, exemplified by moderate elevations in symptoms and distress following a traumatic event that gradually worsen over time. Other identified trajectories included continuous distress associated with enduring impairment that predates the event; and distress-improvement, in which elevated distress prior to the event decreases markedly after it.

Measurement and Instrumentation

The methods and instruments used to collect data and the informants from which the data are collected are crucial to the design of disaster protocols and represent other key elements in research methodology. Thus, they influence the drawing of valid inferences. This section covers informants, the quality of measures, and methods of assessment.

Informants

The most obvious source of information is provided by the individuals under study. In this review, 118 (81.4 %) studies used some form of child self report. When it is difficult to access youth samples, others may provide information about children. The child’s parents can provide descriptions of the child’s externalizing behavior; teachers can identify deviations from the child’s peer group; and clinicians can determine what constitutes pathological reactions (Myers and Winters 2002). In this review, 65 (44.8 %) studies used some form of parent or significant other report. See Table 1. Siblings and other family members, classmates, and friends may provide unique perspectives of a child’s situation, but they may be difficult to access. Children are considered more accurate in describing their own experiences, perceptions, and internalizing emotions as these may not be apparent to others (Myers and Winters 2002). Thus, the report of others, even parents or knowledgeable adults, should not replace the child’s self report (Balaban 2006).

Disaster studies have included parental reports about their children on a number of topics. Researchers have used parent report to examine parents’ perceptions of their children’s reactions and the factors that affected their reactions (e.g., Fairbrother et al. 2003; Schlenger et al. 2002; Schuster et al. 2001; Stein et al. 2004); the services their children received (e.g., Stuber et al. 2002); and children’s event-related television viewing (e.g., Fairbrother et al. 2003). Studies also have used adult samples to access children for direct assessment (e.g., Pfefferbaum et al. 2006b). Some studies requested information from parents about their own reactions as well. For example, in a study of the September 11 terrorist attacks, parents responded to questions regarding their own disaster-related stress and coping as well as inquiries about their children’s distress and event-related television viewing (Schuster et al. 2001).

Three studies did not assess children directly or through their parents or teachers (Balas and Guttman 2003; Olteanu et al. 2011; Ward et al. 2008). With the objective of describing children’s mental health needs in the aftermath of Hurricane Katrina, Olteanu et al. (2011) reviewed medical records of pediatric patients who were seen in a mobile mental health clinic in New Orleans between July 2007 and June 2009. Ward et al. (2008) assessed the effects of Hurricane Katrina on the behavior and academic performance of displaced students in Mississippi through records provided by the Mississippi Department of Education. Balas and Guttman (2003) collected data from children’s psychoanalysts rather than directly from the children or their relatives which, according to the authors, had the advantage of being less intrusive while obtaining accurate information elicited in the context of the trusted relationship between the children and their therapists. Table 1 identifies types of informants used for studies of each of the three disasters.

Ideally, researchers should use multiple informants as collateral sources of information to obtain a comprehensive appraisal of children’s disaster reactions. In this review, 45 studies (31.0 %) collected data from more than one informant. See Table 1. The use of multiple informants may improve the validity of the data obtained (Kraemer et al. 2003), potentially decreasing systematic errors caused by informants’ characteristics, viewpoints, or distortions in memory (Achenbach et al. 1987; Kraemer et al. 2003; Wagner et al. 2010). The use of multiple informants also allows data to be collected on reactions that may exist only in specific settings or circumstances. For example, parents may not notice symptoms that emerge in the school setting where children must concentrate, perform academically, and conform to peer standards for achievement and interactions, while teachers are in a position to assess these reactions. Thus, informants whose viewpoints are independent from one another should be selected to provide diverse perspectives (Kraemer et al. 2003). Otherwise, collecting data from informants with similar points of view might result in collinear information without improving the validity of the outcome measurement.

Discrepancies Across Multiple Informants

While capturing multiple perspectives is theoretically ideal, the use of multiple informants requires researchers to reconcile discrepancies in their reports and to consider these differences when interpreting results. Discrepancies across informants or situations may be random or they may reflect biases in perceptions or perspectives (e.g., Kraemer et al. 2003; Rowe and Kandel 1997). Discrepancies may be informative (e.g., Holmbeck et al. 2002). For example, children may underreport their own anxiety for reasons of social desirability, and parents may underreport relative to the child because they are unaware of the child’s internal states. Differing reports from multiple informants also may reflect variations of the measured construct across different settings. For example, Chemtob et al. (2010) observed discrepancies between mothers’ and teachers’ reports on children’s psychological outcomes in the aftermath of the September 11 World Trade Center attacks. Teachers reported fewer somatic complaints and more aggressive behavior than did mothers, perhaps because children are more likely to report physical symptoms to their mothers (rather than teachers); and in the school environment, children have greater opportunities to interact with their peers and, thus, engage in aggressive behavior observable by teachers.

Statistical Analysis of Multiple-Informant Data

Discrepancies in data collected from multiple informants are a challenge in statistical analysis. Several methods for dealing with these discrepancies have been proposed. These include conducting separate analyses of each data set; aggregating the data from multiple sources into one single variable for analysis; applying confirmatory factor analysis (CFA) and structural equations modeling (SEM); and using regression methods with the full data set (Holmbeck et al. 2002; Horton and Fitzmaurice 2004; Horton et al. 1999; Van Dulmen and Egeland 2011).

Quality of Measures

Researchers should consider several characteristics of a measure before using it in a study (Kazdin 2003). Two general considerations—reliability and validity—are important in deciding what instruments to use.

Reliability

A reliable measure is one that measures a construct consistently. Reliability is required for the validity of the instrument in measuring the construct. Good reliability does not necessarily ensure validity of the instrument in measuring the construct, however. Reliability includes several types of estimates including inter-rater, test–retest, and alternate-form reliability as well as internal consistency (Kazdin 2003). The most commonly reported reliability statistics are internal consistency and inter-rater reliability when more than one rater is used.

Internal Consistency

Internal consistency, the most frequently computed reliability statistic, determines whether items designed to measure the same construct yield similar results. Its calculation does not require multiple raters or multiple administrations of the instrument. Internal consistency generally is measured with Cronbach’s alpha (Cronbach 1951) which is applicable to both continuous and dichotomous variables. There is no consensual guideline for the interpretation of the computed alpha, although some authors have proposed criteria for the levels of reliability.

Inter-Rater Reliability

Inter-rater reliability assesses the degree to which a construct is consistently identified when observations are made by different people (i.e., raters). Researchers may confuse inter-rater reliability and inter-rater agreement, which are two distinct concepts. While the former assesses consistency in ratings, the latter determines the degree of consensus among raters. A high inter-rater reliability may coexist with a low inter-rater agreement and vice versa (Gisev et al. 2013; Wagner et al. 2010). Inter-rater reliability can be measured with the Pearson product moment, the Spearman’s rho, and Kendall’s statistics. Statistics used to assess inter-rater agreement include: percent agreement which does not take into account agreement occurring by chance; Cohen’s kappa, a chance-corrected statistic applicable when there are only two raters; and Fleiss’ kappa which is chance-corrected as well and can be computed for any number of raters. There is no consensus for the interpretation of inter-rater agreement statistics, although the guidelines proposed by Landis and Koch (1977) often are used to define the level of agreement for the reported kappa. Studies which reported kappa statistics using data from their own sample include, among others, studies by Halberstadt et al. (2008) for coding children’s coping strategies; Lutz et al. (2007) for coding responses to the Child’s Reaction to Crisis Interview; Sprung (2008) for coding children’s unwanted intrusive thoughts; and Sprung and Harris (2010) for coding ratings of concentration and distractibility, cognitive cuing of emotions, and children’s unwanted intrusive thoughts. Kilmer et al. (2009) reported a Kendall’s tau-b assessing reliability among raters of the Posttraumatic Growth Inventory for Children-Revised.

Reliability Statistics

Reliability statistics are based on the scores of the informants and, thus, are sample dependent. Their values may differ from a previous assessment if the respondents are drawn from a population different from the one in the previous assessment. Thus, investigators should determine reliability of the instrument in their study samples rather than relying on values obtained in previous research, especially when prior studies were conducted in a different socio-cultural setting. Some studies included in this review reported reliability statistics of the instruments used for their study sample (e.g., Berger and Gelkopf 2009; Kelley et al. 2010; Moore and Varela 2010; Spell et al. 2008; Vigil and Geary 2008; Vigil et al. 2010). A large majority of studies, however, demonstrated instrument reliability by providing evidence of good to excellent reliability statistics from previous publications, which may be relevant if the instrument is administered in the same language and socio-cultural framework as in the referenced literature. Nevertheless, as a general rule, the psychometric properties of instruments in the disaster field should be reevaluated with each new sample (e.g., determining internal consistency of a measure in the given sample) (Streiner 2003; Wilkinson and The Task Force on Statistical Inference 1999). The easiest reliability statistic to report is internal consistency as it requires no additional methodological changes to the study.

Validity

Validity (including its estimates) is a vast topic with no single, universally-accepted definition. For the purposes of this review, validity refers to the extent to which a concept, conclusion, or measurement is well-founded and corresponds accurately to the construct under consideration. Types of validity include construct, content, concurrent, predictive, criterion, face, convergent, and discriminant validity. Each type of validity estimate can contribute to estimates of a tool’s psychometric properties in assessing the domain/construct of interest (Kazdin 2003). In planning disaster studies, researchers should choose instruments with established validity estimates. For example, Brown and Goodman (2005) assessed criterion validity of the Traumatic Grief subscale of the Extended Grief Inventory (EGI-TG). This subscale showed good convergent validity with the depression and anxiety subscales of the Behavioral Assessment System for Children (BASC) and with the Child PTSD Symptom Scale. EGI-TG also showed good discriminant validity with the Sensation-seeking and the Self-esteem subscales of the BASC.

According to Kazdin (2003), a particular construct or variable should be assessed with multiple tools to ensure adequate measurement of the different components of the construct, to account for specific situational factors, and to reduce variance in the outcome variable being measured and in the method of assessment (e.g., different raters). The use of multiple measures and multiple methods reduces the impact of errors associated with methodological factors in measuring a specific construct (Kazdin 2003). For example, Aber et al. (2004) assessed depression, anxiety, and conduct disorder (among other disorders), using both the Seattle Personality Questionnaire and the Computer-Diagnostic Interview Schedule for Children-IV in a sample of adolescents after the September 11 attacks. They reported significant correlations between the two instruments for each of these outcomes.

Methods of Assessment

The studies in this review measured outcomes, correlates, risk factors, and intermediary variables using (1) inventories or rating scales administered to children themselves or to their parents or others (e.g., teachers, clinicians) who observe them; (2) interviews including structured diagnostic assessments; (3) cognitive assessment; and/or (4) psychobiological measurement. Decisions about instruments and informants are guided by the research questions and populations being studied but also often reflect time and resource constraints. Questionnaires and rating scales are relatively easy to administer, and they usually require less time and fewer resources than do more comprehensive assessments conducted through interviews or structured testing. The following sections describe these assessment methods and the status of the field. Measuring disaster exposure is covered under inventories and rating scales though it also is an important component of other disaster assessment approaches. PTSD symptoms are covered under inventories and rating scales, and PTSD and other diagnoses are covered under interviews. Studies using cognitive and psychobiological measurement techniques are covered in a separate section on special issues in assessment approaches.

Inventories, Questionnaires, Surveys, and Rating Scales

In his 2006 review, Balaban offered specific recommendations for assessing children’s disaster reactions: (1) examine the type and severity of children’s trauma (including exposure); (2) assess a wide range of symptoms; (3) obtain adults’ ratings of children’s behavior; (4) evaluate parent and family disaster reactions and adjustment; and (5) address functional impairment in addition to subjective and objective ratings of symptomatology. Balaban (2006) also emphasized the importance of considering age and developmental issues, risk and resilience factors, and the cultural setting as part of assessment.

Inventories, questionnaires, surveys, and rating scales typically have been administered in paper-and-pencil format but other approaches, such as web-based technology, have been used as well (e.g., Gil-Rivas et al. 2007; Schlenger et al. 2002). Lack of consistency in the instruments used in disaster studies makes it difficult to compare data across events and to draw conclusions from multiple studies. There also may be limitations associated with particular instruments. Children may not understand terminology associated with the constructs being measured. For example, young children may not comprehend terms and concepts such as “numbing.” Symptom rating scales that do not query all diagnostic criteria may result in inflated estimates of outcome rates such as PTSD if, for example, the questions emphasize symptoms without acknowledging the requisite exposure, duration of symptoms, or effects on functioning (North and Pfefferbaum 2002).

Measuring Disaster Exposure

In addition to gathering basic information (e.g., demographics) and measuring specific variables under study (e.g., posttraumatic stress, depression, coping, resilience), disaster studies routinely query various aspects of the child’s exposure experiences. As discussed above, the child’s disaster exposure can be described specifically in terms of geographic proximity to the disaster’s epicenter, injuries suffered, witnessing death or injury to others, losses, facing direct life threat, or other experiences.

The difficulty in interpreting results and developing cohesive theories in disaster studies stems, in part, from the lack of a consensus regarding definitions of exposure and the extent to which each disaster is characterized by its own unique and salient features, underscoring the importance of addressing children’s personal experiences using questions tailored to capture aspects of the specific disaster. For example, Hurricane Katrina and its aftermath brought distinct features of exposure (e.g., wading through water, seeing floating bodies) and secondary adversities (e.g., mass evacuation, living in severely overcrowded facilities, extended displacement), the results of which could not be captured without inquiries designed specifically for that disaster. McLaughlin et al. (2009) created 10 categories of hurricane stressors in their study of serious emotional disturbance among Katrina survivors including victimization due to lawlessness, extreme psychological adversity such as lack of privacy, and ongoing problems with housing among other experiences.

The most common way to index exposure in the studies in this review was via a checklist or survey items. Many investigators included their own set of questions to assess various aspects of exposure. Others used existing measures. One of the most frequently-used tools measuring exposure to hurricanes and their aftermath, the HURTE (“Hurricane-Related Traumatic Experiences”), is based on the work of La Greca et al. (1996). Participants are asked a set of yes/no questions to assess exposure to hurricane-related traumatic events, such as seeing windows and doors breaking, as well as experiencing physical injury.

The quality of a measure is, in part, determined by how well it captures the nuanced experiences of the individual children who are studied. Measures of disaster exposure typically have not been the subject of reliability and validity assessment. Highly similar exposure checklists have shown varying estimates of internal consistency (Cronbach’s alpha). For example, in a study by Weems et al. (2009), internal consistencies for the summed exposure events was poor (α = .54), with similar estimates in a study by Vigna et al. (2009) who reported alpha for threat = .41 and loss = .64. Other studies, however, have reported very high estimates of alpha (e.g., = .98 in Spell et al. 2008) and acceptable estimates of alpha (e.g., = .76 in Hensley and Varela 2008, and = .77 in Yelland et al. 2010).

It is likely that variation among participants in the experience of certain traumatic events affects estimates of internal consistency as Cronbach’s alpha is based on inter-correlations among items. In a sample of children all reporting very low disaster exposures, internal consistency could be very high due to non-endorsement of multiple items. Theoretically, it is not clear that low internal consistency of exposure experiences is an accurate indicator of an unreliable measure of exposure. Correlations of exposure indices with measures of proximity, test–retest reliability which assesses consistency from one test administration to the next for the same informant, or inter-rater reliability might be better indices of the reliability of exposure measures. In general, data on test–retest or inter-rater reliability of exposure measures were not reported in the studies reviewed.

Measuring Disaster Outcomes

Disaster reactions may be identified with discrete data, such as the presence or absence of particular symptoms or diagnoses, or with dimensional ratings of severity of symptoms or changes in symptom levels over time. A number of instruments using inventories and rating scales have been administered to study children’s disaster reactions across a wide range of symptoms, including PTSD and PTSD reactions, anxiety, depression, behavioral disorders, somatic disturbances, learning problems, and functioning (Balaban 2006). Balaban (2006) emphasized the importance of using psychometrically-sound, standardized, brief questionnaires in recommending instruments to assess PTSD and PTSD symptoms, depression, anxiety disorders, and behavioral problems. Table 2 presents the major instruments used to measure key outcomes for each of the three disasters studied in this review. PTSD or PTSD symptoms were more frequently measured in 2004 Tsunami (χ² = 9.98; df = 1; p = .0016) and Hurricane Katrina (χ² = 5.77; df = 1; p = .016) studies than in September 11 studies. Depression was more frequently measured in 2004 Tsunami studies than in September 11 (χ² = 13.21; df = 1; p = .0003) and Katrina (χ² = 15.45; df = 1; p = .000084) studies.

Table 2.

Research instruments used to measure the four most frequently assessed psychological outcomes

Outcomes/instruments	September 11	2004 Tsunami	Hurricane Katrina	Total
	n = 68 (46.9 %)	n = 28 (19.3 %)	n = 49 (33.8 %)	n = 145 (100 %)
PTSD
PTSD-RI (Steinberg et al. 2004)	5 (15.6)	12 (52.2)	18 (52.9)	35 (39.3)
IES (Weiss and Marmar 1997)	3 (9.4)	4 (17.4)	4 (11.8)	11 (12.4)
DISC/DPS (Lucas et al. 2001; Shaffer et al. 2000)	7 (21.9)	–	–	7 (7.9)
CPSS (Foa et al. 2001)	5 (15.6)	–	–	5 (5.6)
Others	11 (34.4)	7 (30.4)	12 (35.3)	30 (33.7)
Not specified	2 (6.3)	–	2 (5.9)	4 (4.5)
Totala	32 (47.1)b	23 (82.1)	34 (69.4)c	89 (61.4)d
Anxiety
CBCL (Achenbach 1991)	4 (21.1)	2 (22.2)	1 (6.3)	7 (15.9)
RCMAS (Reynolds and Richmond 1985)	2 (10.5)	1 (11.1)	3 (18.8)	6 (13.6)
DISC/DPS (Lucas et al. 2001; Shaffer et al. 2000)	3 (15.8)	–	2 (12.5)	5 (11.4)
STAIC (Spielberger 1973)	2 (10.5)	–	2 (12.5)	4 (9.1)
Others	6 (31.6)	6 (66.7)	9 (56.3)	21 (47.7)
Not specified	2 (10.5)	–	1 (6.3)	3 (6.8)
Totala	19 (27.9)	9 (32.1)	16 (32.7)c	44 (30.3)c
Depression
CBCL (Achenbach 1991)	4 (21.1)	3 (15.8)	1 (9.1)	8 (16.3)
DISC/DPS (Lucas et al. 2001; Shaffer et al. 2000)	7 (36.8)	–	1 (9.1)	8 (16.3)
CDI (Kovacs 1985)	2 (10.5)	2 (10.5)	1 (9.1)	5 (10.2)
CES-D (Radloff 1977)	2 (10.5)	–	2 (18.2)	4 (8.2)
Others	3 (15.8)	14 (73.7)	5 (45.4)	22 (44.9)
Not specified	1 (5.3)	–	1 (9.1)	2 (4.1)
Totala	19 (27.9)	19 (67.9)	11 (22.4)	49 (33.8)
Behavior
CBCL (Achenbach 1991)	9 (40.9)	3 (50.0)	4 (22.2)	16 (34.8)
BASC (Reynolds and Kamphaus 1992, 2004)	5 (22.7)	–	2 (11.1)	7 (15.2)
SDQ (Goodman 1997)	1 (4.6)	2 (33.3)	4 (22.2)	7 (15.2)
DISC/DPS (Lucas et al. 2001; Shaffer et al. 2000)	3 (13.6)	–	2 (11.1)	5 (10.9)
Others	3 (13.6)	1 (16.7)	6 (33.3)	11 (23.9)
Not specified	1 (4.6)	–	1 (5.6)	2 (4.3)
Totala	22 (32.4)	6 (21.4)	18 (36.7)b	46 (31.7)c

The number of studies that assessed the outcome was used as the denominator to calculate percentages. Studies using two different instruments to assess the same outcome are counted twice; thus, percentages may not add to 100

BASC Behavior Assessment Scale for Children, CBCL Children behavior checklist, CDI children depression inventory, CES-D Center for Epidemiological Depression Scale, CPSS Child PTSD Symptom Scale, DISC/DPS Diagnostic Interview Schedule for Children/Diagnostic Predictive Scale, IES Impact of Event Scale, PTSD-RI Post-Traumatic Stress Disorder Reaction Index, RCMAS Revised Children’s Manifest Anxiety Scale, SDQ Strengths and Difficulties Questionnaire, STAIC State-Trait Anxiety Inventory for Children

^aThe total represents the number of studies that have assessed the outcome

^bOne study used two instruments to measure the outcome

^cTwo studies used two different instruments to assess the outcome

^dThree studies used two different instruments to assess PTSD

Posttraumatic stress disorder and PTSD symptoms are the prototypic outcome measured in child disaster studies. The majority of studies in this review (n = 89, 61.4 %) measured PTSD or PTSD symptoms. The PTSD Reaction Index (PTSD-RI), used in 35 studies in this review (39.3 % of the studies which assessed PTSD), is one of the most widely-chosen instruments to assess posttraumatic stress reactions. See Table 2. The instrument can be administered through paper-and-pencil self report, individual interview, or group application (e.g., in classrooms), and a parent-report version is available (Steinberg et al. 2004). Researchers often make disaster-specific modifications to the PTSD-RI for youth to rate their symptoms in response to the disaster they experienced (e.g., Hurricane Katrina in Hensley and Varela 2008). The most recent of a number of versions of PTSD-RI is mapped to the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) (American Psychiatric Association 1994) criteria for PTSD (Steinberg et al. 2004). The DSM-IV version contains three sections. Part I is a brief, lifetime trauma screen in which the child identifies presence or absence of various events and the “one currently most bothersome” if more than one event is endorsed (Steinberg et al. 2004, p. 97). Part II assesses both objective and subjective features of Criterion A (exposure) of the PTSD diagnostic criteria. Part III assesses the frequency of DSM-IV PTSD Criteria B (intrusive reexperiencing), C (avoidance/numbing), and D (hyperarousal), and two additional experiences—fear of recurrence and trauma-related guilt. A score sheet identifies instructions to calculate a total PTSD severity score and severity scores for the B, C, and D diagnostic criteria (Steinberg et al. 2004). When the exposure criterion is met, endorsement of the appropriate number of symptoms “much of the time” or “most of the time” for the three symptom criteria is scored as having a likely diagnosis of PTSD, with a cutoff of 38 or greater having the greatest sensitivity and specificity for detecting the diagnosis (Steinberg et al. 2004, p. 97). Other cutoff and severity classifications have been used as well (Hensley and Varela 2008; Steinberg et al. 2004).

Research has shown internal consistency (Cronbach’s alpha) estimates for the PTSD-RI across versions to be greater than .80 (Hafstad et al. 2010; Hensley and Varela 2008; Roussos et al. 2005; Steinberg et al. 2004). In a large sample of children and adolescents between 7 and 18 years of age who were exposed to a wide range of traumas and were referred to healthcare facilities across the United States, Steinberg et al. (2013) found that the internal consistency of the PTSD-RI for DSM IV was not significantly different across age groups, race/ethnicity, and gender, ranging from .86 to .91 for the total score, thus supporting the use of the PTSD-RI in assessing PTSD symptoms across different population subgroups. The instrument also showed a strong convergent validity (r = .75; p < .001) with the Posttraumatic Stress subscale of the Trauma Symptom Checklist for Children-Alternative devised by Briere (1996). Test–retest reliability, which has been reported less often, was .84 for the DSM-IV version in a study by Roussos et al. (2005).

In addition to PTSD and PTSD symptoms, child disaster studies often assess other emotional and behavioral problems. For example, 44 of the reviewed studies measured anxiety, 49 measured depression, and 46 measured behavioral problems. A frequently-used instrument, the Child Behavior Checklist (CBCL) (Achenbach 1991), was administered in 20 (13.8 %) studies in this review to assess anxiety, depression, and/or behavior difficulties. See Table 2. The CBCL is a standardized rating scale that provides a total score, scores for internalizing and externalizing subscales, and scores for several narrow-band behavior problem areas such as avoidance/withdrawal. Items, scaled scores, and clinical cutoffs have been found to discriminate between clinic-referred and non-referred children, and normative data are available (Achenbach 1991). The CBCL has good reliability and has been extensively validated in the assessment of emotional and behavioral problems (Achenbach 1991). In a sample of New Orleans youth affected by Hurricane Katrina, Roberts et al. (2010) reported mixed internal consistency statistics for the CBCL in measuring depression and anxiety symptoms (Cronbach’s alpha = .65–.90) but good to excellent statistics for posttraumatic stress symptoms (Cronbach’s alpha = .81–.92).

Interviews

Interviews are another frequently-used assessment approach in child disaster studies. Studies in this review used interviews with the child only, the parent only, or both child and parent. One study interviewed a teacher and social workers as well (Hestyanti 2006). Interviews allow the researcher to collect more detail than is possible with surveys or questionnaires and to clarify potential misunderstandings by the child. For example, in addition to asking children about their coping strategies after Hurricane Katrina, Salloum and Lewis (2010) used brief, semi-structured interviews (lasting approximately 45 min) to determine what helped children and parents to cope. The interviewer was able to address children’s lack of comprehension regarding the meaning of “cope” by restating the question and explaining that they were asking “what has helped you feel better?” (p. 34). Interviews using multiple informants to query child reactions or symptoms offer the opportunity to obtain data from multiple perspectives, but they are subject to discrepancy among reports. Interviews also are susceptible to interviewer biases, such as inconsistency in asking questions and deviation from the standardized tool. Methods for limiting or detecting interviewer bias include the use of standardized procedures or instruments, training of the assessor on interview procedures, and tape-recording and reviewing interview sessions for quality control.

Diagnostic Interviews

Interviews can be an important tool for determining diagnoses. One of the most frequently-used interviews is the National Institute of Mental Health Diagnostic Interview Schedule for Children Version IV (DISC-IV) (Shaffer et al. 2000), administered in 12 (8.3 %) of the reviewed studies. The DISC-IV is a highly structured diagnostic interview designed to assess the psychiatric disorders common in children and adolescents. Its structure fosters administration by interviewers after minimal training, and it has well-established reliability and validity estimates (Shaffer et al. 2000). A Predictive Scales (DISC-PS) version of the DISC-IV also is available to identify probable presence of common childhood disorders based on DSM-IV criteria (American Psychiatric Association 1994). The DISC-PS can be administered to both youth and parents to assess the child’s psychiatric symptoms and the likelihood of psychiatric diagnoses in the past 12 months. This instrument has demonstrated adequate psychometric properties and excellent sensitivity and specificity compared to the full DISC for major depressive disorder, generalized anxiety disorder, oppositional defiant disorder, and conduct disorder (Cubo et al. 2010; Leung et al. 2005). While a number of the reviewed studies used the DISC-IV or DISC-PS, most relied on the psychometric properties established in prior studies rather than calculating psychometrics with their own samples (e.g., Hoven et al. 2005). Berger and Gelkopf (2009) reported some psychometric results for the DISC-PS in their own sample with an internal consistency (Cronbach’s alpha) of .73 for subjective functional impairment and .64 for somatic complaints.

Special Issues in Approaches to Assessment

A number of theoretically important variables and methods of assessment are needed to advance our understanding of children’s reactions. Assessment approaches developed in other research areas are being applied in the disaster context. These include the use of cognitive and biological assessment techniques to evaluate important areas of children’s functioning affected by disaster exposure.

Cognitive Assessment

While children are maturing and developing physically, their emotional and cognitive skills are rapidly advancing. Disaster studies have identified elements of cognition as predictors and outcomes of children’s post-disaster functioning. Indeed, children devote varying degrees of cognitive effort to the disaster and its aftermath, likely in part depending on the nature of the exposure and other individual and environmental factors. Faced with an extraordinary stressor such as a disaster, cognitive processes are first engaged as the appraisal process is activated to assist children in identifying whether the event is stressful and threatening and then in ascertaining the extent to which it may cause them and their loved ones harm or loss (Lazarus and Folkman 1984). Following the initial appraisal process, children’s other cognitive skills are engaged and become susceptible to the deleterious effects of disaster exposure. Exposure to trauma has been linked to impairment in several cognitive domains, including deficits in sustained attention, concentration, mental manipulation, and memory (e.g., Gil et al. 1990; Gilbertson et al. 2001; Vasterling et al. 1998, 2002). One of the most important elements of cognition is appraisal.

Appraisal

Stress appraisal is the cognitive process by which individuals assess potentially threatening stimuli to evaluate their impact on well-being. An individual considers the possible harm or loss that may come from an environmental stressor such as a disaster, while simultaneously assessing his or her own personal resources (Compas et al. 2001; Lazarus 1993). Subsequently, involuntary and conscious cognitive efforts are activated to reduce any perceived discrepancy between the demands of the stressor and one’s own personal resources (Compas et al. 2001; Lazarus 1993). As such, the thoughts, reasoning, judgment, and decision making that ensue interact with other individual and environmental factors to produce an adaptive or maladaptive disaster response.

It is common for assessments of disaster experiences and even exposure to include at least one item reflecting children’s cognitive appraisal process. For example, Holmes et al. (2007) asked children in inner-city London schools to report retrospectively on their initial appraisals of the threat caused by the September 11 attacks, including, for example: “When you saw the attack on the buildings did you feel scared?” and “Did it feel like your life was in danger?” (p. 479). Pina et al. (2008) developed and administered a yes/no questionnaire to child and adolescent survivors of Hurricane Katrina which contained an item related to children’s fear, at the time of the Hurricane, that they might die or be badly hurt. A 2004 Tsunami study asked children if they felt their own or a family member’s life had been in danger (Thienkrua et al. 2006). Similarly, several 2004 Tsunami studies (e.g., Hafstad et al. 2011) have utilized the PTSD-RI (Steinberg et al. 2004) as a measure of subjective distress and, thus, appraisal, by asking children whether during or immediately after the event they were afraid they would die or afraid they would be hurt badly. Assessing later appraisals, Levine et al. (2005) measured adolescents’ and their parents’ self-reported appraisal of the impact of the September 11 terrorist attacks at three and 8 months post disaster. This geographically-distant sample responded to four questions concerning the extent to which they currently believed the attacks affected America, their futures, themselves, and their peers.

Instead of, or in addition to, constructing or selecting a small number of items to assess cognitive appraisal, some child disaster researchers used extant measures of the cognitive appraisal process. For example, Lengua et al. (2006) administered the “What I Felt Scale” (Sheets et al. 1996) to children geographically distant from the September 11 attacks to assess the extent to which they “thought each thought” (with both positive and negative items listed) in response to recent stressors other than the disaster (referred to as pre-attack, dispositional threat appraisal style) (p. 1221). For threat appraisal specific to the September 11 attacks, these researchers substituted language about September 11 for that regarding general stressors. The Child Post-Traumatic Cognitions Inventory (CPTCI) (Meiser-Stedman et al. 2009) is a measure that shows promise with respect to capturing distorted beliefs about the condition of the world and one’s ability to cope after exposure to traumatic events. Using the CPTCI in an intervention study of children who had experienced single-incident trauma, Smith et al. (2007) found a strong correlation between changes in PTSD symptoms and changes in appraisals, a significant effect of the intervention on both PTSD symptoms and misappraisals, and a mediating effect of misappraisals on PTSD symptoms.

Attention and Concentration

As cognitive building blocks, attention and concentration are critical to memory and other high-level cognitive skills; however, few studies assessed them as predictors or outcome measures. Attention includes several components, including selectivity, duration, intensity, and speed of processing. “Paying attention” may refer to maintaining short- or long-term focus on one (selective) or more than one (divided) stimulus. The quality of attention is affected by inhibition, impulsivity, distractibility, and fatigue. Visual attention typically is evaluated with computerized tools, whereas most auditory attention measures involve a stimulus being read aloud by the examiner. These measures may be supplemented by behavioral observations, collateral reports by teachers and parents, and even self reports.

Attention and concentration may be assessed with items on self-report symptom checklists or questionnaires (e.g., Chemtob et al. 2008; Hendricks and Bornstein 2007; Hestyanti 2006; Holmes et al. 2007; Terranova et al. 2009), by children’s responses to standardized testing procedures for attention (e.g., Sprung 2008; Sprung and Harris 2010), by parent report (e.g., Scheeringa and Zeanah 2008), or by dot-probe and similar tasks of attention bias (e.g., Lindstrom et al. 2011). The effects of a disaster on attention also may be assessed through novel approaches, such as that undertaken by Sprung (2008) and Sprung and Harris (2010). These researchers tested the impact of disaster-related intrusive thoughts on attention among children exposed to Hurricane Katrina with the visual attention task of the NEPSY (“A Developmental NEuroPSYchological Assessment”) (Korkman et al. 1998). Specifically, children completed three trials of the visual attention subtest: one with no sound distraction; one while simultaneously listening to sounds of wind and rain from Hurricane Katrina; and another with no sound. Errors, time to complete the task, and off-task behavior were compared with performance by a control group. Results indicated that exposure to the hurricane was associated with more off-task behavior at 7 and 10 months post hurricane.

Using another novel approach, Lindstrom et al. (2011) asked parents exposed to the September 11 World Trade Center attacks and their children to complete a dot-probe task of attention bias. This task involves a computer screen on which a pair of faces (one neutral and the other happy or angry) appears, one face each on the left and right sides of the screen. On the next screen, participants must identify the location of an asterisk-probe (in place of the faces), which appears in either the right or left visual field, as quickly and accurately as possible. Congruent trials are those with asterisk-probes that appear in the same location on the screen (right or left) as the emotional face in emotion-neutral pairs. In incongruent trials, the asterisk-probe appears in the location of the neutral face in the emotion-neutral pair. Exposure intensity was associated with increased attention bias, such that highly exposed parents showed a bias toward attending to the angry faces. Interestingly, no such difference was found among the children with differentially-exposed parents; however, in the group of highly exposed parents, children’s attention bias was actually inversely related to their parents’ attention bias to threat.

Memory

Memory of the disaster and of the events preceding and following it affects children’s emotions and behaviors and may manifest itself in pathological ways, including intrusive thoughts. Images, thoughts, and perceptions generated from disaster exposure can become distressing and overwhelming to individuals of any age and, if persistent, recurrent, and intrusive, they may constitute criterion B (intrusive reexperiencing) symptoms for a PTSD diagnosis. Vivid visual images of a disaster often are accompanied by negative emotions in both short- and long-term contexts.

Heightened feelings of terror, panic, confusion, anger, and sadness that emerge from disaster exposure may actually contribute to enhanced memory of the event, as emotionally-charged events are remembered better than those not linked to significant emotion. In the case of emotionally-charged events, memory for emotional information may be preserved more than contemporaneous, non-emotional information. The results of a study with adults exposed to various types of trauma supported this notion and underscored the importance of (1) assessing the effects of trauma on memory for the event as well as general memory functioning and (2) distinguishing between those who develop PTSD and those who do not following trauma exposure, as there may be noteworthy differences in the way trauma affects memory in those with and without PTSD (Steinmetz et al. 2012).

Directly assessing emotional memory, Levine et al. (2005) surveyed adolescent and parent recall of the intensity of sadness, anger, and anxiety that they felt at the time of the September 11 attacks. Both groups recalled feeling more sadness than anger and anxiety at the time of the attacks; however, adolescents’ intensity ratings were lower than those of their parents. In fact, while recalled intensity ratings of parents’ emotions increased from three to 8 months post disaster, adolescents’ recalled intensity ratings of their emotions decreased during that time period.

The role of memory in children’s post-disaster adjustment is uncharted territory, with no studies in this review assessing pre-disaster memory as a predictor. Indeed, even the consistency of children’s memories of the events they reported having experienced during a disaster has been unstudied. Studies have incorporated long-term memory into outcome measures in the form of inquiries about the child’s experiences with intrusive disaster-related imagery (e.g., Calderoni et al. 2006; Hendricks and Bornstein 2007; Salloum and Overstreet 2008; Sprung 2008; Sprung and Harris 2010; Vigil and Geary 2008, 2009; Vigil et al. 2009, 2010). Intrusive imagery may be measured as an element of memory; however, repetitive, bothersome thoughts or ideas about a disaster do not necessarily reflect one’s actual exposure to, or experience with, the event.

One issue is the reliability of the reports of memories of an event. The extant research in adults suggests that reports of exposure change over time (e.g., Southwick et al. 1997), and a number of factors may be implicated (McNally 2003). Longitudinal studies have indicated that the more PTSD symptoms reported at a second testing, the more severe the person remembered the traumatic experience to have been (e.g., Southwick et al. 1997). In a study involving a large sample of adults exposed to the 2004 Tsunami, Heir et al. (2009) asked the question “How great do you think the danger was that you would die?” rated on a five-point scale from “none” to “overwhelming” (p. 511). Results indicated that mean ratings increased over time from 6 to 24 months post disaster. Increased recalled threat intensity was associated with stability in posttraumatic stress symptoms over time but not with other disaster-related variables (e.g., degree of exposure) or with personal (e.g., self-efficacy, mood symptoms) or social (e.g., social support) factors. Similar studies have not been conducted in youth exposed to disasters. Many tools used in child disaster studies include some measure of intrusive thoughts. For example, the Impact of Events Scale-Revised (Weiss and Marmar 1997) and the PTSD-RI for DSM–IV (Steinberg et al. 2004) include items addressing intrusive imagery but are not designed to evaluate comprehensively short- or long-term memory of disaster.

Sprung (2008) developed four questions to measure the frequency, recurrence, and content of children’s unwanted thoughts related to Hurricane Katrina but did not explicitly assess memory of actual events. In another Hurricane Katrina study by Kilmer and Gil-Rivas (2010), child survivors completed an adapted version of a rumination scale containing items related to intrusive and deliberate rumination (i.e., intentionally thinking about the Hurricane as a means to cope). Holmes et al. (2007) emphasized the importance of assessing intrusive imagery as a component of disaster reactions and have contributed to a better understanding of its role and varying intensity. They provided a definition and description of visual memory to children in London who had experienced the September 11 attacks through the media and then asked the children to recall whether they had experienced any unwanted “picture memories” of the day of the attacks over the past 2 weeks, allowing for free recall as well as utilizing a pre-existing list of possible image categories.

Children’s maturational status affects the formulation, storage, and recall of memories and, thus, the age of the exposed child is likely to influence the content and effects of disaster-related recollections. For example, Bonanno et al. (2010) asserted that younger children’s less-developed abilities to encode or remember certain elements of their disaster experiences are not necessarily beneficial, as their disaster reactions may be worse and recovery slower as compared with older children. Drawing from the literature more broadly, Pynoos and Nader (1989) found that children’s recall accuracy and their retrospective reports of perceived threat may be affected by actual proximity to a traumatic event. Specifically, children who were present on their elementary school playground at the location where a shooting occurred seemed to minimize their level of threat. These children failed to mention their injuries and recalled themselves being in safer locations or at a greater distance from deceased or injured persons than was actually the case. However, children who were safely inside the school building or even absent from school recalled themselves being closer to the danger than they truly were. Research on the stability of memory also may be facilitated, as recommended by Salmond et al. (2011), by expanding self-report questionnaires to include narratives about the disaster experiences.

Executive Functions

There is a dearth of research on the effects of trauma on children’s high level cognitive skills and a virtual absence of data on the effects of disasters on children’s executive functions including planning, judgment, problem solving, reasoning, inhibition, and mental flexibility. Developing concomitantly with maturation and experience, the cognitive tasks required for children to learn new information, retrieve previously-acquired information, and use the information to solve problems of everyday life depend on lower-level cognitive skills (e.g., attention, memory). Planning, judgment, and problem solving are central components of evaluations following traumatic brain injury and have been included in assessments of children affected by abuse and neglect (e.g., Bucker et al. 2012; Spann et al. 2012).

To date, few child disaster researchers (e.g., Vigil and Geary 2008, 2009) have assessed problem solving behavior as a component of coping, measured with self-report questionnaires such as the Kidcope (Spirito et al. 1988). Child disaster studies have not yet incorporated measures of executive functioning such as the Trail Making Test, Wisconsin Card Sorting Test, or Stroop Test. Certainly, interpreting the meaningfulness of post-disaster data without information about pre-disaster executive functioning is problematic. The extent to which children’s executive functions are affected by disaster exposure remains undetermined and is a topic in need of future research.

Coping and Emotion Regulation

Coping includes automatic and deliberate cognitive, behavioral, and emotional efforts to resolve a perceived discrepancy between the demands of environmental stressors and one’s own personal resources (Compas et al. 2001; Lazarus 1993). Assessing the cognitive processes activated in response to a disaster is crucial for identifying the relationship between children’s coping and their positive and negative disaster outcomes. While there is no unified conceptualization of coping, the process is thought to consist of various cognitive techniques including problem-focused coping (e.g., seeking information or attempting to change one’s circumstances), emotion-focused coping (e.g., seeking support, expressing emotions), attempts at emotion regulation, acceptance, cognitive restructuring, distraction, and denial (Compas et al. 2001).

Children’s general coping style and situation-specific coping typically are assessed using checklists of particular coping strategies wherein children retrospectively identify the behavioral, emotional, and cognitive strategies used in response to stressors (Pfefferbaum et al. 2012b). Other tools include interviews and informant reports. The Kidcope (Spirito et al. 1988), a self-report tool that assesses children’s frequency of use of a wide range of coping strategies including social withdrawal, distraction, wishful thinking, cognitive restructuring, social support, problem solving, self-criticism, emotion regulation, resignation, and blaming others, is one of the most widely-used coping measures (e.g., Vigna et al. 2010). More recent studies have utilized a wide range of tools to assess general (e.g., Brief COPE in Carver 1997; Children’s Coping Strategies Checklist and How I Coped Under Pressure Scale in Ayers et al. 1996) and specific (e.g., Coping Activities Checklist in Wadsworth et al. 2004) coping.

Assessed as a component of coping, emotion regulation broadly refers to the ability to monitor and manage one’s experience and expression of emotion. A common definition of emotion regulation is “the process of initiating, maintaining, modulating, or changing the occurrence, intensity, or duration of internal feeling states and emotion-related physiological processes, often in the service of accomplishing one’s goals” (Eisenberg et al. 2000, p. 137). As indicated above, the Kidcope contains items related to both negative and positive emotion regulation coping strategies (e.g., “I tried to calm myself by talking to myself, praying, taking a walk, or just trying to relax”). While measures of coping tap aspects of emotion regulation, dispositional differences in children’s emotion regulation more often are indexed with questionnaires such as the Early Adolescent Temperament Questionnaire (EATQ) (Capaldi and Rothbart 1992). Studying children exposed to Hurricane Katrina, Terranova et al. (2009) examined EATQ scores and found that effortful control protected against PTSD symptoms both initially and across time but that a negative coping style enhanced vulnerability to PTSD symptoms.

Academic Achievement

Academic functioning represents a significant domain of developmental competence and may be affected by disaster exposure. In the post-disaster context, children must contend with harm and devastation in all layers of their social ecology, including the academic environment. Damage to school buildings and property and disruptions in schedules and routines may result in diminished time devoted to learning, decreased concentration and attention in the classroom, and intrusive anxiety, all of which are likely to affect children’s learning and retention. Research supports a link between different types of trauma exposure and impaired academic performance and increased disciplinary issues (Cook et al. 2003). Measures of academic achievement or performance range from one item of self-reported difficulty with schoolwork (e.g., Kronenberg et al. 2010), grades or grade point average (e.g., Weems et al. 2009), and standardized test scores (e.g., Weems et al. 2013) to more comprehensive reports such as that by Ward et al. (2008) that examined state-wide data on school attendance, disciplinary problems, achievement test scores (math, reading, language arts), and retention-in-grade in children post Hurricane Katrina. Weems et al. (2013) found that the association of disaster-related traumatic stress to academic achievement was indirect via linkages from earlier (Time 1, 24 months post Katrina) PTSD symptoms which predicted later (Time 2, 30 months post Katrina) test anxiety. Time 2 test anxiety was negatively associated with academic achievement, assessed through standardized test scores at the end of the school year approximately 31 months post Katrina.

The literature is mixed as to the relationship between intelligence quotient (IQ) and PTSD symptoms. However, some data suggest a protective relationship (e.g., Saltzman et al. 2006). Saltzman et al. (2006) found that Full scale and Verbal IQ were significantly negatively associated with the number of traumas, re-experiencing symptoms, and impairment. Performance IQ was negatively associated with impairment. A brief assessment of IQ may facilitate assessments. For example, the Peabody Picture Vocabulary Test (PPVT) (Dunn and Dunn 1997) is a norm-referenced test of receptive verbal ability designed for use with a wide age range of participants. The test was standardized on a nationally representative stratified sample, has good psychometric properties, and is a relatively easy to administer and time-efficient measure. It can be used to determine whether participants have adequate comprehension to complete assessments validly.

Biological Assessment

Biological consequences of disasters on children range from injury and health effects to negative neurodevelopmental and hormonal outcomes; however, biological assessments, which provide objective measurement, are only now being explored in the context of disasters. These assessments include hormonal assays [e.g., research on the limbic–hypothalamic–pituitary–adrenal (LHPA) axis], physiological assessments (e.g., heart rate, blood pressure reactivity), and Magnetic Resonance Imaging (MRI) techniques (e.g., research on brain structure and function). Psychoneuroimmune effects, which have been reported in studies of adult disaster survivors (e.g., Ironson et al. 1997; Tucker et al. 2010), were not examined in child disaster studies reviewed as part of this report. While somewhat difficult to obtain due to resource limitations, biological assessments hold promise for an understanding of both emotional and physical effects of disasters on children. The methods for assessing biological consequences of disasters and the results of the sparse research in these areas are described below.

Injury and Health

The most obvious biological and health effects of disasters are those resulting directly from exposure to the event. In addition to medical assessments for children who sustain physical injury in a disaster, other health effects are assessed primarily by self or parent report or through review of medical and health records. The assessment of injury often is part of the determination of exposure. Questions on the HURTE include items asking, for example, “Did you get hurt during the hurricane?” (La Greca et al. 1996, p. 715). Other health effects include, for example, sleep disturbance, use of alcohol and drugs, smoking, somatic complaints, medical conditions, and relapse of symptoms or conditions in those with previous disabilities (Norris et al. 2002). Somatic complaints are a growing area of interest, with several studies in this review finding associations of exposure and PTSD symptoms with physical symptoms using, for example, the Somatic Problems scale of the CBCL or the Children’s Somatization Scale (e.g., Hensley and Varela 2008; Roberts et al. 2010). Results of a study of children exposed to Hurricane Katrina suggested that children with chronic health conditions were more likely than those without to experience negative health outcomes, to develop more new symptoms, and to experience more disruption in care (Rath et al. 2007).

The exploration of novel issues calls for diverse methodological approaches to health assessment. For example, in a series of studies of the effects of the September 11 attacks on children, researchers examined adverse effects on gestational duration and intrauterine growth in infants of mothers who had been exposed to the attacks (Landrigan et al. 2008). The study used blood samples and other biological measures to document exposure to environmental toxins in pregnant women and correlated the location of the women with geocoded data on these environmental exposure levels. Infant growth, development, and health status were assessed at birth and periodically thereafter over the first several years of the children’s lives.

Limbic–Hypothalamic–Pituitary–Adrenal (LHPA) Axis

Activity of the limbic–hypothalamic–pituitary–adrenal (LHPA) axis is an important biological component of the fight-or-flight reaction and, therefore, is likely to be salient in children’s reactions to disasters. Cortisol is used to examine this reaction. Findings on the linkage between PTSD symptoms and cortisol have been fairly consistent in showing dysregulation of cortisol levels in youth with trauma exposure generally and with PTSD symptoms; however, findings on the specific nature of the dysregulation have been inconsistent. In particular, the question of whether trauma exposure results in relative increases or decreases of cortisol has not been definitively answered (De Bellis 2001; Weems and Carrión 2007). Efforts to clarify the findings of various studies have focused on methods of specimen collection, diurnal rhythm in cortisol levels, the nature of the trauma, the sex and age of the exposed participants, PTSD symptom levels, and the time since the traumatic event (e.g., De Bellis 2001; Weems and Carrión 2007, 2009).

Methodologically, cortisol levels have been assessed using blood, urine, and saliva samples (Dahl et al. 1999; Nader 2008). A widely used collection technique is via saliva because this method is relatively convenient. Use of blood samples, requiring a trained collector and appropriate blood handling techniques, may alter cortisol responses if their collection induces fear in participants. For salivary collection, researchers have used cotton swab techniques, whereby a cotton swab is placed in the mouth and then placed in a sealed tube. Saliva is then centrifuged out of the cotton for assay. Salivary assay results for testosterone, progesterone, and other hormones may be artificially elevated when samples are collected using cotton, but salivary cortisol levels do not appear to be affected by the use of cotton collection methods (Shirtcliff et al. 2001).

There are wide individual differences in normal cortisol levels in youth (i.e., what is elevated for one person may not be so for another), and optimal levels of cortisol have not been established (Weems and Carrión 2007). Additionally, various methodological factors, such as variations in the situation, time of day, and time since the trauma, can alter outcomes (Gunnar et al. 2009). There is a strong and consistent pattern of diurnal rhythm with an increase in early morning cortisol and a steady decline throughout the day (Cicchetti 2003; Susman et al. 2007). Consequently, at a minimum, it is essential to hold the hour of collection constant across study participants. Given large individual differences and diurnal rhythm, collecting data at multiple consistent points in the day (e.g., pre breakfast, pre lunch, pre dinner, pre bedtime) and/or collecting samples at a specific time across several days (to obtain individual averages) permits a fairly rigorous assessment of basal cortisol levels.

The time since the traumatic event also has emerged as a potentially important component to understanding the relationship among trauma, PTSD symptoms, and cortisol levels (De Bellis 2001). Research, primarily from non-disaster, trauma-exposed (e.g., interpersonal violence) youth, suggests that after a period of relative cortisol hypersecretion, elevated levels may reverse (De Bellis 2001; Weems and Carrión 2007) to relatively low levels of cortisol (Boscarino 1996; Heim et al. 2000; Stein et al. 1997; Yehuda and McFarlane 1995). This low cortisol may result from an enhancement of the negative feedback loop at the pituitary-adrenal level of the axis (Heim et al. 2000; Yehuda and McFarlane 1995). In one of the first studies examining cortisol response in traumatized youth, Goenjian et al. (1996) examined basal cortisol levels in adolescents a full 5 years after they were exposed to the 1988 earthquake in Armenia. Those living close to the epicenter had more severe PTSD symptoms as well as lower basal salivary cortisol levels and greater cortisol suppression following dexamethasone. More recently, cortisol has been found to be positively associated with PTSD symptoms among those with recent traumas (within 1 year), but this association has been found to be negative when the time since the trauma was greater than 1 year (Weems and Carrión 2007). Studies of disaster-exposed youth may help to clarify whether this apparent discrepancy is related to the time since the trauma. Indeed, disasters may provide a methodological opportunity to clarify these issues with samples that have relatively homogenous dates of exposure to traumatic stress (compared to samples with individuals who have experienced community violence or interpersonal trauma, for instance). Such consistency in timing may allow researchers to test theories about changes in cortisol regulation following trauma.

The studies in this review highlight both methodological potentials and pitfalls. For example, Vigil et al. (2010) reported that adolescents exposed to Hurricane Katrina who were assessed 2 months after the disaster showed relatively lower cortisol levels than non-exposed youth. This would seem at odds with the hypothesis regarding time since the trauma, as one would expect relatively increased cortisol just 2 months post disaster. Unfortunately, it is not clear from the report that the time of cortisol collection was matched in the two samples and so the study precludes meaningful interpretation with regard to differences in cortisol. This is because the reported differences may have been due to the wide variation in the time of day that the saliva samples were collected (the cortisol index used was a single assessment, derived from two samplings 60–90 min apart, with collection times varying from 11 am to 7 pm). Similarly, in another study conducted 2 months after Hurricane Katrina, it is not clear that the time of cortisol collection was matched in the exposed participants who were living in a relocation camp and the control group (Vigil et al. 2009), making meaningful interpretation related to cortisol differences impossible. In this study, however, Vigil et al. (2009) also demonstrated that while the displaced children did not differ from the demographically-matched controls in psychological functioning, the children had higher salivary alpha-amylase activity relative to the controls. Mothers also had higher salivary alpha-amylase activity relative to maternal controls as well as greater depression, anxiety, and distress. Maternal anxiety was positively related with children’s cortisol level activity.

Using carefully matched cortisol collection times, Pfeffer et al. (2007) found that bereaved children had higher cortisol levels than non-bereaved children 19 months after the September 11 attacks. Bereaved children with PTSD showed relatively lower evening cortisol levels compared to bereaved children without PTSD, however. Such findings highlight the complexity of the role of cortisol in children’s disaster reactions. In this study, the time since the trauma was more than 1 year, and the findings with regard to high versus low cortisol differed depending on whether the bereaved children had PTSD (i.e., those with PTSD had relatively lower cortisol).

Autonomic Nervous System (ANS)

The autonomic nervous system (ANS) is a complex system consisting of two distinct branches called the sympathetic nervous system and parasympathetic nervous system. Theoretically, anxious or fearful responses to a frightening situation involve vagal withdrawal and the sympathetic response (i.e., an increase in heart rate) (Porges 2007). A growing body of research suggests that anxiety disorders in youth are characterized by differences in arousal as expressed by increased heart rate, heart rate variability (HRV), blood pressure, and electrodermal response (e.g., Scheeringa et al. 2004; Weems et al. 2005).

Measuring increases in heart rate in response to a stressor is one simple technique for assessing sympathetic arousal. For example, studies of traumatized children after accidental injury have demonstrated relationships between early post-event objective autonomic measures (e.g., heart rate) and later posttraumatic stress over relatively short time periods (e.g., 6 months) (e.g., De Young et al. 2007; Kassam-Adams et al. 2005; Nugent et al. 2006). In contrast, vagal induction may occur, reducing sympathetically-mediated heart rate response (i.e., parasympathetic response). Vagal tone or parasympathetic-mediated heart rate control typically is estimated by assessing respiratory sinus arrhythmia (RSA) or the degree of rhythmic heart rate fluctuation (i.e., increasing and decreasing HRV) during inhalation and exhalation of the respiratory cycle (Beauchaine 2001). A popular technique for deriving RSA-related HRV is power spectral analysis of artifact-free interbeat intervals as recorded using an electrocardiogram over time (Scheeringa et al. 2004).

Scheeringa et al. (2004) found that, compared to a matched control group of children without trauma exposure, young children who developed PTSD symptoms after exposure to any of a variety of traumatic events (accidents, witnessing domestic violence, medical procedures) exhibited objective signs of physiological reactivity, measured through electrocardiogram heart period (interbeat interval), in response to memories of the traumatic event. In a study of children of directly-exposed Oklahoma City bombing survivors conducted 7 years after the incident, Pfefferbaum et al. (2011) measured heart rate and systolic and diastolic blood pressure in response to a trauma-cue interview. They found that despite generally low levels of subjectively-reported posttraumatic stress and depressive symptoms, the children of Oklahoma City bombing survivors showed heightened objectively-measured physiological reactivity relative to a demographically-matched community comparison group. While it is unclear the extent to which the physiological reactivity in the survivors’ children was pathologic, the results suggest that the physiological effects of trauma may endure disconnected to subjectively-experienced affect in the offspring of highly exposed disaster survivors. Only one study in this review conducted physiological tests to examine children’s disaster reactions (Gump et al. 2005). Using electrocardiogram assessment, Gump et al. (2005) found that children assessed after the September 11 attacks exhibited significantly greater stroke volume to acute stress tasks compared with their responses 1 year later, and this change in reactivity differed significantly from the normal 1-year change in reactivity among children not exposed to the September 11 attacks (data collected pre September 11).

Brain Structure and Functioning

Magnetic Resonance Imaging (MRI) and related imaging techniques have allowed research of brain structure and function in youth exposed to trauma (e.g., Carrión et al. 2001, 2007, 2008, 2010; De Bellis et al. 1999; Garrett et al. 2012). The systematic literature search conducted for this review produced only one study of functional MRI in disaster exposed youth. Yang et al. (2004) examined 11 Taiwanese adolescents who had experienced a severe earthquake within 14 months of their trauma exposure. When adolescents were presented with traumatic reminders (words) of an earthquake, those with PTSD demonstrated activation in the bilateral visual cortex, bilateral cerebellum, and left parahippocampal gyrus. These findings were not evident in the non-PTSD control group. No studies in this review identified structural brain differences in disaster exposed youth, but research on other traumatic events highlights the importance of extending this methodology to disaster populations (e.g., Carrión et al. 2010). The use of MRI technology is costly, requiring access to equipment that is not universally available, thus limiting research using these techniques.

Intervention Studies

As noted above, intervention studies that randomly assign participants to an intervention or control condition can be used to address theoretical questions about disaster exposure. Given the importance of creating interventions to address the multitude of problems that beset children in the context of disasters, intervention studies also are important in their own right. Unfortunately, despite the extensive literature documenting the adverse effects of disasters on children, only recently has attention turned to developing and evaluating interventions to address these effects. Of the studies in this review, relatively few (n = 11, 7.38 %) investigated disaster interventions.

The clear methodological preference for intervention studies is randomized controlled trials (American Academy of Child and Adolescent Psychiatry 2010; Foa et al. 2000, 2009; Ursano et al. 2004). For example, the American Academy of Child and Adolescent Psychiatry practice parameter for PTSD (2010) identified the order of preference for evidence: (1) randomized controlled trials in which participants are randomly assigned to two or more treatment conditions; (2) controlled trials in which participants are non-randomly assigned to two or more treatment conditions; (3) uncontrolled trials in which participants are assigned to one treatment condition; and (4) case series or a case report.

Random assignment of participants to treatment groups decreases bias (Foa et al. 2009), and the use of a control group makes it possible to determine if observed changes can be attributed to the intervention rather than to unrelated factors such as the mere passage of time (Nock et al. 2008). Different types of controls support various interpretations of research findings. For example, comparison to no treatment or to a waitlist control is used to determine if an intervention is more efficacious than no treatment (Chambless and Hollon 1998). Three intervention studies conducted with children exposed to the three disasters examined in this review used waitlist controls (Berger and Gelkopf 2009; Scheeringa et al. 2011; Weems et al. 2009). In addition to providing an opportunity to determine whether one treatment is more efficacious than another, comparison of two interventions controls for processes that are independent of treatment and those that are common to all treatments (Chambless and Hollon 1998). Studies comparing two interventions (e.g., Brown et al. 2006; Catani et al. 2009; Jaycox et al. 2010; Salloum and Overstreet 2008, 2012) have been conducted with children following the three disasters reviewed in this report.

In their critical review of psychosocial treatments for PTSD, Foa and Meadows (1997) elaborated on other aspects of methodological rigor for the study of psychosocial interventions for PTSD. They enumerated seven gold standards for methodological rigor: (1) clearly defined target symptoms; (2) reliable and valid measures; (3) use of blinded evaluators; (4) assessor training; (5) manualized, replicable, specific treatment programs; (6) unbiased assignment to treatment; and (7) treatment adherence. Pfefferbaum et al. (submitted) used these standards to review child interventions that employed controlled conditions to examine posttraumatic stress outcomes in children exposed to disasters, terrorism, war, and other single incident traumas such as accidents. They found that most studies clearly described the interventions being tested or used manuals to guide application and most used standardized instruments to measure outcomes. Many studies used random assignment and provided assessor training. Fewer studies used blinded assessment or measured treatment adherence, and sample size in most studies was not adequate to detect small effects generally expected when comparing two active interventions. In addition to meta-analyses, pooling data across studies (Resick et al. 2007) or conducting multicenter studies (Stallard 2006) may be used to address the issue of power. Simple pooling of data, however, may lead to a Simpson paradox, the artifactual reversal of treatment effects as observed across the single studies (Bravata and Olkin 2001).

The extant research has yet to establish clear superiority for any single type of treatment, to identify the unique critical processes of intervention efficacy, or to determine if positive results are due to commonalities across interventions (e.g., receiving professional attention, the expectation of improvement). There does seem to be support for cognitive and behavioral techniques for treating posttraumatic stress in youth exposed to a variety of traumatic events (e.g., Silverman et al. 2008). A host of issues warrant attention in future studies. The relative benefit of various characteristics of the interventions (e.g., type of treatment, number and length of sessions, application to individual children or groups) and/or of intervention delivery (e.g., time since the disaster, location of service delivery) have not been well examined. Moreover, studies to date have been inconsistent in the sophistication of research design, such as in the use of follow-up assessment (e.g., number of assessments) and in addressing concerns that arise during application (e.g., sample attrition). Unfortunately, numerous challenges make such studies difficult, including ethical and practical concerns as well as difficulty securing funding (La Greca and Silverman 2009; Weems 2010).

Analysis and Conclusions

This report on methodological issues in child disaster mental health research was generated from an analysis of studies of three major disasters—the September 11, 2001, terrorist attacks; the 2004 Indian Ocean Tsunami; and Hurricane Katrina. The magnitude of each of these disasters drew widespread attention of both professionals and the public. The sheer number of studies for the three disasters is impressive attesting to the interest in, and ability to conduct, child disaster mental health research. Differences in the number of studies across the three events, with many more focused on September 11 and fewer on the 2004 Tsunami, may reflect interest in the populations, sophistication and curiosity of the professionals in the communities involved, available resources, and adequacy of the infrastructure to support research as well as other factors. For example, the September 11 attacks garnered unconditional attention in communities rich in clinical and scientific expertise and existing networks of institutions able to launch major research initiatives. The 2004 Tsunami occurred in one of the least developed areas of the world where resources, infrastructures for response and for conducting research, and sophistication of local providers and scholars were more limited. Demonstrating the concerns raised by this grave event, research on this disaster was conducted by international teams and much of the work was exceptional in quality despite difficulties associated with the setting. Hurricane Katrina ravaged an entire geographical region of the United States and damaged already-neglected infrastructures for response in communities with relatively few existing resources. Hurricane Katrina also resulted in mass evacuation and displacement, stimulating interesting research questions but complicating the acquisition of samples.

A diversity of methods has been brought to bear in understanding children’s reactions to disasters. Ruling out alternative explanations is central to drawing valid conclusions from research investigations and is facilitated by experimental designs that include manipulation, control, and randomization. In the absence of experimental control, case–control, longitudinal, and before-and-after designs can rule out some, but not all, alternative explanations for a study’s findings. While experimental studies that involve random assignment and double-blind procedures may not be possible or ethical, interventions can be employed to test causal hypotheses. Alternatively, qualitative design offers the advantage of providing detail-rich information that may be particularly useful in identifying specific predictors and outcomes and in generating hypotheses for future study. The extant literature on the three recent disasters evidenced both exploratory approaches and attempts to conduct theory-based hypothesis testing in both assessment and intervention studies. The initiation of intervention research after the September 11 attacks marked a real change in focus for the field as few such studies had been conducted prior to that time. Decades of prior research supported the need to create and study interventions to help children deal with their reactions to mass events. The higher percentage of Hurricane Katrina studies relative to September 11 studies that evaluated interventions reflected a continuing focus on developing and testing interventions. While still inadequate to declare any one approach superior to others, an evidence base for child disaster mental health interventions is developing and includes studies of interventions administered to address these three disasters.

This review revealed a reliance on convenience and purposive sampling. Random sampling is necessary to generalize the results to the target population, and it is preferred when there is a large pool of potential participants from which to draw the study sample. Securing samples is difficult in the aftermath of disasters given the chaos that ensues and because disaster communities may be suspicious of research and protective of victims. The studies reviewed used both self and other report and most used standardized instruments. Interestingly, the 2004 Tsunami studies were more likely than September 11 studies to measure PTSD or PTSD symptoms and depression. Noteworthy, studies of cognition and biological indicators were prominent after Hurricane Katrina.

There is a clear dearth of studies using longitudinal design with long-term follow up which is vital to work with children given the potential for disasters to disrupt development. While cross-sectional studies continue to predominate, archival data and data from studies examining non-disaster topics provided baseline and pre-event data for some investigations, especially for September 11 studies some of which used national or distant samples. While samples in disaster-prone areas might be assessed before a disaster strikes, using serendipitous data collected before an event is another possible avenue to increase the design rigor of disaster research. Before-and-after studies also are difficult to plan but represent another strong design available to disaster researchers. Not unexpectedly, there were no before-and-after studies of the 2004 Tsunami. Future longitudinal studies with long-term follow-up, particularly those with pre-event data, should help to clarify what effects are actually caused by disaster experiences as opposed to findings that merely reflect pre-existing risk factors. Furthermore, clarity in documenting the time since the trauma (and before the trauma for before-and-after studies) should be a basic requirement of disaster studies. Selecting the appropriate frequency of, and time period for, assessment is a central concern in disaster studies.

Disaster exposure has been described with respect to the child’s physical presence at, or geographic proximity to, the disaster site; interpersonal relationships with victims; various objective and subjective experiences; and contact with media coverage. In this review, the majority of the studies included participants who were directly exposed to the disaster or who were exposed through interpersonal relationships. A sizeable minority studied indirect or distant trauma. The study of distant trauma in remotely-affected participants has contributed to an understanding of the effects of media coverage and has illuminated the impact of disasters on society as a whole. Studies with children and their families located at distances from the disaster site were found primarily in research exploring the September 11 attacks, which was nationally threatening with potential implications for children throughout the country. Differences in the proportion of studies including remotely-affected participants may have a pragmatic basis explained by the difference in the number of potential participants directly exposed to the disaster. Both of the natural disasters (the 2004 Tsunami and Hurricane Katrina) involved much larger geographic areas, with more potential research participants available though accessibility was complicated by mass displacement. Studies have established a dose–response relationship between adverse outcomes and the intensity of children’s experiences, including their geographic proximity to the disaster site, subjective appraisals, interpersonal exposures, and media contact. The extant research underscores the need to address children’s specific individual experiences. While most investigations have incorporated an assessment of exposure, studies on the reliability and validity of these measures are needed.

The nature and extent of the influence of risk and protective variables beyond disaster exposure itself are not fully understood. These factors act directly on disaster outcomes and, importantly, they also mediate and moderate the relationship between children’s immediate disaster experience and short- and long-term outcomes. Understanding the roles of exposure and other individual, family, and social factors depends upon the extent to which measures and assessment techniques are valid and reliable, as well as on the sources of data and specifics of data collection (e.g., the child’s self report and/or collateral reports, questionnaires, interviews, cognitive and biological assessments). Comprehensive assessments that extend beyond traditional questionnaires and checklists to include interviews and cognitive and biological measures are needed to elucidate the negative and positive effects of disaster on the child as a whole. More work is needed to: (1) discover and test theoretical concepts; (2) explore issues related to the biology of stress reactions in children; (3) develop more comprehensive knowledge of the relationships among variables not yet substantiated (e.g., the role of prior trauma in disaster vulnerability); and (4) clarify the role of family and community and societal influences in children’s disaster reactions and recovery. Finally, open-ended, qualitative investigations complement other assessment techniques and allow children to express what they believe were the worst aspects of their disaster experience.