Abstract
Background
Disasters often set the stage for scientific inquiry within the field of occupational safety and health. This is especially true when the long-term consequences of exposures associated with a particular disaster are unclear. However, a responder research study can be costly and difficult to design, and researchers must consider whether the proposed study will produce useful, reliable results and is a prudent public health investment.
Methods
Senior NIOSH scientists, experienced with disaster response and representing the disciplines of epidemiology, occupational medicine and psychiatry, and industrial hygiene, were convened at the request of the NIOSH Director to develop a decision process to help determine when to conduct responder health research following disasters.
Results
The decision process can be broken down into various components, including scientific rationale that should be formally recognized as critical to efficiently and effectively determine whether a research study is warranted. The scientific rationale includes certain controlling or “gatekeeper” factors that should be present to proceed with research. Providing the foundation for responder disaster research also requires strategizing before an event occurs, so that critical baseline and comparison data can be collected.
Conclusions
The recommended framework should ensure that research that is most needed and justified will be identified and prioritized.
Keywords: disaster research, responder research, post-disaster research, surveillance, health monitoring, responder
Introduction
Disasters often set the stage for scientific inquiry that could help mitigate potential short- and long-term health effects among responders, as well as improve capabilities for responding to future events. However, the disaster environment presents many challenges for the conduct of research, including the immediate emphasis on critical response activities, limited access to incident leadership, inability to engage response personnel in scientific research activities not immediately pertinent to the event response, marshalling of necessary resources in an expeditious manner, and timely situational awareness of important occupational safety and health events during the response or recovery operation. The disaster environment is typically dynamic and often hazardous, chaotic, and highly charged with conflicting scientific opinions, political pressures, and disparities in knowledge or the capability to safely perform the required work.1
Emergency response workers and others involved in response, remediation, and recovery efforts (referred to herein as “responders”) are often engaged in non-routine activities in uncharacterized, potentially hazardous environments and may encounter novel or unique exposures or experience unexpected health effects. As such, there is frequently a pressing public health need to conduct research to answer critical questions regarding the health impact of these exposures and/or determine the factors that are responsible for any adverse health effects experienced in both the short- and long-term. Issues related to conducting research during and following disasters have gained increasing attention following the World Trade Center disaster in 2001 and, more recently, the Deepwater Horizon disaster.2,3 For example, the National Biodefense Safety Board, an advisory committee to the Assistant Secretary of Preparedness and Response within the U.S. Department of Health and Human Services (HHS), recently engaged on the topic of inclusion of scientific investigations as a component of disaster planning.4 A responder research study, however, can be costly and difficult to design; therefore, one must carefully consider whether a proposed or requested study will be able to produce useful, reliable results and is a prudent public health investment.
Providing the foundation for responder health research within the context of a disaster and subsequently determining the need to pursue a study requires careful consideration and planning before an event occurs. Unless the potential for such a study is anticipated, the ability to collect critical baseline data can be lost. Research studies need to address clearly stated, important questions or hypotheses, and be appropriately designed to maximize the likelihood of producing a meaningful study. However, the factors and decision-making processes relevant to determining whether to conduct responder health research, or how to best perform such research, have been relatively unexplored.
Background
For the purposes of this paper, health studies conducted in conjunction with the response to emergency events are divided into four basic types: (1) those that involve routine or baseline health monitoring, health surveillance, industrial hygiene or environmental assessments, responder interviews/focus groups, and roster/registry activities (discussed in detail later), optimally planned in a generic way prior to an event (termed Baseline Activities); (2) those that investigate and respond to immediate health problems and exposures, and are designed to expeditiously provide useful and actionable information that directly affects the health and safety of current responders (termed Public Health Investigations); (3) those that are exploratory or preliminary in their approach, often to determine the need for or feasibility of a more comprehensive research study (termed Pilot Investigations), and (4) those that entail a systematic and rigorous investigation, typically require peer-reviewed protocols, usually extend well beyond the duration of the emergency, and are designed to develop or contribute to generalizable scientific knowledge (termed Responder Health Research). Public health activities to prevent or control disease or injury and improve health, or to improve a public health program, are not generally considered research, as the purpose of research is to generate or contribute to generalizable knowledge through a systematic and scientifically defensible process.5
Baseline Activities include health monitoring, health surveillance, documentation of exposures early in and throughout disaster events, and roster/registry activities. Health monitoring refers to ongoing and systematic clinical evaluation of individual responders' health status, while health surveillance is the term used for an ongoing and systematic population-based data collection approach. Baseline activities may include routine exposure assessment, which refers to systematic industrial hygiene or environmental measurements to evaluate workers' exposures to chemical, biological, or physical hazards. Baseline activities also include compiling a “roster,” which is a list with contact information for all responders scheduled to perform a duty. Alternatively, a “registry” may be implemented to collect, store, and manage information about affected populations at risk of exposure or who have developed a specific disease. The absence of rosters or registries, or sufficient, well-organized exposure data, makes subsequent responder health research studies difficult to conduct, and quite probably less accurate because identifying the universe of response and remediation workers and reconstructing potential exposures after a response has concluded can be very difficult if not impossible. While not usually considered research, per se, baseline activities nonetheless provide critical information that is fundamental to supporting any future investigation or study and should be standard practice in any emergency that entails workers entering uncharacterized or uncontrolled environments, regardless of whether a decision to conduct a responder health research study has been made. Additionally, baseline activities often inform the need for responder research, assist in hypothesis generation, and/or help determine the feasibility of the proposed research.
Public Health Investigations are designed to provide actionable information that will have a direct impact on the health and safety of current responders. Public health investigations involve medical/epidemiologic investigations, industrial hygiene assessments, and/or biological monitoring to assess individual exposure levels and health effects. The focus and timing of these investigations are often to resolve questions arising during the course of the response to help mitigate exposure or disease and identify early indications of significant adverse health effects among responders. Because of the time elements inherent in these studies, they primarily address acute effects, conditions, or concerns and are not designed to detect delayed or chronic effects. Typically, these investigations are not considered research because they are not designed to produce generalizable knowledge, although information derived from the investigations can sometimes inform future responses. As with baseline studies, public health investigations may serve to generate or explore hypotheses and stimulate research ideas. Examples of public health investigations include cross-sectional surveys (“snapshot” surveys of workers that collect information about health issues, risk factors, and exposures at the same point in time), cluster investigations (evaluations of reported conditions and illness above the expected level), exposure assessment investigations, case series (tracking individuals by examining similar exposures and health outcomes), and site-specific investigations, such as NIOSH Health Hazard Evaluations.6
Pilot Studies are typically small-scale studies designed to collect information on the feasibility and value of conducting a more comprehensive responder health research study. A pilot investigation also may test, inform or improve the design of an eventual larger research project. Such studies can provide valuable insights; for example, should any research component be identified as missing in the pilot study, it can be added to the larger research study to improve the chances of a clear outcome. Additionally, if insufficient rationale is present to justify a larger research study, pilot research studies can sometimes provide this justification. Pilot studies can be particularly useful in identifying design issues (e.g., a planned procedure or assessment is impractical in the field) before more comprehensive research is conducted. Furthermore, pilot studies may also be considered when data are needed expeditiously because it may not be available later once a larger research study is designed and approved.
Responder Health Research Studies are designed according to a detailed research protocol to test explicit scientific hypotheses about associations between defined exposures (including “psychological exposures”) in the work environment and adverse health outcomes or analogous questions. These studies entail rigorous and/or complex scientific methodology, create generalizable knowledge, typically have peer-reviewed protocols, and usually extend beyond the duration of the emergency. These studies can address acute, sub-acute, and/or chronic effects and conditions. Examples of such studies include (1) cohort studies (designed to follow over time a defined group of individuals who are exposed in varying degrees to a factor that is hypothesized to influence the probability of the occurrence of a given disease or other outcome), and (2) case-control studies (designed to compare differences in exposures and other risk factors among those with an illness/condition (cases) and those without the illness/condition (controls)).
Methods
Upon review of the response activities related to the Deepwater Horizon oil release in the Gulf of Mexico in 2010, a need was identified for a systematic decision-making process to determine if post-disaster responder health research should be initiated. In response to this need, the Director of NIOSH convened a work group consisting of senior NIOSH scientists experienced with disaster response and representing the disciplines of epidemiology, occupational medicine and psychiatry, and industrial hygiene, and that included former directors of the NIOSH emergency preparedness and response program, medical staff involved in the World Trade Center Health Program, and epidemiologists involved in industry-wide studies. Drawing on direct experience in multiple large emergency/disaster responses, including the terrorist attacks of September 11, 2001, anthrax events, severe acute respiratory syndrome (SARS), Hurricane Katrina, and the Deepwater Horizon oil release, the workgroup collaborated to devise a comprehensive rationale for determining when to conduct responder health research studies.
An extensive literature search, including MEDLINE,®EMBASE,® Cumulative Index to Nursing and Allied Health Literature (CINAHL®), Health and Safety Science Abstracts, and NIOSHTIC, yielded no publications directly addressing this topic. Search terms including “emergency responder, disaster worker, emergency worker, first responder” and “long-term study, post-disaster study, surveillance, health monitoring” yielded numerous studies that investigated the health status of emergency response workers in various response scenarios; however, none of these studies addressed the thought processes or criteria involved in the decision to conduct such a study, and no general treatments of this issue were identified. The Agency for Toxic Substances and Disease Registry (ATSDR), however, has developed guidance for determining if an environmental assessment should be conducted in a non-emergency situation.7 The National Biodefense Science Board recently released a report discussing the need for disaster research and the significant barriers to conducting disaster research, but this report does not specifically address the decision process on whether to proceed with research.4
After defining the types of responder health studies and activities pertinent to a disaster response, the work group discussed previous emergency response studies as well as current trends in emergency response research methodologies. While considering various criteria potentially necessary for determining the need for a study, it became clear that developing a focused hypothesis (or hypotheses) based on varying pre-existing information would be a critical component of an effective decision-making process. The work group then discussed factors that should be evaluated to assess whether a study was warranted, including issues such as feasibility and research significance. Using these considerations, the group conducted a table top exercise (described later in the paper) to test the validity of this decision-making process.
This paper summarizes the Work Group's recommendations for determining when a responder health research study should be initiated, including the critical factors that need to be considered when making the decision to carry out research.
Results
Hypothesis Development for Responder Research
As described above, a responder health research study is designed to address a specified scientific query, which is often expressed as a hypothesis or set of hypotheses. Hypotheses may arise from the results of recently concluded or on-going public health investigations begun during or just after an emergency response. Hypotheses may also be derived from data produced by health surveillance, individual worker health monitoring efforts, pilot studies, or previous research. An ideal scientific hypothesis should be testable and precise in its construction, make specific and unambiguous predictions, and clearly define the intent of the research study. Research studies should be designed to address potential confounders and have sufficient statistical power to detect a meaningful effect.
The hypothesis or set of hypotheses under study may be expressed or operationalized in different ways, and tends to vary in a manner consistent with a given scientific discipline. Additional discussions on hypothesis development and research project design are available elsewhere.8,9,10
Improperly formulated hypotheses or inappropriately applied theory can be detrimental to the overall research initiative. For example, when broad concerns are voiced about potential health effects in a disaster, a research organization may translate the problem into one that can be investigated with available resources. While the translation often narrows and simplifies the focus to something manageable and potentially resolvable, researchers need to be vigilant to ensure that the hypothesis is constructed so that it remains capable of evaluating the original health concerns. If the research is capable of only addressing a narrow aspect of the original health concern, or if it can only address a tangential issue, then serious consideration is necessary as to whether it is prudent to proceed with the research. Alternatively, a research organization may attempt to initiate research around an overly expansive or complex hypothesis in an attempt to satisfy the original broad health concerns. This may result in research that is unfocused, inefficient, subject to constant ad hoc modification, and result in scientifically ambiguous results or conclusions.
Factors to Consider When Proposing a Responder Health Research Study
The various factors to be considered when determining if a responder health research study should be conducted are presented below. These factors can be broadly organized into six categories: 1) exposure-related, 2) observed adverse health events, 3) public health significance and scientific importance, 4) societal considerations, 5) feasibility, and 6) level of research interest. Several of these factors were adapted to an occupational context from the previously referenced ATSDR guidance document.7
- Exposure-related factors –
- Presence of actual exposures to hazardous substances, conditions, trauma, etc.
- Existence of unique, novel, or unusual exposures
- Presence of complex environments or combined exposures
- Potential implications of exposures on worker health
- Types of science/research methodologies necessary to address/answer exposure questions
- Adverse health event-related factors –
- Observance or anticipation of unique, novel, particularly serious, or unusual adverse health events
- Presence of unexpected or unforeseen occupational health issues that are observed or may manifest during an event
- Presence of higher than expected numbers or rates of a specific adverse health event – or of overall events
- Occurrence of adverse health problems associated with exposures below applicable occupational limits
- Public health significance and scientific importance –
- Ability to provide new knowledge or information about an exposure-outcome relationship
- Ability to evaluate specific exposures or outcomes that have not been adequately studied
- Ability to generalize to other situations or populations
- Ability to confirm or refute a preliminary or pre-existing hypothesis or theory
- Ability to answer questions that need to be answered and cannot be answered in any other way
- Ability to contribute to or directly improve the public health response to disasters
- Magnitude of event, for example, a large number of workers exposed or considered at risk
- Societal factors –
- High profile or traumatic event
- Beliefs about harm or resource disparities, particularly among high-risk groups
- Unique vulnerability of the worker population
- Socioeconomic, legal, political, and psychological implications of the event
- Feasibility factors –
- Access to the work site(s)
- Ability to quickly collect reliable data, particularly if data could be lost if not collected immediately
- Ability to document or validate human health outcomes
- Ability to reasonably estimate or document individual exposure
- Ability to assign workers into exposure categories to permit exposure-response assessment
- Adequate study size and statistical power
- Ability to identify and locate subjects and records
- Availability of an appropriate control or comparison population
- Ability to address potential confounding factors
- Ability to measure and disentangle the relevant environmental, behavioral, or other factors
- Adequacy of resources to support, conduct, and complete the study
- Adequacy of support from employers and unions or other relevant stakeholders (e.g., other federal agencies, state or local agencies or components, trade groups, etc.)
- Ability to provide participants with necessary confidentiality (for instance, in research psychology)
- Ability to address potential ethical issues and obtain expeditious Institutional Review Board (IRB) approval for time-sensitive research
- For federal agencies, ability to obtain timely emergency clearance from the Office of Management and Budget (OMB) for survey instruments that fall under the jurisdiction of the Paperwork Reduction Act
- Adequacy of preliminary or baseline data to support the study (this is implied in some of the above bullets)
- Level of research interest –
- Research arising from academic/research areas of interest
- Contribution to established institutional program goals, such as emergency response research priority areas
The Work Group conducted a tabletop exercise to test and refine the above factors and assess their utility to adequately inform the decision to conduct a responder health research study. The March 11, 2011, earthquake and tsunami that devastated northeastern Japan and damaged the Fukushima Daiichi nuclear power facility was the example used for this exercise. The Work Group focused specifically on the release of radioactive contamination from the severely damaged Fukushima Daiichi facility into the surrounding environment and the ensuing potential for radiation exposures to workers during efforts to contain the damage. Health Physicist subject matter experts participated in the exercise to assist with defining the research needs associated with worker exposure to ionizing radiation. The exercise revealed that the above set of factors/questions served as a useful tool in determining priority research questions resulting from the incident, helping to elucidate various information, exposure, and resource needs, as well as public health significance. For example, it was determined that this event would provide a unique opportunity to address a number of questions of significant public health importance, including the adequacy of current exposure standards, issues regarding chronic exposure, impact of work involving long-term decontamination, utility of chromosome and biomarker data for evaluating effects on different exposure cohorts, and psychological effects on responders. The exercise thus suggested that research could readily be justified, but the group did not have access to information that would have allowed it to consider other factors, such as feasibility.
This exercise further illuminated the need to conduct pre-planning in order to provide a timely science response to a large-scale disaster. For example, such pre-planning could focus on predictable logistical needs, resource identification, interagency coordination, mechanisms for the systematic collection of data, etc. for a variety of possible disaster scenarios. All of these are fundamental requirements for conducting a research study, and waiting to address these issues until post–event can result in significant delays, inefficiency, and affect the likelihood of a successful study.
Proposed Process for Determining When to Conduct a Responder Health Research Study
The process for determining whether to conduct a research study is informed by multiple inputs and considerations, as illustrated in Figure 1, but should begin with the development of a scientific query, expressed as a hypothesis (or hypotheses). Public health investigations that evaluate exposures and health outcomes among responders during the event, or health monitoring and surveillance efforts that document the health status of the responders, may result in one or more scientific hypotheses worthy of further study. Pilot investigations that are exploratory in nature may be hypothesis-generating as well. Once the hypotheses have been developed, it is important to evaluate them in a logical and transparent manner to determine if they truly merit the time and investment required for a research study.
The hypotheses will be informed by the “factors to consider” that were described in the preceding section. This is a complex process entailing considerable expert scientific judgment and should be conducted by a science planning team (described later). The process, however, should be thorough, deliberative, systematic, and transparent, considering as many of the factors as possible based on available information.
Although consideration of the factors generally occurs after hypotheses have been generated, it may be necessary to evaluate some or all at earlier stages, and many of the factors can be evaluated in parallel. Thus, for example, exposures measured or inferred during the event or observed adverse outcomes (e.g., a cluster of cases) may identify significant knowledge gaps, question current understanding, and inform the need for a responder research study. Feasibility issues, such as access, support from workers and agencies, etc., are other examples of factors that can be considered prior to committing resources for more extensive studies.
While there are no predetermined thresholds for these factors, such as a particular level of increased incidence of adverse outcomes or degree of excessive exposure that necessarily trigger a decision for or against conducting a research study, it is evident that there are a several controlling (or “gatekeeper”) factors that must be satisfied for post-disaster responder research to proceed (see Table 1). If any of these gatekeeper factors are not adequately satisfied, a responder research study should not proceed, or at a minimum, the research should be seriously reconsidered. It will often be advisable for the initial activities to be limited to baseline activities or a pilot investigation to evaluate study feasibility or help determine whether health concerns justify a full-scale study. Even if the gatekeeper factors are met, prudence dictates that pilot investigations should be considered prior to the initiation of large, expensive responder health research projects.
Table 1. Critical gatekeeper factors for determining if post-disaster research should be conducted.
Critical Gatekeeper Factors | |
---|---|
Scientific Query |
|
Exposure-Related |
|
Study Design |
|
Feasibility Factors |
|
A critical gatekeeper factor that significantly affects the feasibility of responder research following a large-scale disaster is the ability to identify and locate subjects and records. Thus, development of a responder registry or roster is particularly important, especially when the proposed research involves large numbers of responders. The absence of a registry or roster makes research studies difficult to conduct and possibly less accurate, because identifying the universe of response and remediation workers after the response can be very difficult, if not impossible. If only a subset of workers is identified and recruited as participants, they may not be representative of the entire workforce, and the study results will likely be skewed.
Discussion
Because of the many complexities associated with both the decision to conduct responder research and the actual conduct of responder research, analyzing a prioritized set of potential disaster scenarios could help identify research opportunities. For example, it could be useful to evaluate research questions and knowledge gaps likely to arise and be amenable to study during a variety of incident types, including those foreseen in Federal Emergency Management Agency (FEMA) National Planning Scenarios.11 Careful analysis in the planning stages enables the appropriate application of multidisciplinary expertise to identify scientific gaps, formulate well-defined hypotheses, and design potential studies within the context of the life cycle of a disaster and the requisite phases of emergency management. Planning includes pre-identification of subject matter experts in a variety of scientific disciplines and pre-determination of data needs and systems for obtaining the necessary data (e.g., exposure assessment, worker rosters, etc.).
To facilitate the decision process, the organization or agency contemplating research should, as part of its pre-planning efforts, create a science planning team of experienced subject matter experts from diverse scientific disciplines (e.g., occupational medicine, industrial hygiene, toxicology, epidemiology, psychology, emergency response). The team should be substantially separate from those primarily responsible for coordinating the immediate response. Once a disaster commences, the science planning team should be rapidly convened to review, refine, and prioritize any previously developed generic research plans to identify potential research needs, including a statement of rationale and importance. Scope of the effort (including approximate numbers of staff and potential costs) should also be estimated. This information should be reviewed and a determination made as to whether the research should be pursued. In some cases, the level of research interest may inform determinations. A pre-determined level of research interest may be reflected in an agency or institution's research agenda.12
The science planning team, which can be supplemented by specific subject matter experts depending on the type of disaster, should be able to evaluate the information available and come to a consensus on whether there are valid, testable, and relevant hypotheses or appropriate theoretical foundations, and whether a responder research study can feasibly be conducted. The team should also be required to periodically reassess the need for conducting responder research because of the rapidly evolving nature of disasters and worker exposures.
Disaster events, particularly those where responder health research is anticipated, should routinely include collection of a core set of baseline data, such as exposure data, rosters of exposed individuals, and baseline health status. Failure to begin collecting critical data as soon as possible can result in data that are unavailable, lost, or cannot be reconstructed, which can significantly affect study feasibility. Research plans as described above should include provisions for anticipatory data collection, including standard instruments for emergency or generic OMB clearance (in the case of federally sponsored research) and advance preparations for IRB approval, even if it is ultimately decided not to move forward with a research study.
As part of planning efforts for research initiatives, strong interagency communication and cooperation will be needed among federal, state, and local governmental agencies (e.g., health departments, workers compensation bureaus, labor departments, Federal Emergency Management Agency) in the jurisdiction of the disaster. These discussions may be expedited prior to an event through discussions with relevant umbrella organizations, such as the Council of State and Territorial Epidemiologists, Association of State and Territorial Health Officials, and National Association of County and City Health Officials.
Finally, discussions with other public safety, public service, and nongovernmental organizations, including labor unions and educational institutions, are necessary to establish expectations, roles, and responsibilities related to research that may be considered during or following a disaster. In order for research projects—especially those requiring data collection from a large number of people—to go forward efficiently, it is imperative that workers and other participants understand the importance of the work, trust that it is being carried out for their or for society's benefit (and not solely to fulfill institutional needs), and understand that the results will be shared with them when the data are collected and analyzed. Moreover, the affected workers themselves will be an asset in many projects, supplying specific knowledge and skills, including appropriate language and cultural sensitivities, which can strengthen data collection efforts.
Conclusions
The decision process on whether to proceed with a responder health research study can be broken down into various components that should be formally recognized as critical to determining whether a research study of responder health is warranted. As described in this paper, the decision process should be formalized to ensure the best possible judgment and recommendation, and may benefit from advanced planning in anticipation of possible events. Several critical gatekeeper factors are provided that must be satisfied if a research study is to be initiated. A determination to conduct a study without sufficient information to develop the exposure-outcome relationship or without an initial hypothesis is not generally recommended and may be subject to ethical concerns. In the current austere fiscal environment, the recommended framework should ensure that research that is most needed and justified will be identified and prioritized.
Table 2. Critical gatekeeper factors for determining if post-disaster research should be conducted.
Critical Gatekeeper Factors | |
---|---|
Scientific Query |
|
Exposure-Related |
|
Study Design |
|
Feasibility Factors |
|
Contributor Information
John A. Decker, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health, Atlanta, Georgia.
Max Kiefer, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health, Denver, Colorado.
Dori B. Reissman, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health, Washington, DC.
Renée Funk, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health, Atlanta, Georgia.
John Halpin, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health, Atlanta, Georgia.
Bruce Bernard, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health, Cincinnati, Ohio.
Richard L. Ehrenberg, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health, Atlanta, Georgia.
Christine R. Schuler, Disease Control and Prevention, National Institute for Occupational Safety and Health, Morgantown, West Virginia.
Elizabeth Whelan, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health, Cincinnati, Ohio.
Kyle Myers, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health, Atlanta, Georgia.
John Howard, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health, Washington, DC.
References
- 1.Jackson BA, Baker JC, Ridgely MS, Bartis JT, Linn HI. Protecting emergency responders: safety management in disaster and terrorism response. [Accessed August 8, 2011];2004 DHHS (NIOSH) Publication No. 2004-144, Rand Publication No. MG-170. Available at http://www.cdc.gov/niosh/docs/2004-144/pdfs/2004-144.pdf.
- 2.Government Accountability Office. September 11: HHS needs to develop a plan that incorporates lessons from the responder health programs. [Accessed August 8, 2011];2008 May; GAO-08-610. http://www.gao.gov/new.items/d08610.pdf.
- 3.Kitt MM, Decker JA, Delaney L, Funk RM, Halpin J, Tepper A, Spahr J, Howard J. Protecting workers in large-scale emergency responses. JOEM. 2011;53(7):711–715. doi: 10.1097/JOM.0b013e31822543b6. [DOI] [PubMed] [Google Scholar]
- 4.Department of Health and Human Services, Office of the Assistant Secretary for Preparedness and Response, National Biodefense Safety Board. Call to Action: Include Scientific Investigations as an Integral Component of Disaster Planning and Response. Available at http://www.phe.gov/Preparedness/legal/boards/nbsb/recommendations/Pages/default.aspx.
- 5.Protection of Human Subjects, 45 Code of Federal Regulations, part 46.
- 6.National Institute for Occupational Safety and Health, Health Hazard Evaluation Program. 2011 Sep 23; Available at http://www.cdc.gov/niosh/hhe/
- 7.Agency for Toxic Substances and Disease Registry. Guidance for ATSDR Health Studies. Apr 1996; Available from the Division of Health Studies, Agency for Toxic Substances and Disease Registry, 4770 Buford Highway, Mailstop F-57, Atlanta, GA 30341-3717. [Google Scholar]
- 8.Popper K. The Logic of Scientific Discovery. New York: Basic Books, Inc.; 1959. [Google Scholar]
- 9.Singleton RA, Straits BC. Approaches to Social Research. 3rd. New York: Oxford University Press; 1999. [Google Scholar]
- 10.Rothman KJ, Greenland S. Modern Epidemiology. 2nd. Philadelphia: Lippincott-Raven; 1998. [Google Scholar]
- 11.FEMA Fact Sheet, National Planning Scenarios. [Accessed September 23, 2011]; Available at http://www.fema.gov/pdf/media/factsheets/2009/npd_natl_plan_scenario.pdf.
- 12.NIOSH Emergency Preparedness Program Goals. [Accessed September 23, 2011]; Available at http://www.cdc.gov/niosh/programs/epr/goals.html.