Lack of Evidence Supporting the Effectiveness of Disaster Supply Kits

Abstract

We reviewed the available evidence in support of the effectiveness of disaster supply kits presently used in household emergency preparedness in the United States. The expectation that people should take responsibility for their own disaster preparedness has largely not taken into account contextual influences on disaster preparedness.

The efficiency of current disaster supply kits used during critical postdisaster periods has not been empirically tested. Professional recommendations regarding the composition of disaster supply kits containing at least water, food, first aid, hygiene, and clothing have not been universally defined. This lack of consensus may lead to the assembling of disaster supply kits yielding suboptimal results.

The use of disaster supply kits should continue to be nationally recommended, although additional research is needed to demonstrate their beneficial impact on survival and resilience after a disaster.

Disaster research in the United States over the past decade on individual or household preparedness has found that the majority of the population under study, vulnerable or not, no matter the demographic characteristics or geographic location, is unprepared for a disaster.^1–8 Only two studies revealed that a majority of the participants were considered adequately prepared for survival; data collection was limited by the timing because of the urgent circumstances of the hazardous conditions. One investigation was conducted during hurricane season, shortly after a hurricane,⁹ and the other was conducted after a household emergency preparedness media campaign with a subsequent tornado outbreak.¹⁰

Household emergency preparedness is a public health concern, and lack of preparedness, especially in medically frail populations, consumes responders’ time, taking them away from relief and recovery efforts, and can easily deplete community health resources.³ Better training in household emergency preparedness could save lives, prevent worsening of chronic medical conditions, and decrease the likelihood of responders having to brave dangerous situations to assist those in need. Community education and disaster preparedness have been credited with saving lives in disasters.¹ A single accepted definition does not seem to exist for household emergency preparedness, and the concept can be used to describe either an individual’s or a household’s preparedness.

The most popular definition used in public health preparedness literature,^{1–4,6,7,9-12} comes from the Federal Emergency Management Agency’s (FEMA) Ready campaign, which states that individuals or households are prepared for a disaster if they have thought about and planned for the types of disasters for which they are at most risk, have developed a family communication and evacuation plan in the event of a disaster, and have assembled a complete disaster supply kit (battery-powered weather radio, flashlight and extra batteries, first aid kit, whistle, dust mask, plastic sheeting, duct tape, personal sanitation items, wrench, pliers, manual can opener, and local maps) that can sustain each member of the household with food, water, and medication for up to three days without any outside assistance.¹³

Al-Rousan et al. expanded the FEMA definition, adding that families and individuals should also have working smoke detectors, more than one exit in the home, and the ability and knowledge of how to turn off utility connections if needed.² Reininger et al. added access to emergency funds, ability to obtain disaster alerts, possession of adequate social support networks, and aptitude to access and leverage community resources.⁸ Murphy et al. added the need to have at least a half-tank of gasoline in a personal vehicle.⁷ The US Department of Health and Human Services combined the FEMA supplies list with that of the American Red Cross.¹⁴ Regardless of which definition of household emergency preparedness was used, all definitions included a disaster supply kit.

Using the disaster supply kit as a variable in household emergency preparedness research can be problematic. Literature is lacking to support the claims that having a disaster supply kit results in self-sufficiency or contributes to disaster-related resilience. Additionally, literature is lacking on how the contents of the kits were determined. Perman et al. conducted a literature review to examine published disaster supply kit lists to determine differences, commonalities, specificity, and comprehensiveness.¹⁵ They discovered 71 different published checklists, with a huge disparity in the number of recommended contents, the specificity of the items, and the time frame needed to be self-sufficient. The only item that all 71 of the checklists recommended was water.

It has been established that no evidence exists for determining what should be in the disaster supply kits or determining whether having the kits results in self-sufficiency or disaster resilience. Nonetheless, the kits are still used as one of the primary variables for household emergency preparedness research. The quantitative studies used surveys, instruments, and objective disaster preparedness checklists to determine whether respondents were prepared. They sought to assess the degree of household emergency preparedness and make correlations between demographic characteristics and levels of preparedness. One major limitation in most of these studies was having the ability to find only correlations and not causations because these studies were observational and nonexperimental. Also, none of the researchers have been able to describe the majority of the variance for why people are or are not prepared. Quantitative studies are unable to glean the social, economic, and environmental influences that have an impact on preparing for disaster with a supply kit.¹⁶

There are also issues with measuring household emergency preparedness via checklists or instruments. The first issue is that there is no gold standard household emergency preparedness instrument. Three potential instruments were identified: (1) the Behavioral Risk Factor Surveillance System Questionnaire 2012 Module 19: General Preparedness Module¹⁷; (2) the California Health Interview Survey 2009 Adult Questionnaire Version 3.4: Emergency Preparedness Module¹⁸; and (3) the Readiness Quotient.¹⁹

Unfortunately, no official publications detailing the original instrument development of the Behavioral Risk Factor Surveillance System Questionnaire 2012 Module 19: General Preparedness Module or the California Health Interview Survey preparedness modules were available for review. The authors of the Readiness Quotient provided an instrument development publication but did not provide the psychometric properties of the instrument. Thus, it is impossible to discern the conceptual basis that guided the development of these instruments, critique the methods used to generate items, compare the items to an original concept definition, or evaluate the psychometric properties. Instrument development information needs to be shared, and if it does not exist, psychometric testing needs to be conducted immediately and with each subsequent study using these instruments.

The second problem with measuring household emergency preparedness via checklists or instruments is that household emergency preparedness is not a stable construct but is considered a dynamic characteristic. This means that household emergency preparedness level is expected to change over relatively short periods and from one situation to another, such as during hurricane season or immediately after a disaster. Because disaster supply kits are not empirically confirmed, using supplies kits as an indicator of household emergency preparedness calls into question the validity of the instruments. Validity refers to whether an instrument measures what it purports to measure or the ability of the instrument to measure the attributes of the phenomenon of interest.²⁰

It has not been empirically established that being prepared for disaster with a disaster supply kit results in actually surviving a disaster without need for outside assistance. Culture bias is also a major factor that threatens the validity of household emergency preparedness instruments. Culture bias is manifested when an instrument is not conceptually equivalent for cultural subgroups.²⁰ It is very likely that there are cultural differences in preparing a household for disaster, even between responders and lay people living in the same community. The FEMA definition of preparedness comes from educated experts with extensive experience in disaster response and recovery. The lack of conceptual equivalence is likely the reason for the low level of preparedness seen in the studies. Perhaps the most appropriate form of validity for instruments to measure household emergency preparedness, predictive validity, has not yet been explored.

Knowledge of the factors that affect how individuals prepare for a disaster would be advanced with qualitative studies.¹⁶ Although often criticized for not being generalizable, qualitative studies provide participants a venue to voice their concerns and make suggestions that could persuade the consumer of the research to take action on the basis of the findings. Qualitative methodologies are credible research designs because they can establish the causal relationship of barriers to effective disaster preparedness that participants face.²¹

Few qualitative studies on the topic of household emergency preparedness have been conducted in the United States.^5,10-12,22 These studies offered insights into both community members’ and responders’ perceived barriers and potential solutions to increase household emergency preparedness. The authors of these qualitative studies also found that a majority of their participants were unprepared for a disaster.

However, there are several major barriers to adequately preparing a disaster supply kit:

1. people lack the knowledge of how best to prepare,^{2,3,7,10,11,16,22}

2. people lack the means to purchase and maintain necessary supplies,^{2,7,11,15,22,23}

3. people feel that it is inconvenient to maintain necessary supplies,^5,23

4. household emergency preparedness information is often written only in English,^11,22,23

5. household emergency preparedness information is often written at a too advanced literacy level,¹¹

6. Internet-based disaster preparedness education campaigns do not reach people who lack Internet access,^11,15

7. people do not have adequate space to store disaster supply kits,^11,15

8. people do not believe that they are at risk (e.g., for death, injury, property damage, and disruption to daily activities),²³

9. people do not believe that they are able to effectively mitigate risks,²³

10. people already living in survival mode cannot be convinced to prepare for an event that may never happen,²⁴

11. people have an unrealistic sense of optimism, do not take threats seriously, are in denial, or feel subjectively prepared but are not actually prepared objectively,^1,15

12. people are medically frail,⁴

13. household emergency preparedness information demonstrates a lack of consensus,¹⁵ and

14. people feel that there is no point in assembling a kit because it may not be accessible (e.g., in a building collapse, flood, fire, or precipitous evacuation).

Facilitators to assembling a disaster supply kit have also been noted in the literature. For example, individuals are most likely to prepare for disaster when they have identified that a risk exists and have a combination of perceived high vulnerability, perceived high self-efficacy for preparing for disaster, and perceived low costs.^5,7,23 Age, past experience with disaster, and gender were found to be either barriers or facilitators to disaster preparedness, depending on the study.

The literature has revealed an expectation that people will take responsibility for their own disaster preparedness but has largely not taken into account contextual influences on disaster preparedness.¹⁶ The composition and use of disaster supply kits has not been empirically tested. Instruments with items that represent the contents of a disaster supply kit to determine level of household emergency preparedness effectiveness may not actually be valid measures of household emergency preparedness. Disaster supply kit recommendations are not uniformly defined, but contents needed during the most crucial postdisaster period include water, food, first aid, hygiene, and clothing.¹⁵ Disaster supply kits are difficult for people to assemble because of significant barriers. The facilitators to assembling a kit include clear and consistent recommendations for the contents, people understanding that a risk exists, and people understanding that they are able to do something to help alleviate that risk, as long as it is convenient and not too costly.

The composition and use of disaster supply kits should continue to be recommended, but more research is needed on how they affect survival and resilience after a disaster. Household emergency preparedness research is still mostly in the assessment stage, with plenty of work still to be done on discovering which populations are prepared and which are unprepared and how to target specific populations for communication about disaster preparedness. Research needs to address where preparedness gaps are rooted.^1,2,8,10,24 Bethel et al. call for studies to investigate why vulnerable groups do not have complete disaster preparedness kits, particularly those living in areas prone to natural disasters.⁴ McCormick et al. suggest exploring the impact of a disaster event on the levels of personal preparedness.⁶

Research still needs to be conducted to determine whether disaster supply kits actually result in self-sufficiency or contribute to disaster-related resilience. Once a gold standard household emergency preparedness instrument is developed and adequately tested, it should be used to determine whether there is an association between being prepared for disaster and surviving the disaster without the need for rescue or outside assistance. For medically frail community members, it should be determined whether there is an association between being prepared for disaster and surviving the disaster without an acute exacerbation of a chronic illness and with no change in baseline functional status. Additional qualitative work is needed to acquire data that will contribute to developing an in-depth theoretical understanding of why people do or do not prepare for disaster.¹⁶

A gold standard household disaster supply kit recommendation list needs to be developed, embraced, and disseminated through trusted disaster information sources.¹⁵ Campaigns should be expanded throughout social media and disseminated via television, radio, newspaper, community- and faith-based organization newsletters, and mass mail. These various methods of communication should have a consistent message and be offered in several languages and at a low literacy level. Public health professionals should educate their communities on household emergency preparedness. The problem of knowledge and means could be resolved if premade disaster kits with illustrated evacuation instructions were distributed to vulnerable community members on request. Public health spending would have a dual purpose by simultaneously directing resources to the people most likely to need assistance during disaster and increasing a community’s resilience.²⁵

Household emergency preparedness is a dynamic, multifaceted, large-scale public health concern that needs to be addressed. Community education and disaster preparedness, when carried out properly, are credited with saving lives in urgent circumstances, whereas insufficient disaster preparedness has cost lives.¹ This is a public health concern that has not been successfully addressed yet but shows promise for collaboration between responders, government agencies, and the community because preparedness is a universal concern.²⁵