Implementation and Evaluation of the Value of Improved and Sustained Information Access By Library Expertise (VISIBLE) Program

Abstract

Introduction

This study’s purpose was to increase knowledge of National Library of Medicine (NLM) resources using a train-the-trainer (TTT) approach.

Methods

Workshops were held in spring, 2016 to increase knowledge of four NLM tools. Data was collected both before and after the workshop; post-intervention data was collected immediately, 3 months, and one year after the workshop. Knowledge questions were scored as 1 point per question; an aggregated knowledge score could range from 0 – 16 points. A paired t-test assessed a change in knowledge from pre- to post workshop.

Results

Four workshops were hosted, with a total of 74 attendees. The response rate for the surveys ranged from 50%–100%. Knowledge scores changed significantly from 7.2 to 11.9 (t= 15, p < .001). One year after the workshop, more participants reported having informally trained others compared to those who had provided at least one formal training session (56.8% vs. 8.1%, respectively; p < .001).

Discussion

Based on objectively measured knowledge and information dissemination, the NLM workshop was successful and resulted in both short- and long-term gains.

Conclusion

This workshop could be repeated in other populations to further disseminate information regarding the NLM tools, which could help improve disaster response.

Introduction

Disasters claim a large number of lives through the destruction of the physical environment and disruption to the social and economic structures of a community. For example, the 2010 Haitian earthquake resulted in more than 200,000 deaths and displacement of another 1.5 million individuals.¹ According to the United Nations Office for Disaster Risk Reduction², between 1995 – 2015, more than 4.1 billion individuals have been affected and more than 606,000 have died due to disasters. Having a more prepared and resilient community can minimize the impact of disasters, but this requires collaborative planning prior to an event and access to appropriate resources after a disaster occurs.^3,4

An important resource during disasters is having access to critical health information relevant to the event. Without access to such information, emergency responders’ effectiveness is limited. Libraries and librarians play a critical role in emergency management, because the information they provide can improve disaster preparedness and response.⁵ Librarians can serve as institutional supporters, collection managers, information disseminators, internal planners, community supporters, government partners, educators and trainers, and information community builders in any phase of disaster and emergency management.⁶ For example, librarians can develop accurate and current disaster and health online resources prior to an event that can be accessed and disseminated to emergency responders, health professionals, and the public during a disaster.⁶ As educators and trainers, librarians can also provide training for emergency responders in the use of emergency reference services and tools.⁶ Emergency responders’ critical information needs and librarians’ skills in mobilizing and making health information accessible call for the close collaboration between these two professions both before and after a disaster occurs.⁷

The National Library of Medicine (NLM) created the Disaster Information Management Research Center (DIMRC) in 2008 to collect, organize, and disseminate critical health information to emergency responders, health professionals, and the general public during mitigation, preparedness, response, and recovery from disasters.⁸ The NLM’s DIMRC resources include training courses, databases, research and after-action reports, guidelines, and preparedness and response tools.⁹ The NLM also provides training for librarians to become Disaster Information Specialists; this specialized education prepares librarians to serve as support for providing information related to disaster medicine, emergency management, and public health preparedness.⁴ It is vital that emergency responders and librarians be made aware of and trained in the use of these NLM’s DIMRC resources so that they can be actively incorporated into disaster-related training and reference materials. However, training is needed to increase awareness of the DIMRC and the free NLM tools.⁹ In addition, there is a need for emergency responders and librarians to collaborate, so that both professions become aware of the others’ skills and capabilities.⁹ This multidisciplinary collaboration should result in better prepared emergency responders and more resilient communities.

This study’s primary purpose was to increase awareness and knowledge of NLM resources among emergency responders and librarians, and evaluate the impact of this education on individual competence. Secondary aims were to determine whether dissemination of DIMRC materials could be increased using a train-the-trainer model and expand the role of Missouri librarians by partnering with local emergency responders in disaster planning efforts. This project’s title is the Value of Improved and Sustained Information Access By Library Expertise (VISIBLE).

Methods

VISIBLE consisted of four workshops held in St. Louis, Kansas City, Springfield, and Jefferson City, Missouri in spring, 2016 to increase knowledge of the following NLM DIMRC tools: Wireless Information System for Emergency Responders (WISER), Radiation Emergency Medical Management (REMM), Chemical Hazards Emergency Medical Management (CHEMM), and Toxicology Data Network (TOXNET). The workshops were multi-disciplinary and included emergency managers, disaster planners, public health professionals, and librarians. Individuals were recruited using posters/flyers that were emailed to potential participants. Multiple organizations aided in distribution of recruitment materials: St. Louis Area Regional Response System (STARRS), Mid-America Regional Council (MARC), Missouri’s State Emergency Management Agency (SEMA), Boone County Department of Emergency Management, Health Sciences Library Network of Kansas City (HSLNKC), Midcontinental Chapter of the Medical Library Association (MCMLA), Missouri Library Association (MLA), MOBIUS Consortium, and the St. Louis Chapter of the Special Library Association (SLA).

Intervention/Workshop

The VISIBLE workshop consisted of five scenarios, the first of which was conducted “cold” (i.e., prior to education about the NLM tools). Next, workshop attendees were provided information about the NLM tools and walked through examples of how and when to use them. Participants were then given four additional scenarios to work through that required use of the tools. The scenarios consisted of a cesium-137 theft (the “cold” scenario), a train crash involving release of radiological material, a chemical suicide in a car, sarin gas release on a bus, and a dirty bomb explosion in an urban area. The scenarios were customized for each workshop through incorporation of local landmarks to make the scenarios more realistic. Workshop seating was arranged so that each group was multidisciplinary and would facilitate small groups discussions that would force team members to work together. The workshop utilized a train-the-trainer (TTT) format that consisted of providing the workshop’s slides, handouts, and scenario information to the participants with the hope that they would share the workshop materials formally or informally with others.

Data Collection and Variables of Interest

Data was collected both before and after the VISIBLE workshop using a pre/post-intervention study design. Post-intervention data was collected during three time frames: immediately after the workshop, 3 months later, and one year later. Data collected included: 1) awareness and use of the NLM DIMRC materials, 2) perceived confidence in accessing and using the NLM tools, 3) knowledge of the NLM WISER, REMM, CHEMM, TOXNET tools, 4) intent and extent to which workshop participants disseminated their knowledge through formal and informal trainings, and 5) reasons why the information was not shared, when applicable. Awareness and perceived confidence were measured on a five-point Likert-scale. Knowledge was measured by 16 true-false and/or multiple-choice questions, and all knowledge-related question content was covered during the workshop. In addition, participants were asked eight questions related to the perceived quality and usefulness of the workshop; items were measured using 5-point Likert scales. The workshop was pilot-tested with a group of 31 graduate public health students and faculty from Saint Louis University prior to study implementation. The Saint Louis University Institutional Review Board approved this study.

Data Analysis

All data analyses were performed using Statistical Package for the Social Sciences (SPSS®) 24.0. Knowledge questions were scored as 1 point per question, with the subsequent calculation of a knowledge score that could range from 0 – 16 points. Descriptive statistics were conducted with all variables. Wilcoxon signed rank tests were run to compare pre-workshop responses to immediate and one-year post-workshop responses for awareness and perceived confidence. Chi square tests were used to compare perceived confidence across use of the NLM tools. A paired t-test was conducted to assess a change in knowledge from pre- to immediate-post workshop. Chi square tests were used to compare intent to share training materials formally versus informally at each time period; when cell counts were below 5, a Fisher’s exact test was used. A Bonferonni correction was used for the pre-post awareness and perceived confidence comparisons, resulting in an adjusted critical value of .008 for these tests (calculated as .05/6 comparison tests). A critical p value of .05 was used for all other analyses.

Results

Four workshops were hosted, with a total of 74 attendees; participation ranged from 11 – 28 per workshop. Just over half (56.8%, n=42) were male and most (90.5%, n=67) were white. Twenty percent (n=15) of the attendees were librarians; the remaining 80% were first responders, public health professionals, emergency managers, and healthcare professionals. More than half (62.2%, n=46) identified as being in a management role. A little more than a third (37.9%, n=29) had a master’s degree or more education or a bachelor’s degree (37.9%, n=29). Approximately one-half (51.4%, n=38) had 11 or more years of work experience. Of the 15 librarians, one-third (n=5) were reference librarians and the remaining were public librarians. None of the librarians reported holding a certificate in Disaster Information Specialization (DIS) from the Medical Library Association. Of the first responders, public health professionals, emergency managers, and healthcare professionals (n=59), about a quarter (28.8%, n=17) were a current or past member of a hazardous materials (HAZMAT) team. Participant demographics are outlined in Table 1. The response rate for the surveys ranged from 50%–100%. All participants (N=74) completed a pre- and immediate-post workshop survey; 58% (n = 43) completed a 3-month post-workshop survey, and 50% (n = 37) completed a one-year post-workshop survey.

Table 1.

Participant Demographics

Individual Characteristics	N=74 % (n)*
Gender (male)	56.8 (42)
Race (white)	90.5 (67)
Age
≤ 30 years	10.8 (8)
31 – 40 years	25.7 (19)
41 – 50 years	31.1 (23)
51 – 60 years	17.6 (13)
≥ 61 years	14.9 (11)
Education Level
Associate’s degree or less	24.2 (18)
Bachelor’s degree	37.9 (28)
Master’s degree or higher	37.9 (28)
Occupation
Emergency manager	23.0 (17)
Healthcare professional	17.6 (13)
Disaster planner	16.2 (12)
Public service librarian	13.5 (10)
Public health professional	12.2 (9)
Reference librarian	6.8 (5)
Fire professional	5.4 (4)
Law enforcement or military	5.4 (4)
Manager (yes)	62.2 (46)
Employing Organization
Emergency management	23.0 (17)
Public health	20.3 (15)
Healthcare facility	17.6 (13)
Public library	12.2 (9)
Academic library	8.1 (6)
Other	18.9 (14)
Work Experience
≤ 1 year	13.5 (10)
2 – 5 years	25.7 (19)
6 – 10 years	9.5 (7)
≥ 11 years	51.4 (38)

^*Likert scales used: quality scale, 1 = poor to 5 = excellent; perception scale, 1 = strongly disagree to 5 = strongly agree.

Perceived Quality and Relevance of the Workshop Materials

Immediately following the workshop, participants were asked questions about perceived quality of the workshop, whether they believed they accomplished the workshop’s goals, and perceived relevance and usefulness of the workshop materials for themselves or when they train others. The questions were measured using Likert-scales related to perceived quality (1 = poor to 5 = excellent) and perceived relevance (1 = strongly disagree to 5 = strongly agree). Average scores on all perceived relevance and quality questions were high (range of average scores: 3.95 – 4.53; Table 2), meaning that most participants found the workshop materials to be of high quality, useful, and relevant to them. Table 2 outlines participants’ responses to questions related to perceived quality and usefulness of the workshop materials.

Table 2.

Participants’ Perceived Quality and Usefulness of the Workshop

Statement	Mean*	SD
Quality of the course content overall	4.46	.67
Quality of the PowerPoint slides	4.35	.65
Quality of the speakers’ presentation styles	4.53	.67
I accomplished the workshop objectives	4.0	.81
The workshop was relevant to my job	3.95	.91
I will be able to use the skills taught at the workshop	4.19	.79
It increased my ability to train others	4.04	.88
The materials provided will be useful when I train others	4.15	.93

^*Likert scales used: Quality scale: 1=Poor; 5 = Excellent Perception scale: 1 = Strongly disagree; 5 = Strongly agree

Awareness and Use of the NLM Tools Before Vs. After the Workshop

Awareness and use of the NLM tools was measured on a 5-point scale, ranging from 1 – 5 (1=“never heard of them”; 5 = “aware and used them often”). There was a significant increase in awareness from pre- to post-workshop and this awareness remained increased one year after the workshop. There was a significant increase in awareness from pre- to post-intervention (average 1.9 pre-workshop vs. 3.5 immediately post-workshop and 3.1 one year post-workshop; p < .001 for both pre/post comparisons). Almost none of the participants reported having ever received any training on REMM, CHEMM, or TOXNET prior to the workshop (91.9%, 91.9%, and 90.5%, respectively). Most (80.0%, n=59) reported having received no training on WISER, though 10% (n=8) had received 30 minutes or less training and the remaining 10% (n=7) had received 31 minutes or more WISER training. Almost none of the participants reported having ever used REMM, CHEMM, or TOXNET prior to the workshop (95.9%, 93.2%, and 91.9%, respectively). Very few indicated that, prior to the workshop, they had used WISER often (2.7%, n=2) or sometimes (8.1%, n=6). The rest had either never used it (78.4%, n=58) or used it rarely (10.8%, n=8). The librarians (n=15) were asked if an emergency responder had ever approached them for assistance in identifying literature relevant to disaster planning or response; 93.3% (n=14) indicated that they had never been asked by an emergency responder to find such resources.

Perceived Confidence in Using the NLM Tools Before Vs. After the Workshop

Participants were asked about their perceived confidence in using four NLM tools: WISER, CHEMM, REMM, and TOXNET, using a 5-point Likert-type scale ranging from 1 (very unconfident) to 5 (very confident). Changes in perceived confidence from pre-workshop to immediate post workshop were statistically significant and remained statistically significant at the one-year follow-up for all four NLM tools: CHEMM, REMM, TOXNET, and WISER (Table 3). Participants were significantly more likely to report being confident in their use of WISER compared to the other NLM tools (p < .001 for all comparisons).

Table 3.

Participants’ Perceived Confidence In Using the National Library of Medicine Tools Before Versus After the Workshop

	Perceived Confidence*
NLM Tool	Before Mean (sd)	Immediately After Mean (sd)	One Year After Mean (sd)	p value**
CHEMM	2.4 (1.1)	4.0 (.38)	3.8 (.86)	< .001
REMM	2.4 (1.2)	3.8 (.42)	3.8 (.87)	< .001
TOXNET	2.4 (1.1)	3.5 (.49)	3.8 (.75)	< .001
WISER	2.8 (1.2)	4.1 (.31)	4.0 (.96)	< .001

^*Perceived confidence measured on a 5-point Likert-type scale: 1 = very unconfident; 5 = very confident

^**Determined by a Wilcoxon signed rank test with Bonferonni correction

Knowledge of the NLM Tools Before Versus After the Workshop

Sixteen questions were used to assess participants’ knowledge of the NLM tools and one point was assigned to each item; therefore, participants’ knowledge scores could range from 0 – 16. The average pre-workshop knowledge score was 7.2 and average immediate post-workshop was 11.9, a statistically significant change (t= 15, p < .001), with a 95% confidence interval of (4.1, 5.3). Knowledge scores were not found to differ when comparing occupation, age, gender, education level, years of work experience, perceived confidence in using the tools, or past reported use of the tools.

Intent and Extent to Which Participants Disseminated Materials After the Workshop

Immediately after the workshop, about a third of participants (36.5%, n=27) indicated that they planned to host a formal training using the TTT materials in the next month; significantly more individuals (54.1%, n=40; p< .001 for all comparisons) indicated that they planned such a formal training in the next 6 months or year. About three-quarters (78.4%, n=58) reported that they planned to share the TTT materials informally with others in the next month. Participants were significantly more likely to indicate that they planned to offer informal training compared to formal training for each of the time frames (Table 4). There were no differences between occupation & intent to train formally or informally.

Table 4.

Intent to Share Workshop Materials Formally Versus Informally in the Short- and Long-term

	Intent to share via formal training* % (n)	Intent to share via informal training* % (n)	p value**
Immediately After (N=74)
In the next month	36.5 (27)	78.4 (58)	< .001
In next 3 months	50.0 (37)	75.7 (56)	< .001
In next 6 months	54.1 (40)	78.4 (58)	< .001
In next year	54.1 (40)	70.3 (52)	< .001
In the next month to year	68.9 (51)	87.8 (65)	< .001
3 Months After (N=43)	Intent to share via formal training* % (n)	Intent to share via informal training* % (n)	p value**
In the next month	7.0 (3)	32.6 (14)	= .01
In next 3 months	16.3 (7)	37.2 (16)	< .01
In next 6 months	27.9 (12)	53.5 (23)	< .001
In next year	39.5 (17)	65.1 (28)	< .001
In the next month to year	44.2 (19)	67.4 (29)	< .001
1 Year After (N=37)	Intent to share via formal training* % (n)	Intent to share via informal training* % (n)	p value**
In the next month	0	16.2 (6)	< .001
In next 3 months	8.1 (3)	21.6 (8)	= .05
In next 6 months	5.4 (2)	32.4 (12)	< .05
In next year	21.6 (8)	43.2 (16)	< .01
In the next month to year	24.3 (9)	45.9 (17)	< .01

^*Participants could answer “yes” to more than one time frame.

^**Determined by the X² test; Fisher’s exact test used when cell sizes were ≤ 5

Three (7%) of the 43 participants who responded to the 3-month post-workshop survey reported that they had provided formal training to others using the TTT materials since the workshop; in total, they reported providing formal training to approximately 31 individuals. Twenty-four (55.8%) of the 43 participants who responded to the 3-month post-workshop survey reported that they had provided informal training to others since the workshop; in total, they reported having informally trained approximately 157 individuals. Librarians and first responders were equally likely to have provided informal training.

Among the respondents who answered the one-year post-survey (n = 37), significantly more reported having informally trained at least one person since the workshop versus indicating they had provided at least one formal training session (56.8% vs. 8.1%, respectively; p < .001). In these reported trainings by workshop attendees, approximately 42 individuals received formal training and 203 individuals received informal training. Those who responded to the one-year post-workshop survey who indicated that they had not used the TTT materials to provide formal training (n=34) were asked to identify the reason(s) they did not provide such training to others; they could select more than one. The most frequently reported reasons were a lack of time (61.8%, n=21) or the belief that it was not their responsibility to train others (52.9%, n=18). About a third (32.4%, n=11) indicated that they lack the resources to host a formal training, 26.5% (n=9) lacked confidence in their knowledge of the tools, and 11.8% (n=4) believed it would be too expensive.

Of those who responded to the one-year post-workshop survey (n=37), about a quarter (24.3%, n=9) indicated that they planned to host a formal training session using the TTT materials in the next month to a year. Even more reported planning to host informal training sessions on the NLM tools in the next month to a year (45.9%, n=17; formal vs informal comparison X² = 8.8, p < .01; Table 4). There were no differences between occupation & intent to train formally or informally.

Qualitative Comments During and After the Workshop

In addition to the quantitative data collected and analyzed, qualitative data was collected via the post-workshop surveys and through the research team’s observations during the workshops. A recurring theme identified was the participants’ desire for more training on radiological events. Two of the scenarios involved radiological events, with the intent of having participants gain mastery over the NLM tools by having them access and use the tools to respond to the scenarios. Though this was successful, multiple workshops almost became severely sidetracked when participants requested specific information regarding radiological disasters that was outside the workshop’s scope. Another common theme expressed by attendees was the unanticipated benefit of having librarians and emergency responders partner together in the workshop. Many attendees, both librarians and emergency responders, commented that they had not understood librarians’ role in disaster preparedness prior to the workshop, but afterwards expressed the benefit of this collaboration.

Discussion

The primary purpose of the VISIBLE Program was to increase awareness and knowledge of NLM resources among emergency responders and librarians. To that end, this project was highly successful as evidenced by the significant rise in awareness, perceived confidence in use, and knowledge of the NLM tools covered by the workshop. In addition to the common short-term change seen in education-related intervention studies^10,11, this study found that the attendees maintained their increased awareness and perceived confidence in using the NLM tools for one year after the workshop. Though assessing long-term outcomes of public health disaster preparedness educational interventions are less common, two other studies reported similar findings: at one year after the training, attendees had retained their knowledge and/or confidence in the content provided.^12,13 The VISIBLE project focused on four tools available for free through the NLM: WISER, REMM, CHEMM, and TOXNET. These tools can be used for both pre-event planning and responding to actual events. The sustained strong confidence in using these tools increases the likelihood that emergency responders will use the NLM resources in developing more comprehensive disaster plans and/or having a more successful response to an actual event. This should result in more resilient communities.

Findings from this study indicate that the TTT format resulted in dissemination of the VISIBLE Program workshop materials to others by both the librarian and emergency responder participants. In total, approximately 245 individuals received training on the NLM tools during the “second generation” of this project in the one year after the workshop. In addition, a quarter reported that they plan to host a formal training session and almost half indicated that they would disseminate information on the NLM tools informally to others in the upcoming year. Assuming that they follow through with these intentions, information on the NLM tools will be greatly disseminated beyond the initial workshops held in this study. This supports the intention of using a TTT format: to spread the training more widely than could be done by only hosting a few workshops.^14,15

It is difficult to quantify the expected percentage of participants who would disseminate the information to others or the number of individuals that would be trained formally or informally after a TTT program, as a lack of long-term follow-up to document exactly how many individuals have been trained during the “second generation” is a common limitation in TTT programs.^14,16 Only three fully evaluated TTT programs related to disaster preparedness could be identified. A TTT program in New York focused on a disaster mental health education intervention reported that 60% of the 140 participants had provided some training to others in the one year after the intervention and that, in total, 559 individuals were trained in the “second generation”.¹⁴ An online TTT program aimed at Bangladeshi health professionals reported that there was almost no dissemination of knowledge beyond the initial groups of individuals trained; the reason for the lack of follow-up was postulated to be a lack of confidence in the knowledge gained.¹³ A state-wide TTT program focused on public health preparedness in Maine involved 118 individuals in the initial training, and only 20% of those disseminated their training to others.¹⁶ The findings from the VISIBLE Program are in line with the better results reported in the New York TTT study in terms of the percentage of workshop attendees who disseminated the training to others and the approximate number of people educated in the “second generation”. Further research should be conducted to examine the long-term outcomes of other TTT programs to better quantify the extent of dissemination of programs or the reasons why some programs are more successful than others.

Findings from this study indicate that VISIBLE Program workshop participants were significantly more likely to offer informal training compared to formal training on the NLM tools. One explanation for this could be the time and resource commitment needed to provide formal training. For example, two commonly reported reasons for not providing formal training in this study included a lack of time and lack of resources. Another possible reason why participants in this study offered more informal than formal training is that those who attended the workshop were not aware before arrival that the program was intended to be a train-the-trainer format. It is possible that those who attended were seeking training for themselves only and are not in a position to host formal training for others. There is some evidence for this; half of those who did not offer formal training reported the belief that it was not their responsibility to train others. In general, TTTs are designed to provide education to instructor-trainees who have the responsibility to train others.^14,15 If workshop attendees do not perceive themselves to be instructor-trainees, it lessens the likelihood that they will use the TTT materials to disseminate to others. It may be prudent to target advertising for future workshops to indicate the TTT aspect of the program. Dissemination of TTT materials may also be increased in future workshops by more heavily emphasizing the importance of sharing the TTT materials, and offering ongoing assistance to attendees if they request it.

A secondary aim of the VISIBLE Program was to expand the role of Missouri librarians by partnering with local emergency responders in an educational setting to improve disaster planning efforts. Though this aim was not measured quantitatively through the study, there is qualitative evidence that both emergency responders and the participating librarians benefited from working together at these workshops. Attendees commented at the workshop and in the qualitative comments section of the surveys about the benefits of having this diverse group of professionals working together on these complex scenarios. It seems that, prior to this workshop, neither group understood how librarians could contribute to disaster preparedness. Given that none of the librarians in this study reported holding a certificate in Disaster Information Specialization (DIS) from the Medical Library Association¹¹, this workshop hopefully served as an introduction to the librarian’s role in disaster preparedness for many of the participating librarians. It is important to build this awareness because librarians can play a significant role in preparing healthcare facilities and emergency response agencies for disasters.¹² In particular, librarians could aid emergency responders in identifying critical literature needed to develop more comprehensive disaster plans.¹² It is notable that, in this study, only one of the librarians had ever been asked by an emergency responder to assist in identifying disaster-related literature. Perhaps this is because emergency responders are not aware of this resource in their community. Future studies and educational programs should be focused on increasing awareness about these resources and providing more collaborative opportunities for emergency responders and librarians.

This study had multiple strengths. It was the first known study to partner librarians and emergency responders in an educational setting specifically to enhance the benefits of each group. TTT formats have long been used to aid in disseminating training materials, but formal evaluation, especially long-term, of these programs is rare.^14,15 This study included an extended post-workshop follow-up time period (i.e., one year), which allowed for examination of longer-term outcomes, which is a major strength. The main limitation to this study was the lower response rate at the 3-month and 1-year post-workshop time periods; however, the response rate remained reasonably high at 58% (3 months) and 50% (1 year) in the follow-up time periods and is similar to the response rate seen in other studies using a TTT approach.^14–16 The lower response rate in the follow-up surveys may introduce some response bias in those who chose to complete a 3–month or 1-year post-workshop survey. It is possible that non-responders may have answered differently than those who chose to complete a survey. This study may also have social bias, as the reporting of post-workshop training dissemination was only self-report. Lastly, because this study was only conducted in Missouri, the results may not be generalizable to other regions.

Conclusion

This study used an education intervention to increase awareness, confidence, and knowledge of emergency responder and librarians in the use of free tools available through the National Library of Medicine for improving disaster preparedness and response. Attendees perceived the workshop to be of high quality and useful to their careers, and both short- and long-term positive changes were found. In addition, the train-the-trainer model resulted in further dissemination of the workshop information through both formal and informal training. The NLM tools are free of charge and could be used to host train-the-trainer programs in other communities.