An Academic-Government-Faith Partnership to Build Disaster Mental Health Preparedness and Community Resilience

Abstract

Objectives

Faculty and affiliates of the Johns Hopkins Preparedness and Emergency Response Research Center partnered with local health departments and faith-based organizations to develop a dual-intervention model of capacity-building for public mental health preparedness and community resilience. Project objectives included (1) determining the feasibility of the tri-partite collaborative concept; (2) designing, delivering, and evaluating psychological first aid (PFA) training and guided preparedness planning (GPP); and (3) documenting preliminary evidence of the sustainability and impact of the model.

Methods

We evaluated intervention effectiveness by analyzing pre- and post-training changes in participant responses on knowledge-acquisition tests administered to three urban and four rural community cohorts. Changes in percent of correct items and mean total correct items were evaluated. Criteria for model sustainability and impact were, respectively, observations of nonacademic partners engaging in efforts to advance post-project preparedness alliances, and project-attributable changes in preparedness-related practices of local or state governments.

Results

The majority (11 of 14) test items addressing technical or practical PFA content showed significant improvement; we observed comparable testing results for GPP training. Government and faith partners developed ideas and tools for sustaining preparedness activities, and numerous project-driven changes in local and state government policies were documented.

Conclusions

Results suggest that the model could be an effective approach to promoting public health preparedness and community resilience.

The focus of our studies at the Johns Hopkins Preparedness and Emergency Response Research Center (JH-PERRC), in the Johns Hopkins Bloomberg School of Public Health, has been mental and behavioral public health systems research, a field of investigation we consider important for several reasons, including:

• There is overwhelming evidence^1–3 that the majority of injuries or trauma in most disaster settings are psychological, as opposed to physical, with ratios ranging from 4:1 to as much as 50:1—ratios consistently reflected in the National Planning Scenarios.⁴

• Even for emergencies that might be defined as strictly physical or biological (e.g., a dirty bomb or an epidemic), the connection between physical responses and emotional, cognitive, and social-psychological processes is substantial.⁵

• Individuals with preexisting mental illnesses represent an important, highly vulnerable population.^6,7

• Public health system personnel are themselves at risk for all of the aforementioned reasons.^8–10

Critical among the capabilities needed to mitigate the impact of behavioral health surge are strategies to remedy (1) the shortage of disaster responders with mental health expertise to aid individual disaster survivors and (2) the dearth of communities with formal disaster preparedness plans (and planning activities) to safeguard residents before, during, and following disasters. Accordingly, we believe the public health significance of the approach to be described is that it incorporates validated and replicable interventions to address those challenges directly.

Delivered within the framework of partnerships among our academic health center (AHC), faith-based organizations (FBOs), and local health departments (LHDs), the interventions are training in psychological first aid (PFA) and guided preparedness planning (GPP). The FBO serves two important roles in the collaborative structure of our model. First, the FBO acts as a link between LHDs and the community as a whole, enabling LHDs and AHCs to transfer subject-matter expertise through training in psychological crisis intervention to individuals and disaster preparedness planning to 2- to 4-person teams. Second, the FBO serves as a partner with the LHD representative to foster an enduring, post-training preparedness alliance.

We developed our model using a three-phase investigational strategy:

• Phase 1: A pilot study involving administrations of early versions of PFA and GPP delivered to urban populations in the state of Maryland, without the collaboration of LHDs, and conducting traditional posttest-only assessments of participant reactions to the training^11–13

• Phase 2: A study of more refined iterations of PFA and GPP, administered to rural populations in Maryland, with the collaboration of LHDs and, again, with posttest assessments of participant reactions to the training^14,15

• Phase 3: A series of validation studies with final versions of PFA and GPP administered to cohorts in Maryland and other states, with assessments of pre- and post-training self-reports, objective tests, and behavioral indices of changes in relevant knowledge, skills, and attitudes (KSAs)

Phase 3 work was conducted in partnership with multiple LHDs and, on several occasions, with the collaboration of national Preparedness and Emergency Response Learning Centers (PERLCs).

We chronicled the development and evaluation of the practice-relevant features of this dual-intervention approach to enhancing public mental health preparedness and community resilience and provide descriptions of (1) strategies for establishing and maintaining the systems-based partnerships, (2) methods for designing and delivering the interventions, (3) objective testing data, (4) representative outcomes at multiple levels of the public health emergency preparedness system (PHEPS), and (5) lessons learned that could be useful to prospective adopters of the model.

METHODS

Partners, participants, and settings

Partners were leaders of LHDs and FBOs collaborating with members of an AHC, comprising faculty and affiliates of the JH-PERRC, and with representatives in the Johns Hopkins School of Medicine and Health System. Participants (i.e., trainees) were adult male and female members of congregations and communities recruited by participating FBO partners representing Christian faiths in the study (and Christian, Jewish, and Muslim faiths in earlier studies). Settings included urban and rural areas of Maryland, Illinois, and Iowa.

Interventions

We viewed training lay citizens in PFA as a logical means to create mental health extenders in communities where professional mental health expertise is in short supply, particularly during and after large-scale emergencies. We viewed training and technical assistance with FBO-designated planning teams in GPP as a reasonable strategy to begin to enhance the preparedness of participating organizations, member families, and larger communities. Figure 1 provides an outline of component modules of the respective training curricula.

Figure 1.

Summary of modular content of PFA and GPP training interventions delivered to all cohorts^a in Illinois, Iowa, and Maryland, March 2010–June 2012

^aRural cohorts included Cambridge and Centreville, Maryland, and Cedar Rapids, Iowa. Urban cohorts included Baltimore and Turner Station, Maryland, and Chicago, Illinois.

PFA = psychological first aid

GPP = guided preparedness planning

KSA = knowledge, skills, and attitudes

FBO = faith-based organization

ICS = incident command system

Typically, PFA and GPP were delivered on back-to-back days (e.g., Fridays and Saturdays) or on the same day of sequential weeks (e.g., two consecutive Saturdays).

Specific aims

The aims of the study and the component questions included the following:

1. Aim 1: Confirm the feasibility of the partnership model (i.e., proof of concept).

   1. Question 1.1: Can an AHC successfully engage and sustain LHDs and FBOs in a collaborative preparedness venture?
   2. Question 1.2: Can partners jointly define and execute their respective roles and responsibilities (initially, not fully articulated)?
   3. Question 1.3: Can barriers and facilitators to successful collaboration be characterized?
2. Aim 2: Develop and validate the training interventions.

   1. Question 2.1: Can partners jointly design curricular content and support materials for the PFA and GPP workshops?
   2. Question 2.2: Will the interventions be effective with both urban and rural populations at risk for various disasters and public health emergencies?
3. Aim 3: Explore methods of promoting sustainability and translational impact of the overall model.

   1. Question 3.1: Will LHD partners be willing and able to generate ideas for sustaining LHD-FBO preparedness alliances beyond the term of the project?
   2. Question 3.2: Does the model hold potential for translational impact on policy or practice at the level of local and/or state government?
   3. Question 3.3: Can the model be replicated beyond its state of origin (i.e., Maryland)?

Research design

We applied formative research strategies, spanning both quantitative and qualitative methods, to answer the component questions. Included among the data collection approaches were face-to-face and telephone interviews, focus group discussions, structured and unstructured surveys, and checklists. We conducted content analyses on several unstructured data sources. The primary quantitative outcome data in this article were derived from objective pre- and posttesting to validate the effectiveness of final versions of the companion interventions to improve KSAs delivered to seven cohorts.

Data collection methods and measures

Feasibility of the model.

Inferences about the feasibility of the approach were derived from process-evaluation data (e.g., mean number of participants by intervention type) and the willingness and ability of partners to define and execute their roles.

Design/development of PFA and GPP curriculum content.

We conducted a literature review of PFA disaster-planning approaches, including publications in major databases (e.g., Medline, PsychINFO, and Thomson Reuters Web of Knowledge). Selected elements from relevant planning protocols were integrated with the input of partners and an advisory committee, and with recommendations from disaster planning guidance provided by the Office of Preparedness and Response within the Maryland Department of Health and Mental Hygiene.

Evaluation of PFA and GPP effectiveness.

Instruments used to confirm knowledge acquired by PFA and GPP trainees, respectively, were the Psychological First Aid Knowledge Test (an objective test comprising nine true/false and five multiple-choice questions) and the Disaster Planning Knowledge Test (an objective test comprising 10 true/false and five multiple-choice questions).

Sustainability and impact.

No a priori operations were selected to quantify accomplishment of these longer-term goals. Rather, as ideas and opportunities for their achievement emerged, we pursued them. The results are reported in the form of illustrations and examples.

Data analysis

We summarized outcome evaluation data in the form of responses on the knowledge-acquisition tests as percent correct and mean total correct items with 95% confidence intervals (CIs). Changes between pre- and post-training administrations were evaluated with general linear model analyses accounting for the within-cohort correlation of responses. Identical analyses were performed within rural and urban cohorts.

RESULTS

Aim 1: Feasibility of the partnership model

Question 1.1: Engaging and sustaining partner collaboration.

The viability of the AHC-LHD-FBO partnership concept was demonstrated in all phases of our research series, as hundreds of FBOs and scores of LHD representatives partnered with the Johns Hopkins AHC to implement the approach.^11–15 In our most recent (i.e., Phase 3) study, involving four urban and three rural cohorts, the mean number of individual participants attending PFA and GPP trainings was 31 and 25, respectively; the typical number of organizational partners at these trainings was 10 FBOs and three LHDs. Although the participation of rural FBOs was nearly twice that of urban FBOs, no safe conclusions may be drawn about the difference (data not shown).

Question 1.2: Defining and executing partner roles.

Despite representing organizations with diverse missions and cultures, all partner types proved able to define their roles in early research projects, and to adhere to their respective responsibilities in more recent projects. -Figure 2 summarizes the partner-specific responsibilities in the final model. Although the AHC partner served key roles as convener, coordinator, and scientific lead in the development of the model prototype, we would emphasize that all of the AHC responsibilities could be provided, or contracted for, by an LHD or other public health organization interested in implementing the approach without AHC involvement.

Figure 2.

Essential responsibilities of the AHC, LHD, and FBO, by stage of model implementation

AHC = academic health center

LHD = local health department

FBO = faith-based organization

PFA = psychological first aid

GPP = guided preparedness planning

MRC = Medical Reserve Corps

Question 1.3: Barriers and facilitators to engaging FBOs.

It was not unusual for FBOs to require multiple recruitment efforts before a final decision to collaborate. Critical to securing leadership buy-in and ultimate approval for partnering was finding at least one advocate in the organization who could voice and model enthusiasm for the coventure. As for traditional program marketing and promotion strategies, the most effective were e-mail messages, bulletin inserts, flyers, meetings with ministerial associations, and word-of-mouth transmission; however, it was necessary to follow up on these activities with numerous personal contacts. Recruiting and retaining LHD partners posed some of the same challenges as engaging FBOs. We have published a detailed report on how certain theoretical and conceptual frameworks in the behavioral sciences can be applied successfully to motivate prospective partners to participate in disaster-preparedness activities (data not shown).¹⁶

Aim 2: Development and validation of interventions

Question 2.1: Design of intervention curricula.

The PFA curriculum (Figure 1), based on the Johns Hopkins RAPID approach (Rapport-Building/Reflective Listening, Assessment, Prioritization, Intervention, and Disposition),¹⁷ was successfully adapted and refined for both public health professional and paraprofessional (i.e., lay, non-mental health) audiences.^18,19 Curricular content of the most recent paraprofessional version includes a case formulation approach based on a comprehensive logic model.²⁰ The GPP disaster-planning protocol is grounded in four core assumptions: (1) the importance of developing the plan in partnership with a representative of the LHD, (2) the need for an all-hazards orientation, (3) a focus on the whole of the community, and (4) the imperative of meeting the challenge of (the disproportion of) psychological casualties attending all major public health emergencies.

Although a few participants suggested that GPP should be delivered before PFA, most indicated that not only was PFA a critical hook that pulled them into the project in the first place, but it also raised their consciousness about the lack of preparedness planning in their community. Two types of early evaluative comments that helped determine the final format for the PFA workshop were concerns expressed about not having adequate time to practice techniques and not having content about the kinds of practical information one should have before agreeing to volunteer as a PFA responder (data not shown).

Question 2.2: Evaluation of effectiveness.

Our Phase 3 outcome data are consistent with previously cited findings in Phases 1 and 2. For example, Table 1 shows that of the 14 test items intended to randomly sample learning of PFA content, 11 confirmed significant pre- and post-training improvements.

Table 1.

Comparison of percent correct on PFA test between pre- and post-training test administrations to participants in rural, urban, and all cohorts^a in Illinois, Iowa, and Maryland, March 2010–June 2012

^aRural cohorts included Cambridge and Centreville, Maryland, and Cedar Rapids, Iowa. Urban cohorts included Baltimore and Turner Station, Maryland, and Chicago, Illinois.

^bThe number of respondents for rural cohorts pre-training ranged from 87 to 109, and for post-training ranged from 104 to 107 across these questions.

^cThe number of respondents for urban cohorts pre-training ranged from 68 to 76, and for post-training ranged from 67 to 69 across these questions.

^dP-values are based on two-sample (administration) tests, as responses were not matched for all cohorts.

PFA = psychological first aid

CI = confidence interval

Targeted content spanned the technical aspects of PFA (e.g., knowing how and when to use open- vs. closed-ended questions with disaster survivors [p<0.001]) and the more practical considerations (e.g., knowing important pre-deployment questions to ensure informed responder decisions [p<0.001]). Comparable testing results for GPP are summarized in Table 2. Significant improvements were observed in scores on eight of the 15 test items for all cohorts. As with several PFA items, the relatively high levels of pre-training knowledge on some variables reduced the likelihood of statistically significant training effects being documented.

Table 2.

Comparison of percent correct on GPP test between pre- and post-training administrations to rural, urban, and all community cohorts^a in Illinois, Iowa, and Maryland, March 2010–June 2012

^aRural cohorts included Cambridge and Centreville, Maryland, and Cedar Rapids, Iowa. Urban cohorts included Baltimore and Turner Station, Maryland, and Chicago, Illinois.

^bThe number of respondents for rural cohorts pre-training ranged from 85 to 96, and for post-training ranged from 85 to 91 across these questions.

^cThe number of respondents for urban cohorts pre-training ranged from 62 to 71, and for post-training ranged from 60 to 63 across these questions.

^dP-values are based on two-sample (administration) tests, as responses were not matched for all cohorts.

GPP = guided preparedness planning

CI = confidence interval

LHD = local health department

ICS = incident command system

SWOT = strengths, weaknesses, opportunities, threats

We observed some differences between the rural and urban groups. Rural participants demonstrated learning in knowing types of at-risk citizens during disasters (p<0.001), conducting drills and evaluations of disaster plans (p<0.008), and recognizing examples of preparedness tools and resources (p<0.007); urban participants showed no knowledge improvement on those items. Urban participants, on the other hand, exhibited learning effects in the basic phases of public health emergencies (p<0.001) and the responsibilities of the operations chief in the incident command system (ICS) (p<0.001), while rural participants did not (Table 2).

GPP workshops were effective in guiding >90% of the urban and rural planning teams in the crafting of disaster plans for their communities.²¹ Importantly, the success of early versions of GPP was compromised by attendees who did not possess sufficient knowledge of their FBO leadership to be effective planners, and by trainers devoting too much workshop time to lecturing about disaster planning.

Aim 3: Sustainability and translational impact: representative success stories

Question 3.1: Sustaining LHD/FBO preparedness alliances.

Another problem with beginning versions of GPP was the absence of any framework for LHD and FBO leaders to perpetuate their nascent preparedness alliances beyond the term of the project. Eventually, LHD partners were urged to brainstorm a list of ideas to sustain post-project LHD-FBO contacts, and, following a content analysis, we incorporated the consolidated ideas into a checklist tool. LHD and FBO leaders now review the checklist at the end of the GPP workshop and agree on those relationship-sustaining activities that they are willing to implement jointly in the -subsequent year. (The post-intervention section of Figure 2 lists examples of sustainability activities.)

Question 3.2: Translational impact of the overall model at levels of government.

We have observed impacts of our model on practice and policy at local and state government levels. For example, at the local level, one LHD leader (author Charlene Perry) now has in place a durable arrangement by which FBOs in her jurisdiction make their facilities accessible to disseminate public health messages and guidelines. During pandemic influenza A(H1N1), 22 FBOs operated as points of dispensing, where 536 people were vaccinated.

Another successful translational initiative has been implemented in the form of an innovative mini-grant program for both previous and new FBO partners. The program has been effective in two ways: First, it has upgraded the preparedness status of families, FBOs, and communities by enabling them to acquire practical resources (e.g., equipment, supplies, and shelter-in-place guidelines) for enhancing their resilience during public health emergencies. Second, it has strengthened the relationships between faith and health department representatives through ongoing, trust-building contacts at events such as advisory meetings, plan drills, and town exercises.

Maryland's Medical Reserve Corps (MRC), MD Responds, now accepts our PFA-trained citizens as volunteer responders during disasters and emergency situations. Previously, network membership was available only to board-licensed health professionals. Most recently, 60%–70% of trainees have been submitting applications for network membership, a status that confers professional liability insurance and workers' compensation coverage to responders. Locally, the process is facilitated by an MRC representative attending PFA workshops, providing a brief overview of the MRC program, and making available laptop computers for direct online application.

Question 3.3: Portability and replicability of the model.

Our model travels well, as judged by field testing in other states. For example, with promotional and logistical support provided by the Illinois Preparedness and Emergency Response Learning Center (IL-PERLC), we implemented the program with FBOs and government representatives during a two-day visit to Chicago. An independent evaluation conducted by the IL-PERLC yielded overall program satisfaction ratings for the PFA and GPP trainings, respectively, of 4.95 and 4.55 on a 5-point Likert scale (where 5 = strongly agree). We recorded comparable participant program satisfaction ratings when we conducted trainings in Cedar Rapids, Iowa, with administrative support provided by the U.S. Disaster Program of Episcopal Relief and Development and its local Iowa diocese (data not shown).

DISCUSSION

We gained many insights that may be of potential use to prospective adopters of our model.

Engaging community partners

Use direct contacts and incentives as primary recruitment strategies.

The most successful community-engagement strategies involve direct contact, as might be effected through third parties, meetings with ministerial associations, and outreach workers. Postal mailings and e-mail messages to faith leaders are much more effective when followed by phone calls. Employ a broad range of incentives, including simple considerations to make cooperation easier, such as offering convenient meeting times and locations. Resources permitting, provide snacks and lunch at preliminary meetings and workshops, and always try to sit with your community partners during meals and breaks, not just with your own coworkers and peers.

Locate a “champion” in the organization.

Many faith leaders have full-time jobs outside of their faith duties, and they may not respond immediately even to overtures. Although the formal leader typically will need to provide final approval, often a prospective advocate can be identified who, if she or he is a trusted thought-leader, can be critical to the ultimate partnering agreement. One reason such allies are effective is that they encourage collaboration using the language of the organizational culture.

Adopt a participatory partnership philosophy.

Major differences in organizational cultures exist among the three partner categories. Know that challenges occasioned by diversity of missions, values, and norms can be mitigated by adopting a collaborative philosophy that embodies principles of mutual respect, compatible goals, clear objectives, joint decision making, and shared credit.

Designing and delivering interventions

Deliver PFA before GPP.

A day of training in PFA is typically more attractive to prospective community participants than a workshop in disaster planning. However, the PFA training session will raise the consciousness of participants about the potential gravity of disasters, and the spotlighting of a community's lack of preparedness for public health emergencies galvanizes motivation to participate in community preparedness planning.

Allow ample time for the practice of PFA techniques.

Participants are attracted to PFA training for its potential to impart practical helping skills, both for survivors of disasters and people experiencing everyday mini-disasters. An especially popular part of PFA training is the opportunity afforded to practice listening skills and stress-management techniques.

Incorporate practical information.

Aside from technical content, PFA training should cover important information prospective PFA responders should possess before deployment; for example, workshop leaders should emphasize the importance of responders knowing (1) how, when, and where to report; (2) what their roles will be; and (3) what resources and supports will be available for survivors and themselves. A related issue to accentuate is the importance of responding within the framework of the state MRC, in contrast with the well-meaning but often-problematic practice of just showing up at disaster sites.

Ensure that members of GPP teams are prequalified.

GPP workshops will be successful only if participants have adequate knowledge of the primary community for which the disaster plan is being developed. Particularly vital is knowledge about the FBO's leaders and members so that specific people can be proposed for ICS leadership positions, and disaster-related resource surpluses and shortages in the community (e.g., availability or absence of health-care professionals, temporary shelters, and generators).

Avoid discussing disaster planning at the cost of doing disaster planning.

The GPP session requires a task orientation: a product is developed. Accomplishing that objective will be facilitated by distributing the planning template to prospective participants before GPP, and by ensuring that workshop leaders review and provide feedback on plan drafts throughout the workshop day.

Appreciate that the GPP intervention generates drafts of “basic” disaster plans.

As workshops are of only one-day duration, FBOs will need to continue to flesh out their plan drafts, including various functional annexes (e.g., lockdown, shelter-in-place, and evacuation). Critical to plan advancement are two post-workshop activities: (1) ongoing preparedness planning efforts by each newly identified ICS position holder, particularly the planning leader; and (2) continued support and guidance of the FBO by the LHD partner. Collectively, all should focus efforts on identifying at-risk subpopulations in the community.

Promoting model sustainability and translation

Target long-term sustainability goals for new LHD/FBO preparedness alliances.

Immediately following the GPP workshops, it is important that LHD and FBO leaders identify the specific ways that they will be jointly advancing FBO plans and strengthening their nascent preparedness relationships. Especially important for promoting model sustainability and translational impact is the LHD fostering the development of coalitions with other FBOs, and connecting FBOs with other public emergency personnel, programs, and agencies.

CONCLUSIONS

The living laboratories embodied in each LHD/FBO partnership have shown that leaders from these diverse organizational cultures can work effectively to implement an approach with the potential for being a widely applicable model for enhancing community disaster resilience. The model aligns strongly with numerous legislative and executive directives^22–24 and with the Institute of Medicine's research recommendations to create sustainable preparedness and response systems²⁵ and to explore the benefits of public-private preparedness coalitions.²⁶

Our next steps are to continue refining the PFA intervention following feedback from its national dissemination, and to integrate GPP content into a technology platform for online training through current funding by the Centers for Disease Control and Prevention. We have recently conducted alpha-testing of the draft protocol with subject-matter experts, key informants, and practice partners who, because they already participated in the face-to-face training, can make a comparative evaluation with the online product. This first-stage refinement of the online training will be disseminated to FBOs and LHDs throughout the country for feedback and improvement by those who are naïve to GPP. This national feedback will ensure broad representation from FBOs, who, with their LHD partners, will bring differing and varying experiences and expectations to the planning process.

We trust that these activities will provide an opportunity for a more widespread application of the model, thereby enhancing its public health significance.