Introduction
The important role of public health in emergency management during disasters is widely recognized.[1] Within emergency management, Continuity of Operation Planning (COOP) encompasses actions taken to ensure delivery of essential services before, during, and after a crisis and is dependent upon technologies designed to support these actions.[2] However, many public health departments lack the funds and staff to maintain preparedness activities mandated by law[3] and so may lack the technology to support COOP.
Our experiences working with professional COOP planners at a large municipal public health agency confirm a lack of decision support technology for public health Continuity of Operations Planning. Decision-makers rely on timely, up-to-date information to coordinate the delivery of essential services to clients. Although decision-makers have access to a wide variety of disparate data sources, they need the data aggregated, processed and displayed in a user-friendly and usable format that will enhance rather than impede the decision-making process. Given the resource-constrained environments within which public health agencies typically operate and a lack of available COOP decision support technology, we were presented with an opportunity to demonstrate the usefulness of scenario-based design in a public health context as well as begin preliminary development of a public health COOP decision support framework.
In order to design appropriate decision support tools for COOP decision-makers, one needs to understand the complexity of the decision-making process in emergency situations and specifically in the public health context. Scenarios are powerful tools in assessing end users’ perceptions and information needs and provide a solid way of explaining workflow features, challenges and solutions. By using a narrative explanation style, one may achieve greater success in explaining how a complex or unexpected situation may arise or which steps are involved in completing tasks and notifying stakeholders. In Five Reasons for Scenario-Based Design, Carroll notes “[d]esigners often have little more than craft practices to guide them in these regions. But if we see design as inherently a process of inquiry, this lag between codified knowledge and practice is transformed into a significant opportunity.”[4] Scenarios in general provide a valuable basis for communication and common understandings within a project group, especially in large studies where complex systems are to be designed. Given the success of scenario-based design in other domains, we implemented this methodology to create user scenarios for decision-makers in a large municipal public health agency as a first step to bridge this apparent technology gap.
The purpose of this study is to document and assess public health leadership decision-making information needs during emergency operations and explore the use of scenario-based design for decision support tools for public health COOP. Toward this end, we created user scenarios based on information needs during a pandemic flu and demonstrated the need for support of COOP efforts in a large urban area through description of required user actions. The result of these efforts is a set of twelve pandemic decision-making scenarios that mark the starting point in the creation of PHARAOH - the Public Health Awareness, Response And Operations Help framework.
Materials and Methods
User scenarios were developed based on decision-making information needs that were documented during an in-house service improvement initiative at a large municipal public health agency. Beginning in January of 2008, a senior software designer and two COOP managers participated in on-going meetings to help document and design a COOP decision support system following the principles of scenario-based design.[4–6] Six such design meetings took place from January, 2008 to March, 2008. One COOP manager had 30 years experience within the organization and has held several distinct decision-making roles. The other COOP manager directed COOP efforts across divisions for the organization. Meetings lasted from 60–90 minutes.
Four of these meetings were conducted where the two COOP managers discussed organizational roles and information needs in the context of various disaster scenarios. During these meetings, the senior software designer took notes and asked questions to clarify details or probe leads for information needs. Specifically, the question protocol addressed issues related to necessary information for decision-making with regard to work situations to discover end user context, work flows and processes.[7, 8] For instance, if the interviewee stated the need to see a staff roster to know who is available at a given site, one follow-up question was: “What types of things do you need to know about available staff to make decisions about staffing and services?” Or, if the interviewee stated a need to work from home, one follow-up question was: “When you say you need to work from home, are there things that you do at the office that you need to do there?” In addition, one meeting included a larger group whose members also included public health division and clinic managers engaged in the same types of discussions. Lastly, one meeting included a different larger group whose members included senior staff from both a leading public health informatics center and a leading academic informatics program.
Notes from each meeting were summarized by the senior software designer. Questions and clarifications about notes were sent to the COOP managers via e-mail. Finalized summaries were then sent to the COOP managers. These documents formed the basis for open-ended questions and the start of discussions at the following meeting using an iterative qualitative data collection approach.[7–10] Nine total pages of summarized notes related to COOP information needs resulted from these meetings.
In April 2008, the senior software designer analyzed the complete set of notes containing the recorded information needs to create a set of scenarios of use to inform preliminary design of a decision support system based on principles of qualitative analysis and scenario-based design.[4–10] Information needs and hypothetical cases were grouped according to similarity based on qualitative data analysis[7, 9] and scenarios of use for different roles emerged through this process. Documented roles include site manager (responsible for one clinic), area manager (responsible for multiple clinics) and COOP manager (responsible for Continuity of Operations for a subunit of the organization). Some scenarios of use are more likely for certain roles than others. For purposes of presentation, the scenarios here are shown with relation to one role. This presentation strategy was validated by both COOP managers in keeping with the principles of member-checking[10] and end user involvement as design evolves.[8] All resulting scenarios of use were member-checked and validated by e-mail with the two COOP managers in May, 2008, adjusted, and then validated again in a face-to-face meeting in July, 2008. Both COOP managers and the senior software designer attended a COOP table-top exercise with 31 decision-makers and COOP managers from six different organizational divisions in October 2008. Scenarios of use were subsequently modified based on changes in service prioritization and other changes within the organizational COOP plan. These updated scenarios of use were member-checked and validated a third time by one of the COOP managers in November 2008.
Results
Scenario-Based Design
Data collection lead to the generation of a user profile and twelve user scenarios for the use of a COOP decision support system during a pandemic.
User Profile
Bill is a senior-level administrator at a large public health department and the Continuity of Operations Manager for a subunit of the organization. His job is to direct the day-to-day service delivery operations for several public health clinics. In addition, he is responsible for development and management of the COOP plan for his sites for pandemics, earthquakes and other emergencies. During an emergency, Bill must assess the facility, staff and supply situation to make staff management decisions for his sites.
All services in these scenarios are prioritized on a scale from 1 to 4. Priority 1 services are services that cannot be suspended while priority 4 services are services that can be suspended indefinitely. Priority 2 and 3 services, if suspended, must be put back into service within 24 and 72 hours, respectively.
User Scenarios
-
Scenario Name: See a roster for staff attendance at a selected public health site
The public health department is managing the effects of a flu pandemic. Bill must run a report for staff attendance as reported by the site manager at the Location A Public Health Center facility. Bill needs to see the job class and skills of each employee at Location A in order to cope with staff reductions.
-
Scenario Name: See a roster for staff attendance at all public health sites
Bill must run an aggregate report to see overall staff attendance at all public health facilities in order to cope with staff reductions as a result of the pandemic flu. Bill needs to see the job class, location and skills of each employee as reported by each site manager.
-
Scenario Name: See individual staff member information regarding job class, job skills, home address, assigned work site, etc.
Bill must be able to quickly and securely access information for each individual employee as automatically updated from the human resources database to assess possibilities for remote work, movement to another site and other staff management logistics. Bill needs to see address, job class and other skills information for each service provider.
-
Scenario Name: See and manage a list and map of public health site status
Bill needs to find out which facilities are functioning or non-functioning. In order to do so, he runs a report that shows a list of all public health sites as well as a map with the status of each site. He sees that the Location B Public Health Center is shown as a functioning site but knows that it is closed due to excessive staff reductions. He edits the status of the Location B site to indicate its non-functioning status.
-
Scenario Name: See and prioritize a list of resource requests by requesting staff member and site
Bill checks the list of resource requests as reported by each site manager. The Location C Public Health Center reports that they are running low on vaccination supplies and has made a resource request. Bill flags the request as Priority 1 and checks the inventory of other public health centers.
-
Scenario Name: See a list of supply inventories at selected public health site
The Location C Public Health Center has made a Priority 1 request for vaccination supplies. Bill checks the supply inventory at the Location A Public Health Center and finds that they have excess inventory. Bill reports the situation to the Emergency Operations Center. He gets the go ahead to make arrangements for delivery of the necessary supplies to Location C from Location A.
-
Scenario Name: See an aggregate list of supply inventories for all public health sites
Location C Public Health Center is still low on vaccination supplies but Location A cannot spare any more of their inventory. Bill checks the aggregate list of all supplies by site to determine what other sites can spare supplies for Location C.
-
Scenario Name: See a list of prioritized services and the necessary skills and resources for delivery
Staff reductions due to the pandemic are negatively affecting the ability of all public health centers to deliver mission critical services. Bill checks the total list of services delivered by the public health department. Bill sees that employees who deliver non-essential services have skills that can be used to deliver essential services. Bill reports the situation to the Emergency Operations Center. He gets the go ahead to temporarily suspend the on-site delivery of services related to community education, the foster care passport program and budget administration to free staff resources. He informs all public health sites of the temporary suspension.
-
Scenario Name: See a list of potential team member configurations to maximize service delivery capacity
Both the Location D and Location E Public Health centers have closed due to staff reductions. Bill would like to best maximize the skill sets of the remaining available staff. He checks the list of available staff to see their skill sets and finds that he can create a new team of service providers. He reports the situation to the Emergency Operations center and gets the go ahead to create the new team.
-
Scenario Name: See where individual staff members live in relation to Location C and Location E Public Health Centers
Bill needs to see where the members of a newly created team of service providers live in relation to Location C and Location E. Both Location C and Location E have similar local service demand so he checks a map of public health sites with regard to the addresses of all five team members. He finds that four of the five team members live closer to Location C than Location E. Bill assigns the new team to the Location C Public Health Center.
-
Scenario Name: Remotely log into the Continuity of Operations Decision Support System
Bill is at home after a long day at work but he still has administrative tasks to complete for an early morning Emergency Operations Center meeting. He remotely logs into the system to create some reports.
-
Scenario Name: Create reports modeled after current hard-copy reports
Bill needs to fill out some reports from home. In the past, he would have to bring paper forms home but the Continuity of Operations decision support system provides standard public health department reports in an electronic format. Bill completes his administrative tasks and will print the saved reports in the morning for use in a meeting.
The user profile and resulting scenarios demonstrate that public health decision-makers need to rely on timely, up-to-date information to coordinate the delivery of essential services to clients. In the setting under study and at the time of completion of this study, decision-making data were aggregated and formatted by hand in spreadsheets. In addition, as participants pointed out, up-to-date data regarding employees required ad hoc requests from planners to IT staff or Human Resources. Data update requests were historically not documented with a formal protocol and turn-around time could often take days. This highlights the need for a situation assessment tool that automatically pulls necessary data from different sources, aggregates that data, and presents them as manageable information in an appropriate format for decision-making.
The defined user information needs provided the framework for a new system that aims to address the identified gaps, labeled Public Health Awareness, Response And Operations Help (PHARAOH). The system based on PHARAOH aims to meet the information needs of public health decision-makers in times of normal and emergency operations. During the course of normal operations, decision-makers have a need for up-to-date reports of facility status, staff attendance and supply inventory; during times of emergency, the need for timely information becomes much more critical. Reports and maps of facility, staff and supply information filtered by individual site and on an aggregate site basis are needed for decision-making. In addition, decision-makers must be able to prioritize essential services to manage service delivery capacity in the face of staff reductions and surges in local service demand.
PHARAOH furthermore recognizes the required flexibility pertaining to service coordination. Some services are delivered by individual service providers. However, some services are delivered by a team of service providers with various skill sets working together. Each service provider has a minimum skill set determined by job class. In addition, some service providers have necessary skills that are not indicated by job class but are determined by certifications and experience (example: ability to issue burial permits or give injections). Information regarding the total set of skills for each service provider, as well as service provider location, must be available to public health leadership for decision-making support.
By assessing supply inventories and the skills of available service providers as indicated by job class and experience, decision-makers can direct the movement of service providers and supplies between functioning facilities to build service provider teams with the necessary resources to deliver essential services. This approach maximizes site-local service delivery capacity to meet local service demand based on the total available pool of service providers and resources.
Discussion
The need for Continuity of Operations Planning for public health agencies is critical to maintain delivery of essential services to save lives and protect population health when disaster strikes. However, technology support for COOP in public health has not been extensively studied nor has the design of such technology been heavily pursued. As designers approach this under-explored area, they must be sensitive to the needs of many public health practitioner roles to encourage adoption of new decision support systems. Given that public health practitioners typically operate in low resource settings and have many constraints upon their time, any new technology must provide immediate, obvious value to the end user with a low impact training cost. One way to provide such value is to tailor decision support systems to the information needs of each user role.
This study highlights the creation of scenarios as narrative of a use episode. The set of these textual descriptions reflects the end user point of view and therefore allows for inclusion of parameters such as background information, unique circumstances and resources that may not be otherwise revealed to a designer entering the setting as an external entity. Scenarios provide insight into the users' goals, and their operational context making the workflow process and information needs explicit. They also serve as an archive documenting design rationale for future system revisions.
Understanding the information needs of public health leadership in critical situations is essential to the design of a decision support system that is reliable, valid and acceptable to the end users. This study highlights the complex and timely needs of public health decision-makers during a pandemic and the powerful contribution of scenario-based design as a tool to assess these needs and translate the assessment into system specifications. The study is limited in its sample size as it focused on COOP planners of one public health department. It does however present a methodology and set of scenarios that can be applicable to other settings.
Although decision-making needs vary by disaster, geographic area and the size of a public health agency, these scenarios are applicable in other settings in that all public health leaders must manage staff, services, resources, and facilities during a crisis. The need for situational awareness is similar in disasters of the same type. Demand for services is similar in disasters of similar magnitude that occur in similarly-sized population areas. As such, we believe that these scenarios of use can be applied in the design of public health COOP decision support technology in other large urban areas.
Acknowledgments
The authors would like to thank John Hartman and Dee Totten for their invaluable contributions and feedback.
References
- 1.Logue JN. The public health response to disasters in the 21st century: reflections on Hurricane Katrina. Journal of environmental health. 2006;69(2):9–13. [PubMed] [Google Scholar]
- 2.Somers S. Survey and Assessment of Planning for Operational Continuity in Public Works. PUBLIC WORKS MANAGEMENT AND POLICY. 2007;12(2):451–464. [Google Scholar]
- 3.Buehler JW, Whitney EA, Berkelman RL. Business and public health collaboration for emergency preparedness in Georgia: a case study. BMC Public Health. 2006;6:285. doi: 10.1186/1471-2458-6-285. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Carroll JM. Five reasons for scenario-based design. Interacting with computers. 2000;13(1):43–60. [Google Scholar]
- 5.Carroll JM. Scenario-based design : envisioning work and technology in system development. New York: Wiley; 1995. [Google Scholar]
- 6.Rosson MB, Carroll JM. Morgan Kaufmann series in interactive technologies. San Fancisco: Academic Press; 2002. Usability engineering : scenario-based development of human-computer interaction. [Google Scholar]
- 7.Friedman CP, Wyatt JC, Owens DK. Evaluation and Technology Assessment. In: Shortliffe EH, Cimino JJ, editors. Biomedical Informatics: computer applications in health care and biomedicine. Springer; 2006. pp. 403–443. [Google Scholar]
- 8.Blomberg J, et al. Ethnographic Field Methods and Their Relation to Design. In: Schuler D, Namioka A, editors. Participatory design : principles and practices. Hillsdale, N.J.: L. Erlbaum Associates; 1993. pp. 123–155. [Google Scholar]
- 9.Ulin PR, Robinson ET, Tolley EE. Qualitative methods in public health : a field guide for applied research. San Francisco, CA: Jossey-Bass; 2005. [Google Scholar]
- 10.Krathwohl DR. Methods of educational & social science research : an integrated approach. New York; Harlow, England: Longman; 1998. [Google Scholar]