Abstract
Responding to public health emergencies requires rapid and accurate assessment of workforce availability under adverse and changing circumstances. However, public health information systems to support resource management during both routine and emergency operations are currently lacking. We applied scenario-based design as an approach to engage public health practitioners in the creation and validation of an information design to support routine and emergency public health activities. Methods: Using semi-structured interviews we identified the information needs and activities of senior public health managers of a large municipal health department during routine and emergency operations. Results: Interview analysis identified twenty-five information needs for public health operations management. The identified information needs were used in conjunction with scenario-based design to create twenty-five scenarios of use and a public health manager persona. Scenarios of use and persona were validated and modified based on follow-up surveys with study participants. Scenarios were used to test and gain feedback on a pilot information system. Conclusion: The method of scenario-based design was applied to represent the resource management needs of senior-level public health managers under routine and disaster settings. Scenario-based design can be a useful tool for engaging public health practitioners in the design process and to validate an information system design.
Keywords: Information needs, scenario–based design, public health, continuity of operations planning, emergency preparedness
1. Introduction
Disasters, such as the earthquake in Haiti and Hurricane Katrina, underscore the need for a public health system that is both responsive to emergencies and capable of carrying out routine public health services. During disasters informed decision-making requires up-to-date assessment and coordination of available resources under dynamic conditions. Continuity of Operations Planning (COOP) refers to an organization’s ability to deliver essential services before, during, and after an emergency [1–3]. In the case of public health clinics, COOP involves the accurate and rapid assessment of needed staff under changing and possibly adverse conditions. Essential public health functions must be delivered in the face of competing resources during an emergency. Targeted planning, an agreed-upon chain of command, and ongoing assessment of resources are critical for meeting the changing needs for public health services during and in the aftermath of an emergency. A prior systematic review of the published literature indicates a lack of studies about technology support for public health COOP[4].
Developing public health information systems that assist with COOP during routine, as well emergency situations, requires an understanding of the day-to-day local health jurisdiction operations, the continuity of operations plan, and the effect of emergencies on routine operations. This in turn requires a design methodology that involves practitioners to accurately model current and envisioned work practices. This paper explores the use of scenario-based design as an approach to engage public health practitioners in the creation and validation of an information designs to support routine and emergency work activities.
Scenario-based design is an established method that focuses on the activities of practitioners to engage them in the design process of information systems to support their work[5–9]. Scenario-based design has been used successfully in a variety of contexts, including shared home care technology[10], clinical protocol authoring[11] and e-health[8]. Scenarios are descriptions of common work activities performed by individuals who occupy specific roles in specific contexts[6, 9]. Scenarios describe everyday tasks in plain language, making it easier for informaticians and designers to discuss designs with professionals with specialized domain knowledge. Scenarios are narratives of a successful path through an information system. There are few documented efforts of scenario-based design in the public health context.
Personas are representations of stakeholder roles that are useful to guide the information design process[12–15]. Personas describe the type of person who will interact with an information system. Designers use personas as a reference aid design. Used in conjunction with scenario-based design, personas are an effective way to help designers improve information designs[15].
The requirements for information systems to support public health operations during routine and emergency operations are currently under-explored and poorly understood. With the knowledge that technologies designed to support continuity of operations during emergencies are most effective when they conform to the day-to-day activities[16], we worked with public health professionals that oversee operations for multiple clinics in a large local health jurisdiction to document their routine work practices during normal and emergency situations. Our early work demonstrated that qualitative methods are valuable in documenting workflow in public health settings[17] and that scenario-based design can be an effective way to create and validate designs for continuity of operations planners at a large local health jurisdiction[18]. This study afforded us the opportunity to further explore the value of scenario-based design in the public health domain. In addition, we tested the concept of reusable design knowledge[19] applied to a different public health role by implementing scenarios from previous design work[18] in a operations support prototype.
2. Methods
We investigated the information needs, communications and work processes of managers of multiple facilities that require support during routine and emergency operations at a large local health jurisdiction. We engaged participants using a process of participatory design[20, 21] that included semi-structured interviews, qualitative data analysis[22, 23], scenario-based design[5–11, 24–33], validation surveys and a think aloud protocol[34]. Figure 1 illustrates an overview of the study methods and design process. The research team consisted of a public health informatics faculty member (AT) and a biomedical and health informatics doctoral student (BR). Prior approval for this study was granted by the Institutional Review Boards (IRB) of the University of Washington and the local health jurisdiction.
Figure 1.
Overview of study methods and design process
2.1. Setting
The study setting is the largest health local health jurisdiction in Washington State, and the 11th largest local health jurisdiction in the US. The Community Health Services Division (CHS) runs 14 public health centers and clinics throughout the local county. Since 2003, each division of the local health jurisdiction has been responsible for developing a continuity of operations plan. In the event of an emergency, one strategy that was suggested called for fourteen clinics to consolidate into a smaller number of major service delivery areas.
2.2. Participants
In consultation with CHS division managers, we focused our interviews on area managers, who play a key role in decision-making during routine and emergency operations. Area managers are responsible for overseeing the operations of several public health centers within an extended geographic area of the local health jurisdiction. Their job responsibilities include making decisions regarding what services to provide at each clinic based on available staff and client demand, and communicating recommendations up the command chain during emergencies. Three of the five area managers have training as nurses and all had at least ten years of supervisory experience.
2.3. Interviews
We created an interview guide for semi-structured interviews based on prior field work at the study setting, analysis of COOP documents from the local health jurisdiction and a systematic review of the literature covering public health, information technology and continuity of operations planning[4]. The guide was piloted with two former area managers experienced with COOP at the local health jurisdiction. The interview guide covered the following six areas: 1) Professional public health background of each participant; 2) Normal duties, decisions and use of technologies; 3) Decisions and communication channels used during routine vs. emergency operations; 4) Barriers and needs to successful communication with staff; 5) Current understanding of the continuity of operations plan; 6) Information about a recent regional snow storm as critical incident.
We conducted semi-structured interviews with all five area managers. To better understand organizational lines of communication, we identified two staff that frequently interacted with the area managers to relate work processes to other roles: 1) a float pool manager who assigns temporary staff to cover work force gaps and 2) a clinical supervisor. Informed consent was obtained prior to each interview. Interviews were conducted over a four-month period from January 2009 to April 2009. All interviews were conducted on-site and lasted approximately one hour. With the permission of each participant, the interviews were audio recorded using a digital recorder. Interviews were transcribed and imported into atlas.ti software[35] for coding and analysis.
2.4. Data Analysis
Two researchers (AT and BR) iteratively coded the interview transcripts for area manager information needs. Using constant comparative method we identified characteristics of the context of work, modes of communication, common information needs and work activities. Characteristics of the context of work were descriptively coded and grouped into themes[22]. Information needs were descriptively coded based on goals and activities. A public health area manager persona was developed from background characteristics and job responsibilities of study participants. Using scenario-based design methods, scenarios of use were developed from information needs and work activities identified from interviews. The context of use for the persona and scenarios is a snow storm based on a recent critical event at the time of the study. Where applicable, we altered scenarios of use from prior work and applied them to the snow storm context to test the concept of reusable design in a public health setting. Specifically, scenarios of use for a public health continuity of operations planner persona in the context of a pandemic[36] were selected and repurposed to support the activities of an area manager persona in a snow storm context based on similarity of information needs.
2.5. Validation studies
Identified information needs, the persona and corresponding scenarios of use were validated through member-checking with study participants[37]. Twenty-five information needs and scenarios of use were validated through a survey sent to all participants. All five participants completed the survey. Response to all survey items was 100% for all five participants. Participants were asked to assess each information need and the scenarios of use by answering the following questions:
Is this information need accurately reflected?
Does this scenario accurately describe activities that meet this need?
Participants were also asked to provide comments about each information need and scenario of use if they felt descriptions were not accurate or if additional feedback might be helpful.
3. Results
We developed descriptions that characterized the context of work for area managers and identified twenty-five individual information needs. In addition, we created a persona based on common participant characteristics and developed twenty-five scenarios of use designed to satisfy identified information needs. The design of the scenarios was guided by the persona description and a critical incident ( a recent snow storm).
3.1. Characterizing the context of work
Several themes emerged that describe common characteristics of the area managers’ work. These emergent themes include regular management of small emergencies during routine operations, colleague commitment to serving the local community in all circumstances and operating in an environment that gives responsibility without ultimate decision-making authority. Two emergent themes that were barriers to management of operations were identified as non-standard communications and work practices and lack of access to up-to-date information. In addition, we described and characterized the differences between communication during routine operations and communication during emergency operations.
3.1.1. Management of small emergencies during routine operations
Nearly all participants expressed the perception that urgent routine public health operations share many characteristics of continuity of operations planning. The need to respond to changing workplace conditions, staff shortages, and budget cuts have become routine operations in public health management. Decisions regarding rescheduling and reassigning personnel, continuing operations in resource- and technology-poor environments, and making decisions with often insufficient data are not isolated to natural disasters.
“We’re basically continuity of operations everyday because we have a significant amount of call-out with our staff and are basically understaffed…You’re having to do that juggling every day”
Participant #1
“We’re so stretched there’s almost always somebody who’s out or gone or vacant for some reason so I end up having to step in more often.”
Participant #2
3.1.2. Commitment to serving the local community
Another recurrent theme was dedication and commitment to providing essential public health services to communities in times of need. Participants had worked in public health an average of 15 years. The critical incident story revealed a dedication to do whatever was necessary to ensure the safety of their staff and at the same time provide essential services to the community. One manager reported walking two miles in the snow to check the physical facility and make sure WIC checks were available for local health jurisdiction clients. Another manager discussed the culture of commitment in public health in the context of a recent snow storm:
“I have a doc[tor] that lives right up the hill. She came. Everyday. I had another doc[tor] who lives by Children’s. She skied in. Took her four hours.”
Participant #2
3.1.3. Responsibility without ultimate decision-making authority
A commonly expressed frustration expressed was the lack of decision-making authority during emergency operations. Participants felt they had the expertise to make sound decisions regarding personnel and clinic operations during an emergency, but that because of the chain of command they were only in a position to inform administrators at a higher level but not to make the decision. This led to frustrations and what was described as a culture of “learned helplessness.”
“I can make decisions all day long but I need to know if I am really the decision-maker or not. I think if anything creates paralysis it is that. It’s the sense that there’s been a lack of recognition for proactive decision-making in the past so people sit back and say, “I’m only going to do what I am told because I’m worried about the consequences of acting.”
Participant #1
3.1.4. Non-standard communications and work practices
Participants described difficulty due to lack of standardized communications and work processes across clinics that hindered routine and emergency operations.
“We have no standardization, which makes it very hard. Processes and procedures are so different because the sites have evolved differently and we do not have a centralized infrastructure for our clinical services, so we’ve all kind of developed our own processes. So when you bring somebody from another site, it’s like you are bringing them on to another planet, sometimes.”
Participant #4
3.1.5. Lack of access to up-to-date information
Lack of access to up-to-date information to facilitate decision-making was a commonly expressed barrier. Managers wanted information about clinic scheduling, staff contact information and availability, clinical and emergency protocols, and current information about the physical state of the clinic facility. In the case of emergency situations, nearly all participants said they needed remote access to a centralized repository of key information so they could make informed decisions. Participants indicated that they required information for decision-making from a variety of sources including employee contact information, emergency protocols and clinic schedules. As one manager commented:
“As a manager and a supervisor, I always like to have as much information as I possibly can, so I am really deliberate when it comes to making recommendations and making decisions.”
Participant #3
3.1.6. Communication during routine operations
Because area managers oversee more than one site, communication with other sites was important. The major modes of communication with staff from the other clinic facility were telephones including smart phones (primarily Blackberries) and e-mail. Routine communications between area managers and clinic staff were generally face-to-face. However, telephones and e-mail communication were also used frequently. Monthly meetings with staff and administrators generally took place face-to-face.
3.1.7. Communication during emergency operations
The main change from routine communications is teleconferencing with senior health administrators and incident command personnel that takes place several times per day before, during, and after a disaster. During the most recent snow storm, participants continued to rely heavily on phones and e-mail during emergencies, but relied more heavily on mobile phones during emergencies to assess clinic status, gather information, and make decisions regarding opening or closing the clinics. E-mail was used to send more general messages out to staff. Text messaging via mobile phone was not used either routinely or during emergencies. Communication with the public was primarily through the website, by phone or in the case of clinics closing was by a written note placed on the entrance door by a designated staff member who lived near to the facility.
3.2. Information needs
Twenty-four of twenty-five information needs were validated as accurately reflected by at least 4 out of 5 of participants. High levels of agreement make these information needs good candidates for information design to support the work of area managers. Levels of agreement for validation of information needs are shown in Table 1.
Table 1.
Participant validation of information needs as accurately reflected
| Level of agreement | Validated information needs |
|---|---|
| 5 of 5 participants | 15 information needs |
| 4 of 5 participants | 9 information needs |
| 3 of 5 participants | -- |
| 2 of 5 participants | 1 information need |
|
| |
| Total | 25 information needs |
Information needs are grouped into four categories: staff management, client management, general operations and continuity of operations. Staff and client management information needs refer to features that involve specific information required to make decisions related to staff and clients. General operations information needs refer to features that could support many facets of work. Continuity of operations information needs refer to features that could support work during emergency situations.
3.3. Area manager persona
The area manager persona, Mary, was created by identifying common features of each participant’s experiences, education, skills, training, and job history. We drew from what we learned about the critical incident to create the winter snow storm context description. The persona description was validated and revised by soliciting feedback from all five area managers. Box 1 describes the area manager persona.
Box 1. Mary: A Public Health Area Manager Persona.
Mary is an area manager who oversees the operations of two public health centers in a large local health jurisdiction that has ten major centers and a number of smaller satellite clinics. She is a registered nurse by training and holds an MPH degree. As such Mary has 20 years experience in public health with 12 of them in supervisory roles. Prior to her career in public health, she had experience as a provider of direct client services in hospital and outpatient venues. Mary has incident command experience managing clinic operations during inclement weather emergencies and is a contributing member of her division’s continuity of operations workgroup.
Mary’s responsibilities as an area manager include the following:
Direct clerical and clinic supervisors who manage line staff
Direct line staff as necessary
Manage public health center systems at an overarching level
Manage multiple clinics within a dispersed geographic area
Ensure availability of resources so each clinic can deliver program services as part of day-to-day operations
Know key personnel within each program at each public health site
Spend time at each of her sites to maintain personal management relationships with supervisors and staff
Attend division meetings at the local health jurisdiction’s administration building (“downtown”)
Communicate and plan with other area managers via projects and work groups (example: managing interpretation services capacity)
Communicate and plan with public health program managers who direct programs (example: family planning and immunizations)
Communicate recommendations to division leadership
Participate in joint leadership decision-making with other area managers, public health program managers and division leadership
Debrief with other area managers after emergencies
Receive updates from site supervisors about staff scheduling issues
Receive updates from site supervisors who monitor staff attendance and vacation and sick leave balances
Maintain up-to-date supervisor and staff contact information
Notify staff of clinic open/closure status
3.4. Context of use
Scenarios of use were designed to reflect the activities that Mary would engage in during a snow storm. The snow storm context is described below in Box 2.
Box 2. Snow Storm Context of Use.
Critical Incident: Managing the Effects of a Snow Storm
It is mid-December and a snow storm has shut down most public and private operations in the area. The county covers a wide geographic area with many microclimates, so conditions vary from bad to worse. Many areas are without power. Roads are blocked by snow and ice. Public transit is running sporadically or not at all. Many public health staff members commute to work via car or bus. Some staff members live far from the public health sites where they work. Mary must use the local health jurisdiction’s operations support information system to help manage the effects of the snow storm at her public health centers (Location A and Location B).
3.5. Scenarios of use
Twenty of the twenty-five scenarios of use were validated by at least 4 out of 5 participants as accurately describing activities that met identified information needs. Twenty-three scenarios of use were validated by a majority of participants. Of seven scenarios of use employed as a test of reusable design knowledge for a different role in a different context, six of the seven were validated by 5 of 5 participants as accurately describing activities that met identified information needs. One of the scenarios employed for design reuse was validated by only two out of five participants. Only two scenarios of use were not validated by a majority of participants. Validated scenarios of use are shown in Table 3.
Table 3.
Participant validation of scenarios as describing activities that meet information needs
| Level of agreement | Validated scenarios of use | Validated scenarios from prior work |
|---|---|---|
| 5 of 5 participants | 15 scenarios of use | 6 scenarios of use (reusable design test) |
| 4 of 5 participants | 5 scenarios of use | -- |
| 3 of 5 participants | 3 scenarios of use | -- |
| 2 of 5 participants | 2 scenarios of use | 1 scenarios of use (reusable design test) |
|
| ||
| Total | 25 scenarios of use | 7 scenarios of use (reusable design test) |
3.6. Impact of feedback for design
Scenarios provide a means for a dialog between the designer and the public health practitioner. A majority of the scenarios of use were validated by most of participants. Below are three examples of scenarios that were validated by 5 of 5 participants (Box 5, 6 and 7). These three scenarios are examples of successful reusable design. They were implemented and further validated in an operations support prototype. Scenarios were revised, when necessary, based on participant comments from the survey. Descriptions of participant concerns and comments follow each of these successfully validated scenarios to illustrate the importance of iterative engagement with the people who will use an information design.
Box 5. Scenario of Use – Successful Design Reuse.
Scenario Name: See where individual staff members live in relationship to each public health centerd
Mary needs to see where individual staff live in relationship to each public health center for staffing purposes. She checks a map of public health sites with regard to the addresses of a list of staff members. Mary can only see the staff she manages and their direct reports from the public health centers she manages.
Box 6. Scenario of Use – Unsuccessful Design Reuse.
Scenario Name: See and manage a list and map of public health site statuse
Mary needs to find out which facilities are functioning or non-functioning. In order to do so, she runs a report that shows a list of all public health sites as well as a map with the status of each site. She sees that the Location B Public Health Center is shown as a functioning site but knows that it is closed due to excessive staff reductions. She edits the status of the Location B site to indicate its non-functioning status.
Box 7. Scenario of Use – Unsuccessful Design Based on a Validated Information Need.
Scenario Name: Move Women, Infants and Children (WIC) case load records to another public health centerf
The Location B public health center is closed for the foreseeable future. Mary must move records for the WIC program case load to the Location A public health center. Mary logs into the operations support information system and creates a batch process that automatically creates a temporary facility assignment for all Location B WIC program participants. When the current inclement weather emergency passes, Mary will simply delete the temporary facility assignment and the entire Location B WIC case load will return to its normal facility assignment. A log of the case load records movement is created.
Feedback also provided a means for communicating potential concerns or contextual information about the proposed scenarios. As an example, we have included two scenarios of use that were not validated by a majority of participants (see Box 6 and 7). One of these scenarios is an example of unsuccessful design reuse (Box 6) while the other shows an example of unsuccessful design based on a new information need (Box 7).
Participants validated the scenario in Box 3 but also noted the need to make functionality available to other roles.
Box 3. Scenario of Use – Successful Design Reuse.
Scenario Name: See individual staff member information regarding job class, job skills, home address, assigned work site and contact informationb
Mary needs to quickly and securely access information for individual staff members as automatically updated from the human resources database to assess possibilities for remote work and other staff management logistics. Mary runs a report to see job class, job skills, home address, assigned work site and contact information for each staff person. Mary can access information for the staff she manages and their direct reports
“Since assignment may be coordinated centrally… it’ll be important for a dispatcher to easily access this info.”
Participant #5
Participants noted that a program – the foster care program – had been eliminated since the beginning of the project and creation of the scenario of use in Box 4. In addition, they noted that organizational hierarchy required that they report information up to receive commands. The scenario in Box 4 was revised accordingly.
Box 4. Scenario of Use – Successful Design Reuse.
Scenario Name: See a list of prioritized services and the necessary skills and resources for deliveryc
Staff reductions due to the snow storm are negatively affecting the ability of all public health centers to deliver mission critical services. Mary checks the total list of services delivered by her public health centers. Mary sees that employees who deliver non-essential services have skills that can be used to deliver essential services. Mary reports the situation to division leadership who report to the Emergency Operations Center. She gets the go ahead to temporarily suspend the on-site delivery of services related to community education, maternity case management and budget administration to free staff resources. She informs her public health sites of the temporary suspension.
“Onsite services don’t vary too much except that some sites provide primary care and others don’t. In this scenario, foster care passport wouldn’t apply. Something like maternity case management would apply.”
Participant #3
“I’d report status to CHS Division. CHS would contact Area Command for offers to deploy site staff.”
Participant #5
The scenario in Box 5 was validated by 5 of 5 participants. One participant noted that automation for data visualization would be useful since a manual process for mapping staff already existed as a result of emergency preparedness activities for flooding. One participant was unsure of whether geospatial visualization was necessary for staffing but nevertheless validated the scenario. Privacy issues with regard to mapping staff information were raised.
“It would be great to have this info, but even nicer to know any… staff in the area since staff from all centers live all over the county. Would there be a privacy issue here?”
Participant #2
A majority of participants did not validate the scenario in Box 6 as describing an activity that supported one of their information needs (2 of 5 participants). Neither was the information need that forms the goal for this scenario validated by a majority (2 of 5 participants). Participants noted that it would not be Mary’s job to determine facility closure and the need for a clear authority about edits and updates.
“I’m not sure how geospatial visualization would indicate a site was functional or not as I’m not familiar with this technology. If this were available, this could be done by a centralized operations response team. An area manager would just be informed…”
Participant #3
The scenario in Box 7 was not validated by a majority of participants (2 of 5). However, a majority (4 of 5 participants) validated the information need that forms the goal of this scenario. This scenario is an example of unsuccessful design based on a new, validated information need. Participants raised concerns about whether or not moving case load would fall to an area manager and whether or not case load should be moved but instead should be viewable from other facilities. In addition, they noted organizational and technical issues for technology administration, client information management and client scheduling between state and local information systems
“Functionally this sounds great, but we are constrained by State requirements, mostly limited by the way the technology is set up. CIMS is a set system owned and managed by the State. They have agreed in principle to this need, and have agreed to work with us in an extreme emergency, but they don’t seem to be moving towards a more flexible system for the long term”
Participant #1
3.7. Prototype development
Although resource management systems exist for clinical settings, we are unaware of a commercial or open source information system that meets public health needs for integrating and visualizing employee data from different sources to assist with decision-making. Access to data such as job title, job summary, distinguishing characteristics, essential duties, required knowledge and skills, informal experience, licensing, certification requirements and employee ZIP code are key to decision-making about delivery of necessary services under varied conditions of operation. As a step to address this gap, we modified an existing open source public health GIS tool[38] to support the scenarios from Boxes 3, 4 and 5.
The resulting prototype was dubbed the Continuity of Operations Data Analysis (CODA) system. CODA was validated through a formative evaluation with two participants using a think aloud protocol[34]. The CODA system and evaluation have been described previously [39, 40]. CODA provides resource management and location information about staff at public health centers. The CODA prototype uses a MySQL database of employee information in conjunction with Google Maps mashups to geospatially visualize the location of personnel and public health centers. Figure 2 shows a selection of county public health centers in relation to employees’ home ZIP codes in CODA.
Figure 2.
CODA visualization of selected public health centers in relation to employee residences
4. Discussion
Comments and feedback about the three selected scenarios that were reused and validated indicate that even when scenarios are reusable, it is important to pay attention to participant comments that situate use in context. Feedback about the scenario that was reused and not validated suggests that when an information need is not validated by a majority of participants, then a corresponding scenario that supports that information need will also not be validated. Comments about the scenario that was developed to satisfy a validated information need but subsequently was not validated suggest that we should not automatically assume that we can design for every identified information need before socio-technical issues are satisfied. Program changes and terminations during this study illustrate the dynamic nature of work and the need for changes to information designs as work processes change. Characterizing the context of work, as we have done in this study, can help inform the design process. However, regular, iterative member-checking with the people who will use an information system is critical to match designs to work processes.
Ninety-six percent of information needs were validated by 4 of 5 participants; 80% of scenarios of use were validated by 4 of 5 participants. The lower level of validation for scenarios of use in comparison to corresponding information needs is a reflection of the design process. Information needs were identified directly from interviews with participants whereas scenarios of use were created or adapted by an informatics researcher working apart from the study group. The survey results validating information needs and scenarios represent an example of a design iteration where designs were created and adjusted collaboratively with public health practitioners.
Personas, in conjunction with scenarios, are used to capture the specific perspective of an individual who uses an information system to accomplish work tasks. However, some tasks are common and may transfer between individuals who have different roles, particularly under the changing demands of emergency situations. Indeed, some information needs of the direct reports of area managers were identified as those of the area managers because leaders change roles based on operational circumstances. Accordingly, some scenarios better support the work of direct reports rather than area managers.
In 1998, public health professionals were asking for information systems that supported scheduling and calendar management[41]. Our study shows the need for integrated support of those activities still exists thirteen years later. In addition, participants reported their daily work entailed many aspects of continuity of operations planning and they employed continuity of operations principles. This finding is unexpected because we assumed that daily work would have a sense of routine. Area managers continuously juggle resources and staff in order to provide essential services to the community. At the time of this study, the local health jurisdiction was responding to an ongoing financial crisis due to state and county budget cuts. There were staff and resource shortages, and many employees were called upon to fulfill more than one role. In this sense, the day-to-day operations reflect continuity of operations planning because local health jurisdictions typically operate in low-resource settings. Our findings underscore the need for technologies that support continuity of operations, but are integral to routine activities. Under emergency conditions, tasks simplify and people generally apply familiar processes to the dynamic situation. Technologies that are familiar and flexible enough to support routine essential activities in a changing context will be most successful during an emergency.
Based on our results, we suggest four areas that informatics research should explore for technology support of continuity of operations in public heath. The areas are not defined by explicit boundaries and are non-exclusive of each other. Areas for further study are: 1) Technology support for staff attendance, personalized contact lists, scheduling, and staff location; 2) Technology support for remote access for all public health roles; 3) Integrated applications to support flexible management of WIC caseload; 4) Information systems to support emergency document and protocol management.
5. Limitations
A limitation of this study is that it looks at one group of practitioners at a single large metropolitan local health jurisdiction; therefore the results may not be generalizable to local health jurisdictions of different sizes or locations. The identified information needs are based primarily on the recall of study participants about a recent snow storm, and these same needs may not transfer to another type of emergency such as an earthquake. However the events of a recent snow storm were fresh in the study participants’ minds and they often summarized their reflections based on the experience of other emergencies such as communicable disease outbreaks and wind storms. This study focuses on clinical services. Most local health jurisdictions carry out a broad set of essential services such as communicable disease tracking and environmental monitoring which may have different information needs and related activities. An expansion of this work is being applied to additional settings and services to better understand the generalizability of the findings.
6. Conclusions
The description of work context, information needs, persona, and scenarios of use resulting from this study are important because they form a baseline understanding of complex public health work processes for information designers. These scenarios can be used as a blueprint for design of systems and may be modified in other settings for different conditions. The methods we used are important to designers in the public health domain because they transcend technology implementation. These methods could just as easily be used in the design of a notifiable conditions information system or an immunization registry. However, an important lesson for researchers and designers to remember is the necessity of member- checking with practitioners for whom an information system is being designed. Toward this end, the usefulness of personas as a tool for engagement became evident when area managers began referring to the Mary persona as if she were a real person in relation to work activities described in the scenarios of use.
The results of this study provide additional evidence that scenario-based design is a useful methodology for information design in the public health context. We adapted seven scenarios that were created for a different public health role[18]. Six of seven of these scenarios were validated by 5 of 5 participants, suggesting that some activities in public health work are common across roles while others are specific to a given role and public health service. Although nearly all information needs and scenarios in this study were validated by a majority of study participants, most of these designs are conceptual in nature.
To explore the matching of conceptual information designs to the real world, we tested scenario-based design by implementing three scenarios in an operations support prototype and validated the implementation with two of our participants. The results demonstrate the value of scenario-based design, from concept to prototype, in the setting of a large local health jurisdiction through validated iterations.
Table 2.
Information Needs of Area Managers
| Information Need | Category | |
|---|---|---|
| 1. | Access to an automatically updated staff roster for use during emergencies | Staff Management |
| 2. | Access to attendance information for a specific location with job class and skills information to cope with staff reductions | Staff Management |
| 3. | Access to attendance patterns and general family medical leave balance information | Staff Management |
| 4. | Access to attendance patterns and personal leave balance information | Staff Management |
| 5. | Access to prioritized services and employees with skills to deliver those services | Staff Management |
|
| ||
| 6. | Access to prioritized services and skills to deliver those services displayed in a matrix | Staff Management |
| 7. | Access to staff information including job class, skills, assigned work site, etc. for staff reassignment | Staff Management |
| 8. | Remote access to all attendance and leave balance information | Staff Management |
| 9. | Ability to move Women, Infants and Children (WIC) case load between facilities | Client Management |
| 10. | Access to attendance information and impact on client appointments | Client Management |
|
| ||
| 11. | Secure access to client appointment and contact for rescheduling | Client Management |
| 12. | Secure, remote access to client appointment and contact for rescheduling | Client Management |
| 13. | Remote access to jurisdiction information systems | General Operations |
| 14. | Remote access to digital versions of hard copy forms | General Operations |
| 15. | Access to contact information for other government facilities near public health centers | Continuity of Operations |
|
| ||
| 16. | Access to emergency and alternate contact information for division leadership and other decision-makers | Continuity of Operations |
| 17. | Access to job cards with standardized procedure information for use during emergencies | Continuity of Operations |
| 18. | Access to staff information related to emergency experience and availability during emergencies | Continuity of Operations |
| 19. | Easy access to updated continuity of operations plans and emergency protocols from the office | Continuity of Operations |
| 20. | Ability to remotely conduct a facility assessment and communicate facility status to others | Continuity of Operations |
|
| ||
| 21. | Geospatial visualization of employees and facilities on a map | Continuity of Operations |
| 22. | Mobile access to continuity of operations plans and emergency protocols | Continuity of Operations |
| 23. | Mobile and remote access to emergency and alternate contact information for division leadership and other decision-makers | Continuity of Operations |
| 24. | Remote access to continuity of operations plans and emergency protocols | Continuity of Operations |
| 25. | Geospatial visualization of functioning facilities on a mapa | Continuity of Operations |
This information need was validated by only two out of five participants. All other information needs were validated as accurately reflected by at least 4 out of 5 of participants.
Acknowledgments
This project was funded in part by the Centers for Disease Control and Prevention (CDC), Grant no. 5P01TP000297 and the CDC Center of Excellence in Public Health Informatics grant P01HK000027. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the Centers for Disease Control and Prevention. Dr. Reeder’s participation was supported by the National Library of Medicine Medical Informatics Training Grant T15LM007442-08. We wish to thank the study participants for their time and cooperation, Jim Wallace for his help with figures, John Hartman for providing public health expertise, Dr. George Demiris and Kathy Hall for their review of this manuscript.
Footnotes
This scenario was validated by 5 of 5 participants as accurately describing an activity that meets an identified information need. It is a successful example of design reuse from previous work[18].
This scenario was validated by 5 of 5 participants as accurately describing an activity that meets an identified information need. It is a successful example of design reuse from previous work[18].
This scenario was validated by 5 of 5 participants as accurately describing an activity that meets an identified information need. It is a successful example of design reuse from previous work[18].
This scenario was not validated by a majority of participants as accurately describing an activity that meets an identified information need. It is an unsuccessful example of design reuse from previous work[18].
This scenario was not validated by a majority of participants as accurately describing an activity that meets an identified information need. It is an example of unsuccessful design from new work.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.Rozek P, Groth D. Business continuity planning. It’s a critical element of disaster preparedness. Can you afford to keep it off your radar? Health Manag Technol. 2008;29:10–2. [PubMed] [Google Scholar]
- 2.Dawes SS, Cresswell AM, Cahan BB. Learning From Crisis: Lessons in Human and Information Infrastructure From the World Trade Center Response. Social Science Computer Review. 2004;22:52–66. [Google Scholar]
- 3.Somers S. Survey and Assessment of Planning for Operational Continuity in Public Works. PUBLIC WORKS MANAGEMENT AND POLICY. 2007;12:451–464. [Google Scholar]
- 4.Reeder BP, Turner AM, Demiris G. Use of Technology to Support Information Needs for Continuity of Operations Planning in Public Health: A Systematic Review. Online Journal of Public Health Informatics. 2010;2:20. doi: 10.5210/ojphi.v2i1.2855. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Carroll JM. Scenario-based design: envisioning work and technology in system development. New York: Wiley; 1995. [Google Scholar]
- 6.Carroll JM. Five reasons for scenario-based design. Interacting with computers. 2000;13:43–60. [Google Scholar]
- 7.Carroll JM. Making use is more than a matter of task analysis. Interacting with Computers. 2002;14:619–627. [Google Scholar]
- 8.Sutcliffe A, Thew S, De Bruijn O, Buchan I, Jarvis P, McNaught J, Procter R. User engagement by user-centred design in e-Health. Philosophical transactions Series A, Mathematical, physical, and engineering sciences. 2010;368:4209–24. doi: 10.1098/rsta.2010.0141. [DOI] [PubMed] [Google Scholar]
- 9.Go K, Carroll JM. The Blind Men and the Elephant: Views of Scenario-Based System Design Interactions. 2004;11:44. [Google Scholar]
- 10.Hägglund M, Scandurra I, Koch S. Scenarios to capture work processes in shared homecare--From analysis to application. International Journal of Medical Informatics. 2008 doi: 10.1016/j.ijmedinf.2008.07.007. In Press, Corrected Proof. [DOI] [PubMed] [Google Scholar]
- 11.Weng C, Gennari JH, McDonald DW. Scenario-based participatory design of a collaborative clinical trial protocol authoring system. AMIA … Annual Symposium proceedings/AMIA Symposium. AMIA Symposium; 2003. p. 1051. [PMC free article] [PubMed] [Google Scholar]
- 12.Cooper A. The inmates are running the asylum. Indianapolis, IN: Sams; 1999. [Google Scholar]
- 13.Pruitt J, Grudin J. Personas: practice and theory. Proceedings of the 2003 conference on Designing for user experiences; San Francisco, California: ACM; 2003. [Google Scholar]
- 14.Chang YN, Lim YK, Stolterman E. Personas: from theory to practices. Proceedings of the 5th Nordic conference on Human-computer interaction: building bridges; Lund, Sweden: ACM; 2008. [Google Scholar]
- 15.Grudin J, Pruitt J. Personas, Participatory Design and Product Development: An Infrastructure for Engagement. Proceedings of the Participatory Design Conference; 2002. pp. 144–161. [Google Scholar]
- 16.Alesi P. Building enterprise-wide resilience by integrating business continuity capability into day-to-day business culture and technology. Journal of Business Continuity & Emergency Planning: Henry Stewart Publications LLP. 2008:214–220. [PubMed] [Google Scholar]
- 17.Turner AM, Ramey J, Lee S. Connecting public health IT systems with enacted work: report of an ethnographic study. AMIA … Annual Symposium proceedings/AMIA Symposium. AMIA Symposium; 2008. pp. 737–41. [PMC free article] [PubMed] [Google Scholar]
- 18.Reeder B, Demiris G. Building the PHARAOH Framework Using Scenario-Based Design: A Set of Pandemic Decision-Making Scenarios for Continuity of Operations in a Large Municipal Public Health Agency. Journal of Medical Systems. 2010:34. doi: 10.1007/s10916-009-9288-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Reeder B, Hills RA, Demiris G, Revere D, Pina J. Reusable design: A proposed approach to Public Health Informatics system design. BMC Public Health. 2011 doi: 10.1186/1471-2458-11-116. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Blomberg J, Giacomi J, Mosher A, Swenton-Wall P. Ethnographic Field Methods and Their Relation to Design. In: Schuler D, Namioka A, editors. Participatory design: principles and practices. Hillsdale, N.J: L. Erlbaum Associates; 1993. pp. 123–155. [Google Scholar]
- 21.Oostveen A-M, Besselaar Pvd. From small scale to large scale user participation: a case study of participatory design in e-government systems. Proceedings of the eighth conference on Participatory design: Artful integration: interweaving media, materials and practices; Toronto, Ontario, Canada: ACM; 2004. [Google Scholar]
- 22.Boyatzis RE. Transforming qualitative information: thematic analysis and code development. Thousand Oaks, CA: Sage Publications; 1998. [Google Scholar]
- 23.Miles MB, Huberman AM. Qualitative data analysis: an expanded sourcebook. Thousand Oaks: Sage Publications; 1994. [Google Scholar]
- 24.Kaindl H. An integration of scenarios with their purposes in task modeling. Proceedings of the 1st conference on Designing interactive systems: processes, practices, methods, \& techniques; Ann Arbor, Michigan, United States: ACM; 1995. [Google Scholar]
- 25.Antón AI, Potts C. A Representational Framework for Scenarios of System Use. Requirements Engineering. 1998;3:219–241. [Google Scholar]
- 26.Dzida W, Freitag R. Making use of scenarios for validating analysis and design. Software Engineering, IEEE Transactions on. 1998;24:1182–1196. [Google Scholar]
- 27.Filippidou D. Designing with scenarios: A critical review of current research and practice. Requirements Engineering. 1998;3:1–22. [Google Scholar]
- 28.Haumer P, Pohl K, Weidenhaupt K. Software Engineering, IEEE Transactions. 1998. Requirements elicitation and validation with real world scenes; pp. 1036–1054. [Google Scholar]
- 29.Rolland C, Ben Achour C, Cauvet C, Ralyté J, Sutcliffe A, Maiden N, Jarke M, Haumer P, Pohl K, Dubois E, Heymans P. A proposal for a scenario classification framework. Requirements Engineering. 1998;3:23–47. [Google Scholar]
- 30.Stiemerling O, Cremers AB. The use of cooperation scenarios in the design and evaluation of a CSCW system. Software Engineering, IEEE Transactions on. 1998;24:1171–1181. [Google Scholar]
- 31.Galal GH, Paul RJ. A Qualitative Scenario Approach to Managing Evolving Requirements. Requirements Engineering. 1999;4:92–102. [Google Scholar]
- 32.Carroll JM. Making use: scenario-based design of human-computer interactions. Cambridge, Mass: MIT Press; 2000. [Google Scholar]
- 33.Rosson MB, Carroll JM. Usability engineering: scenario-based development of human-computer interaction. San Fancisco: Academic Press; 2002. [Google Scholar]
- 34.Jaspers MW, Steen T, van den Bos C, Geenen M. The think aloud method: a guide to user interface design. International Journal of Medical Informatics. 2004;73:11–12. doi: 10.1016/j.ijmedinf.2004.08.003. [DOI] [PubMed] [Google Scholar]
- 35.atlas.ti. Available at http://www.atlasti.com
- 36.Reeder BP, Demiris G. Building the PHARAOH Framework using Scenario-Based Design: A Set of Pandemic Decision-Making Scenarios for Continuity of Operations in a Large Municipal Public Health Agency. Journal of Medical Systems. doi: 10.1007/s10916-009-9288-3. (In Press) - Accepted for publication 3/30/2009 - JOMS418R1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Krathwohl DR. Methods of educational & social science research: an integrated approach. New York; Harlow, England: Longman; 1998. [Google Scholar]
- 38.Yi Q, Hoskins RE, Hillringhouse EA, Sorensen SS, Oberle MW, Fuller SS, Wallace JC. Integrating open-source technologies to build low-cost information systems for improved access to public health data. Int J Health Geogr. 2008;7:29. doi: 10.1186/1476-072X-7-29. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Turner AM, Reeder B, Wallace JA. CODA: Developing an Open Source Resource Management Tool for Public Health Preparedness. 2012 ACM SIGHIT International Health Informatics Symposium (IHI 2012); Miami, FL. 2011. (in review) [Google Scholar]
- 40.Wallace JA, Reeder B, Turner AM. CODA: A Continuity of Operations Data Analysis System for Public Health; Hyatt Regency Hotel, Atlanta, GA: Public Health Informatics Network - Centers for Disease Control; 2009. [Google Scholar]
- 41.O’Carroll P, Cahn M, Auston I, Selden C. Information needs in public health and health policy: Results of recent studies. Journal of Urban Health. 1998;75:785–793. doi: 10.1007/BF02344508. [DOI] [PMC free article] [PubMed] [Google Scholar]