Event Communication in a Regional Disease Surveillance System

Abstract

When real-time disease surveillance is practiced in neighboring states within a region, public health users may benefit from easily sharing their concerns and findings regarding potential health threats. To better understand the need for this capability, an event communications component (ECC) was added to the National Capital Region Disease Surveillance System, an operational biosurveillance system employed in the District of Columbia and in surrounding Maryland and Virginia counties. Through usage analysis and user survey methods, we assessed the value of the enhanced system in daily operational use and during two simulated exercises. Results suggest that the system has utility for regular users of the system as well as suggesting several refinements for future implementations.

Introduction

Disease surveillance systems allow users the ability to view large amounts of population health information ^{1, 2} and examine automated alerts that suggest increased disease activity. These systems require users to view and interpret which of these alerts or data streams are epidemiologically important. This interpretation is valuable information that can benefit other users.

In addition to the daily interpretation of data done by the users of disease surveillance systems, the ability to communicate local concerns and findings during a public health event to neighboring jurisdictions is of great public health importance. Systems such as the Health Alert Network and ProMed allow public health officials to communicate findings to peers ^{3, 4}. These systems, however, focus more on events after they are confirmed. However, public health officials also need constant situational awareness and a venue to share their concerns about increases in disease activity before a health emergency is declared. The ECC provides this venue.

Methods

System

The usefulness of the ECC was tested in the Electronic Surveillance System for the Early Notification of Community based Epidemics (ESSENCE) system in the National Capital Region (NCR)⁵. In the NCR, Maryland, Virginia, and the District of Columbia each have local copies of the ESSENCE system to monitor the health of their population. In addition, a regional version of the system allows them to share de-identified data across all three jurisdictions and provides a better understanding of the regional public health situation.

Users

NCR ESSENCE users include epidemiologists and public health officials from different levels of government. Maryland, Virginia, and the District of Columbia have users at the state/district level. Montgomery and Prince George’s counties in MD and Arlington, Alexandria, Fairfax, Loudoun, and Prince William counties in VA have users that review county level data. In addition, Virginia also has users from the northern Virginia regional health office.

Event Communication Component

The Johns Hopkins University Applied Physics Laboratory developed the ECC as a module for the ESSENCE system. The component is accessible by an “Event List” button in the navigation header of each screen. An Event List (Figure 1) displays the most recent events made by users. These events consist of: a text block where the user describes the situation; meta information such as location, age group, syndrome, and data sources affected; the onset date and duration; a status indicator that tells if the event is open or closed; and a ‘severity’ rank which indicates the user’s level of concern. The severity rank has five possible levels of concern: Info, Low, Guarded, Elevated, and High.

Figure 1.

The ECC Event List.

More details for each event are found by clicking a hypertext link that takes the user to a detailed event page (Figure 2) containing a running list of user comments regarding the event. This is where other users post responses to the events and discuss their interpretations, findings, or share similar event information.

Figure 2.

The ECC Event Details page.

In addition, the ESSENCE summary alert list screen (Figure 3) has been updated to include event information. The summary alert list is displayed as a table with each syndrome assigned to a column and each geographic region assigned to a row. Each syndrome/location cell in the table contains a row of stars. When mathematical detection algorithms identify a statistically significant increase in disease ‘activity’ in a syndrome/location cell the stars change from grey to yellow or red depending on the significance level of the alert. Thus the summary alert list gives a single page overview of alarming data streams by syndrome and location. A second row of stars was added to each cell in the table to visualize user’s level of concern about events occurring in specific syndromes and locations. This updated summary alert list provides both statistical and epidemiologic information about potential disease threats to the community in a single view.

Figure 3.

The Summary Alert List with both mathematical and user-based alerts.

Exercises

The NCR users have participated in two regional exercises since the ECC was added to the NCR ESSENCE system. The first exercise took place from May 30, 2006 to June 2, 2006 and included three days of system use and a concluding meeting. During the first three days users were allowed one hour and thirty minutes to monitor simulated data in the ESSENCE system for the occurrence of a disease outbreak. Users worked from their offices, allowing them to fully simulate their normal ESSENCE review. Users were asked to complete a survey at the end of each day. At the wrap-up meeting, users were asked to summarize their experiences and present their findings to the entire group.

The scenario for this exercise was a Hepatitis A outbreak that spanned the three jurisdictions and was spread by a catering service. Two hundred and sixty-seven cases were injected on top of simulated background data over a three-day period.

The second exercise occurred October 3–6, 2006. Users consisting of many of the same participants from the previous exercise again worked from their offices over a three day period and met on the fourth day for a summary meeting. In this exercise, human cases of H5N1 influenza had been found in the UK, and users had to find a small outbreak of H5N1 influenza in the NCR that was buried in a wave of “worried well” patients flocking to local emergency rooms. Thirty-six cases of H5N1 influenza and 1,627 cases of “worried well” were injected over three days onto a background dataset.

Operational Use

The ECC became operational in the NCR ESSENCE system on May 1, 2006.

Data Analysis

During the first exercise 21 surveys were collected. The survey asked questions about the overall utility and specific features of the ECC. We were also able to analyze the events and comments created during the exercise.

In addition, website usage logs recorded when and how pages of the ECC were viewed by each user. These logs allowed us to determine component utilization.

During the second exercise 22 surveys were collected from a very similar group of users who participated in the previous exercise. The questions in this survey were focused on how users communicated with other users during an outbreak (Table 2). Again, website usage logs and records of the created events and comments were also available for this exercise.

Table 2.

Events breakdown by category.

Category	# Events
Neurological	10
Severe Illness / Death	9
Fever	8
Gastrointestinal	5
Respiratory	5
Rash	4
Localized Lesion	4
General	3
Hemorrhagic Illness	2
Lymphatic	2
Botulism-Like	2

Each jurisdiction’s summary presentation was also recorded during the wrap-up meetings for each exercise.These presentations give insights into the utility of the ECC from each jurisdiction’s perspective.

Website usage logs and records of the created events and comments are also available for the ten month period of operational use of the ESSENCE regional NCR system.

Results

Observations, and data collected from 21 participants via participant feedback and survey forms provided several significant findings and comments focused at the design and utility of the ECC. The following is a summary of findings:

• Approximately 50% (11 of 21) of the participants reviewed comments posted in the ECS before conducting their initial data review during the exercise.

• Two thirds of participants commented that they think the ECS serves a critically needed function.

• 18 of 21 participants agreed that the ECS is intuitively designed.

• 18 of 21 participants agreed that using the ECS is worthwhile.

• 11 participants were in favor of using the ECS for communicating suspected events, outbreaks, and non-events (routine daily findings), 7 participants thought the ECS should only be used for communicating suspected events or actual outbreaks, and 3 participants were unsure.

• Of the seven participants in favor of using the ECS for only communicating suspected or actual events, three agreed that a separate set of web pages would be valuable to communicate routine findings and/or daily observations.

• Of the several common definitions, labels, abbreviations, and terminology used in the ECS, one label was not understood by 1 of the 21 participants.

• Of seven potential system enhancements listed on the participant survey, developing an e-mail notification tool (allowing users to be aware of new events posted in the ECS), was voted as the number 1 priority by participants.

• Of seven potential enhancements listed on the participant survey, developing access controls, on-line chat capabilities, and including a daily discussions page were voted as the least desired features.

These results suggest users found the ECC useful and a positive addition to disease surveillance system. More than 67 percent thought it filled a critical need and most (86%) stated they would use the ECS in practice. Most also found this particular implementation easy to use. A criticism of the survey is that it had a positive bias to its questions, but even accounting for that the overwhelming majority of users stated they would use the ECC.

There was some disagreement on whether an ECC should be used to communicate information daily or just for a suspected event. Users also expressed a desire to use the event communications component without having to constantly check the web page. This could be done by automatically emailing new events or marking events that have already been viewed.

The question about potential system enhancements gave the users a chance to pick the features they most wanted to see added to future versions of the ECC. The email notifications feature was the highest ranked and describes a process by which users could have emails sent to them when new events or comments were created. The second most popular feature would allow events to be marked as read in order to show users what they had already viewed. Changing the page layout of the ECC from a threaded discussion of comments under events to an unthreaded discussion of just comments was the third most popular. Allowing users to link to pages and have the linked page become cached and its state saved for future viewings was the fourth highest ranked feature. In the current system, all pages are being updated with real-time data and so linked pages are constantly changing. This would allow users to link to what they saw at the time instead of what the situation is when a different user clicks on their link. The lowest ranked features included the online chat capability, stricter access control to allow messages to be viewed only by certain users, and the idea to daily discussions on a separate page from more pressing event discussions.

Each jurisdiction formally presented their opinions during the Friday meeting. While each jurisdiction was pleased with the ECC as a tool, there were issues faced during the exercise by each jurisdiction. These issues included data from the rest of the site could not be viewed simultaneously while reading comments in the ECC. There also needed to be an ability to move comments from local ESSENCE system to the regional ESSENCE system. Users expressed a need for clear protocols for using the ECC. There was also a majority of users who were confused by the ranks and what they meant.

During the Friday meeting most of the informal comments about the ECC were positive and two additional system changes were noted during the discussions: the need for a notepad area in the disease surveillance site, and clarification of the ranking scores.

The second exercise survey forms results are in the table below (Table 1).

Table 1.

Survey results from exercise 2.

Sent info to other public health jurisdictions
Question	Day 1	Day 2	Day 3
Found Anomalies	7 / 8	9 / 9	4 / 5
Contacted other PH agencies	7 / 7	9 / 9	5 / 5
Used Event Communications	6 / 7	9 / 9	4 / 5
Used Email	1 / 7	8 / 9	3 / 5
Used Phone	2 / 7	1 / 9	1 / 5
Used Fax	0 / 7	1 / 9	1 / 5

These surveys showed that the event communications component was heavily used during an outbreak, but the ability to discuss events online did not completely replace existing forms of communication.

During the Friday meeting two main comments were expressed by the participants. The first is that the information posted in the ECC made the users aware of what happening in other jurisdictions in the region. The second was that the ECC should be integrated into the inter- and intra-jurisdictional communications protocols.

These comments reiterate sentiments from previous surveys and comments. They continue to show that the event communications component is useful and that a better understanding of when to use it is needed.

During operational use from May 1, 2006 until March 15, 2007 54 events were created by 8 different users. There were 98 comments associated with the 54 events created by the same 8 users. These events were distributed among all syndromic categories (Table 2).

The rank associated with each event is shown in Table 3 and when they were created is shown in Figure 4.

Table 3.

Events breakdown by rank.

Rank	# Events
	Operational	Exercise 1	Exercise 2
Info	15	0	1
Low	26	0	4
Guarded	10	2	4
Elevated	3	1	1
High	0	5	7

Figure 4.

The number of events by rank and week.

The operational use of the event communications component has ranks that are skewed to the less concerning because few outbreaks or suspicious activity occurred during the period. During operational use events of Low concern level were most frequent. Figure 4 also shows the number of events created did slow down over time. This could show the users were experimenting with the tool during the first few weeks, but also may show that the ECC use changed from a daily summary tool to more event focused tool, or might show there were fewer events to discus during this time.

Table 4 shows that during the operational period the Event List was accessed nearly as often as the home page, or nearly every time a user logged on.

Table 4.

Website page views.

Web page	# Views
Event List	973
Home Page	1219
Time Series	3788
Data Details	1972
Summary Alert List	3279

Discussion

Users found the ECC prototype useful but suggested several improvements for the next iteration of the system. Using the ECC operationally and in exercises allowed us to better understand the shortfalls of the system and begin improvements for future versions. Feature changes, like converting ranks to action-oriented words such as “No Concern”, “Monitoring”, “Investigating”, and “Responding”, enhanced user communications. The event list is also currently being converted from an event focused system to a comment focused system. The exercises showed us that users were more likely to create new events than they were to find a similar event and post comments under it. This has led us to believe a threaded system of events and comments should be replaced with a flat system of comments that could be rolled-up into an event.

The ECC also demonstrates the need for communications between disease surveillance systems in contiguous jurisdictions. It is clear from this experience that standardized meta-information is a necessary component of any information sharing system. By having standardized meta-information the ECC could be integrated with the summary alert list and other parts of the ESSENCE site. The combination of standardized meta-information and standardized user information (i.e. concern levels) allowed the users a feedback loop into the system to balance the mathematical algorithms with epidemiologic judgment. Eventually this labeled dataset of counts and user concern levels may be incorporated into mathematical algorithms to improve the specificity of system alerts.

While it would be difficult for another system to pickup and use the ECC, the lessons learned can certainly be applied to similar features being developed for other disease surveillance systems.

Conclusion

Users found the ECC to be a useful feature of the regional ESSENCE system in the NCR. The ability to share concerns and interpretations of data with peers both within and outside their own jurisdiction facilitated communication in the region. The ECC makes review of ESSENCE quicker for users by allowing them to use epidemiologic conclusions made by other users to inform their system review. It also provides a forum for users to share concerns about potential health events in their population early in the course of an ‘investigation’.