Interfacing a Biosurveillance Portal and an International Network of Institutional Analysts to Detect Biological Threats

Abstract

The Early Alerting and Reporting (EAR) project, launched in 2008, is aimed at improving global early alerting and risk assessment and evaluating the feasibility and opportunity of integrating the analysis of biological, chemical, radionuclear (CBRN), and pandemic influenza threats. At a time when no international collaborations existed in the field of event-based surveillance, EAR's innovative approach involved both epidemic intelligence experts and internet-based biosurveillance system providers in the framework of an international collaboration called the Global Health Security Initiative, which involved the ministries of health of the G7 countries and Mexico, the World Health Organization, and the European Commission. The EAR project pooled data from 7 major internet-based biosurveillance systems onto a common portal that was progressively optimized for biological threat detection under the guidance of epidemic intelligence experts from public health institutions in Canada, the European Centre for Disease Prevention and Control, France, Germany, Italy, Japan, the United Kingdom, and the United States. The group became the first end users of the EAR portal, constituting a network of analysts working with a common standard operating procedure and risk assessment tools on a rotation basis to constantly screen and assess public information on the web for events that could suggest an intentional release of biological agents. Following the first 2-year pilot phase, the EAR project was tested in its capacity to monitor biological threats, proving that its working model was feasible and demonstrating the high commitment of the countries and international institutions involved. During the testing period, analysts using the EAR platform did not miss intentional events of a biological nature and did not issue false alarms. Through the findings of this initial assessment, this article provides insights into how the field of epidemic intelligence can advance through an international network and, more specifically, how it was further developed in the EAR project.

The Early Alerting and Reporting (EAR) project is aimed at improving global early alerting and risk assessment and evaluating the feasibility and opportunity of integrating the analysis of CBRN and pandemic influenza threats. EAR's innovative approach involved both epidemic intelligence experts and internet-based biosurveillance system providers in the framework of an international collaboration called the Global Health Security Initiative. The EAR project was tested in its capacity to monitor biological threats, proving that its working model was feasible and demonstrating the high commitment of the countries and international institutions involved. During the testing period, analysts using the EAR platform did not miss intentional events of a biological nature and did not issue false alarms. This article provides insights into how the field of epidemic intelligence can advance through an international network.

The link between health concerns and human security has been traced back to the 1994 UNDP Human Development Report,¹ but the first major effort to define the concept of “health security” came in 2001 with the World Health Assembly Resolution 54.14, “Global Health Security: epidemic alert and response.”² At that time, the way “health security” was considered in the scientific community underwent fundamental changes. It went from being associated mostly with domestic social welfare to becoming more global in nature and encompassing activities aimed at containing public health threats, including all things likely to cause health-related damage or danger. While the word “risk” became more generally associated with naturally occurring events, the word “threat” started to be used specifically in reference to intentional acts.³ Although this process helped to expand interest in events of international importance, the concept and framework for health security remained incomplete, with coexisting incompatible and at times ambivalent definitions.^2,4

This is the context in which, in November 2001, health ministers of the G7 countries plus Mexico, the EU Health Commissioner, and the Director-General of the World Health Organization (WHO) set up the Global Health Security Initiative (GHSI), a new international partnership to collectively reduce the potential health impact of biological, chemical, and radionuclear (CBRN) terrorism and pandemic influenza.⁵ This framework guided the development of several activities that had in common the formal creation of informal collaborations among like-minded countries to fill a gap in addressing health issues related to global health security. Among the issues that GHSI identified for concerted global action was an agreed process for international collaboration on risk assessment of potential health threats. This was the focus of the Early Alerting and Reporting (EAR) project, which began in 2008 with the aim of improving the capacity for detection and risk assessment of events that could be related to the intentional release of CBRN agents based on information from unstructured public sources.

The notion that rumors concerning outbreaks can be extremely useful for early detection and that methodologies for verification of this information for the purposes of public health are needed date back to the 1990s.⁶ This approach further evolved as technological advances affected the ability to detect and assess public health threats⁷ through a growing number of internet-based applications and rapid communication tools. The concept of epidemic intelligence (EI)⁸ was defined in the early 2000s to include all activities related to the early identification of potential health hazards and their verification, assessment, and investigation in order to provide information to guide appropriate actions in public health.^2,9

In addition to relying on indicator-based surveillance, which generally analyzes data routinely collected from healthcare facilities through institutional disease reporting, EI integrated the monitoring of rumors by formalizing the methodology of event-based surveillance (EBS).^10-12 Event-based surveillance captures information from immediately available unstructured data, from sources that go beyond the health sector.¹³ The limitation of this information is that it is often not official, not verified, abundant, and available in various formats and levels of detail. Once signals of potential interest are detected through appropriate filtering mechanisms, they need to be assessed and validated to be used as a basis for decision making in the management of health threats. By integrating an indicator-based and an event-based component in a single surveillance system, epidemic intelligence expanded the approach of threat detection through the use of information technology (IT).

With the birth of event-based surveillance came the development of internet-based biosurveillance systems (IBBSs)^12,14-20 aimed at detecting health threats by using information available on the internet, and the need for professional analysts able to monitor, detect, and assess potentially relevant information captured from news items, blogs, or other sources of information.²¹

Between 2008 and 2010, the EAR project developed and implemented a new synergistic combined epidemic intelligence technical tool (“EAR portal”), which pooled information from 7 IBBS providers. The performance of the EAR portal highlighted in a previous evaluation a higher sensitivity for relevant information when compared to each individual internet-based biosurveillance system.²² The EAR project also built a team of international experts in biological threats from public health institutions of participating countries and international organizations (hereafter bioanalysts) that helped to develop this innovative tool. They provided user feedback and created a collaborative working model to jointly use the EAR portal for biological threat detection. The feasibility and usefulness of this working model underwent a proof of concept trial run shortly following the EAR portal development phase. The aim of this exercise was to explore the operationalization of the use of the EAR portal by analysts, through an assessment of the portal-analyst performance. Based on this proof of concept trial run, the feasibility and usefulness of the collaborative working model proposed by the EAR project was assessed. The gaps and lessons learned that were detected were the basis for the future development of the subsequent phases of the EAR project.

Materials and Methods

The EAR Portal

Following a survey on the biological threat analysts' needs,²² the EAR portal, a web-based, password secured, overarching portal that pooled data from 7 internet-based biosurveillance systems, was designed by a technical portal development team composed of experts from the Joint Research Centre (JRC) of the European Commission. The IBBSs that participated in the pilot phase of the EAR project included research-based systems (BioCaster, Japan;¹⁵ HealthMap, US;¹⁷ ProMed, US;^14,23 and PULS, Finland¹²) and systems with a national or transnational mandate (Argus, US; Global Public Health Intelligence Network, Canada;¹⁶ and MedISys, European Commission^18-20).

With the exception of ProMED,²³ which is a contributor-based system, and taking into account inherent differences, all of these internet-based biosurveillance systems were designed to scan multiple and nonpredefined sources on the internet to detect information on potential health threats.²² These diverse sources included both rumors from the media and more reliable information published by official agencies and organizations. Unstructured information was filtered and deduplicated automatically on the basis of the technology developed for the MedISys IBBS.^18-20,24 In the processing chain, more than 1,000 multilingual categories for public health threats were available. Two algorithms were used for categorization: one based on a simple keyword list, which was used for unambiguous categories (such as “chikungunya”), and one based on Boolean combinations of keywords, which allowed more complex categories and elimination of ambiguity (eg, “anthrax” AND “outbreak” BUT NOT “rock concert”). All incoming items were processed in their original language, and therefore the keywords also were translated in different languages.

Sixteen CBRN threats were defined based on the opinion of subject matter experts in biological, chemical, and radionuclear threats involved in the EAR project during the development phase of the portal, and they were labeled as follows: anthrax, avian influenza, botulism, cesium, chlorine, cyanide, Ebola virus, iridium, organophosphate, Q fever, ricin, Rift Valley fever, smallpox, tularemia, unknown disease, and plague.

The portal compared all items within a 4-hour window with each other, using the metadata (categories and filters) to prefilter. A cosine similarity measure was used to compare incoming items. Items with a 99% similarity of title and description were automatically tagged as duplicates. EAR portal analysts had the option to display only the first original news items, or the first items plus the duplicate news items subsequently published.

All news items filtered in the platform could be viewed by analysts both in the original language and in English, using statistical machine translation. Analysts could also access the original source of the news item and select and copy the text to apply their chosen software and language for statistical machine translation. External links to Google Translate were also provided.

The EAR platform stored all the data generated in an internal database that allowed for descriptive statistical analysis (metrics). Metrics provided in the EAR portal included the number of incoming items, the number of items categorized as CBRN (before and after automatic deduplication), and the number of items assessed as relevant by EAR analysts.

The Bioanalyst Network

The consolidation of the analyst network for the biological component (bioanalysts) was achieved during the first phase of the EAR project; the analyst networks for the chemical and radiological/nuclear components of CBRN were rudimentary and further refined later. Bioanalysts were subject matter experts involved in biological threat health risk analysis in the following public health government institutions: Public Health Agency of Canada (Canada), European Centre for Disease and Prevention Control (ECDC), Institut de Veille Sanitaire (France), Robert Koch Institute (Germany), Istituto Superiore di Sanità (Italy), National Institute of Infectious Diseases (Japan), Health Protection Agency (United Kingdom), and the US Centers for Disease Control and Prevention (United States). The EAR project did not receive dedicated funding for its activities but benefited from voluntary and in-kind contributions from participating countries and institutions. The bioanalysts contributed their work as part of this in-kind contribution to the EAR project activities.

The EAR Portal Bioanalyst Working Model

In order to enable bioanalysts involved in the EAR project to work on a rotation in a coordinated way, a standard operating procedure for biological threat event-based surveillance was designed. This document described monitoring, manual deduplication, and risk assessment functions that bioanalysts were expected to carry out during their duty periods. All those functions were designed to integrate, in a feasible and sustainable way, the work required under the EAR mandate with the risk analysis activities that the bioanalysts were already conducting in their national and international working context and that, for the most part, included both CBRN and public health scopes. The standard operating procedures and the outputs of the work carried out by the bioanalysts were discussed and developed during EAR project meetings in the presence of representatives of the WHO HQ, the European Commission (EC), and of the contributing internet-based biosurveillance systems. The bioanalysts interacted with the EAR portal development team to set up functions in the portal to better apply the risk assessment matrix and facilitate the production of standardized outputs.

The EAR project bioanalysts defined the use of the terms “item” and “event” for the purposes of the EAR project as follows: An item is data in the form of discrete articles or snippets of information filtered in the EAR portal following automatic filtering and deduplication and displayed on the EAR portal for examination and review by human analysts. An event is an item of potential interest highlighted as such by the bioanalyst on duty through the creation of a dedicated event page in the EAR portal. The main activities conducted by bioanalysts are described below.

Monitoring

On-duty bioanalysts read each item filtered in the threat categories of the EAR portal and assessed content for nonexact duplication and relevance. Nonrelevant items or those that were duplicates were tagged for removal from the user's view.

Risk Assessment

In order to assess potentially relevant items in a consistent way, the bioanalysts designed a set of 6 standardized unweighted questions for risk assessment of biological threats that could be independently scored between 1 and 3, with a total score that could range for each item between 6 and 18. This tool was called the EAR risk assessment matrix. Aspects assessed included the etiology, health impact, clinical presentation, and epidemiology of the incident or outbreak described in an item as well as the source of information and the likelihood that it could be caused by an intentional release (Table 1).

Table 1.

The Pilot EAR Project Risk Assessment Matrix (active between July and September 2010)

Question	Score	Comments
Public Health Impact
What is the impact of the event reported (in terms of cases and severity)?	1-3	1–Low impact 2–Medium impact 3–High impact
Suspicion of Deliberate Act
Is the disease endemic in the country in which it has occurred?	1-3	1–Endemic 2–Sporadic or disease not known 3–Not known to occur/new
Does the clinical presentation (including outcome and response to treatment) suggest an unnatural event?	1-3	1–Clinical presentation as expected/no information 2–Clinical presentation not typical but plausible 3–Clinical presentation very unusual
Does the epidemiology (including geo-distribution and origin of event) of the case/outbreak suggest an unnatural event?	1-3	1–Epidemiology as expected 2–Epidemiology unusual but within accepted norms 3–Epidemiology very unusual
Does the etiology or suggested etiology of the infection suggest an unnatural event?	1-3	1–Etiology entirely consistent with natural infection 2–Etiology unusual but possible or etiology not mentioned/known 3–Etiology very unusual
Source of Information
Is the report from a trusted source?	1-3	1–Report from a potentially unreliable source 2–Report from a potentially reliable source (eg, NGO, news sources of good reputation, or reporting national health authorities) 3–Report from trusted source (eg, WHO, national health authorities)

This scoring tool was tested by analyzing the original news or internet-based biosurveillance system items of a set of 24 historical events. Bioanalysts independently assessed each item for risk and decided in each case if the information warranted the issue of an alert under the EAR project mandate and, if so, if it should be given high or low priority. At the same time, the analysts independently scored each report using the 6 standardized questions. A frequency analysis of scores was made by priority of reporting. Higher scores were associated with high-priority reporting and low scores with no reporting (Figure 1).

Figure 1.

Test of the EAR Risk Assessment Matrix by EAR Bioanalysts. Number of reports assessed, by score and suggested action, and empirical thresholds

Based on the findings, scoring thresholds were empirically defined as follows: If items scored over 14, the issue of an alert was suggested; if items scored below 10, the suggestion was not to issue alerts; if an item scored between 10 and 14, it was considered low priority, and an alert was not strongly suggested. The EAR risk assessment matrix was included in the user interface of the EAR portal.

Reporting

The EAR project working model had 2 types of reporting outputs: alerts and hand-over emails. The creation of an “alert” was a process through which the on-duty bioanalyst, having compiled the EAR risk assessment matrix on the EAR portal for an item considered relevant, decided to trigger the automatic generation of an email alert to senior officials of the GHSI Risk Management and Communication Working Group. This choice remained with the on-duty bioanalyst, who would base his or her decision partially, but not exclusively, on the outputs of the scoring. Alerts included all situations that suggested intentional release of biological agents.

At the end of their duty rotation, bioanalysts were also required to compile a hand-over email addressed to the bioanalyst next on duty and copy the other bioanalysts. This email summarized the work done during the week, including the issuance of alerts. It also highlighted any events that might need to be monitored in the coming week.

Proof of Concept Trial Run

The EAR project working model was tested between July and September 2010. The exercise involved the bioanalyst network and the EAR portal development team. During this time, bioanalysts from various public health institutions rotated each week and applied the described standard operating procedures.

Monitoring and risk assessment focused only on biological threats (11 threat categories, consisting of the 10 mentioned biological threats and the “unknown disease”). Metrics of the items filtered into the EAR portal by date, trigger words, and language were automatically recorded.

Given the rarity of intentional releases of biological agents, and in order to verify the collaborative risk assessment capacity, bioanalysts were asked to assess and consider relevant items of public health importance in addition to those under the EAR mandate. At the end of each rotation, the on-duty analyst completed an end-of-duty questionnaire to assess the workload of the activities performed on the EAR portal, the level of consultations conducted to perform the risk assessments needed, and the usefulness of the information assessed in his or her working environment. In addition, bioanalysts were required to provide feedback on the functioning of the EAR portal, which was relayed to the portal development team.

The questionnaire was designed to capture input, process, output, outcome, and impact indicators (Table 2) for the main functions carried out by EAR bioanalysts: the monitoring function, the risk assessment function, and the reporting function. For each function, indicators were defined by type and associated to proxy measures when adequate. Event generation was considered the main output of the EAR risk analysis function, while the main report output considered was the hand-over email.

Table 2.

Functions and Indicators Assessed During the Proof of Concept Trial Run of the EAR Portal Bioanalyst Working Model

Function	Indicator Type	Definition	Proxy	Results
Monitoring	Input	Average number of incoming news items filtered into the EAR platform/week	Average workload expected for biological threat monitoring	716 items/week
Monitoring	Input	Number of hours/week dedicated by on-duty analysts to EAR platform monitoring	Average weekly working hours required to apply the EAR working model in the testing period	10 hrs or more (64% of rotations); 3-10 hrs (36%)
Monitoring	Process	Continuity of rotations maintained throughout the testing period (% of expected rotations covered)	Possibility of managing EAR-related duties alongside bioanalyst's other activities	92%
Risk assessment	Process	% of rotations involving other institutional/EAR bioexperts in risk assessments	Capacity of the EAR work model to involve the bioanalyst EAR network and other subject matter experts within participating institutions in the risk assessment	55%
Risk assessment	Output	Number of events generated/testing period		11
Risk assessment	Outcome	Number of false alarms issued during the testing period	Provides some indication of the reliability of the outputs of the EAR portal bioanalyst work model	None
Risk assessment	Outcome	Number of intentional releases of biological agents missed by the system during the testing period	Provides some indication of the sensitivity of the EAR portal bioanalyst work model	None
Risk assessment	Outcome	Number of correctly identified false news items (hoaxes) on intentional releases of biological agents	Provides some indication of the specificity of the EAR portal bioanalyst work model	3
Risk assessment	Impact	% of rotations for which the on-duty analysts used information assessed in the EAR platform also to make decisions or take further action in their country/institution	Added value of the EAR working model for participating bioanalysts at institutional (national/international) level	45%
Reporting	Output	% of rotating bioanalysts who produced a hand-over email during the testing period	Possibility of managing EAR-related duties alongside bioanalyst's other activities	100%
Reporting	Output	Number of alerts generated during the testing period		3

Results

From July 5 to September 27, 2010, 7 of 8 EAR bioanalysts, each from a different public health institution, succeeded in monitoring, assessing, and analyzing the items filtered into the 11 selected threat categories of the EAR portal on an individual weekly rotation basis for 11 weeks. The 7 rotating analysts completed 1 duty week during the testing period; 4 of these analysts were on duty for 2 nonconsecutive weeks. (One duty week rotation was not covered because of a misunderstanding related to the rotation assignments.)

A total of 5,845 items were automatically filtered into the EAR portal, with a mean of 716 items per week (ranging from 447 to 991). Items filtered were in 32 different languages. In the testing time window, the most frequent languages were English (3,371 items, 57.7%), Romanian (1,260 items, 21.6%), Russian (492 items, 8.4%), French (105 items, 1.8%), and Spanish (101 items, 1.7%). The most frequent items were filtered into the portal by the trigger words used for plague and anthrax (Figure 2). From week 31 to week 34, the volume of information on plague increased and remained stable at about 200 items filtered per week, as outbreaks were taking place in Peru and Paraguay. In the following weeks, the volume of these items slowly decreased even though the outbreaks progressed over time. In week 38, an evolving plague outbreak in China contributed to the total item volume observed.

Figure 2.

Events Monitored and Alerts Issued During the Pilot Test of the EAR Portal, July-September 2010

Anthrax items initially peaked during the fifth week of the 3-month period (week 31), as news about outbreaks in Siberia and Canada was being detected. After an initial decrease, anthrax items peaked again between weeks 34 and 36, as epidemics were being reported in Bulgaria and Bangladesh. The volume rose again in week 38 because of information updates on Russia and Bangladesh and to media resonance triggered by a new anthrax death in the UK related to the injection of contaminated heroin.

As shown in Figure 2, all other biological threats monitored were comparatively negligible in item volume except for Ebola, which peaked in week 34 when US authorities approved the beginning of human trials for a drug proven to be effective against Ebola virus in monkeys.

Ten hours or more were spent by bioanalysts to conduct risk monitoring and assessment in most duty weeks (64%) and, in particular, by 6 of 7 first-time users (86%) compared to 1 of 4 second-time users (25%).

In about half of the duty weeks (55%), bioanalysts consulted other subject matter experts to evaluate items filtered in the EAR portal. When consultation occurred, it always involved colleagues in the same country (100%) and, in half of cases, also other members of the EAR bioanalyst network. Analysts reported in 55% of duty week rotations that they had no previous knowledge of the detected relevant items through other sources. On-duty analysts indicated that they used the items from the portal to make decisions or take further action in their institutions in almost half (45%) of all the total duty-week rotations. Bioanalysts were also asked if working on the EAR portal had an added value, given their institutional professional role. On-duty analysts answered that this had been the case during 5 of the 11 duty weeks (45%) and that it had not been the case during 1 duty week (9%). Bioanalysts of 4 duty weeks (36%) relayed the impression that the system had the potential to provide an added value but needed further development. One on-duty analyst did not answer.

No intentional biological CBRN events took place during the pilot period, and no false alarms were issued by the bioanalysts. Apart from 2 system tests, to create an event and an alert, performed during the pilot period by analysts and announced as tests to other participants, only 3 alerts were published during the testing period. All were issued in order to discredit false information circulating on possible intentional releases of biological agents (Figure 2) that could generate undue alarm (plague among Syrian troops, suspicious mail in US embassy in France, white powder detection in Chile).

Overall only 2 analysts did not create an event during their test week. Five analysts created 1 event, 1 analyst created 2, 2 analysts 3, and 1 created 7. Events created without issuing alerts allowed analysts to select and gather information in a dedicated section of the EAR portal as the situation unfolded and to exchange comments and insights. This facilitated the monitoring of events of public health relevance and supported international collaboration among concerned institutions participating in the EAR network.

As an example of this model of collaboration, we describe an event created by an EAR analyst on August 23, 2010, in response to items reporting 80 human cases of anthrax occurring in Bangladesh. The country is considered endemic for anthrax, but the number of cases was considered unusual. Although flooding had been occurring in the country for several weeks, this was not thought to explain totally the spreading outbreak as it did not differ from similar events in previous years. The initial assessment was low in terms of scoring under the EAR mandate (8-9), and no alert was issued. In the following weeks, the situation worsened with cases increasing and reported in various parts of the country. On-duty EAR analysts monitored the situation from one week to the next through a dedicated event page and weekly comprehensive hand-over emails. Although a deliberate release was not suspected, the outbreak had become highly relevant from a public health standpoint, with hundreds of cases reported in the country at the beginning of September. As international attention was growing, with travel advisories being posted by several countries, international media started strongly to suggest the possibility of an intentional act. During the following days, ECDC and CDC members of the EAR network shared information and assessment findings. A deliberate act was excluded by the collaborating bioanalysts based on the evidence available, and a new risk assessment was posted on the EAR portal to inform all EAR members of the final evaluation of the event.

Discussion

The EAR project operates with a mandate for CBRN in the framework of GHSI, drawing its human resources mainly from the public health domain. While this required the bioanalyst team to adjust to this double nature in the development phase, with some bioanalysts being more oriented to CBRN and others to public health, this situation was fruitful and productive. It generated a team that could integrate both focuses by respecting the mandate of the EAR project and its outputs and also benefiting from the project activities in their work in national and international public health institutions.

The main pattern that emerged in the EAR portal use during the pilot test run was that in the first weeks of the trial, bioanalysts tended to create events only in order to issue alerts. As the weeks proceeded, analysts started creating events without issuing alerts in order to maintain a monitoring function of episodes of public health relevance that were taking place during their duty week but that were not relevant enough to the EAR mandate to issue an alert, as in the case of naturally occurring outbreaks.

This was a shift from “alerting” to “alerting and monitoring.” The initial EAR approach had been to analyze information to issue alerts valid at a point in time without following the evolution of events. This also reflected the findings of the mentioned user requirement survey²² that had prioritized “timeliness” over “completeness” and therefore a function more oriented to “warning” than “monitoring.”

The creation of events, with their dedicated webpage, scoring, and risk assessments, were de facto a monitoring tool, and most events created were for items of public health relevance. This is likely because all the items detected by the bioanalysts that indicated a potential intentional release were rapidly recognized as hoaxes and reported as such. With this spontaneous and progressive shift from alerting to monitoring, analysts adapted the EAR portal to the evolving nature of early reports, whereby intentionality can either be initially suspected and later dismissed, or it might be suspected in a later stage, as in the case of the anthrax outbreak event described.

As a result, the creation of alerts (the main external output of the EAR project) showed consistency and credibility, since no false alerts were issued and no incidents missed. Greater variability was observed in the use of the portal as bioanalysts kept an eye also on public health events on the basis of individual needs and preferences.

The EAR project was developed in the context of the GHSI and of the international legal and operational frameworks at the UN and EU levels that bound participating countries, such as the International Health Regulations (IHR),²⁵ which entered into force in 2007, and the EU Early Warning and Response System.²⁶ The project was particularly in line with the IHR, which had formalized and stated the link between health and security, defined formal communication channels, and, in its article 9, had described paths of international collaboration for the detection of public health emergencies of international concern through alternative sources of information. EAR operated at a technical scientific level in direct collaboration with the WHO and the EC, which guided and directly contributed to the design of its methodology and tools, and reported its outputs to senior officials who were part of the GHSI Risk Management and Communication Working Group (RMCWG). The RMCWG is made up of official representatives of the ministries of health from all participating countries.

Although not financed by a funding institution, the EAR project stimulated enough interest in the WHO, the EC, and in the participating international institutions, ministries, national public health institutions, and internet-based biosurveillance systems for them to provide inputs and manpower to carry out the project activities on a voluntary basis. This interest speaks to the need to collaborate in an environment that is institutionalized enough to enable the exchange of confidential information, but also informal enough to allow the exchange of early unverified reports that might be discredited as further information is collected.

The EAR project's working model supported the core mission of the involved institutions of detecting and characterizing events as early as possible so that appropriate and rapid action can be taken. This fueled motivation to participate in the activities even without dedicated staff. For this reason, excessive workloads were considered a critical issue in the functioning of the EAR project and were specifically assessed in the test run. Results were promising. First, shorter time requirements were recorded among second time users of the EAR portal. This was probably related to week-to-week variations in the number of items filtered, to the bioanalysts' familiarity with the portal, which led to increased efficiency, and to the weekly technical improvements that were made to the portal itself on the basis of the feedback provided through questionnaires. Second, most bioanalysts reported that activities conducted for the EAR project had an added value for their institutional work.

The EAR project had also an unexpected outcome, which was to raise awareness of the advantages of unstructured information collection and assessment and consequently stimulate further development of event-based surveillance in those participating countries where it was less developed. The exchange of information and tools among participants was therefore also conducive to building capacity and helping to shape evolving epidemic intelligence systems.²⁷ These accomplishments can be attributed not only to the contributions of knowledgeable and experienced individual group members, but also to the collective experience garnered in establishing and implementing the EAR portal and common analytic processes.

Prior to the GHSI EAR project, event-based surveillance for CBRN was more fragmented because of the lack of a common portal capturing and filtering open-source unstructured information. Each institution, organization, and agency involved in public health and CBRN epidemic intelligence activities normally needed to access multiple sources of web-based medical intelligence, each source having its own strengths and weaknesses, to gain as full a picture of the current situation as possible. Before the efforts of the EAR network, this was also aggravated by a general dearth of information on the ways the existing portals were functioning in terms of their simplicity, usefulness, timeliness, sensitivity, completeness, representativeness, and flexibility.²²

Risk assessments for CBRN threats were being conducted in each country, but methods for capturing, validating, and assessing information were diverse and more often based on the individual analyst's experience and capacity, lacking a standardized approach. This was, and still is, aggravated by the fact that stakeholders for risk assessments of CBRN threats are diverse in each country's government, including the ministries of internal affairs, defense, foreign affairs and health, institutes of public health, and civil protection agencies. A consequence of this is that although risk analysts with slightly different focuses are usually present both in the same country and in different countries, they may not have regular opportunities to interact and exchange information.

EAR proved that not only was it feasible and useful to pool data from existing internet-based biosurveillance systems into a unique CBRN-oriented portal, but it is also feasible to establish a collaborative and synergistic working model in the field of event-based surveillance that focused on CBRN without losing a wider public health perspective. Bioanalysts coming from a common basis of expertise but working in different settings achieved a level of professional trust that enabled them to accept interchangeability in functions and to reach consensus on the working model to adopt and the approach to use.

Limits and Lessons Learned

The aim of this exercise was to demonstrate conceptually whether the different actors involved in the EAR project could work together, accepting and contributing to the risk assessment and monitoring relevant items. It was not designed to assess the level of expertise of the EAR bioanalysts nor their level of harmonization in applying the EAR working model. There were 2 reasons for this. First, because the EAR project was a pilot, bioanalysts selected had a specific expertise for biological threats both from a CBRN and public health perspective, and the team included some of the principal subject matter experts in event-based surveillance. Their expertise was considered the basis for a correct development of the EAR tools and processes for biological threats and was therefore not being evaluated. Second, as the working model was being proved, this exercise aimed also to gather insights on its feasibility and ways of improving it by building on the different approaches that the bioanalysts adopted. While harmonization could be the objective of the current EAR project, at the time of its development it was considered premature.

Another aspect to take into account is that in this exercise the outcomes and impact were measured uniquely from a bioanalyst perspective. It would be interesting, in a future evaluation of the EAR project, to gather the opinions also of the senior officials of the RMCWG and, when officers are different, of the EU Early Warning and Response System and IHR national focal points of concerned countries.

The EAR project moved beyond development into a consolidation phase. Since 2010, EAR has continued working on refining the portal and has established a continuous and sustainable international rotation of analysts. The development phase guided subsequent evolutions of the EAR working model. The lack of regular communications to senior officials of the RMCWG during duty weeks that did not see the generation of alerts was a limitation, as it did not provide any visibility for the project activities beyond the alerts themselves. In the following EAR phase, this was changed with the creation of a standardized weekly report to senior officials. In addition, the monitoring function introduced by the creation of “events” was further refined following inputs from this exercise, which highlighted its usefulness in generating continuity and collaboration among the different rotating bioanalysts. In addition to “events” that can be scored and can lead to generation of alerts, “incidents under review” (IUR) were introduced with new functionalities, such as item tracking and a forum in which analysts can share information and opinions in a password-protected secure environment. Incidents under review can be upgraded to events if necessary.

Conclusions

In the context of the formal GHSI framework, the EAR project represented a unique collaborative experience and is a proof of concept that large-scale intercountry partnerships are feasible with promising results in the field of biological health threat detection and assessment. EAR enabled the establishment of an informal technical international collaboration, and it improved the continuity and enhanced the knowledge of existing internet-based biosurveillance systems. It also fostered a more systematic and rational approach to biological CBRN threat detection and assessment through a public health lens and enabled the establishment of a trusted network of stakeholders in biological health threat surveillance (Table 3).

Table 3.

Gaps identified and addressed by the GHSI EAR project

Gap Identified	GHSI EAR Response	Result	Outcome
Fragmented and personalized risk assessment with great variability between countries.	To define analyst needs and expectations through a survey;22 to formalize risk assessment tools and develop a common portal for item capture, filter, validation, and assessment.	A common portal for risk analysts to use for the detection and assessment of threats was set up and SOPs for risk assessment for biological threats produced.	Development of a more systematic and rational approach to biological threat detection and assessment.
Lack of a communication framework between risk analysts and general reluctance in information sharing.	To create a network of epidemic intelligence expert analysts and system providers that acts as a framework for information sharing. To establish a rotation system of risk assessment involving 7 countries and an international organization.	An information-sharing framework that has gradually built trust and confidence in each other's abilities, reaching the point of becoming a rotation system for risk assessment across 3 continents.	Establishment of a common, trusted network of stakeholders in biological threat surveillance.

Sharing information provided most participants with otherwise unavailable data in real time and an environment to share risk analysis expertise and disseminate verified epidemic intelligence to all participants. However, the fact that all participating countries are characterized by strong health systems, and therefore by a potential response capacity that could be clearly empowered by the early detection of threats, should not be underestimated.

Considering the threat protection aspect of health security, a sustainable response capacity will need to be promoted in order for the climate of trust in the current interpretation of health security to be maintained and expanded in more countries. As eloquently stated by William Aldis in 2008,² health systems need to be strengthened to ensure that surveillance data is of use and that the added value of this specific approach to health security is perceived as such by a wider number of stakeholders.

Acknowledgments

The authors thank the EAR working group for their participation and the fruitful discussion raised by this manuscript. We gratefully acknowledge GHSI, which supports the project and allows countries and institutions to provide human and financial support. EAR project core members: John Brownstein (Children's Hospital, Boston, US), Nigel Collier (National Institute of Informatics, Japan), Ricardo Cortes (Ministry of Health of Mexico), Pamela S. Diaz (US Centers for Disease Control and Prevention), Christian Herzog (Robert Koch Institute, Germany), Mike Hiley (Health Protection Agency, UK), Nigel Lightfoot (Health Protection Agency, UK), Lawrence C. Madoff (ProMED-mail), Jas Mantero (European Centre for Disease Prevention and Control), Noele P. Nelson (Georgetown University, Washington, DC), Agnes Rortais (European Food Safety Authority), Johannes Schnitzler (World Health Organization), Germain Thinus (European Commission, DG-SANCO), Roman Yangarber (University of Helsinki, Finland), Nicholas Palanque (Public Health Agency Canada), Heinrich Maidhof (Robert Koch Institute, Germany), Mark Salter (Public Health England), and Josep Jansa (European Centre for Disease Prevention and Control). The EAR project was not financed by a funding institution, and the authors hereby declare no conflicts of interest whether financial, personal, political, or academic.