Event-based surveillance at health facility and community level in low-income and middle-income countries: a systematic review

Abstract

Background

The International Health Regulations require member states to establish “capacity to detect, assess, notify and report events”. Event-based surveillance (EBS) can contribute to rapid detection of acute public health events. This is particularly relevant in low-income and middle-income countries (LMICs) which may have poor public health infrastructure. To identify best practices, we reviewed the literature on the implementation of EBS in LMICs to describe EBS structures and to evaluate EBS systems.

Methods

We conducted a systematic literature search of six databases to identify articles that evaluated EBS in LMICs and additionally searched for grey literature. We used a framework approach to facilitate qualitative data synthesis and exploration of patterns across and within articles.

Results

We identified 778 records, of which we included 15 studies concerning 13 different EBS systems. The 13 EBS systems were set up as community-based surveillance, health facility-based surveillance or open surveillance (ie, notification by non-defined individuals and institutions). Four systems were set up in outbreak settings and nine outside outbreaks. All EBS systems were integrated into existing routine surveillance systems and pre-existing response structures to some extent. EBS was described as useful in detecting a large scope of events, reaching remote areas and guiding outbreak response.

Conclusion

Health facility and community-based EBS provide valuable information that can strengthen the early warning function of national surveillance systems. Integration into existing early warning and response systems was described as key to generate data for action and to facilitate rapid verification and response. Priority in its implementation should be given to settings that would particularly benefit from EBS strengths. This includes areas most prone to outbreaks and where traditional ‘routine’ surveillance is suboptimal.

Key questions.

What is already known?

• Event-based surveillance (EBS) aims to contribute to rapid detection of acute public health events, and supplements indicator-based surveillance (IBS), the backbone of national surveillance systems.

• To date, no assessment of evidence for best practices exists.

What are the new findings?

• The scope of EBS is broader than IBS in several aspects: (1) In outbreaks, EBS was able to identify outbreak-disease cases and programming gaps helping to guide the response; (2) EBS was able to rapidly identify outbreaks and other health hazards that were not detected by IBS, or before they were detected by IBS; (3) EBS was able to reach remote areas and was most used where health facilities were scarce.

• Good communication with communities and other stakeholders were described as key; in addition, linkage to IBS surveillance and rapid response was described as essential for all systems as every verified event needs an assessment and potentially a response.

What do the new findings imply?

• EBS is a valuable source of data that can strengthen the early warning function of national surveillance systems, if integrated into existing surveillance and linked to response structures.

• Priority in the implementation of EBS should be given to settings that would particularly benefit from EBS’ strengths such as areas most prone to outbreaks, where access to the formal health system is limited, where traditional ‘routine’ surveillance is suboptimal and where timeliness of notification of epidemic-prone diseases is of particular importance to mounting a rapid response.

Background

The International Health Regulations (IHR) require member states to have “the capacity to detect, assess, notify and report events”.¹ An event is defined as “a manifestation of disease or an occurrence that creates a potential for disease”.¹ Event-based surveillance (EBS) is the organised collection, monitoring, assessment and interpretation of mainly unstructured, ad hoc information regarding health events or risks, which may represent an acute risk to human health.² EBS should be monitored and responded to in real-time. EBS, alongside indicator-based surveillance (IBS), is a component of the early warning, alert and response (EWAR) function of a national surveillance system.² The objective of EWAR is to support the early detection of, and rapid response to, acute public health events of any origin.² Thus, EWAR is a key function of surveillance, particularly in low-income and middle-income country (LMIC) settings and for humanitarian emergencies where the epidemic risk is high and other elements of the surveillance architecture may be limited or absent.² EBS may provide the earliest detection of any acute public health event, particularly those that are not well detected by IBS, or not detected rapidly enough.

Guidance documents discuss the implementation of EBS within national surveillance systems, prior to and during humanitarian emergencies.^2–6 However, to date no assessment of evidence for best practices exists. We evaluated the peer-reviewed and grey literature relating to the implementation of EBS for outbreak detection in LMICs in both emergency and non-emergency settings in order to provide a systematic description of EBS and to evaluate the strengths and weaknesses of EBS across both contexts.

Methods

Key outcomes

The key outcomes included summaries of the (1) quality of available evaluations, (2) EBS system structure and event definitions in use, and (3) evaluations of surveillance attributes of the EBS systems.

Search strategy

Following PRISMA guidelines, we conducted a systematic review to identify peer-reviewed literature that presented data collected by EBS systems in LMICs and/or evaluated EBS systems in LMICs. We searched six databases: MEDLINE (Ovid MEDLINE and Epub Ahead of Print, In-Process & Other Non-Indexed Citations and Daily), EMBASE, Global Health, Northern Light Life Sciences Conference Abstracts, CINAHL Plus with full text and Africa Wide Information. We included literature published since the earliest date indexed in each database up to 20 August 2018 in English, French, German, Portuguese or Spanish. The search terms were related to (1) event-based surveillance and early warning alert and response, or (2) different types and channels of surveillance that are used to detect outbreaks or epidemics and (3) LMICs and regions (online supplementary file 1). The search strategies included indexed terms where possible and the list of LMICs was adapted from the standard list available at OVID MEDLINE based on World Bank Group classification 2017–2018.⁷ We adapted the search strategy for MEDLINE and tailored it to each database. Furthermore, we conducted a backward citation search and searched the references for each included article for relevant literature. We repeated all steps of the systematic assessment of titles and abstracts and full texts for all cited references.

Additional grey (unpublished) literature was retrieved by contacting epidemiologists at agencies and networks known to be involved in implementation or evaluation of EBS at the London School of Hygiene and Tropical Medicine, Médecins Sans Frontières, Norwegian Red Cross, CARE, US Centers for Disease Control (US CDC), WHO, and the Training Programs in Epidemiology and Public Health Interventions Network (TEPHINET).

Definitions

We defined the terms EBS and associated terminology in accordance with WHO guidance^{2 3} (table 1).

Table 1.

Systematic and grey literature review on event-based surveillance in the field: definitions

EBS	For the purpose of this review, we defined EBS as “the organised collection, monitoring, assessment and interpretation of information of mainly unstructured ad hoc information regarding health events or risks, which may represent an acute risk to human health”2 and require rapid reporting and assessment.2 3 The events can relate to the occurrence of epidemic-prone diseases and other chemical and environmental hazards in humans (eg, cluster of cases of disease, unusual disease patterns, unexplained deaths, chemical spills) or related to potential exposure for humans (eg, mass deaths among animals).2 3 We included EBS systems (as distinct from IBS2) characterised by the following attributes: (1) the source of signal can be a report from the general public, healthcare workers, community-level volunteers and it is not limited to the use of data and other information that arrive via a formal IBS system; (2) notifiable events are not defined by symptoms or syndromes but by (specified or unspecified) events related to health threats; (3) notification from the field to the EBS system is immediate (rather than planned weekly or monthly); (4) every notified signal requires rapid verification; (5) every verified event requires a timely response. EBS as defined by WHO2 can include an epidemic intelligence component that is based on the systematic automated analysis of various media sources (such as Epidemic Intelligence from Open Sources29 and the Global Public Health Intelligence Network30 on global scale). However, given the difference in structure and remit, we restricted our review to field-based EBS sourced from informants in health facilities or in the community.
Signal	A signal is reported data or information which represent a potential acute risk to human health. It is transmitted immediately and has not yet been verified to meet the event definition of the EBS system.2
Event	The IHR define an event as “(…)a manifestation of disease or an occurrence that creates a potential for disease; (…)”.1 In the context of event-based surveillance, an ‘event’ refers to ‘a signal’ that has been verified to meet the event definition of the EBS system.2
Alert	In this document (as in the WHO guidance 20142), “an alert will refer to a public health event that has been (i) verified and (ii) risk assessed and (iii) requires an intervention (an investigation, a response or a communication).”2
Community-based surveillance (CBS)	“CBS is the systematic detection and reporting of events of public health significance within a community by community members”.11 It is a surveillance system (IBS or EBS) that relies on defined individuals from the community (often called community volunteers) who received special training on event definition (for EBS) or case definitions (for IBS) and who notify signals to the surveillance system.
Health facility–based surveillance	A surveillance system (IBS or EBS) that relies on health professionals notifying signals to a next level in the surveillance system.
Open surveillance	We use the term ‘open surveillance’ to describe systems that did not specify who can notify signals but were open to receive signals from anyone (including lay people, media, NGOs, health professionals, teachers etc).
Outbreak setting	Refers to a setting in which EBS was implemented during an outbreak to enhance outbreak-specific surveillance.
Routine setting	Refers to a setting in which the purpose of the EBS system is to contribute to routine surveillance of defined or undefined events to detect outbreaks and other public health emergencies outside outbreaks.

CBS, community-based surveillance; EBS, event-based surveillance; IBS, indicator-based surveillance; IHR, International Health Regulations; NGO, non-government organisation; WHO, World Health Organisation.

Selection of literature

Following the elimination of duplicates, two researchers independently assessed the potential relevance of all papers, starting with titles and abstracts (screening) and full text (data extraction) using set inclusion and exclusion criteria. A consensus was reached for any discrepancy through discussion between the two researchers. For conference abstracts that were identified during the literature search, we contacted the corresponding author to obtain additional information.

Inclusion criteria

The full text referred to EBS as defined above and to LMICs as defined by the World Bank Group 2017–2018.⁷

Exclusion criteria

We excluded any full texts that described the EBS system but did not provide any data (quantitative or qualitative) on its functioning or effectiveness. We excluded any full texts that referred exclusively to (1) either case-based or syndromic IBS systems, or (2) systems using thresholds (of numbers of observations) to generate an alert. We excluded full texts that described mixed surveillance systems (IBS and EBS) if the evaluation did not provide detailed information on the EBS component. In addition, descriptions of EBS that focused on animal health without any link to the human health were excluded. Finally, we did not include global EBS that relied on algorithms for detection of media or social media activity only.

In the case of multiple publications based on the same EBS system/dataset, the quality and content were assessed simultaneously. Structure and components are described by EBS system (and not by publication or by dataset).

Assessment of quality of studies

We assessed the quality of the evidence using the GRADE approach.⁸ In addition, we developed a score to evaluate the quality of the evaluation based on the guidance by CDC for evaluating surveillance systems.⁹ We based our approach on the eight key elements of surveillance system evaluation described by CDC⁹ and added two additional categories: “response mechanism described” and “feedback mechanism to stakeholders described”. We scored the quality of description of the surveillance system (0–2 points). Zero points applied for “not stated/missing information”, one point for “incomplete description” and two points for “comprehensive description”. We calculated a summary score (0–20 points) for the quality of description. In addition, we graded each attribute evaluated and the comprehensiveness of evaluation using a simple colour scheme (green: comprehensively evaluated; yellow: partly evaluated; red: not evaluated).

Data extraction

We extracted information for each study using a standardised form assessing quality of the description, context, event definitions, characteristics and attributes evaluated.

Data synthesis

We used a framework approach¹⁰ to facilitate qualitative data synthesis and exploration of patterns within and across articles. We summarised quantitative results without pooling the data as the lack of standardisation between surveillance systems and evaluations does not allow for the calculation of quantitative summary measures. Following an inductive process, we developed typologies of EBS systems separately for outbreak and routine contexts. We synthesised available information for each of 10 defined attributes of surveillance systems⁹ using the framework approach.¹⁰

Patient involvement

Design, literature search, data synthesis and interpretation were done without patient involvement.

Results

Selection of literature

After deduplication, we screened titles and abstracts of 778 records (systematic literature search, grey search and backwards search combined) (figure 1). We screened full texts for 90 records, of which we included 17 (figure 1).

Figure 1.

Systematic and grey literature review on event-based surveillance in the field: PRISMA flow chart for systematic literature review (black ink letters) and grey literature search (blue ink letters) and backward citation search (green ink letters) describing identification, screening, eligibility and inclusion. EBS, event-based surveillance; IBS, indicator-based surveillance; LMIC, low-income and middle-income countries.

Among the 17 included records were 13 peer-reviewed publications, one manuscript under peer review, two conference abstracts including presentations and one agency report.

All full texts assessed and the reason for exclusion are available in online supplementary file 2.

Quality assessment

The 17 identified records referred to 15 studies (based on individual sets of data) and 13 different EBS systems.^8–24 For the quality appraisal, we assessed each of the 15 studies. Three studies were of low evidence (observational studies with comparison groups) and 12 were of very low evidence (observational studies without any comparison group), according to GRADE.

The quality of description of the EBS system varied from a summary score of 5/20 to 20/20 (online supplementary file 3). All 15 studies described the objective of the surveillance system, the event(s) under surveillance and the components of the EBS system to some extent (online supplementary file 3). With regards to the attributes described, 9 of the 17 studies described the positive predictive value of signals to events for the EBS system to some extent. All other attributes were rarely described (online supplementary file 3).

Data extraction

Type of studies and context of EBS systems

Of the 15 included studies, 11 studies were evaluations of EBS systems, three were descriptions of EBS systems and one was an assessment of the response to EBS alerts (table 2).

Table 2.

Systematic and grey literature review on event-based surveillance in the field: summary of types of studies (n=15) and context and setting of event-based surveillance systems (17 publications on 15 studies)

Publication		Context and setting			Methodology
Title	Type of publication	Place and scope	Country, area and population	Time, scale and frequency of reporting	Type of data	Methods and attributes evaluated
Outbreak setting	Ratnayake R, et al (2016). “Assessment of Community Event-Based Ssurveillance for Ebola Virus Disease, Sierra Leone, 2015.” Emerging Infectious Diseases 22(8): 1431–143719 and ERC (2015) Evaluation of the Functionality and Effectiveness of Community Event-Based Surveillance (CEBS) in Sierra Leone20	Peer-reviewed publication. Evaluation of EBS.	Sierra Leone. Emergency setting/outbreak. Primarily rural area. Community-based.	Sierra Leone, 9/14 districts. Population 3.9 million.	Feb 2015–Sep 2015 in nine districts. Exhaustive surveillance. Immediate reporting.	Quantitative EBS data. Quantitative comparison with other surveillance data.	Description of overall type of signals over time, usefulness: identification of EVD and other outbreaks, PPV (confirmed cases/all suspect, probably, confirmed cases), sensitivity of CEBS (CEBS cases/all confirmed cases). Description of Kambia CEBS cases with no epi link: sensitivity of CEBS (CEBS cases/all confirmed cases), timeliness (days): onset to detection.
	Stone E, et al (2016). “Community Event-Based Surveillance for Ebola Virus Disease in Sierra Leone: Implementation of a National-Level System During a Crisis.” PLoS currents 8. and ERC (2015) Evaluation of the Functionality and Effectiveness of Community Event-Based Surveillance (CEBS) in Sierra Leone20	Peer-reviewed publication. Evaluation of EBS.	Sierra Leone. Emergency setting/outbreak. Primarily rural area. Community-based.	Sierra Leone, 9/14 districts. Population 3.9 million.	Mar 2015–Aug 2015. Exhaustive surveillance. Immediate reporting.	Quantitative EBS data. Quantitative survey among personnel. Qualitative interviews among personnel.	Description of data quality (proportion of community health monitor (CHM) who correctly recalled trigger events), acceptability (proportion of CHM reporting weekly and proportion of district stakeholders finding CEBS useful), other: process evaluation of implementation.
	Lee CT, et al. (2016). "Evaluation of a National Call Centre and a Local Alerts System for Detection of New Cases of Ebola Virus Disease—Guinea, 2014–2015.” MMWR. Morbidity and Mortality Weekly Report 65(9): 227–23018	Peer-reviewed publication. Evaluation of EBS.	Guinea. Emergency setting/outbreak. Countrywide. Anyone (person/agency) can notify event.	Guinea. Population 11.8 million.	Nov 2014–Aug 2015. Exhaustive surveillance. Immediate reporting.	Quantitative EBS data. Quantitative comparison with other surveillance data.	Description of number of signals over time, sensitivity of (1) National Call Centre and (2) Local Alerts System.
	Miller LA, et al (2015). “Use of a nationwide call centre for Ebola response and monitoring during a 3 day house-to-house campaign—Sierra Leone, September 2014.” MMWR. Morbidity and Mortality Weekly Report 64(1): 28–2921	Peer-reviewed publication. Assessment of EBS response.	Sierra Leone. Emergency setting/outbreak. Countrywide. Anyone (person/agency) can notify event.	Sierra Leone. Poplation 7 million.	19–21 Sep 2014. Exhaustive surveillance. Immediate reporting.	Quantitative EBS data. Quantitative survey among lay people who notified alerts.	Description of number of signals over time, other: response: proportion calls that resulted in action (assessment of the situation on site).
	Santa-Olalla P et al (2013). “Implementation of an alert and response system in Haiti during the early stage of the response to the cholera epidemic.” The American Journal of Tropical Medicine and Hygiene 89(4): 688–69725	Peer-reviewed publication. Description of EBS.	Haiti. Emergency setting/outbreak following natural disaster (UN clusters activated). Countrywide. Anyone (person/agency) can notify event.	Haiti. Population 10 million.	Nov 2010–Nov 2011. Exhaustive surveillance. Immediate reporting.	Quantitative EBS data. Quantitative comparison with other surveillance data. Case study.	Description of number of signals over time and type of alerts, usefulness: action taken based on EBS’ data quality: proportion of documented responses and validity: comparison with IBS data, acceptability: transition to local ownership, flexibility: change of case definitions, other: exit strategy. Case study illustrating sensitivity and timeliness.
Routine setting	Clara A et al: Factors Influencing Community Event-based Surveillance: Lessons Learned from Pilot Implementation in Vietnam. Health Security Volume 16, Number S1, 2018. DOI: 10.1089/hs.2018.0066 (not published yet)13	Peer-reviewed publication. Evaluation of EBS.	Vietnam. Routine setting. Urban and rural area. Community-based.	Vietnam, 6/63 provinces. Population 8 million; 9% of the Vietnamese population.	Sep 2016–Dec 2017. Exhaustive surveillance. Immediate reporting.	Quantitative EBS data. Quantitative survey among personnel.	Including only new information (compared with previous publication): Description of type of signals and event incidence over time, village health worker characteristics, PPV (signal:event ratio), acceptability: willingness to participate via quantitative questionnaire. Evaluation of factors influencing event incidence rate (MVA).
	Clara A, et al (2018). “Event-Based Surveillance at Community and Healthcare Facilities, Vietnam, 2016–2017.” Emerging Infectious Diseases 24(9): 1649–1658.14	Peer-reviewed publication. Evaluation of EBS.	Vietnam. Routine setting. Urban and rural area. Community-based.	Vietnam, 4/63 provinces. Population 6 292 800; 7% of the Vietnamese population.	Sep 2016–May 2017. Exhaustive surveillance. Immediate reporting.	Quantitative EBS data. Quantitative survey among personnel. Qualitative interviews among personnel Case study.	Description of signals over time and sources of signals, usefulness: proportion agreeing EBS supports outbreak detection via quantitative questionnaire, PPV: events/signal, acceptability (and sustainability): willingness to participate via quantitative questionnaire and motivation via QI and FGD, timeliness (hours): detection to notification and detection to response. Evaluation of event definitions via QI and FGD. Case study from detection to response.
	Merali S, et al (2018). “Lessons Learned from Community Event-Based Surveillance Implementation in Ghana.” ICEID. 26.–29.08.2018 Atlanta	Conference presentation. Description of EBS.	Ghana. Routine setting. Urban and rural area. Community-based.	Ghana, 2 pilot districts. Population 264 536.	Jun 2017–Aug 2018. Exhaustive surveillance. Immediate reporting.	Quantitative EBS data. Case study.	Description of type of signals, PPV (signals-events-responses), other: lessons learnt. Case study from detection to response.
	Larsen TM et al (2017). "Red Cross volunteers’ experience with a mobile community event-based surveillance (CEBS) system in Sierra Leone during- and after the Ebola outbreak—a qualitative study”. Health Prim Car 1 (3):1–715 and (2016). “A Qualitative Study of Volunteer Experiences With a Mobile Community Event based Surveillance (CEBS) System In Sierra Leone.” IJID 53 Suppl: S11616	Peer-reviewed publication and conference presentation. Evaluation of EBS.	Sierra Leone. Routine setting (post-outbreak). Primarily rural area. Community-based.	Sierra Leone, 3/14 districts. Population not specified	Jul 2015/Dec 2015/Jan 2016. Exhaustive surveillance. Immediate reporting.	– Qualitative interviews among personnel.	Description of acceptance, experiences of volunteers.
	Toyama Y et al (2015). “Event-based surveillance in north-western Ethiopia: experience and lessons learnt in the field.” Western Pacific Surveillance and Response Journal: WPSAR 6 (3): 22–2722	Peer-reviewed publication. Evaluation of EBS.	Ethiopia. Routine setting. Rural area. Community-based.	Ethiopia, Amhara region, 3 zones with 175 Health Centres (HCs). Population 4.5 million.	Oct 2013–Nov 2014. Sentinel surveillance in 59 HC, each serving 25 000 population. Immediate reporting.	Quantitative EBS data. Quantitative comparison with other surveillance data.	– Description of type signals and sources of signals, usefulness: action taken based on EBS, data quality: completeness of rumour log books and validity of measles signals, PPV: proportion of verified rumours, sensitivity: comparison with IBS data, acceptability: proportion of rumours that were notified by the community, timeliness (days): onset to reporting and reporting to response.
	Oum S et al (2005). “Community-based surveillance: a pilot study from rural Cambodia.” Tropical Medicine & International Health 10(7): 689–69726	Peer-reviewed publication. Evaluation of EBS.	Cambodia. Routine setting. Rural area. Community-based.	Cambodia, 7 communities; served by four health centres. Population 30 000.	Sep 2000–Aug 2002. Exhaustive surveillance. Immediate reporting.	Quantitative EBS data. Quantitative comparison with other surveillance data.	Description of type of signals, PPV: proportion of verified outbreaks, other: resources: costs, training and time, additional indicators for IBS component of the system evaluated but not considered here.
	Naser AM, et al (2015). “Integrated cluster- and case-based surveillance for detecting stage III zoonotic pathogens: an example of Nipah virus surveillance in Bangladesh.” Epidemiology & Infection 143(9): 1922–193024	Peer-reviewed publication. Evaluation of EBS.	Bangladesh. Routine setting. Predominantly rural area. Health facility based.	Bangladesh, 10 sentinel hospitals. Population not specified.	Feb 2006–Sep 2011. Sentinel surveillance. Immediate reporting.	Quantitative EBS data. Quantitative comparison with other surveillance data.	Description of: number of Nipah clusters and non-Nipah clusters identified, PPV: proportion of Nipah clusters/non-Nipah clusters, sensitivity: meningo-encephalitis cases identified with cluster surveillance among all meningo-encephalitis cases.
	Sharma R et al (2009). “Communicable disease outbreak detection by using supplementary tools to conventional surveillance methods under Integrated Disease Surveillance Project (IDSP), India.” Journal of Communicable Diseases 41(3): 149–15927	Peer-reviewed publication. Description of EBS.	India. Routine setting. Countrywide. Health facility based.	India. Population 1.2 billion.	Apr 2008–Jun 2009. Exhaustive surveillance. Immediate reporting.	– Quantitative EBS data.	Description of number of calls received over time. Further surveillance systems outside the scope of this review.
	Tante S et al (2015). “Which surveillance systems were operational after Typhoon Haiyan?” Western Pacific Surveillance and Response Journal: WPSAR 6(Supplement 1): 66–7023	Peer-reviewed publication. Evaluation of EBS.	Philippines. Routine EBS surveillance evaluated in emergency setting/natural disaster. Areas affected by typhoon. Anyone (person/agency) can notify event.	Philippines (3 regions including 11 surveillance units affected by typhoon). Population not specified.	18 weeks following 11 Aug 2013 (day typhoon hit). Exhaustive surveillance. Immediate reporting.	– Quantitative survey among personnel.	Description of stability: operationality by area (yes/no) and functionality on Likert scale (1–5), other: complementary function on Likert scale.
	Dagina R et al (2013). “Event-based surveillance in Papua New Guinea: strengthening an International Health Regulations (2005) core capacity.” Western Pacific Surveillance and Response Journal: WPSAR 4 (3): 19–2528	Peer-reviewed publication. Evaluation of EBS.	Papua New Guinea. Routine setting. Countrywide. Anyone (person/agency) can notify event.	Papua New Guinea. Population ~7 million.	Sep 2009–Nov 2012. Exhaustive surveillance. Immediate reporting.	– Quantitative EBS data.	Description of type of signals over time and sources of signals, usefulness: action taken based on EBS, PPV: proportion of verified events, timeliness (days): onset to reporting and reporting to verification, other: laboratory confirmation of signals.

Yellow: EBS systems in outbreak settings; Blue: EBS systems in routine settings. The colors are already labelled in all tables.

CBS, communinty-based surveillance; CEBS, Community-event-based surveillance; CHM, Community Health Monitor; EBS, event-based surveillance; FGD, Focus group discussion; IBS, indicator-based surveillance; MVA, multi variable analysis; PPV, Positive predictive value; QI, Qualitative interviews.

Structure, components and functioning of EBS systems

Event definitions were disease specific in the four EBS systems in outbreak settings and one routine EBS system. In other routine surveillance systems, definitions were a mix of events targeting emerging diseases, unusual events and early detection of outbreaks (table 3).

Table 3.

Systematic and grey literature review on event-based surveillance in the field: event definitions as described in identified EBS systems (n=13)

	Publication	Setting: routine/outbreak context	Event	Event definition
Outbreak setting	Ratnayake et al (2016)19 and Stone et al (2016)17 and ERC (2015)20	Community-based/trained volunteers notify events.	Trigger events for EVD transmission.	(1) >2 sick or dead members in a household, (2) a sick or dead person after an unsafe burial or corpse washing, (3) a sick or dead health worker or traditional healer, (4) a sick or dead traveller, (5) a sick or dead contact of an EVD patient, (6) an unsafe burial or corpse washing, (7) ‘other’ was included so that community health monitors could report and describe other unusual events that did not fall under any of the six defined trigger events.
	Lee et al (2016)18	Anyone (person/agency) can notify events.	Any community deaths and suspect Ebola cases.	No further specification.
	Miller et al (2015)21	Anyone (person/agency) can notify events.	Possible Ebola cases and deaths and any Ebola information.	No further specification.
	Santa-Olalla et al (2013)25	Anyone (person/agency) can notify events.	Cholera event requiring immediate response. Unusual events.	(1) First cases or deaths (in previously cholera-free areas); (2) deaths in the community (cholera or other cause); (3) significant increases of numbers of cases or deaths (in areas with confirmed cases of cholera as indicated by local partners and/or field team members); (4) insufficient treatment capacity (hospitals, CTCs and CTUs); (5) need for partners, drugs, and/or equipment supplies or staff for the CTCs, CTUs or ORPs; (6) lack of access to healthcare services and/or potable water; (7) lack of a sanitation strategy (body management and disposal and waste management); (8) lack of training (eg, in case management or prevention). Any event that could pose a public health risk using the framework established by the IHR (2005).
Routine setting	Clara et al (2018)14 and (2018)13	Community-based/trained volunteers notify events.	Emerging new diseases, rabies, avian influenza, vaccine-preventable diseases, cholera (no specification which indicator for which event). Revised Aug 2017.	>2 hospitalised persons and/or death with similar type of symptoms occurring in the same community, school, or workplace in the same 7 days. Unexpected large numbers of children absent from school because of the same illness in the same 7-day period, sales at pharmacies of many people buying medicines for the same kind of illness, people sick with the similar type of symptoms at the same time, deaths of poultry or other domestic animals. A dog that is suspected to be rabid or a sick dog that has bitten someone or any dog that has bitten >2 persons in the past 7 days. one child <15 years of age with sudden weakness of limbs or fever, rash, respiratory infection and possibly red eyes. A single case severe enough to require hospital admission or causing death of any of the following: >3 rice watery stools in 24 hours in any person >5 years of age with dehydration, a new respiratory infection with fever in a person who has travelled abroad in the past 14 days, a new respiratory infection with fever after contact with live poultry, illness within 14 days after vaccination, illness never seen before or rare symptoms in the community.*
	Merali et al (2018)	Community-based/trained volunteers notify events.	Unusual health events.	Unexpected large numbers of children absent from school due to the same illness. Increase in number of animal deaths, including poultry, within 1 week. Two or more persons presenting with similar illnesses in the same community within 1 week. Two or more persons that pass watery stools and/or vomiting after eating/drinking at a given setting (eg, wedding, funeral, festival, canteen, food sellers etc). Two or more persons dying in the same community within 1 week. Any person who has been bitten, scratched or whose wound has been licked by a dog, cat or other animal. Any person with fever and rash, any person with worms emerging from any part of the body. Any person 5 years or more with lots of watery diarrhoea and sometimes vomiting profusely as well. In case of an outbreak anybody who passes watery/loose stool. Any newborn who is able to suck and cry at birth and then, after 2 days, is unable to suck or feed and becomes stiff. Any person with fever and neck stiffness. Any person who develops sudden weakness in the limbs.
	Larsen et al (2017)15 and (2016)16	Community-based/trained volunteers notify events.	6 community events.	VHF including Ebola+Lassa, AWD as proxy for cholera, measles, community deaths, floods, fire. No further specification.
	Toyama et al (2015)22	Community-based/trained volunteers notify events.	Unusual events.	Any communicable disease outbreaks and unusual health events, particularly events with multiple deaths from unknown causes.
	Oum et al. (2005)26	Community-based/trained volunteers notify events.	Cluster of cases.	A group of five or more similar cases occurring unusually closely together in any village within a week.
	Naser et al (2015)24	Health facility–based/health workers notify events.	Suspect meningo-encephalitis cluster.	≥2 meningo-encephalitis cases, aged ≥5 years, living within 30 min walking distance of each other who developed illness within 3 weeks of one another.
	Sharma et al (2009)27	Health facility–based/health workers notify events.	Any unusual health events.	No further specification.
	Tante et al (2015)23	Anyone (person/agency) can notify events.	Rare, unusual events.	No further specification.
	Dagina et al (2013)28	Anyone (person/agency) can notify events.	Potential public health events.	No further specification.

*The system included IBS community-case definitions, in addition to EBS event definitions.

AWD, acute watery diarrhoea; CTC, cholera treatment centre; CTU, cholera treatment unit; EBS, event-based surveillance; EVD, Ebola virus disease; IHR, International Health Regulations; ORP, oral rehydration points; VHF, viral haemorrhagic fever.

Trained lay community volunteers (CVs) notified events for 6/13 EBS systems, healthcare workers notified events in 2/13 EBS systems, and any member of the public was able to notify events using the two public telephone hotlines and in the three centralised surveillance systems (table 4). A verification and risk assessment mechanism was described for 11/13 EBS systems and a standardised algorithm for verification was mentioned in 3/13 EBS systems (table 4). All EBS systems were integrated into the routine surveillance system to some extent (table 4). Routine surveillance staff were involved in the response in all systems that specified the responding body (12/13). Six of 13 EBS systems specified a feedback mechanism to stakeholders (table 4).

Table 4.

Systematic and grey literature review on event-based surveillance in the field: structure and components of EBS systems (n=13)

Publication	Components	Data flow	Integration into routine surveillance	Feedback to stakeholders	Response mechanisms	Resources needed
Outbreak setting	Ratnayake et al (2016)19 and Stone et al (2016)17 and ERC (2015)20	Notifier: Community health monitors (CHMs). CHM were volunteers or existing community health workers (CHWs), who received training to detect the six trigger events. Assigned to villages. Receiver: Community surveillance supervisors (CSSs) with assigned a larger area. Verification: CSS, sometimes with support from community health officers (CHOs). CHOs were trained professionals in the public health system. Standard verification process not specified. A standard alert log included: type of alert, age, sex, location and action taken as a result of the alert and if District Ebola Response Centre (DERC) was notified. Risk assessment: Not applicable. Response: Any verified suspect events were reported to local DERC for follow-up. Local social mobilisation teams were notified to provide health education activities. Data collection and transmission: Mobile phones. All CEBS staff received phones and phone credit.	CHM notified→CSS verified and kept log→CHO support verification if needed→local DERC responds. Second additional data flow: CSS→district surveillance supervisor→the national CEBS coordination in Freetown.	Alerts that remained suspect after verification were reported to DERC for response.	CEBS district lead confirmed final alert status with the DERC database. Feedback mechanism not specified.	Response by DERC. Case investigation.	Staff: 7416 CHM, 137 surveillance supervisors. Costs: Start-up costs: US$1.3 million. Monthly costs US$129 000 covered training, telephones, motorbikes, fuel and incentives.
	Lee et al (2016)18	Notifier: (a) Anyone, nationwide for National Call Centre; (b) anyone in the prefecture for Local Alert Number. Receiver: (a) National Call Centre; (b) Local Alert Number. Verification: (a)+(b) prefecture is informed about the call. Verification algorithm not specified. Risk assessment and response: (a)+(b) prefecture is informed about the call. Data collection and transmission: Not specified.	(a) National Call Centre→dispatch team→prefecture. (b) Local Alert Number→prefecture.	Not specified.	Not specified.	Not specified.	Not specified.
	Miller et al (2015)21	Notifier: Anyone, nationwide. Volunteers encouraged public to report to the hotline during 3-day campaign. Receiver: Hotline. Verification: District Ebola Response Teams (DERC). Verification algorithm not specified. Risk assessment and response: DERC. Data collection and transmission: Hotline.	Public→Hotline→District-level Ebola response teams.	Information received is shared with usual state or district Ebola surveillance officers.	Not specified.	Case investigations and follow-up actions: Transport of ill persons. Safe and dignified burials.	Not specified.
	Santa-Olalla et al (2013)25	Notifier: All partners (UN response clusters activated). Field teams reported daily, including zero-reporting. Receiver: “The operational hub of the A&R System was in Port-au-Prince, where alerts were received by the national alert team”. Verification: A&R coordinator or field teams or partners. Information received from ‘official or reliable sources’ was considered verified. All other information was verified. Risk assessment: Assessment by field teams. The assessment consisted of “(a) assess the public health risk posed by the reported event, (b) assess the need for and urgency of such a response, and (c) define the type of response and appropriate partners”. Response: Coordinated by A&R coordinator at national hub. In the field through field teams and ministry of health staff. Data collection and transmission: Telephone and email.	From any partner/field teams→operational hub in Port-au-Prince→field teams for assessment.	A&R system complements existing national surveillance system.	Daily alerts bulletin sent to response partners: alerts and hotspots and assessment of the need for response. Alert and responses also in a weekly bulletin with IBS data.	The system was set up to “organise a rapid response with partners to provide immediate support based on needs identified in the field (eg, supplies, training, social mobilisation, water, and sanitation)”.	Staff: National alert team and 5–8 PAHO/WHO field teams, broad network of partners. UN response cluster mechanism activated.
Routine setting	Clara et al (2018)14 and (2018)13	Notifier: 7167 village health workers (VHWs) and health collaborators (HCs) received training, phone minutes and communication materials. HCs are mostly important community members. Information source of signals: VHW, community members, teachers, pharmacy, clinics, media, office, factories, religious leaders, healers, others. Receiver: Commune Health Station (CHS). Verification: Initial verification: CHS. Comprehensive verification: District Health Centre (DHC). Verification algorithm not specified. Risk assessment: DHC. Response: Provincial Preventive Medicine Centres (PPMC) and/or Regional Institutes. Data collection and transmission: Phone, email, in person.	VHW/HC→CHS→DHC→PPMC→Regional Institutes→General Department of Preventive Medicine.	Complete integration into IBS system. IBS information flow and organisational structure used. Personnel received extra training. VHW existed in theory previously; structure revitalised.	DHC and CHS conduct regular meetings with VHW to ensure a feedback loop is completed.	Not specified. In case study: case finding and laboratory testing of food.	Training and training materials. One-off funding of communication material and infrastructure improvements. All staff positions previously existed.
	Merali et al (2018)	Notifier: Community CBS volunteers Receiver: Public health personnel at local health facility and district, regional, national level Verification, risk assessment and response: not specified. In the case study, district level conducted investigation and response. Data collection and transmission: Tools to report were distributed but not specified.	CBS volunteer→health facility→district→regional→national. Communication with veterinary side at district, regional and national level.	Complete integration into IDSR. Personnel received additional training. Community volunteers existed before.	Not specified.	Not specified.	Not specified beyond training, training materials and reporting tools. Roles of staff existed before.
	Larsen et al (2017)15 and (2016)16	Notifier: Community-based volunteers (CBVs) recruited from their communities and trained in event definitions and on how to report using SMS text messages with specified codes for reporting. CBVs reported suspect cases and measures taken on community level. Receiver: CBVs inform volunteer surveillance supervisors (VSSs). Verification: VSS. Verification algorithm not specified. Risk assessment: VSS informs community health officer (CHO) and assess the event together. Response: CHO reports to DERC/DHMT to initiate response. Data collection and transmission: SMS to a local number that is synced with online data collection tool Magpi. The information is automatically analysed in an analytical visualising database.	CVS→VSS→CHO→DERC/DHMT.	VSS reaches out to CHO, who channels the information into the routine surveillance and response system.	Not specified.	DERC/DHMT/health facility.	Not specified.
	Toyama et al (2015)22	Notifier: Health Development Armies (HDAs)=volunteer-based community health team. Received training on EBS by health extension worker (HEW). Report EBS to health centre (HC). HEWs, who work at health posts and serve about 5000 people, also reported EBS to HC. Other information sources of signals: community, health post workers and others. HC also report into routine indicator-based surveillance. Receiver: HC surveillance focal person at each health facility registers rumours in rumour log book. Verification: Initial Verification: HDAs and HEWs. Assist if needed: Surveillance officers at the district health office. Verification algorithm not specified. Risk assessment: Not specified. Response: Surveillance officers at the district health office instruct response activities and communicate with zonal and regional health departments for further assistance. Data collection and transmission: Not specified.	HDA/HEW/other source→HC surveillance focal person→surveillance officer at district health office.	All roles existed before the rumour log book was introduced, the system was integrated into the routine surveillance system.	Not specified.	Case management, active case finding, vaccination, patient referral.	“The cost of establishing the system was minimal, requiring only a brief orientation for the surveillance focal persons and printing and distribution of the rumour logbooks to the HCs”.
	Oum et al (2005)26	Notifier: Lay VHV chosen by village elder or health facility staff, receiving training, incentive and supervision. "A series of 3 day initial training workshops was held for both VHVs and health staff at each project site shortly before the implementation of the system. It was followed by a monthly half day of refresher training separately for VHVs and health staff and further training in collation and analysis of data for the health staff”. Receiver: Health centre staff. Verification: Not specified if verification was undertaken. Verification algorithm not specified. Risk assessment: Operational district office. Response: Operational district office and local health centre staff. Data collection and transmission: Not specified.	VHV→dedicated health centre staff→operational district staff→provincial health department.	“built on the existing health system and resources, following the Ministry of Health policy and strategy to strengthen the Operational District structure”.	Monthly meeting between VHV and focal points at health centre discussed data, decisions, response.	Outbreak investigation; implementation of control measures.	Travel costs, per diem, food, free medical care for VHVs. Annual cost ~US$0.5 per capita including visits from Phnom Penh for training, supervision, and evaluation.
	Naser et al (2015)24	Notifier: Any physician in sentinel hospitals. Receiver: Surveillance physicians in sentinel hospitals. Verification: Institute of Epidemiology, Disease Control and Research (IEDCR) of the Government of Bangladesh, with the International Centre for Diarrhoeal Disease Research, Bangladesh (icddr,b). Verification through standard questionnaire. Risk assessment and Response: IEDCR and icddr,b. Data collection and transmission: Not specified.	Physician→surveillance physicians→IECDR and icddr,b.	Runs parallel to routine IBS.	Surveillance physicians. IEDCR and icddr,b investigation teams.	Outbreak investigation, active case finding, collect detailed exposures, determine epi link between cases.	Not specified.
	Sharma et al (2009)27	Notifier: Healthcare workers and health professionals. Receiver: Call centre staff. Call centre provides information to state and district surveillance officers for assessment and response. Verification, Risk assessment and Response: State/district surveillance officers. Verification algorithm not specified. Data collection and transmission: 24/7 toll-free call centre, available in multiple languages.	Healthcare workers→Call centre→State/district surveillance officers.	Information received is shared with state or district surveillance officers.	Not specified.	Not specified.	Cost and staff intensive. 65% of call centre budget spent on human resources.
	Tante et al (2015)23	Notifier: Anyone: media, health workers or non-governmental organisations and other informal channels. Receiver, Verification and Verification algorithm, Risk assessment, Response: Not specified. Data collection and transmission: Formal reporting system, phone calls, text messages any type of reporting.	Entry at any point into the national surveillance system.	Established to complement the national IBS system. During disasters, syndrome-based system is added.	Not specified.	Not specified.	Not specified.
	Dagina et al (2013)28	Notifier: Any source, including health workers, non-governmental organisations, embassies, media and general public. No specific training mentioned. Receiver: EBS coordinator within the Command Centre of the Communicable Diseases Surveillance and Emergency Response Unit of National Department of Health (NDOH). Verification: EBS coordinator verifies events reported from non-health sources by contacting the nearest health authorities or provincial health offices (PHOs) who are responsible for disease surveillance and control. Structured and standardised report and assessment form available for documentation of transmitted information and verification. Risk assessment: EBS coordinator provides guidance to PHO. Response: Primarily PHO. Support from higher levels up on request. Occasionally third parties. Data collection and transmission: Not specified.	Any source→EBS coordinator→PHO. “By routing data directly from the ground level to the national level, the system bypasses established reporting channels, that is, from local/district to provincial to national levels”.	Runs in parallel to existing system. EBS coordinator reaches out to existing system for verification and response.	“All events investigated through the EBS system are reported back to stakeholders(…)through a weekly National Surveillance Bulletin”.	Outbreak investigation. Not specified.	Staff: 2 part-time staff members on national level.

A&R, alert and response ; CBS, community-based surveillance; CBV, community-based volunteer; CEBS, community-event-based surveillance; CHM, community health monitor; CHO, community health officer; CHS, commune health station; CHW, community health worker; CSS, community health supervisor; DERC, District Ebola Response Centre; DHC, district health centre; EBS, event-based surveillance; HC, health centre; HEW, health extension worker; IBS, incicator-based surveillance; PPMC, Provincial Preventive Medicine Centre; VHW, village health workers; VSS, volunteer surveillance supervisors.

Data synthesis

Types of EBS systems in outbreak contexts

Community-based surveillance in outbreak contexts

We identified one community event-based surveillance (CEBS) system operating during an outbreak: the Sierra Leonean CEBS implemented during the Ebola outbreak at national level.^{17 19 20} CVs and their supervisors were specifically identified and rapidly trained; supervisory staff provided verification and assessment. The district-level Ebola response system that was already in place at the local level conducted follow-up if needed.^{17 19 20} CVs notified defined Ebola-related events (table 4).^{17 19 20} No verification or assessment algorithm was specified.¹⁹

Open surveillance

We identified three EBS systems that were ‘open’ to receive outbreak-specific event notifications from anyone: the telephone hotlines during the Ebola outbreak in Guinea¹⁸ and Sierra Leone²¹ allowed anyone to notify unspecified Ebola-related events; signals received by call centres were forwarded to the Ebola response structure in place for verification and response.^{18 21} Similarly, the EBS system in place in Haiti at the start of the cholera outbreak allowed partners within and outside the UN health cluster to notify an additional list of defined cholera-related events, including the lack of access to healthcare and sanitation.²⁵ The focus was expanded to unusual events in general after the outbreak.²⁵ Notifications were received at central level; verification, assessment and response processes were co-ordinated centrally but supported locally by field response teams. The assessment consisted of several standard items to answer.²⁵

Types of EBS systems in routine contexts

Community-based surveillance in routine contexts

We identified five CBS systems that were functioning as routine EBS systems in Vietnam,^{13 14} Ghana,¹² Ethiopia,²² Cambodia²⁶ and Sierra Leone.^{15 16} CVs were already in place before the implementation of all five systems.10 12 13 19–22 CVs notified defined syndromes to IBS in addition to events to EBS in four systems.^{12–16 26} CVs working in EBS-only systems were instructed to report any outbreak or unusual event²²; CVs working in combined IBS and EBS systems had specific definitions for the events (table 4).^{12–16 26} All EBS systems in communities were built on existing IBS systems and infrastructure. No verification or assessment algorithm was specified for any of the systems.

Health facility-based surveillance in routine contexts

We identified two different types of health facility-based surveillance: in Bangladesh, routine health facility-based IBS in a set of hospitals was complemented by EBS in a way that clusters of meningo-encephalitis cases were notified immediately and investigated to identify small-area Nipah outbreaks.²⁴

In India, a hotline was set up to notify unusual events that was available to health professionals.²⁷ No verification or assessment algorithm was specified for any of the systems.

Open surveillance in routine contexts

We identified two ‘open’ surveillance systems: the EBS system in the Philippines after Typhoon Haiyan could receive event information through routine surveillance channels or informal reporting by anyone; steps for verification, assessment and response were not specified in the paper.²³ In Papua New Guinea (PNG), anyone could notify events to a national EBS coordinator, who reached out to provincial health authorities for verification, assessment and response. A standardised verification and assessment form was used and kept as a log.²⁸

Surveillance attributes of EBS systems

Data extraction for each surveillance system attribute can be found in online supplementary file 4.

Usefulness: events reported and actions taken

All EBS systems detected local outbreaks. All non–disease-specific and disease-specific EBS systems reported to have identified confirmed cases and outbreaks among humans (eg, measles, acute respiratory infections, chickenpox, acute watery diarrhoea, malaria, whooping cough, meningitis, acute flaccid paralysis) and among animals (eg, poultry die-off, rabies, anthrax) (online supplementary file 4).12–15 22 25–28 In addition, natural and man-made events (eg, floods, riots, chemical events, nutritional crisis) were reported.^{22 25 28}

In outbreak contexts, community event-based surveillance (CEBS) during the Ebola outbreak in Sierra Leone generated a large number of signals (>90%) that were mainly reported in the category ‘other’ (rather than in any defined category) and the majority (>85%) concerned deaths in the community.^{17 19 20} Alerts were investigated by response teams that were in place.¹⁹

Open Ebola telephone hotlines during the Ebola outbreak in Sierra Leone and Guinea received signals for cases: during a 3-day campaign in Sierra Leone, 75% of calls were verified as alerts,²¹ while only 12% of calls were verified as alerts over 10 months in Guinea.¹⁸ Alerts were investigated by the response teams in place though the number of true events (identified case(s) meeting the case definition) was not reported.

In the open cholera EBS system in Haiti, 90% of the signals received related to cholera (increase in cases, deaths, lack of supplies etc) but also relevant non-cholera signals were received including rabies, acute flaccid paralysis, anthrax and non-infectious hazards.²⁵ Alerts were of use to indicate underserved geographical areas within which to set up treatment centres, investigate outbreaks, and carry out community mobilisation and water and sanitation activities²⁵ (online supplementary file 4).

In routine contexts, CEBS systems identified human and animal health events and other hazards: CEBS in Vietnam identified mostly signals of hand-foot-mouth disease, suspect dengue, chickenpox, suspect mumps, acute respiratory infections, foodborne diseases and avian influenza in poultry.^{13 14} CEBS in Ghana identified increases in animal deaths, vaccine-preventable diseases and foodborne diseases.¹² CEBS in Ethiopia identified signals for outbreaks of measles, suspect rabies, anthrax, whooping cough, acute flaccid paralysis, neonatal tetanus, meningitis, acute watery diarrhoea, flood and malnutrition²²; 67% of signals were verified and resulted in response, such as case management, active case finding, patient referrals and vaccinations.²² In Cambodia, CEBS identified outbreaks of malaria, acute watery diarrhoea, measles and viral haemorrhagic fever.²⁶

Health facility-based EBS in Bangladesh for meningo-encephalitis clusters identified 62 signals that were investigated to identify chains of transmission.²⁴ The telephone hotline for health professionals in India identified nine outbreaks but was used for non-health-related calls more than 95% of the time.²⁷

The open generic EBS system in PNG received 61 signals from provincial health authorities, health professionals and media, NGOs and communities notified signals related to acute watery diarrhoea, influenza-like illness, acute gastrointestinal syndromes, acute fever and rash, acute neurological syndromes, acute febrile illness and other respiratory diseases.²⁸ Alerts were investigated by provincial health authorities with support from higher levels²⁸ (online supplementary file 4).

Data quality (validity and completeness)

In the context of outbreaks, for CEBS in Sierra Leone, five of six Ebola-related trigger events were correctly recalled by less than two-thirds of CVs 3 months after implementation.¹⁷

Santa-Olalla et al ²⁵ compared trends of EBS signals for cholera-related events with IBS data for validity of data and saw comparable trends with regards to time and geographical distribution with the exception that more EBS signals came from hard-to-reach areas with the lowest number of partners²⁵ (online supplementary file 4).

In a routine context in Ethiopia, similar trends in EBS signals about measles rumours and IBS measles data were reported over time.²² With regards to data completeness, 23% of signals had no records of verification and 29% had complete information on reporting and response times²² (online supplementary file 4).

Positive predictive value (PPV) for true events

The interpretation of PPV across EBS systems is dependent on the types of events used for the calculation. In the context of the Ebola outbreak, 12 126 signals were received through CEBS for Ebola in Sierra Leone, of which 287 were ultimately confirmed to meet the suspected, probable or confirmed case definition (PPV 0.024, 287/12 2126) and 16 Ebola cases were confirmed¹⁹ (online supplementary file 4).

In the routine surveillance context of Vietnam, an overall PPV of 0.07 (176 events/2520 signals) for CEBS was reported based on a list of defined events. PPV increased significantly over time after evaluation visits and improved event definitions from 0.06 initially to 0.12.¹³ CEBS in Ghana, Ethiopia and Cambodia reported PPVs of >0.60 for the signals to be an event^{12 22 26} (online supplementary file 4).

The EBS telephone hotline in India received 44 484 calls from health professionals, of which 1185 were health related, 112 were outbreak signals and 9 were true outbreak alerts (PPV for a signal to be an alert 0.08; 9/112).²⁷ The PPV for meningo-encephalitis cluster reported by health professionals to be a true Nipah cluster was 0.17 (10 clusters/62 signals).¹⁵

In PNG’s centralised routine EBS system, of 61 signals, 51 proved to be true events (PPV 0.84)²⁸ (online supplementary file 4).

Sensitivity

In outbreak context, Ratnayake et al ¹⁹ reported that 30% (16/53) of all confirmed cases of Ebola in the area where CEBS for Ebola was implemented were identified by CEBS. In Guinea, 3.9% (71/1838) of all confirmed Ebola cases were identified by signals received at the national Ebola call centre and 54% (120/221) of all confirmed Ebola cases were identified by local Ebola alert numbers¹⁸ (online supplementary file 4).

In the routine context of Ethiopia, the verified events reported by CEBS for rabies, anthrax, acute flaccid paralysis and neonatal tetanus outnumbered the cases reported IBS during the same time in areas where CEBS was active, indicating better sensitivity of EBS compared with IBS²² (online supplementary file 4).

Acceptability

In the context of the Ebola outbreak, Stone et al ¹⁷ evaluated acceptability as the proportion of CVs reporting for CEBS in Sierra Leone at least once per week; this increased from 69% to 93% during implementation. In addition, 74% of district stakeholders agreed that CEBS increased case detection and benefited their district through increased linkage with communities.¹⁷ CEBS was based on community health worker (CHW) and Red Cross volunteer networks that are largely known to the community prior to CEBS.^{16 17 19}

Santa-Olalla et al ²⁵ reported that the near real-time information and response increased the acceptance by local authorities during the outbreak and willingness to institutionalise the system into the Ministry of Health immediately after the end of the outbreak in Haiti (online supplementary file 4).

In routine context, CEBS in Sierra Leone was well accepted by communities and CVs were appreciated, and CVs wanted to continue CEBS after the outbreak despite the lack of incentives.^{15 16} Clara et al ^{13 14} reported “Key motivating factors for participation expressed by the VHWs (village health workers) were a sense of service to the community and opportunities to increase community ties, and improvement in community trust”. Toyama et al ²² interpreted the proportion of rumours reported by the community to CEBS in Ethiopia as community acceptance: 30% of rumours were reported directly from the community (online supplementary file 4).

Timeliness

During the Ebola outbreak in Sierra Leone, the time from onset to detection for six confirmed Ebola cases in districts with active transmission was 1–3 days for four cases identified with CEBS and 5–7 days for two confirmed Ebola cases identified through other sources (online supplementary file 4).¹⁹

For the routine context in Vietnam, for EBS times from detection to notification at district level was reported to be <24 hours, mean time from detection to response was reported to be within 48 hours.¹⁴ The authors conclude that such a rapid response would not have been possible before the implementation of EBS.¹⁴ For CEBS in Ethiopia, a median time from onset to reporting of 3.8 days (95% CI 2.2 to 5.3 days) was reported and from reporting to response of 0.6 days (95% CI 0.1 to 1.2 days).²² In PNG’s centralised routine EBS system, median time from onset to reporting was 10 days (range, 0–109 days) and median time from reporting to response was the same day²⁸ (online supplementary file 4).

Flexibility and stability

For the centralised EBS in Haiti and for CEBS in Sierra Leone, EBS was described as flexible to adapt and was broadened in scope, from EBS systems based on events in connection to one outbreak disease to generic EBS systems.^{15 16 25}

Sharma et al ²⁷ described a successful shift in focus of the EBS system from generic EBS to influenza A (H1N1) specific events.

Tante et al ²³ described the stability of the routine EBS system in the Philippines after a natural disaster and found that 6/11 affected areas had no interruptions of EBS operation, 3/11 areas were interrupted for 1 week while the two hardest hit areas had limited operationality for more than 7 weeks due to human resource and logistical limitations; 73% of all areas rated the EBS system as functional post-typhoon (online supplementary file 4).

Cost estimates

Several additional characteristics were evaluated (see online supplementary file 4), among them annual costs: data from Cambodia estimated the annual cost of CBS at US$0.5 per capita for training, supervision and evaluation, including CEBS and the IBS and vital events component but excluding staff costs as all staff were previously in place.²⁶ For the Sierra Leone CEBS, total costs were estimated at US$1.3 million start-up costs and US$129 000 monthly costs that covered training, telephones, motorbikes, fuel and incentives for 7416 CHWs and salaries for 137 surveillance supervisors¹⁹ (given a population of 3.9 million, ~US$0.4 per capita annual costs).

Discussion

What are the strengths of EBS?

The scope of EBS is broader than IBS in several aspects: (1) In outbreaks, EBS was able to identify outbreak-disease cases and programming gaps (eg, poor-quality cholera care) helping to guide the response^{19 20 25}; (2) in outbreaks and routine context alike, EBS was able to identify outbreaks that were not detected by IBS, both for notifiable and non-notifiable diseases12–14 19 20 22 24–26 28; (3) in outbreaks and routine contexts, EBS was able to identify other potential hazards to human health such as natural events or animal die-offs12 14 22 25 28; (4) in outbreaks and routine context, EBS at the community level and open EBS were able to reach remote areas14–17 19 20 22 25 26 28 and EBS was most used where health facilities and response was scarce in Haiti.²⁵ In these settings, EBS provided an important channel for rapid reporting of events, of particular benefit to remote settings where IBS was poor. In addition, data from the Sierra Leone CEBS suggest that EBS systems were faster to identify cases from onset to detection¹⁹ than IBS and other CEBS systems were also reported to be faster than IBS.^{14 22} However, across studies, no systematic and direct comparisons of timeliness of IBS and EBS were presented.

What are the trade-offs for EBS?

Many questions with regards to sensitivity and specificity of event definitions remain unanswered. For CEBS in outbreak settings, specified event definitions that indicate disease transmission had low specificity and event definitions were not always remembered long after trainings,^{17 19 20} suggesting that definitions should be more rigorously tested prior to implementation.¹⁹ Telephone hotlines were mostly used for information by the public (and useful for improving communication with the public) but also identified cases that went under the radar of IBS¹⁸ and from areas outside the reach of IBS.²⁵ EBS in communities showed a higher specificity than EBS using open hotlines, at the costs of limited reach (limited to communities with CEBS vs anywhere with phone connection).^{18 19} All EBS systems involved an increased workload for verification and assessment given the low specificity of the events and the volume of alerts produced.^{17–21 25}

In routine contexts, in which events were defined by a list of specific events (eg, for CEBS in Vietnam, Ghana and Sierra Leone), the main reported challenges included balancing sensitivity and specificity^{12 13} and uncertainties around event definitions.^{15 16} In Vietnam, initially more than half of the reported events were endemic diseases that were routinely and rapidly reported through IBS, and therefore, event definitions that could indicate such diseases were subsequently removed.¹³ Balancing sensitivity against specificity is particularly challenging as there is no gold standard for detection of events and non-events that would allow determination of sensitivity and specificity and available evidence is anecdotal, that is, reports of outbreaks identified by EBS but not IBS.²² In addition, balance of sensitivity versus specificity will inevitably depend on the setting, existing surveillance, access to healthcare and whether an outbreak is ongoing. CEBS systems that were based on undefined events such as ‘unusual events’ showed a PPV >0.6.^{22 26} The open EBS system in PNG notifying ‘public health events’ also exhibited a PPV >0.8, possibly because it was mainly used by local public health authorities to notify to central level.²⁸ There seems to be some indication that the PPV of undefined events is not worse than the PPV of defined events pointing to specific diseases; however, we do not know if this comes at the costs of lower sensitivity to detect outbreaks.

The amount of work for verification and response is higher in EBS systems than IBS systems, as every notified signal requires rapid verification and potentially response. In outbreak settings, all EBS were set up with international support^{17–21 25}; in routine contexts, dedicated national government leadership was often paired with regional or international support to sustain verification and response systems.12–16 22–24 26–28

With regard to costs, while most reports did not account for total direct costs,^{19 26} costs structures would be influenced by the provision of additional capacity for verification of the potentially large number of signals produced by EBS. Where human resources and response elements are not already in place, costs might be associated with EBS, that not currently absorbed by the surveillance system’s budget (ie, incentives for CHWs, additional verification capacity, including rapid response teams etc).

How to implement it?

In outbreak and non-outbreak settings, early engagement and meetings with all stakeholders (including community leaders) were described as crucial14 17 19 20 to ensure ownership, increase coordination and gain “better understanding of the existing surveillance landscape”.¹⁴

With regards to surveillance, Santa-Olalla et al ²⁵ described the importance of not undermining existing responsibilities and capacities for EBS cholera surveillance in Haiti. Routine context CEBS in Vietnam was developed at ministry of health level with involvement of all stakeholders to ensure integration into existing surveillance infrastructure.¹⁴ In Cambodia and Ethiopia, CEBS was reported to be functional mainly because it was simple and built on the existing surveillance systems and resources.^{22 26}

Good linkage to response was described as essential for all systems as every verified event needs an assessment and potentially response.^8–24 However, response was challenging in some systems and required scaling up.21 22 25 28 All outbreak-specific EBS systems were linked to pre-existing response structures set up specifically for the outbreak.^{17–21 25} In order to ensure timely reporting of defined events and sustainability, stakeholders in EBS need training and, if possible, monitoring, supervision and refresher trainings.14–17 20 22 Several reports suggested that standardised documentation and assessment improved accountability and ensured a consistent approach.^{14 22 28} In routine contexts, where the capacity to conduct risk assessment and field investigations is low, specific resources and training are required.^{14 28}

Several EBS systems incorporated feedback mechanisms to stakeholders to inform about findings and actions; all records that reported such feedback loops identified them as essential instruments to sustain motivation, especially in routine settings, and to improve cooperation.14 25 26 28 Communication improved between actors in the surveillance system and the community²⁶ but also between human and animal health¹⁴ and stakeholders at different levels at the surveillance system.²⁸

Limitations

There are some limitations stemming from the variety of EBS systems as well as evaluation designs and the poor evidence rating of the studies. The number of studies with low-quality evidence grading is due to the nature of surveillance systems evaluations which are inherently descriptive and seldom have a comparator. All studies were of low evidence and none compared EBS and IBS systems systematically, controlled or randomised. Most importantly, surveillance attributes, especially PPV, were not always reported or analysed in the same way, complicating meaningful comparisons. For the PPV, information on if and how signals that did not result in alerts were documented was frequently lacking.

Additionally, the number of studies was low, and more descriptions and rigorous evaluations are therefore needed to generate more conclusive evidence. Despite the increasing focus on EBS as a central component for early detection of outbreaks, there is limited guidance on EBS evaluation. The development of guidance more tailored to the evaluation of EBS would be useful as would be planning and publishing evaluations of EBS systems more systematically to ensure lessons learnt are documented and communicated.

Furthermore, all EBS systems succeeded in identifying events and outbreaks; however, we cannot rule out that publication bias towards successful systems is the reason for that finding.

In addition, other countries may have EBS systems in place, but their description might not be included in the paper given the absence of literature on them.

Conclusion

Health facility and community-based EBS provide valuable information that can strengthen the early warning function of national surveillance systems. Every signal generated by EBS requires verification, and each EBS system is only as good as the capacity to respond to verified events to ensure that EBS can lead to meaningful public health action.^8–24 This requires close integration with existing surveillance systems and this needs to happen in a way that does not undermine existing responsibilities and capacities.10 12–17 19 20 22 23 25 28 EBS can be a powerful tool to improve rapid communication between actors in healthcare and surveillance, and feedback loops to all stakeholders including communities were described as essential instruments for sustaining motivation and cooperation.16 17 19 20 23 25 28 Where EBS is not a fully integrated function of a surveillance system, priority in its implementation should be given to settings that would particularly benefit from EBS strengths. This includes areas most prone to outbreaks, where otherwise undetected health threats will most likely occur, where access to the formal health system is limited and where traditional ‘routine’ surveillance is suboptimal.