Evaluation of the early warning, alert and response system after Cyclone Winston, Fiji, 2016

Abstract

Objective

To assess the performance of an early warning, alert and response system (EWARS) developed by the World Health Organization (WHO) – EWARS in a Box – that was used to detect and control disease outbreaks after Cyclone Winston caused destruction in Fiji on 20 February 2016.

Methods

Immediately after the cyclone, Fiji’s Ministry of Health and Medical Services, supported by WHO, started to implement EWARS in a Box, which is a smartphone-based, automated, early warning surveillance system for rapid deployment during health emergencies. Both indicator-based and event-based surveillance were employed. The performance of the system between 7 March and 29 May 2016 was evaluated. Users’ experience with the system was assessed in interviews using a semi-structured questionnaire and by a cross-sectional survey. The system’s performance was assessed using data from the EWARS database.

Findings

Indicator-based surveillance recorded 34 113 cases of the nine syndromes under surveillance among 326 861 consultations. Three confirmed outbreaks were detected, and no large outbreak was missed. Users were satisfied with the performance of EWARS and judged it useful for timely monitoring of disease trends and outbreak detection. The system was simple, stable and flexible and could be rapidly deployed during a health emergency. The automated collation, analysis and dissemination of data reduced the burden on surveillance teams, saved human resources, minimized human error and ensured teams could focus on public health responses.

Conclusion

In Fiji, EWARS in a Box was effective in strengthening disease surveillance during a national emergency and was well regarded by users.

Introduction

Humanitarian emergencies, such as conflicts and natural disasters, increase the risk of communicable disease outbreaks.¹ Although effective and timely surveillance and response measures can mitigate risks,¹ public health systems are frequently disrupted during emergencies, particularly in developing countries where existing surveillance systems are fragile.² Several early warning, alert and response systems have been developed to enhance surveillance and response capacities during health emergencies.³ The World Health Organization (WHO) has produced guidelines on surveillance during health emergencies and has supported the development and implementation of surveillance systems in developing countries,⁴ such as the Disease Early Warning System (DEWS) used in Pakistan after the 2005 earthquake and the Surveillance in Post Extreme Emergencies and Disasters (SPEED) system used in the Philippines in 2010, in the Solomon Islands after the 2013 tsunami and in Vanuatu following Cyclone Pam in 2015.^3,5–7

Few low- and middle-countries have plans for enhancing surveillance systems during emergencies.⁴ Usually these systems are established during crises and therefore lack standardized methods for data collection, management and analysis.³ Moreover, the need to collect and process large volumes of data places a burden on health systems that are often struggling to manage other urgent priorities.^6,8–11 To address these challenges, WHO developed a portable, field-ready toolkit for an early warning, alert and response system (EWARS) that can be deployed within 24 hours in a major emergency: EWARS in a Box, hereafter referred to as EWARS.^12,13 The toolkit includes smartphones with preinstalled, open-source EWARS applications, laptops, a mobile and locally hosted server, and solar chargers. The system was first deployed in South Sudan in 2015, but its performance was not formally assessed.¹²

On 20 February 2016, Cyclone Winston, one of the most powerful storms recorded in the South Pacific, made landfall in Fiji. The cyclone affected around 400 000 people, damaged or destroyed 40 000 homes and displaced 55 000 people. Immediately afterwards, Fiji’s Ministry of Health and Medical Services and WHO conducted a rapid public health risk assessment to assess health priorities.¹⁴ They identified several factors that increased the risk of disease transmission and outbreaks: (i) large displaced populations; (ii) overcrowded emergency shelters; (iii) limited access to clean water; (iv) disruption of the sanitation infrastructure; and (v) increased exposure to mosquitos and other disease vectors. In addition, there was a moderate to high risk of outbreaks of several diseases prone to epidemics: (i) leptospirosis; (ii) diarrhoea (including dysentery); (iii) typhoid; (iv) dengue; (v) chikungunya; (vi) Zika virus infection; and (vii) acute respiratory infection.¹⁴ Due to this increased risk, the Ministry of Health and Medical Services implemented EWARS within 2 weeks with assistance from WHO.

Prior to Cyclone Winston, the 12 sentinel health-care facilities in the Fiji Syndromic Surveillance System reported weekly on five syndromes: (i) diarrhoea; (ii) influenza-like illness; (iii) prolonged fever; (iv) acute fever and rash; and (v) dengue-like illness.¹⁵ Following the cyclone, EWARS was installed at 34 health-care facilities (Fig. 1), including 11 of the 12 existing sentinel facilities, and reported weekly on nine syndromes. Surveillance sites were selected based on population density, proximity to severely affected areas, the number of displaced persons and access to transport and telecommunications. In addition to reporting cases that met specific case definitions (i.e. indicator-based surveillance), EWARS included event-based surveillance, which is the reporting of events that may not meet the reporting criteria for indicator-based surveillance, but may have important public health implications.¹⁶ For example, a cluster of unusual neurological disease cases that do not meet prespecified case definitions for indicator-based surveillance would not be reported and would, therefore, not trigger an alert. However, it may indicate a serious outbreak and would typically be reported through event-based surveillance. Most early warning systems rely on only indicator-based surveillance for outbreak detection.¹⁶

Fig. 1.

Path of Cyclone Winston and EWARS in a Box surveillance sites, Fiji, 2016

EWARS: early warning, alert and response system.

Notes: Projected population figures for 2016 were estimated by Fiji’s Ministry of Health and Medical Services using data from the 2007 census conducted by the Fiji Bureau of Statistics. Fiji is divided into four administrative divisions: Central Division includes Tailevu, Naitasiri, Rewa, Serua and Namosi Provinces; Western Division includes Ra, Ba and Nadroga-Navosa Provinces; Northern Division includes Bua, Macuata and Cakaudrove Provinces; and Eastern Division includes Lomaviti Province.

Source of shapefile: Paul Jaskierniak, World Health Organization Division of the Pacific Technical Support, Suva, Fiji.

We assessed the performance of EWARS during the post-disaster phase in Fiji. The primary objective was to assess the system’s functionality during a health emergency and the secondary objective was to assess its ability to provide timely alerts for suspected disease outbreaks.

Methods

Fiji has a population around 900 000 and is divided into four administrative divisions: Central, Western, Northern and Eastern.¹⁷ We evaluated the performance of EWARS in Fiji during a national state of emergency between 7 March and 29 May 2016. Implementation of the system started immediately after the cyclone. Three teams, each comprising one staff member from the health ministry and a WHO epidemiologist with expertise in EWARS, conducted 2-hour workshops at surveillance sites for focal points (i.e. health officials in charge of surveillance at individual EWARS sites) and other doctors and nurses responsible for data collection. Training covered: (i) the importance of early outbreak detection and responses; (ii) syndromes and diseases under surveillance; (iii) case definitions; (iv) use of smartphones and the EWARS application; and (v) reporting protocols and deadlines. Focal points were given smartphones with the EWARS application and were responsible for data entry. Although a network connection was not required for data entry, it was essential for transmitting data to the central EWARS database.

Our quantitative and qualitative evaluations of EWARS were based on previously published methods, including the United States’ Centers for Disease Control and Prevention’s framework for evaluating public health surveillance systems,^18–22 existing early warning and response guidelines,¹⁶ and the methods used to evaluate the Pacific Syndromic Surveillance System^15,23–25 and Fiji’s Syndromic Surveillance System (Fiji Ministry of Health and Medical Services, unpublished report, 2015). The system attributes evaluated are described in Table 1. We assessed the performance of EWARS using a semi-structured questionnaire in interviews with stakeholders, including four staff members from the health ministry, four from WHO and one from Fiji’s Health Sector Support Program, a bilateral programme supported by the Australian government. These individuals represented organizations involved in the design and implementation of EWARS or in its governance and performance. Other stakeholders represented partner organizations that were not involved in surveillance, but used EWARS data for planning responses, including staff from other United Nations agencies and the Fiji Red Cross Society.

Table 1. Attributes of EWARS in a Box evaluated after Cyclone Winston, Fiji, 2016.

EWARS attribute	Criterion	Data sources
Surveillance quality
Timelinessa	Submission of data on time (i.e. by the Monday following the week under surveillance, which lasted from Monday to Sunday)	EWARS database and cross-sectional survey of EWARS usersb
Completenessa,c	Submission of data by Sunday of the week following the week under surveillance	EWARS database and cross-sectional survey of EWARS usersb
Data validitya,c	Accuracy of the data recorded by the surveillance system	EWARS database, surveillance site visits and retrospective reviews of clinic records
Representativeness	Geographical appropriateness and coverage	Review of implementation protocol, EWARS database, population data from Fiji’s Ministry of Health and Medical Services and stakeholder interviews
System performance
Usefulnessa,c	EWARS’ contribution to monitoring disease trends and the early detection of disease clusters and outbreaks	Cross-sectional survey of EWARS usersb
Flexibilitya,c	Ability to adapt rapidly to changing information needs or operating conditions without substantial demands on staff or funding	Cross-sectional survey of EWARS usersb and stakeholder interviews
Simplicitya,c	Ease of use of surveillance system	Cross-sectional survey of EWARS usersb
Acceptabilitya,c	Willingness of users to participate in surveillance and data collection and analysis	Cross-sectional survey of EWARS usersb
Stability	Reliability and resilience of EWARS	EWARS database and cross-sectional survey of EWARS usersb
Cost	Expenditure on equipment and on implementing and operating the system	Fiji Ministry of Health and Medical Services, Fiji Health Sector Support Program and WHO Division of Pacific Technical Support

EWARS: early warning, alert and response system; WHO: World health Organization.

^a This attribute was evaluated only for indicator-based surveillance.

^b EWARS users were people who submitted data to the system (i.e. surveillance focal points or individuals in charge of surveillance sites).

^c For reasons of confidentiality, qualitative data are not reported.

We reviewed data collection and reporting processes between 28 March and 1 May 2016 during site visits to 11 EWARS sites. These sites were chosen for ease of access to both buildings and patient registers and for other logistical considerations. Data from patient registers were compared with corresponding EWARS data. Experience with using EWARS was evaluated in a cross-sectional survey of users and surveillance officers; users were people who submitted data (i.e. focal points or individuals in charge of sites). Users and surveillance officers were emailed self-administered online surveys designed using survey development software (SurveyMonkey, San Mateo, United States of America). All five surveillance officers (100%) and 27 of 34 focal points (79%) completed the surveys. All analyses were performed in Microsoft Excel 2010 (Microsoft Corporation, Redmond, USA), unless otherwise specified. The study was approved by the Fiji’s National Health Research and Ethics Review Committee (application number 2017.86.NW).

Results

The operation of EWARS, including indicator-based and event-based surveillance, is illustrated in Fig. 2. Under indicator-based surveillance, surveillance sites reported weekly on individuals aged younger and older than 5 years who satisfied the definitions for nine syndromes (Table 2): (i) acute fever and rash; (ii) prolonged fever; (iii) influenza-like illness; (iv) acute watery diarrhoea; (v) acute bloody diarrhoea; (vi) acute jaundice syndrome; (vii) suspected dengue; (viii) suspected meningitis; and (ix) Zika-like illness, which was added 3 weeks after surveillance started. Proportional morbidity was calculated from the total number of consultations in the relevant week. Under event-based surveillance, sites reported unusual public health events, such as clusters of deaths, unusually severe disease or widespread animal deaths that did not meet criteria for indicator-based surveillance.¹⁶ These events were immediately reported to a dedicated surveillance officer using the EWARS application, by email or on a toll-free phone number. An alert was generated if: (i) for indicator-based surveillance, the number of cases of a syndrome rose above the weekly threshold for the reporting site; or (ii) for event-based surveillance, an event occurred that EWARS site staff judged to have adverse public health implications. Designated surveillance team members were automatically notified about alerts by email. Five surveillance officers monitored and verified alerts, collected preliminary information about them and investigated any delays in reporting. Weekly EWARS epidemiological bulletins were produced using automated algorithms that compiled and analysed data. These bulletins were disseminated by email to EWARS focal points and other stakeholders. All data collected through EWARS remained the property of the national health authorities and were stored in a secure, cloud-based database (Table 3).

Fig. 2.

Operational flowchart, EWARS in a Box, Fiji, 2016

EWARS: early warning, alert and response system; SMS: short message service.

Note: An unexpected public health event was defined as an event that may have important public health implications, but that does not involve cases of syndromes monitored by indicator-based surveillance. For example, a cluster of unusual neurological disease cases that do not meet prespecified case definitions for indicator-based surveillance.

Table 2. Syndromes monitored by EWARS in a Box after Cyclone Winston, Fiji, 2016.

Syndrome	Case definition	Reporting thresholda	No. cases	Frequency (cases per 100 cases of all syndromes)b
Acute fever and rash	Fever (> 38 °C) reported or measured plus nonblistering rash	1 case	672	2.0
Prolonged fever	Fever (> 38 °C) reported or measured that lasted ≥ 3 days	Twice the average number of cases seen in the previous 2 weeks	1 461	4.3
Influenza-like illness	Fever (> 38 °C) reported or measured plus cough or sore throat or both	Twice the average number of cases seen in the previous 2 weeks	16 426	48.2
Acute watery diarrhoea	Three or more loose or watery (non-bloody) stools in 24 h	Twice the average number of cases seen in the previous 2 weeks	10 054	29.8
Acute bloody diarrhoea	Episode of acute bloody diarrhoea	3 cases at one location in 1 week or twice the average number of cases seen in the previous 2 weeks	293	0.7
Acute jaundice syndrome	Jaundice (i.e. yellow discoloration of the whites of the eyes or dark urine) and severe illness, with or without fever	3 cases	71	0.2
Suspected dengue	Fever for ≥ 2 days plus at least two of: (i) nausea or vomiting; (ii) muscle or joint pain; (iii) severe headache or pain behind the eyes; (iv) rash; or (v) bleeding	Twice the average number of cases seen in the previous 3 weeks	4 520	13.3
Suspected meningitis	Sudden onset of fever, plus at least one of: (i) severe headache; (ii) neck stiffness; (iii) altered consciousness; or (iv) petechial or puerperal rash	1 case	33	0.1
Zika-like illnessc	Generalized maculopapular rash plus at least two of: (i) arthralgia or myalgia; (ii) red eyes or non-purulent conjunctivitis; (iii) oedema of the hands or feet; (iv) low grade fever (< 38 °C); or (v) pain behind the eyes	3 cases	583	1.7

EWARS: early warning, alert and response system.

^a The initial reporting thresholds listed here were based on standard World Health Organization thresholds. They were changed using aberration detection algorithms when sufficient historical data had been collected.¹⁵

^b The total number of reported cases of all syndromes between 7 March 2016 and 29 May 2016 was 34 113.

^c Zika-like illness was added 3 weeks after surveillance started.

Table 3. Integrating EWARS in a Box into routine surveillance after a health emergency.

Integration optiona	Description	Advantages	Disadvantages
Full cloud computingb	EWARS application and all data are stored on a secure server hosted by a public cloud service provider, but data are still owned by the country where they were collected; the cost of the EWARS application and the server are met by WHO	Automatic software updates are applied centrally and are immediately available throughout the country; the country controls access to data	There is a perception that the country does not own the data, though WHO can provide an additional layer of security and technical support, if desired
Cloud computingb with a local server	EWARS application is stored on a public cloud, but data are stored on a local server; the cost of the EWARS application is met by WHO and that of the local server is met by the country	Software is updated automatically	Data may be lost if the server is not always accessible; data are safer and encryption is stronger with public cloud service providers, such as Amazon Web Services
Local server only	EWARS application and data are stored on a local server; the cost of maintaining EWARS and the server are met by the country	The country has complete control of the server, software and data	Software updates are not applied automatically; software maintenance, including installing upgrades, is more difficult to carry out as WHO has limited access to the server, though regular updates can be provided if requested, but will not be installed every week as usual

EWARS: early warning, alert and response system; WHO: World health Organization.

^a There are no licensing fees or other costs for using the EWARS in a Box application.

^b Cloud computing involves the sharing of computer resources at remote sites, usually over the internet.

Indicator-based surveillance

Between 7 March 2016 and 29 May 2016, 34 113 cases of the nine monitored syndromes were reported following 326 861 consultations (Table 2). Fig. 3 and Fig. 4 show the number of cases and Fig. 5 and Fig. 6 show proportional morbidity. The most frequently reported syndrome was influenza-like illness, which accounted for 48.2% of cases. Its incidence increased from the week beginning 28 March (Fig. 5) and corresponded with an outbreak of severe acute respiratory infection that was investigated by the health ministry.²⁶ In addition, indicator-based surveillance detected several clusters of watery diarrhoea and measles. Investigations into these clusters were not fully reported through EWARS (personal communication, subdivisional medical officer, 2016). Stakeholders reported that responses to disease outbreaks at divisional and subdivisional levels were better with automated alerts than in previous national emergencies.

Fig. 3.

Weekly cases of influenza-like illness, acute watery diarrhoea, prolonged fever and suspected dengue after Cyclone Winston, Fiji, 2016

Notes: The figure shows the data available on 31 May 2016. Data for the last week may not be complete due to delayed reporting. The dates represent the end of epidemiological weeks.

Fig. 4.

Weekly cases of acute jaundice syndrome, acute fever and rash, acute bloody diarrhoea, suspected meningitis and Zika-like illness after Cyclone Winston, Fiji, 2016

Notes: The figure shows the data available on 31 May 2016. Data for the last week may not be complete due to delayed reporting. Zika-like illness started to be monitored in the week ending 3 April 2016 during deployment of the early warning, alert and response system (EWARS in a Box). The dates represent the end of epidemiological weeks.

Fig. 5.

Trends in proportional morbidity due to influenza-like illness, acute watery diarrhoea, prolonged fever and suspected dengue after Cyclone Winston, Fiji, 2016

Notes: The figure shows the data available on 31 May 2016. Proportional morbidity was calculated from the total number of consultations in the week. The dates represent the end of epidemiological weeks.

Fig. 6.

Trends in proportional morbidity due to acute jaundice syndrome, acute fever and rash, acute bloody diarrhoea, suspected meningitis and Zika-like illness after Cyclone Winston, Fiji, 2016

Notes: The figure shows the data available on 31 May 2016. Proportional morbidity was calculated from the total number of consultations in the week. Zika-like illness started to be monitored in the week ending 3 April 2016 during deployment of the early warning, alert and response system (EWARS in a Box). The dates represent the end of epidemiological weeks.

Surveillance quality

The national average for the timeliness and completeness of reporting over the study period was 64% and 90%, respectively, but variations between administrative divisions were common (Fig. 7 and Fig. 8). Overall, 88% (21/24) of sites experienced at least one delay in reporting, most frequently due to a high workload (Table 4; available at: http://www.who.int/bulletin/volumes/97/3/18-211409). During the evaluation period, 325 alerts were generated through indicator-based surveillance, 88% (range: 52–100) of which were verified. Three of the five surveillance officers reported delays in the verification process, most commonly due to difficulty contacting the surveillance sites. Data validity was assessed using the complete patient registers that were available for the weeks from 28 March to 1 May 2016 at 3 of the 11 sites reviewed. There was a considerable variation in data collection and reporting practices between sites: different methods were used to record cases and case counts were totalled in different ways. This variability probably decreased data quality. The 34 EWARS surveillance sites (Fig. 1) were located in 16.2% of health-care facilities nationally (Table 5). The system was representative because it covered the cyclone’s trajectory and most of the severely affected population.

Fig. 7.

Timeliness of reporting of monitored syndromes after Cyclone Winston, by geographical location, Fiji, 2016

Notes: Cases were reported using the EWARS in a Box early warning, alert and response system (EWARS in a Box) at national and divisional levels. The nine syndromes monitored were: (i) acute fever and rash; (ii) prolonged fever; (iii) influenza-like illness; (iv) acute watery diarrhoea; (v) acute bloody diarrhoea; (vi) acute jaundice syndrome; (vii) suspected dengue; (viii) suspected meningitis; and (ix) Zika-like illness. Fiji is divided into four administrative divisions: Central, Western, Northern and Eastern. Timeliness is defined in Table 1. The dates represent the end of epidemiological weeks.

Fig. 8.

Completeness of reporting of monitored syndromes after Cyclone Winston, by geographical location, Fiji, 2016

Notes: Cases were reported using the EWARS in a Box early warning, alert and response system (EWARS in a Box) at national and divisional levels. The nine syndromes monitored were: (i) acute fever and rash; (ii) prolonged fever; (iii) influenza-like illness; (iv) acute watery diarrhoea; (v) acute bloody diarrhoea; (vi) acute jaundice syndrome; (vii) suspected dengue; (viii) suspected meningitis; and (ix) Zika-like illness. Fiji is divided into four administrative divisions: Central, Western, Northern and Eastern. Completeness is defined in Table 1. The dates represent the end of epidemiological weeks.

Table 4. User survey, EWARS in a Box, Fiji, 2016.

Survey question and responsesa	No. of usersb responding (%)c
Q. In which administrative division is the EWARS site located?
Central, no. (% of all surveillance sites in division)	12 (100)
Eastern, no. (% of all surveillance sites in division)	2 (100)
Northern, no. (% of all surveillance sites in division)	5 (50)
Western, no. (% of all surveillance sites in division)	8 (80)
Total, no. (% of all EWARS sites in Fiji)	27 (79)
Q. What do you think is the purpose of EWARS?d
Theme: outbreak detection	18 (67)
Theme: outbreak response	9 (33)
Theme: general disease surveillance or monitoring	12 (44)
Total respondents	27 (100)
Q. How well do you think EWARS is able to signal an early warning for potential disease outbreaks?
Very well	18 (67)
Somewhat well	7 (26)
Not very well	2 (7)
Not at all well	0 (0)
Total respondents	27 (100)
Q. Do you think EWARS has had any impact on public health in Fiji?
Yes	20 (77)
No	0 (0)
Unsure	6 (23)
Total respondents	26 (100)
Q. In your opinion, how easy is it to use EWARS on the mobile phone?
Very easy	24 (89)
Somewhat easy	3 (11)
Not very easy	0 (0)
Very difficult	0 (0)
Total respondents	27 (100)
Q. Have you ever had difficulty accessing EWARS on the mobile phone (e.g. application not working)?
Unsure	0 (0)
No	12 (44)
Yes	15 (56)
Total respondents	27 (100)
Q. How often did you experience difficulty accessing EWARS?e
Very often (most weeks)	2 (13)
Somewhat often (> once a month)	6 (40)
Not very often (≤ once a month)	3 (20)
Not often (≤ twice during the study period)	4 (27)
Total respondents	15 (100)
Q. At your health facility, what is the process used to record patients who meet the case definitions?
Medical officers record cases directly on the EWARS tally sheet at the time the patient is seen	4 (20)
Medical officers record cases on an EWARS line list	1 (5)
Weekly review of register or logbook by medical officer or nurse	13 (65)
Not known	0 (0)
Other	2 (10)
Total respondents	20 (100)
Q. How do you send/transmit the EWARS weekly reports?f
EWARS mobile phone application	25 (93)
EWARS website (using computer)	1 (4)
Email	4 (15)
Telephone call	8 (30)
SMS	6 (22)
Other	3 (11)
Total respondents	27 (100)
Q. What is your preferred reporting method?
EWARS mobile phone application	24 (92)
EWARS website (using computer)	0 (0)
Email	0 (0)
Telephone call	2 (8)
SMS	0 (0)
Total respondents	26 (100)
Q. Have there been situations where you could not submit the EWARS weekly report on time (i.e. before 6 pm Monday)?
Yes	21 (88)
No	3 (12)
Unsure	0 (0)
Total respondents	24 (100)
Q. What are the most common challenges for timely reporting?f
Tally sheet not received on time from other staff	11 (46)
No access to internet (no credit)	4 (17)
No access to internet (no signal)	11 (46)
No access to phone	2 (8)
Not enough time/workload too heavy	14 (58)
Unsure	0 (0)
Other	4 (17)
Total respondents	24 (100)
Q. Are you aware of the EWARS case definitions?
Yes	27 (100)
No	0 (0)
Unsure	0 (0)
Total respondents	27 (100)
Q. How easy is it to classify cases into the syndrome categories?
Very easy	11 (41)
Somewhat easy	16 (59)
Not very easy	0 (0)
Very difficult	0 (0)
Total respondents	27 (100)
Q. How easy was it to amend the reporting process when an additional syndrome (i.e. Zika-like illness) was added to EWARS?
Very easy	12 (44)
Somewhat easy	13 (48)
Not very easy	2 (7)
Not at all easy (very difficult)	0 (0)
Total respondents	27 (100)
Q. Why was it not easy?
Theme: similar case definitions and patient presentation	2 (100)
Total respondents	2 (100)
Q. Did you receive any feedback when an EWARS alert was generated for your health facility?
Yes	22 (81)
No	3 (11)
Unsure	1 (4)
Not applicable	1 (4)
Total respondents	27 (100)
Q. Do you receive the EWARS weekly bulletin?
Yes	19 (70)
No	7 (26)
Unsure	1 (4)
Total respondents	27 (100)
Q. How useful is the information in the EWARS weekly bulletin for your health facility?
Very useful	9 (33)
Somewhat useful	12 (44)
Not very useful	3 (11)
Not at all useful	3 (11)
Total respondents	27 (100)
Q. How have you used the information in the EWARS weekly bulletin?
Theme: information sharing	6 (32)
Theme: to compare with other reporting areas	5 (26)
Theme: to initiate preventive or responsive public health actions	5 (26)
Total respondents	27 (100)
Q. How could the EWARS weekly bulletin be improved?d
Theme: include health facility-specific surveillance data	3 (16)
Theme: include outcome of the previous week's case investigations	1 (5)
Theme: extend access to other staff members at the health facility	2 (11)
Total respondents	19 (100)
Q. Do you ever distribute the information in the weekly bulletin to other persons or organizations?
Yes	9 (35)
No	16 (62)
Unsure	1 (4)
Total respondents	26 (100)
Q. Who do you distribute the information to?f
Theme: health facility colleagues	8 (89)
Theme: community health-care workers	3 (33)
Theme: regional public health staff	1 (11)
Total respondents	9 (100)
Q. How satisfied do you feel with the training that you received when EWARS was implemented?
Very satisfied	11 (41)
Somewhat satisfied	12 (44)
Not very satisfied	3 (11)
Not at all satisfied	1 (4)
Total respondents	27 (100)
Q. How supported do you feel to be able to carry out your EWARS responsibilities?
Very supported	13 (48)
Somewhat supported	14 (52)
Not very supported	0 (0)
Not at all supported	0 (0)
Total respondents	27 (100)
Q. Overall, how satisfied are you with EWARS?
Very satisfied	16 (59)
Somewhat satisfied	11 (41)
Not very satisfied	0 (0)
Not at all satisfied	0 (0)
Total respondents	27 (100)

EWARS: early warning, alert and response system; Q: question; SMS: short message service; WHO: World health Organization.

^a For confidentiality reasons, qualitative data is not reported.

^b Users were people who submitted EWARS data (i.e. site focal points or individuals in charge of surveillance sites).

^c All values in the table represent the absolute number of users responding to the specific question and the percentage of the total responding to the question, unless otherwise stated.

^d This was an open question and answers were grouped thematically.

^e This question was asked to those respondents who answered yes to the preceding question.

^f Respondents were permitted to give more than one response to this question.

Table 5. Administrative divisions and surveillance sites, EWARS in a Box, Fiji, 2016.

Administrative division	Populationa	Health-care facilities	EWARS surveillance sites	Health-care facilities covered by EWARS sites
	No. (%)	No.	No. (%)	%
Central	361 895 (41.6)	52	12 (35.3)	23.1
Eastern	36 870 (4.2)	52	2 (5.9)	3.8
Western	344 663 (39.6)	61	10 (29.4)	16.4
Northern	127 556 (14.6)	45	10 (29.4)	22.2
Nationally	870 984 (100)	210	34 (100)	16.2

EWARS: early warning, alert and response system.

^a Projected population figures for 2016 were estimated by Fiji’s Ministry of Health and Medical Services using data from the 2007 census conducted by the Fiji Bureau of Statistics.

System performance

Stakeholders and 93% (25/27) of users who completed the cross-sectional surveys regarded EWARS as an effective early warning system (Table 4). In addition, 89% (24/27) of users thought the number and types of syndromes monitored were appropriate for the setting and 77% (21/27) regarded the weekly EWARS epidemiological bulletins as useful. In addition, national public health staff reported that the bulletins were helpful for providing updates and for coordinating public health responses. The bulletins also provided feedback on surveillance to EWARS sites.

Based on their experience with modifying surveillance to include Zika-like illness, system developers and surveillance officers reported that EWARS was flexible and could easily be modified. In addition, 93% (25/27) of users also thought the surveillance process was easy to modify. The smartphone reporting system was highly acceptable and most EWARS users found that the smartphone application was simple and very easy to use (89%; 24/27). Most preferred it to email or telephone communication (92%; 24/26). However, 58% (14/24) of users stated that the reporting process substantially increased their workload. Most users (81%; 22/27) received feedback from surveillance officers after an alert was generated by their site and 70% (19/27) received weekly epidemiological bulletins. Four of the five surveillance officers found the EWARS website simple to navigate and easy to use.

The system was stable: the EWARS server in Fiji required little maintenance and there were no outages of the whole system. Although most users (56%; 15/27) reported some difficulty accessing EWARS via the smartphone application or website, these problems occurred in the early phase and were quickly resolved by system developers.

Event-based surveillance

All 10 alerts triggered through event-based surveillance were verified. Four were confirmed outbreaks: (i) two were large outbreaks of viral conjunctivitis (880 cases combined); (ii) one was a typhoid outbreak (4 cases); and (iii) one was a cluster of prolonged fever (13 cases) whose etiology was not identified. The time between an event being reported and the start of the public health response ranged from 0 to 4 days. All stakeholders and users thought event-based surveillance was useful but underused. Overall, 59% (16/27) of users indicated they would directly contact the medical officer in their administrative division (i.e. the most senior, local member of the health ministry) on encountering an unusual public health event, whereas only 11% (3/27) indicated they would report the event using the EWARS toll-free number.

Costs

During the evaluation period, the total direct costs associated with the implementation and operation of EWARS was approximately 185 000 United States dollars (US$), which did not include the salaries of repurposed health ministry staff (e.g. EWARS users who performed surveillance in addition to their routine activities) or repurposed WHO staff. Equipment costs were relatively small: US$ 7500 for smartphones and US$ 13 450 for laptops for surveillance officers. Approximately US$ 95 000 was spent on the fees and travel costs of consultants with expertise in infectious disease surveillance and response who supported the implementation and operation of the system and provided training. The consultants were located in Western, Northern and Central Division and Eastern Division was supported by consultants based in Central Division. Other expenditure totalled US$ 70 400, which included US$ 2000 for mobile phone connection costs, US$ 14 000 for surveillance officers’ salaries, US$ 12 000 for staff travel within Fiji and US$ 42 000 for contractual services. The health ministry contributed 3.5% of total expenditure, Fiji’s Health Sector Services Program contributed 7.5% and WHO provided the remainder.

Discussion

Our study of the implementation of EWARS in a Box in Fiji following Cyclone Winston in 2016 found that the system was well regarded across all ten quality attributes (Table 1) assessed. During the 12-week study period, data from more than  326 000 consultations (including 34 113 with patients who had a syndrome being monitored) were processed, 325 alerts were generated and three large outbreaks (i.e. influenza, conjunctivitis and typhoid) that required public health interventions were identified. No large outbreak was missed. Moreover, it was clear that the system could be rapidly deployed during a health emergency. Although mobile phones have been used during surveillance in several settings (e.g. after the 2008 Sichuan earthquake in China),^27,28 the use of highly automated surveillance systems during a natural disaster is relatively new and unusual.²⁹ The main lessons learnt are presented in Box 1.

Box 1. Lessons learnt in implementing EWARS in a Box after Cyclone Winston, Fiji, 2016.

• Syndromes for surveillance should be selected using rapid risk assessments before EWARS implementation and, where possible, thresholds for case numbers should be set using baseline epidemiological data.

• All EWARS elements should be designed and customized to suit the needs of the country and health emergency.

• Event-based surveillance compliments standard indicator-based surveillance and is useful for outbreak detection.

• The establishment of a local public health workforce should be prioritized and its ability to respond to future health emergencies and outbreaks should be maintained by providing ongoing technical support for surveillance site focal points^a and surveillance officers.

• Surveillance training should follow the train-the-trainer format and should emphasize understanding surveillance data, alert verification and investigating disease outbreaks.

• Strong leadership is critical during a public health emergency to coordinate surveillance and outbreak responses because good surveillance depends on effective interactions between all involved.

A unique and important feature of EWARS that could not be quantified was its automation of data analysis, alert-generation and the distribution of weekly bulletins. Most traditional systems rely on spreadsheet software, emails and telephone communication for data analysis and dissemination, which is time-consuming. Surveillance teams may have little time left for verifying alerts, making rapid risk assessments or investigating outbreaks. In Fiji, EWARS saved human resources, minimized human error and ensured surveillance teams could focus on data collection, management and responding to alerts.^3,6

Timely data reporting is crucial for rapid outbreak detection and public health responses.²² We observed few delays, most of which were associated with an increased workload at surveillance sites. Variations in reporting quality were mainly attributed to the size of the health-care facility and to staff motivation, training and supervision. On rare occasions, delays were due to a lack of mobile phone credit or to a poor connection. There is an ongoing need for technical support and feedback for everyone involved in surveillance activities.²²

In Fiji, we found that event-based surveillance was important for detecting disease outbreaks: it was able to detect an outbreak of conjunctivitis that was not captured by indicator-based surveillance. Moreover, as previously reported, event-based surveillance is often simpler and faster to implement than indicator-based surveillance.³⁰ In Fiji, only health-care workers were permitted to report events, which we believe minimized the false-positive alerts that can be generated by reports from community members.

Our evaluation of EWARS in a Box had some limitations. As system implementation relied on access to affected areas and functional telecommunications, some remote locations (e.g. islands in Eastern Division) were probably missed. In addition, there were some discrepancies between EWARS data and patient registry data, these discrepancies were similar to those observed in previous evaluations of syndromic surveillance systems.^3,15,24,25 They could be minimized by ensuring that data are reported and case definitions are applied consistently, by classifying cases using automated symptom-based algorithms and by improving links to surveillance laboratories.^25,31,32 In addition, better access to point-of-care tests could improve diagnostic accuracy.¹ Another limitation is that surveillance officers from the health ministry helped design the cross-sectional surveys and collected responses, which may have influenced how EWARS users responded. However, as the surveys were administered using SurveyMonkey and data were anonymized, we believe there was little reporting bias.³³ Further, in the absence of a gold-standard method of detecting outbreaks, we had to rely on proxy measures of accuracy, such as the timeliness and completeness of reporting. We were unable to compare the effectiveness of EWARS with the pre-existing surveillance system, because EWARS was implemented throughout most of the area affected by the cyclone. Comparisons of EWARS with routine surveillance should be considered and would be feasible in large, chronic or complex emergencies in which EWARS is implemented in stages.

Since its implementation in Fiji, EWARS in a Box has been deployed in Nigeria (2016), Ethiopia (2016), Cox’s Bazar, Bangladesh (2017) and the Democratic Republic of the Congo (2018). The system has been modified to incorporate laboratory surveillance, geospatial mapping of cases and case-based reporting during outbreaks. In Fiji, EWARS continued to operate at the 34 surveillance sites 3 years after Cyclone Winston. As a result, the country’s epidemic surveillance system has been strengthened and tools, infrastructure and knowledge are available for emergencies in the future.

The early detection of disease outbreaks is particularly important and difficult during humanitarian emergencies, especially in developing counties. Tools that facilitate and standardize early warning surveillance after disasters, such as EWARS in a Box, can improve the detection of disease outbreaks while minimizing the reporting burden on the public health system. In Fiji, this system strengthened disease surveillance during a national emergency and was well regarded by users. Emergency early warning systems should be incorporated into routine national surveillance systems (Table 3) to strengthen them and expand their capabilities.^6,9