Abstract
Background
It is a challenge in low-resource settings to ensure the availability of complete, timely disease surveillance information. Smartphone applications (apps) have the potential to enhance surveillance data transmission.
Methods
The Central African Republic (CAR) Ministry of Health and Médecins Sans Frontières (MSF) conducted a 15-week pilot project to test a disease surveillance app, Argus, for 20 conditions in 21 health centers in Mambéré Kadéi district (MK 2016). Results were compared to the usual paper-based surveillance in MK the year prior (MK 2015) and simultaneously in an adjacent health district, Nana-Mambére (NM 2016). Wilcoxon rank sum and Kaplan-Meier analyses compared report completeness and timeliness; the cost of the app, and users’ perceptions of its usability were assessed.
Results
Two hundred seventy-one weekly reports sent by app identified 3403 cases and 63 deaths; 15 alerts identified 28 cases and 4 deaths. Median completeness (IQR) for MK 2016, 81% (81–86%), was significantly higher than in MK 2015 (31% (24–36%)), and NM 2016 (52% (48–57)) (p < 0.01). Median timeliness (IQR) for MK 2016, 50% (39–57%) was also higher than in MK 2015, 19% (19–24%), and NM 2016 29% (24–36%) (p < 0.01). Kaplan-Meier Survival Analysis showed a significant progressive reduction in the time taken to transmit reports over the 15-week period (p < 0.01). Users ranked the app’s usability as greater than 4/5 on all dimensions. The total cost of the 15-week pilot project was US$40,575. It is estimated that to maintain the app in the 21 health facilities of MK will cost approximately US$18,800 in communication fees per year.
Conclusions
The app-based data transmission system more than doubled the completeness and timeliness of disease surveillance reports. This simple, low-cost intervention may permit the early detection of disease outbreaks in similar low-resource settings elsewhere.
Keywords: Surveillance, mHealth, Limited resource settings, Innovation and health
Background
Since 1998, the World Health Organization (WHO) Regional Office for Africa and Member States have adopted the Integrated Disease Surveillance and Response (IDSR) strategy [1]. The Technical Guidelines to support this strategy emphasize the identification of priority diseases using standardized case definitions, reporting mechanisms, epidemiological data analysis and field investigation, outbreak response, communication and feedback [1]. Completeness and timeliness of disease reporting are essential attributes of an effective disease surveillance system [1–3]. Yet, in low-resource settings it is challenging for health facilities to deliver paper-based surveillance reports to national authorities in a timely manner [4, 5]. Smartphone applications (known as apps) have been developed to support data transmission between health facilities and district offices in a number of low-income countries, including several in sub-Saharan Africa [2, 6, 7]. Apps have the potential to improve surveillance [8, 9] and to hasten control of potential epidemics. To our knowledge, apps have been used to collect and transmit data on a limited number of diseases [3, 5, 7] or in an emergency context [10, 11] but not on a large portfolio of health conditions in a context of post-conflict insecurity. This report describes the implementation and evaluation of an app surveillance system for the notification of Alerts and transmission of Weekly Reports on 20 conditions from 21 health facilities in the southwestern part of Central African Republic (CAR), in a pilot study entitled “Projet d’Alerte Précoce” (PAP; Early Warning Project, in French).
Study context and objectives
The district of Mambéré Kadéi (MK) is the second largest health district in CAR with an area of 30,203 km2 and estimated population of 460,000. For the purpose of this pilot study, the Ministry of Health (MOH) identified 21 ‘sentinel’ health facilities (3 hospitals, 18 health centers) as these were the principal facilities still functioning within the district and represented all seven sub-districts. Eleven of the health facilities were located in the sub-district of Berberati, two in each of the sub-districts of Gadzi, Gamboula and Amada Gaza, and one in each the sub-districts of Carnot, Dédé Makouba and Sosso Nakombo. The district health office, which includes the office of the Director of Surveillance, is located in the city of Berbérati. Travel time by car or motorcycle from the health facilities to the district health office ranges from 25 min for the closest facility to 9 h for the most distant. Communities in the district are connected by earth roads but lack public systems of water, sewer and electricity. Cellular network service in the district is provided by three telecommunications operators, while villages located on the border with Cameroon are covered by two Cameroonian operators. Only a basic Global System for Mobile communication (GSM) telecommunication service is available.
Health facilities in CAR are expected to submit disease surveillance reports to the district health office according to the national protocol: diseases requiring immediate notification (Alerts) within 24 h; a summary of all notifiable diseases by 5:00 pm Wednesday each week. Both reports are normally transmitted on standard paper forms, although some staff in the health facilities use their own mobile phones and telephone credit to do so.
In 2013–14 CAR experienced a period of violent conflict between religious and ethnic communities which compromised the health care system, including the disease surveillance system as shown by changes in the level of completeness of disease reporting from 2011 to 2015 (Fig. 1). In some districts of the country, that conflict continues to the present day, while in other districts, such as Mambéré-Kadéi (MK), tensions persist, but only scattered episodes of violence have occurred. During the period of the PAP study (2015–2016), differences in the level of security within the district of MK due to religious violence and non-targeted armed conflict were modest: higher in the more distant sub-districts of Gadzi and Amada Gaza (European Interagency Security Forum (EISF) risk level ‘medium’) and lower in those districts near to the district capital of Bérbérati (EISF risk level ‘low’) [12].
Fig. 1.
Overview of the Completeness (%)* of Weekly Reports at the District of Mambere-kadei during years 2011 through 2015 [28]. *According to WH0, the accepted cutoff point of completeness (%) is 80% [16]; w = week
Shortly after the onset of the unrest, Médecins Sans Frontières (MSF) deployed its mission in CAR and in 2015 the MOH and MSF explored how to support the disrupted national disease surveillance system. Due to movement of displaced populations, there was considerable risk of outbreaks of epidemic and vaccine-preventable diseases [13]. To accelerate data transmission and a public health response, MOH and MSF proposed a pilot project using an app reporting system in the district of MK. The main objectives were: 1) to improve the completeness and timeliness of disease surveillance information reaching the district health office, and 2) to evaluate the use of the app to enhance data transmission between the health facilities and the district health office.
Methods
The software
The disease surveillance app, Argus, has been developed and made available by the Support to National Surveillance team in the World Health Organization (WHO) Office in Lyon, France [14]. Argus is an integrated solution that comprises a smartphone app and complementary laptop ‘server’ program that permits the reception, validation, analysis and presentation of data in a dashboard format.
The Argus app performs three operations: i) Alert, to provide an immediate notification of cases that may signal a potential outbreak; ii) Report, to transmit Weekly and Monthly surveillance reports, and iii) Archive, to review the status of previously submitted reports. Although the app functions optimally with a 3G cellular telephone network or greater, it requires only a GSM network to transmit basic disease data using a series of Short Message Service (SMS). For an Alert, the numbers of cases, hospitalized patients and deaths are submitted in an open text box format. In the Weekly Reports, the number of cases and deaths for each notifiable disease are submitted as a single SMS; hence a complete report requires a total of 20 SMS messages. Data transmission of the SMS messages occurs through a Universal Serial Bus (USB) modem connected to the laptop computer server maintained by the Director of Surveillance of the health district. The modem contains a Subscriber Identity Module (SIM) card that receives and transfers SMSs into a MySQL database, using Frontline SMS software [15]. After having submitting data to the server, users receive an acknowledgment on their smartphones for each disease notification successfully transmitted. In case of failure in delivery to the server, the app displays an error message on their smartphone screen. In this situation users are required to re-submit the messages.
The main page of the app (Fig. 2) displays four tabs: to activate the Alert, Weekly Report, Monthly Report, and Archive functions; the Monthly Report function was not utilized in this pilot. The electronic format of the Alert and Report functions was developed to mirror the existing paper-based disease surveillance report forms in French and designed with a graphical user interface to facilitate data entry, reduce typing, and ensure zero-reporting (Fig. 2b) [16]. Once data has been entered in all 40 required fields (case and death numbers for 20 conditions), users have simply to touch the Submit Report tab (paper airplane icon), and the notifications are sent in the form of 20 individual SMS, in a predetermined standardized format using a comma separator to the server which visually displays the data on a dashboard (Fig. 3). Only reports with data entered in all fields (including the reporting of zeros) are able to be sent by the app. All Alerts and Weekly Reports were reviewed by the Director of Surveillance as soon after receipt as possible. Weekly reports were considered: i) Complete if case and death numbers for all 20 conditions were received in the server; ii) Incomplete when some, but not all, of the information on the 20 conditions were received by the server; iii) Rejected if the Director of Surveillance identified an error in the reports after reviewing them and checking the details of suspected cases with the respective health center by means of an open text SMS or telephone call; or iv) Validated when the content of completed reports was deemed to be correct by the Director of Surveillance. With respect to Alerts, the Director of Surveillance investigated the reported cases by telephone with the reporting health facility, however validation of Alerts is not a step included in the app.
Fig. 2.
a Screenshot of the Main Page of the Argus android phone app. The boxes, explained clockwise, from top left-hand side: a) Tab to submit Weekly Reports; b) Tab to submit Monthly Reports, (not activated in this pilot study; c) Tab to submit Alerts; d) Archive button to review previously submitted reports and Alerts. b Screenshot of the Report function of the Argus android phone app From top to bottom: the epidemiological week, list of diseases, and the Submit (paper airplane icon)
Fig. 3.
Screenshot of GIS map function of Argus server dashboard showing the district of Mambéré-Kadéi, geographical locations of the health facilities, sub-districts, and cases of acute severe malnutrition during the pilot period
Materials and equipment
The smartphones were Android-based, loaded with the Argus app [17]. Technical support was provided by WHO, Lyon [14, 18], the Geographic Information System (GIS) Unit of MSF and an IT consulting firm based in Geneva [19]. This support comprised programming the app to specify CAR health facilities, telephone and SIM card numbers, diseases under surveillance, and the resolution of periodic technical problems. One full-time local staff member was hired and trained by MSF to provide IT support and to train the Director of Surveillance of the health district to manage the server.
A single smartphone was distributed to each surveillance ‘focal point’ in each participating health facility and a laptop server, together with a backup laptop server, was located in the district health office in Berbérati. Solar energy panels were installed in all health facilities and the district health office to ensure the permanence of power for the mobile telephones and the central server. A contract for a ‘fleet’ of telephone numbers was made with each of the three cellular network providers of service in the district to permit free voice communication between focal points and the Director of Surveillance. The cost of submission of the reports by SMS was covered by prepaid credit with each of the three providers. Two health centers used prepaid Cameroon cellular networks to submit their reports. Finally, in three health centers without mobile telephone network coverage, the cost of motorcycle taxi transport was covered to permit the focal points of these centers to travel weekly to a neighboring village to access the mobile phone network and send their Weekly Report, or to do so at any time to send an Alert.
Training participants
An initial four-day training workshop on the principles and practice of surveillance was conducted for the surveillance focal points (October 2015). The app was then tested for functionality, using fictitious data, in six health centers in the city of Berbérati (December 2016). Prior to launching the pilot study, the 21 focal points participated in a two-day practical training workshop on electronic data transmission using the app, then received a two-day booster training at week four of the pilot (epidemiological week 14). The booster-training workshop permitted troubleshooting of technical problems and included an agreement by focal points to begin transmission of Weekly Reports early (48 h before the submission deadline). The Director of Surveillance of the health district received training and supervision on the use of Argus on the laptop server for 3 hours weekly during a period of 3 months (March – May 2016). The 15-week pilot study was implemented from 6 March to 11 June, 2016 (epidemiological weeks 9–23).
Data analysis and study outcomes
We conducted a set of analyses for completeness and timeliness of reporting, duration of transmission, and usability of the app (see definitions in Box) [1]. First, we compared these parameters for MK during the pilot period (epidemiological weeks 9–23, 2016), here termed ‘MK 2016’, with the data for the same district in the same time period the year prior when standard paper-based reporting had been used (epidemiological weeks 9–23, 2015), termed ‘MK 2015’. Second, we compared the data from MK 2016 to that from the district of Nana-Mambéré (NM) during the same time period (epidemiological weeks 9–23, 2016), termed ‘NM 2016’. Wilcoxon rank sum test was used to compare data completeness and timeliness for MK 2016 with MK 2015 and NM 2016. As the surveillance focal points were absent from their health facilities for the booster-training workshop in Berberati during epidemiological week 14, they did not submit their Weekly Reports until returning the following week. Therefore completeness, but not timeliness of reports, was calculated for week 14. For analysis, the 15-week period was divided into five three-week periods and the distribution of transmission times for these periods compared using a Kaplan-Meier survival analysis (Stata/SE Version 12.0 stcox Efron procedure) [20].
The cost of the project was calculated as the total value of expenses [21], including equipment, telephone fleet services, SMS message charges, human resources and Information Technology (IT) consultation services.
The comparison district, NM, is a district adjacent to MK, with an area nearly equivalent to MK (26,600 km2 versus 30,203 km2) and half the population of MK (N = 233,666 persons). Both districts share a comparable epidemiological profile and use similar surveillance reporting procedures, however the app was only piloted in MK.
Results
A total of 3403 cases and 63 deaths were notified in the Weekly Reports during the 15-week period (Table 1). The three most frequent conditions reported were: Severe Acute Malnutrition (30%), Moderate Acute Malnutrition (25%) and Severe Acute Respiratory Infection (17%). The three most frequent causes of death were: Maternal Mortality (33%), Severe Acute Malnutrition (21%) and Severe Acute Respiratory Infection (19%). A total of 26 cases and 4 deaths were notified as Alerts.
Table 1.
Diseases under surveillance, cases and deaths reported in weekly reports and alerts during the 15-week pilot period, Mambéré Kadéi district 2016
| Diseases (alphabetical order) | Weekly Reports | Alerts | ||||||
|---|---|---|---|---|---|---|---|---|
| Cases | Deaths | Cases | Deaths | |||||
| N | % | N | % | N | % | N | % | |
| Acute Flaccid Paralysis | 5 | 0.1% | 0 | 0 | 2 | 7.7% | 0 | 0 |
| Adverse Events Following Immunization | 50 | 1.5% | 0 | 0 | 1 | 3.6% | 0 | 0 |
| Bacillary Dysentery | 66 | 1.9% | 0 | 0 | – | – | – | – |
| Cholera | 0 | 0 | 0 | 0 | – | – | – | – |
| Chronic Malnutrition | 29 | 0.9% | 0 | 0 | – | – | – | – |
| Diphtheria | 1 | < 0.1% | 0 | 0 | – | – | – | – |
| Dracunculiasis | 0 | 0 | 0 | 0 | – | – | – | – |
| Hemorrhagic Fever | 0 | 0 | 0 | 0 | – | – | – | – |
| Influenza | 215 | 6.3% | 1 | 1.6% | – | – | – | – |
| Malnutrition | – | – | – | – | 8 | 31% | 0 | 0 |
| Maternal Mortality | – | – | 21 | 33.3% | – | – | 2 | 50% |
| Maternal/Neonatal Tetanus (MNT) | 21 | 0.6% | 9 | 14.3% | 6 | 23% | 2 | 50% |
| Measles | 25 | 0.7% | 0 | 0 | 2 | 7.7% | 0 | 0 |
| Meningococcal Meningitis | 78 | 2.3% | 5 | 7.9% | 4 | 15.4% | 0 | 0 |
| Moderate Acute Malnutrition | 855 | 25.1% | 0 | 0 | – | – | – | – |
| Rabies | 12 | 0.4% | 1 | 1.6% | 3 | 11.5% | 0 | 0 |
| Severe Acute Malnutrition | 1018 | 29.9% | 13 | 20.6% | – | – | – | – |
| Severe Acute Respiratory Infection | 567 | 16.7% | 12 | 19% | – | – | – | – |
| Severe Acute Respiratory Syndrome (SARS) | – | – | – | – | – | – | – | – |
| Typhoid Fever | 448 | 13.2% | 1 | 1.6% | – | – | – | – |
| Whooping Cough | 10 | 0.3% | 0 | 0 | – | – | – | – |
| Yellow Fever | 3 | 0.1% | 0 | 0 | – | – | – | – |
| TOTAL | 3403 | 100% | 63 | 100% | 26 | 100% | 4 | 100% |
*A total of 271 Weekly Reports and 15 Alerts were received during the 15-week pilot period.The sign’ –‘indicates that no data was reported for this condition, whereas the value ‘0′ indicates there were no reported cases of this condition, in compliance with the zero reporting policy of WHO
Over the 15-week period, a total of 530 Weekly Reports and 15 Alerts (together comprising 8378 SMS messages) were received by the server. Of these 530 weekly reports, 271reports (51%) (comprising 5420 SMS) were complete and validated; 71 (13%) were incomplete, and 188 (36%) were rejected. Incomplete and rejected reports were corrected and re-sent by the focal points, sometimes multiple times, thereby increasing the time to report completion; reports would be considered complete and validated only after review and approval by the Director of Surveillance.
Overall, the median completeness of Weekly Reports was significantly higher in MK 2016 (81% (IQR 81–86%)) than in MK 2015 (29% (IQR 24–36%)) and NM 2016 (52% (IQR 48–57%)) (p < 0.01; Fig. 4a). Similarly, the median timeliness of complete reports was significantly higher in MK 2016 (50% (IQR 39–57%)), than in MK 2015 (19% (IQR 19–24%)) and NM 2016 (29% (IQR 24–36%)) (p < 0.001; Fig. 4b). The median duration of transmission of complete reports was 12 h (IQR 1–63 h). If one were to consider timeliness as the receipt of the first SMS transmission of a Weekly Report, the timeliness in MK 2016 would be 74% (IQR 60–80%). Figure 5 shows the Kaplan-Meier survival analysis for time to transmit a complete Weekly Report grouped by 3-week time periods during the pilot. There was a significant progressive reduction in transmission time over the 15-week period (p < 0.01).
Fig. 4.
a Completeness (%) of Weekly Reports in Mambéré-Kadéi 2016 (PAP), versus paper surveillance in Mambéré-Kadéï 2015 and in the comparison district, Nana-Mambéré 2016. b Timeliness (%) of Weekly Reports in Mambéré-Kadéï 2016 (PAP) versus paper-based surveillance in Mambéré-Kadéi 2015 and in the comparison district Nana-Mambéré 2016. * p value for both figures was < 0.01 and it was calculated using the Wilcoxon sum test to compare the completeness of Mambéré-Kadéi 2016 (PAP) with the paper surveillance in the districts of Mambéré-Kadéï 2015 and Nana-Mambére 2016
Fig. 5.
Kaplan-Meier Survival Analysis for the duration of transmission for complete Weekly Reports by 3 week period
Consistently each week, health centers across the district initiated electronic transmission of the Weekly Reports as required by the study protocol (96% of occasions); notably, sub-districts showed no significant difference in this regard (Table 2). Yet, completed transmission and report timeliness was significantly lower in a number of sub-districts in both centrally-located low-risk security zones, and in more distant medium-risk security zones. The mean scoring of the usability of the app reported by the 21 focal points was 4.5–5/5 for all 27 questions on the three dimensions studied: ease of use, usefulness and efficiency (data not shown). The total cost for the 15-week pilot was US$40,575. This was composed of i) US$10,950 in capital costs: 30 smartphones (US$2250), 30 solar chargers (US$2700), and two laptop computers (US$6000)); ii) US$5200 communication fees (US$1800 fleet services, US$3400 SMS charges); iii) US$225 transportation costs in three isolated health facilities iv) US$4100 local personnel, and v) IT consultant services based in Geneva (US$20,100). It is estimated that to maintain the system in the 21 health facilities of MK will cost approximately US$18,800 per year in communication fees; this represents US$17.20 per health facility per week.
Table 2.
Level of Weekly Report Completeness and Timeliness by Sub-district
| Sub-district | Distance (Km) from district capital (Travel time by motorbike, hours) |
Level of security risk (EISF) | Completeness | ||
|---|---|---|---|---|---|
| Transmission initiated (%)a | Transmission complete (%)b | Timeliness (%)c | |||
| Berberati | 5 Km (0.1 h) |
Low | 153/154 (99.3%) | 160/165 (97%) | 126/154 (81.8%) |
| Sosso-Nakombo | 58 Km (1–2 h) |
Low | 14/15 (93.3%) | 10/15 (66.7%) | 5/14 (35.7%) |
| Dede-Makouba | 71 Km (2–5 h) |
Low | 29/30 (93.3%) | 17/30 (56.7%) | 12/28 (42.9%) |
| Gamboula | 86 Km (2–5 h) |
Low | 30/30 (100%) | 26/30 (86.7%) | 11/28 (39.3%) |
| Carnot | 76 Km (2–5 h) |
Low | 15/15 (100%) | 14/15 (93.3%) | 11/14 (78.6%) |
| Gadzi | 206 Km (5–10 h) |
Medium | 29/30 (96.7%) | 22/30 (73.3%) | 16/28 (57.1%) |
| Amada-Gaza | 179 Km (5–10 h) |
Medium | 21/30 (70%) | 13/30 (43.3%) | 13/28 (46.4%) |
| p value | p = 0.72 | p < 0.01 | p < 0.01 | ||
aThe proportion of transmission initiated = Total reports partially or fully received × 100 / total N reports expected from sub-districts)
bThe proportion of transmission complete = Total reports fully received × 100 / total N reports expected from sub-districts)
cAs the surveillance focal points were absent from their health facilities for the booster-training workshop in Berberati during epidemiological week 14, they did not submit their Weekly Reports until returning the following week. Therefore completeness, but not timeliness of reports, was calculated for week 14
Discussion
The objective of the pilot project was to enhance the quality of an integrated strategy for disease surveillance [9] through improved reporting using smartphones. In comparison with paper-based surveillance in both the pilot district the year prior and in a nearby comparison district during the same time period, median completeness and timeliness of reporting from the health facilities more than doubled to 81% and 50%, respectively. Timeliness, itself, progressively improved over the study period.
Mobile phone-based surveillance systems have been pilot tested in several contexts in the past decade. General syndromic surveillance projects in Papua New Guinea [22], Cambodia [2], and Madagascar [23] have produced moderate improvements in the completeness and timeliness of reporting, e.g. an increase in completeness from 40% prior to 70% during the pilot in Papua New Guinea. Targeted surveillance programs for malaria [24], rabies [3], and influenza in Uganda, Tanzania, and Kenya [7], respectively, produced considerable improvements in report timeliness, eg. a reduction in the median reporting delay for influenza in Kenya from 21 to 7 days. Mobile technology appears also to have enhanced surveillance in emergency settings, such as following the earthquakes in Sichuan, China [10] and Haiti [11]. However, larger, comprehensive mobile phone-based surveillance systems have not been assessed in post-conflict settings with ongoing insecurity. This project assesses the impact, feasibility, and cost of using mobile technology for the surveillance of a broad range of notifiable conditions using a simple SMS-based application in a post-conflict setting. Our study shows that, within weeks, the completeness and timeliness of weekly surveillance reports can be considerably improved, and the improvement maintained, despite the remoteness of the healthcare centers and primitive road conditions. Although sub-districts were highly compliant in submitting weekly reports on their mobile devices, the proportion whose transmissions were successfully completed differed significantly (Table 2). Of the two distant sub-districts with the medium-level security risk (Gadzi and Amada-Gaza), one managed a high level of completeness (Gadzi 73.3%) relative to the other (Amada-Gaza 43%). Conversely, two sub-districts in areas of low security risk near the capital (Sosso-Nakombo, Dede-Makouba) managed only 66.7% and 56.7% levels of completeness, respectively. The consistently high level of submission of weekly reports from all sub-districts suggests that remoteness and level of insecurity did not constrain the use of the technology in the health centers. Rather, we suspect that the differences seen between sub-districts in report completeness and timeliness may relate more to the quality of the communications network, the aptitude of the participating staff and, in several settings, the need for motorcycle taxi transport to a nearby community to access a telecommunications network.
The electronic reporting system in the present project was rated as highly usable by participating staff. This may reflect a readiness of the staff to adopt new technology for digital surveillance, however we do not have baseline information with which to measure changes in perceptions or technical skills. The cost of the pilot was higher than mobile disease surveillance interventions reported in other contexts [6]. This is due to the necessary use of smartphones, the provision of these devices to all health facilities, the purchase of ‘fleet’ services from multiple cellular network providers, the costs of SMS transmission, the adherence to the ‘zero reporting’ protocol on a relatively large number of diseases (N = 20), and the need for considerable IT consultant services in the developmental stage. The projection of local operating costs beyond the pilot stage, US$17.20 per health facility per week, includes communication fees, but does not estimate the cost of ongoing technical support and staff training. Although still high, this cost is more comparable to that reported elsewhere [2, 3, 7, 22–24].
Several challenges were encountered in this pilot study that require consideration. Smartphones were used in this project to permit use of the app with a user-friendly interface and its associated convenient server dashboard summary of the surveillance data. However, smartphones are not yet commonly used in most sub-Saharan African countries [25]. This necessitated the provision of smartphones to each participating health facility, several days of training prior to the pilot and a booster training at 4 weeks to ensure correct use of the app by healthcare personnel.
The smartphones enabled not only electronic data transmission from health facilities to the district health office via SMS, but since they were configured as a ‘fleet’, telephone contact between the sites was possible to correct errors, provide feedback, and further delineate the epidemiological features of reported cases as needed. The negotiation of ‘fleet’ services with multiple mobile phone service providers proved cumbersome, however, and the project may have been better served at lower cost by the use of a toll-free number to which reports could be sent and calls directed [24].
All submitted Weekly Reports were reviewed and required validation by the Director of Surveillance before entering the server database. Through telephone or SMS contact with the focal points, the Director notified health facilities of incomplete reports that were thereafter re-sent, and investigated details of reports that were rejected. Since reports could only be transmitted once all fields were complete, reports identified in the pilot as ‘incomplete’ were most often incomplete as a result of technical problems during the transmission process. Reports were ‘rejected’ most often as a result of evident data entry errors; these errors were corrected by the focal points and the report resubmitted.
The time taken to transmit complete Weekly Reports in this project was substantial (median 12 h) and highly variable (IQR 1–63). This was due to several factors: i) the poor quality and unreliable nature of GSM telephone network connectivity in the district; ii) the reception of transmissions by a single SIM card of the health district server; iii) the arrival of the bulk of transmission in the few hours before the reports were due. Variation in the proportion of reports that were timely, complete and validated did not differ significantly according to the distance of the healthcare centre from the district office and the associated degree of insecurity, but rather appeared related more to the quality of the communications network and aptitude of the participating staff. It is clear from the progressive reduction in transmission time over the 15-week period that staff quickly became more adept at sending complete, correct reports, and submitting them adequately in advance of the deadline. Participating staff reported developing a sense of comfort and ownership of the electronic surveillance system [26], and could become the future trainers for the implementation of this surveillance program in other districts, should the Ministry of Health proceed to scale-up this project to the national level.
As only a GSM network was available in most of the district, feedback to health facilities on epidemiological trends and issues in the district, and comparison of their performance with that of their peers, could not efficiently be given by such means of an electronic bulletin. Further experimentation is needed to identify a suitable means to do so.
A number of limitations of this work should be recognized. This pilot project focused only on electronic transmission of data from health facilities to the district office; there was no extension to the national level, although the app provides such functionality. Surveillance data that were aggregated and analyzed at the health district office were transmitted verbally by mobile phone to the national office. Clearly, smooth electronic integration of the district program with the national surveillance and health information systems will be vital in the future. We have assessed the impact of smartphones on the quality of disease reporting in a district disease surveillance program. No attempt was made to conduct a comprehensive evaluation of the technology’s effects on the broader dimensions of the IDSR system. According to the national IDSR protocol, cases and deaths of most of the listed conditions are to be notified immediately as Alerts as well as in a summative manner in the Weekly Reports, however, it is apparent from Table 1 that the Alert function is underused. Clearly more intensive training and supervision is required to ensure an immediate reporting of these conditions so as to enable a timely public health response. Comparison was made of the completeness and timeliness of reporting in the district of MK during the pilot period (MK 2016) with that observed in the district during the same time period the previous year (MK 2015). However, the poorer quality of reporting in 2015 may have reflected not only the limitations of a paper-based system, but also the residual effects of civil unrest on the functioning of health facilities in the district at that time. Only a simple cost analysis is presented; a full cost comparison and cost-benefit analysis merits future study. Usability of the system was measured with a short semi-quantitative questionnaire. The responses of the focal points show little variability and the app were ranked highly on all dimensions. It is possible that the instrument used was not adequately sensitive and/or that participants chose to provide a positively biased evaluation of the app to the investigators.
The project was of a limited 15-week duration in a post-conflict setting with only a modest degree of insecurity. Generalization of these results to a setting of more active conflict and greater insecurity should be done cautiously. Finally, it is recognized that sustainability of this pilot initiative and its scale-up to a national level are challenging [8, 27] and will require long-term political commitment, training and resources.
Conclusion
This SMS-based electronic data transmission system demonstrated a significant improvement in the completeness and timeliness of disease surveillance reports in a remote setting affected by recent conflict, and appears to perform equally well in low and medium risk security zones. This relatively simple intervention may permit the early detection of disease outbreaks in similar settings elsewhere.
Acknowledgments
We thank the staff of the participating health centers, district health office and national Ministry of Health, WHO and MSF for their contributions and commitment to this project.
Definitions
We adopted the definitions provided by WHO Afro IDSR [18]:
Completeness (%) = (Total number of reports delivered × 100) / (Total number of expected weekly reports; N = 21);
Timeliness (%) = (Total number of completed reports delivered on time each week × 100) / (Total number of expected weekly reports; N = 21). To be considered ‘on time’, weekly reports were to be received by the Director of Surveillance of the health district by 17:00 h on the Wednesday following the end of the epidemiological week of the report (24:00 h Sunday), that is, within 65 h. Timeliness was measured only for Weekly reports, not Alerts, in this pilot study.
Duration of transmission: calculated as the time interval between the reception of the first SMS of a complete weekly report and the last SMS of the report.
Usability of the app was assessed at the end of the 15-week period by the 21 surveillance focal points who completed a custom tool adapted from that described by Schnall et al. [29] and Lewis et al. [30] to assess three aspects: ease of use, usefulness, and efficiency of the app. Responses on the self-administered 27 item questionnaire were given on a 5 point Likert scale (1-low to 5-high)
Funding
Funding for the project was provided by MSF.
Disclaimer
The authors alone are responsible for the views expressed in this article and they do not necessarily represent the views, decisions or policies of the institutions with which they are affiliated.
Authors’ contributions
ZEK, MS, AP, SNZ, CPS, JPK, HM, MA, GD, DI, ABB, TD, PC, MQ, BW, SR, HdR and BR planned, conceived and implemented the study; ZEK analyzed the data; ZEK and BR wrote a first draft of the paper; all authors contributed to critical interpretation of the results and revision of the manuscript.
Ethics approval and consent to participate
This project was an operational innovation that was approved and directed by the CAR Ministry of Health with technical and financial support from MSF. No personal data were collected.
Consent for publication
Not applicable.
Competing interests
We have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Contributor Information
Ziad El-Khatib, Email: ziad.khatib@gmail.com.
Maya Shah, Email: Maya.SHAH@geneva.msf.org.
Samuel N Zallappa, Email: samuelnzallapan@yahoo.fr.
Pierre Nabeth, Email: nabethp@who.int.
José Guerra, Email: guerraj@who.int.
Casimir T Manengu, Email: manenguc@who.int.
Michel Yao, Email: yaom@who.int.
Aline Philibert, Email: philibert.aline@gmail.com.
Lazare Massina, Email: massinalazare@yahoo.fr.
Claes-Philip Staiger, Email: claes-philip@staiger.se.
Raphael Mbailao, Email: mbailaoraphael@yahoo.fr.
Jean-Pierre Kouli, Email: koulizou@yahoo.fr.
Hippolyte Mboma, Email: hippomboma@gmail.com.
Geraldine Duc, Email: geraldineduc@hotmail.com.
Dago Inagbe, Email: MSFCH-CAR-HOM@geneva.msf.org.
Alpha Boubaca Barry, Email: alphaboub2000@yahoo.fr.
Thierry Dumont, Email: zemsfdocteur2014@outlook.com.
Philippe Cavailler, Email: pcavailler@gmail.com.
Michel Quere, Email: michel.quere@geneva.msf.org.
Brian Willett, Email: bdw_@hotmail.com.
Souheil Reaiche, Email: souheilreaiche@gmail.com.
Hervé de Ribaucourt, Email: Herve.DeRIBAUCOURT@geneva.msf.org.
Bruce Reeder, Email: bruce.reeder@usask.ca.
References
- 1.World Health Organization (WHO). WHO technical guidelines for integrated disease surveillance and response in the African region. 2015. http://afro.who.int/publications/technical-guidelines-integrated-disease-surveillance-and-response-african-region-0.
- 2.World Health Organization mHealth: New horizons for health through mobile technologies. Observatory. 2011;3:112. [Google Scholar]
- 3.Mtema Z, Changalucha J, Cleaveland S, et al. Mobile phones as surveillance tools: implementing and evaluating a large-scale Intersectoral surveillance system for rabies in Tanzania. PLoS Med. 2016;13:e1002002. doi: 10.1371/journal.pmed.1002002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Nsubuga P, Nwanyanwu O, Nkengasong JN, et al. Strengthening public health surveillance and response using the health systems strengthening agenda in developing countries. BMC Public Health. 2010;10(Suppl 1):S5. doi: 10.1186/1471-2458-10-S1-S5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Mghamba JM, Krekamoo W, Senkoro KP, et al. Challenges of implementing an Integrated Disease Surveillance and Response strategy using the current Health Management Information System in Tanzania. Tanzan J Health Res. 2004;6:57–63. [Google Scholar]
- 6.Hall CS, Fottrell E, Wilkinson S, et al. Assessing the impact of mHealth interventions in low- and middle-income countries – what has been shown to work? Glob Health Action. 2014;7:25606. doi: 10.3402/gha.v7.25606. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Njuguna HN, Caselton DL, Arunga GO, et al. A comparison of smartphones to paper-based questionnaires for routine influenza sentinel surveillance, Kenya, 2011- 2012. BMC Med Inform Decis Mak. 2014;14:5. doi: 10.1186/s12911-014-0107-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Tomlinson M, Rotheram-Borus MJ, Swartz L, et al. Scaling up mHealth: where is the evidence? PLoS Med. 2013;10:e1001382. doi: 10.1371/journal.pmed.1001382. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.World Health Organization (WHO). eSurveillance implementation in the context of IDSR in the WHO African Region; 2015. p. 2015. https://afro.who.int/sites/default/files/2017-06/esurveillance-implementation-in-the-context-of-integrated-disease-surveillance-and-response-in-the-who-african-region.pdf.
- 10.Yang C, Yang J, Luo X, et al. Use of mobile phones in an emergency reporting system for infectious disease surveillance after the Sichuan earthquake in China. Bull World Health Organ. 2009;87:619–623. doi: 10.2471/BLT.08.060905. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Freifeld CC, Chunara R, Mekaru SR, et al. Participatory epidemiology: use of mobile phones for community-based health reporting. PLoS Med. 2010;7:e1000376. doi: 10.1371/journal.pmed.1000376. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Davis J, Reilly L. Security to go: a risk management toolkit for humanitarian aid agencies. 2nd edition; 2017.
- 13.World Health Organization (WHO). WHO helps Central African Republic assess damage and restore its health system. Bangui; 2014. http://www.who.int/features/2014/car-health-system/en/.
- 14.World Health Organization (WHO). Strengthening health security by implementing the International Health Regulations; 2005. p. 2006. http://www.who.int/ihr/finalversion9Nov07.pdf.
- 15.Frontline. FrontlineSMS Desktop. 2016. https://www.frontlinesms.com/.
- 16.World Health Organization. WHO recommended surveillance standards, 2nd ed. Geneva: World Health Organization; 1999.http://www.who.int/iris/handle/10665/65517.
- 17.World Health Organization (WHO). Support to countries for strengthening public health capacities required under the international health regulations. Lyon; 2016. http://apps.who.int/iris/bitstream/handle/10665/251696/WHO-HSE-GCR-LYO-2016.17-eng.pdf?sequence=1.
- 18.World Health Organization (WHO). Communicable disease surveillance and response systems. Lyon; 2006. http://www.who.int/csr/resources/publications/surveillance/WHO_CDS_EPR_LYO_2006_2.pdf
- 19.Novel-T . Novel-T Innovative Solutions. 2016. [Google Scholar]
- 20.StataCorp . Stata Statistical Software: Release 12. 2011. [Google Scholar]
- 21.European centre for disease prevention and control. Data quality monitoring and surveillance system evaluation - a handbook of methods and applications. Stockholm; 2014. https://ecdc.europa.eu/sites/portal/files/media/en/publications/Publications/Data-quality-monitoring-surveillance-system-evaluation-Sept-2014.pdf.
- 22.Rosewell A, Ropa B, Randall H, et al. Mobile phone-based syndromic surveillance system, Papua New Guinea. Emerg Infect Dis. 2013;19:1811–1818. doi: 10.3201/eid1911.121843. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Randrianasolo L, Raoelina Y, Ratsitorahina M, et al. Sentinel surveillance system for early outbreak detection in Madagascar. BMC Public Health. 2010;10:31. doi: 10.1186/1471-2458-10-31. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Asiimwe C, Gelvin D, Lee E, et al. Use of an innovative, affordable, and open-source short message service-based tool to monitor malaria in remote areas of Uganda. Am J Trop Med Hyg. 2011;85:26–33. doi: 10.4269/ajtmh.2011.10-0528. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Poushter J, Stewart R. Smartphone ownership and internet usage continues to climb in emerging economies - but advanced economies still have higher rates of technology use. Pew Research Center. 2016. http://www.pewglobal.org/2016/02/22/smartphone-ownership-and-internet-usage-continues-to-climb-in-emerging-economies/.
- 26.Agarwal S, Perry HB, Long LA, et al. Evidence on feasibility and effective use of mHealth strategies by frontline health workers in developing countries: systematic review. Trop Med Int Heal. 2015;20:1003–1014. doi: 10.1111/tmi.12525. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Zurovac D, Talisuna AO, Snow RW. Mobile phone text messaging: tool for malaria control in Africa. PLoS Med. 2012;9:e1001176. doi: 10.1371/journal.pmed.1001176. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Prefecture Sanitaire - Mambere Kadei. Completness of weekly reports in Mambere-Kadei. Berberati: Ministry of Health, Central African Republic; 2016.
- 29.Schnall R, Rojas M, Bakken S, et al. A user-centered model for designing consumer mobile health application (apps) J Biomed Inform. 2016;60:243–251. doi: 10.1016/j.jbi.2016.02.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Lewis JR. IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int J Hum Comput Interact. 1995;7:57–78. doi: 10.1037/t32698-000. [DOI] [Google Scholar]