Cost Analysis of Health Facility Electronic Integrated Disease Surveillance and Response in One District in Sierra Leone

Abstract

Global health security depends on effective surveillance systems to prevent, detect, and respond to disease threats. Real-time surveillance initiatives aim to develop electronic systems to improve reporting and analysis of disease data. Sierra Leone, with the support of Global Health Security Agenda partners, developed an electronic Integrated Disease Surveillance and Response (eIDSR) system capable of mobile reporting from health facilities. We estimated the economic costs associated with rollout of health facility eIDSR in the Western Area Rural district in Sierra Leone and projected annual direct operational costs. Cost scenarios with increased transport costs, decreased use of partner personnel, and altered cellular data costs were modeled. Cost data associated with activities were retrospectively collected and were assessed across rollout phases. Costs were organized into cost categories: personnel, office operating, transport, and capital. We estimated costs by category and phase and calculated per health facility and per capita costs. The total economic cost to roll out eIDSR to the Western Area Rural district over the 14-week period was US$64,342, a per health facility cost of $1,021. Equipment for eIDSR was the primary cost driver (45.5%), followed by personnel (35.2%). Direct rollout costs were $38,059, or 59.2% of total economic costs. The projected annual direct operational costs were $14,091, or $224 per health facility. Although eIDSR equipment costs are a large portion of total costs, annual direct operational costs are projected to be minimal once the system is implemented. Our findings can be used to make decisions about establishing and maintaining electronic, real-time surveillance in Sierra Leone and other low-resource settings.

Effective disease surveillance and response is essential to preventing the spread of disease and is a core capacity of the International Health Regulations (IHR 2005).¹ Establishing real-time surveillance systems capable of detecting public health events of international concern (PHEIC) is a key initiative under the Global Health Security Agenda (GHSA),² which was launched in 2014 to support countries’ IHR compliance. GHSA real-time surveillance objectives encourage the development of “interoperable, interconnected electronic reporting systems”² to improve reporting and analysis of disease surveillance data.

Many African countries use the Integrated Disease Surveillance and Response (IDSR) framework to implement surveillance for public health events of international concern.³ Use of electronic tools to conduct IDSR (eIDSR) reporting and analysis is increasing and has been reported to improve timeliness, completeness, and availability of data.^4–6 Economic costs of IDSR have been assessed in Burkina Faso, Eritrea, and Mali.⁷ However, no evaluations have assessed costs associated with electronic IDSR (eIDSR).

Sierra Leone has limited public health resources, and the 2014–2016 West African Ebola outbreak highlighted the need for a functional disease surveillance system.⁸ Sierra Leone revitalized IDSR in 2015⁹ and, with the support of the US Centers for Disease Control and Prevention (CDC) and partners, developed an electronic platform to streamline and enhance IDSR weekly reporting and analysis of diseases and conditions. eIDSR was developed on the national web-based data reporting platform, DHIS2 (https://www.dhis2.org/), starting at the district level, where district surveillance officers enter data into DHIS2 on behalf of health facilities. Development of an Android mobile app for eIDSR reporting directly from health facilities into DHIS2 also began in 2015, with the goal of improving data timeliness, completeness, analysis, and reporting from areas with low connectivity. After completion of a pilot in 2016–17 in the Port Loko district, the Sierra Leone Ministry of Health and Sanitation (MoHS) and partners began a national rollout of health facility eIDSR in April 2018.

We assessed the economic costs and the primary cost drivers associated with establishing health facility eIDSR in the Western Area Rural district in Sierra Leone. Additionally, we estimated the direct costs of implementation to inform financial plans for future district rollouts, projected ongoing annual operational costs of eIDSR, and conducted alternative operational models to identify cost changes when varying operational inputs.

Methods

Rollout Description

The Ministry of Health and Sanitation and the eIDSR technical working group planned for Western Area Rural to be the third district to implement health facility eIDSR. Western Area Rural is 1 of 14 districts in Sierra Leone, with 63 IDSR-reporting health facilities and a population of 444,270. Sierra Leone has a total of 1,305 IDSR-reporting facilities and a population of 7,092,113.

Preparatory activities included procuring and setting up tablets, planning logistics, and connecting devices. Tablets used in health facilities were provided by 2 organizations, and 1 organization installed mobile device management functionality to restrict use to work calls and data reporting. Training of trainers for district staff, followed by 3 health facility staff training sessions, took place near the Western Area Rural district office. A standardized curriculum was used that covered smart device use, data entry, report submission, and troubleshooting. Six district staff and 126 health facility staff were trained. After training, the Ministry of Health and Sanitation and partners provided intensive follow-up for 4 weeks to support facilities learning to use the eIDSR app; after that, the Ministry of Health and Sanitation provided support as needed.

Study Design

We conducted a retrospective cost analysis of the eIDSR rollout in the Western Area Rural district. This district was assessed because it was the first district rollout after all app development and training materials were finalized. We estimated costs from the perspective of the program provider, which included costs from the government and partners. We used microcosting to collect quantities and prices associated with rollout activities.¹⁰ Costs were recorded in local currency (Sierra Leonean Leones) and converted to US dollars using the 2017 average exchange rate of 7,384.4 Leones per US dollar;¹¹ costs originally in US dollars were converted to Leones for data entry using that rate. Costs were not adjusted for inflation. Costs were assessed by 4 rollout phases, identified by the Ministry of Health and Sanitation as planning (4 weeks: April 22 to May 19, 2018), training (2 weeks: May 20 to June 2, 2018), supervision (4 weeks: June 3 to 30, 2018), and maintenance (4 weeks: July 1 to 28, 2018).

Sampling and Data Collection

The national Ministry of Health and Sanitation identified 6 organizations that provided personnel or resources through direct implementation or technical assistance: (1) national ministry staff oversaw the rollout and conducted weekly data analysis; (2) district ministry staff led the health facility training sessions; (3) eHealth Africa (eHA), funded by the US CDC, developed the app for Sierra Leone and was the main implementing partner for Western Area Rural responsible for logistics, technical support during training, and device preparation and connectivity for 50 tablets; (4) FOCUS1000, funded by the Department for International Development (DFID), provided technical assistance and device preparation and connectivity for 13 tablets shared with a similar Ministry of Health and Sanitation electronic data entry project; and (5) CDC and (6) the World Health Organization (WHO) provided technical support.

We collected data between May 2018 and March 2019 through emails, phone meetings, and field visits. We identified activities in each phase and conducted semi-structured interviews with key personnel from each organization, such as project leads, who agreed to participate. Individuals were asked about activities that their organization was involved in and the costing details of activities (Table 1). When necessary, we interviewed organization leaders, accountants, procurement officers, and logisticians to obtain additional costing details.

Table 1.

List of Example Activities and Associated Costs, by eIDSR Rollout Phase

Phase	Activity Examples	Cost Examples
Planning	Technical working group meetings Preparation meetings and visits at district location Connectivity assessment eIDSR equipment procurement Device set-up Create user accounts in DHIS2	Person time Fuel Tablet procurement Cellular data packages
Training	Training trainers Training health facility staff	Person time Fuel Catering Venue rental Internet hot spot Generator fuel Travel expenses Printing materials
Supervision	Visits to district and health facilities to support submission Data monitoring and analysis Maintain tablet cellular data connectivity	Person time Fuel Data packages
Maintenance	Data monitoring and analysis Maintain tablet cellular data connectivity	Person time Fuel Data packages

Cost data were obtained from invoices, project agreements, and local market prices. If no cost documentation was found, the interviewee was asked to estimate costs. If costs could not be estimated, costs for a similar item from another organization were used or the data were treated as missing. Population data were collected from the Sierra Leone Census 2015.¹² To fit the context of the eIDSR rollout, we modified the CDC SurvCost tool¹³ for data entry using Microsoft Excel software.

Cost Categories and Calculations

Each organization provided information on resources used to support the rollout, including both recurrent and capital costs (see Appendix Table 1 in supplemental material at https://www.liebertpub.com/doi/suppl/10.1089/hs.2019.0082). Costs were organized into 4 cost categories adapted from SurvCost¹³—personnel, office operating, transport, and capital investments—and subcategories (see Appendix Table 2 in supplemental material at https://www.liebertpub.com/doi/suppl/10.1089/hs.2019.0082). Within each category, costs for each input were estimated and apportioned to the 4 rollout phases (see Appendix Table 3 in supplemental material at https://www.liebertpub.com/doi/suppl/10.1089/hs.2019.0082).

Analysis

We compiled the data into an analysis sheet developed in Microsoft Excel. We determined the average costs per health facility and per capita costs. We calculated total economic and direct costs (see Appendix Table 4 in supplemental material at https://www.liebertpub.com/doi/suppl/10.1089/hs.2019.0082) and estimated costs by cost category and phase. Economic costs were direct costs plus the use of existing resources and personnel time. Direct costs were monetary payments required for the rollout. Using the eIDSR operating costs from the time of the study, we projected annual direct operational costs; we assumed 5% annual device replacement, and mobile device management functionality exists on all devices.

We modeled 3 alternative scenarios. Model 1 estimated rollout if more travel costs were required. We assumed partners and national MoHS involved in training incurred lodging and per diem each day. Model 2 estimated implementation primarily by the MoHS. We assumed technical roles, logistical roles, attendance at training, and field support were performed by the MoHS. Model 3 estimated annual direct operational costs using the cellular connectivity plans active at the time of analysis.

Results

The total economic cost to roll out eIDSR to Western Area Rural across 14 weeks was $64,342. eHA accounted for $48,552 (75%) of the total cost (Figure 1). Average rollout cost per health facility was $1,021, and cost per capita was $0.14 (Table 2).

Figure 1.

Total Economic Costs by Organization, in 2017 US$.

Table 2.

Costs per Health Facility and per Capita, in 2017 US$

	Cost per Health Facility	Cost per Capita
Total economic rollout costs	1,021	0.14
Total direct rollout costs	604	0.09
Projected annual operating costs	224	0.03
Model 3 annual operating costs	243	0.03

By cost category (Figure 2), capital was the primary cost driver ($30,232, 47.0%), followed by personnel ($22,630, 35.2%). Table 3 presents cost category by organization. Personnel costs were driven by partner organizations: eHA (66.0%), CDC (18.3%), and FOCUS1000 (10.7%). Capital costs were mostly borne by eHA (75.4%) and FOCUS1000 (22.8%). Within capital, the primary cost driver was procurement of eIDSR equipment, which was 96.9% of capital category costs and 45.5% of total costs (Table 4).

Figure 2.

Total Economic Costs by Cost Category, in 2017 US$.

Table 3.

Total Economic Costs of Rollout to Western Area Rural District by Cost Category and Organization, in 2017 US$

	Total Economic Cost 2017
Organization	Personnel, No. (%)	Office, No. (%)	Transport, No. (%)	Capital, No. (%)
CDC	4,146 (18.3)	195 (1.9)	71 (5.3)	373 (1.2)
District Ministry of Health and Sanitation	565 (2.5)	0 (0.0)	0 (0.00)	0 (0.00)
eHA	14,940 (66.0)	9,710 (95.7)	1,113 (83.2)	22,790 (75.4)
FOCUS1000	2,420 (10.7)	162 (1.6)	58 (4.3)	6,887 (22.8)
National Ministry of Health and Sanitation	271 (1.2)	77 (0.8)	23 (1.7)	51 (0.2)
WHO	288 (1.3)	0 (0.0)	72 (5.4)	131 (0.4)
Total	22,630 (100.0)	10,144 (100.0)	1,337 (100.0)	30,232 (100.0)

Table 4.

eIDSR Equipment Total and Percentages of Economic and Direct Costs^a

	Economic Costs	Direct Costs
eIDSR equipment total	$29,286	$29,286
eIDSR equipment % of capital cost category	96.9%	100.0%
eIDSR equipment % of planning phase	71.5%	93.2%
eIDSR equipment % of total costs	45.5%	76.9%

^aeHA provided equipment for 50 health facilities (tablet, case, carry bag, and screen protector) at $450 per facility; FOCUS1000 provided equipment for 13 health facilities (tablet, case, solar charger) at $522 per health facility.

Assessing economic costs by phase (Figure 3) revealed the planning phase accounted for 63.7% ($40,957) of total costs, followed by training at 19.9% ($12,811). The primary cost driver of the planning phase was procurement of eIDSR equipment ($29,286, 71.5%) (Table 4).

Figure 3.

Total Economic Costs by Rollout Phase, in 2017 US$.

Direct costs were 59.2% ($38,059) of total economic costs. As shown in Table 5, eIDSR equipment accounted for 76.9% ($29,286) and training costs for 12.9% ($4,928). The estimated total direct costs were $604 per health facility (Table 2).

Table 5.

Total Direct Costs of Rollout to Western Area Rural District by Phase and Subcategory, in 2017 US$

Category	Subcategory	Planning	Training	Supervision	Maintenance	Total (% of total direct costs)
Office	Supplies	19	427	0	0	446 (1.2)
	Training rentals	0	3,500	0	0	3,500 (9.2)
	eIDSR operating	2,075	0	829	829	373 (9.8)
Transport	Vehicle fuel	40	221	44	9	314 (0.8)
	Travelers	0	780	0	0	780 (2.0)
Capital	eIDSR equipment	29,286	0	0	0	29,286 (76.9)
Total (% of direct costs total)		31,420 (82.6)	4,928 (12.9)	873 (2.3)	838 (2.2)	38,059 (100.0)

Projected annual direct operational costs for eIDSR in the Western Area Rural were $14,091 (Table 6), which is $224 per health facility and $0.03 per capita (Table 2). The mobile device management functionality (Table 6) accounted for 80.5% ($11,340 per year, $180 per health facility), and the projected cellular and internet connectivity costs were 6.8% ($951 per year, $15 per health facility).

Table 6.

Projected Annual Direct Operational Costs for All Health Facilities in Western Area Rural District, by Input, in 2017 US$ (% of column total)

	Rollout Cost Projectiona No. (%)	Model 3 Cost Projectiorb No. (%)
Tablet connectivity	831 (5.9)	2,048 (13.4)
District connectivity	120 (0.9)	120 (0.8)
Mobile device managementc	11,340 (80.5)	11,340 (74.1)
Device replacementd	1,800 (12.8)	1,800 (11.8)
Total	14,091 (100.0)	15,308 (100.0)

^aActual rollout projection estimates total cost using cellular connectivity prices as paid during the rollout.

^bModel 3 projection estimates total cost using cellular connectivity prices as was being paid at the time of analysis.

^cMobile device management functionality was assigned to all 63 tablets.

^dDevice replacement calculated assuming a 5% annual replacement at the highest tablet price of the 2 organizations; 4 replacement phones are accounted for at the eHA cost of $450.

Table 7 shows results for models 1 and 2. Model 1 included $54 for lodging/per diem for training days. Estimated economic costs increased to $80,842, and direct costs increased to $54,559. In model 2, shifting functions to the Ministry of Health and Sanitation decreased costs by $12,185. The third model (Table 6) shows an increase to $243 per health facility per year for annual direct operational costs with the active cellular connectivity plans in country.

Table 7.

Alternative Operational Models for Rollout, in 2017 US$

	Actual Rollout		Model 1a		Model 2b
Category	Economic Cost, $ (%)	Direct Cost, $ (%)	Economic Cost, $ (%)	Direct Cost, $ (%)	Economic Cost, $ (%)	Direct Cost, $ (%)
Personnel	22,630 (35.2)	0 (0)	22,630 (28.0)	0 (0)	10,444 (20.0)	0 (0)
Office	10,144 (15.8)	7,680 (20.2)	10,144 (12.5)	7,680 (14.1)	10,144 (19.4)	7,680 (20.2)
Transport	1,337 (2.1)	1,094 (2.9)	17,836 (22.1)	17,593 (32.2)	1,337 (2.6)	1,094 (2.9)
Capital	30,232 (47.0)	29,286 (76.9)	30,232 (37.4)	29,286 (53.7)	30,232 (58.0)	29,286 (76.9)
Total	64,342 (100.0)	38,059 (100.0)	80,842 (100.0)	54,559 (100.0)	52,157 (100.0)	38,059 (100.0)

^aModel 1 assumes all persons involved in training incur lodging and per diem in the amount of $54 per day as used for Western Area Rural. We did not alter fuel costs, since a specific distance was not used.

^bModel 2 assumes all technical roles, logistical roles, attendance at training, and field support are performed by the Ministry of Health and Sanitation.

Discussion

This study estimates costs to expand an established electronic system, health facility eIDSR, to a new district and is the first estimation of operational costs needed to sustain eIDSR. As use of electronic platforms to perform IDSR reporting and analysis becomes more popular,^4–6 understanding the costs to roll out the system is valuable to Sierra Leone and other IDSR implementers.

The largest cost driver of health facility eIDSR rollout in the Western Area Rural district was eIDSR equipment. This highlights the infrastructure investment, such as tablets, required for an electronic reporting system. Although cheaper devices could reduce overall cost, devices must meet required eIDSR functional specifications, such as adequate screen size. Personnel costs, driven by partner support, were the second largest cost driver. eHA contributed the largest share because their technological skill set was critical for many activities. Direct costs representing monetary payments for the rollout were 59% of total economic cost and were driven by eIDSR equipment procurement. Training costs were a small share of the direct costs and reflect the minimal travel expenses for the Western Area Rural. We estimated training costs in districts with greater geographic dispersion using an alternative model (Table 6), but we still found direct costs were driven by eIDSR equipment.

Projected annual direct operational costs per health facility were driven by the mobile device management functionality installed on the tablets—a sizeable cost, but its functionality contributes to the program’s success. Identifying alternative software applications or methods to provide the same functionality would greatly reduce costs. Also included in projected costs are tablet cellular connectivity and district internet connectivity. Combined, these costs were a small portion of operational costs.

The alternative operational models further explored the costing structures. In model 1, the addition of costs for lodging and per diem increased overall training costs, but the primary cost drivers remain eIDSR equipment and personnel. Even in districts with higher training costs, any efforts to reduce training costs should be balanced with maintaining quality training. Model 2 illustrates what implementation costs might be without partner support. Results suggest that total economic costs could be reduced by 19% by shifting responsibilities to the MoHS, due to salary differences between organizations. However, because the rollout was dependent on partners with technical capacities for electronic system implementation and training, any shifts in responsibilities should maintain these necessary skill sets. Finally, model 3 provides an estimate of annual operating costs with the connectivity plan that was in place at the time of analysis. Although annual operating costs could continue to fluctuate with changes to connectivity prices, we expect the connectivity costs will remain a lower share of operating costs.

There have been few studies assessing the economic costs of surveillance systems, and this is the first to assess eIDSR. However, although other surveillance system costing studies may not be directly comparable, they also reported that personnel time was a primary cost driver.^7,14,15 Evaluations of eIDSR systems in other countries have noted financial challenges, such as internet data and infrastructure, especially when health facility staff must use personal phones and cellular data.^4–6 In Sierra Leone, tablets and cellular data are provided to health facility staff, but funding has come from donors following the Ebola outbreak. The government of Sierra Leone could consider future funding opportunities to sustain eIDSR and decrease dependence on donor support, including costs for connectivity, replacement devices, and possibly refresher training. In Sierra Leone, several programs are considering using tablets for reporting from health facilities; cost sharing between programs could help with ongoing costs.

Health facility eIDSR costs should be considered against the country’s needs, current health expenditures, and the global perceived benefits of a functional real-time surveillance system. The eIDSR per capita rollout costs ($0.14) and annual direct operating costs ($0.03) are a small proportion of the annual per capita health expenditures ($86 in 2016¹⁶) for a system to improve notification and response to public health events. Additionally, although the upfront cost and the resources required for eIDSR are not trivial, the estimated costs of outbreaks and pandemics may exceed investments such as eIDSR that target early, rapid detection to prevent wide-scale pandemics.¹⁷

Limitations

There are several limitations to our study. First, data from interviews, including time estimates, were self-reported and thus subject to recall bias. Second, some organizations had difficulty identifying operating costs. Alternative methods were used to develop approximations. Analysis does not include equipment shared across districts or programs (eg, computers, servers). Third, estimated costs are not directly reflective of other districts, although we attempted to approximate that in model 1. Fourth, our projections of annual operational costs only reflect direct costs, as we were unable to project total costs because ongoing personnel needs had not yet been established at the time of the study. However, we believe that personnel costs will be lower in sustained operations, as health facility staff become more efficient and less support from the national ministry is needed. Finally, development costs of eIDSR are not included in our estimates. However, many of the features developed in eIDSR will be included in a DHIS2 open source app, which could reduce future development costs for this or similar systems.

Conclusion

This cost analysis revealed several lessons for understanding and identifying resources required to implement health facility eIDSR in a low-resource country. First, although upfront eIDSR equipment costs are a large portion of total costs, annual operational costs are projected to be minimal once the system is implemented. Funding for the substantial upfront costs should be part of the early decision-making process to implement health facility eIDSR through mobile devices, and cost sharing can ease the financial burden to a single program. Second, while donor and partner personnel support were costly, in countries like Sierra Leone with limited technical skills such as software development and server maintenance, partner support may be essential for developing electronic, real-time surveillance systems. Third, project budgets alone do not capture the full extent of the resources required for program implementation. To understand the true economic cost and appropriately allocate resources, methods to assess costs for all components of implementation are necessary.

Future research should examine the benefits of eIDSR in Sierra Leone, as establishment of eIDSR is expected to improve data timeliness, completeness, quality, and access,^4–6,18 but these have yet to be assessed. Additionally, cost-saving opportunities should be explored, such as alternative mobile device management solutions. Finally, the annual costs of the IDSR system, including all functions and levels, should be assessed after a year of eIDSR implementation to better understand operational economic costs and allow for comparison to other IDSR costing studies.

This study provides information to the Ministry of Health and Sanitation on district-level expansion and annual operational costs of health facility eIDSR. Results are useful to advocate for funding from the government and donors to ensure sustainability of the system. These results can also be used to consider improvements to efficiency, identify potential cost savings, and plan resource allocation. The methods and results provide other countries, particularly those in low-resource environments, with costing methods and benchmark costs when considering initiating or expanding a health facility eIDSR or similar electronic surveillance systems.