Feasibility and challenges in sustaining a community based surveillance system in post-Ebola Sierra Leone

Abstract

Background

In outbreak-prone settings, community-based surveillance (CBS) systems can alert health authorities to respond in a timely manner where suspected cases of disease are being reported. After the 2014–2016 Ebola outbreak, the WHO and other stakeholders supported the establishment of CBS in Sierra Leone, for which community health workers (CHW) were trained to collect and report symptoms data of 11 priority health conditions in their communities. Our study objective was to assess feasibility and challenges to sustain CBS in a low resource setting as part of a World Bank evaluation of Sierra Leone’s Ministry of Health and Sanitation’s (MoHS) CBS and electronic Integrated Disease Surveillance & Response (eIDSR) systems.

Methods

In 2019 we conducted a mixed methods assessment consisting of a household incidence survey, health facility survey, household case verification survey, a costing analysis, and in-depth interviews and focus group discussions with key stakeholders of the CBS system in eight chiefdoms of 4 purposefully selected districts in Sierra Leone. The study period for primary data collection was February through April 2019. We also conducted secondary data analysis of surveillance data in DHIS2 of all 32 chiefdoms.

Results

In districts where CBS was ‘fully functional’, the number and type of CBS alerts corresponded to the number and type of diseases reported through facility based eIDSR system in the same period. However under-reporting of diarrhea and measles suspects from the community still appeared to occur, and reporting deteriorated when primary health care staff including CHWs reported the stoppage of stipends. The annual budget impact for CBS was estimated at 4.4 million USD in 2018. The majority of costs were made at community level (73%) compared to regional (0.3%), primary health unit (21%), district (4%) and national (2%) level. The most important costs drivers were training of CHWs (59% of total costs) and salaries (including stipends of the CHWs) of human resources (15%). Barriers included sustainable financing of human resources, internet connectivity, as well as limited trainings and supportive supervision, and unsupported transportation costs for CHWs and peer supervisors (PS). CHWs and community members reported that communities are more willing to share information about health issues compared to the pre-CBS implementation period.

Conclusions

The similarity between CBS and IDSR reports support the possibility that CBS increases the sensitivity of disease surveillance to the level of the community, which would enable local authorities to take early prevention measures when and where impact will be the greatest. Qualitative interviews suggest that CBS has improved the interface between the community and primary level of the health system. However if the barriers to sustainability are left unaddressed, opportunities for CBS to prevent disease outbreaks will go unrealized.

Introduction

Health facility-based Integrated Disease Surveillance and Response (IDSR) systems have been active throughout West Africa since 2001 [1], with and without community based surveillance (CBS). CBS is primarily run by community volunteers and peripheral health facility staff, to increase the overall reach and sensitivity of public health surveillance. Sierra Leone’s Ministry of Health and Sanitation (MoHS) has used IDSR since 2008 to notify 26 priority health conditions, events and diseases [2]. As of 2018, Sierra Leone’s IDSR was implemented in each of its 14 districts. At the onset of the 2014–2016 West African Ebola outbreak, no CBS was functional in Sierra Leone, leaving the health system vulnerable to delayed detection and reporting of symptoms and deaths caused by Ebola hemorrhagic fever. The subsequent deaths of an estimated 11,291 people in Sierra Leone and the devastating impact of the virus on the health system led to renewed international and domestic interest to support CBS to enable more timely detection of outbreak-prone disease and life-threatening conditions [3].

Community based surveillance (CBS) has been implemented in many settings where routine surveillance may not capture a sufficient (suspected) number of cases of disease to effectively and in a timely manner limit the spread of infection. Throughout the past fifteen years, CBS has been implemented in West African countries including Sierra Leone, Liberia, Cote d’Ivoire, and Benin, and South(east) Asian countries including Bangladesh and Laos [4–6]. In Sierra Leone, a 2015 pilot deployment of CBS saw 82% of 7142 community volunteers submitting reports on a monthly basis [7], and in Bangladesh’s Cox Bazaar, CBS detected suspected cases of acute watery diarrhea (AWD) with a positive predictive value (PPV) of 88.8% and for measles a PPV of 73.7% [8]. Such evidence suggests that CBS can both be effective and improve the reach of a country’s routine public health surveillance system for priority conditions, events, and outbreak-prone diseases included on the WHO’s IDSR list.

Sierra Leone’s MoHS rolled out CBS in nine of its 14 districts between 2016 and 2018 with technical support from the WHO and United States CDC [7, 9]. However due to limited government ownership and multiple competing financial constraints, CBS was discontinued. While active between 2016 and 2018, the purpose of Sierra Leone’s CBS was to extend the reach of IDSR and to improve the timeliness with which the MoHS rapid response team (RRT) could respond to life-threatening conditions including suspected cases of outbreak-prone disease. The CBS system gathered surveillance on 11 priority diseases, conditions or events: acute flaccid paralysis (polio), acute watery diarrhea, cluster of deaths, guinea worm, maternal death, measles, neonatal death, neonatal tetanus, suspected Ebola, yellow fever, and other unusual events such as a cluster of animal deaths.

The World Bank Group commissioned an assessment of Sierra Leone’s electronic IDSR and CBS systems in 2018 which was supported by the country’s Regional Disease Surveillance Systems Enhancement (REDISSE) fund. The purpose of the assessment was to inform the MoHS, domestic and international stakeholders of both systems’ technical- and cost-performance in anticipation of the (then) planned scale-up to achieve national coverage. While IDSR was successfully scaled up and is still implemented in all districts [2], CBS stopped functioning in 2019. The most recent version of the MoHS’s IDSR guidelines do restate the value of a CBS system, and the MoHS and development partners are working to digitize the system using the OneHealth platform [10].

This manuscript describes the World Bank-commissioned assessment of the CBS system. The assessment’s objectives for CBS specifically were to measure and describe the technical performance, i.e. its capacity to function as an extension of the IDSR into communities; to measure the costs and cost-effectiveness of running the system; and to evaluate its contribution to the health system and ability to be sustained.

Methods

Study design

The study team carried out a mixed methods study to assess performance of Sierra Leone’s IDSR and CBS systems, combining seven quantitative and qualitative research components: (i) desk review of MoHS surveillance documents and international literature on CBS for outbreak prone diseases, (ii) spatial and time series analysis of CBS alerts, (iii) household survey to estimate incidence of symptoms and health seeking behavior, (iv) health facility survey on IDSR and CBS experiences, (v) household case verification study, (vi) costing analysis and vii) qualitative study focusing on health system factors (Fig. 1).

Fig. 1.

Overview of all assessment research components

The study team convened a stakeholders’ workshop with the MoHS, international and country partners to agree on assessment objectives and methods. To operationalize the study objectives we identified research indicators for each study component, based on international literature on surveillance system evaluation and stakeholder consultation [11].The assessment was performed on both CBS and IDSR, but the methods and results presented in this manuscript address only the CBS portion of the assessment therefore results of the case verification survey are not presented. As CBS is intended to supplement the IDSR system, the interface between CBS and IDSR is addressed where relevant throughout this manuscript [9].

Data management and analysis

Data collection for all studies was done in April 2019. Eight chiefdoms within four districts (two per district) were purposefully selected with stakeholder input based on three criteria: (1) availability of functioning IDSR and CBS for specific lengths of time prior to data collection, (2) representative rural and urban areas, and 3) representative geography. Chiefdoms were selected in Port Loko, Koinadugu and Pujehun districts, and Tonkolili was selected as a control, where CBS reporting was not taking place (Fig. 2).

Fig. 2.

Locations of Assessment Districts and Chiefdoms (2018)

We developed data collection tools for all study components, and programmed household, health facility, and case verification survey tools in OpenDataKit (ODK). Forms were stored on a central server and uploaded onto the ODK Collect app on Android devices. Questions were formulated to mirror Demographic and Health Survey language for comparability with Sierra Leone’s and other DHS surveys. Quantitative and qualitative data collection teams were trained on tools, survey methodology and ethical issues. Tools were piloted in a health facility before beginning field work.

Household survey

While the primary aim of the household incidence survey was to serve the assessment of the IDSR system, we also collected information on health seeking behavior and awareness of CBS. Household survey locations were selected as part of a multi-stage cluster sampling design. Within each chiefdom, 13 enumeration areas (EAs) were randomly selected by Statistics Sierra Leone, and per EA 20 households were randomly selected following a systematic skip pattern for a total sample size of 2060 households. The household survey was conducted primarily in Krio with the primary caretaker of under 5 year old children. Household survey field teams collected and uploaded 2079 digital forms to the server, averaging 260 households per chiefdom. Seven households with incomplete data were dropped, leaving 2072 households in the final dataset. 9793 case-based records were created to enumerate all household individuals including those who moved away or died in the prior year. Age data were collected from 9452 individuals, of which 26.2% (n = 2475) were under age five, and 73.8% (n = 6977) who were 5 or above.

Health facility survey

Four health facilities were surveyed in each of two chiefdoms of Port Loko, Pujehun and Koinadugu for a total of 24 health facilities. The purpose of the health facility survey was to describe CBS recording and reporting practices from facility staff perspectives, including their interactions with CHWs, PSs, and the District Health Management Team (DHMT). Facilities were selected purposefully to ensure a sufficient number of CBS alerts was available for the case verification survey (to verify notified cases, not described here). Data collectors also extracted patient information from CBS case report forms (CRFs) showing suspected cases of acute watery diarrhea (AWD), measles, or maternal deaths for the 1st through 17th epidemiological week of 2019. Data collection teams uploaded 24 health facility forms to the server.

Quantitative analysis of the household and health facility surveys was conducted in Stata version 15 SE software including descriptive statistics, and for the costing analysis descriptive statistics were calculated in Microsoft Excel.

Qualitative study (IDIs and FGDs)

For the qualitative study we applied various methods including the desk review, and IDIs and FGDs with stakeholders at all levels in Sierra Leone’s health system on IDSR and CBS. At the district level, 15 IDIs were conducted with the DHMT including members of the RRT, hospital and district lab staff. At chiefdom level 15 IDIs were conducted with primary health unit (PHU) staff. At community level 15 IDIs were carried out with CHWs, PSs, and village leaders (VL), and FGDs were conducted with community members. In total we performed 45 IDIs and 6 FGDs.

Following transcription of qualitative interviews, two researchers identified main themes from a sample of 10 transcripts to develop a coding framework. Coding was performed in Microsoft Excel by three researchers. When new themes were identified in the remaining transcripts, they were discussed and added to the framework. After coding, descriptive summaries were written for each code and linked the assessment framework’s research questions.

DHIS2 data extraction (spatial and time series analysis)

We extracted measles and diarrhea disease notifications from the MOHS’s IDSR DHIS2 module for all chiefdoms across Sierra Leone from 2017 through March 2019. We performed time series analysis of CBS suspected cases of diarrhea and measles and IDSR disease notification rates per 100,000 population from October 2016–March 2019. We also performed Local Indicators of Spatial Autocorrelation (LISA) analysis in GeoDa using the Bayesian shrinkage technique to identify whether CBS alerts of 2018 per 100,000 population showed spatial clustering or outliers at the chiefdom level. CBS LISA maps were compared with IDSR LISA maps to identify similarities and dissimilarities in observed spatial patterns of comparable CBS alerts and IDSR notifications.

Costing analysis

We designed a retrospective micro-costing study from a health system perspective based on WHO guidelines and relevant literature [12–16]. CBS surveillance activity costs were collected and aggregated for a one year period from 1 January to 31 December 2018. Cost prices were obtained from governmental disease surveillance budgets, program records and expert opinion. We used paper based questionnaires to interview surveillance program staff about resources used. Four interviews were conducted with representatives of MoHS Disease Surveillance Office, CDC, eHealth Africa and WHO. We visited regional laboratories in Port Loko, Makeni and Bo and conducted one interview per lab. We conducted 13 district level interviews in Koinadugu, Pujehun and Tonkolili, each. Per district, eight interviews were conducted in the urban chiefdom with a representative from the DHMT, district hospital, private hospital, Community Health Center (CHC), Community Health Post (CHP), Mother and Child Health Post (MCHP), one CHW peer supervisor (PS) and one CHW. In the rural chiefdom five interviews were conducted with representatives from CHC, CHP, MCHP, one CHW PS and one CHW.

Cost categories included building, equipment, human resources, training, internet, travel, office supplies, booklets and lab supplies. District level population size estimates and number of health system entities (e.g. number of PHUs and CHW per district) were collected from the 2015 census and MoHS program data respectively [17–19].

Cost data were recorded in Sierra Leonean Leones (SLL) and US dollars (US$) using the mean annual 2018 exchange rate [20]. Data were entered into Microsoft Excel and resource cost components were multiplied by prices. Costs were aggregated to calculate the annual 2018 unit cost per CBS activity and health system entity. The budget impact (total annual costs in 2018) was estimated based on projections. The unit costs were multiplied by the number of health system entities for Koinadugu, Tonkolili, Pujehun and for Sierra Leone nationwide. Based on the budget impact we calculated cost per capita and the distribution across health system levels. To analyze cost drivers we divided the costs by the budget impact for all cost categories. Cost effectiveness estimates were calculated by dividing the budget impact by the number of CBS suspected measles, diarrhea and maternal death cases alerted in 2018 for Koinadugu, Tonkolili and Pujehun.

Ethical considerations

Ethical approval was obtained from the Sierra Leone Ethics and Scientific Review Committee (SLESRC) prior to the commencement of this study. Collection of personal identifiers were required as part of the assessment’s primary data analysis component and were deleted after creation of the final study dataset.

Results

Results of the assessment of CBS performance are presented in line with the CDC’s Framework for Evaluating Public Health Surveillance Systems for Early Detection of Outbreaks [10]. The framework identifies the following components of a comprehensive evaluation of surveillance systems: (1) system description; (2) outbreak detection description; (3) system experience description; and (4) provision of conclusions and recommendations. As this was a mixed methods study, we triangulated findings from all study components to develop integrated recommendations to the MoHS.

System description

Between 2017 and 2018 community leaders of nine districts were trained to conduct CBS as either a CHW or PS. Surveillance started with CHWs visiting households in their communities at least once per week to interview residents, although they sometimes skipped households or villages due to poor weather or distance. During a visit, if someone in the household had at least one symptom matching a suspected case definition, the CHW filled out a CRF. CHWs reported the household data from their community to their PS or to the PHU, and PS collected CHW case report forms (CRF), SMSs or verbal reports and brought them to PHUs weekly. PHU staff sometimes compiled paper forms and entered data electronically in tablets into the DHIS2 CBS module, or reported by phone to DHMT if tablets were unavailable. If CBS alerts were reported by phone or paper, DHMT staff entered PHU level CBS alerts digitally into DHIS2 and investigated high priority CBS alerts in their respective district. On a weekly basis district-level CBS alerts were reviewed in meetings convened by the MoHS and technical partners. Stakeholders generally reported the same CBS processes and information flow through various channels (phone, SMS, verbal communication, paper). CBS suspected cases reported to PHUs were not de-duplicated from IDSR cases identified in the same PHU when they concerned the same individual. If suspected cases were identified by CHWs conducting CBS, specimens for testing were either collected in the community or individuals were asked to present to the nearest PHUs for specimen collection. Testing could be done in PHU laboratories (mostly blood testing) or if lab services were not available at the PHU, specimens were sent to district, regional, or the national reference laboratory, or outside of the country.

18 of 24 health facility survey respondents reported that at least one CHW reported CBS forms to them in the prior 12 months, with at least half of CHWs actively reporting alerts. Non-reporting CHWs faced limited transportation, unrealistically large catchment areas, poor road infrastructure, or rough terrain combined with the lack of (functioning) motorbikes and/or fuel. Several CHWs and PSs used their own money to cover their catchment area or deliver reports, or said that they could not, especially during the rainy season.

It is difficult. The distance from one community is far, therefore they [CHWs] used to pay bike fare from their own pocket (…). I used to walk on foot and pay bike fee to hand over their reports.” (PS9, Koinadugu).

[My] area during the raining season is covered with water. (…) I use a boat…it is fearful because it is a big river and it is not a big canoe. (CHW 18, Port Loko)

According to the MoHS policy, a performance incentive should be paid to all CHWs on a three-monthly basis; however, findings from the costing and qualitative study showed that this performance incentive was not being paid to all CHWs consistently.

Outbreak detection description

Timeliness

The IDIs and FGDs revealed that CHWs, PSs, village leaders and PHU staff were aware of CBS weekly reporting deadlines and that the majority were able to meet them. Delays resulted from limited transportation, lack of phone credit or a poor phone network to call the PS or PHU. Community members verified that CHWs usually visited households daily or several times per week. While weekly reporting at each link of the reporting cascade was standard in most districts from Q3 2017 to Q1 2019, PHU staff confirmed that high priority symptoms (i.e. bleeding from the mouth, concurrent rash and fever, cluster of deaths) triggered immediate alerts. Over time CBS reporting shifted to a monthly schedule, while one respondent reported that CBS reports never arrived on time.

Timeliness of the RRT could not be measured quantitatively as RRT logs were not identified; however, interviews at district, PHU and community level indicated that RRT investigations were conducted within 24 h up to one week of the alert depending on urgency and transport options. The costing study revealed that district surveillance officers mostly used public transport and self-payment when they needed to verify CBS alerts in the community. An in-charge from Port Loko remarked about the timeliness of the RRT:

Sometimes the response team do not come on time. …if they [RRT] delay and we don’t have facilities to keep the patients, they return back to their communities. We often lose those cases because of that delay…If they come for specimen the same day, it will be better. (I/C 16; Port Loko)

Validity

PSs sometimes attempted to verify symptoms in person which were reported by CHWs. One PS chose to verify suspected cases alerted by the CHW by visiting the household before reporting to the PHU:

If anything happened in the community the CHW will call me on phone I will go there and view the case to know what happened before I send the report to the CHO at the health center. (PS 11; Port Loko).

Concurrence of CBS and IDSR over time

CBS AWD reporting started in nine out of fourteen districts in the first five weeks of 2018 but continued throughout 2018 only in Koinadugu and Pujehun. Koinadugu’s CBS AWD alerts largely reflected concurrent trends of IDSR AWD notifications while in Pujehun, CBS AWD alerts spiked between January and July 2018 while IDSR notifications remained stable (Fig. 3(1) and (2)). Very few suspected measles cases were alerted through CBS in the first half of 2018. No CBS alerts for any symptoms were reported in either Port Loko or Tonkolili during the study period.

Fig. 3.

(1) Suspected diarrhea with severe dehydration notifications in IDSR and CBS, Koinadugu (2017–2019). (2) Suspected diarrhea with severe dehydration notifications in IDSR and CBS, Pujehun (2017–2019). The blue line shows IDSR notifications while the orange line shows cumulative CBS and IDSR notifications while both systems were active. In Koinadugu CBS trend mirrors IDSR trend, while in Pujehun CBS notifications sharply decrease while IDSR notifications remain stable in April to July 2018

Concurrence of CBS and IDSR in space

In 2018, IDSR diarrhea with dehydration notifications and CBS AWD alerts were clustered in many of the same northern chiefdoms (Fig. 4(1) and (2)). Figure 4(1) shows spatial clustering of high rates across the north including Koinadugu and near the southwest border. The Moran’s I value of 0.27 for the IDSR U5 diarrhea with dehydration notification rates (not presented) indicates modest spatial clustering across all chiefdoms. The CBS AWD Moran’s I value of 0.54 shows strong spatial clustering, likely attributable to the clustered high rates shown in the north and along the southwest border, and clustered low rates in the center.

Fig. 4.

Local Indicators of Spatial Autocorrelation for diarrhea (suspected) case notification rates in IDSR and CBS, 2018

System experience description

Questions on general health-seeking behavior were asked as part of the household incidence survey. In total, 1751 (85%) of 2072 respondents reported having ever contacted health services. Respondents in Pujehun district sought contact least, with 76% reporting any contact. Amongst them, 47% (n = 829) reported their last health-related contact was with a CHW, which was evaluated by most as helpful (99%, n = 822) (Fig. 5). CHWs interactions were assessed positively because they offered helpful or knowledgeable advice (66.3%), made the individual feel heard (11.3%) or because the CHW explained health information clearly (8.9%). If illness prevented daily activities for the respondent or if a child was ill, most respondents sought care from the community health worker or the nearest peripheral health facility worker, at 84.3% and 85.5% respectively (Fig. 6(1) and (2)).

Fig. 5.

General Health Seeking Behavior

Fig. 6.

(1) Place of treatment seeking when illness prevents daily activities. (2) Place of treatment seeking when child is ill

Community respondents often mentioned that people no longer hid symptoms or sick family members and were willing to share information more readily with CHWs and PHU staff:

…community people are afraid to report case of illness… Now as a result of the introduction of CHWs in community, and the sensitization of community people on health hazards, [this has] greatly [helped to] increase the level and turnout of community who reported CBS case to the DHMT. (PS 16; Mongo).

System stability

Although some CHWs and PSs mentioned not having received the trainings committed by MoHS, most also reported that they received some CBS trainings and guidelines, which they reportedly used regularly. CHWs provided fewer details about CBS training than PSs, and CHWs did not differentiate between CBS and non-CBS trainings. Although village leaders often contributed to CBS, all indicated that they have not received any CBS training.

We are provided with manuals, guidelines, protocols; basic case definition handbook and other set of materials to guide us do our work effectively. This package was given to us during our training as CHW at DHMT in Kabala. (CHW 18; Koi).

WHO and MoHS performed quarterly integrated supportive supervision at the district level, and WHO and the Expanded Program on Immunization provided supportive supervision of adherence to IDSR and CBS guidelines at the PHU level. On average DHMT staff made three supervisory trips to PHUs per week in Pujehun, two in Koinadugu and one in Tonkolili. PHU surveillance focal points reported that they have been trained by DHMT/MoHS and WHO on the paper to tablet transition and use of internet for CBS data reporting. Improvements around CBS training were widely suggested, citing insufficient new and refresher trainings. Lab staff reported receiving trainings and SOPs on sample collection, management and analysis, but that refresher trainings on handling suspected cases were needed.

Although government and donors supported fuel costs for DHMTs vehicles, transport was mentioned as a serious challenge at all levels for CBS related travel. Respondents mentioned poor (road) infrastructure, insufficient fuel and (motor)bikes and vehicles, and difficulty and fear of accessing riverine communities. Some interviewees commented that “maintenance and fueling is all on [them]” (CHO, Puj15) and that they try in “[their] own little ways” to overcome transportation problems. Public transport was the main mode of transport among PHU staff (66%), CHWs and PSs (75%). 43.8% of respondents reported funding transport themselves.

System costs

Budget impact of CBS

Table 1 shows the estimated costs made by different health system entities to execute CBS in 2018 in Koinadugu, Tonkolili, and Pujehun. (Although MoHS stakeholders explained that CBS was not fully rolled out in Tonkolili, findings showed that some CBS activities were conducted.) The unit costs largely differ per health system entity and across the districts. The cost per CHW to conduct CBS activities ranged between USD 94–319 for rural and USD 95–250 for urban areas. The cost for PS was higher in urban areas at USD 571–1021 compared to rural areas USD 674–1124. Table 6 shows the budget impact for CBS in 2018 in Koinadugu (USD 213,338), Tonkolili (USD 365,631) and Pujehun (USD 392,727), which were used to project costs for all 14 districts in Sierra Leone: USD 4,423,204.

Table 1.

Estimated annual unit costs (in USD for 2018) per health sector role involved in CBS in Koinadugu, Tonkolili and Pujehun

Health system level	Health sector role	District costs (USD)
		Koinadugu	Tonkolili	Pujehun
District	DHMT	9938	11,556	14,859
	District hospital	0	0	1402
	Private hospital	134	0	67
PHU	CHC urban	0	2048	1999
	CHP urban	0	1837	966
	MCHP urban	505	1923	845
	CHC rural	0	2521	983
	CHP rural	0	1520	885
	MCHP rural	0	1182	366
Community	CHW PS urban	571	1012	821
	CHW PS rural	674	861	1124
	CHW urban	94	200	319
	CHW rural	127	95	250
Regional laboratories	Port Loko	4191
	Makeni	0
	Bo	4913
National level actors	MoHS surveillance office	70,061
	CDC	7015
	e-Health Africa	0
	World Health Organization	12,734

CBS cost drivers and cost effectiveness

The majority of costs are made at community level ranging between 59 and 95% in the three districts and 73% nationwide (Table 2). Based on the nationwide budget impact projection, costs are driven by training (59% of budget impact), mainly for the CHW and their PSs, human resources salaries and incentives (15%), use of rooms in clinics and offices (8%) and equipment (6%) (Fig. 7).

Table 2.

Total projected annual CBS costs per health system level in 2018 (in USD) for Sierra Leone nationally, and Koinadugu, Tonkolili and Pujehun

Health system level	CBS costs (% of total budget impact)
	Koinadugu	Tonkolili	Pujehun	Nation-wide
National	n.a	n.a	n.a	89,811 (2%)
Region	n.a	n.a	n.a	12,138 (0.3%)
DHMT & hospitals	10,072 (5%)	11,556 (3%)	16,328 (4%)	177,125 (4%)
PHU	0 (0%)	139,613 (38%)	44,317 (11%)	922,239 (21%)
Community	203,338 (95%)	214,462 (59%)	332,082 (84%)	3,221,890 (73%)
Total annual costs (budget impact)	213,338 (100%)	365,631 (100%)	392,727 (100%)	4,423,204
Total population size (2015)	409,372	531,435	346,461	7,092,113
Total annual cost per capita	0.52	0.90	1.13	0.62

Fig. 7.

Cost drivers of eIDSR and CBS (% of total national budget impact in 2018 in US$

Table 3 shows the cost-effectiveness estimates for Koinadugu, Pujehun and Tonkolili in 2018. The estimated cost to detect a maternal death using CBS was USD 213,409 per suspected case in Koinadugu. CBS was more efficient in detecting diarrhea, with costs of USD 241 and USD 1094 per suspected case in Koinadugu and Pujehun respectively. The estimated costs to detect measles was relatively high at USD 106,705 and USD 392,727 per suspected case for Koinadugu and Pujehun respectively.

Table 3.

Estimated cost per suspected case from CBS in Koinadugu, Pujehun and Tonkolili

	Budget impact (annual costs 2018 US$)	Suspected cases detected (2018)		Cost per suspected case detected (US$ 2018)
Koinadugu	213,409	Maternal death	1	213,409
		Diarrhea*	884	241
		Measles	2	106,705
Pujehun	392,726	Maternal death	0	n.a
		Diarrhea*	359	1,094
		Measles	1	392,726
Tonkolili	365,631	Maternal death	n.a.**	n.a
		Diarrhea*	n.a	n.a
		Measles	n.a	n.a

*Acute watery diarrhea for CBS, diarrhea with severe dehydration under 5 years for eIDSR

**CBS was not officially introduced in Tonkolili, but health workers reported CBS related activities during costing study. However, no suspected cases were reported in DHIS2 for Tonkolili

Discussion

The objective of this assessment was to evaluate the performance of Sierra Leone’s CBS system from an operational standpoint; evidence of its ability to extend the reach of the IDSR system; its cost and cost effectiveness; and its sustainability. CBS AWD alerts mirrored and added to the IDSR diarrhea notifications in Koinadugu and Pujehun respectively, suggesting that CBS can detect suspected cases where IDSR might not. This was shown in both the time series and spatial analysis and strengthens the likelihood that CBS increased overall surveillance sensitivity while it functioned. This can enable well supported rapid response teams to contain outbreak prone diseases near the index case, and limit public health consequences and the strain on the health system. However, delayed and halted payments to CBS CHWs and PSs meant that it could not be sustained, thereby also limiting the amount of CBS alert data available to analyze.

A study of Ghana’s CBS system, conducted within a six month timespan from September 2018 to March 2019, showed that 317 health events were captured through CBS which would not have been reported through routine surveillance, including unexpected animal deaths, acute hemorrhagic conjunctivitis and foodborne illnesses [21]. The CBS system of Sierra Leone demonstrated the same capacity as shown in the spatial and temporal analyses.

Positive appraisal of CBS by community stakeholders was widespread, with many individuals mentioning that they were receptive to sharing health information to protect their communities. Even though there were multiple barriers to collect and report data to PHUs, CHWs and PSs showed commitment to supporting CBS, sometimes even after payments stopped; this type of engagement has been reported as a key driver of success of CBS systems by IFRC [22]. Although internet connectivity and phone network outages were mentioned as major problems, there was no evidence that this stopped surveillance. Given this level of community engagement, sustainable financing appeared to be the key missing link. Health system resources were leveraged well to implement CBS; therefore, CBS profited from the health system’s assets but suffered from its weaknesses such as a shortage of finances, and as a direct result, limited transportation and possibly the felt need for more trainings, both of which have been identified as essential pillars of functional CBS systems in crisis-affected settings [23, 24].

Financial resources to implement CBS were not integrated in district budgets, leading to limited capacity to operate the program. This resulted in delays of RRT, decreased motivation of CHWs, and frustration of community leaders and members, particularly with regard to limited resources for transportation, which was also listed as a challenge for functioning community surveillance in Niger [25]. CBS was expensive due to the number of CHWs and PSs who had to be trained nationwide and paid at the chiefdom and village level. However with IDSR and CBS amounting to only 3.2% of Sierra Leone’s 2018 health budget, the costs involved in sustaining payments to CHWs and PSs would have been minor compared to the more than 3.6 billion USD response to the 2014–2016 ebola outbreak in West Africa from international donors [26].

Taking the feasibility and costs of CBS into account, it is worth considering modalities other than a CBS system which aims to provide national geographic coverage. For example sentinel or risk-based CBS to measure symptoms prevalence if costs for full CBS coverage cannot be managed [23, 27]. CHWs can be trained to screen for selected symptoms in geographic areas at higher risk for specific high impact outbreak prone diseases on a routine or intermittent basis, for example for ebola and other viral hemorrhagic fevers [27, 28]. When outbreak-prone disease epidemics do occur, existing CHW networks can be called upon to conduct door-to-door screening for symptoms. Our findings build evidence that CHWs could successfully screen for and alert suspected cases of COVID-19 to their nearest PHU, given that Sierra Leone’s CBS priority symptoms include fever, cough, upper respiratory symptoms, and diarrhea, all of which can identify presumptive COVID-19 cases [29]. Sierra Leone’s MoHS and National COVID-19 Emergency Response Center (NaCOVERC) did in fact implement strict infection control measures and performed routine COVID-19 surveillance, and updated the definition of acute respiratory infection in IDSR to include COVID-19 like symptoms, but the extent to which the definition in CBS was also updated is unknown [30, 31]. While Sierra Leone’s IDSR technical guidelines updated in 2021 contain detailed guidance on setting up a CBS system, training responsible individuals and providing feedback and supportive supervision, they do not specifically mention whether the CBS priority conditions list encompasses any new symptoms consistent with a suspected SARS-COV-2 case definition [2].

While CHWs can be trained to identify symptoms of common illnesses and alert their PHU as needed, this does not mean that the current health system should be expected to diagnose and treat all suspected cases for priority conditions [23]. For relatively common conditions such as diarrhea, the health system could be overwhelmed if all suspected cases presented to health facilities, and it would be unnecessary as the vast majority of cases recover without professional care. Most COVID-19 cases also recover without professional care, and for this reason CHWs can serve the important role of informing the community about infection risks and advising when self-isolation or seeking care at the nearest PHU is appropriate [28]. This could be combined with supervisor verification to confirm alerts of suspected cases. Continuously updating the CBS list of priority conditions to account for newly emerging diseases (and use thereof) may inform the MoHS about where geographic risk zones may be, so that targeted responses can be implemented if and where they are needed [23].

Conclusion

Our assessment methodology covered all major perspectives of a CBS’s functionality i.e. timeliness, accuracy, validity, cost efficiency, sustainability and added value for health. When CBS was operational it showed promise that it could function as an extension of IDSR into harder to reach communities. This was shown in different ways by the spatial and time trend analyses and qualitative interviews. However the system was disrupted before enough evidence could be gathered to prove that it could provide early alerts for outbreak-prone diseases. The case for CBS as an early alert system for a list of priority diseases, conditions and events is strong, especially when health-seeking behavior may be reduced due to fear of infection, as was the case with COVID-19. The CBS data storage system in DHIS2 is still ready to be adapted for use in emergent scenarios, but can only be sustained with supportive financing, policies and governance.

Author contribution

M.B. and A.v.d.B. conceived of all study methods. C.M., N.T., J.J. and K.N. carried out the primary data analysis, and C.M., N.T., and K.N. wrote the main manuscript text. S.W., A.A.K., M.V., O.S., M.K.K and TT.S. provided technical guidance during data collection and analysis. All authors reviewed the manuscript.

Funding

Funding for this study was provided by the World Bank Group.

Availability of data and materials

All data generated or analyzed during this study are owned by Sierra Leone’s Ministry of Health & Sanitation, and may be approached with questions about data availability.

Declarations

Ethics approval and consent to participate

Ethical approval was obtained from the Sierra Leone Ethics and Scientific Review Committee (SLESRC) prior to the commencement of this study. Collection of personal identifiers were required as part of the assessment’s primary data analysis component and were deleted after creation of the final study dataset. Verbal informed consent was obtained from all participants responding to questionnaires and participating in key informant interviews. All methods were carried out in accordance with the Declaration of Helsinki.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.