Abstract
Background
Impact evaluations allow countries to assess public health gains achieved through malaria investments. This study uses routine health management information system (HMIS) data from Zanzibar to describe changes in confirmed malaria incidence and impact of case management and vector control interventions during 2000–2015.
Methods
HMIS data from 129 (82%) public outpatient facilities were analyzed using interrupted time series models to estimate the impact of artemisinin-based combination therapy (ACT), indoor residual spray, and long-lasting insecticidal nets. Evaluation periods were defined as pre-intervention (January 2000 to August 2003), ACT-only (September 2003 to December 2005) and ACT plus vector control (2006–2015).
Findings
After accounting for climate, seasonality, diagnostic testing rates, and outpatient attendance, average monthly incidence of confirmed malaria showed no trend over the pre-intervention period 2000–2003 (incidence rate ratio (IRR) 0.998, 95% CI 0.995–1.000). During the ACT-only period (2003–2005), the average monthly malaria incidence rate declined compared to the pre-intervention period, showing an overall declining trend during the ACT-only period (IRR 0.984, 95% CI 0.978–0.990). There was no intercept change at the start of the ACT-only period (IRR 1.081, 95% CI 0.968–1.208), but a drop in intercept was identified at the start of the ACT plus vector control period (IRR 0.683, 95% CI 0.597–0.780). During the ACT plus vector control period (2006–2015), the rate of decline in average monthly malaria incidence slowed compared to the ACT-only period, but the incidence rate continued to show an overall slight declining trend during 2006–2015 (IRR 0.993, 95% CI 0.992–0.994).
Interpretation
This study presents a rigorous approach to the use of HMIS data in evaluating the impact of malaria control interventions. Evidence is presented for a rapid decline in malaria incidence during the period of ACT roll out compared to pre-intervention, with a rapid drop in malaria incidence following introduction of vector control and a slower declining incidence trend thereafter.
Keywords: Malaria, Impact evaluation, Zanzibar, Health management information systems (HMIS)
Highlights
-
•
Health Information Management System (HMIS) data can be used successfully in rigorous impact evaluation analyses.
-
•
An interrupted time series model identified a rapid decline in malaria incidence in Zanzibar after introduction of ACT.
-
•
The low incidence was sustained with both ACT and vector control, with continued slight declining trend during 2006-2015.
Research in Context
Evidence Before This Study
Following unprecedented investment and scale-up in malaria control interventions, many countries have experienced substantial reductions in malaria burden. Impact evaluations provide valuable evidence to support advocacy for continued and accelerated investment in malaria control interventions, but probabilistic evaluations are often not feasible in settings where interventions were scaled up to all areas at risk, with no control areas. Evolving improvements in availability and access to parasitological diagnosis of malaria at health facilities and improvements in health management information systems (HMIS) make routine data collected at health facilities increasingly valuable and relevant for impact evaluations. To seek examples of previous evaluations using routine HMIS data to estimate impact of interventions on malaria burden, we searched PubMed using the terms “malaria (incidence OR cases) AND (health information system OR surveillance system OR facility records OR hospital admission OR routine surveillance data) AND (impact OR evaluat*)”. We applied no language or publication date restrictions. Most previous evaluations of the impact of universally applied malaria control interventions that used routine HMIS data to assess changes in burden either restricted analysis to a small number of sentinel facilities or short time periods, relied on simple pre-post intervention comparisons, or struggled to adequately account for potential confounders and contextual factors.
Added Value of This Study
Few examples exist in the literature of malaria impact evaluations effectively using routine HMIS data from a large number of facilities to assess the impact of interventions over periods of greater than 5 years, particularly for settings where the program successfully transitioned from controlling malaria to eliminating malaria over the evaluation period. Our analysis has attempted to control for a range of potential confounding factors, including temperature and rainfall anomalies, increasing access to parasitological diagnosis, changes in reporting practices, and access to healthcare.
Implications of the Available Evidence
We propose that our methods can be adapted to conduct robust impact evaluations in other countries using existing HMIS data. Particularly in low malaria transmission settings, where nationally-representative surveys are under-powered or become prohibitively expensive to assess spatial and temporal changes in malaria prevalence, there is an urgent need to identify and use alternative data sources, more sensitive indicators, and improved analytic approaches to demonstrate continued impact of interventions.
Alt-text: Unlabelled Box
1. Outstanding Questions
The interrupted time series approach presented here assumes that interventions are applied at a consistent level and effect during the intervention time period, which may not be appropriate in contexts where interventions are increasingly targeted sub-nationally or where development of insecticide resistance reduces the effect of vector control (see Box 1). Additional research is needed to explore impact evaluation designs that are suited to these types of highly targeted program, and to account for endogeneity in intervention targeting.
Box 1. What is an Interrupted Time Series?
Interrupted time series is a study design in which a string of consecutive observations is interrupted by the imposition of a treatment/intervention to determine if the slope or intercept of the series changes as a result of the treatment/intervention [11]. Interrupted time series is a type of quasi-experimental study, which can be used to estimate causal effects using observational data [12]. Segmented regression is an analytical approach that is used as part of ITS designs, which enables fitting of regression lines to each ‘segment’ of the time series, for example, before and after the introduction of an intervention [13]. Each segment therefore has a level (the value at the start of the time series segment, equivalent to intercept for the first segment) and a trend. ITS designs typically include changes in level, changes in trend, or changes in both trend and level between each ‘segment’ [11]. The most appropriate ITS design should be hypothesized a priori, including if effects of interventions are expected to be immediate or delayed, permanent or time-limited [14]. In the current study in Zanzibar, we have assumed that ACT and vector control interventions would have an immediate effect on both the level and trend of malaria incidence, and that these effects would persist over time (assuming continued drug and insecticide susceptibility). Alternative scenarios and models, together with modelled counterfactuals presenting estimated cases averted are presented in the Supplementary Information.
Alt-text: Box 1
2. Background
Malaria control programs have seen rapid increases in intervention coverage following unprecedented funding support since 2000. Zanzibar has substantially reduced the burden of malaria over this period, and is now targeting malaria elimination. Interventions applied in Zanzibar included effective treatment using artemisinin-based combination therapy (ACT) and appropriate vector control; primarily long-lasting insecticidal nets (LLINs) and indoor residual spraying (IRS). Access to prompt parasitological diagnosis of malaria has been improved through use of rapid diagnostic tests (RDTs). As other countries strive to replicate Zanzibar's transition from malaria control to elimination, there is considerable interest in further defining the contributions of the interventions applied in Zanzibar to the observed decline in malaria burden.
Interventions in Zanzibar were applied with the aim of universal coverage. The lack of a contemporaneous control group precludes directly measuring what would have happened had malaria intervention scale-up never occurred; therefore probabilistic evaluations comparing intervention and control areas are not feasible [1]. An alternative approach is to assess the plausibility that reductions in malaria-related health outcomes can be attributed to malaria intervention scale-up, while simultaneously considering the contextual factors known to affect the impact indicator(s) of interest [2], [3].
Routinely collected health facility-based malaria indicators in Health Management Information System (HMIS) data have the potential to be used for impact evaluation [4], particularly where parasitological diagnosis of malaria is available at all levels of the health system. HMIS data are increasingly valuable in low malaria transmission settings such as Zanzibar, where periodic national cross-sectional surveys may not be adequately powered to assess trends in low-level malaria parasite prevalence over time [5]. However, HMIS data are infrequently used to target interventions or to provide rigorous evidence of program effectiveness due to concerns about data quality and potential for bias [4], [6]. With the inclusion of surveillance as a core intervention in the 2016–2030 Global Technical Strategy for Malaria [7], it is possible that improvements in HMIS data completeness, timeliness, quality and use will accelerate.
Zanzibar achieved large reductions in malaria burden following intensive interventions in the 1960s, but transmission resurged following withdrawal of funding to support maintenance of intervention coverage [8]. An impact evaluation in Zanzibar in 2007 used survey, health facility and demographic data for descriptive analyses in a plausibility framework [9]. A subsequent analysis focused on routine data from six inpatient facilities, using a segmented regression approach where impact was defined as the ratio between observed and counterfactual indicator estimates [10]. While these evaluations provide encouraging indications that interventions were associated with declines in malaria burden in Zanzibar, they were limited by relatively short time periods, small spatial scale, and analysis that did not fully explore potential for confounding. This study presents estimates of the impact of interventions on malaria case incidence at health facilities in Zanzibar from 2000 to 2015.
3. Methods
3.1. Study Site
Zanzibar is a collection of islands in the Indian Ocean, approximately 40 km from mainland Tanzania (Fig. 1). The total population of Zanzibar was estimated to be 1·3 million in the 2012 census. Zanzibar has historically experienced perennial malaria transmission, with seasonal peaks following rainfall from March to May and October to December. Malaria in Zanzibar is primarily caused by Plasmodium falciparum. Small numbers of P. vivax infections have been identified historically by microscopy, and more recently using molecular methods [15], but it is not fully understood if these are autochthonous or imported infections.
Fig. 1.
Map of Pemba and Unguja Islands, Zanzibar, with district boundaries (left). Locater map with Tanzania Mainland and Zanzibar shaded (right).
All primary health care units (PHCUs) providing outpatient or inpatient services in Zanzibar were eligible to be included. Access to health services for diagnosis is high in Zanzibar; a 2015–16 survey estimated that 72·7% of children under five years of age with recent fever were taken to a health facility, pharmacy or private health provider [16]. RDTs were initially introduced at facilities in Central, South, Wete and Micheweni districts in 2004 with Médecins sans Frontières-Spain assistance [17], [18]. A subsequent scale-up led to all PHCUs having access to confirmatory diagnosis from the start of 2007.
3.2. Health Facility Data
Facility-level, monthly aggregate outpatient department (OPD) data were extracted from existing electronic HMIS reports between January 2000 and December 2015 for all public PHCUs. Available indicators included total outpatient attendance, malaria diagnostic tests conducted, and total parasitologically confirmed (by microscopy or RDT) malaria cases. HMIS data were aggregated at the district-level by month. District population data were compiled from censuses in August 2002 and August 2012, then estimated by linear interpolation and extrapolation for remaining months.
3.3. Malaria Control Interventions
In November 2001 Zanzibar changed the recommended first- and second-line treatment for uncomplicated malaria from chloroquine and sulfadoxine-pyrimethamine to ACT (artesunate-amodiaquine). The new ACT policy was implemented from September 2003 [9].
Insecticide-treated mosquito nets (ITNs) were promoted in Zanzibar from 2004, but ownership and use of ITNs remained low until a policy of mass distribution was adopted in 2006 [9]. The first long-lasting insecticide-treated net (LLIN) distribution in February 2006 targeted pregnant women and children under five years, with subsequent mass LLIN campaigns in 2008–9, 2012 and 2016 targeting all-ages. Indoor residual spraying commenced in January 2006 using a pyrethroid (ICON-CS). The IRS insecticide changed to a carbamate (bendiocarb) in 2011 and an organophosphate (pirimiphos methyl) in 2014 [19]. During 2008–2011 a blanket approach was used where all shehias were sprayed. From 2012 onwards IRS was targeted to the highest incidence shehias. The Urban district (Stone Town) was excluded from IRS campaigns from 2012 onwards.
Indoor residual spray and LLINs were considered as a single ‘vector control’ intervention at a consistent level, commencing in January 2006. Further data describing survey-derived estimates of vector control coverage are presented in Supplementary Information.
For the purpose of this study, we define January 2000 to August 2003 as the “pre-intervention period”, September 2003 to December 2005 as the “ACT-only period”, and January 2006 to December 2015 as the “ACT plus vector control period”. The mean incidence of confirmed malaria across Zanzibar in the first and last month of each intervention period is shown in Table 1.
Table 1.
Incidence of confirmed malaria per 100,000 population across Zanzibar during the month at the start and end of each intervention period. Note that May to July is generally considered to be the seasonal peak of malaria transmission.
| Study period | Month | Mean incidence per 100,000 population across Zanzibar |
|---|---|---|
| Start of baseline | January 2000 | 111 |
| End of baseline | August 2003 | 101 |
| Start of ACT-only period | September 2003 | 106 |
| End of ACT-only period | December 2005 | 38 |
| Start of ACT plus vector control period | January 2006 | 20 |
| End of ACT plus vector control period | December 2015 | 12 |
3.4. Covariate Data
Estimates of total monthly rainfall were obtained from the USGS Famine Early Warning System African Data Dissemination Service from 2000 to 2015 [20]. The monthly enhanced vegetation index (EVI) at 1 km resolution was obtained from MODIS satellite data for 2000–2015 [21]. Daily mean, maximum and minimum daytime and nighttime land surface temperature were sourced from MODIS data for 2000–2015 [21]. Temperature and rainfall raster data were summarized to district-month means, and monthly anomalies calculated for the each district-month. Further details are available in Supplementary Information. District-level covariates derived from monthly HMIS data included number of facilities reporting data, total all-cause outpatient attendance and proportion of all-cause OPD attendees receiving a malaria parasitological test.
3.5. Statistical Analysis
Statistical analysis was conducted in Stata 14·0. An interrupted time series (ITS) approach was used to assess the change in trend of monthly malaria incidence following the introduction of ACTs in September 2003 and vector control in January 2006 [13]. Monthly confirmed malaria case count by district served as the primary outcome estimated in models, fit to district-level covariates in a random-effect negative binomial regression model, with a district population offset. ITS models require binary terms for presence or absence of each intervention, and time in months since intervention was introduced [13], [14]. Briefly, the time variables (time in months since the start of the study, months since ACT introduction, months since vector control introduction) were used to estimate the baseline trend over the entire period 2000–2015, the change in trend in ACT-only period and change in trend in vector control period, respectively, Binary terms for presence/absence of each intervention were used to estimate intercept change after introduction of each intervention [13], [14]. Model predictions were generated using non-linear combination of coefficients (stata nlcom). Wete and North B Districts did not have access to parasitological diagnosis until after ACTs were introduced, therefore are excluded from models due to lack of confirmed malaria count data in the pre-intervention period. Environmental data were tested for inclusion in ITS models using a range of biologically-plausible lags. Combinations of covariates hypothesized to influence malaria transmission in Zanzibar were tested for inclusion in ITS models. Akaike's Information Criterion (AIC), mean square error and visual inspection of residuals were used to inform model selection. Model specification and selection is detailed further in the Supplementary Information. Modelled counterfactuals and estimates of number of cases averted by the interventions are presented in Supplementary Information.
The ITS model included a priori terms of total all-cause outpatient attendance, number of facilities reporting data and proportion of all outpatients who received a parasitological test to control for changes in health facility attendance, reporting and access to parasitological diagnosis, respectively. The best fitting ITS model also included one month lags of EVI anomaly and anomaly in minimum daytime temperature, and two month lags of the rainfall anomaly and of the anomaly in minimum nighttime temperature. A square-root transformation of the previous month's malaria case count was included to account for autocorrelation, and a categorical month variable to account for seasonality. A binary dummy variable was included for Urban and West districts, with another dummy variable for South district.
3.6. Study Ethics
This study involved secondary analysis of individually unidentifiable aggregate surveillance data routinely collected at health facilities, and received approval from the U.S. Centers for Disease Control as non-human subjects research (category II.C).
4. Results
4.1. Descriptive Analysis
A total of 129 PHCUs in 8 districts were included in analysis. Of the 87 facilities operating in January 2000, 80 remained operational throughout 2000–2015, and 7 closed during the study period. An additional 42 facilities opened or began reporting data during the study period, with 35 of these remaining open until the study end. From a total 19,882 expected facility-months of data, 327 (1.6%) facility-months were completely missing, and all-cause OPD attendance data was missing from 1.7% facility-months. Further description of missingness over time is presented in Supplementary Information.
17% of facilities operating during 2000–2005 had access to parasitological diagnosis, increasing to 39% of operational facilities by the end of 2006. From the start of 2007, 100% of PHCUs reporting data had access to confirmatory diagnosis. Confirmed malaria case count was missing from 3.0% of facility-months where confirmatory diagnostics were expected to be available at the facility. Due to inconsistencies in the approaches used to report clinical malaria, it is not possible to determine the proportion of suspected malaria cases that received confirmatory testing. However, Fig. 2 shows stepped increases in the number of individuals being tested in 2006 and 2007, and all-cause outpatient attendance, which remained stable over the study period.
Fig. 2.
Total monthly all-cause outpatient department (OPD) attendance, total tested for malaria using microscopy or rapid diagnostic test, and total of confirmed malaria cases by island (Unguja and Pemba), plotted on a log scale.
Monthly incidence of confirmed malaria among all ages varied by district (Fig. 3). Micheweni District had the highest observed incidence (568 per 100,000) in June 2000. North A, Urban and West Districts reported low incidence of confirmed malaria (< 100 per 100,000) throughout the study period. Seasonal peaks in incidence can be observed in all districts across all years, generally following the March to May seasonal rains. Malaria became more strongly seasonal during the study period (Supplementary Information).
Fig. 3.
Monthly incidence of confirmed malaria per 100,000 population reported to HMIS, displayed by district. Wete and North B districts, although excluded from the ITS model, are included for the period in which parasitological diagnosis was available at facilities (from January 2006 in Wete, from January 2007 in North B). Vertical reference lines indicate the timing of interventions: introduction of ACTs as first line malaria treatment in September 2003 (leftmost vertical line), and large-scale introduction of IRS and LLINs in January 2006 (rightmost vertical line).
4.2. Impact of Interventions Estimated by ITS
The pre-intervention trend (before ACT or vector control) in confirmed monthly malaria incidence showed no change over time (IRR 0.998, 95% CI 0.995, 1.000, p = 0.089) (Table 2). There was no significant intercept change when transitioning from pre-intervention period to the start of ACT-only period (IRR 1.081, 95% CI 0.968, 1.208, p = 0.169). The malaria incidence rate declined in the ACT-only period compared to the pre-intervention trend (IRR 0.987, 95% CI 0.980, 0.993, p < 0.001), and showed a the overall significantly declining trend during the ACT-only period, with an average 1·6% reduction in incidence rate each month (IRR 0.984, 95% CI 0.978, 0.990, p < 0.001) after accounting for the baseline trend. A negative change in the intercept was seen when transitioning from the ACT-only to ACT plus vector control period (IRR 0.683, 95% CI 0.597, 0.780, p < 0.001). During the ACT plus vector control period, the rate of decline in confirmed monthly malaria incidence slowed slightly compared to the trend during the ACT-only period (IRR 1.009, 95% CI 1.003, 1.020, p = 0.006), but the monthly incidence of confirmed malaria during the ACT plus vector control period continued to show an overall decline (IRR 0.993, 95% CI 0.992, 0.994, p < 0.001), with average reduction of 0.7% in incidence rate each month during 2006–2015. Full model output and coefficients are presented in Supplementary Information, along with explanation of coefficients for change in trend between each intervention period, and overall trend during each period.
Table 2.
Selected terms from the final interrupted time series model, describing baseline trend in confirmed malaria incidence, and change in level and trend of confirmed malaria incidence in Zanzibar for each intervention period, after accounting for confounding variables. Both the model coefficients and exponentiated coefficients (incidence rate ratio, IRR) are reported. Additional indicators describing trends during periods of specific intervention implementation are estimated using an alternative specification of the ITS model (details in Supplementary Information), but can also be estimated by combining coefficients for baseline and intervention periods.
| ITS model: changes in level and trend | Coef | IRR | IRR 95% CI | P |
|---|---|---|---|---|
| Trend in confirmed malaria incidence (2000–2015) | − 0.0025 | 0.9975 | 0.9947, 1.0004 | 0.089 |
| Intercept change at ACT introduction (September 2003) | 0.0779 | 1.0810 | 0.9676, 1.2078 | 0.169 |
| Change in trend after ACT introduction (ACT period) | − 0.0135 | 0.9866 | 0.9801, 0.9932 | < 0.001 |
| Intercept change at vector control introduction (January 2006) | − 0.3821 | 0.6825 | 0.5968, 0.7804 | < 0.001 |
| Change in trend after vector control introduction (ACT plus vector control period) | 0.0087 | 1.0087 | 1.0025, 1.0150 | 0.006 |
| ITS model: estimated trend during each period | ||||
| Trend in confirmed malaria incidence during period of ACT implementation (Sept 2003–Dec 2005) | − 0.0160 | 0.9842 | 0.9782, 0.9902 | < 0.001 |
| Trend in confirmed malaria incidence during period of ACT plus vector control implementation (2006–2015) | − 0.0073 | 0.9927 | 0.9916, 0.9939 | < 0.001 |
ITS model predictions by district are plotted against observed incidence in Fig. 4. The model tended to under-predict in Micheweni in the pre-intervention period, and also to under-predict the seasonal peaks in incidence observed in Central and South Districts during the ACT plus vector control period. The observed malaria incidence and model prediction aggregated across Zanzibar is shown in Fig. 5.
Fig. 4.
District-level plots of the observed confirmed malaria incidence per month (gray line), predictions of incidence from the final interrupted time series model (blue line) and 95% confidence interval of the model prediction (pale blue shading). Vertical reference lines indicate the timing of interventions: introduction of ACTs as first line malaria treatment in September 2003 (leftmost vertical dotted line), and large-scale introduction of IRS and LLINs in January 2006 (rightmost vertical dotted line).
Fig. 5.
Zanzibar-level plots of the observed confirmed malaria incidence per month (gray line), predictions of incidence from the final interrupted time series model (blue line) and 95% confidence interval of the model prediction (pale blue shading). Vertical reference lines indicate the timing of interventions: introduction of ACTs as first line malaria treatment in September 2003 (leftmost vertical dotted line), and large-scale introduction of IRS and LLINs in January 2006 (rightmost vertical dotted line).
5. Discussion
This study evaluated the impact of ACT as first line treatment and vector control scale-up on malaria case incidence at primary health care units in Zanzibar, in a context of expanding access to parasitological diagnosis. By generating interrupted time series models considering climate data, changing parasitological testing rates, and temporal autocorrelation, we estimated that substantial reductions in confirmed malaria incidence occurred in Zanzibar which can be plausibly attributed to the widespread scale-up of ACTs, with some evidence of further reductions in the already low incidence during the period of LLIN and IRS scale up.
These findings concur with previous evaluations in Zanzibar that found a declining burden of malaria following introduction of ACTs and LLINs, but we add further weight to this conclusion with rigorous analysis of a much larger dataset [9], [10]. Bhattarai et al. presented one of the first studies exploring the public health impact of introduction of ACTs and scale-up of insecticide-treated nets, using a combination of cross-sectional survey, outpatient and inpatient facility data [9]. However, facility data were limited to annual summary statistics from one district, with assessment of linear relationships between monthly rainfall and malaria outcome indicators. Aregawi et al. focused on six inpatient health facilities in Zanzibar, using ITS to compare outpatient and inpatient indicators in 2008 to a counterfactual generated by continuing the average 1999–2003 trend [10]. While rainfall and temperature trends over the Aregawi et al. study period were described, these potential confounders were not included in the model, potentially introducing bias into impact estimates [4].
The current analysis improves on these previous evaluations in Zanzibar. Firstly, data have been collated over a wider spatial and temporal scale, including all PHCUs from eight of Zanzibar's ten districts over 16 years, representing a fuller picture of malaria burden changes over the period of intervention scale-up and transition to an elimination setting. Our models incorporate satellite-derived environmental data to account for changes in malaria transmission intensity attributable to climate variations, estimates of outpatient visits and facility reporting and parasitological testing rates, and a categorical month variable to adjust for seasonality. This increases the plausibility that model-estimated impacts are attributable to the interventions.
Including multiple years' data prior to interventions is valuable in estimating the pre-intervention malaria trends after accounting for confounding variables, which improves validity of estimates of changes in trend from the baseline after introduction of the interventions. Finally, our analysis enables exploration of the impact of ACT and vector control over an extended period, when there were risks of erosion in intervention impact due to degradation and loss of LLINs, resistance to IRS insecticides and artemisinin resistance, but the combination of interventions nevertheless was able to sustain reductions in malaria incidence.
The current study could have been improved by inclusion of district-level intervention coverage, enabling a district-level dose–response analysis which could have accounted for any variation in intervention coverage over time or between districts, or the assessment of the differential impact of LLINs and IRS [22]. While the ITS methodology does allow for multiple breakpoints reflecting changes in interventions or coverage (e.g. IRS insecticide changes), testing a large number of breakpoints increases the plausibility of changes in outcome being attributable to other potentially confounding factors [11]. As a result, a change from blanket IRS to targeted spraying in only the highest incidence shehias in 2012 was not accounted for in the ITS design. Furthermore, it was not possible to distinguish the effect of large-scale introduction of RDTs in 2006 and 2007 from the impact of vector control, which was scaled up at the same time. Previous studies in Zanzibar have acknowledged that reductions in malaria cases may reflect some transmission blocking effect of artemisinin derivatives through its gametocytocidal activity [9]. Considering the limitations of the ITS study design, we are unable to fully assess or quantify the potential confounding effect of gametocytocidal properties of artemisinins in Zanzibar. Complete facility geographical coordinates were not available, precluding assessment of small-scale variations in intervention impact over this study period. In addition, facilities from two districts were excluded from the analysis and a small proportion of monthly HMIS reports from facilities were missing, therefore this does not represent a fully comprehensive assessment of malaria burden in Zanzibar over the 16 year evaluation period. A sensitivity analyses could further explore if ITS results are influenced by methods used to estimate missing HMIS data: by carrying forward the count from the previous month, using estimate from the subsequent month, or calculating the average number of cases from immediately before and after the missing data. While we present estimates of cases averted due to ACT and vector control in the Supplementary Information, the approach used was limited by the need to either adopt a level change only ITS model, or assumptions that decreasing trends would continue indefinitely rather than reach a new equilibrium.
For the current study, it was not possible to access individual-level admission data, to test whether an age-shift in severe malaria cases occurred due to the reduced malaria transmission in Zanzibar. The case-based surveillance system used since 2012 in Zanzibar will facilitate future in-depth investigation of malaria patients.
Increased access to parasitological testing as a result of RDT availability in Zanzibar from 2006 onwards has the potential to bias confirmed malaria incidence in the intervention period upwards, since facilities previously only able to report clinical malaria were now able to report confirmed cases. To address this, parasitological testing rate was included in models. Access to healthcare in Zanzibar is higher than on mainland Tanzania, and has been considered by inclusion of all-cause OPD attendance in models. However, there is some evidence from Kenya that treatment seeking from health facilities increases when efficacious treatment is available [23], which may have biased our impact estimates downwards. The linear extrapolation method used to estimate monthly district population based on 10-year census data does not account for seasonal fluctuations in population, and assumes a consistent linear growth in population. While errors in estimation of catchment population size can drastically alter analysis results [24], [25], in many settings it remains challenging to accurately define population size for HMIS data, particularly in defining facility catchment populations.
The use of confirmed malaria case counts derived from surveillance data provides a more direct indicator of malaria intervention impact than approaches using all-cause child mortality (ACCM) as the primary outcome indicator. Changes in ACCM could be attributable to a wide range of factors, necessitating inclusion of a large number of contextual factors in evaluations using ACCM [26]. While contextual factors should still be considered in analyses using malaria surveillance data, these are narrower in scope and commonly available from HMIS indicators, program data and remotely sensed environmental data.
Impact evaluations in individual countries have demonstrated substantial reductions in confirmed malaria incidence attributable to interventions such as LLINs [27], [28]. Based on models evaluating the impact of malaria control interventions across sub-Saharan Africa, LLINs are estimated to have accounted for 68% of declines in annual P. falciparum prevalence, while ACT and IRS accounted for 19% and 13% of reductions, respectively [29]. However, this finding is driven partly by the scale and duration of implementation for each intervention, and is not a direct comparison of intervention effectiveness [29]. The current study found that substantial declines in malaria incidence occurred during the period of ACT introduction, with incidence having fallen to 0.4 per 1000 population by the time of vector control introduction. With incidence already very low in 2006, further reductions in incidence during the period of IRS and LLINs were modest, but showed evidence of a continued decline in incidence. Previous small scale surveys have documented an impact of both ACT and vector control on malaria parasitemia prevalence in two districts over the same time period [9], [30].
Zanzibar is targeting elimination of locally acquired malaria by 2018, and plans to achieve this by maintaining high intervention coverage, effective treatment and parasitological diagnosis, and adding primaquine to the standard malaria treatment (commenced November 2016). Additional interventions include an expanded surveillance system with case-level resolution and real-time reporting [31], to allow for investigation of 100% of cases and reactive case detection, as well as national- and district-level coordination structures to facilitate effective data sharing and decision making on the path to elimination.
Zanzibar has achieved substantial reductions in burden of malaria between 2000 and 2015. The current analysis presents evidence that these reductions can be plausibly attributed to introduction of ACTs, as well as the expansion of vector control interventions, specifically IRS and LLINs. The use of routine HMIS data is appropriate for impact evaluations using an interrupted time series approach where relevant contextual factors and confounders are incorporated in models and considered in interpretation, as part of a plausibility evaluation. This approach may be appropriate for use in other settings where routine HMIS data are available from multiple years before and after scale-up of interventions using a universal coverage approach, but where district-level intervention data may not be available.
Authors' Contributions
All authors contributed to study concept and design. RAA and ABe conducted data analysis, with input on methods and interpretation from all other authors. RAA prepared the manuscript, with revisions made by all other authors.
Role of the Funding Source
Co-authors employed by the U.S. President's Malaria Initiative and U.S. Centers for Disease Control and Prevention had roles in the conceptualization and design of this study, interpretation of findings and preparation of the manuscript. The opinions expressed herein are those of the authors and do not necessarily reflect the views of the President's Malaria Initiative or the Centers for Disease Control and Prevention. As corresponding author, RAA had full access to all data in the study, and had final responsibility for the decision to submit for publication.
Funding
U.S. Agency for International Development, President's Malaria Initiative.
Declaration of Competing Interest
The authors declare no conflicts of interest.
Acknowledgements
This research was supported by the President's Malaria Initiative (PMI) through the United States Agency for International Development (USAID) under the terms of MEASURE Evaluation cooperative agreement AIDOAA-L-14-00004. MEASURE Evaluation is implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill, in partnership with ICF International; John Snow, Inc.; Management Sciences for Health; Palladium; and Tulane University. The findings and conclusions presented in this manuscript are those of the authors and do not necessarily reflect the official position of the U.S. Centers for Disease Control and Prevention, PMI, USAID, or the United States Government. We are grateful to George Greer, Lynn Paxton, René Salgado, and Steven Yoon for their contributions to this study. We also thank Christian Geneus for assistance in preparing satellite-derived environmental data.
Footnotes
Supplementary data to this article can be found online at https://doi.org/10.1016/j.eclinm.2019.05.011.
Appendix A. Supplementary material
Supplementary material
References
- 1.Ye Y., Eisele T.P., Eckert E. Framework for evaluating the health impact of the scale-up of malaria control interventions on all-cause child mortality in Sub-Saharan Africa. Am J Trop Med Hyg. 2017;97(3_Suppl):9–19. doi: 10.4269/ajtmh.15-0363. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Victora C.G., Habicht J.P., Bryce J. Evidence-based public health: moving beyond randomized trials. Am J Public Health. 2004;94(3):400–405. doi: 10.2105/ajph.94.3.400. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Rowe A.K., Steketee R.W., Arnold F. Viewpoint: evaluating the impact of malaria control efforts on mortality in sub-Saharan Africa. Trop Med Int Health. 2007;12(12):1524–1539. doi: 10.1111/j.1365-3156.2007.01961.x. [DOI] [PubMed] [Google Scholar]
- 4.Ashton R.A., Bennett A., Yukich J., Bhattarai A., Keating J., Eisele T.P. Methodological considerations for use of routine health information system data to evaluate malaria program impact in an era of declining malaria transmission. Am J Trop Med Hyg. 2017;97(3_Suppl):46–57. doi: 10.4269/ajtmh.16-0734. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.A research agenda for malaria eradication: monitoring, evaluation, and surveillancePLoS Med. 2011;8(1) doi: 10.1371/journal.pmed.1000400. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Rowe A.K., Kachur S.P., Yoon S.S., Lynch M., Slutsker L., Steketee R.W. Caution is required when using health facility-based data to evaluate the health impact of malaria control efforts in Africa. Malar J. 2009;8:209. doi: 10.1186/1475-2875-8-209. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.World Health Organization . World Health Organization; Geneva: 2015. Global technical strategy for malaria 2016–2030. [Google Scholar]
- 8.Schwartz E., Pener H., Issa S.M., Golenser J. An overview of the malaria situation in Zanzibar. J Community Health. 1997;22(1):33–44. doi: 10.1023/a:1025194823592. [DOI] [PubMed] [Google Scholar]
- 9.Bhattarai A., Ali A.S., Kachur S.P. Impact of artemisinin-based combination therapy and insecticide-treated nets on malaria burden in Zanzibar. PLoS Med. 2007;4(11) doi: 10.1371/journal.pmed.0040309. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Aregawi M.W., Ali A.S., Al-Mafazy A.W. Reductions in malaria and anaemia case and death burden at hospitals following scale-up of malaria control in Zanzibar, 1999–2008. Malar J. 2011;10:46. doi: 10.1186/1475-2875-10-46. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Shadish W.R., Cook T.D., Campbell D.T. Wadsworth Cengage Learning; Belmont, CA: 2002. Experimental and quasi-experimental designs for generalized causal inference. [Google Scholar]
- 12.Kontopantelis E., Doran T., Springate D.A., Buchan I., Reeves D. Regression based quasi-experimental approach when randomisation is not an option: interrupted time series analysis. BMJ. 2015;350:h 2750. doi: 10.1136/bmj.h2750. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Wagner A.K., Soumerai S.B., Zhang F., Ross-Degnan D. Segmented regression analysis of interrupted time series studies in medication use research. J Clin Pharm Ther. 2002;27(4):299–309. doi: 10.1046/j.1365-2710.2002.00430.x. [DOI] [PubMed] [Google Scholar]
- 14.Bernal J.L., Cummins S., Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int J Epidemiol. 2017;46(1):348–355. doi: 10.1093/ije/dyw098. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Cook J., Xu W., Msellem M. Mass screening and treatment on the basis of results of a plasmodium falciparum-specific rapid diagnostic test did not reduce malaria incidence in Zanzibar. J Infect Dis. 2015;211(9):1476–1483. doi: 10.1093/infdis/jiu655. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Ministry of Health CD, Gender, Elderly and Children (MoHCDGEC) [Tanzania Mainland], Ministry of Health (MoH) [Zanzibar], National Bureau of Statistics (NBS), Office of the Chief Government Statistician (OCGS), ICF International . 2016. Tanzania demographic and health survey and malaria Indicator survey (TDHS-MIS) 2015–16. Dar es Salaam, Tanzania, and Rockville, Maryland, USA. [Google Scholar]
- 17.Msellem M.I., Mårtensson A., Rotllant G. Influence of rapid malaria diagnostic tests on treatment and health outcome in fever patients, Zanzibar: a crossover validation study. PLoS Med. 2009;6(4) doi: 10.1371/journal.pmed.1000070. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Médecins sans Frontières-Spain . 2006. Integrated health program focusing on malaria and childhood illnesses amongst the vulnerable population of Zanzibar archipelago. [Google Scholar]
- 19.Haji K.A., Thawer N.G., Khatib B.O. Efficacy, persistence and vector susceptibility to pirimiphos-methyl (Actellic 300CS) insecticide for indoor residual spraying in Zanzibar. Parasit Vectors. 2015;8:628. doi: 10.1186/s13071-015-1239-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.United States Geological Survey Famine early warning systems network. http://earlywarning.usgs.gov/fews/datadownloads/Continental%20Africa/Dekadal%20RFE
- 21.Moderate Resolution Imaging Spectroradiometer (MODIS) http://modis.gsfc.nasa.gov/
- 22.Bennett A., Yukich J., Miller J.M. A methodological framework for the improved use of routine health system data to evaluate national malaria control programs: evidence from Zambia. Popul Health Metr. 2014;12(1):30. doi: 10.1186/s12963-014-0030-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Burgert C.R., Bigogo G., Adazu K. Impact of implementation of free high-quality health care on health facility attendance by sick children in rural western Kenya. Trop Med Int Health. 2011;16(6):711–720. doi: 10.1111/j.1365-3156.2011.02752.x. [DOI] [PubMed] [Google Scholar]
- 24.Gething P., Atkinson P., Noor A., Gikandi P., Hay S., Nixon M. A local space-time kriging approach applied to a national outpatient malaria dataset. Comput Geosci. 2007;33(10):1337–1350. doi: 10.1016/j.cageo.2007.05.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Zu Erbach-Schoenberg E., Alegana V.A., Sorichetta A. Dynamic denominators: the impact of seasonally varying population numbers on disease incidence estimates. Popul Health Metr. 2016;14:35. doi: 10.1186/s12963-016-0106-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Eckert E., Florey L.S., Tongren J.E. Impact evaluation of malaria control interventions on morbidity and all-cause child mortality in Rwanda, 2000–2010. Am J Trop Med Hyg. 2017;97(3_Suppl):99–110. doi: 10.4269/ajtmh.17-0281. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Karema C., Aregawi M.W., Rukundo A. Trends in malaria cases, hospital admissions and deaths following scale-up of anti-malarial interventions, 2000–2010, Rwanda. Malar J. 2012;11:236. doi: 10.1186/1475-2875-11-236. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Aregawi M., Lynch M., Bekele W. Time series analysis of trends in malaria cases and deaths at hospitals and the effect of antimalarial interventions, 2001–2011, Ethiopia. PLoS One. 2014;9(11) doi: 10.1371/journal.pone.0106359. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Bhatt S., Weiss D.J., Cameron E. The effect of malaria control on Plasmodium falciparum in Africa between 2000 and 2015. Nature. 2015;526(7572):207–211. doi: 10.1038/nature15535. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Bjorkman A., Shakely D., Ali A.S. From high to low malaria transmission in Zanzibar-challenges and opportunities to achieve elimination. BMC Med. 2019;17(1):14. doi: 10.1186/s12916-018-1243-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Cressman G., McKay M., Al-Mafazy A.W. Using mobile technology to facilitate reactive case detection of malaria. Online J Public Health Inform. 2017;9(1):e68. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplementary material