Skip to main content
eLife logoLink to eLife
. 2015 Dec 29;4:e09672. doi: 10.7554/eLife.09672

Coverage and system efficiencies of insecticide-treated nets in Africa from 2000 to 2017

Samir Bhatt 1, Daniel J Weiss 1, Bonnie Mappin 1, Ursula Dalrymple 1, Ewan Cameron 1, Donal Bisanzio 1, David L Smith 1,2,3, Catherine L Moyes 1, Andrew J Tatem 3,4,5, Michael Lynch 6, Cristin A Fergus 6, Joshua Yukich 7, Adam Bennett 8, Thomas P Eisele 7, Jan Kolaczinski 9, Richard E Cibulskis 6, Simon I Hay 3,10,11, Peter W Gething 1,*
Editor: Catherine Kyobutungi12
PMCID: PMC4758960  PMID: 26714109

Abstract

Insecticide-treated nets (ITNs) for malaria control are widespread but coverage remains inadequate. We developed a Bayesian model using data from 102 national surveys, triangulated against delivery data and distribution reports, to generate year-by-year estimates of four ITN coverage indicators. We explored the impact of two potential 'inefficiencies': uneven net distribution among households and rapid rates of net loss from households. We estimated that, in 2013, 21% (17%–26%) of ITNs were over-allocated and this has worsened over time as overall net provision has increased. We estimated that rates of ITN loss from households are more rapid than previously thought, with 50% lost after 23 (20–28) months. We predict that the current estimate of 920 million additional ITNs required to achieve universal coverage would in reality yield a lower level of coverage (77% population access). By improving efficiency, however, the 920 million ITNs could yield population access as high as 95%.

DOI: http://dx.doi.org/10.7554/eLife.09672.001

Research Organism: Human

eLife digest

Malaria is a major cause of death in many parts of the world, especially in sub-Saharan Africa. Recently, there has been a renewed emphasis on using preventive measures to reduce the deaths and illnesses caused by malaria. Insecticide-treated nets are the most prominent preventive measure used in areas where malaria is particularly common. However, despite huge international efforts to send enough nets to the regions that need them, the processes of delivering and distributing the nets are inefficient. This problem is compounded by the fact that little information is available on how many nets people actually own and use within each country.`

Bhatt et al. have now created a mathematical model that describes the use and distribution of nets across Africa since 2000. This is based on data collected from national surveys and reports on the delivery and distribution of the nets. The model estimates that in 2013, only 43% of people at risk of malaria slept under a net. Furthermore, 21% of new nets were allocated to households that already had enough nets, an inefficiency that has worsened over the years. Nets are also lost from households much more rapidly than previously thought.

It’s currently estimated that 920 million additional nets are required to ensure that everyone at risk from malaria in Africa is adequately protected. However, Bhatt et al.’s model suggests that given the current inefficiencies in net distribution, the extra nets would in reality protect a much smaller proportion of the population. Taking measures to more effectively target the nets to the households that need them could improve this coverage level to 95% of the population. The next challenge is to devise distribution strategies to send nets to where they are most needed.

DOI: http://dx.doi.org/10.7554/eLife.09672.002

Introduction

Insecticide-treated nets (ITNs), which comprise conventional (cITNs) and long-lasting insecticidal nets (LLINs), are the single most widely used intervention for malaria control in Africa, proven to significantly reduce morbidity and mortality via direct protection and community-wide reductions in transmission (Lim et al., 2011; Lengeler and Lengeler, 2004; Eisele et al., 2010; Killeen et al., 2007). The World Health Organization (WHO) promotes a target of universal coverage for all populations at risk with either ITNs or indoor residual spraying (IRS), with the former representing the primary vector control tool in nearly all endemic African countries (WHO, 2013a). The international community has invested billions of dollars in the provision of at least 700 million LLINs since 2004 (WHO, 2013a). While these investments have led to enormous scale up in population access to ITNs (Noor et al., 2009; Monasch et al., 2004), the target of universal coverage remains distant and millions of African households at risk remain unprotected (WHO, 2013a).

Bridging this gap is a key component of future strategies to reduce further the burden of malaria in Africa (WHO, 2014), and will require sustained commitment from donors, policy makers and national programmes. Central to these efforts is the capacity to monitor reliably current levels of ITN coverage in populations at risk and evaluate the systems that give rise to this coverage. This, in turn, enables progress towards international goals to be tracked and opportunities for efficiency gains to be identified. Such information is essential for evaluating the existing commodity and financing shortfalls and assessing future requirements if the target of universal coverage is to be achieved.

Modelling coverage

To facilitate standardised and comparable monitoring of ITN coverage through time, WHO and the Roll Back Malaria Monitoring and Evaluation Reference Group (RBM-MERG) has over the past decade defined a series of indicators to capture two different aspects of ITN coverage: access and use. Gold standard measurements of these indicators are provided by nationally representative household surveys such as Demographic and Health Surveys (DHS) (Measure, 2014), Multiple Indicator Cluster Surveys (MICS) (UNICEF, 2012), and Malaria Indicator Surveys (MIS) (RBM, 2014a). These surveys are carried out relatively infrequently, however, meaning they cannot be used directly for evaluating year-on-year coverage trends or for generating timely estimates of continent-wide coverage levels. In contrast, programmatic data such as the number of ITNs delivered and distributed within countries, while not describing coverage directly, are available for most countries and years (WHO, 2013a ). In a 2009 study, Flaxman and colleagues (Flaxman et al., 2010) used a compartmental modelling approach to link these programmatic and survey data, generating annual estimates of the two ITN indicators recommended at that time on access (% households with at least one ITN) and use (% children < 5 years old who slept under an ITN the previous night).

Since that study, there has been increasing recognition that a richer set of indictors is required to identify the complex nature of ITN coverage (Kilian et al., 2013). An intra-household 'ownership gap' may exist whereby many households with some nets may not have enough for one net between two occupants (the recommended minimum level of protection (WHO, 2013b). Similarly, a 'usage gap' may exist whereby individuals with access to a net do not sleep under it. In response, the measurement of two additional indicators was recommended: % households with at least one ITN for every two people and % population with access to an ITN within their household (assuming each net was used by two people) (RBM, UNICEF, WHO, 2013; RBM, 2011). In addition, the indicator on usage was extended to include the entire population rather than only children under 5 years old. This updated set of four indicators, used individually and in combination, has the potential to provide a nuanced picture of ITN access and use patterns that can directly guide operational decision making (Kilian et al., 2013). To achieve this, there is a need to develop modelling frameworks to allow all four to be tracked through time.

Evaluating efficiency

Countries have an ongoing struggle to maintain high LLIN coverage in the face of continuous loss of nets from households due to damage, repurposing, or movement away from target areas. In response, systems need to be responsive to emerging coverage gaps by ensuring nets are distributed to households that need them and avoiding over-allocation (i.e. distribution of nets to those that already have them). Together, the rate of net loss and the degree of over-allocation of new nets play a key role in determining how efficiently delivery to countries will translate into household coverage levels. These factors are not currently well understood but triangulation of survey and programmatic data allows new insights into both.

Estimating future needs

The WHO define universal access to ITNs on the basis that two people can share one net. Using the working assumptions of a 3-year ITN lifespan and a 1.8 person-per-net ratio (one-between-two but allowing for odd-numbered households), a simple calculation yields an indicative estimate of 150 million new nets required each year to provide universal coverage to an African population at risk of around 810 million (WHO, 2013a). To support country planning and donor application processes (RBM-HWG, 2014), a more elaborate needs assessment approach has been developed by the RBM Harmonization Working Group (RBM-HWG) and implemented by 41 of the 47 endemic African countries (RBM, 2014; Paintain et al., 2013). The tool takes into account the size and structure of national target populations, a 1.8 person-per-net ratio for mass campaigns, additional routine distribution mechanisms employed by countries, and volumes of previously distributed nets and their likely rates of loss through time. Countries have used these inputs to calculate requirements for new nets to achieve national coverage targets, leading to an estimated continent-wide need for 920 million ITNs over the 2014–2017 period (approximately 230 million per year) (RBM, 2014). This tool provides a transparent, intuitive and standardised mechanism for comparing forecasted needs against current financing levels and identifying likely shortfalls. However, calculated needs are sensitive to assumptions about how a given volume of new nets will translate into population coverage, and inefficiencies in the system such as such as over-allocation and rate of net loss are not accounted for explicitly in the current needs assessment exercise.

The purpose of this study is to define a new dynamic modelling approach, triangulating all available data on ITN delivery, distribution and coverage in sub-Saharan Africa in order to (i) provide validated and data-driven time-series estimates for all four internationally recommended ITN indicators; (ii) explore and quantify different aspects of system efficiency and how these contribute to reduced coverage levels; and (iii) estimate future LLIN needs to achieve universal access by 2017 under different efficiency scenarios and how these compare to existing needs assessment estimates.

Results

Net stock estimates

Figure 1A summarises the main inputs to and outputs from the stock-and-flow model for LLINs when aggregated at the continental level. Some 718 million LLINs have been delivered across the 40 endemic countries since their introduction in 2004. As is well documented (WHO, 2013a), annual LLIN deliveries increased year-on-year from 2004 to 2010, reaching 145 million in that year, but then declined dramatically in 2011 and 2012 to less than half that amount before rising again to 143 million in 2013 (green line). Taking into account rates of loss in households, these LLIN deliveries led to a continental net crop shown by the red line. We estimate that there were 252 million LLINs in sub-Saharan households by the end of 2013, with that net crop growing approximately linearly from 2004, with the exception of a slow-down resulting from the reduced supply of nets in 2011–2012. Figure 1B shows equivalent distribution and resulting net crop estimates for cITNs, which constituted nearly all ITNs prior to 2005 but diminished rapidly in importance following the introduction of LLINs thereafter.

Figure 1. Time series of ITN delivery, distribution, and estimated net crop in sub-Saharan households 2000–2013 for (A) LLINs and (B) cITNs. .

Figure 1.

Manufacturer data on deliveries were available for LLINs only. cITNs, conventional insecticide-treated nets; HHs, households; ITNs, insecticide-treated nets; LLINs, long-lasting insecticidal nets; NMCP, National Malaria Control Programme.

DOI: http://dx.doi.org/10.7554/eLife.09672.003

Coverage estimates

Figure 2 shows continent-level time-series estimates of the four internationally recommended ITN indicators, along with the 'access gap' indicator. All four indicators show a similar temporal trend: very low coverage levels and modest year-on-year increases for the first 5 years from 2000, with a marked inflexion point in 2005 and much more rapid gains thereafter. Importantly, however, all four indicators show that the pace of increase has, overall, slowed since 2005. By the end of 2013, we estimate that around two-thirds (66%, 95% CI 62%–71%) of households at risk owned at least one ITN. However, less than one-third (31%, 29%–34%) owned enough for one ITN between two people. This much lower level of adequate ownership is reflected in the levels of access and use, with 48% (45%–51%) of people at risk having access to an ITN within their household (on a one-between-two basis) and 43% (39%–46%) sleeping under an ITN the previous night. Comparison of Figure 2A,B demonstrates that many households that own some ITNs do not own enough for one-between-two, and this is captured in the time-series for the 'ownership gap' (Figure 2E). Encouragingly, this gap has been narrowed from 77% (76%–78%) of net-owning households having insufficient nets in 2000 to 56% (54%–57%) in 2013. Analysis of the 'use gap' suggested a large majority (89%, 84%–93%) of those with access to an ITN in the household slept under it the previous night, and we found no evidence of significant change in this proportion through time.

Figure 2. Continental-level time series of estimated ITN coverage indicators for the years 2000–2013.

Figure 2.

(A) % households with at least one ITN; (B) % households with at least one ITN for every two people; (C) % population with access to an ITN within their household; (D) % population who slept under an ITN the previous night; (E) 'ownership gap', the % of ITN-owning households with insufficient ITNs for one-between-two. Black circles are the annual estimates; pink envelopes denote the 95% posterior credible interval. ITNs, insecticide-treated nets.

DOI: http://dx.doi.org/10.7554/eLife.09672.004

The relatively smooth temporal trends seen at continental level obscure a great deal of complexity in the patterns of ITN scale-up occurring at national level (Figure 3). Nearly all countries began with very low coverage levels in 2000 and display a marked inflection point towards the middle of the decade, although there was considerable variation in the timing of onset of concerted scale-up activities. Importantly, the monotonic increases in coverage seen at the aggregated continental level are often replaced at national level with pronounced periods of rise and fall, and in many cases, 2013 does not represent the peak year. Variation in contemporary levels of coverage remains stark. The population with access to ITNs within the household, for example, was at or below 15% in seven countries in 2013, while above 70% for the top four.

Figure 3. Country-level time series of estimated ITN coverage indicators 2000–2013.

Figure 3.

Each plot shows the four ITN coverage indicators: % households with at least one ITN (black); % households with at least one ITN for every two people (red); % population with access to an ITN within their household (green); % population who slept under an ITN the previous night (blue). CAR = Central African Republic; DRC = Democratic Republic of Congo; ITNs, insecticide-treated nets; HH = household.

DOI: http://dx.doi.org/10.7554/eLife.09672.005

Over-allocation

Over the 14-year period since 2000, on average 15% (12%–18%) of all ITNs distributed to households were over-allocated (owned by households already owning sufficient nets for one-between-two). Figure 4 illustrates how these over-allocation rates have changed through time. Around 7% (6%–9%) of ITNs were over-allocated in 2000, and this has risen steadily to 27% (22%–32%) in 2013. The year-on-year increase in over-allocation is to some extent an expected consequence of the overall growth in ITN provision: we found that over-allocation increased approximately 15 percentage points for each one-ITN-per-capita increase in net crop. Over-allocation also varied substantially between countries, for example ranging in 2013 from 50% (36%–65%) in the Republic of the Congo to 11% (9%–15%) in Côte D’Ivoire.

Figure 4. Time series of over-allocation for the combined set of 40 sub-Saharan endemic countries, 2000–2013.

Figure 4.

Over-allocation refers to insecticide-treated nets distributed to households already owning enough nets for one-between-two, measured as the percentage of over-allocated nets among all nets in households.

DOI: http://dx.doi.org/10.7554/eLife.09672.006

Net loss

Averaged over all years and all countries, we found the median retention time for LLINs in households was 23 (20–28) months. We found no statistically significant evidence of continent-wide temporal trends in retention times, but substantial between-country variation. Figure 5 plots the LLIN loss function representing the most recent three years (2011–2013) for each country individually (blue lines), along with the aggregated continental-level curve (red line). For reference, we also overlay on Figure 5 some alternative loss functions that have been proposed. Flaxman et al. (orange line) fitted very small annual loss rates (5%) for years 1, 2 and 3 - with all LLINs then assumed lost after 3 years (Flaxman et al., 2010). The RBM-HWG proposed rate of loss (green line) is 8, 20 and 50% of LLINs to remain after 1, 2 and 3 years, respectively, with all nets being lost thereafter (Networks, 2014). As can be seen, we found rates of loss for the first 3 years to be greater than both these alternatives for all countries. Both alternatives impose a three-year maximum retention time and our decision not to do so meant that we modelled a small proportion of LLINs lasting some years beyond that point.

Figure 5. Insecticide treated netretention.

Figure 5.

Estimated long-lasting insecticidal net retention curves for each country individually (blue lines) and combined (red line), in both cases relating to the average of the most recent 3 years, 2011–2013. Also shown for reference are the rate of loss recommended in the Roll Back Malaria Harmonization Working Group needs assessment exercise (green line) and the loss rate fitted by Flaxman et al. (orange line).

DOI: http://dx.doi.org/10.7554/eLife.09672.007

ITN requirements to achieve universal coverage

Figure 6 shows the projected levels of coverage that we estimate would be achieved by the end of 2017 with LLIN deliveries for the 2014–2017 period varying from zero to 2.5 billion and under a range of different efficiency scenarios. The most important characteristic of our results is the pronounced shallowing of the delivery-coverage curves: proportionately smaller gains are made in coverage as more LLINs are delivered in an archetypal 'law of diminishing marginal returns'. This means that under a business-as-usual scenario, where current levels of over-allocation and LLIN loss persist, very large increases in LLIN delivery would be required to achieve high coverage. Under this scenario, we estimate that 1 billion LLINs (i.e. an average of 250 million per year) would be required to achieve 80% of the population with access to an LLIN in the household by the end of 2017, although this would only translate into 70% population use.

Figure 6. Projected 2017 coverage for sub-Saharan Africa in relation to number of LLINs delivered over 2014–2017 period.

Figure 6.

(A) % households owning at least one ITN; (B) % households owning enough ITN for one between two; (C) % population with access to ITN within the household; (D) % population sleeping under an ITN the previous night; (E) 'ownership gap', the % of ITN-owning households with insufficient ITNs for one-between-two. For each indicator, we project likely coverage under four scenarios: current levels of over-allocation and net loss (i.e. 'business as usual'); with minimised over-allocation; with longer average net retention (3-yr median); and with both minimised over-allocation and longer net retention. The vertical dashed lines indicate the number of LLINs calculated as required over the period under the country programmatic needs assessment supported by Roll Back Malaria Harmonization Working Group. LLINs, long-lasting insecticidal nets; ITNs, insecticide-treated nets.

DOI: http://dx.doi.org/10.7554/eLife.09672.008

The extent to which coverage gains diminish as deliveries increase is mitigated substantially when over-allocation and ITN loss rate are reduced. In a scenario with minimised over-allocation (where over allocation is set to zero), 80% population access in 2017 would be achievable with just 700 million nets (175 million per year). Reducing ITN loss rate to a 3-year median retention time would have a broadly similar impact, acting in isolation, to minimising over-allocation. If these two hypothetical efficiency gains were combined, however, 80% access could be reached in 2017 with around 560 million nets (140 million per year). We found that the relative importance of the over-allocation and LLIN loss rates changed as more LLINs were introduced. Increasing LLIN retention times was the most important factor at low levels of net delivery, but as more and more nets were provided, over-allocation became progressively more important. This is intuitive since it becomes increasingly difficult to avoid over-allocation as more households obtain adequate numbers of nets.

For reference, we also plot on Figure 6 the 920 million additional LLINs calculated by countries as required for universal coverage of targeted populations by 2017 under the RBM-HWG needs assessment exercise. Under current levels of over-allocation and net loss, we estimate that by the end of 2017 this quantity of new LLINs would translate into 77% access (among those populations targeted by countries for ITN coverage) and, assuming current behaviour patterns continue, 68% sleeping under an ITN. Under the combined efficiency scenario with minimised over-allocation and 3-yr median ITN retention time, however, the 920 million nets would approach universal access (slightly over 95%).

Discussion

By linking manufacturer, programme and national survey data using a conceptually simple model framework, the intention has been to provide a transparent and intuitive mechanism for tracking net crops and resulting household coverage that reflects the input data while simultaneously providing a range of insights about the system itself. In doing so we have been able to (i) provide a new approach for estimating past trends and contemporary levels of ITN coverage; (ii) explore the effects of uneven net distribution between households and the rates of net loss once in households; and (iii) use these insights to estimate how many LLINs are likely to be required to achieve different coverage targets in sub-Saharan Africa.

We have, for the first time, extended dynamic model-derived estimation of ITN coverage to all four internationally recognized indicators, along with the two 'gap' metrics. Our results reinforce a simple message: while gains in ITN coverage have been impressive, there remains an enormous challenge if the goal of universal access is to be achieved and sustained. The importance of the new expanded suite of indicators is also exemplified: while an encouraging two-thirds of households now own at least one ITN, less than half of these have enough to protect everyone who lives there. This ownership gap is narrowing but the disparity remains evident across nearly all countries. Conversely, there is little evidence that non-use of available nets contributes substantially to low coverage levels. We therefore reinforce earlier studies that suggest the overwhelming barrier to not sleeping under an ITN is lack of access rather than lack of use WHO, 2013a; Eisele et al., 2009; 2011; Koenker and Kilian, 2014; Koenker et al., 2014). Of course, non-use may be important in certain local contexts, and finer-scale analysis can support identification of areas where behaviour change communication interventions may be appropriate to reduce it (Kilian et al., 2013).

We found substantial over-allocation of nets to households already owning a sufficient quantity, and that this became more pronounced as overall ownership levels increased through time. Mass distribution campaigns can, in principle, be designed to minimise over-allocation and maximise evenness of nets allocated to households strictly on the basis of households members and pre-existing nets. As other studies have highlighted, however, any possible commodity savings achieved by such strategies must be compared against the operational cost of these more complex distribution mechanisms (Yukich et al., 2013). What is certain is that over-allocation becomes a major barrier to achieving universal coverage when levels of ITN provision are high because most new incoming nets are simply leading to surpluses in many households, while elsewhere there remains a shortfall. This may have a disproportionately high public health impact if those surplus nets are concentrated in households at lowest risk. Wealthier, better educated and more urban households may be better placed to obtain available nets but are often located in regions of lower transmission (Steketee and Eisele, 2009; Webster et al., 2005). While beyond the scope of the present study, the approaches we have developed here could be extended to consider these issues of equity in coverage versus risk in more detail.

One of the most important observations in our study is that LLINs may be lost from households at a substantially faster rate than is currently assumed. Importantly, we assess loss by comparing total inputs to countries (from deliveries) to total numbers in households (net crop), and so we measure real losses rather than, for example, reallocation of nets between relatives (Koenker et al., 2014). Longer retention times of the sort observed in some local studies are not supported by the body of evidence we have provided by triangulating large-scale net distributions and household survey data. This more rapid loss rate has potentially important implications for existing guidelines. Current RBM guidance is for mass ITN campaigns to be conducted every 3 years, complemented by continuous distribution of nets via routine channels in order to maintain coverage levels between those campaigns. However, whatever levels of coverage are achieved by a given campaign, we estimate that one-half of the campaign nets distributed, on average, will not be present in households just 2 years later. Our coverage time-series for many countries suggest that routine distribution channels are not yet compensating fully for this rate of loss, often displaying pronounced dips in coverage levels between mass campaigns. Maintaining higher continuous coverage therefore clearly requires some combination of more frequent campaigns, greater ongoing distribution between campaigns, or more durable nets and improved care behaviour by users that lead to longer overall retention times.

We considered nets in households as simply present or absent, with no allowance for their condition. In reality, of course, nets may be retained by households (and thus 'present' in our calculations) even when they are badly torn, or have diminished insecticidal properties. As such, our estimates of 'coverage' would be revised downwards if additional measures of net efficacy were included. Our model is able to provide an estimate for every country and every year of the age-profile of ITNs in households. This raises the possibility of extending the predictions to incorporate modelled or observational data on average rates of net degradation in different contexts (Briët et al., 2012) to explore measures of entomologically effective coverage.

Tools developed to assist countries to calculate LLIN requirements, have tended to define need using a simple ratio to populations at risk (such as 1.8 people per net), and have made allowances for net loss from households using pre-defined rates of loss. We have been able to show that true LLIN requirements are likely to be considerably larger when the more rapid rates of loss are taken into account, along with the additional effect of likely over-allocation patterns. This more realistic framework not only provides the basis for more accurate needs assessments but identifies the relative importance of these different factors in determining the coverage that can be achieved for a given delivery level. Our analysis of future LLIN needs from the present time to 2017 demonstrates how these factors lead to a pronounced law of diminishing returns: as more nets are introduced to a population, proportional increases in coverage diminish, with over-allocation a particular problem at high net provision levels.

Under business-as-usual, the number of nets required to approach full coverage is prohibitively large. Clearly, however, reducing current system inefficiencies and increasing net retention are not straightforward and already the subject of much attention by countries and international partners. Over-allocation is the complex result of different distribution strategies and varying levels of population access to services, and any solution comes with its own cost. Net retention can doubtless be increased by improved LLIN technology coupled with behaviour-change communication efforts, although it is also feasible that retention times may reduce when overall net provision increases (with new nets displacing older ones). Additionally, we look only at the RBM definition of use and ignore the effectiveness of nets in repelling mosquitoes once they are being used. This is potentially an important confounder when considering retention times. While not aiming to provide solutions to these complex challenges, the results we present here provide an analytical framework in which the impact of theoretical efficiency gains can be assessed and this could be extended to include formal cost–effectiveness analysis.

In conclusion, our results provide evidence that LLIN requirements to achieve universal coverage have been underestimated. If obtaining higher coverage remains an accepted goal of the international community, then larger LLIN volumes must be considered and planned for at national and international levels. We emphasise, however, that this would be best achieved in parallel with a renewed focus on maximising the efficiency of coverage achieved for each new net financed. Given that the pattern of diminishing coverage returns for each dollar spent is likely to be unavoidable, the cost–effectiveness of pursuing universal coverage rather than a lower operational target must ultimately be weighed against alternative malaria control investments.

Materials and methods

Overview

Two important preceding studies have sought to model national-level ITN delivery, distribution, and coverage: the Flaxman et al. study (Flaxman et al., 2010) and the work of Albert Killian culminating in the NetCALC tool (Networks, 2014) and a series of related publications (Paintain et al., 2013; Yukich et al., 2013). Although very different in implementation, both approached the problem in a similar two-stage process. First, a mechanism was defined for estimating net crop — the total number of ITNs in households in a country at a given point in time—taking into account inputs to the system (e.g. deliveries of ITNs to a country) and outputs (e.g. the discard of worn ITNs from households). Second, empirical modelling was used to translate estimated net crops into resulting levels of coverage (e.g. access within households). We have adopted a similar analytical outline, but the models we have developed for each stage differ structurally and conceptually from these earlier efforts. Our underlying principle has been to represent the ITN system in a simple and intuitive way and to parameterise that system using a data-driven approach that minimises the reliance on assumptions or small external datasets. In this Methods section, we describe: (i) the main data sources used; (ii) a new compartmental model for estimating net crop that also offers insights into rates of ITN loss from households; (iii) a new coverage model linking net crop to household net access and use that also assesses the efficiency of between-household distribution (i.e. the extent of over-allocation); and (iv) the use of our models to predict future ITN requirements to meet the goal of universal access. A schematic overview of our analytical framework is provided in Figure 7, and additional methodological detail is provided in the Supplementary Information.

Figure 7. Schematic showing overall analytical framework linking data, model components, and outputs.

Figure 7.

HH = household; ITN, insecticide-treated net; NMCP = National Malaria Control Programme.

DOI: http://dx.doi.org/10.7554/eLife.09672.009

Data

We used three principal sources of data to fit our models. These are described briefly below and in more detail in Supplementary Information.

  1. LLINs delivered to countries: data provided to WHO by Milliner Global Associates on the number of LLINs delivered by approved manufacturers to each country each year (WHO, 2013aAMP, 2014). These were complete for each country from 2000 to 2013 inclusive.

  2. ITNs distributed within countries: data provided to WHO by National Malaria Control Programmes (NMCPs) on the number of cITN and LLINs distributed annually within each country (WHO, 2013a). Data were available for 365 of the 560 country-years addressed in the study. We treated these data as only partial records of distribution activities because the extent to which NMCP reporting captures distribution by non-government agencies is not known for all countries.

  3. Nationally representative household surveys. We assembled 99 national surveys from 39 sub-Saharan African countries from 2001–2013, covering 18% of all possible country-years since 2000 (Figure 8). More recent surveys provided household-level data on the number of cITNs, LLINs, people within each dwelling, and people sleeping under nets the previous night. RBM-MERG guidelines detail the conversion of these data into the standardised ITN indicators (RBM, UNICEF, WHO, 2013) and, in combination with national population data (UNPD, 2012), they can yield an estimate of national net crop (see Supplementary Information). Older surveys had less information: providing data on use but not ownership, for example, or for cITNs but not LLINs (see Supplementary Information). For most surveys (95/99), we were able to access the underlying data, while for the remaining four we used only the survey report.

Figure 8. Distribution of national survey data on ITN net crop and household ownership, access and use used in this study, by country and year.

Figure 8.

The different types of survey are shown in the key: CAR = Central African Republic; DHS = Demographic and Health Survey; DRC = Democratic Republic of Congo; ITN, insecticide-treated net; MICS = Multiple Indicator Cluster Survey; MIS = Malaria Indicator Survey; STP = São Tomé and Príncipe.

DOI: http://dx.doi.org/10.7554/eLife.09672.010

Countries and populations at risk

Our main analysis covered 40 of the 47 (WHO, 2013a) malaria endemic countries of sub-Saharan Africa. We excluded six endemic countries on the basis that ITNs do not form an important part of their vector control programme, as reported by the respective NMCPs to the African Leaders Malaria Alliance, ALMA (M. Renshaw, pers. comm. 3rd August 2014). These were Botswana, Cape Verde, Namibia, São Tomé and Príncipe, South Africa and Swaziland. We also excluded the small island nation of Mayotte, for which no ITN delivery or distribution data were available. We limited all analyses to those populations categorized as being at risk by NMCPs (WHO, 2013a). When interpreting NMCP distribution and household ownership data, we made the simplifying assumption that all reported ITNs were distributed among, and owned within, households situated in malaria endemic regions (Burgert et al., 2012). Additionally, we used data from African Leaders Malaria Alliance (ALMA) on the proportion of populations at risk targeted for ITNs versus IRS, and downscaled targeted populations at risk accordingly. It should be noted that restricting the distribution of ITNs to populations at risk makes the assumption that no ITNs are distributed to populations not at risk.

Estimating national net crops through time

Like Flaxman et al.(Flaxman et al., 2010), we represented national ITN systems using a discrete time stock-and-flow model. In this structure, a series of compartments were defined that contained a given number of nets at each time-step, with possible movement of nets from one compartment to another between time-steps (see Supplementary Information). Nets delivered to a country by manufacturers were modelled as first entering a 'country stock' compartment (stored in-country but not yet distributed to households). Nets were then available from this stock for distribution to households by the NMCP or other distribution channels. Years where NMCP distributions were smaller than available country stock represented potential ‘under-distribution’, with nets left to stockpile rather than reaching households. However, because of the uncertainty associated with NMCP distribution data, these discrepancies could simply reflect under-reporting of distribution levels. To accommodate this uncertainty, we specified the number of nets distributed in a given year as a range, with all available country stock as one extreme (the maximum nets that could be delivered) and the NMCP-reported value (the assumed minimum distribution level) as the other.

New nets reaching households joined older nets remaining from earlier time-steps to constitute the total household net crop, with the duration of net retention by households described by a loss function. In this representation, the net crop simply reflected the differences over time between inputs to and outputs from households. This meant that distribution, net crop, and the loss function together formed a closed system: the three must triangulate exactly and knowledge of any two components allowed the third to be calculated directly. Flaxman et al. (Flaxman et al., 2010) assembled data from six studies on ITN durability and rates of loss. Using a loss function fitted to these data, however, they found that the three components tended not to triangulate: net crops observed in surveys were too small, given the data on nets distributed to households and their modelled rate of loss. Their interpretation was that the number of ITNs distributed each year may be systematically over-reported by NMCPs, and a 'bias parameter' was included in the model, adjusting downward the volume of nets entering households in each country compared with reported levels. As described above, we took a different approach: with no a priori expectation that NMCP distribution reports exaggerate distribution levels. Rather than fitting the loss function to a small external dataset, we fitted this function directly to the distribution and net crop data within the stock-and-flow model itself. Conceptually, this reflected the view that the 560 country-years of distribution data triangulated against the 102 survey-derived national net crop values represented a more impartial and data-driven way of inferring rates of loss than using limited data from local ITN retention studies. Loss functions were fitted on a country-by-country basis, allowed to vary through time, and defined separately for cITNs and LLINs. We compared these fitted loss functions to existing assumptions about rates of net loss from households. The stock-and-flow model was fitted using Bayesian inference and Markov chain Monte Carlo (MCMC), providing time-series estimates of national household net crop for cITNs and LLINs in each country along with evaluation of under-distribution, all with posterior credible intervals. A complete technical description is provided in the Supplementary Information.

Estimating national ITN access and use indicators from net crop

Levels of ITN access within households depend not only on the total number of ITNs in a country (i.e. net crop), but on how those nets are distributed between households. In simple terms, a more even distribution yields a greater proportion of households owning nets than if those same nets are concentrated in fewer households. Many recent national surveys report the number of ITNs observed in each surveyed household. This allows, a histogram to be generated that summarises the net ownership pattern (i.e. the proportion of households with zero nets, one net, two nets and so on). By analysing such data from multiple surveys, previous studies have demonstrated that histograms for different countries vary in a broadly predictable way according to national net crop (Flaxman et al., 2010; Yukich et al., 2013). By representing these histograms using a formal statistical distribution (such as the negative binomial), and linking its parameters to net crop, predicted histograms can be generated for any country-year for which a net crop estimate is available (Flaxman et al., 2010; Yukich et al., 2013). These histograms, in turn, allow direct calculation of the first access coverage indicator (% households owning one or more ITN). We took the view that this approach—linking net crop to a statistical distribution, and using the distribution to calculate access indicators—is preferable to the alternative of regressing the access indicators against net crop directly. The latter approach, used in the NetCalc tool (Networks, 2014), is simpler but provides less direct insight into the patterns of between-household ITN distribution that ultimately link net crops to access levels.

One aspect that is known to strongly influence the relationship between net crop and household ownership distribution is the size of households found in different countries (Networks, 2014; Yukich et al., 2013), which varies greatly across sub-Saharan Africa (Swaziland, for example has an average household size of around three members, while in Senegal the average is nearly ten). Household size also, of course, determines whether a given number of owned nets will be sufficient to provide access to all residents. We extended earlier analyses (Flaxman et al., 2010; Yukich et al., 2013) to explicitly account for household size: using a bivariate (i.e. two- rather than one-dimensional) histogram model to link net crop to ownership distributions for each household size stratum (see Supplementary Information). We replaced the negative binomial distribution with a 2-d zero-truncated Poisson distribution and, for each household size stratum, fitted the distribution using two parameters: (i) the proportion of households with zero ITNs and (ii) the mean number of ITNs per ITN-owning household. Using the household-level data from 83 national surveys, we found that both parameters were strongly related to national net crop, allowing bivariate histograms to be generated for every country-year that were closely representative of the true ITN ownership distribution.

Stratifying our analysis by household size had three important advantages over earlier approaches. First, the distribution of net ownership tended to vary substantially between households of different sizes within a given country and this variation would be missed if all households were considered together. Accounting for this enabled better fits to the data. This makes sense: all else being equal, larger households would be expected to own more nets than smaller ones and so distribution patterns would differ systematically. Second, the bivariate ownership histograms predicted for each country-year could be used to directly calculate all three indicators of household access. While a simple univariate histogram allows calculation of % households with at least one ITN, a bivariate histogram means the number of both ITNs and people in every household can be triangulated which, in turn, allows direct calculation of the two additional indicators: % households with at least one ITN for every two people and % population with access to an ITN within their household, along with the 'ownership gap' (see Supplementary Information). Linking these bivariate histograms to our annual net crop estimates for each country meant we could predict time-series of the access indicators at the national level from 2000–2013, with all parameters fitted in a Bayesian framework providing posterior credible intervals around each time-series. We also combined the country-level results to generate a set of continent-level indicator time-series, representing overall coverage levels among populations at risk in the 40 endemic countries. Third, the bivariate histograms allowed analysis of over-allocation: certain cells of the histogram represented households owning more ITNs than were required to achieve access on a one-between-two basis, and the proportion of the total net crop falling in this category was examined through time for every country.

We took a different approach for the final indicator, % population who slept under an ITN the previous night. ITN use is less directly linked to national net crop and is primarily determined by the availability of nets within households (Eisele et al., 2009). A total of 83 of the 102 national surveys contained data allowing the relationship to be explored between ITN use and each of the three access indicators with, perhaps unsurprisingly, % population with access to an ITN within their household displaying the largest correlation (adjusted R2= 0.96). We fitted this relationship across the 83 surveys using a simple Bayesian regression model (see Supplementary Information) and used it to predict time-series of the ITN use indicator for every country. The ratio of population use to access revealed the 'usage gap'—the fraction of the population with access to ITNs not using them—and between-country variation in this ratio was also explored.

Estimating ITN requirements to achieve universal access

Our two-stage modelling framework represented the pathway from ITN delivery into countries through to resulting levels of net access and use in households. It also accounted for two potential factors that act to reduce access levels, and allowed these to be quantified through time for each country. Using this architecture, it was possible to simulate delivery of any hypothetical volume of ITNs to a given country over a given future time period, to predict the levels of access and use that would result, and to examine the impact of different amounts of over-allocation and net loss. The current needs assessment exercise that countries are undertaking (RBM-HWG, 2014; RBM, 2014) is designed to identify the number of LLINs required to achieve coverage targets by 2017. We used our model to estimate the levels of access likely to be achieved if these forecast LLIN commodity needs were met across the 2014–2017 period under a 'business as usual' scenario, that is, with current levels of over-allocation and net loss, and compared these predicted levels with the objective of universal access among target populations. We then generalized this experiment to predict the likely level of coverage (for all four indicators) achievable by 2017 under a broad spectrum of LLIN delivery levels, equivalent to a total for sub-Saharan Africa (two of the 40 endemic countries in our study did not participate in the RBM-HWG needs assessment exercise [Djibouti and Equatorial Guinea], and so our scenario analysis is based on the set of 38 remaining countries; to maintain comparability through time, we combined needs assessment data for mainland Tanzania and Zanzibar, and for Sudan and South Sudan) of between zero and 2.5 billion nets across the 4-year period. Further, we ran these simulations under four scenarios: (i) 'business-as-usual' (where current levels of over-allocation and net loss were maintained); (ii) with no over-allocation (new LLINs are distributed preferentially to those households with zero LLINs, then to those with less than one-between-two); (iii) with reduced LLIN net loss by households (using a modelled 3-year median retention time); and (iv) with both no over-allocation and a 3-year median retention time.

Acknowledgements

The authors acknowledge the valuable assistance from Melanie Renshaw in providing information from the Roll Back Malaria Harmonization Working Group Programmatic Gap Analysis and other guidance in the interpretation of our results. We thank members of the Roll Back Malaria Monitoring and Evaluation Reference Group (RBM-MERG) and the World Health Organization Surveillance Monitoring and Evaluation Technical expert Group (SME-TEG) for their feedback and suggestions. We thank Clara Burgert of the DHS Program for her assistance with DHS Survey access and interpretation.

Appendix 1. Supplementary information on data and methods

1.0 Outline of document

In this Supplementary Information document, we augment the main manuscript by providing additional details on the data and modelling architecture developed in this study to allow prediction of insecticide treated bednet coverage, system efficiency and future needs. In Section S2, we describe the acquisition and processing protocols for data from national household surveys, NMCPs and ITN manufacturers. In Section 3, we describe the Bayesian compartment model which estimates, at quarterly intervals, the total number of ITNs in households in each country (the net crop). In Section 4, we describe a second modelling stage that predicts national-level ITN coverage indicators as a function of predicted net crop. Finally, in Section 5, we describe the use of the modelling framework to predict future ITN coverage levels under a range of hypothetical ITN delivery volumes and efficiency scenarios.

2.0 Data collection

The various modelling stages were fitted to data from three principal sources.

2.1 ITN manufacturer reports

Manufacturer reports provided information on the number of LLINs delivered to each country each year by international manufacturers. These data were provided to the WHO by Milliner Global Associates and were complete for each country from 2000 to 2013 inclusive.

2.2 National malaria control program reports

NMCP reports provided information about the number of LLINs and cITNs distributed in a county within a given year. These data were provided to WHO by NMCPs and were available for 365 of the 560 country-years addressed in the study. We treated these data as only partial records of distribution activities because the extent to which NMCP reporting captures distribution by non-government agencies is not known for all countries.

2.3 Household survey reports

We identified and obtained data for the ownership and use of ITNs from household surveys conducted in sub-Saharan countries since 2000, including DHS, MIS, MICS, AIDS indicator surveys (AIS) and a malaria and anaemia prevalence survey (EA & P) (Supplementary file 1). Data at the household level were acquired from 95 national surveys from 39 countries from 2001 to 2014. In addition, we acquired national level data from four household survey reports for which we were unable to obtain the household level dataset. The range and number of the household survey data collected is depicted in Appendix Figure 7. For those surveys where household-level data were available on the type of nets owned (see Supplementary file 1), the number of ITNs owned was determined by the sum of each ITN in the household. A net was considered an ITN if it was an LLIN, or a pre-treated net obtained within the past 12 months, or a net that has been soaked with insecticide within the past 12 months. ITNs were then subdivided into the two classes, LLINs and cITNs.

For the surveys where data on the type of net was only available for one net in each household (see Supplementary file 1), the overall survey-level proportion of total nets for each net classification (non-ITN, LLIN, cITN) was determined and multiplied by the number of nets in the surveyed households to estimate the number of LLINs and conventional ITNs owned by each surveyed households.

3.0 Compartment model

3.1 Introduction

This section describes our implementation of a Bayesian hierarchical model to impute, at quarter-annual intervals, the total number of insecticide-treated bed nets (which can be LLINs or cITNs) in a given country.

To achieve this goal, we built a model incorporating the three distinct sources of information on ITNs described in the preceding section: manufacturer reports on ITNs delivered to countries, NMCP reports on ITNs distributed to households within countries, and household survey data providing direct cross-sectional estimates of net crop in households at a given time point.

The key challenge in our model was linking the delivery and distribution data sources to the net crop measurements. Figure 2 shows our schematic representation of the system and the evidence synthesis of these three data sources. In a given year, a given volume of nets are delivered to a country from manufacturers (Appendix figure 8, green arrows), giving rise to a (as yet undistributed) country stock (Appendix figure 8, orange arrow). Nets from this stock then may be distributed to households in that or subsequent years, as captured by NMCP distribution data (Appendix figure 8, blue arrows). To link the NMCP distributed nets to the direct estimates of the net crop existing within households, we needed to account for the rate of loss of nets from households. That is, once nets are distributed, they remain within households for a given length of time until they are discarded. By tracking the net loss from NMCP deliveries (Appendix figure 8, brown arrows), we were able to estimate the total number of nets in a country at a given time point (e.g. in 2008, when a household survey was conducted [ Appendix figure 8, red arrow] by summing the nets of all ages [Appendix figure 8, purple line]). The compartment model was therefore calibrated by parameterising the loss function in such a way that the net crop observed in each national survey was consistent with the known influx of nets to households (following manufacturer delivery and NMCP distribution) and an estimated rate of loss from households. Our loss function was modelled to be temporally varying for a given country and parameterises the proportion of nets discarded through time.

3.2 Model specification

3.2.1 Observed data

3.2.1.1 Observations of net crop from household surveys

Conceptual description

Household survey reports provided a direct measure of how many ITNs, of any age, existed in households in a country over a sampling period. For use in fitting the compartment model, household-level survey data on ITN ownership needed to be translated into a representative estimate for the total number of nets in a given country. To accomplish this, we summarised each household survey into two variables: the average number of LLINs and cITNs per household and the average household size (i.e. number of residents). The total number of ITNs was then calculated as the product of the population of the country at the time of the survey and the estimated ITNs per-capita as observed in the survey data, taking into account the survey weighting to ensure the arithmetic mean was nationally representative. The calculation of the total number of nets was completed by specifying probability distributions that allowed propagation of uncertainty from the household surveys.

Formal description

To estimate national net crop (total ITNs in households) from each national survey, three summary statistics were required:

a) The average household size, h hμ,, with an associated standard error, hhσ2.

b) The average number of LLINs per household, AvgLLINμ', with an associated standard error, AvgLLINσ2 .

c) The average number of cITNs per household, AvgcITNμ', with an associated standard error, AvgcITNσ2.

It should be noted here that, throughout this Supplementary document, we use the term standard error when referring to the standard deviation of the sample mean or the variability of an estimator.

Survey data came from four different sources—DHS Program, MICS4, MICS3 and other surveys for which only the published report (and not underlying data) were available, mainly MIS undertaken unilaterally by NMCPs or supporting partners. Different protocols were required to obtain summary statistics (a), (b), and (c) above for each survey type.

DHS and MICS4 data: These household survey reports provided direct data about the number of LLINs and cITNs per household and the household size. Therefore, means across all households were calculated using a weighted average that incorporated the published survey sample weights and the standard errors obtained through Taylor linearization (Kish and sampling, 1965).

MICS3 data: These household surveys provided direct data about the household size and number of bed nets per household, but did not provide information on the type of each net observed (e.g. LLINs, cITNs or untreated bet nets). The surveys did, however, provide a full description of a single net within each household. Therefore, across all such single nets in each survey, we determined the mean proportion of nets that were LLINs, cITNs or untreated nets along with the standard errors. We then multiplied these proportions with the computed average number of nets (of any kind) to determine the average number of LLINs and cITNs per household. Propagation of uncertainty was achieved using MCMC sampling (Plummer, 2003; Gelman et al., 2013).

Other reports: These surveys contained no disaggregated household data and only reported the averages of household size and numbers of LLIN and cITN nets with no standard errors. We therefore assumed a small 1% error on these estimates, consistent with the magnitude of sampling errors seen in other surveys.

Using metrics (a), (b) and (c) obtained from the 95 processed surveys, the total number of LLINs and associated standard error in a country reported by a survey were defined probabilistically as:

μSURVEYLLINt¯,c=Ε[Population t¯,cNormal(AvgLLINμ,AvgLLINσ2)Normal(hhμ,hhσ2)] (1)
σSURVEYLLINt¯,c=VAR[Populationt¯,cNormal(AvgLLINμ,AvgLLINσ2)Normal(hhμ,hhσ2)] (2)

And similarly the total number of cITNs:

μSURVEYcITNt¯,c=EPopulationt¯,cNormal(AvgcITNμ,AvgcITNσ2)Normal(hhμ, hhσ2) (3)
σSURVEYcITNt¯,c=VARPopulationt¯,cNormal(AvgcITNμ,AvgcITNσ2)Normal(hhμ,hhσ2) (4)

Where Ε[] is the expected value, VAR[] is the standard error, c is a given country and t¯ is the mean sampling time of the survey. Equations 1–4 μSURVEYLLINt¯,c,σSURVEYLLINt¯,c,μSURVEYcITNt¯,c and σSURVEYcITNt¯,c were therefore observed data inputs into the compartment model.

3.2.1.2 Manufacturer reports of LLIN deliveries

As shown in Appendix figure 8, manufacturer reports provided data on the number of LLINs delivered to a country in a given year. There were no corresponding reports for cITNs.

Conceptual description

The main purpose of the manufacturer data was to determine stock levels and ‘cap’ estimated NMCP distributions in each year (i.e. more nets could not be distributed than were potentially available in country stock). The manufacturer reports were complete (no missing values) and assumed to be of high fidelity. We therefore modelled the likelihood of the manufacturer data in a given country-year as a normally distributed random variable. The manufacturer reports did not report standard errors in the delivery numbers and therefore we assumed a uniformly distributed prior probability on the error. Because the manufacturer data served only to inform stock and cap NMCP distributions, no considerations of sub-annual timing were required.

Formal description

We define Manufacturerc,t (observed data) as the number of manufacturer LLIN nets delivered to a country c at year t. The number of manufacturer LLIN nets sent (μc,t) was modelled as:

Manufacturerc,t~Normal(μc,t,Manufacturerc,tσm,t2) (5)

With error σm,t=Uniform(0,0.075)0.3

3.2.1.3 NMCP reports of ITN distributions

As shown in Appendix figure 8, NMCP reports provided information on the number of both LLIN and cITNs nets distributed in a given country c at year t.

Conceptual description

As with the manufacturer data, the NMCP reports were assumed to be of reasonably high fidelity. Unfortunately, NMCP reports were not complete and contained missing values where countries failed to report the number of nets distributed in a year. To impute these missing values, we defined an informative prior probability distribution on the NMCP distributions for both LLINs and cITNs. We tested multiple different parameterisations of NMCP prior distributions, and evaluated performance of parameterisations using out-of-sample cross validation to choose the best model.

Our final choice of NMCP prior distributions was data-driven, using combined reports across all African country-years that had NMCP data. We scaled the reports to per-capita NMCP distributions to remove country-specific differences. We observed that this combined per-capita distribution approximately followed an exponential distribution with a zero inflation hurdle (to account for no deliveries). Additionally, when looking at the combined reports across time, it was clear that combined distribution varied temporally. We therefore modelled NMCP LLINs and cITNs separately for each year by disaggregating the continent wide per-capita distribution temporally into separate exponentials with zero inflation. Finally, to account for variability due to sample size, we fitted splines through these time series.

Formal description

We defined NMCP_LLINc,t (observed data), NMCP_ITNcc,t (observed data) and NMCP_TOTALc,t (observed data) as the number of NMCP LLIN nets, NMCP cITN nets and the sum of LLIN and ITN NMCP nets reported distributed in a country c at time t.

As described above, the NMCP reports did not have standard errors and contained missing values. We therefore defined informative prior distributions on NMCP_LLINc,t and NMCP_cITNc,t. First, we defined two sets t and t as the combined set of per-capita rates of NMCP LLIN and cITN across all African countries at times t respectively. The sets T and T with T[2000,2001,,2012] therefore contained all NMCP LLIN and cITN distributions across all country-years.

To characterise the hurdle exponential distribution, we used two parameters: the zero inflated components (pLLIN0t,pcITN0t), which defined the probability that in a given country-year no nets were distributed, and the exponential component (pLLIN1t,pcITN1t) which, given some distribution of nets took place, gave the per-capita rate of distribution (i.e. how many nets per-capita were distributed).

We therefore define the proportion of zero NMCP deliveries for LLINs and cITNs as:

pLLIN0t=it1i,t=0it1i,t (6)
pcITN0t=iIt1Ii,t=0iIt1i,t (7)

And the mean delivery rate at time t as:

pLLIN1t=iti,t1i,t0it1i,t0 (8)
pcITN1t=iItIi,t1Ii,t0iIt1Ii,t0 (9)

Using these two parameters we then defined the prior distributions on NMCP LLIN and NMCP cITN as:

rateLLIN,t~Exponential(1pLLIN1t),ΩLLIN,t=1                                         0, otherwise (10)
ratecITN,t~Exponential(1pcITN1t), ΩcITN,t=1                                      0, otherwise (11)

Where:

ΩLLIN,t~Bernouli(pLLIN0t) (12)
ΩcITN,t~Bernouli(pcITN0t) (13)

The terms rateLLIN,t and ratecITN,t therefore characterised the prior distribution on the per-capita rates of NMCP LLIN and cITN distributions, respectively. The values of terms rateLLIN,t and ratecITN,t also contained missing values for some years and showed variability due to different sample sizes. Therefore, we fitted penalised regression splines through the time series for each parameter to create smooth prior parameters. The spline fitting was done using restricted maximum likelihood with rigorous selection done to find the optimal number of basis functions for the spline (see Appendix figure 1 for the spline fits).

To form the likelihoods, these prior rates needed to be scaled by the population in a given country-year. Therefore, we defined terms δc,t=Populationc,trateLLIN,t and πc,t=Populationc,tratecITN,t as the final prior distribution on the NMCP distributions (not per-capita).

Using these prior distributions and allowing for some uncertainty, we model the likelihoods for the observed NMCP_LLINc,t, NMCP_ITNcc,t and NMCP_TOTALc,t as random normal variables with added error:

NMCP_LLINc,t~Normal(δc,t,σd1,t2) (14)
NMCP_cITNc,t~Normal(πc,t,σd2,t2) (15)
NMCP_TOTALc,t~Normal(δc,t+πc,t,σd3,t2) (16)

Where σd1,t=σd2,t=σd3,t=Uniform(0,0.01). Additionally, all distributions (equations 14 and 16) were zero truncated to prevent negative numbers of nets.

It should be noted that NMCP distribution numbers are not always accurate and potentially under- or over-estimate the number of nets distributed. This may result, for example, from nets being distributed through other sources and therefore not contributing to the total NMCP distribution sum. To account for this uncertainty, we allow the number of nets distributed to take a uniformly distributed value with a lower bound of the NMCP distribution and an upper bound of the number of nets able to be distributed (found from the net stock—see Equation 22).

3.2.2 Compartment model structure

Following the methods outlined above, we arrived at a set of observed data, with standard errors, of the number of nets delivered for all country-years (manufacturer), the number of nets distributed within-country for all country-year (NMCP) and sparse estimates of the number of nets of any age in households (i.e. net crop) in 95 country-years with available surveys

Our compartment model linked all these processes together by modelling four different processes. (i) Using delivery and distribution information to allow for LLIN net stock to accumulate, thereby allowing for more LLIN nets to be distributed than were delivered in a country-year. (ii) Disaggregating the total nets delivered in a country-year into quarter-yearly intervals to allow for a more realistic modelling of the temporal dynamics. (iii) Linking year-by-year distributions through a loss function that accounted for the rate of nets being lost from households after distribution as a function of the time from distribution. (iv) Calibrating the compartment model quarter-yearly estimates on the observed survey reports. These four modelled processes are now described in turn.

3.2.2.1 Accumulation of LLIN net stock

Conceptual description

Following the methodology described in Flaxman et al (2010) (Flaxman et al., 2010), we define a net stock variable Stockc,t for a given country-year. Stockc,t links together LLIN manufacturer deliveries and NMCP LLIN distributions by allowing for a surplus of nets to be built up in a country. Essentially, Stockc,t added a ‘cap’ on the number of nets NMCPs report to be delivered, and therefore created an upper bound on erroneously reported deliveries.

We also allowed for NMCP data to represent an under-estimate of true distribution levels. This could occur, for example, if the NMCP reporting system did not capture those nets being distributed by non-governmental agencies. To accommodate this uncertainty, we specified the number of nets distributed in a given year as a range, with all available country stock as one extreme (the maximum nets that could be delivered) and the NMCP-reported value (the assumed minimum distribution level) as the other. It should be noted that due to the lack of manufacturer data for cITNs, this uncertainty was only incorporated for LLINs.

Formal description

First we define δ^c,t, the adjusted δc,t, as the modelled parameter for the number of NMCP LLIN nets delivered in a country-year. We define δ^c,t0 and Stockc,t0 at the first time point t0=2000 as:

δc,t0~μc,t0,   δc,t0>μc,t0    δc,t0,   Otherwise (17)
δc,t0adj~Uniform(δc,t0,μc,t0) (18)
Stockc,t0=μc,t0δc,t0adj (19)

And then define δ^c,t and Stockc,t at subsequent time points recursively as:

δc,t=μc,t+ Stockc,t1,  δc,t>μc,t+ Stockc,t1                       δc,t,   Otherwise     (20)
δc,tadj~Uniform(δc,t,μc,t+ Stockc,t1) (21)
Stockc,t=Stockc,t1+μc,tδc,tadj (22)

As shown in the above equations, if a country did not distribute as many nets as were delivered, stock levels can increase, but with a limit that a country cannot deliver more nets than stock permits. δ^c,t has a probabilistic interpretation reflecting our uncertainty about whether the NMCP values reported the total number of LLINs that were able to be distributed or if the calibration of the stock and flow model on the survey data required more nets to be delivered. For cITNs, it was not possible to include a stock component in the compartment model as there were no manufacturer reports for cITNs.

3.2.2.2 Temporal disaggregation of NMCP LLIN and cITN deliveries

Conceptual description

Modelled variables δ^c,t and πc,t defined the number of LLINs and cITNs distributed within a country-year. However, these variables were modelled on data that did not provide any information about when in the given year nets were distributed. This led to the potential for temporal inconsistency when calibrating survey estimates at an average time in a country-year against NMCP distribution information with no-sub annual temporal resolution. Therefore, we needed to specify a prior on when NMCP distributions occurred within a given country-year. In striving to keep the model as parsimonious as possible, we first modelled a scenario where all NMCP nets were distributed at either the start or end of the year. This, however, did not represent reality adequately, and led to poor calibrations with survey estimates in some instances. We then relaxed this assumption to allow all nets to be delivered at a random point in the year, but again this led to poor calibrations. Finally, we opted for a more realistic distribution scenario where we disaggregated distributions to a quarter- yearly temporal resolution. We then assigned priors on NMCP quarterly net distributions to allow any proportion of nets to be delivered at the start, first quarter, second quarter, third quarter or end of the year. This scheme allowed for maximum flexibility in the model with minimal subjective prior assumptions, and yielded excellent calibrations with survey estimates.

Formal description

We disaggregated modelled variables δ^c,t and πc,t (the number of LLINs and cITNs distributed within a country-year) into intervals Q[0.25,0.5,0.75,1], representing the number of nets distributed by the first, second, and third quarter or end of a given year. We defined the proportions of δ^c,t and πc,t in each interval as:

gi~Uniform(0,1) (23)
qi=gijQgj (24)

Where iQ and iQ qi=1 and of course iQ δ^c,tqi  = δ^c,t and iQπc,tqi=πc,t i.e. the sum across the year is preserved.

3.2.2.3 Rate of net loss

Conceptual description

Our compartment model estimates NMCP distributions at quarterly intervals through time from manufacturer delivery data and the estimated stock accumulation. The final link to calibrate these quarterly distributions with survey observations of net crop is the rate of net loss. We model the net loss function as a smooth compactly supported function defined previously as part of the NetCALC tool (Koenker et al., 2013). We also model the loss function as non stationary in time, and represent this change through time using a moving average. By using a moving average as opposed to individual loss functions for each year, or quarter, we were able to learn temporal changes given the sparse data and not over represent the prior.

Formal description

We tried several different functional forms for net loss (Weibull, exponential, hill) and decided on specifying the form using a smooth-compact function defined previously (Koenker et al., 2013):

Loss(t,k,L)=ek-k/(1-(t/l)2), t < L                       0, Otherwise (25)

Where k and L are loss function parameters with k,L>0. The smooth-compact loss function produced models with a the lowest information criteria (DIC) from the other forms and has been validated in previous studies (Koenker et al., 2013; Yukich et al., 2013).

For both LLINs and ITNs we use the same functional form in Equation 25 but restrict the bounds on parameter k as uniform priors on this parameter produced strongly non-uniform functions (Bornkamp, 2012). Therefore, to achieve a diffuse uniform prior on the loss functions, we allowed L to vary within large bounds, thereby producing priors that allowed candidate loss functions with half-lives from 0.7 to 5 years. These priors were necessarily vague to allow for adequate flexibility in fitting country-specific loss functions.

k~Uniform(16,18)L~Uniform(4,20.7) (26)

To model the loss function through time, we define a moving average on parameters k and L. Therefore the moving average on the loss function for both LLINs and cITNs is defined as:

kmv,t=kt +kt1++ kt(n1)n (27)
Lmv,t=Lt +Lt1+ + Lt(n1)n (28)

Where n is the moving average lag and t[2000,2001,,2013], and any terms with t<2000 are ignored. From out-of-sample cross validation we found the optimal lag to be n=5, i.e. a balance between over- and under-smoothing. It should be noted that t is restricted to the range 2000t2013, which is the range for which we have real data on NMCP reports, manufacturer reports and household surveys. For the future scenario predictions (described later), we assume any future net loss behaviour is the same as that occurring in 2013, that is:

kmv,t>2013=kmv,t=2013 (29)
Lmv,t>2013=Lmv,t=2013 (30)

3.2.2.4 Calibrating the compartment model against survey information

Conceptual description

Given the net distributions defined at quarterly intervals and the temporally varying loss function, a continuous prediction of the number of nets of any age in a country could be found at quarterly intervals simply by summing across nets of all ages for a given quarter. These quarterly predictions of the number of nets of any age were then calibrated in a likelihood of the observed survey estimates. In the presence of observed survey estimates, this likelihood helped ‘learn’ values for all the prior probability parameters outlined in the compartment model. In the absence of survey information, the model defaulted to the prior probabilities for all parameters and relied on the NMCP and Manufacturer reports.

Formal description

Consider the stock and flow model evaluated over a period 2000:2017, which yielded 73 quarterly intervals. Now consider two 73× 73 matrices labelled MLLIN and McITN. The rows and columns of these matrices represent the entire time period in quarterly intervals.

Consider the stock and flow model progressing column wise through these matrices, at year t and quarter t^Q, the column and row index ind=4t + t^ (e.g 2004.5 or year 5 quarter 3 would be index 23) stores δ^c,tqt^ and πc,tqt^ (LLINs and cITNs in year t and quarter t^Q).

Then for each quarter after the distributions δc,tadjqt^ and πc,tqt^, the remaining nets in subsequent quarters were filled row wise according to the loss function defined in Equation 25.

By summarising the stock and flow process in this manner, the total number of LLIN and cITN nets of all ages in a given quarterly time period was simply the column sums j=173MLLIN,ind, j=173McITN,ind.

Finally, we calibrated estimates of the total number of nets of all ages, against those reported from the household survey reports:

μSURVEYLLINt¯,c~Normal(PLLIN,σSURVEYLLINt¯,c) (31)
μSURVEYcITNt¯,c~Normal(PcITN,σSURVEYcITNt¯,c) (32)

Where the normal standard deviation is given by those found from the survey reports (Equations 2 and 4). Additionally, PLLIN and PcITN was defined as a linear interpolation between the two closest quarters and the average survey time t¯.

It should be noted that, when calculating the nets per-capita, two scalings were used: (1) Countries with the proportion of population at risk being less than 1 were scaled according to the WHO-defined populations at risk proportion. (2) Countries partially dependant on IRS as a means of vector control were scaled as the proportion of the population at risk targeted with ITNs.

4.0 Indicators model

4.1 Introduction

Section 2 provides details on how, using yearly data on manufacturer deliveries and NMCP distributions calibrated using household survey reports, we estimated the number of LLINs and cITNs in households in each country-year at quarterly intervals. From these net crop estimates, we used population information to derive the nets per-capita for LLINs and cITNs. Standardized ITN coverage indicators (Kilian et al., 2013) were then estimated from net crop by leveraging the household survey information with the estimates of nets per capita to derive a set of indicators on net ownership and usage. These were:

  • Indicator 1: % households with at least one ITN

  • Indicator 2: % households with at least one ITN for every two people

  • Indicator 3: % population with access to an ITN within their household

  • Indicator 4: % population who slept under an ITN the previous night

  • Indicator 5: The ratio of population use to access or the ‘ownership gap’

4.2 Indicator model structure

Previous models attempting to evaluate total nets and nets per-capita have utilised negative binomial models unstratified by household size to estimate Indicator 1 (Flaxman et al., 2010). However, using these previous approaches, it was impossible to estimate Indicators 2–5. Here we introduce a new zero-truncated Poisson model stratified by household size, which has the ability to estimate all Indicators 1–5 with excellent precision.

To begin the model derivation, consider a household survey H. Contained within H we were able to calculate a density/histogram of the number of households with a given number of ITNs (both LLINs and cITNs). Appendix figure 2 summarises this density plot, and it is clear that Indicator 1 is trivially calculated (sum of the red bars divided by the total) from this histogram, but Indicators 2–5 are not. Previous modelling approaches (Flaxman et al., 2010) used this unstratified density and assigned a probability distribution (e.g negative binomial or Poisson) parameterised such that the observed density could be recreated using a small number of model parameters (for the negative binomial 2 and Poisson 1).

There are two key problems with this approach, first as highlighted above Indicators 2–5, which provide additional richness of information for decision makers, cannot be directly calculated from this one-dimensional histogram. Second, after experimenting with a large suite of probability distributions, we found that fitting two-dimensional summary histograms to household survey data often provided very poor fits. The key to these poor fits is the lack of stratification of the number of ITNs by household size, the absence of which ignores an important determinant on the number of ITNs per household. A more useful summary of H is the inclusions of a second dimension for household size (Appendix figure 3). From this two-dimensional model, it becomes possible to estimate Indicators 1–3 (Appendix figure 4). However, the problem remains: how do we recreate this two-dimensional density when we have no household survey information?

To accomplish this, we developed a model which, given a household size distribution, translates an estimate of nets per-capita (derived from the compartment model) into an accurate realisation of the three-dimensional histogram.

4.2.1 Zero truncated poisson model

The most logical model to recreate the three-dimensional densities for a given household strata is the Poisson distribution (or a negative binomial distribution for added over dispersion). However, we found that these models did not recreate the observed pattern accurately. We tried more complicated zero-inflated versions but these did not improve fits.

After looking across all 83 surveys for which we had all the relevant information to recreate the three-dimensional histograms, we realised the process to create the histograms had to be separated into two processes: (a) a process which, for a given household strata, gave the density of households with no nets (P0), and (b) a process which, for a given household strata owning nets, gave the density of a given number of nets (1,2,3…) (P1). To think about this intuitively, consider a process that first fills the zero category of ITNs per household in Appendix figure 3, and then fills the categories 1,2,3 etc.

Consider a household strata h (e.g. households of size three persons) from H, it is easy to calculate the proportion of households with no nets. This is the P0,h parameter. Of the remaining households owning one or more nets, we calculated P1,h as the average number of nets in a household strata. The most logical probability distribution to fill the densities given P1,h is again the Poisson distribution as P1,h is simply the mean of the Poisson.

However, because we have already filled the household with no nets density, the correct distribution is a zero truncated Poisson distribution. Unfortunately, the mean of the zero truncated Poisson distribution is now no longer just P1,h but μ=λ/(1eλ), which does not have the same useful interpretation. Therefore, we solved (using a simple root finding) for the value of λ that gave a zero truncated Poisson with the same mean as a standard Poisson with mean P1,h, but excluded a zero category.

Using this model, parameterised by just two parameters per household strata P0,h and P1,h, we evaluated Indicators 1–3 across the 83 relevant surveys with correlation values of more than 0.98 showing that the model reproduced the complex density pattern in the two-dimensional histograms with excellent accuracy.

4.2.2 Zero truncated poisson model in the absence of survey information

The zero truncated Poisson is a probabilistic distribution that translates into parameters P0,h and P1,h that were able to re-create the two-dimensional densities in Appendix figure 4 from which Indicators 1–3 could be calculated. However, we still needed to estimate P0,h and P1,h for country years without survey information.

Given the logical dependence of P0,h and P1,h on the underlying nets per-capita, we created two functions which, given a household strata, h, translated nets per-capita, npc, into P0,h and P1,h. i.e.:

f0(npc,h)=P0,h   and  f1(npc,h)=P1,h (33)

After experimenting with non parametric spline models, we found that simple polynomial surfaces worked remarkably well and had the added benefits of computational efficiency and compatibility with the compartment model.

We divided household sizes into 10 sizes (1,2,3,4,5,6,7,8,9,10) and then modelled f0(npc,h) as:

f0(npc,h)=α0 + β1h + β2h2 + β3npc + β4npc2 + β5npc3 (34)

and f1(npc,h) as:

f1(npc,h)=α1,h + β6,hnpc (35)

The model for f0(npc,h) is therefore a two-dimensional surface that varies according to household size but the model for f1(npc,h) is a separate linear straight line function for each household category.

Equations 34 and 35 were fitted using Bayesian linear regression with the uncertainty in the coefficients being propagated through the compartment model.

When performing 10-fold out-of sample cross validation (leaving out entire surveys), we found that f0(npc,h) predicted P0,h with a correlation of 0.98 and f1(npc,h) predicted P1,h with a correlation of 0.97, indicating extremely good fits.

Given these two functions, which parameterise the zero truncated Poisson model, we can calculate Indicators 1–3 from an estimate of nets per-capita for a country-year (whether we have a survey or not) from the resulting two-dimensional density.

4.2.3 Estimating % population who slept under an ITN the previous night and the 'ownership gap'

The proportion of people who slept under an ITN (Indicator 4) was highly correlated with the proportion of people with access to an ITN (the ‘use gap’, Indicator 5). Therefore, to evaluate Indicator 4, we used a simple linear relationship between access and use evaluated across all 83 surveys with the relevant information (see Appendix figure 5). Therefore, all that was required to evaluate Indicator 4 was to take Indicator 3 (which contained all the rich information about household size strata) and translate it through a linear relationship with noise:

Indicator 4~Normal(0.8838889*Indicator 3,0.06258131^2 (36)

Finally, Indicator 5 (the ownership gap) was calculated as 1–(Indicator 4/Indicator3).

4.2.4 Additional note on household sizes

It should be noted that one missing piece in this analysis is the distribution of household sizes for every country-year. This information does not exist and is very difficult to model. Therefore, we make two assumptions. First, while the populations are known to change over time, we assume that in the 13-year window of our analysis the distribution of household sizes stays constant within each country. This assumption is, to some degree, warranted as countries with serially sampled surveys showed extremely similar household size distribution patterns, and the resulting indicators do not change significantly if a different time point household size distribution is used. Second, for countries with no household size information (due to no relevant surveys), we use an average across all surveys.

5.0 Future predictions

Using the methods described in sections 2 and 3, we were also able to simulate the delivery of any volume of ITNs to a given country over a given future time period to predict the nets per-capita and full suite of indicators. Additionally, we were able to change the dynamics of this simulated future period to allow nets to be retained for a longer period (by varying the net loss function prior) and account for over-allocation of nets (where there is a skewed distribution of net distributions in households with some households having too many nets and some too few).

When simulating forwards in time from 2013, we made several assumptions:

  1. No cITNs were distributed or delivered. This is a justifiable as, with the exception of Gabon in 2013, none of the 40 countries in our analysis delivered or distributed any ITNs in 2012 or 2013. Therefore, it is reasonable to assume that for future years, these countries continued using LLINs exclusively. This assumption also follows country recommendation by the WHO to distribute LLINs and not cITNs (Measure, 2014).

  2. Because only projected information on manufacturer data is known, we assume that all nets delivered to a county are distributed. It should be noted that the long-term consequences of this assumption are insignificant as data from 2000–2013 show that nets that are delivered are ultimately distributed, however, short-term particulars of how many nets are distributed and how many are retained as stock are not captured.

  3. Given a number of nets delivered to all of Africa during the period 2014–2017 (e.g. 500 million LLINs), we use the yearly proportions defined by the harmonisation working group (RBM-HWG) to determine how many nets a country gets in a given year. The working group estimates represent the most reliable projection estimates available.

  4. If the loss function was chosen to be that fitted within the model, we assume the loss function for future years was fixed to that learnt for 2013. We justify this assumption by looking at the changes in loss function on fitted data from 2000–2013, which suggests that the loss function does not vary dramatically and is stable temporally.

5.1 Future scenarios

For the future scenarios the number of nets delivered to all of Africa during the period 2014–2017 was varied from 0 nets to 2.5 billion nets with increments of 100 million nets.

Business as usual

In the business-as-usual scenario, the loss function was fixed to that learned for 2013 and no accounting for over-allocation was performed.

Minimised over-allocation

In the minimised over-allocation scenario (where over allocation is set to zero), the loss function was fixed to that learned for 2013. To account for over-allocation, we ran the compartmental model to estimate net crop, as we would have under the business as usual scenario. However, when evaluating the coverage indicators, we implemented a 'redistribution' algorithm that operated on the modelled two-dimensional histogram, summarising the proportion of households of a given size having a given number of nets for each country-year. From this histogram, we removed those 'surplus' nets from households that that would normally occur due to over-distribution under business-as-usual. These were then redistributed to households with two few nets. This reallocation started with the biggest households with the largest shortfalls and, while surplus nets remained, continued into progressively smaller households until, potentially, every single household had sufficient nets for one-between-two.

Longer net retention

In the longer net retention scenario, we did not use the loss function fitted in the model, but rather defined a fixed loss function with a mean half-life of 3 years. The prior uncertainty was taken as the standard deviation across all actual fitted loss functions calculated retrospectively.

Appendix figure 1. Number of household surveys by country: Each country is shaded to represent the number of household surveys datasets and reports acquired with data on insecticide-treated net ownership and use.

Appendix figure 1.

DOI: http://dx.doi.org/10.7554/eLife.09672.012

Appendix figure 2. Compartment model dynamics for LLINs.

Appendix figure 2.

The compartment model predicts the total number of nets in a country (thick purple line). For a given country at given years there will be LLINs delivered to the country from manufacturers (green arrows), and LLINs distributed at some point in that or subsequent years within the country (blue arrows). Country-wide survey reports give a cross-sectional measure of the total number of nets in households (red line). Evidence synthesis of three data sources can then be inferred by estimating a rate of loss (via a loss function). LLINs, long-lasting insecticidal nets.

DOI: http://dx.doi.org/10.7554/eLife.09672.013

Appendix figure 3. Parameter fits for the hurdle exponential NMCP prior.

Appendix figure 3.

(A,C) are the exponential rate priors on the per capita rates of cITN (A) and LLIN (C) distributions in a given year (see Equations 10 and 11). (B,D) are the Bernoulli hurdle probabilities that zero cITNs (B) or LLINs (D) are distributed in a given year (see Equations 12 and 13). Red points are the data, and blue line are optimised spline fits. cITNs, conventional insecticide-treated nets; LLINs, long-lasting insecticidal nets; NMCP, National Malaria Control Programme.

DOI: http://dx.doi.org/10.7554/eLife.09672.014

Appendix figure 4. Histogram of #ITNs per household as observed in the Nigeria 2010 Malaria Indicator Survey.

Appendix figure 4.

Red bars represent the number of households with one or more nets, and the grey bar represents households with no nets. ITNs, insecticide-treated nets.

DOI: http://dx.doi.org/10.7554/eLife.09672.015

Appendix figure 5. Nigeria 2010 Malaria Indicator Survey household survey summary stratified by household size.

Appendix figure 5.

ITNs, insecticide-treated nets.

DOI: http://dx.doi.org/10.7554/eLife.09672.016

Appendix figure 6. Calculation of indicators 1–3 from two-dimensional histograms for Nigeria 2010 Malaria Indicator Survey.

Appendix figure 6.

(A) % HHs with one or more net, obtained by dividing the sum of red bars (HHs with one or more ITN) by the sum of all bars. (B) % HHs owning enough nets for one-between-two, obtained by dividing sum of red bars (i.e. those HHs meeting or exceeding the one-between-two criteria) by the sum of all bars. (C) % population with access to ITN in the HH, obtained by first converting bars to represent people rather than HHs, then dividing sum of individuals that would be able to access an ITN sharing with one other (red bars in right-hand histogram) with the sum of all individuals (grey bars in left-hand histogram). HHs, households; ITNs, insecticide-treated nets.

DOI: http://dx.doi.org/10.7554/eLife.09672.017

Appendix figure 7. The regression relationship between the proportion of people who slept under an ITN and the proportion of people with access to an ITN.

Appendix figure 7.

ITN, insecticide-treated net.

DOI: http://dx.doi.org/10.7554/eLife.09672.018

Funding Statement

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Funding Information

This paper was supported by the following grants:

  • Medical Research Council K00669X to Samir Bhatt, Peter W Gething.

  • Bill and Melinda Gates Foundation OPP1068048 to Daniel J Weiss, Bonnie Mappin, Ursula Dalrymple, Ewan Cameron, Donal Bisanzio, Peter W Gething.

  • Foundation for the National Institutes of Health U19AI089674 to David L Smith.

  • Bill and Melinda Gates Foundation OPP1110495 to David L Smith.

  • Fogarty International Center to Simon I Hay, David L Smith.

  • Wellcome Trust 091835 to Catherine L Moyes.

  • World Health Organization to Michael Lynch, Cristin A Fergus, Richard E Cibulskis.

  • Global Fund to Fight AIDS, Tuberculosis and Malaria to Jan Kolaczinski.

  • Bill and Melinda Gates Foundation OPP1106023 to Simon I Hay, Peter W Gething.

  • Bill and Melinda Gates Foundation OPP1119467 to Simon I Hay.

  • Wellcome Trust 095066 to Simon I Hay.

  • Bill and Melinda Gates Foundation OPP1093011 to Simon I Hay.

  • Department for International Development to Peter W Gething.

Additional information

Competing interests

SIH: Reviewing editor, eLife.

The other authors declare that no competing interests exist.

Author contributions

SB, Conception and design, Acquisition of data, Analysis and interpretation of data, Drafting or revising the article.

DJW, Analysis and interpretation of data, Drafting or revising the article.

BM, Acquisition of data, Drafting or revising the article.

UD, Acquisition of data, Drafting or revising the article.

EC, Analysis and interpretation of data, Drafting or revising the article.

DB, Analysis and interpretation of data, Drafting or revising the article.

DLS, Analysis and interpretation of data, Drafting or revising the article.

CLM, Analysis and interpretation of data, Drafting or revising the article.

AJT, Acquisition of data, Drafting or revising the article.

ML, Acquisition of data, Analysis and interpretation of data, Drafting or revising the article.

CAF, Acquisition of data, Analysis and interpretation of data, Drafting or revising the article.

JY, Acquisition of data, Drafting or revising the article.

AB, Acquisition of data, Drafting or revising the article.

TPE, Analysis and interpretation of data, Drafting or revising the article.

JK, Acquisition of data, Analysis and interpretation of data, Drafting or revising the article.

REC, Analysis and interpretation of data, Drafting or revising the article.

SIH, Acquisition of data, Drafting or revising the article.

PWG, Conception and design, Acquisition of data, Analysis and interpretation of data, Drafting or revising the article.

Additional files

Supplementary file 1. Surveys used in model fitting, including type of data available.

HH = households; CAR = Central African Republic; DRC = Democratic Republic of Congo; STP = São Tomé and Príncipe.

DOI: http://dx.doi.org/10.7554/eLife.09672.011

elife-09672-supp1.docx (142.7KB, docx)
DOI: 10.7554/eLife.09672.011

References

  1. AMP  net mapping project, working group of the alliance for malaria prevention. http://allianceformalariaprevention.com/working-groups/net-mapping-project/ 2014
  2. Bornkamp B. Functional uniform priors for nonlinear modeling. Biometrics. 2012;68:893–901. doi: 10.1111/j.1541-0420.2012.01747.x. [DOI] [PubMed] [Google Scholar]
  3. Briët OJ, Hardy D, Smith TA. Importance of factors determining the effective lifetime of a mass, long-lasting, insecticidal net distribution: a sensitivity analysis. Malaria Journal. 2012;11:20. doi: 10.1186/1475-2875-11-20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Burgert CR, Bradley SEK, Arnold F, Eckert E. Improving estimates of insecticide-treated mosquito net coverage from household surveys: using geographic coordinates to account for endemicity. Malaria Journal. 2014;13:254. doi: 10.1186/1475-2875-13-254. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Eisele TP, Keating J, Littrell M, Larsen D, Macintyre K. Assessment of insecticide-treated bednet use among children and pregnant women across 15 countries using standardized national surveys. The American Journal of Tropical Medicine and Hygiene. 2009;80:209–214. [PubMed] [Google Scholar]
  6. Eisele TP, Larsen D, Steketee RW. Protective efficacy of interventions for preventing malaria mortality in children in plasmodium falciparum endemic areas. International Journal of Epidemiology. 2010;39 Suppl 1:i88–i101. doi: 10.1093/ije/dyq026. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Eisele TP, Thwing J, Keating J. Claims about the misuse of insecticide-treated mosquito nets: are these evidence-based? PLoS Medicine. 2011;8:e09672. doi: 10.1371/journal.pmed.1001019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Flaxman AD, Fullman N, Otten MW, Menon M, Cibulskis RE, Ng M, Murray CJL, Lim SS, Mueller I. Rapid scaling up of insecticide-treated bed net coverage in africa and its relationship with development assistance for health: a systematic synthesis of supply, distribution, and household survey data. PLoS Medicine. 2010;7:e09672. doi: 10.1371/journal.pmed.1000328. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Gelman A, Carlin JB, Stern HS, Dunson DB, Vehtari A, Rubin DB. Bayesian Data Analysis. CRC press; 2013. [Google Scholar]
  10. Kilian A, Koenker H, Baba E, Onyefunafoa EO, Selby RA, Lokko K, Lynch M. Universal coverage with insecticide-treated nets – applying the revised indicators for ownership and use to the nigeria 2010 malaria indicator survey data. Malaria Journal. 2013;12:314. doi: 10.1186/1475-2875-12-314. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Koenker HM, Yukich JO, Mkindi A, Mandike R, Brown N, Kilian A, Lengeler C. Analysing and recommending options for maintaining universal coverage with long-lasting insecticidal nets: the case of tanzania in 2011. Malaria Journal. 2013;12:150. doi: 10.1186/1475-2875-12-150. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Kilian A, Koenker H, Baba E, Onyefunafoa EO, Selby RA, Lokko K, Lynch M. Universal coverage with insecticide-treated nets - applying the revised indicators for ownership and use to the nigeria 2010 malaria indicator survey data. Malaria Journal. 2013;12:314. doi: 10.1186/1475-2875-12-314. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Killeen GF, Smith TA, Ferguson HM, Mshinda H, Abdulla S, Lengeler C, Kachur SP. Preventing childhood malaria in africa by protecting adults from mosquitoes with insecticide-treated nets. PLoS Medicine. 2007;4:e09672. doi: 10.1371/journal.pmed.0040229. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Kish L. Survey Sampling. New York: Wiley; 1965. [Google Scholar]
  15. Koenker H, Kilian A. Recalculating the net use gap: a multi-country comparison of ITN use versus ITN access. PloS One. 2014;9:e09672. doi: 10.1371/journal.pone.0097496. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Koenker H, Kilian A, Zegers de Beyl C, Onyefunafoa EO, Selby RA, Abeku T, Fotheringham M, Lynch M. What happens to lost nets: a multi-country analysis of reasons for LLIN attrition using 14 household surveys in four countries. Malaria Journal. 2014;13:464. doi: 10.1186/1475-2875-13-464. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Lengeler C. Insecticide-treated bed nets and curtains for preventing malaria. The Cochrane Database of Systematic Reviews. 2004:CD000363. doi: 10.1002/14651858.CD000363.pub2. [DOI] [PubMed] [Google Scholar]
  18. Lim SS, Fullman N, Stokes A, Ravishankar N, Masiye F, Murray CJ, Gakidou E. Net benefits: a multicountry analysis of observational data examining associations between insecticide-treated mosquito nets and health outcomes. PLoS Medicine. 2011;8:e09672. doi: 10.1371/journal.pmed.1001091. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Measure DHS. Demographic and health surveys. http://www.dhsprogram.com/ 2014
  20. Monasch R, Reinisch A, Steketee RW, Korenromp EL, Alnwick D, Bergevin Y. Child coverage with mosquito nets and malaria treatment from population-based surveys in african countries: a baseline for monitoring progress in roll back malaria. The American Journal of Tropical Medicine and Hygiene. 2004;71:232–238. [PubMed] [Google Scholar]
  21. NetWorks The NetCALC tool. http://www.vector-works.org/resources/netcalc-planning-tool/ 2014
  22. Noor AM, Mutheu JJ, Tatem AJ, Hay SI, Snow RW. Insecticide-treated net coverage in africa: mapping progress in 2000–07. The Lancet. 2009;373:58–67. doi: 10.1016/S0140-6736(08)61596-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Paintain LS, Kolaczinski J, Renshaw M, Filler S, Kilian A, Webster J, Lokko K, Lynch M. Sustaining fragile gains: the need to maintain coverage with long-lasting insecticidal nets for malaria control and likely implications of not doing so. PloS One. 2013;8:e09672. doi: 10.1371/journal.pone.0083816. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Plummer M. JAGS: a program for analysis of bayesian graphical models using gibbs sampling. Proceedings of the 3rd International Workshop on Distributed Statistical Computing (DSC 2003) 2003:20–22. [Google Scholar]
  25. RBM Changes to guidance for vector vontrol indicators; Meeting Report of the 17th MERG Meeting; 15–17th June 2011.; New York, USA. 2011. [Google Scholar]
  26. RBM Malaria indicator surveys. http://www.malariasurveys.org/ 2014
  27. RBM RBM partnership programmatic gap analysis. http://archiverbm.rollbackmalaria.org/toolbox/tool_ProgrammaticGapAnalysis.html 2014
  28. RBM. UNICEF. WHO Household survey indicators for malaria control. http://www.rollbackmalaria.org/files/files/resources/tool_HouseholdSurveyIndicatorsForMalariaControl.pdf 2013
  29. RBM-HWG Malaria implementation guidance in support of the preparation of concept notes for the global fund. http://www.rollbackmalaria.org/microsites/gmap/hwg.html 2014
  30. Steketee RW, Eisele TP, Sutherland CJ. Is the scale up of malaria intervention coverage also achieving equity? PLoS One. 2009;4:e09672. doi: 10.1371/journal.pone.0008409. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. UNICEF http://www.childinfo.org/ecd_indicators_mics.html ECD Indicators in Multiple Indicator Cluster Surveys (MICS) 2012
  32. UNPD . World Population Prospects: The 2012 Revision Population Database. New York: United Nations Population Division; 2012. [Google Scholar]
  33. WHO . Methods for Achieving Universal Coverage with Long-Lasting Insecticidal Nets in Malaria Control Vector Control Technical Expert Group Report to MPAC. Geneva: World Health Organization; 2013b. [Google Scholar]
  34. WHO . Global Technical Strategy for Malaria 2016–2030. Geneva: World Health Organization; 2014. [Google Scholar]
  35. WHO . World Malaria Report. Geneva: National Press Club; 2013a. [Google Scholar]
  36. Webster J, Lines J, Bruce J, Armstrong Schellenberg JR, Hanson K. Which delivery systems reach the poor? a review of equity of coverage of ever-treated nets, never-treated nets, and immunisation to reduce child mortality in africa. The Lancet. Infectious Diseases. 2005;5:709–717. doi: 10.1016/S1473-3099(05)70269-3. [DOI] [PubMed] [Google Scholar]
  37. Yukich J, Bennett A, Keating J, Yukich RK, Lynch M, Eisele TP, Kolaczinski K. Planning long lasting insecticide treated net campaigns: should households’ existing nets be taken into account? Parasites & Vectors. 2013;6:174. doi: 10.1186/1756-3305-6-174. [DOI] [PMC free article] [PubMed] [Google Scholar]
eLife. 2015 Dec 29;4:e09672. doi: 10.7554/eLife.09672.021

Decision letter

Editor: Catherine Kyobutungi1

eLife posts the editorial decision letter and author response on a selection of the published articles (subject to the approval of the authors). An edited version of the letter sent to the authors after peer review is shown, indicating the substantive concerns or comments; minor concerns are not usually shown. Reviewers have the opportunity to discuss the decision before the letter is sent (see review process). Similarly, the author response typically shows only responses to the major concerns raised by the reviewers.

Thank you for submitting your work entitled "Coverage and system efficiencies of insecticide-treated nets in Africa from 2000 to 2017" for peer review at eLife. Your submission has been favorably evaluated by Prabhat Jha (Senior editor), Catherine Kyobutungi (Reviewing editor) and two reviewers, one of whom has agreed to reveal her identity: Eline Korenromp.

The reviewers have discussed the reviews with one another and the Reviewing editor has drafted this decision to help you prepare a revised submission.

Summary:

The authors, using data from 102 national surveys, triangulated against delivery data and distribution reports, developed a Bayesian model to generate year-by-year estimates of four ITN coverage indicators: i)% households with at least one ITN; ii)% of children under five who slept under an ITN the previous night; iii)% household with at least one ITN for every two people; and iv)% of the population with access to an ITN within their household.

The authors also explored the impact of two potential 'inefficiencies': uneven net distribution among households and rapid rates of net loss from households. The findings show worsening of over-allocation (a measure of inefficiency) over time as overall net provision has increased, and more rapid ITN loss from households than previously thought. This has implications on the universal coverage (only 77%) that can be expected from the current estimate of 920 million additional ITNs that have been computed as the need through 2017. By improving efficiency, however, the 920 million ITNs could yield population access as high as 95%.

The study presents an important situation analysis, of progress in distribution and resulting protective coverage with ITNs, the primary malaria prevention strategy in most of the high-malaria-burden countries in Africa. The dynamic model developed for this purpose is elegant, building on and refining earlier analysis, and adds value by producing more credible and considerably different estimates of the extent that ITNs are mis-allocated to some households, and the high rates of loss of ITNs from households – with the resulting inefficiencies varying among countries. The up-to-date estimates of ITN use will be useful to other researchers. Building on similar previous work, the authors have developed a model which allows them to break down coverage into estimates of multiple indicators that take account of the variation in household size, and this is an important advance on previous work. The paper also has projections of the coverage achieved if distribution can be done more efficiently or if people retain their nets for longer – this is also novel.

Essential revisions:

The areas of concern include the following:

1) The absence of concrete suggestions on how national malaria control programs could aim to reduce the identified inefficiencies, specifically over-allocation of ITNs. The authors should give examples of the operational strategies for better targeting and/or pre-screening of households' current ITN situation, and indicate which of these has proven to work, in what settings.

2) Different choices could have been made in the modelling, and this makes some of the conclusions more uncertain than what is currently presented. The authors should discuss these points and consider which outputs are most sensitive to the modelling choices. The estimates of the various indicators up to the present might not be much affected, but the projections forwards would be.

3) The net loss function (the rate at which people discard nets) is estimated by combining the numbers of nets entering a country, the number reported as distributed by control programmes, and the numbers owned by households. Flaxman et al. found that nets were retained were much longer than would be expected from matching up the distribution data and coverage data from cross-sectional surveys and as a result they introduced bias parameters to account for the discrepancy between data sources. It is not clear how this was handled in the modelling. The authors should describe this aspect of the methodology and the implications on the findings.

Minor points:

The estimated sharp fluctuations over time in ITN coverage indicators in some countries: are these real (if periodic campaigns dominate distributions?), or reflecting missing or incomplete data, e.g. in NMCP-reported distributions? In particular, comment on the odd time patterns in DRC and Ghana and/or a few other countries with irregular patterns.

[Editors' note: further revisions were requested prior to acceptance, as described below.]

Thank you for resubmitting your work entitled "Coverage and system efficiencies of insecticide-treated nets in Africa from 2000 to 2017" for further consideration at eLife. Your revised article has been favorably evaluated by Prabhat Jha (Senior editor), a Reviewing editor, and two reviewers. The manuscript has been improved but there are some remaining issues that need to be addressed before acceptance, as outlined below:

The reviewers still have concerns about the conclusions on net retention. The model in this paper only estimates ITN retention indirectly while the six studies that Flaxman depended on were directly measuring it. The bias parameters of Flaxman et al. were introduced because of a discrepancy between the different sources of data, if a loss function of longer duration is used. This discrepancy could be due to systematic differences between the data sources, and so the reviewers feel that the conclusion of the present paper that nets are retained for a shorter time than was previously thought is less certain than the authors suggest. This also has implications for the point about sharp fluctuations in the predicted ITN coverage over time, which presumably would be smoother with a loss function of longer duration.

We recommend that the authors conduct a sensitivity analysis that assumes longer retention of nets, showing how any projections change. In any case this uncertainty should be acknowledged and the implications for the projections discussed.

In addition, net loss and retention consider the physical presence of the ITNs in households, but not their effectiveness in repelling mosquitoes. According to manufacturer and WHOPES evidence and guidance on ITN's effective durability, is it justified to assume a median 36-month retention (in the optimized scenario), or does this imply a proportion of ITNs modelled as effective for longer than is realistically assumed? The authors should expound on this point in the discussion – for LLINs specifically

For the country-level estimates: Are the 4 international indicators (ownership, access, usage) calculated and shown for each country's overall national population (including sub-populations at no risk, just as the DHS, MICS and MIS surveys) or for the sub-population at risk of malaria? What assumptions, if any, are made about (mis-)allocation of ITNs to populations not at risk of malaria? Please clarify this in the appropriate section of the manuscript

Minor suggested edits:

If the world limit allows, please define 'Population access' in the Abstract.

Paragraph three, subsection “Net Loss”, 'minimised over-allocation': From the annex (and subsection “Estimating ITN Requirements to Achieve Universal Access”) it seems that this is ZERO over-allocation; please qualify or rephrase accordingly.

Appendix 1—figure 2:

a) Parts of the diagram are confusing or not clear – please correct as needed:b) An arrow (or, equality sign) is missing from HH-level ITN ownership to the box below (National total ITNs in HH?). c) An arrow missing from National net crop to, Observed bivariate histogramsd) National net crop equals National total ITNs in HH 5 boxes below, but the equality is not indicated.

Appendix 1: The scaling across countries to get to continent-totals is not very clear.

eLife. 2015 Dec 29;4:e09672. doi: 10.7554/eLife.09672.022

Author response


Essential revisions:

The areas of concern include the following:

1) The absence of concrete suggestions on how national malaria control programs could aim to reduce the identified inefficiencies, specifically over-allocation of ITNs. The authors should give examples of the operational strategies for better targeting and/or pre-screening of households' current ITN situation, and indicate which of these has proven to work, in what settings.

In this work we sought simply to identify and measure the magnitude of over-allocation and under-retention that exists and the changes in these factors over time. We felt this was a vital initial step and, of course, agree that it now paves the way for subsequent work to explore strategies for better targeting. We would propose, though, that this substantial body of new work would form one (or likely many) new studies in their own right, and would not be appropriately placed within the current submission which already covers a lot of new ground.

Solving the over allocation problem from an operational standpoint is something currently being discussed in policy fora. In the early years of integrated campaigns nets were allocated to a specific target group (e.g. children under five), but since the shift of policy to universal coverage this has become more difficult. Currently the distribution of nets requires a registration process within a county in order to identify beneficiaries thereby leaving large scope for under- and over-allocation. Work has been done at the Global Fund to strengthen the registration process but they have yet to identify concrete solutions that work at scale. Mozambique, Burundi and Madagascar have implemented systems based on village hierarchy that have been partially effective at small scales, but these do not lend to implementations at the scale of our analysis (country or continent level). Increasingly it is thought that solutions to the problem of over allocation need to be addressed in the planning stages rather than post hoc.

Given these complications, it is clear that defining strategies to address over-allocation in the field will remain a challenge – one to which modelling may be able to make a contribution alongside those with expertise of the reality of the logistics on the ground. For now, however, we maintain that by identifying that the phenomenon of over-allocation exists, is widespread, and contributing substantially to lower coverage yields, our study makes an important first step in motivating future work to improve distribution systems in-country.

2) Different choices could have been made in the modelling, and this makes some of the conclusions more uncertain than what is currently presented. The authors should discuss these points and consider which outputs are most sensitive to the modelling choices. The estimates of the various indicators up to the present might not be much affected, but the projections forwards would be.

We thank the reviewers for this comment. We felt that, because of the policy nature of this paper, most modelling assumptions were better described in full in the Appendix. For every component of the model we provide a conceptual and formal distribution of the model that outlines all the assumptions made and justifications for why we made them. As the reviewers expressed concern for the projection assumptions, we have added additional text to the assumptions section for future projections in the Appendix (Future predictions).

3) The net loss function (the rate at which people discard nets) is estimated by combining the numbers of nets entering a country, the number reported as distributed by control programmes, and the numbers owned by households. Flaxman et al. found that nets were retained were much longer than would be expected from matching up the distribution data and coverage data from cross-sectional surveys and as a result they introduced bias parameters to account for the discrepancy between data sources. It is not clear how this was handled in the modelling. The authors should describe this aspect of the methodology and the implications on the findings.

The Flaxman et al. paper was the inspiration for this work. However, there were several fundamental problems with their work. Their loss function was constant over a given year with very minor reductions for subsequent years until all nets are discarded at once after three years (see brown line Figure 7). This function is almost piece-wise linear and does not resonate with available evidence on net retention from field settings (e.g. see Killian et al.’s Net Calc). This limitation of their loss function was compensated for by introducing a ‘bias’ parameter that helped triangulate the entire modelling chain. As a result of this statistical compensation we do not believe their loss function represented reality particularly well – it is inconceivable that everyone in a given country discards all their nets at once at the three year mark. Additionally the Flaxman et al. net loss was fitted from 6 studies outside the main compartment model and fixed, whereas we learn a data driven, time varying loss function directly from the survey, manufacturer and NMCP data. We have discussed these limitations in the Methods section.

Minor points:

The estimated sharp fluctuations over time in ITN coverage indicators in some countries: are these real (if periodic campaigns dominate distributions?), or reflecting missing or incomplete data, e.g. in NMCP-reported distributions? In particular, comment on the odd time patterns in DRC and Ghana and/or a few other countries with irregular patterns.

We believe these fluctuations to be real. The priors on distributions are set as uniform and therefore in the absence of concrete data driving these sharp fluctuations the model will prefer smoother distributions. Sharp fluctuations only arise when triangulation between the NMCP, manufacturer and survey data demand it. In the absence of data the model will choose a more parsimonious and even distribution pattern. It should also be noted that these sharp distributions are accompanied by full Bayesian credible intervals capturing our uncertainty about the distribution pattern (if that uncertainty is there).

[Editors' note: further revisions were requested prior to acceptance, as described below.]

The reviewers still have concerns about the conclusions on net retention. The model in this paper only estimates ITN retention indirectly while the six studies that Flaxman depended on were directly measuring it. The bias parameters of Flaxman et al. were introduced because of a discrepancy between the different sources of data, if a loss function of longer duration is used. This discrepancy could be due to systematic differences between the data sources, and so the reviewers feel that the conclusion of the present paper that nets are retained for a shorter time than was previously thought is less certain than the authors suggest.

The key point of contention here revolves around how the NMCP distribution data triangulates with the loss function. We disagree that the measurement of the loss function, as modelled in Flaxman et al., is suitable and disagree that the bias parameter allows a loss function of longer duration. Below we explain the shortcomings of the Flaxman et al. approach and the true purpose of the bias parameter.

There are two modelling choices that can be made here:

1) The approach of Flaxman et al. – use a loss function based on 6 in field studies, and adjust the NMCP distribution data for a hypothesised systematic bias.

2) Our approach – learn a highly flexible loss function directly from the data and only adjust the NMCP distribution data for random noise.

To convince you that our approach is inherently a less subjective modelling choice consider the two following issues with the Flaxman et al. approach:

a) Net loss function:

Flaxman et al. paramaterised their loss function based on 6 studies with differing time points under no standardised conditions between studies. In these highly controlled studies key factors of net loss predominantly driven by living conditions, household behaviour and attitudes (Killian et al., 2015) are unaccounted for by the very nature of the study. Additionally some of the studies used in Flaxman et al. are retrospective studies where recall bias was a significant and unaccounted for confounder (Killian et al., 2015). We also must stress again just how implausible the loss function resulting from these studies is – the function is essentially constant (reductions with an average of 5% per year) for 3 years and then a complete stepwise reduction to zero at the 3-year mark. Under this loss function, using the upper 95% confidence interval, at most only 30% of nets can be lost after 3 years; immediately after which all nets are discarded. We believe that the implausibility of this loss function highlights the inconsistencies in paramaterising a function based on a small sample of unstandardized studies.

In contrast to this approach, we learn the loss function directly from the data, imposing an agnostic uninformative prior on the underlying half-life and duration. In our approach nets are lost smoothly over time (with no sudden discards) and the model has the choice of a long or short half-life depending the country and the year. From our fitted function (Figure 7) we find large variability between countries that would not be able to be represented using the Flaxman et al. loss function. Additionally, we actually say the duration of nets is longer than what Flaxman et al. say, with nets lasting up to 5 years on average (Figure 7 red curve). The key disagreement between our fitted loss function and that of Flaxman et al. is the year-to-year rate of reduction, and that all nets are lost exactly on the three year mark.

To further assure the reviewers of the implausibility of Flaxman et al. loss function, consider looking at the raw household survey data and extracting information on net age directly from each survey. When averaged across all surveys used in our study the average proportion of the total number of nets that were:

Less than one year old was 57%,

Between one and two years old was 20%

Between two and three years old was 5%

Greater than three years old was 18%

Obviously there are a lot more complexities to these simple summaries, including when the survey was sampled, the distribution pattern, recall bias etc. But one thing is clear, the year-to- year loss rate is very non-linear and probably much greater than 5% (which Flaxman et al. claim), and a large number of nets are older than 3 years (which the Flaxman et al. does not allow). This once again calls into question the plausibility of the Flaxman et al. loss function, but is consistent with our loss function.

b) The bias parameter:

The bias parameter in Flaxman et al. is used to systematically adjust the NMCP distribution data to match the household survey data. This is essential in their approach as given their constrained loss function (described above) the survey data would not match the distribution data without some form of adjustment. There are several conceptual and implementation problems with this bias parameter. Firstly this bias parameter is a very poor linear fit (see Figure 5, Supplementary information Flaxman et al.), and does not account for the sampling time of the survey or the recall bias inherent in the survey questions. The poor linear fit suggests substantial country specific variation. Second the prior does not model country specific variation making the assumption that all NMCP distribution data is biased in the same way.

Conceptually the problem with the bias parameter is a reliance on the assumption: the NMCP data is systematically biased (constant) across all countries and years. Flaxman et al. provide no justification for this assumption, and we believe it to be very unlikely that all NMCP data, reported from different countries, in different years are systematically biased in the same way. This is the equivalent to saying that for every country the number of nets reported to be distributed should be scaled by the same constant.

To further prove to the reviewers that using a bias parameter is not a sound modelling choice to explain the data, consider a like for like re-run of the Flaxman et al. bias parameter analysis. We take the weighted survey data totals for the survey questions:

Is your net less than one year old?

Is your net between 1 to 2 years old?

Is your net between 2 to 3 years old?

And regress these (on a logarithmic scale) against the corresponding NMCP distribution results in that year (with linear interpolation to match the average year of the survey). If the Flaxman et al. hypothesis of the NMCP data being systematically biased holds the intercepts of the three linear models (in Author response image 1 where the intercepts are in the brackets, the regression bias line is in green and the one-to-one line is in red) should be the same. However, these intercepts are orders of magnitude different (note the logarithmic scale) suggesting steep losses of nets and no systematic bias. A far more parsimonious explanation is that the differences in the number of LLINs from the Survey and NMCP are explained by the loss of nets occurring through time. This is supported by the increasing bias parameter, as nets get older. Indeed this analysis suggests that the sole purpose of Flaxman et al. bias parameter is to compensate for the deficiencies in their poorly characterised loss function.

Author response image 1.

Author response image 1.

DOI: http://dx.doi.org/10.7554/eLife.09672.019

In contrast we do not assume any systematic bias on the NMCP data, we only assume that the NMCP data contains random noise and let the variation be described by a flexible loss function.

Therefore in summary we hope we have made clear that opting for modelling approach (2) over (1) is a considerably more parsimonious choice making far fewer assumptions and letting the data speak. We therefore have high confidence in the fits of our modelled loss functions. We stress that we have been completely transparent with the modelling assumptions in the Appendix, and have made very few subjective decisions and focused on a data driven approach.

This also has implications for the point about sharp fluctuations in the predicted ITN coverage over time, which presumably would be smoother with a loss function of longer duration.

We believe the reviewers are mistaken that the cause of sharp fluctuations in ITN coverage is due to the loss function. The predominant and overwhelming cause of these fluctuations is the delivery and distribution data, not the loss function. Sharp fluctuations arise due to real, large distributions of nets captured in both the distribution and survey data. See a typical example of this below for Togo.

In Author response image 2, green dots are the manufacturer deliveries, purple triangles are the NMCP distributions, red triangles are the house-hold surveys, the red line is the number of LLINS, the yellow line is the number of cITNs and the black line is the residual stock. The fluctuations are caused by real large deliveries and distributions causing a saw tooth pattern in the number of nets. The reason the number of nets does not increase exponentially with the deliveries is due to loss, but this loss is smooth over time. Using the Flaxman et al. loss function would cause far less smoothness due to restriction of all nets being discarded at the three-year mark. To highlight this consider what would happen in the above plot if all nets were discarded after 3 years. As highlighted in the previous rebuttal the distribution of NMCP ITNs is uniform intra- years, as nothing in the data suggests a non-uniform pattern, thereby implicitly choosing a smooth distribution pattern unless the data would suggest otherwise.

Author response image 2.

Author response image 2.

DOI: http://dx.doi.org/10.7554/eLife.09672.020

We recommend that the authors conduct a sensitivity analysis that assumes longer retention of nets, showing how any projections change. In any case this uncertainty should be acknowledged and the implications for the projections discussed.

Given the above discussion about the suitability of our loss function we feel the reviewers will agree that a sensitivity analysis is unnecessary given our framework. If we used a loss function such as that in Flaxman et al., this sensitivity analysis would be warranted to test the validity of using such a constrained loss function. However, given the data driven, assumption free nature, of our analysis this is unnecessary as the sensitivity analysis is implicitly contained within the Bayesian framework. That is, during every fit, we explore a full range of candidate functions from differing retention times to sharp and smooth transitions and average over these to find the best function that fits the data. Therefore, during every fit, we perform a statistically robust sensitivity analysis. To state this differently our model allows for a loss function similar to that of Flaxman et al., but this loss function is not supported by the data at all.

That said, one of the important results of this paper is exploring the effects of a longer retention net half-life (3 year half-life). These results are presented and discussed in the main paper (Figure 8 red line) and in Appendix 1.

In addition, net loss and retention consider the physical presence of the ITNs in households, but not their effectiveness in repelling mosquitoes. According to manufacturer and WHOPES evidence and guidance on ITN's effective durability, is it justified to assume a median 36-month retention (in the optimized scenario), or does this imply a proportion of ITNs modelled as effective for longer than is realistically assumed? The authors should expound on this point in the discussion – for LLINs specifically

The scope of this paper was restricted to ownership and usage indicators and not on the effectiveness of nets once they are being used. Of course the insecticidal and barrier effectiveness of a nets diminish in a complex multifaceted way through time but due to these complexities, we did not expand our analysis to effectiveness and focus on the standard RBM definitions of ownership and use. We have expanded on this point in the Discussion.

For the country-level estimates: Are the 4 international indicators (ownership, access, usage) calculated and shown for each country's overall national population (including sub-populations at no risk, just as the DHS, MICS and MIS surveys) or for the sub-population at risk of malaria? What assumptions, if any, are made about (mis-)allocation of ITNs to populations not at risk of malaria? Please clarify this in the appropriate section of the manuscript

We use population at risk as the denominator in all the analysis in this paper and in doing so we avoid the problem of biased estimates by including populations not at risk. This however does make the assumption that there is no allocation of nets outside populations at risk. We have added a line in the main manuscript explicitly stating this assumption and thank the reviewers for pointing out this omission. However it is also vital to recognise that this adjustment by populations of risk only applies to a handful of countries in sub-Saharan Africa (e.g. Ethiopia, Kenya etc.) as all others are universally at risk.

Minor suggested edits:

If the world limit allows, please define 'Population access' in the Abstract.

These terms are fully described in the Introduction, subsection “Modelling coverage”.

Paragraph three, subsection “Net Loss”, 'minimised over-allocation': From the annex (and subsection “Estimating ITN Requirements to Achieve Universal Access”) it seems that this is ZERO over-allocation; please qualify or rephrase accordingly.

We thank the reviewers for this suggestion and have clarified this statement in the Methods and Appendix.

Appendix 1—figure 2:

a) Parts of the diagram are confusing or not clear – please correct as needed:b) An arrow (or, equality sign) is missing from HH-level ITN ownership to the box below (National total ITNs in HH?). c) An arrow missing from National net crop to, Observed bivariate histogramsd) National net crop equals National total ITNs in HH 5 boxes below, but the equality is not indicated.

Appendix 1: The scaling across countries to get to continent-totals is not very clear.

Figure 2 describes the Bayesian compartment model only, we choose not to tie together the Poisson model for the indicators as felt this would make the figure too complex.

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    Supplementary file 1. Surveys used in model fitting, including type of data available.

    HH = households; CAR = Central African Republic; DRC = Democratic Republic of Congo; STP = São Tomé and Príncipe.

    DOI: http://dx.doi.org/10.7554/eLife.09672.011

    elife-09672-supp1.docx (142.7KB, docx)
    DOI: 10.7554/eLife.09672.011

    Articles from eLife are provided here courtesy of eLife Sciences Publications, Ltd

    RESOURCES