Does ignoring transmission dynamics lead to underestimation of the impact of interventions against mosquito-borne disease?

Abstract

New vector-control technologies to fight mosquito-borne diseases are urgently needed, the adoption of which depends on efficacy estimates from large-scale cluster-randomised trials (CRTs). The release of Wolbachia-infected mosquitoes is one promising strategy to curb dengue virus (DENV) transmission, and a recent CRT reported impressive reductions in dengue incidence following the release of these mosquitoes. Such trials can be affected by multiple sources of bias, however. We used mathematical models of DENV transmission during a CRT of Wolbachia-infected mosquitoes to explore three such biases: human movement, mosquito movement and coupled transmission dynamics between trial arms. We show that failure to account for each of these biases would lead to underestimated efficacy, and that the majority of this underestimation is due to a heretofore unrecognised bias caused by transmission coupling. Taken together, our findings suggest that Wolbachia-infected mosquitoes could be even more promising than the recent CRT suggested. By emphasising the importance of accounting for transmission coupling between arms, which requires a mathematical model, we highlight the key role that models can play in interpreting and extrapolating the results from trials of vector control interventions.

Summary box

• A cluster-randomised trial has established that the release of Wolbachia-infected mosquitoes could be a highly effective intervention to reduce dengue virus transmission.

• Cluster-randomised trials of interventions for infectious diseases can be subject to several biases when people or vectors move between areas covered by different arms of the trial, making the trial arms more similar.

• We highlight three sources of bias that could be present in cluster-randomised trials of interventions against vectorborne diseases, the largest of which is due to not accounting for the coupling of transmission between trials arms.

• The amount of bias introduced is likely to be greater for interventions of intermediate efficacy than those with high or low efficacy.

• When designing and interpreting future trials it is important to properly account for transmission dynamics, and perhaps where needed to employ mathematical modellers in the analysis.

Introduction

Dengue virus (DENV) poses a risk to around half the world’s population due to the widespread abundance of its Aedes mosquito vectors.¹ Historically, the success of dengue control has been limited by challenges such as the expanding distribution of Aedes aegypti due to urbanisation and land-use changes, and ineffective or suboptimally applied control strategies.^{2 3} One novel control strategy that holds promise is the release of mosquitoes infected with Wolbachia, a vertically transmitted intracellular bacteria that reduces the ability of Aedes aegypti mosquitoes to transmit DENV.⁴ A cluster-randomised, controlled trial conducted between 2018 and 2020 in Yogyakarta, Indonesia (Applying Wolbachia to Eliminate Dengue, AWED)^{5 6} estimated that release of Wolbachia-infected mosquitoes had a protective efficacy against symptomatic, virologically confirmed dengue of 77.1% (95% CI 65.3% to 84.9%).⁷

Potential sources of bias in Wolbachia trials

There are at least three factors that can result in underestimated efficacy in this type of trial. All operate by making outcomes in treatment and control clusters appear more similar than if these factors were not at play, although they result in this for different reasons. First, the movement of humans between control and treated clusters means that individuals spend time outside of their allocated arm. This can reduce the time which study subjects residing in treatment clusters spend exposed to Wolbachia, as they will be spending more time in the control clusters, and vice versa.⁸ Second, the movement of mosquitoes between areas covered by different arms of the trial can lead to an appreciable proportion of mosquitoes in control clusters infected with Wolbachia, as Wolbachia-infected mosquitoes move from treatment to control clusters.⁷ This will lower these mosquitoes’ ability to transmit DENV and introduce a source of contamination across trial arms. This effect will occur much more strongly for the replacement strategy, when both male and female mosquitoes are released, and it is this strategy which we focus on here. Third, the dynamic, spatially localised nature of DENV transmission^{9 10} implies that suppression of transmission in treated clusters could influence transmission in neighbouring control clusters, thereby reducing incidence in both trial arms. While the first two effects directly affect individuals’ exposure to the intervention, this third effect describes the indirect effect due to DENV infection incidence depending on the prevalence in both arms. This occurs because reduced prevalence in the intervention arm results in less transmission and fewer introductions to the control arm, and vice versa. Hereafter, we refer to each of these three forms of bias as ‘human movement’, ‘mosquito movement’ and ‘transmission coupling’, respectively.

In their per-protocol analyses, Utarini et al⁷ acknowledged the potential effects of human and mosquito movement in their per-protocol analysis, and by incorporating recent travel and Wolbachia prevalence into their efficacy calculations did not detect a difference in efficacy from that estimated in the intention-to-treat analysis. Nevertheless, the analysis of the AWED trial by Utarini et al⁷ did not account for transmission coupling, and they noted that follow-up analyses were needed to further explore the potential for bias due to human and mosquito movement.

Understanding the magnitude of such biases is important when seeking to extrapolate the impact of interventions across contexts. Such extrapolation has been recently undetaken for the RTS,S/AS01 vaccine^{11 12} and the endectocide ivermectin¹³ for malaria. If failing to account for such transmission dynamics contributes to an underestimated biological effect of Wolbachia on DENV, we risk incorrectly assessing its broader impact. Given the myriad intervention options available to public health officials for dengue control,¹⁴ it is important for the potential impacts of each to be understood as well as possible.

Using mathematical models to understand sources of bias

In this analysis, we used a mathematical model of DENV transmission to gain insight into the possible magnitudes of the three aforementioned sources of bias. Our approach involved translating model inputs of the basic reproduction number (R₀), the spatial scale of human movement (b), and the proportional reduction in R₀ afforded by Wolbachia-infected mosquitoes (ε) into outputs of the infection attack rate (IAR) in control and treatment arms of a trial, in accordance with a seasonal, two-patch susceptible-infectious-recovered model.¹⁵ We used the outputs of IAR in treatment and control arms (IAR_t and IAR_c, respectively) to obtain an estimate of the OR of infection and, thereby, an estimate of the efficacy of the intervention, Eff=1 - OR. We constructed six different model versions for estimating efficacy, each of which includes different combinations of the three biases, all of them or none of them. Henceforth, we refer to the efficacy observed in the AWED trial as ‘observed efficacy’, and the efficacy estimated by a given model and ε as ‘estimated efficacy’. Finally, we quantify each bias as the difference in the efficacy estimated by a model including that bias and a model which does not include that bias (see online supplemental methods for more details).

We assumed a checkerboard pattern of control and treatment arms of 1 km² to approximate the design used in the AWED trial, which covered the entire city of Yogyakarta, with neighbouring areas assigned to one arm or another in an (approximately) alternating pattern (figure 1A).⁷ We assume that individuals are evenly distributed within each cluster such that they have no internal spatial structure. The time that humans spend away from their home is assumed to follow a Laplace distribution (figure 1A, top right), which takes a single parameter, b, that we refer to as the scale of human movement. One way to conceptualise this would be that, at a randomly selected time in the day, the probability that an individual will be a given distance from home is given by the Laplace distribution. By assuming that individuals are evenly distributed within each cluster, we can then estimate the average proportion of time that individuals in each trial arm spend in their own arm (ρ_tt and ρ_cc) and in the opposite arm (ρ_tc and ρ_ct—see the apportionment of time at risk section in Supplementary Methods for details). Larger values of b imply that people spend less time in their allocated arm, and for large values of b individuals spend roughly equal amounts of time in both arms (figure 1B).

Figure 1.

The spatial scales of transmission and trial design. (A) Idealised trial design. We used a checkerboard pattern to approximate the design of the AWED trial of Wolbachia-infected mosquitoes to control dengue.⁷ ρ_ij represents the amount of time an individual who lives in arm i spends in arm j, where i and j can represent either control (c) or treatment (t). b describes the scale of human movement. The Laplace distribution shown in the top right is illustrative, and does not represent the distribution used in the model, which was much narrower. (B) The relationship between the scale of human movement and the amount of time individuals spend in clusters of the same type as their home cluster. (C) The relationship between the reduction in R₀ (ε) required to reproduce the observed efficacy in the AWED trial and the time people spend in their allocated arm. In this panel and panels (E, F), the dark blue line corresponds to the observed mean estimated in the AWED trial whereas the light blue line and shaded region correspond to the 95% CIs. (D) The relationship between ε and the estimated efficacy when b=36.9 m. The black line shows the theoretical relationship between a reduction in R₀ and observed efficacy, assuming no mosquito movement and no human movement between arms. The blue line shows this relationship if we include these two factors as well as the effect of transmission coupling. The dark and light blue squares indicate the mean and the 95% CI, respectively, of the observed efficacy in the AWED trial and the corresponding reduction in R₀. (E) The relationship between the amount of time people spend in their allocated arm and the estimated efficacy. (F) The relationship between the size of the clusters and the estimated efficacy. The dashed line indicates the estimated efficacy at the baseline cluster size (1000 m). In all panels, parameters are at their baseline given in online supplemental table S1 unless otherwise stated. AWED, Applying Wolbachia to Eliminate Dengue.

How might the spatial scale of transmission and the dimensions of trial clusters affect efficacy estimates?

The relationship between the efficacy estimated by the model with all three forms of bias (the estimated efficacy) and the reduction in R₀ (ε) was dependent on the amount of time people spent in their allocated arm (figure 1C)—the less time individuals spent in their allocated arm, the higher the reduction in R₀ that was needed to recreate the observed efficacy from the AWED trial. If individuals spent less than 83.9% of their time in their allocated arm, it was impossible to generate the observed efficacy (77.1%), as that would have implied that ε exceeded 1. Assuming that individuals spent 92.9% of their time in their allocated arm (ie, ρ_ii= 92.9%, corresponding to b=36.9 m—see the Spatial Scale of Human Movement section in Supplementary Methods for details and justification), we found that the observed efficacy (77.1% (95% CI: 65.3% to 84.9%)) corresponded to an ε of 49.9% (95% CI: 30.8% to 73.1%) (figure 1D, blue line). If we instead assumed that there was no movement between trial arms, we observed that much smaller values of ε were needed to explain the observed efficacy (6.3% (95% CI: 4.8% to 8.1%)). The difference between these estimates provides an indication of the extent of bias introduced by assuming that humans and mosquitoes remain in their allocated arms, when they in fact do not (figure 1D). We also explored the impact of assuming higher levels of movement between trial arms (b=58.4 m, the maximum amount of movement such that our model can reproduce the 95% CI of efficacy observed in the trial), finding qualitatively similar patterns (online supplemental fig. S11). In this case, the observed efficacy corresponded to an ε of 72.8% (95% CI: 47.2% to 100%).

When we fixed ε to the value that reproduces the observed efficacy in the AWED trial and increased human movement between arms by increasing b, the estimated efficacy by the model accounting for all three forms of bias decreased (figure 1E). For example, increasing the average distance in one direction between transmission pairs (b) from 36.9 m to 70 m caused a relative reduction of 20.0% in estimated efficacy, highlighting the sensitivity of efficacy to the spatial scale of human movement. This effect occurs for two reasons: first, as people spend less time in their allocated arm, the proportion of time that people spend under the intervention becomes more similar between arms; and second, in the presence of transmission coupling, a reduction in prevalence in the intervention arm reduces transmission in the control arm more as people spend less time in their allocated arm. Relatedly, estimated efficacy depended on the dimensions of the trial clusters, which we set to 1 km² by default (figure 1F). When we reduced the cluster dimensions to 500 mx500 m, estimated efficacy dropped from 77.1% to 60.3%, representing a 21.8% relative reduction. This effect occurs because, as the cluster dimensions are reduced, people spend less time in their home cluster. Hence, the time spent in each trial arm approaches parity (ie, 50%). Increasing cluster dimensions above 1 km² had somewhat less of an effect on estimated efficacy. For example, increasing the cluster dimensions to 2 km x 2 km resulted in an estimated efficacy of 86.7%, a relative increase of 12.4%.

What are the relative contributions of the different sources of bias in efficacy estimates?

Our approach enabled us to directly and separately model each of the three potential sources of bias: (1) mosquito movement, (2) human movement, and (3) transmission coupling. Movement of Wolbachia-infected mosquitoes is modelled by including a time-varying level of coverage, and we assume that mosquito movement does not contribute to DENV transmission (see Supplementary Methods—wMel coverage). When we assumed that ε was equal to 49.9%, allowing for mosquito movement but not human movement produced an estimated efficacy of 99.1%, because there was almost no transmission in the intervention arm in that case (figure 2A, online supplemental fig. S7). If we allowed for both mosquito movement and human movement, we observed a lower estimated efficacy of 93.6%. Although there was little transmission in the intervention arm in this case, individuals residing in the intervention arm could be infected in the control arm. Additionally, those assigned to the control arm experienced lower overall risk due to their time spent in the intervention arm. When we accounted for transmission coupling between trial arms alongside human and mosquito movement, thereby allowing for more transmission in the intervention arm, risk was the most similar across the trial arms of all scenarios, leading to the lowest estimated efficacy of 77.1% for an ε equal to 49.9%.

Figure 2.

Sources of bias in efficacy estimates. In all panels, yellow refers to mosquito movement, red to human movement and blue to transmission coupling. (A, B) use the baseline value of b=36.9 m, while (C, D) use a larger value of b=58.4 m. (A) The relationship between the reduction in R₀ (ε) and the estimated efficacy for the six possible models. The black line here is the relationship for a model with no human movement or mosquito movement. Where a line has more than one colour, it represents the model which includes each of the types of bias represented by those colours. The difference between this line and each of the coloured lines represents the bias introduced by not accounting for the features present in the model described by that coloured line. (B) The contribution of each source of bias to the total bias. Eff⁽⁰⁾ refers to the estimated efficacy from a model with none of the biases, Eff^(h) to the estimated efficacy from a model with human movement only, Eff^(m) to the estimated efficacy from a model with mosquito movement only, Eff^(hm) to the estimated efficacy from a model with human and mosquito movement, Eff^(ht) to the estimated efficacy from a model with human movement and transmission coupling, and Eff^(hmt) to the estimated efficacy from a model with all three biases. (C, D) As in (A, B) but with b=58.4 m.

We quantified total bias as Eff^(hmt) − Eff⁽⁰⁾, where Eff^(hmt) is the estimated efficacy under the model with all sources of bias and Eff⁽⁰⁾ is the estimated efficacy under the model without human or mosquito movement. We then computed the difference in the bias produced by pairs of models to decompose overall bias into each of its three sources (figure 2B, online supplemental fig. S8-9, see Supplementary Methods for details). At the baseline ε of 49.9%, 17.6% of the total bias was attributable to mosquito movement, 8.3% to human movement, and 74.1% to transmission coupling. At all values of ε, the greatest source of bias was transmission coupling between trial arms. When ε was below a value of around 10%, the effective reproduction number at the start of the trial exceeded one in both arms. This value of ε varied slightly based on the model used (online supplemental fig. S8-9). If ε was below this critical value, increasing it in the context of coupled transmission reduced incidence in the control arm and caused smaller reductions in incidence in the intervention arm than if transmission had been uncoupled (online supplemental fig. S7, eg, panels D vs F). This implies that the bias introduced by transmission coupling increases as ε increases up to ~10% under our model’s parameterisation (figure 2B). Increasing ε past this point only leads to small reductions in incidence in the intervention arm in an uncoupled model, as incidence is already very low. Assuming more human movement between trial arms (b=58.4 m) led to qualitatively similar findings (figure 2C,D).

How can these biases be mitigated?

Biases arising due to human movement and mosquito movement are potentially addressable through careful statistical analysis of trial data or in the design of the trial.⁸ For instance, in the per-protocol analysis of the AWED trial, Utarini et al accounted for these two forms of bias by combining self-reported recent travel and local Wolbachia prevalence into an individual-level Wolbachia exposure index.⁷ Comparing groups with the highest and lowest Wolbachia exposure did not lead to higher efficacy estimates than their primary analysis. Another approach to addressing contamination involves describing the effectiveness of the intervention at the boundary between clusters using a sigmoid function.^16–18 Our results suggest that failure to take steps such as this to account for human and mosquito movement would typically lead to underestimated efficacy, while failure to account for transmission coupling would lead to an even greater underestimate, particularly at intermediate reductions in R₀.

Bias arising from human and mosquito movement could also be mitigated at the stage of planning the trial. The classical design to achieve this is the ‘fried egg’ design, in which the entirety of each cluster experiences their allocated conditions (ie, either intervention or control), but only the central part of each cluster is used for analysis.¹⁹ The principle behind this is that the samples selected for analysis are surrounded by a region receiving the same conditions, reducing the contamination between clusters, and more closely matching the conditions of full-scale implementation of an intervention. One drawback of the fried egg approach is that it assumes homogeneity within clusters, when in practice the central part of a cluster may be qualitatively different from the outer region—this likely limits the utility of this design for trials taking place in cities, such as the AWED trial. A more recently proposed approach involves excluding a subset of clusters from the trial completely, thereby increasing the distance between clusters and leading to disconnected clusters at less risk of contamination.²⁰ This approach is based on the same principle as the fried egg design, but does not require homogeneity within clusters. While both of these approaches do mitigate the risk of contamination directly, they also necessitate a larger trial area and may be logistically infeasible in a trial taking place in a single city, as was the case for the AWED trial. Another approach could include reducing the number of clusters, but keeping the total area fixed, leading individuals to spend more time in their assigned arm and reducing mosquito movement by reducing the boundary between clusters. This approach also comes with disadvantages: first, fewer clusters is more likely to result in systematic differences between trials arms; and second, fewer clusters will lead to reduced replication in trials where the cluster is the unit of analysis. Our results show that the efficacy estimated from cluster-randomised, controlled trials of interventions against mosquito-borne diseases is highly sensitive to cluster size (figure 1F). Had the dimensions of the clusters in the AWED trial been much smaller, then the estimated efficacy may have been substantially lower. However, having fewer, yet larger clusters would likely introduce new biases by making the arms less comparable, which may not be an acceptable trade-off.

While bias due to human and mosquito movement can be mitigated through trial design and statistical methods, our results highlight a third source of bias, transmission coupling, that requires additional tools to fully address. Accounting for this bias first requires data on the spatial distribution of the intervention and on human movement. Data on human movement could be obtained from self-reported travel histories, as was done in the per-protocol analysis of the AWED trial, or from mobile phone data,²¹ for example. However, accounting for transmission coupling also requires interfacing these data with a dynamical transmission model to account for the fact that, in the presence of movement between arms, incidence in each arm depends on prevalence in both arms.²² Many common trial designs will lead to reduced bias due to transmission coupling—for instance, by allocating a greater proportion of the trial area to the control arm, with small intervention clusters situated among larger control clusters so that transmission suppression in the intervention arm has less of a population-level effect. The ratio of area allotted to treatment and control would depend on many factors, including the expected strength of the intervention, the local force of infection, and logistical constraints such as the size and length of the trial. Using a dynamical model synthesising these factors in the design of a trial could aid in understanding how different designs might affect bias due to transmission coupling.²² More work is needed to understand what types of spatial clustering patterns, among other features of trial design, would minimise this form of bias.

Limitations and caveats

Although our modelling approach allowed us to account for different potential sources of bias and to attribute the total bias to each of those sources, it has several caveats. First, our model was deterministic, yet stochasticity could be important for a highly efficacious intervention with potential to reduce transmission to very low levels.²³ This simplification implies that our estimates are likely conservative, as these effects could increase the bias due to transmission coupling if a highly effective intervention increases the probability of transmission fadeouts. Second, our simple model does not reflect all of the complexities of DENV transmission. For example, we did not account for spatial heterogeneities in transmission or interactions between serotypes. Accurately quantifying the contribution of these effects to bias would require a more detailed model, but the qualitative results would likely be similar. Third, we did not calibrate our model to trial data, so incidence in our model may not reflect the actual incidence during the trial. However, our aim here was not to precisely quantify bias in the AWED trial, but rather to highlight some potential sources of bias in trials of that nature and to understand how these biases are influenced by transmission dynamics and human mobility. Moreover, our model was calibrated to actual incidence from past years in Yogyakarta, and so still reflects transmission typical of that location. It is also worth noting that an earlier version of our analysis, which used a simpler static model based on epidemic attack rate formulae, had qualitatively similar findings.²⁴ Finally, we do not account for heterogeneity between clusters, such as regions of the city with systematically higher mosquito abundance, or greater human movement, or within clusters, such as that transmission may be higher at the edge of control clusters.

Conclusions

Without accounting for human movement, mosquito movement and transmission coupling, the efficacy of Wolbachia-infected mosquitoes as an intervention to control dengue is likely to be underestimated. As the estimate of efficacy in the AWED trial was already very high (77.1% (95% CI 65.3% to 84.9%))⁷ and, as we show, likely underestimated, Wolbachia-infected mosquitoes have potential to be a game-changing tool in the fight against dengue. Even as vaccines against dengue become available, a variety of vector control approaches are likely to remain key tools in the fight against dengue.^{2 14} Although we focused our analysis on a trial of Wolbachia-infected mosquitoes, our findings are applicable to any efficacy trial of an intervention that has the potential to contaminate the control arm, such as gene drive mosquitoes or ivermectin as interventions against malaria.^{25 26} As trials of these interventions continue, it will be important to learn what lessons we can from transmission dynamic modelling when designing and interpreting future trials to ensure that we understand the true promise of these interventions.

Data availability statement

Data are available in a public, open access repository. All code and other files to reproduce our results is available at: https://github.com/scavany/awed_trial_modeling/

Ethics statements

Patient consent for publication

Not applicable.