Abstract
Malaria transmission-blocking vaccines (TBVs) are potentially helpful tools for malaria eradication. The standard membrane-feeding assay (SMFA) is considered one of the “gold standard” assays for TBV development. However, lack of consensus in reporting results from SMFA has made it very challenging to compare results from different studies. Two main readouts, % inhibition in mean oocyst count per mosquito (TRA) and % inhibition in prevalence of infected mosquitoes (TBA), have been used widely.
In this study, we statistically modeled the oocyst data in SMFA using data from 105 independent feeding experiments including 9,804 mosquitoes. The model was validated using an independent data set that included 10,790 mosquitoes from 110 feeding studies. The model delineates a relationship between TRA, the mean oocyst count in the control mosquitoes (mo-contl), and TBA. While TRA was independent from mo-contl, TBA values changed depending on mo-contl. Regardless of monoclonal or polyclonal antibodies tested, there were strong concordances between observed TBA and predicted TBA based on the model using mo-contl and observed TRA. Simulations showed that SMFA with lower true control means had increased uncertainty in TRA estimates. The strong linkage between TBA, TRA and mo-contl inspired creation of a standardized TBA, a model-based TBA standardized to a target control mean, which allows comparison across multiple feeds regardless of mo-contl.
This is the first study showing that the observed TBA can be reasonably predicted by mo-contl and the TRA of the test antibody using independent experimental data. This study indicates that TRA should be used to compare results from multiple feeds with different levels of mo-contl. If a measure of TBA is desired, it is better to report standardized TBA rather than observed TBA. These recommendations support rational comparisons of results from different studies, thus benefiting future TBV development.
Keywords: Malaria, transmission-blocking vaccine, standard membrane-feeding assay
1. Introduction
Due to the expanded application of anti-malarial control measures, such as insecticide-treated nets, rapid diagnosis, and antimalarial drugs, the mortality of malaria has been reduced significantly in the last 15 years. However, it is estimated that 438,000 malaria related deaths, mostly due to Plasmodium falciparum, occurred in 2015 [1]. Multiple novel tools are likely to be required to achieve the ultimate goal of malaria eradication, and a transmission-blocking vaccine (TBV) is considered to be one of them [2-4]. TBVs are designed to induce antibodies in human hosts against sexual stage malaria antigens or to antigens found in the mosquito vector, and these antibodies should inhibit parasite development in the mosquito when they are ingested with gametocyte-stage parasites.
Several TBV candidates have reached the preclinical development stage and a few phase 1 human trials have been conducted [4,5]. To accelerate vaccine development, establishment of a robust and functional assay(s) to evaluate TBV candidates is essential [2]. There are several biological assays to determine the functionality of TBV-induced antibodies [6], and the standard membrane-feeding assay (SMFA) is considered one of the “gold standard” assays. In this assay, a mixture of cultured P. falciparum gametocytes and test antibodies are fed to Anopheles mosquitoes through a membrane-feeding apparatus, and approximately one week later the mosquitoes are dissected to enumerate oocysts in the midgut. Not only for TBV development, SMFA has also become popular for the development of drugs targeting sexual stage parasites [7-9]. However, a fundamental question relevant to this assay has not been resolved, viz., there is no consensus whether to use % inhibition in oocyst intensity (also referred to as “transmission reducing activity” or “TRA”), % inhibition in prevalence of infected mosquitoes (also called “transmission-blocking activity” or “TBA”), or both as the main readout(s) of the SMFA. The TBA readout is thought to be the best predictor of vaccine efficacy under field conditions, as a single oocyst can still generate a large number of infectious sporozoites [10]. However, one of the major differences between SMFA and natural infection is the number of oocysts per mosquito. In direct feed assays (DFA), where mosquitoes feed directly on a malaria patient’s skin [11-13], or in a study where mosquitoes were caught in the field [14], most of the mosquitoes had less than 5-6 oocysts. On the other hand, in many SMFA assays, observed mean oocyst intensities in the control groups (mo-contl) are much higher [10,15-17]. There is no systematic approach to judge whether TBA is still a better readout than TRA when mean oocyst intensity in the control (either mo-contl or true mean oocysts in the control, mt-contl) is equal to 20, 50 or 100. Inconsistency in reporting the SMFA results has made it very challenging to compare data from different studies, and has hampered application of this assay for vaccine and drug development. In addition, information on oocyst intensity and prevalence of infected mosquitoes in the control group, by which % inhibition of a test sample is calculated, are generally ignored when researchers compare the results from different assays or studies.
In this study, we first statistically modeled the SMFA using data (model-building data) from 105 membrane-feeding assays involving 9,804 mosquitoes, and then validated the model using an independent data set (validation data) included 10,790 mosquitoes from 110 feeding experiments. We utilized the SMFA model and the validation data to evaluate: 1) the linkage between TRA and TBA, and 2) the impact of control mean oocyst intensity (either mo-contl or mt-contl) on the error in TRA and TBA estimates.
2. Methods
2.1. Test materials
Feeding experiments were conducted with multiple monoclonal antibodies (mAb), protein G purified mouse polyclonal antibodies, and protein G purified IgGs from normal mouse, rabbit, monkey and human sera. The mAbs included 4B7 (anti-Pfs25) [18], 3E12 (anti-Pfs48/45) [19], IIC5B10 (anti-Pfs48/45) [19], and 1B3 (anti-Pfs230) [20] mAbs. The details of mouse polyclonal antibodies have been reported elsewhere [21,22], and the target antigens of those antibodies included Pfs25, Pfs48/45, Pfs230, PfHAP2 and Anopheles gambiae aminopeptidase N (AgAPN1). Multiple normal mouse and rabbit sera were purchased from Sigma-Aldrich (St. Louis, MO, USA), SouthernBiotech (Birmingham, AL, USA) and Invitrogen (Waltham, MA, USA). The monkey sera were obtained from Alpha Diagnostic International (San Antonio, TX, USA), and human serum from Interstate Blood Bank (Memphis, TN, USA).
2.2. SMFA
The standardized methodology for performing the SMFA has been described previously [16]. Briefly, 16-18 day old gametocyte cultures of the P. falciparum NF54 line (200 μl of 50% haematocrit culture adjusted to 0.15-0.2% stage V gametocytaemia) were mixed with 60 μl of a test sample, and the final mixture was immediately fed to ~50 female Anopheles stephensi (Nijmegen strain, three to six days old) mosquitoes through a membrane-feeding apparatus. Mosquitoes were kept for 8 days and dissected (n=~20 per “Container of Mosquitoes” (COM) for most of the cases) to enumerate the oocysts in the midgut. Throughout the paper, COM refers to a group of mosquitoes which were housed in the same container and were fed the same final mixture of gametocyte cultures and control/test antibodies. Only midguts from mosquitoes with any eggs at the time of dissection were analyzed (60-80% of mosquitoes were egg positive in general). The human serum and red blood cells used for the gametocyte cultures and feeding experiments were purchased from Interstate Blood Bank.
2.3. Statistical analysis
Percent (%) inhibition of mean oocyst intensity (TRA) was calculated as: 100 × {1 − (mean number of oocysts in the test group)/(mean number of oocysts in the control groups)}. Similarly, the (unstandardized) % inhibition of oocyst prevalence (TBA) was evaluated as: 100 × {1− (proportion of mosquitoes with any oocysts in the test group)/(proportion of mosquitoes with any oocysts in the control group).
Details of modeling, standardization, and statistical analysis are described in the accompanying manuscript [23]. The model shows that the TBA estimand depends on mt-contl. Briefly, the oocyst data were modeled using a zero-inflated negative binomial random effects model (ZINB model) which was similar to the method described previously [16]. In this study 9,804 mosquito data from 105 feeding experiments with 492 COMs (model-building data) were utilized to determine the best estimate of parameters in the ZINB model. The model-building data consisted of SMFA performed with normal IgGs in various species. For model validation, an independent data set including 10,790 mosquitoes from 110 feeding experiments with 541 COMs was utilized (validation data). Details of determining the 95% prediction region shown in Fig. 2 and the prediction of required number of mosquitoes in Fig. 4 are shown in the supplemental material for this paper. TBA estimates and confidence intervals shown in Fig. 5 used transformations and t-test confidence intervals [23].
Fig. 2. Validation of the model.
Independent SMFA data including 10,790 mosquitoes from 110 feeding experiments with 541 COMs were used to validate the model. The mean oocysts (x-axis in a log-scale) and prevalence (y-axis in a linear-scale) from each COM of the validation data were calculated. The data are presented with jittering and alpha blending to show overlapping points. The best-fit line (red) and the 95% prediction region (PR, blue) was estimated from the ZINB model using the model-building data (the red and blue lines are not the best-fit and the 95% PR of the validation data). The 95% PR of the model was calculated assuming 20 mosquitoes were dissected per COM (see supplement for details).
Fig. 4. Number of mosquitoes to achieve the same level of error in TRA estimates.
At each level of % inhibition (TRA), the number of mosquitoes that are required to achieve the same level of mean squared error in % inhibition measurement is estimated by simulation. Different colors and symbols represent different levels of TRA (0% to 95% inhibition). The reference condition is a feed where 20 mosquitoes are analyzed in each COM (20 each for control and test) and mt-contl =10.
Fig. 5. Standardized TBA.
From the 4B7 mAb data shown in Fig. 1, 17 data points were randomly selected in each specified mo-contl range: mo-contl were less than 6 for “low”; mo-contl were between 6 and 30 for “med”; and mo-contl more than 30 for “high”. Based on the observed TRA, a standardized TBA was estimated assuming mt-contl = 2 (regardless of mo-contl). Unstandardized TBA (A; i.e., observed TBA) and standardized TBA (B) are shown. Both individual data, the best estimate of TBA and the associated 95% confidence intervals of each group are shown. A similar figure using the full set of 4B7 mAb data, not with randomly selected data (n=17 per range in this figure), is shown in the accompanying manuscript [23] with 95% confidence intervals for individual points.
The correlation between mo-contl and TRA (or TBA) for 4B7 mAb (tested at 94 μg/ml) was determined by a Spearman Rank test. Random marginal agreement coefficients (RMACs) were utilized to determine the concordance between observed TBA and model-based TBA [24]. All statistical tests were performed in R (version 3.2.2) or Prism 6 (GraphPad Software), and p-values <0.05 were considered significant.
3. Results
3.1. Impact of mean oocysts in the control
To determine the impact of mo-contl on % inhibition readouts, 4B7 monoclonal antibody (mAb) was tested at a fixed concentration of 94 μg/ml in 104 independent assays (Fig. 1). The median of the mo-contl was 14.1 oocysts per mosquito (interquartile range of 7.3 to 22.8). While there was no significant impact of mo-contl on % inhibition of mean oocyst intensity (TRA; p=0.6226 by a Spearman rank test; Fig. 1A), the % inhibition of oocyst prevalence (TBA) decreased when mo-contl increased (Spearman coefficient = −0.563, p<0.0001, Fig. 1B). The data strongly suggest that it is difficult to evaluate the TBA readout without considering mo-contl.
Fig. 1. SMFA with fixed concentration of 4B7 mAb.
Total of 104 independent feeding experiments (124 COMs) were performed with 94 μg/ml of 4B7 mAb. The results of % inhibition of mean oocyst intensity (TRA, A) and % inhibition of oocyst prevalence (TBA, B) are shown. One data point with mean oocyst in control (mo-contl) = 0.13 and TRA = TBA =100% is not shown in the figures.
3.2. Model validation and a correlation between model-based TBA estimates and observed TBA
We recalculated the best-fit parameters of a zero-inflated negative binomial random effects model (ZINB model), which was similar to the method described previously [16], using data from 105 independent feeding experiments with 492 COMs including 9,804 mosquitoes (model-building data). The model-building data rendered the estimates of five parameters in the ZINB model; the estimate of overall control mean oocyst (log) is 2.57, the zero inflation parameter is 0.056, the standard deviation of the random effects for COM and feed are 0.2306 and 1.042, respectively, and the negative binomial dispersion parameter is 1.93. No species effect of antibodies was observed in the model fit (data not shown). For the validation of the model, an independent data set (validation data) included 10,790 mosquitoes from 110 feeding experiments and 541 COMs were utilized. As previously reported by us [16] and other groups [25,26] (regardless of control feeds or feeds with test samples), there was a strong correlation between the observed mean oocyst intensity and observed prevalence in the validation data (Fig. 2). Fig. 2 overlays the model of the true intensities and prevalence (red line) and the 95% prediction region (PR, blue lines) of the ZINB model (made by model-building data). There is a small systematic lack-of-fit in the model (more points are below and to the right of the red line than expected), but 93.5% of the validation data points were within the 95% PR boundaries of the model.
Using the ZINB model, a model-based TBA can be expressed with only 3 parameters (for details see [23] where the model-based TBA is referred to as the TBA estimand). This model-based TBA for the ith sample is estimated by
where (=1.93) is the estimated negative binomial dispersion parameter from the model, is the ratio of mean oocyst count in the test COM over the mean oocyst count in the control COM (so that the observed TRA for the ith sample is , and is the mo-contl for the ith control. We then assessed how well the observed TBA matched the model-based TBA. Using the validation data, a model-based TBA was estimated for each pair of COMs (i.e., a control and a test COM which were fed in the same experiment). Regardless of the data set analyzed, there were very strong concordances between observed TBA and model-based TBA (Fig. 3): concordance correlation (ρRMAC) = 0.92 (95% confidence interval, 95%CI, 0.89 - 0.95) for 4B7 mAb; ρRMAC = 0.90 (95%CI, 0.79 - 0.95) for other mAbs (3E12, IIC5B10, and 1B3); and ρRMAC = 0.97 (95%CI, 0.94 – 0.98) for polyclonal antibodies. The strong concordances demonstrate TRA and TBA are not independent readouts, and the observed TBA can be reasonably estimated by the model using mo-contl and the observed TRA.
Fig. 3. Concordance between observed TBA and model-based TBA.
Independent SMFA data sets (validation data) were utilized to determine whether the model could estimate TBA based on the mo-contl and TRA of test samples. Each point represents the observed TBA (x-axis) and Model-based TBA (y-axis) with each point calibrated to the target mt-contl that is equal to the mo-contl of a single test sample. Data from the 4B7 mAb which targets the Pfs25 antigen (A), other mAbs targeting Pfs48/45 or Pfs230 (B) and mouse polyclonal antibodies targeting Pfs25, Pfs48/45, Pfs230, PfHAP2 or AgAPN1 (C) are shown.
3.3. Impact of fewer mean oocysts in control on TRA estimates
The data presented strongly indicate that TBA values are difficult to compare unless all samples are tested in a single feed, or multiple feeds but with similar mt-contl. Hence, if one wants to use the TBA readout directly to predict efficacy in a field situation, the SMFA should be performed with similar mt-contl to that seen in the field, i.e., mt-contl should be very low so that most of the mosquitoes have less than 5-6 oocysts. Therefore, we next estimated the impact of mt-contl on the error in TRA estimates using the model (Fig. 4). Our simulations show that when mt-contl is lower, many more mosquitoes are required to attain the same level of mean squared error in the TRA estimate, compared to the reference condition where 20 mosquitoes were analyzed in each COM (control and test) and mt-contl =10. For example, when a sample with (true) TRA = 80 % is tested in a feed of mt-contl =1, then dissection of 109 mosquitoes per COM are required to achieve the same level of mean squared error which could be obtained from 20 mosquitoes per COM with mt-contl =10. The results indicate that SMFA assays targeting fewer mo-contl increase the error in % inhibition estimates unless many more mosquitoes are dissected.
3.4. Standardized TBA
At present it is very challenging to achieve the mo-contl within a certain low restricted range (e.g., the mo-contl is always within 1-4, but not zero) and such an SMFA will likely increase the error in % inhibition estimates (Fig. 4). Therefore, we next assessed the potential of a standardized TBA using the 4B7 mAb data shown in Fig. 1. The standardized TBA values were calculated based on the observed TRA and a fixed target mt-contl of 2, regardless of mo-contl, using the model. In other words, the standardized TBA for the ith sample just replaces the in the model-based TBA with 2. The 4B7 data were categorized based on the mo-contl (less than 6 for “low”; between 6 and 30 for “med”; and more than 30 for “high”) and 17 data points were randomly selected for each group. As expected, unstandardized TBA values, i.e., observed TBA, were affected by mo-contl (Fig. 5A). When the three groups were compared by one way ANOVA tests, there was a significant difference in unstandardized TBA between groups (p<0.001). However, there was no significant effect (p=0.19) of mo-contl on standardized TBA (Fig. 5B). The results suggest that the standardized TBA should be used when SMFA data from different feeding experiments are compared.
4. Discussion
SMFA is considered as one of the “gold standard” assays, and it has been widely used for TBV development. However, different readouts, either TRA or TBA or both, have been selected to report the results, complicating comparison of results from different studies. Even within a study, if mo-contl affects TBA and/or TRA readouts, we cannot compare data from different feeding experiments with different mo-contl. To the best of our knowledge, this is the first study showing observed TBA can be reasonably predicted by mean oocyst intensity in the control (mo-contl) and the TRA of the test antibody (Fig. 3) using independent experimental data. Our 4B7 mAb data from 104 assays (Fig. 1) and the strong concordance between model-based TBA (calibrated to have the target mt-contl equal to the mo-contl) and observed TBA (Fig. 3) convincingly indicate that TBA data cannot be reasonably interpreted without mo-contl; i.e., there is not one true TBA parameter, because a true TBA parameter is determined not only by the biological activity of a test antibody but also by the mt-contl. In addition, our simulations strongly suggest that SMFA with fewer mo-contl has a risk of increasing uncertainty in estimating % inhibition unless many mosquitoes are examined (Fig. 4, the lines in the figure were calculated using mt-contl, not mo-contl). Instead, our data suggest that it is better to use a standardized TBA using TRA data which are obtained from SMFA with larger mo-contl. The standardized TBA can be utilized to compare results from different assays/studies by using any fixed target mt-contl. Once the mt-contl in a field site of interest is estimated, we can calculate standardized TBA using that estimate of mt-contl from the field as the fixed target mt-contl in the model.
Many studies from different investigators, including us, have shown that there is a strong correlation between mean oocyst intensity (either arithmetic mean or geometric mean) and prevalence of infected mosquitoes in SMFA both with P. falciparum and P. berghei parasites in different species of Anopheles mosquitoes (Fig. 2) [17,25,26]. We have also utilized a negative binomial model with zero inflation and random effects for COMs and feeding experiment for SMFA data, similar to models used by other investigators [25,26]. In addition, we have shown that the same model (refitting the means but using the same overdispersion and zero inflation parameter estimates) also fits with the data from the direct membrane-feeding assay (DMFA, where gametocyte parasites from infected patients are used instead of in vitro cultured gametocytes) with both P. falciparum and P. vivax (KM and Sattabongkot, personal communication, December 2015). Furthermore, Medley et al., have reported a strong and a similar correlation between mean oocyst intensity and prevalence of infected mosquitoes in naturally infected A. gambiae and A. funestus mosquitoes which were collected in houses with malaria patients [25]. Similarly, Billingsley et al., have also shown that a negative binomial model fits well to the observed correlation between mean oocyst intensity and prevalence in naturally infected mosquitoes if a large number of mosquitoes were dissected [14]. Therefore, we believe our ZINB model is a reasonable model to explain the oocyst data in SMFA, and it is likely to be suitable for data in DMFA and naturally infected mosquitoes. In this study, we first verified the model (Fig. 2 and 3) using independent data sets, which were not used to generate the model, then performed subsequent analysis. A more detailed mathematical discussion of model development is described in a separate manuscript [23].
It is reasonable to predict that it is more difficult to bring down the mean number of oocysts from 100 (control) to 0 (test), than from 4 to 0. A preceding study with anti-Pbs28 mAb using P. berghei parasites has shown that TBA level decreased as mo-contl increased, while there was no correlation between TRA and mo-contl [27]. Our 4B7 mAb data tested with P. falciparum parasites also reached the same conclusion (Fig. 1). The data from these two studies clearly show that TBA cannot be directly compared unless mo-contl of feeds are the same or similar. However, if the linkage between the two readouts is weak, i.e., the two readouts are the indicators of different aspects of transmission-blocking activity, we need to perform SMFA with the same level of mo-contl and always report both TRA and TBA. Therefore, we next evaluated whether they are independent readouts.
Overall, a strong concordance was observed between model-based TBA and observed TBA (Fig 3), regardless of target antigens. The result confirmed that TBA and TRA readouts are not independent, and the observed TBA is a function of mo-contl and TRA. The lack of perfect concordance in Fig 3 is due to errors in both observed (x-axis) and model-based (y-axis) TBA. The observed TBA values have errors because of biological variability in the observed prevalence in control and test COMs. In other words, it is practically difficult to obtain the same observed TBA values when the same samples are tested in multiple feeds. The model-based TBA also has possible errors due to lack-of-fit of the model, as well as errors from the inputs into the model. For the former source of error, Fig 2 shows a small systematic lack-of-fit between observed data (validation data set) and model-based estimates (red and blue lines). However, in order to get a better fit in Fig 2, extra parameters are required (changing the 5 parameter values in the current ZINB model is not enough), and such parameters make the model more complicated. Further, our interest is not in the prevalence, but in a ratio of prevalences; i.e., TBA is 1 minus the ratio of prevalences (test sample over control sample). Therefore, the lack-of-fit of both numerator and denominator in the ratio may mostly cancel each other out. For these reasons we do not further complicate the model to achieve better fit. In terms of the inputs into the model, each of the observed TRA, the overdispersion parameter, and the mo-contl could have errors from their true values. It is difficult to identify which error(s) dominate the discrepancy between observed TBA and model-based TBA for each data point. However, there were strong concordances overall between the two TBA.
Some investigators have been trying to control the mo-contl levels to directly compare observed TBA values, however, as far as we are aware, no laboratory can tightly control the level of mo-contl for every single feeding experiment. Therefore, currently the only practical method to perform a “field-like” SMFA (i.e., mt-contl of SMFA should be very low as seen in the field) is to discard SMFA data when mo-contl is outside a pre-defined range. This is likely to reduce the throughput of SMFA and slow the development of much needed transmission-blocking vaccines. Further, simulations under the ZINB model show that the model-based TBA (standardized to a target mt-control) has lower mean squared error than the observed TBA (restricted to mo-control values close to a target mt-control) [23]. There may be considerable variability associated with the any TBA measure (model-based/standardized TBA or observed/unstandardized TBA). Thus, it is useful to express the variability for individual standardized TBA values using confidence intervals (for details see [23]). The standardized TBA also has the usual uncertainties from using an imperfect model; however, to make no adjustment at all is to essentially choose a much worse model, i.e., the TBA does not depend on mo-contl. We have shown that the latter “model” is grossly inadequate. Sauerwein et al. specifically discussed the difference between SMFA and natural infection, i.e., mo-contl in SMFA is usually much higher, in their recent review [28], and proposed performing SMFA with multiple levels of mo-contl to estimate TBV efficacy in a field situation. However, the present study has shown that TRA and TBA are not independent measurements, and TBA can be reasonably estimated from TRA at a targeted mt-control. Therefore, the results suggest that performing SMFA with multiple levels of mo-contl is not necessary to estimate TBA in a low mo-contl setting, although testing in multiple feeds is always valuable, in terms of reducing errors in TRA estimates and being more model-free.
Another potential problem of a “field-like” SMFA is the larger error in % inhibition estimates. Since observed TBA depends on mo-contl, it makes sense that the true TBA value from our model depends on the mt-contl (see [23]). In other words, the true TBA value changes when mt-contl changes. Therefore, the effect of mt-contl on error in TRA estimates at each level of true TRA, not true TBA, was examined (Fig. 4). Depending on the usage of data, the tolerable levels of error in the estimates could vary. For example, when the data are used for Go and No-Go decisions in a clinical trial, one might need to modify the assays to reduce the errors, such as dissecting more than 20 mosquitoes per COM, increasing the number of COMs per feed, or testing in multiple feeds. However, the point is that our simulations clearly show that such “field-like” SMFA requires many more mosquitoes to achieve the same level of confidence which we can obtain with 20 mosquitoes per COM when mo-contl =10. Churcher et al. simulated the minimum number of mosquitoes to be dissected to ensure that reported efficacy is within 10% of the true TRA efficacy [26] using their model. Their analysis also showed that many more mosquitoes were required when mo-contl was 1 or 2 compared to an SMFA when mo-contl was 10 or more. The two studies indicate that TRA values from a “field-like” SMFA likely have larger errors in the estimates unless many mosquitoes are dissected. Because TBA is a function of TRA and mo-contl as shown in Fig. 3, if TRA estimates have larger errors, similarly TBA estimates have larger errors as well. There have been several attempts [29,30] to establish a higher throughput SMFA by substituting the dissection and/or oocyst counting steps at endpoint, which are labor-intensive and time-consuming procedures. These methods may be especially useful when using much larger numbers of mosquitoes than are typically used in the SMFA or when there may be limited resources available to dedicate to the labor-intensive endpoint. Regardless of how the SMFA endpoint is analyzed, the number of samples that can be tested will still be limited by the number of mosquitoes that can be produced in the insectary, potentially the volume of the gametocyte culture, and also the amount of test material available.
Taken together, to compare results from different studies (or assays) more appropriately, we recommend that: 1) TRA should be used as it allows comparison across multiple feeds over time, regardless of mo-contl, 2) if TRA is independent from mo-contl as it appears for all samples tested in this study, investigators should target a “higher mt-contl” SMFA rather than a “field-like” (lower mt-contl) SMFA to achieve smaller errors in estimates of true TRA for a given number of mosquitoes in a COM, 3) in a given feeding experiment, if mo-contl is low, then examine more mosquitoes to increase the confidence in the TRA estimates, and 4) if a measure of TBA is desired, it is better to report standardized TBA (model-based TBA based on the observed TRA and a fixed mt-contl, regardless of the mo-contl) rather than observed TBA and mo-contl. We propose these recommendations as a means to support rational comparisons of results from different studies, thus benefiting future TBV development.
Supplementary Material
Acknowledgements
We thank Jetsumon Sattabongkot for the direct membrane-feeding data. This work utilized the computational resources of the NIH HPC Biowulf cluster (http://hpc.nih.gov). This study was supported in part by the Intramural Research Program of the National Institute of Allergy and Infectious Diseases, NIH and also by the PATH Malaria Vaccine Initiative.
List of abbreviations
- TBV
transmission-blocking vaccine
- SMFA
standard membrane-feeding assay
- TRA
transmission reducing activity, % inhibition in oocyst intensity
- TBA
transmission-blocking activity, % inhibition in prevalence of infected mosquitoes
- DFA
direct feed assay
- mo-contl
observed mean oocyst intensity in the control group
- mt-contl
true mean oocyst intensity in the control group
- ZINB model
a zero-inflated negative binomial random effects model
- COM
container of mosquitoes
- PR
prediction region
- DMFA
direct membrane-feeding assay
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Competing Interests:
The authors declare that they have no competing interests.
Author contributions
KM, Conception and design, Acquisition of data, Analysis and interpretation of data, Drafting or revising the article; BJS, Conception and design, Analysis and interpretation of data, Drafting or revising the article; BD, LZ, TPP, AD, TB, Acquisition of data, Interpretation of data; MPF, CAL, Conception and design, Analysis and interpretation of data, Drafting or revising the article. All authors read and approved the final manuscript.
References
- [1]. [accessed 24 February 2016];World Malaria Report. 2015 http://www.who.int/malaria/publications/world-malaria-report-2015/report/en/
- [2].malERA Consultative Group on Vaccines A research agenda for malaria eradication: Vaccines. PLoS Med. 2011;8:e1000398. doi: 10.1371/journal.pmed.1000398. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [3].Nunes JK, Woods C, Carter T, Raphael T, Morin MJ, Diallo D, et al. Development of a transmission-blocking malaria vaccine: progress, challenges, and the path forward. Vaccine. 2014;32:5531–5539. doi: 10.1016/j.vaccine.2014.07.030. [DOI] [PubMed] [Google Scholar]
- [4].Wu Y, Sinden RE, Churcher TS, Tsuboi T, Yusibov V. Development of malaria transmission-blocking vaccines: from concept to product. Adv Parasitol. 2015;89:109–152. doi: 10.1016/bs.apar.2015.04.001. [DOI] [PubMed] [Google Scholar]
- [5].Nikolaeva D, Draper SJ, Biswas S. Toward the development of effective transmission-blocking vaccines for malaria. Expert Rev Vaccines. 2015:1–28. doi: 10.1586/14760584.2015.993383. [DOI] [PubMed] [Google Scholar]
- [6].Sinden RE, Blagborough AM, Churcher T, Ramakrishnan C, Biswas S, Delves MJ. The design and interpretation of laboratory assays measuring mosquito transmission of Plasmodium. Trends Parasitol. 2012;28:457–465. doi: 10.1016/j.pt.2012.07.005. [DOI] [PubMed] [Google Scholar]
- [7].Ruecker A, Mathias DK, Straschil U, Churcher TS, Dinglasan RR, Leroy D, et al. A male and female gametocyte functional viability assay to identify biologically relevant malaria transmission-blocking drugs. Antimicrob Agents Chemother. 2014;58:7292–7302. doi: 10.1128/AAC.03666-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [8].Sanders NG, Sullivan DJ, Mlambo G, Dimopoulos G, Tripathi AK. Gametocytocidal screen identifies novel chemical classes with Plasmodium falciparum transmission blocking activity. PloS one. 2014;9:e105817. doi: 10.1371/journal.pone.0105817. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [9].Baragana B, Hallyburton I, Lee MC, Norcross NR, Grimaldi R, Otto TD, et al. A novel multiple-stage antimalarial agent that inhibits protein synthesis. Nature. 2015;522:315–320. doi: 10.1038/nature14451. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [10].Stone WJ, Eldering M, van Gemert GJ, Lanke KH, Grignard L, van de Vegte-Bolmer MG, et al. The relevance and applicability of oocyst prevalence as a read-out for mosquito feeding assays. Sci Rep. 2013;3:3418. doi: 10.1038/srep03418. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [11].Githeko AK, Brandling-Bennett AD, Beier M, Atieli F, Owaga M, Collins FH. The reservoir of Plasmodium falciparum malaria in a holoendemic area of western Kenya. Trans R Soc Trop Med Hyg. 1992;86:355–358. doi: 10.1016/0035-9203(92)90216-y. [DOI] [PubMed] [Google Scholar]
- [12].Toure YT, Doumbo O, Toure A, Bagayoko M, Diallo M, Dolo A, et al. Gametocyte infectivity by direct mosquito feeds in an area of seasonal malaria transmission: implications for Bancoumana, Mali as a transmission-blocking vaccine site. Am J Trop Med Hyg. 1998;59:481–486. doi: 10.4269/ajtmh.1998.59.481. [DOI] [PubMed] [Google Scholar]
- [13].Gouagna LC, Yao F, Yameogo B, Dabire RK, Ouedraogo JB. Comparison of field-based xenodiagnosis and direct membrane feeding assays for evaluating host infectiousness to malaria vector Anopheles gambiae. Acta Trop. 2013;130C:131–139. doi: 10.1016/j.actatropica.2013.10.022. [DOI] [PubMed] [Google Scholar]
- [14].Billingsley PF, Medley GF, Charlwood D, Sinden RE. Relationship between prevalence and intensity of Plasmodium falciparum infection in natural populations of Anopheles mosquitoes. Am J Trop Med Hyg. 1994;51:260–270. doi: 10.4269/ajtmh.1994.51.260. [DOI] [PubMed] [Google Scholar]
- [15].van der Kolk M, De Vlas SJ, Saul A, van de Vegte-Bolmer M, Eling WM, Sauerwein RW. Evaluation of the standard membrane feeding assay (SMFA) for the determination of malaria transmission-reducing activity using empirical data. Parasitology. 2005;130:13–22. doi: 10.1017/s0031182004006067. [DOI] [PubMed] [Google Scholar]
- [16].Miura K, Deng B, Tullo G, Diouf A, Moretz SE, Locke E, et al. Qualification of standard membrane-feeding assay with Plasmodium falciparum malaria and potential improvements for future assays. PloS one. 2013;8:e57909. doi: 10.1371/journal.pone.0057909. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [17].Li T, Eappen AG, Richman AM, Billingsley PF, Abebe Y, Li M, et al. Robust, reproducible, industrialized, standard membrane feeding assay for assessing the transmission blocking activity of vaccines and drugs against Plasmodium falciparum. Malaria J. 2015;14:150. doi: 10.1186/s12936-015-0665-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [18].Barr PJ, Green KM, Gibson HL, Bathurst IC, Quakyi IA, Kaslow DC. Recombinant Pfs25 protein of Plasmodium falciparum elicits malaria transmission-blocking immunity in experimental animals. J Exp Med. 1991;174:1203–1208. doi: 10.1084/jem.174.5.1203. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [19].Carter R, Graves PM, Keister DB, Quakyi IA. Properties of epitopes of Pfs48/45, a target of transmission blocking monoclonal antibodies, on gametes of different isolates of Plasmodium falciparum. Parasite Immunol. 1990;12:587–603. doi: 10.1111/j.1365-3024.1990.tb00990.x. [DOI] [PubMed] [Google Scholar]
- [20].Quakyi IA, Carter R, Rener J, Kumar N, Good MF, Miller LH. The 230-kDa gamete surface protein of Plasmodium falciparum is also a target for transmission-blocking antibodies. J Immunol. 1987;139:4213–4217. [PubMed] [Google Scholar]
- [21].Miura K, Takashima E, Deng B, Tullo G, Diouf A, Moretz SE, et al. Functional comparison of Plasmodium falciparum transmission-blocking vaccine candidates by the standard membrane-feeding assay. Infect Immun. 2013;81:4377–4382. doi: 10.1128/IAI.01056-13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [22].Kapulu MC, Da DF, Miura K, Li Y, Blagborough AM, Churcher TS, et al. Comparative assessment of transmission-blocking vaccine candidates against Plasmodium falciparum. Sci Rep. 2015;5:11193. doi: 10.1038/srep11193. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [23].Swihart BJ, Fay MP, Miura K. Statistical methods for standardized membrane-feeding assays to measure transmission blocking or reducing activity in malaria. Submitted for publication (accompanying manuscript) [DOI] [PMC free article] [PubMed] [Google Scholar]
- [24].Fay MP. Random marginal agreement coefficients: rethinking the adjustment for chance when measuring agreement. Biostatistics. 2005;6:171–180. doi: 10.1093/biostatistics/kxh027. [DOI] [PubMed] [Google Scholar]
- [25].Medley GF, Sinden RE, Fleck S, Billingsley PF, Tirawanchai N, Rodriguez MH. Heterogeneity in patterns of malarial oocyst infections in the mosquito vector. Parasitology. 1993;106:441–449. doi: 10.1017/s0031182000076721. [DOI] [PubMed] [Google Scholar]
- [26].Churcher TS, Blagborough AM, Delves M, Ramakrishnan C, Kapulu MC, Williams AR, et al. Measuring the blockade of malaria transmission - An analysis of the standard membrane feeding assay. Int J Parasitol. 2012;42:1037–1044. doi: 10.1016/j.ijpara.2012.09.002. [DOI] [PubMed] [Google Scholar]
- [27].Da DF, Churcher TS, Yerbanga RS, Yameogo B, Sangare I, Ouedraogo JB, et al. Experimental study of the relationship between Plasmodium gametocyte density and infection success in mosquitoes; implications for the evaluation of malaria transmission-reducing interventions. Exp Parasitol. 2014;149C:74–83. doi: 10.1016/j.exppara.2014.12.010. [DOI] [PubMed] [Google Scholar]
- [28].Sauerwein RW, Tousema T. Transmission blocking malaria vaccines: Assays and candidates in clinical development. Vaccine. 2015;33:7476–82. doi: 10.1016/j.vaccine.2015.08.073. [DOI] [PubMed] [Google Scholar]
- [29].Delves MJ, Sinden RE. A semi-automated method for counting fluorescent malaria oocysts increases the throughput of transmission blocking studies. Malar J. 2010;9:35. doi: 10.1186/1475-2875-9-35. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [30].Stone WJ, Churcher TS, Graumans W, van Gemert GJ, Vos MW, Lanke KH, et al. A scalable assessment of Plasmodium falciparum transmission in the standard membrane feeding assay using transgenic parasites expressing GFP-luciferase. J Infect Dis. 2014;210:1456–1463. doi: 10.1093/infdis/jiu271. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.