ABSTRACT
The bacterium Wolbachia pipientis is increasingly studied for its potential use in controlling insect vectors or pests due to its ability to induce Cytoplasmic Incompatibility (CI). CI can be exploited by establishing an opportunistic Wolbachia infection in a targeted insect species through trans-infection and then releasing the infected males into the environment as sterilizing agents. Several host life history traits (LHT) have been reported to be negatively affected by artificial Wolbachia infection. Wolbachia is often considered the causative agent of these detrimental effects, and the importance of the host’s genetic origins in the outcome of trans-infection is generally overlooked. In this study, we investigated the impact of host genetic background using an Aedes albopictus line recently trans-infected with wPip from the Culex pipiens mosquito, which exhibited some fitness costs. We measured several LHTs including fecundity, egg hatch rate, and male mating competitiveness in the incompatible line after four rounds of introgression aiming at restoring genetic diversity in the nuclear genome. Our results show that introgression with a wild genetic background restored most fitness traits and conferred mating competitiveness comparable to that of wild males. Finally, we show that introgression leads to faster and stronger population suppression under laboratory conditions. Overall, our data support that the host genome plays a decisive role in determining the fitness of Wolbachia-infected incompatible males.
IMPORTANCE
The bacterium Wolbachia pipientis is increasingly used to control insect vectors and pests through the Incompatible Insect Technique (IIT) inducing a form of conditional sterility when a Wolbachia-infected male mates with an uninfected or differently infected female. Wolbachia artificial trans-infection has been repeatedly reported to affect mosquitoes LHTs, which may in turn compromise the efficiency of IIT. Using a tiger mosquito (Aedes albopictus) line recently trans-infected with a Wolbachia strain from Culex pipiens and displaying reduced fitness, we show that restoring genetic diversity through introgression significantly mitigated the fitness costs associated with Wolbachia trans-infection. This was further demonstrated through experimental population suppression, showing that introgression is required to achieve mosquito population suppression under laboratory conditions. These findings are significant for the implementation of IIT programs, as an increase in female fecundity and male performance improves mass rearing productivity as well as the sterilizing capacity of released males.
KEYWORDS: Wolbachia, Aedes albopictus, introgression, male competitiveness, Incompatible Insect Technique
INTRODUCTION
The α-proteobacterium Wolbachia is among the most abundant symbionts, infecting around 50% of all arthropod species (1–4). This maternally transmitted bacterium induces a wide range of phenotypes in its hosts, including reproductive manipulations to favor its spread in natural populations (5). The most common manipulation induced by Wolbachia infection is a form of conditional sterility named Cytoplasmic Incompatibility (CI) (6, 7), which is an embryonic lethality resulting from mating between a Wolbachia-infected male and a female that is either uninfected or infected with a distinct incompatible Wolbachia strain (8–10). This natural phenomenon can be exploited in biological control through two distinct strategies. Wolbachia-infected mosquitoes can be used to replace the existing mosquito population with one less likely to transmit the pathogen, generating long-term reductions in transmission (a Wolbachia-based population replacement approach). Alternatively, CI can be exploited to suppress the existing mosquito population by releasing males only, a Wolbachia-based population suppression approach also called Incompatible Insect Technique (IIT) (11, 12), in which incompatible males are used as a sterilizing agent to control the population size of the target species (13, 14).
The main advantage of IIT, when compared for example to other techniques such as the sterile insect technique, is that males are “ready to use” and do not require specialized equipment for sterilization. However, IIT is highly sensitive to the accidental release of females, which could lead to the undesirable establishment of Wolbachia-transinfected mosquitoes. This can be prevented by combining IIT with irradiation, which will sterilize accidentally released females (15, 16), or by running IIT as a standalone (17–19) provided a highly effective sexing system based on genetic (20–23) or mechanical/automated processes (17, 24) is available. Whichever method is employed, the use of a transinfected line exhibiting bidirectional CI with the resident population will drastically reduce the probability of invasion (25).
While initially limited to species where natural Wolbachia infections allowed the expression of CI (14, 26–28), or to closely related species allowing horizontal Wolbachia transfer through introgression (29), IIT can be considered in a wider range of species through the artificial transfer of CI-inducing Wolbachia strains (30–33). Wolbachia trans-infected lines have been repeatedly reported to display reduced fitness including reduced fecundity, lower egg hatch rates, and/or survival (34–38). Such fitness costs have been generally attributed to the artificial Wolbachia infection. It has been proposed that these costs could be attenuated by artificial selection expected to stabilize the density of the symbiont together with its distribution in the tissues of the new host (39–42). Alternatively, it has also been proposed that the bottleneck in host diversity induced by the trans-infection process may be responsible for some of the measured deleterious effects, although this has been rarely tested to date (32, 43). In most cases, the selection of one (or few) isofemale lines showing higher maternal transmission during the first generations following trans-infection will result in a loss of genetic population diversity.
In this study, we addressed the importance of host genetic background on the performance of Wolbachia trans-infected lines by comparing different life history traits (LHTs) in an Aedes albopictus trans-infected line to the same line following a few rounds of introgression, or to the wild-type line used for this introgression. Finally, we compared the capacity of incompatible males to suppress a population in laboratory-controlled conditions. The data presented show that a small number of rounds of introgression are sufficient to suppress fitness costs associated with Wolbachia trans-infection.
RESULTS
CI penetrance of wPip-IV in an introgressed background
The mosquito lines used in the experiments include (i) a doubly wAlbA/B infected line, referred herein to as wild-wAlb, (ii) the wPip trans-infected line that was either introgressed with wild-wAlb and referred to as wild-wPip, or (iii) not introgressed and referred to as lab-wPip. Reciprocal crosses between wild-wAlb and wild-wPip lines confirmed that wPip-IV induced bidirectional CI in Ae. albopictus. Indeed, when crossed with wild-wAlb females, males from wild-wPip induced complete CI (0% egg hatching rate). The reciprocal crosses between males from the wild-wAlb line and females from wild-wPip displayed an average hatch rate of 0.46% (±0.46) (Table 1). The same pattern was observed when using the non-introgressed line (20), showing, as expected, that introgression did not modify CI.
TABLE 1.
Hatch rate obtained in control and reciprocal crosses between the transinfected wild-wPip and the wild-wAlb line
| Crosses | Hatch ratea (%) (±SD) | CI (%) | CIcorr (%) |
|---|---|---|---|
| ♂ wild-wPip × ♀ wild-wAlb | 0.00 (±0.00) [9,825] | 100 | 100 |
| ♂ wild-wAlb × ♀ wild-wPip | 0.46 (±0.46) [3,865] | 99.54 | 99.49 |
| ♂ wild-wAlb × ♀ wild-wAlb | 91.22 (±0.20) [3,447] | ||
| ♂ wild-wPip × ♀ wild-wPip | 89.86 (±0.04) [3,512] |
The number of eggs counted per cross is indicated in brackets.
Effect of a wild genetic background introgression on life history traits
We assessed the effect of introgression on the LHTs of the trans-infected line that was either introgressed or not introgressed with the wild-wAlb line. LHTs included longevity, number of laid eggs, hatching rate, and the insemination male capacity. For both sexes, the lab-wPip showed the lowest longevity with a significant difference compared to the two other wild-wPip and wild-wAlb lines (P < 0.001 for each comparison, Fig. 1A and B), for which a similar longevity was observed. We also observed a significant difference in the hatch rates between the lab-wPip, displaying an average hatch rate of 56.03% (±28.83) and both wild-wPip and wild-wAlb lines, for which similar hatch rates were observed, 78.88% (±23.58) and 82.77% (±18.04), respectively (P = 0.681) (Fig. 1C). The number of eggs laid per female was not significantly different between lines although higher for females from the wild-wAlb line that averaged 70.55 (±30.24) eggs per female, compared to 62.64 (±28.86) and 61.65 (±25.53) for females from the wild-wPip and lab-wPip lines, respectively (Fig. 1D). The insemination capacity of males was evaluated by crossing males and females from the same line and by crossing incompatible males from the wild-wPip or lab-wPip with wild-wAlb females. Of note, over 99,5% of dissected inseminated females (957/960) had two filled spermathecae, aligning with a pattern typically reported for Aedes albopictus (44, 45). As expected, decreasing the female/male ratio reduced the percentage of inseminated females for all three mosquito lines regardless of the combination of males and females (Fig. 2A and B). For all three tested ratios, males from wild-wPip and wild-wAlb lines presented a similar insemination capacity when crossed with females from their own line (Fig. 2A). Interestingly, we also observed a similar insemination capacity between these males when crossed with wild-wAlb females (Fig. 2B). Finally, the insemination capacity of lab-wPip males was lower than that of males from a wild genetic background (wild-wAlb and wild-wPip).
Fig 1.
Life history traits of trans-infected lab-wPip and wild-wPip lines compared to wild-wAlb line. (A) Male longevity. (B) Female longevity. (C) Eggs hatching rate (%). (D) Number of eggs laid per female. For panels A and B, the solid lines indicate the mean values and the dotted lines are the 95% CI. For panels C and D, black dots indicate mean values, horizontal bars the medians, and distinct letters significant variations between lines.
Fig 2.
Insemination capacity of incompatible males compared to the wild-wAlb males at different male ratios. (A) Percentage of inseminated females in compatible crosses (male and female from the same line). (B) Percentage of inseminated females in incompatible crosses (male from the lab-wPip and wild-wPip crossed with wild-wAlb female). Distinct letters indicate significant variations between the different crosses. The bar plots represent the means, and the vertical black bars are the min and max values.
Mating competitiveness of incompatible males
We compared the mating competitiveness of lab-wPip and wild-wPip lines by crossing virgin wild-wAlb females with different ratios of incompatible (from lab-wPip or wild-wPip lines) versus compatible (from wild-wAlb line) males. Observed hatch rates in 1:0 and 0:1 ratios (first and second numbers corresponding to the ratio of compatible and incompatible males, respectively) were compared with the expected hatch rates under the assumption of similar mating competitiveness between incompatible and compatible males. We showed that incompatible males from lab-wPip and wild-wPip lines induced a complete CI in crosses with wild-wAlb females (ratio 0:1 in which only incompatible males were present). Accordingly, for both trans-infected lines, we observed a hatch rate decrease with an increasing proportion of incompatible males (Fig. 3). For each ratio, the expected and observed hatch rates in crosses involving wild-wPip males were very close, indicating that these males are as competitive as wild-wAlb males. By contrast, lab-wPip males appeared significantly less competitive than wild-wAlb males in 1:1, 1:5, and 1:10 ratios (exact binomial test, P < 0.01). Altogether, these data suggest that introgression of the original lab-wPip line results in an increase in male mating competitiveness to levels that are not distinguishable from that of wild-wAlb males.
Fig 3.
Mating competitiveness of incompatible males from lab-wPip and wild-wPip lines. The blue line indicates the expected hatch rate under a similar competitiveness between compatible and incompatible males. The red and green lines indicate the observed hatch rate with the min and max values. ***P < 0.001.
Evaluation of incompatible males to suppress an Ae. albopictus population
We compared the capacity of lab-wPip and wild-wPip males to suppress an Ae. albopictus population under laboratory-controlled conditions. In cages where wild-wPip males were introduced weekly, we obtained a suppression efficiency of 100% 15 weeks after the first release, with a suppression level exceeding 99% from the seventh week after the first release. By contrast, only ~75% suppression was measured in cages where males from the lab-wPip line were used under the same conditions (Fig. 4A). Similarly, while hatch rates dropped to 0% in cages where wild-wPip males were released, hatch rates plateaued at around 60% in lab-wPip cages and 80% in control cages (Fig. 4B). In the control cages, where no incompatible males were introduced, the number of eggs laid per week increased substantially over the first 7 weeks before plateauing at roughly 5,000 eggs per week, whereas an average value of 74 eggs was recorded in wild-wPip and 1,845 in lab-wPip treatment cages at the end of the experiment (Fig. 4C).
Fig 4.
Evaluation of incompatible males from lab-wPip and wild-wPip lines effectiveness to suppress Ae. albopictus population. (A) Suppression efficiency (%). (B) Hatch rates of eggs. (C) Number of eggs laid. Black vertical dotted lines represent the beginning of incompatible male releases. Each dot represents the mean value, and the vertical lines indicate the standard deviation.
DISCUSSION
Wolbachia infections may alter the fitness of their hosts, with the observed effects influenced by both Wolbachia strains and the lineage of the insect host (35, 46, 47). This is particularly true when Wolbachia is introduced experimentally into insect hosts, resulting in diverse effects on fitness, which are hardly predictable since densities and tissue distributions can change dramatically upon trans-infection (12, 48). We previously introduced the wPip-IV Wolbachia strain in an Ae. albopictus Wolbachia-free (asymbiotic) laboratory line and showed that this new infection induced a physiological cost, characterized by a decrease in hatch rate and fecundity as compared to a naturally wAlbA/B bi-infected or an asymbiotic line (20). The horizontal transfer of wPip into an asymbiotic Ae. albopictus line from Italy also reduced fitness several generations post-transfer, with the suggestion that a continued selection could attenuate the negative fitness effects associated with artificial Wolbachia infection (39, 40).
In our case, we hypothesized that these fitness costs were due to the genetic drift induced by the successive population bottlenecks resulting from the construction of the trans-infected line, including the generation of the asymbiotic line through antibiotic treatment and the selection of isofemale lines showing perfect maternal transmission of wPip (20). Under such circumstances, inbreeding depression is likely to occur through mating between closely related individuals, resulting in increased homozygosity and the appearance of traits associated with reduced fitness (49, 50). Other factors such as gut microbiota could also explain the variation of fitness between laboratory and field populations because the microbiome, which can greatly influence Aedes mosquitoes LHTs, is drastically affected by laboratory maintenance (51). Indeed, gut microbiota are much less diverse in colonized mosquitoes and tend to be similar in laboratory populations of Ae. aegypti regardless of geographic origin (52), suggesting that in artificial conditions the nuclear compartment plays a greater role in mosquito performance. For example, in Ae. aegypti, a loss of fitness due to inbreeding was strongly correlated with a decreased effective population size (53).
In this study, we demonstrated that introgression with a wild background lineage for as little as four consecutive generations restored most fitness characteristics of a trans-infected line to those of its native wild counterpart. Indeed, we showed that the post-introgression wild-wPip and the native wild-wAlb lines had both comparable fecundity and egg hatch rates. Males from the wild-wPip and the wild-wAlb lines also had undistinguishable longevity and insemination capacities, leading to equal mating competitiveness. Finally, we measured the capacity of incompatible males, whether introgressed or not, to suppress a wild population under laboratory-controlled conditions. Using a protocol based on a closed population (no migration) and without taking into account potential density-dependent effects, a population extinction was expected with incompatible males inducing complete CI. The releases of wild-wPip males allowed us to achieve the expected elimination, with complete suppression obtained 15 weeks after the first release, whereas the use of males from the non-introgressed lab-wPip only resulted in suppression levels stagnating at ~75% until 15 weeks after the first release. An intermediate level of suppression was also obtained in similar conditions when this lab-wPip line was combined with a Genetic Sexing Strain (20). In another investigation, repetitive releases of incompatible males from the triple-Wolbachia-Ae. albopictus HC lines were seen to lead to a suppression level close to 100% ~10 weeks after the first release (15). A limitation of our experiments is that we assessed the effects of introgression only under laboratory conditions. The restoration of incompatible male performance in these conditions does not necessarily indicate similar competitive behavior in the field. Therefore, population suppression experiments should be repeated in semi-field and field conditions before considering the use of this line for large-scale deployment.
Our findings suggest that several rounds of introgression can be an alternative to a selection process for decreasing the negative impacts of a Wolbachia strain post-introduction. It has been shown, for example, in naturally Wolbachia-free Ae. aegypti as well as in a wMelPop-CLA trans-infected line that outcrossing can introduce considerable genetic heterogeneity resulting in improved host fitness (53, 54). Our study also reveals that genetic heterogeneity within a mosquito line, regardless of its geographical origin, is crucial for better characterizing a Wolbachia-host association. This is of direct importance when the horizontal transfer aims to select a Wolbachia candidate for IIT applications.
Furthermore, to maximize the success of a release program, a common precaution is to introgress the genetic background of wild-collected mosquitoes into the strain to be released through repeated crossing (55–57). This recommendation is also relevant for large-scale production, in which lineage maintenance over multiple generations under laboratory conditions can significantly reduce the fitness and heterozygosity of mosquitoes (53). For example, it has been reported that laboratory-adapted Aedes mosquitoes show shorter development and increased body size (53, 58), higher sensitivity to stress (59), and reduced female fecundity relative to their field counterparts (60).
Our findings are highly significant for the implementation of IIT programs as they contribute to improving their efficiency. Introgression should be considered a crucial component in the development of transinfected lines before initiating male release programs. Moreover, in our case, introgression positively impacted females fitness, increasing the number of eggs laid per female, thereby facilitating the mass production process. Because regular introgression by backcrossing during mass production will be practically challenging, we suggest that introgression should be performed at a laboratory scale before mass production and that genetic variability could be preserved by the maintenance of a large number of individuals. It has been demonstrated that repeated outcrossing with a wild-type line and maintaining a large effective population size allowed the mass rearing of the triple-Wolbachia Ae. albopictus HC lines up to 15 million mosquitoes per week for over 50 generations without a reduction in the quality of the artificially bred mosquitoes (61).
The data presented in this study are of general interest for release programs, especially for IIT deployment. We demonstrated the potential of introgression with a wild genetic background into a transinfected laboratory line, resulting in the restoration of male fitness that had been artificially infected with Wolbachia. The restoration of fitness and genetic variability through introgression is an important development toward the successful application of population suppression programs using this trans-infected line.
MATERIALS AND METHODS
Mosquito lines and introgression of a wild genetic background into the trans-infected laboratory line
Three different Ae. albopictus lines were used in the study. The first one, named lab-wPip, is an Ae. albopictus line trans-infected with wPip-IV (20), a Wolbachia strain naturally occurring in Culex pipiens (62). This trans-infected line was originally obtained using a natural population from Reunion Island maintained in the insectary for 40 generations, that was deprived of its natural Wolbachia through an antibiotic treatment during six consecutive generations and finally used as a recipient strain of wPip-IV (20). The second line, named wild-wAlb which provided the nuclear background used for the introgression of the incompatible lab-wPip line, was obtained at the city of Le Port, Reunion island, following a single time point sampling using ~60 ovitraps placed throughout an area of 12.5 hectares. Then, we backcrossed 15-day-old wild-wAlb males (using the F1 post sampling) with 4–5-day-old females from the lab-wPip line for four consecutive generations, resulting in a line named wild-wPip (expected to display 93.75% of the wild genetic background). Adults of all lines were maintained at a temperature of 27°C ± 1°C, a relative humidity of 70% ± 5%, and a 12:12 h light:dark photoperiod. Females were blood-fed with bovine blood provided by the regional slaughterhouse (Saint-Pierre, La Réunion), supplemented with EDTA (0.1%), and using the Hemotek system (Hemotek Ltd., United Kingdom).
Verification of CI penetrance of wPip-IV in a wild genetic background
To confirm CI expression following the introgression of the wild-wPip line, we performed reciprocal en masse crosses using wild-wPip and wild-wAlb lines. Although en masse crosses can mask individual variability, especially in the case of moderate CI, we expected complete or nearly complete CI (20), which led us to favor en masse rather than individual crosses in order to screen a larger number of mosquitoes. Crosses involved 2–5-day-old virgin females and males (100 from each sex) in 15 × 15 × 15 cm cages (Bugdorm, Taiwan). Three replicates were performed for each crossing experiment. Females were given a blood meal 48 h after mating and eggs were collected 5 days later. After 7 days of drying, eggs were allowed to hatch for 48 h (in a jar of 250 mL with tap water supplemented with 5 mg of TetraMin [TETRA]), and the number of hatched and unhatched eggs was counted. A hatch rate was calculated as follows: hatch rate = (number of hatched eggs/total number of eggs) × 100. To account for embryonic mortality not related to CI, we used a corrected index of CI (CIcorr) (47, 63) calculated as follows: CIcorr = [(CIobs − CCM)/(100 − CCM)] × 100, where CIobs is the percentage of unhatched eggs observed in a given incompatible cross, and CCM is the mean mortality observed in the control crosses. The F2 and F5 were used for wild-wPip and wild-wAlb lines, respectively.
Comparison of life history traits
The larvae-rearing conditions were standardized between lines and were the same for all experiments presented in this study. Specifically, eggs from each line were allowed to hatch for 24 h at 31°C in containers containing 250 mL of water supplemented with 50 mg TetraMin (TETRA). For each line, 2,500 L1 were manually counted and transferred to a tray (53 × 325 × 65 cm, MORI 2A) and fed with a controlled quantity of food (TetraMin [TETRA], day 1: 0.45 g, day 2: 1 g, day 3: 1.25 g, day 4: 1 g, day 5: 1 g, day 6: 1 g) until pupal stage. Male and female pupae were initially separated using a pupae sex sorter (Wolbaki, WBK-P0001-V1 model), and then individually inspected under a binocular loupe.
For each line, longevity was measured by introducing 100 newly emerged males or females separately in 15 × 15 × 15 cm cages (Bugdorm, Taiwan) in which sugar meal (5%) was changed weekly. Three replicates were performed for each line and sex and longevity were determined by recording the number of dead mosquitoes for 45 days.
For each line, the number of laid eggs was measured by placing 200 male and 200 female pupae (one cage per line) in 30 × 30 × 30 cm cages (Bugdorm, Taiwan) and left for 3 days following emergence. A blood meal was then provided and engorged females were placed in a separate cage. Five days later, 50 females were randomly selected and placed individually in a small plastic cup for egg laying. Cups with at least one laid egg were conserved for the analyses. After 7 days of drying, eggs were counted, allowed to hatch for 48 h, and hatch rates were measured. The F3 and F6 were used for wild-wPip and wild-wAlb lines, respectively.
The insemination capacity of males was evaluated for each line by placing fifty 2–3-day-old virgin males and females in 15 × 15 × 15 cm cages (Bugdorm, Taiwan). The insemination capacity of males was evaluated by crossing males and females of the same line and crossing incompatible males from the wild-wPip and lab-wPip with the wild-wAlb females. Three different sex ratios: 1:1 (female:male), 1:0.5, and 1:0.25 were tested for each cross. After 48 h of mating, all individuals were stored at −20°C. For each cross and sex ratio, 30 females were randomly selected and dissected to quantify the insemination rate. Females were considered inseminated when presenting at least one spermatheca filled with sperm (64). This experiment was repeated three times, thus 90 females were dissected for each cross and sex ratio. The F4 and F7 were used for wild-wPip and wild-wAlb lines, respectively.
Comparison of the mating competitiveness of introgressed versus non-introgressed males
We evaluated the mating competitiveness of incompatible males by mixing virgin females from the wild-wAlb line with different ratios of males from the wild-wPip and lab-wPip lines. Pupae from both lines were allowed to emerge in separate 30 × 30 × 30 cm cages (Bugdorm, Taiwan) with sugar meal (5%); 2–3-day-old virgin females and males were used for this experiment. Females (N = 100) were first placed inside cages followed by the simultaneous release of all males, and mating was allowed for 48 h. All males were then removed, a blood meal was provided and eggs were collected by oviposition en masse 5 days later. After 7 days of drying, eggs were allowed to hatch and the hatching rate was measured and used as a proxy of mating competitiveness. Five ratios were tested: 1:1 (50♂ wild-wAlb:50♂ wild-wPip or lab-wPip), 1:5 (17♂ wild-wAlb:85♂ wild-wPip or lab-wPip), 1:10 (9♂ wild-wAlb:90♂ wild-wPip or lab-wPip), and two control ratios, 1:0 (100♂ wild-wAlb:0♂ wild-wPip or lab-wPip) and 0:1 (0♂ wild-wAlb:100♂ wild-wPip or lab-wPip). Three replicates were performed for each cross and ratio. The F3 and F6 were used for wild-wPip and wild-wAlb lines, respectively.
Laboratory cage suppression experiment
A wild population was established in each control cage (four replicates) introducing weekly 100 male and female pupae of the wild-wAlb line from the beginning to the end of the experiment. The treatment cages (four replicates) were treated similarly until the end of week 4, then 500 incompatible adult males from the wild-wPip or lab-wPip line were released weekly, resulting in a 5:1 ratio (500 wild-wPip or lab-wPip males:100 wild-wAlb males). We took into account the level of population suppression induced by the repeated releases of incompatible males in the treatment cages. Thus the number of introduced wild-wAlb pupae was determined by the difference of egg hatch rates between control and treatment cages. For example, if the egg hatch rate was 80% and 60% (20% difference) in control and treatment cages, respectively, we introduced 20% less wild-wAlb males and females in the treatment cages the following week (100 × 0.2 = 80), therefore 80 male and female pupae, instead of 100 pupae of each sex in the control cages.
Throughout the experiment, three blood meals were provided weekly and eggs were collected once a week. After 7 days of drying, all eggs were counted and allowed to hatch for 24 h. Thus, the number of eggs and hatch rates were determined weekly for each cage. Altogether, 12 cages (45 × 45 × 45 cm, Bugdorm, Taiwan) were monitored, including four repetitions for each cage type (control, wild-wPip, and lab-wPip). The suppression efficiency (SE) was calculated based on the following equation, where Hc and Ht are the numbers of hatched eggs counted weekly in control and treatment cages, respectively (15, 19). SE (%) = [(µHc – Ht)/µHc] × 100. The F4 and F7 were used at the beginning of the experiment, for wild-wPip and wild-wAlb lines, respectively.
Statistical analyses
Longevity data were analyzed using a log-rank test. Fecundity data (count data) were analyzed using a generalized linear mixed model (GLMM) (quasi-poisson family, log link) in which the different mosquito lines were included as a fixed effect and the repetitions (individual egg laying) as a random factor. We used GLMM to analyze egg hatch rate (binary data, quasi-binomial family, logit link) and insemination rate data (binary data, binomial family, logit link). The overdispersion of the data was checked using an R code proposed by Ben Bolker and others (https://bbolker.github.io/mixedmodels-misc/glmmFAQ.html). An F-test for the GLMM-quasi-binomial and quasi-poisson models was used to analyze deviance. Exact binomial tests were used to compare the observed and expected hatch rates in the mating competitiveness experiment (proportional data). Analyses were performed in R version 4.3.3 (65), using the lme4 package for all mixed models (66), the MASS package (67) for using the quasi-binomial and quasi-poisson families in the GLMMs, and the survival package for longevity data analyses (68). For all data, the significance level was set to α = 0.05.
ACKNOWLEDGMENTS
We thank Dr. David Wilkinson for his helpful proofreading of the manuscript.
This work was supported by a Credit Impôt Recherche provided by the French Ministry of Research and Higher Education to SymbioTIC and by the European Regional Development Fund (ERDF) (INTERREG VI program; OPERATING SEYWOL no: REU004954).
Note
In the Fig 1, the letters A & C are missing.
Footnotes
This article is a direct contribution from Pablo Tortosa, a member of the Applied and Environmental Microbiology Editorial Board, who arranged for and secured reviews by Katerina Nikolouli, International Atomic Energy Agency; Thomas Ant, MRC-University of Glasgow Centre for Virus Research; and Antonios Augustinos, Hellenic Agricultural Organization—DIMITRA.
Contributor Information
Pablo Tortosa, Email: pablo.tortosa@univ-reunion.fr.
Julien Cattel, Email: juliencattel@gmail.com.
Gemma Reguera, Michigan State University, East Lansing, Michigan, USA.
REFERENCES
- 1. Hilgenboecker K, Hammerstein P, Schlattmann P, Telschow A, Werren JH. 2008. How many species are infected with Wolbachia? – a statistical analysis of current data. FEMS Microbiol Lett 281:215–220. doi: 10.1111/j.1574-6968.2008.01110.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Zug R, Hammerstein P. 2012. Still a host of hosts for Wolbachia: analysis of recent data suggests that 40% of terrestrial arthropod species are infected. PLoS One 7:e38544. doi: 10.1371/journal.pone.0038544 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Weinert LA, Araujo-Jnr EV, Ahmed MZ, Welch JJ. 2015. The incidence of bacterial endosymbionts in terrestrial arthropods. Proc Biol Sci 282:20150249. doi: 10.1098/rspb.2015.0249 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Bailly-Bechet M, Martins-Simões P, Szöllosi GJ, Mialdea G, Sagot MF, Charlat S. 2017. How long does Wolbachia remain on board? Mol Biol Evol 34:1183–1193. doi: 10.1093/molbev/msx073 [DOI] [PubMed] [Google Scholar]
- 5. Sanaei E, Charlat S, Engelstädter J. 2021. Wolbachia host shifts: routes, mechanisms, constraints and evolutionary consequences. Biol Rev Camb Philos Soc 96:433–453. doi: 10.1111/brv.12663 [DOI] [PubMed] [Google Scholar]
- 6. Werren JH, Baldo L, Clark ME. 2008. Wolbachia: master manipulators of invertebrate biology. Nat Rev Microbiol 6:741–751. doi: 10.1038/nrmicro1969 [DOI] [PubMed] [Google Scholar]
- 7. Turelli M, Katznelson A, Ginsberg PS. 2022. Why Wolbachia-induced cytoplasmic incompatibility is so common. Proc Natl Acad Sci U S A 119:e2211637119. doi: 10.1073/pnas.2211637119 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Werren JH. 1997. Biology of Wolbachia. Annu Rev Entomol 42:587–609. doi: 10.1146/annurev.ento.42.1.587 [DOI] [PubMed] [Google Scholar]
- 9. Beckmann JF, Ronau JA, Hochstrasser M. 2017. A Wolbachia deubiquitylating enzyme induces cytoplasmic incompatibility. Nat Microbiol 2:17007. doi: 10.1038/nmicrobiol.2017.7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. LePage DP, Metcalf JA, Bordenstein SR, On J, Perlmutter JI, Shropshire JD, Layton EM, Funkhouser-Jones LJ, Beckmann JF, Bordenstein SR. 2017. Prophage WO genes recapitulate and enhance Wolbachia-induced cytoplasmic incompatibility. Nature 543:243–247. doi: 10.1038/nature21391 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Iturbe-Ormaetxe I, Walker T, O’ Neill SL. 2011. Wolbachia and the biological control of mosquito‐borne disease. EMBO Rep 12:508–518. doi: 10.1038/embor.2011.84 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Hoffmann AA, Ross PA, Rašić G. 2015. Wolbachia strains for disease control: ecological and evolutionary considerations. Evol Appl 8:751–768. doi: 10.1111/eva.12286 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Boller EF, Russ K, Vallo V, Bush GL. 1976. Incompatible races of european cherry fruit fly, Rhagoletis cerasi (dipter a: tephritidae), their origin and potential use in biological control. Entomol Exp Applic 20:237–247. doi: 10.1111/j.1570-7458.1976.tb02640.x [DOI] [Google Scholar]
- 14. Riegler M, Stauffer C. 2002. Wolbachia infections and superinfections in cytoplasmically incompatible populations of the European cherry fruit fly Rhagoletis cerasi (Diptera, Tephritidae). Mol Ecol 11:2425–2434. doi: 10.1046/j.1365-294x.2002.01614.x [DOI] [PubMed] [Google Scholar]
- 15. Zheng X, Zhang D, Li Y, Yang C, Wu Y, Liang X, Liang Y, Pan X, Hu L, Sun Q, et al. 2019. Incompatible and sterile insect techniques combined eliminate mosquitoes. Nature 572:56–61. doi: 10.1038/s41586-019-1407-9 [DOI] [PubMed] [Google Scholar]
- 16. Bansal S, Lim JT, Chong C-S, Dickens B, Ng Y, Deng L, Lee C, Tan LY, Kakani EG, Yoong Y, Du Yu D, Chain G, Ma P, Sim S, Ng LC, Tan CH. 2024. Effectiveness of Wolbachia-mediated sterility coupled with sterile insect technique to suppress adult Aedes aegypti populations in Singapore: a synthetic control study. Lancet Planet Health 8:e617–e628. doi: 10.1016/S2542-5196(24)00169-4 [DOI] [PubMed] [Google Scholar]
- 17. Crawford JE, Clarke DW, Criswell V, Desnoyer M, Cornel D, Deegan B, Gong K, Hopkins KC, Howell P, Hyde JS, et al. 2020. Efficient production of male Wolbachia-infected Aedes aegypti mosquitoes enables large-scale suppression of wild populations. Nat Biotechnol 38:482–492. doi: 10.1038/s41587-020-0471-x [DOI] [PubMed] [Google Scholar]
- 18. Beebe NW, Pagendam D, Trewin BJ, Boomer A, Bradford M, Ford A, Liddington C, Bondarenco A, De Barro PJ, Gilchrist J, Paton C, Staunton KM, Johnson B, Maynard AJ, Devine GJ, Hugo LE, Rasic G, Cook H, Massaro P, Snoad N, Crawford JE, White BJ, Xi Z, Ritchie SA. 2021. Releasing incompatible males drives strong suppression across populations of wild and Wolbachia-carrying Aedes aegypti in Australia. Proc Natl Acad Sci U S A 118:e2106828118. doi: 10.1073/pnas.2106828118 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Zeng Q, She L, Yuan H, Luo Y, Wang R, Mao W, Wang W, She Y, Wang C, Shi M, Cao T, Gan R, Li Y, Zhou J, Qian W, Hu S, Wang Y, Zheng X, Li K, Bai L, Pan X, Xi Z. 2022. A standalone incompatible insect technique enables mosquito suppression in the urban subtropics. Commun Biol 5:1419. doi: 10.1038/s42003-022-04332-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Scussel S, Gaudillat B, Esnault J, Lejarre Q, Duployer M, Lebon C, Benlali A, Mavingui P, Tortosa P, Cattel J. 2024. Combining transinfected Wolbachia and a genetic sexing strain to control Aedes albopictus in laboratory-controlled conditions. Proc Biol Sci 291:20240429. doi: 10.1098/rspb.2024.0429 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Koskinioti P, Augustinos AA, Carvalho DO, Misbah-Ul-Haq M, Pillwax G, de la Fuente LD, Salvador-Herranz G, Herrero RA, Bourtzis K. 2021. Genetic sexing strains for the population suppression of the mosquito vector Aedes aegypti. Philos Trans R Soc Lond B Biol Sci 376:20190808. doi: 10.1098/rstb.2019.0808 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Lutrat C, Giesbrecht D, Marois E, Whyard S, Baldet T, Bouyer J. 2019. Sex sorting for pest control: it’s raining men! Trends Parasitol 35:649–662. doi: 10.1016/j.pt.2019.06.001 [DOI] [PubMed] [Google Scholar]
- 23. Lutrat C, Olmo RP, Baldet T, Bouyer J, Marois E. 2022. Transgenic expression of Nix converts genetic females into males and allows automated sex sorting in Aedes albopictus. Commun Biol 5:210. doi: 10.1038/s42003-022-03165-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Gong J-T, Mamai W, Wang X, Zhu J, Li Y, Liu J, Tang Q, Huang Y, Zhang J, Zhou J, Maiga H, Somda NSB, Martina C, Kotla SS, Wallner T, Bouyer J, Xi Z. 2024. Upscaling the production of sterile male mosquitoes with an automated pupa sex sorter. Sci Robot 9:eadj6261. doi: 10.1126/scirobotics.adj6261 [DOI] [PubMed] [Google Scholar]
- 25. Bourtzis K. 2008. Wolbachia-based technologies for insect pest population control. Transgenes Manag Vector-Borne Dis 627:104–113. doi: 10.1007/978-0-387-78225-6_9 [DOI] [PubMed] [Google Scholar]
- 26. Laven H. 1967. Eradication of Culex pipiens fatigans through cytoplasmic Incompatibility. Nature 216:383–384. doi: 10.1038/216383a0 [DOI] [PubMed] [Google Scholar]
- 27. Atyame CM, Cattel J, Lebon C, Flores O, Dehecq JS, Weill M, Gouagna LC, Tortosa P. 2015. Wolbachia-based population control strategy targeting Culex quinquefasciatus mosquitoes proves efficient under semi-field conditions. PLoS One 10:e0119288. doi: 10.1371/journal.pone.0119288 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Atyame CM, Pasteur N, Dumas E, Tortosa P, Tantely ML, Pocquet N, Licciardi S, Bheecarry A, Zumbo B, Weill M, Duron O. 2011. Cytoplasmic incompatibility as a means of controlling Culex pipiens quinquefasciatus mosquito in the islands of the south-western Indian Ocean. PLoS Negl Trop Dis 5:e1440. doi: 10.1371/journal.pntd.0001440 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Brelsfoard CL, Séchan Y, Dobson SL. 2008. Interspecific hybridization yields strategy for South Pacific filariasis vector elimination. PLoS Negl Trop Dis 2:e129. doi: 10.1371/journal.pntd.0000129 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Xi Z, Dobson SL. 2005. Characterization of Wolbachia transfection efficiency by using microinjection of embryonic cytoplasm and embryo homogenate. Appl Environ Microbiol 71:3199–3204. doi: 10.1128/AEM.71.6.3199-3204.2005 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Xi Z, Dean JL, Khoo C, Dobson SL. 2005. Generation of a novel Wolbachia infection in Aedes albopictus (Asian tiger mosquito) via embryonic microinjection. Insect Biochem Mol Biol 35:903–910. doi: 10.1016/j.ibmb.2005.03.015 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Hughes GL, Rasgon JL. 2014. Transinfection: a method to investigate Wolbachia-host interactions and control arthropod-borne disease. Insect Mol Biol 23:141–151. doi: 10.1111/imb.12066 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Zhang M, Xi Z. 2023. Wolbachia transinfection via embryonic microinjection, p 175–188. In Wolbachia: methods and protocols. Springer. [DOI] [PubMed] [Google Scholar]
- 34. McGraw EA, Merritt DJ, Droller JN, O’Neill SL. 2002. Wolbachia density and virulence attenuation after transfer into a novel host. Proc Natl Acad Sci U S A 99:2918–2923. doi: 10.1073/pnas.052466499 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. Fraser JE, De Bruyne JT, Iturbe-Ormaetxe I, Stepnell J, Burns RL, Flores HA, O’Neill SL. 2017. Novel Wolbachia-transinfected Aedes aegypti mosquitoes possess diverse fitness and vector competence phenotypes. PLoS Pathog 13:e1006751. doi: 10.1371/journal.ppat.1006751 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Madhav M, Brown G, Morgan JAT, Asgari S, McGraw EA, James P. 2020. Transinfection of buffalo flies (Haematobia irritans exigua) with Wolbachia and effect on host biology. Parasit Vectors 13:296. doi: 10.1186/s13071-020-04161-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Ant TH, Herd CS, Geoghegan V, Hoffmann AA, Sinkins SP. 2018. The Wolbachia strain wAu provides highly efficient virus transmission blocking in Aedes aegypti. PLoS Pathog 14:e1006815. doi: 10.1371/journal.ppat.1006815 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Ross PA, Turelli M, Hoffmann AA. 2019. Evolutionary ecology of Wolbachia releases for disease control. Annu Rev Genet 53:93–116. doi: 10.1146/annurev-genet-112618-043609 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Boyle L, O’Neill SL, Robertson HM, Karr TL. 1993. Interspecific and intraspecific horizontal transfer of Wolbachia in Drosophila. Science 260:1796–1799. doi: 10.1126/science.8511587 [DOI] [PubMed] [Google Scholar]
- 40. Calvitti M, Moretti R, Lampazzi E, Bellini R, Dobson SL. 2010. Characterization of a new Aedes albopictus (Diptera: Culicidae)-Wolbachia pipientis (Rickettsiales: Rickettsiaceae) symbiotic association generated by artificial transfer of the wPip strain from Culex pipiens (Diptera: Culicidae). J Med Entomol 47:179–187. doi: 10.1603/me09140 [DOI] [PubMed] [Google Scholar]
- 41. Turelli M. 1994. Evolution of incompatibility-inducing microbes and their hosts. Evolution (N Y) 48:1500–1513. doi: 10.1111/j.1558-5646.1994.tb02192.x [DOI] [PubMed] [Google Scholar]
- 42. Weeks AR, Turelli M, Harcombe WR, Reynolds KT, Hoffmann AA. 2007. From parasite to mutualist: rapid evolution of Wolbachia in natural populations of Drosophila. PLoS Biol 5:e114. doi: 10.1371/journal.pbio.0050114 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43. Schneider DI, Riegler M, Arthofer W, Merçot H, Stauffer C, Miller WJ. 2013. Uncovering Wolbachia diversity upon artificial host transfer. PLoS One 8:e82402. doi: 10.1371/journal.pone.0082402 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44. Oliva CF, Damiens D, Vreysen MJB, Lemperière G, Gilles J. 2013. Reproductive strategies of Aedes albopictus (Diptera: Culicidae) and implications for the sterile insect technique. PLoS One 8:e78884. doi: 10.1371/journal.pone.0078884 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45. Parsana D, Nanfack-Minkeu F, Sirot LK. 2022. Insemination in Aedes aegypti and Aedes albopictus. Cold Spring Harb Protoc 2022:413–416. doi: 10.1101/pdb.top107668 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46. Mouton L, Henri H, Charif D, Boulétreau M, Vavre F. 2007. Interaction between host genotype and environmental conditions affects bacterial density in Wolbachia symbiosis. Biol Lett 3:210–213. doi: 10.1098/rsbl.2006.0590 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47. Cattel J, Nikolouli K, Andrieux T, Martinez J, Jiggins F, Charlat S, Vavre F, Lejon D, Gibert P, Mouton L. 2018. Back and forth Wolbachia transfers reveal efficient strains to control spotted wing drosophila populations. J Appl Ecol 55:2408–2418. doi: 10.1111/1365-2664.13101 [DOI] [Google Scholar]
- 48. Martinez J, Ok S, Smith S, Snoeck K, Day JP, Jiggins FM. 2015. Should symbionts be nice or selfish? Antiviral effects of Wolbachia are costly but reproductive parasitism is not. PLoS Pathog 11:e1005021. doi: 10.1371/journal.ppat.1005021 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49. Reed DH, Frankham R. 2003. Correlation between fitness and genetic diversity. Conserv Biol 17:230–237. doi: 10.1046/j.1523-1739.2003.01236.x [DOI] [Google Scholar]
- 50. Wright LI, Tregenza T, Hosken DJ. 2008. Inbreeding, inbreeding depression and extinction. Conserv Genet 9:833–843. doi: 10.1007/s10592-007-9405-0 [DOI] [Google Scholar]
- 51. Scolari F, Casiraghi M, Bonizzoni M. 2019. Aedes spp. and their microbiota: a review. Front Microbiol 10:1–19. doi: 10.3389/fmicb.2019.02036 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52. Dickson LB, Ghozlane A, Volant S, Bouchier C, Ma L, Vega-Rúa A, Dusfour I, Jiolle D, Paupy C, Mayanja MN, Kohl A, Lutwama JJ, Duong V, Lambrechts L. 2018. Diverse laboratory colonies of Aedes aegypti harbor the same adult midgut bacterial microbiome. Parasit Vectors 11:207. doi: 10.1186/s13071-018-2780-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53. Ross PA, Endersby-Harshman NM, Hoffmann AA. 2019. A comprehensive assessment of inbreeding and laboratory adaptation in Aedes aegypti mosquitoes. Evol Appl 12:572–586. doi: 10.1111/eva.12740 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54. Yeap HL, Mee P, Walker T, Weeks AR, O’Neill SL, Johnson P, Ritchie SA, Richardson KM, Doig C, Endersby NM, Hoffmann AA. 2011. Dynamics of the “popcorn” Wolbachia infection in outbred Aedes aegypti informs prospects for mosquito vector control. Genetics 187:583–595. doi: 10.1534/genetics.110.122390 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55. Benedict MQ. 2021. Sterile insect technique: lessons from the past. J Med Entomol 58:1974–1979. doi: 10.1093/jme/tjab024 [DOI] [PubMed] [Google Scholar]
- 56. Ross PA. 2021. Designing effective Wolbachia release programs for mosquito and arbovirus control. Acta Trop 222:106045. doi: 10.1016/j.actatropica.2021.106045 [DOI] [PubMed] [Google Scholar]
- 57. Garcia G de A, Sylvestre G, Aguiar R, da Costa GB, Martins AJ, Lima JBP, Petersen MT, Lourenço-de-Oliveira R, Shadbolt MF, Rašić G, Hoffmann AA, Villela DAM, Dias FBS, Dong Y, O’Neill SL, Moreira LA, Maciel-de-Freitas R. 2019. Matching the genetics of released and local Aedes aegypti populations is critical to assure Wolbachia invasion. PLoS Negl Trop Dis 13:e0007023. doi: 10.1371/journal.pntd.0007023 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58. Yeap HL, Endersby NM, Johnson PH, Ritchie SA, Hoffmann AA. 2013. Body size and wing shape measurements as quality indicators of Aedes aegypti mosquitoes destined for field release. Am J Trop Med Hyg 89:78–92. doi: 10.4269/ajtmh.12-0719 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59. Hoffmann AA, Ross PA. 2018. Rates and patterns of laboratory adaptation in (mostly) insects. J Econ Entomol 111:501–509. doi: 10.1093/jee/toy024 [DOI] [PubMed] [Google Scholar]
- 60. Richardson AJ, Williams CR. 2013. Inter-population mating success in Australian dengue vector mosquitoes: effects of laboratory colonization and implications for the spread of transgenics. J Vector Ecol 38:111–119. doi: 10.1111/j.1948-7134.2013.12016.x [DOI] [PubMed] [Google Scholar]
- 61. Li Y, Zhang M, Wang X, Zheng X, Hu Z, Xi Z. 2021. Quality control of long-term mass-reared Aedes albopictus for population suppression. J Pest Sci 94:1531–1542. doi: 10.1007/s10340-021-01340-z [DOI] [Google Scholar]
- 62. Atyame CM, Delsuc F, Pasteur N, Weill M, Duron O. 2011. Diversification of Wolbachia endosymbiont in the Culex pipiens mosquito. Mol Biol Evol 28:2761–2772. doi: 10.1093/molbev/msr083 [DOI] [PubMed] [Google Scholar]
- 63. Poinsot D, Bourtzis K, Markakis G, Savakis C, Merçot H. 1998. Wolbachia transfer from Drosophila melanogaster into D. simulans: host effect and cytoplasmic incompatibility relationships. Genetics 150:227–237. doi: 10.1093/genetics/150.1.227 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64. Balestrino F, Puggioli A, Carrieri M, Bouyer J, Bellini R. 2017. Quality control methods for Aedes albopictus sterile male production. PLoS Negl Trop Dis 11:e0005881. doi: 10.1371/journal.pntd.0005881 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65. R Core Team . 2023. R: a language and environment for statistical computing. Vienna, Austria. [Google Scholar]
- 66. Bates D, Maechler M, Bolker B, Walker S, Christensen RHB, Singmann H, Dai B, Grothendieck G, Green P, Fox J, Bauer A, Krivitsky PN, Tanaka E, Jagan M. 2015. Package “lme4.” Convergence 12:2
- 67. Ripley B, Venables B, Bates DM, Hornik K, Gebhardt A, Firth D, Ripley MB. 2013. Package ‘mass’. Cran r 538:113–120
- 68. Therneau TM. 2023. A package for survival analysis in R




