Skip to main content
Proceedings of the National Academy of Sciences of the United States of America logoLink to Proceedings of the National Academy of Sciences of the United States of America
. 2017 May 8;114(21):5413–5418. doi: 10.1073/pnas.1700396114

Hybridizing transgenic Bt cotton with non-Bt cotton counters resistance in pink bollworm

Peng Wan a,b, Dong Xu b, Shengbo Cong b, Yuying Jiang c, Yunxin Huang d, Jintao Wang b, Huaiheng Wu b, Ling Wang a,b, Kongming Wu a,1, Yves Carrière e, Andrea Mathias e, Xianchun Li e, Bruce E Tabashnik e
PMCID: PMC5448178  PMID: 28483999

Significance

Crops genetically engineered to produce insecticidal proteins from the bacterium Bacillus thuringiensis (Bt) kill some major pests and reduce use of insecticide sprays. However, evolution of pest resistance to Bt proteins decreases these benefits. We report a strategy for combating resistance by crossing transgenic Bt plants with conventional non-Bt plants and then sowing the second-generation seeds. This strategy yields a random mixture within fields of three-quarters of plants that produce Bt protein and one-quarter that does not. An 11-y field study in China shows this strategy countered resistance to Bt cotton of pink bollworm, one of the world’s most devastating pests. This outcome illustrates that non-Bt plants in a seed mixture can boost survival of susceptible insects and delay resistance.

Keywords: sustainability, evolution, resistance management, genetically modified, refuge

Abstract

Extensive cultivation of crops genetically engineered to produce insecticidal proteins from the bacterium Bacillus thuringiensis (Bt) has suppressed some major pests, reduced insecticide sprays, enhanced pest control by natural enemies, and increased grower profits. However, these benefits are being eroded by evolution of resistance in pests. We report a strategy for combating resistance by crossing transgenic Bt plants with conventional non-Bt plants and then crossing the resulting first-generation (F1) hybrid progeny and sowing the second-generation (F2) seeds. This strategy yields a random mixture within fields of three-quarters of plants that produce Bt toxin and one-quarter that does not. We hypothesized that the non-Bt plants in this mixture promote survival of susceptible insects, thereby delaying evolution of resistance. To test this hypothesis, we compared predictions from computer modeling with data monitoring pink bollworm (Pectinophora gossypiella) resistance to Bt toxin Cry1Ac produced by transgenic cotton in an 11-y study at 17 field sites in six provinces of China. The frequency of resistant individuals in the field increased before this strategy was widely deployed and then declined after its widespread adoption boosted the percentage of non-Bt cotton plants in the region. The correspondence between the predicted and observed outcomes implies that this strategy countered evolution of resistance. Despite the increased percentage of non-Bt cotton, suppression of pink bollworm was sustained. Unlike other resistance management tactics that require regulatory intervention, growers adopted this strategy voluntarily, apparently because of advantages that may include better performance as well as lower costs for seeds and insecticides.


Genetically engineered crops that produce insecticidal proteins from the bacterium Bacillus thuringiensis (Bt) have been planted globally on a cumulative total of over 732 million ha since 1996 (1). The benefits of these Bt crops include pest suppression, reduced insecticide use, enhanced biological control, and increased farmer profits (27). However, increasingly rapid evolution of resistance to Bt crops by pests has eroded these benefits (811). The main strategy for delaying pest resistance to Bt crops aims to increase the survival of susceptible insects with “refuges” of host plants that do not produce Bt toxins (9). Although refuges can delay insect adaptation to Bt crops (2, 3, 9, 12), the optimal spatial scale for planting refuges remains unresolved (1315). Also, because refuges are often perceived to cause short-term economic sacrifices for growers, they are usually imposed by regulations.

Before 2010, regulations in the United States mandated refuges of non-Bt plants in blocks consisting of separate fields, rows, or strips (14). In 2010, the regulations were modified to include mixtures of Bt and non-Bt seeds generating a random array of Bt and non-Bt plants side by side within fields (14). By 2015, all farmers surveyed in the US Corn Belt planted such seed mixtures for some or all of their corn (16). Seed mixtures have several advantages relative to block refuges, particularly convenience for farmers and elimination of noncompliance with block refuge requirements (15, 17). However, seed mixtures increase opportunities for larval movement between Bt and non-Bt plants or plant tissues, which could accelerate resistance evolution by raising the dominance of resistance or by reducing the survival of susceptible insects (1315, 17). Despite potentially profound implications of the seed mixture strategy for the sustainability of Bt crops, large-scale field tests of its efficacy have been lacking.

Here, we test a version of the seed mixture strategy by comparing predictions from computer modeling with field monitoring data for the pink bollworm (Pectinophora gossypiella) based on evaluation of resistance in 96 sets of insects collected during 2005–2015 from 17 sites in six provinces in the Yangtze River Valley of China (total n = 66,648 larvae tested in bioassays). The millions of small-scale farmers in this region first planted transgenic cotton producing Bt toxin Cry1Ac in 2000, with the pink bollworm as one of the primary targets (18). The larvae of this devastating worldwide pest feed primarily on cotton seeds within bolls, and susceptible larvae are killed by the Cry1Ac in Bt cotton (6, 19).

Previous work in the Yangtze River Valley documented a small but statistically significant increase in resistance of the pink bollworm to Cry1Ac from 2005–2007 to 2008–2010 (18). The percentage of populations with one or more larvae surviving at the diagnostic concentration (9 μg of Cry1Ac per milliliter of diet) increased from 0% in 2005–2007 to 56% in 2008–2010 (total n = 51 populations; Fisher’s exact test: P < 0.0001; Fig. 1A and SI Appendix, Fig. S1 and Table S1). In addition, the median percentage survival at the diagnostic concentration increased from 0% in 2005–2007 to 1.6% in 2008–2010 (Mann–Whitney U test: U = 504, P < 0.001; Fig. 1B). The resistance ratio, which was calculated as the concentration of Cry1Ac killing 50% of larvae (LC50) for a field population divided by the LC50 for the susceptible laboratory strain QJ-S tested in the same year, rose from a mean of 2.5 in 2005–2007 to 4.1 in 2008–2010 (t test: t = 2.2, df = 49, P = 0.03; Fig. 1C and SI Appendix, Table S2).

Fig. 1.

Fig. 1.

Pink bollworm resistance to Bt toxin Cry1Ac in the Yangtze River Valley, 2005–2015. (A) Percentage of populations with one or more survivors at the diagnostic concentration of Cry1Ac (SI Appendix, Table S1). (B) Percentage of survivors in each population (SI Appendix, Table S1). (C) Resistance ratio, the LC50 divided by the LC50 for the susceptible QJ-S laboratory strain tested in the same year as an internal control (SI Appendix, Table S2). Values in B and C are means with SE. Data for 2005–2010 are from ref. 18.

Without major changes, rapid increases in resistance were anticipated after 2010 because refuges of non-Bt cotton varieties had decreased to only 6% of all cotton hectares planted (18) and evolution of resistance is an exponential process expected to accelerate after more than 1% of the population is resistant (20). However, the results reported here show that pink bollworm resistance to Cry1Ac did not increase from 2008–2010 to 2011–2015 (Fig. 1). The percentage of populations with one or more larvae surviving at the diagnostic concentration declined from 56% in 2008–2010 (n = 27 populations) to 0% in 2011–2015 (n = 45 populations) (Fisher’s exact test: P < 0.0001; Fig. 1A). Also, the median percentage survival at the diagnostic concentration declined from 1.6% in 2008–2010 to 0% in 2011–2015 (Mann–Whitney U test: U = 945, P < 0.0001; Fig. 1B). The mean resistance ratio, which is a less sensitive indicator than survival at the diagnostic concentration, decreased slightly but not significantly, from 4.1 in 2008–2010 to 3.7 in 2011–2015 (t test: t = 0.77, df = 70, P = 0.55; Fig. 1C). In control tests, survival at the diagnostic concentration was 0% for the susceptible laboratory strain QJ-S every year from 2005 to 2015 (n = 72 larvae per year, total of 792 larvae), indicating consistency of the bioassays over time.

To understand the unexpected decrease in survival at the diagnostic concentration of Cry1Ac, we investigated the possibility that the percentage of non-Bt cotton plants increased in 2010 and subsequent years because growers purchased and planted seed from second-generation (F2) cotton hybrids. Because of heterosis, F1 and F2 cotton hybrids typically have a higher yield than their parent varieties (21, 22). Whereas production of F1 hybrid seeds requires costly hand pollination, self-pollination of F1 hybrids produces F2 hybrid seeds (21, 22). Crossing Bt cotton with non-Bt cotton creates F1 hybrids that make Bt toxin and are hemizygous for this trait because they have one copy of the Bt transgene and no corresponding allele (23, 24). Self-pollination by such F1 hybrids creates F2 hybrid seeds expected to consist of 25% Bt homozygotes and 50% hemizygotes that produce Bt toxin and 25% non-Bt homozygotes that do not (23).

From 2010 to 2015, we determined the percentage of cotton seeds containing Cry1Ac using immunoassays with 100 seeds tested individually per assay in 140 assays of 16 varieties and 68 hybrids recommended for planting in the Yangtze River Valley (total of 14,000 seeds tested; Fig. 2 and SI Appendix, Fig. S2 and Tables S3–S6). We detected Cry1Ac in 99% (SE = 0.6%) of seeds from six Bt cotton varieties, which is close to the expected 100%. In 10 non-Bt cotton varieties, Cry1Ac occurred in 23% (SE = 2) of seeds, which indicates contamination of the non-Bt cotton seed supply. For the 68 hybrids, the percentage of seeds containing Cry1Ac clusters into two groups: 23 hybrids with a range of 89–100% (mean = 96%, SE = 0.5%) and 45 hybrids with a range of 39–84% (mean = 67%, SE = 1%; SI Appendix, Fig. S2). We refer hereafter to these groups as F1 and F2 hybrids, respectively, because they are likely to consist predominantly of these hybrids. However, the observed percentage of seeds containing Cry1Ac is lower than the expected 100% for F1 hybrids and 75% for F2 hybrids, which implies that some of these F1 and F2 hybrids contain seeds from non-Bt cotton varieties, other hybrids, or both. Based on our classification of the hybrids, the price was 35% higher for seeds from F1 than F2 hybrids (SI Appendix, Table S7), which corresponds with the higher production cost of the F1 hybrids. Although our analysis focuses primarily on 2010–2015, we also surveyed 12 major seed companies to estimate the percentage of F2 hybrids planted from 2004 to 2009 (SI Appendix, Table S8).

Fig. 2.

Fig. 2.

Cotton planted in the Yangtze River Valley of China from 2004 to 2015. (A) Percentage of cotton hectares planted to F1 hybrid cotton, F2 hybrid cotton, non-Bt cotton varieties, and Bt cotton varieties (SI Appendix, Tables S8–S11). (B) Percentage of cotton hectares consisting of plants not producing Cry1Ac: total and contributions by F2 hybrids and non-Bt cotton varieties. The contribution of F1 hybrids plus Bt cotton to the percentage of cotton hectares consisting of plants not producing Cry1Ac was ≤2.5% each year (mean = 1.4%).

To calculate the total percentage of non-Bt cotton plants for each year, we multiplied the proportion of non-Bt cotton seeds for each type of cotton (SI Appendix, Tables S3–S6 and S8) by the percentage of cotton hectares planted with the corresponding type of cotton during that year (Fig. 2A and SI Appendix, Tables S8–S11), and then summed the resulting percentages. The overall percentage of non-Bt cotton plants declined from a maximum of 37% in 2004 to 12% in 2008 and 2009, then nearly doubled to 23% in 2010, and was 25–27% from 2011 to 2015 (Fig. 2B). The initial decrease in non-Bt cotton percentage reflects reduced planting of non-Bt cotton varieties, whereas replacement of Bt cotton varieties and F1 hybrids with F2 hybrids caused the subsequent increase (Fig. 2). In each year from 2010 to 2015, F2 hybrids accounted for 78–84% of the total non-Bt cotton plants (mean = 81%; Fig. 2B), which provided an exceptional opportunity to test hypotheses about how this seed mixture affects the evolution of resistance.

We used computer modeling to evaluate two hypotheses: (a) all non-Bt cotton plants act as refuges, including the non-Bt cotton plants in fields of F2 hybrid cotton, or (b) only non-Bt cotton plants in fields of non-Bt cotton varieties act as refuges. In both scenarios, we simulated the period from the end of the season in 2008 to 2015 using a previously described one-locus, two-allele population genetic model, with fitness values for each genotype on Bt and non-Bt cotton plants based on empirical data (25, 26) (SI Appendix, Table S12). Under hypothesis (a), which yielded an annual mean refuge of 23% non-Bt cotton plants, resistance evolution was delayed, which corresponds with the observed outcome in the field (Fig. 3). In striking contrast, hypothesis (b) yielded an annual mean refuge of only 4.1%, and the predicted percentage of resistant individuals surged to 100% by 2013 (Fig. 3). These comparisons between the observed and predicted outcomes support hypothesis (a) but not hypothesis (b). Sensitivity analyses varying assumptions about refuges and insect fitness show that the predicted delay in resistance evolution associated with the increased percentage of non-Bt cotton plants provided by F2 hybrids is robust (SI Appendix, Figs. S3 and S4).

Fig. 3.

Fig. 3.

Predicted versus observed evolution of pink bollworm resistance to Bt toxin Cry1Ac in the Yangtze River Valley of China from 2008 to 2015. The predictions are from computer simulations of hypotheses proposing that (a) all non-Bt cotton plants act as refuges, including the non-Bt cotton plants in fields of F2 hybrid cotton or (b) only non-Bt cotton plants in fields of non-Bt cotton varieties act as refuges. The observed values are bioassay data for survival at the diagnostic concentration of Cry1Ac (Fig. 1B and SI Appendix, Table S1).

The correspondence between the outcome in the field and modeling results under hypothesis (a) demonstrates that the increase in the overall percentage of non-Bt cotton plants, including F2 hybrids, is sufficient to explain the observed lack of increase in resistance from 2011 to 2015 (Fig. 3). Nevertheless, this result does not exclude the alternative hypothesis that an increase in the relative abundance of non-Bt host plants other than cotton contributed to this delay in resistance. To address this alternative hypothesis, we tested the potential of six other common crops in the Yangtze River Valley to serve as pink bollworm host plants. The results of laboratory experiments confirmed previous reports that okra (Abelmoschus esculentus) and tomato (Lycopersicon esculentum) support pink bollworm larval development (19, 27), but no survival occurred on eggplant (Solanum melongena), cucumber (Cucumis sativus), apple (Malus domestica), or kidney bean (Phaseolus vulgaris) (SI Appendix, Table S13). In field experiments, however, pink bollworm larvae and adults were common in cotton, yet none were found in okra or tomato (SI Appendix, Tables S14 and S15). Thus, it is unlikely that non-Bt host plants other than cotton contributed substantially to the observed delay in evolution of pink bollworm resistance to Bt cotton.

A potential drawback of the increased percentage of non-Bt cotton plants from 2011 to 2015 is higher population density of pink bollworm. However, the mean number of pink bollworm eggs per 100 cotton plants was not higher in 2011–2015 (5.4) than in 2006–2010 (11) (Fig. 4A and SI Appendix, Table S16). Furthermore, no significant association occurred between pink bollworm eggs per cotton plant and year from 2011 to 2015 (linear regression: r2 = 0.05, df = 3, P = 0.72; Fig. 4A), indicating no steady rise in population density during this period. Relative to the 5 y before Bt cotton was planted (1995–1999), when the mean number of pink bollworm eggs per 100 cotton plants was 94 (SE = 9.4), the abundance in 2011–2015 declined by 96%.

Fig. 4.

Fig. 4.

Pink bollworm population density and insecticide sprays in the Yangtze River Valley, 1995–2015. (A) Number of F1 pink bollworm eggs per 100 cotton plants was determined annually for 150 plants per county in each of 12–24 counties (mean = 20 counties, total sample = 63,300 plants; SI Appendix, Table S16). (B) Sprays targeting pink bollworm annually per cotton field based on ∼10 fields per county for 14–16 counties per year (SI Appendix, Table S17). Values in both panels are means with SE.

In parallel, the mean number of insecticide sprays applied yearly against pink bollworm per cotton field decreased from 2.0 in 1995–1999 to 0.64 in 2011–2015, a 69% reduction (Fig. 4B and SI Appendix, Table S17). Moreover, sprays were 27% fewer in 2011–2015 relative to 2006–2010 (mean = 0.88) (Mann–Whitney U test: U = 24, P = 0.02).

Whereas previous work has emphasized problems caused by interbreeding between transgenic and nontransgenic plants (1315), we discovered that increased planting of F2 hybrids from crosses between Bt and non-Bt cotton was associated with delayed evolution of pink bollworm resistance to Bt cotton and continued suppression of this voracious pest. Despite extensive global monitoring of resistance to Bt crops during the past two decades and some examples of fluctuations in resistance allele frequency (2, 9, 28), previous reports have not documented any cases where resistance was markedly delayed or reversed after the threshold of 1% resistant individuals was exceeded in a region for several years, as reported here.

The seed mixture generated with F2 hybrids analyzed here may be especially effective against pink bollworm because of the substantial fitness cost associated with its resistance to Cry1Ac and its recessive inheritance of resistance (25), even when feeding on bolls from F1 hybrids, Cry1Ac-producing bolls from F2 hybrids (SI Appendix, Table S18), or a mixture of Bt and non-Bt seeds (24). Selfing of F2 plants in subsequent generations is expected to maintain roughly 25% Bt homozygotes, 50% hemizygotes, and 25% non-Bt homozygotes. The F2 hybrid seed mixture planted in China differs from the standard seed mixtures used now for managing resistance to Bt corn in the United States, which do not entail hybridization and have either 90% or 95% homozygous Bt seeds combined in seed bags with 10% or 5% homozygous non-Bt seeds, respectively (15).

The major advantage of a seed mixture generated with F2 hybrids is a built-in refuge of roughly one-quarter non-Bt plants, which could be especially important in developing countries where planting of separate refuges is low or nil. Whereas compliance with refuge requirements is considered a key factor that delayed evolution of pink bollworm resistance to Bt cotton in the United States, the scarcity of non-Bt cotton refuges probably hastened this pest’s resistance to Bt cotton in India (8, 9). In the Yangtze River Valley, millions of farmers have planted F2 hybrid cotton not as a resistance management strategy, but apparently for immediate benefits that may include higher yield relative to parental varieties, reduced expenses for insecticides relative to non-Bt cotton, and reduced cost of seeds relative to F1 hybrids. The efficacy of this strategy for managing resistance in other regions and against other pests remains to be determined.

Materials and Methods

Monitoring Pink Bollworm Resistance to Cry1Ac.

We conducted field sampling and diet bioassays to monitor pink bollworm resistance to Bt toxin Cry1Ac as described previously (18). Briefly, for 2011–2015, we sampled 60–700 (mean = 329) pink bollworm larvae per site from seven to 10 sites per year in the Yangtze River Valley (SI Appendix, Fig. S1; totals: 45 sets of larvae and 14,799 larvae sampled). Larvae from each site were mass-reared on an artificial diet in the laboratory, and their F1 progeny were tested on a diet containing concentrations of Cry1Ac ranging from 0 to 9 μg of Cry1Ac per milliliter of diet, which is a diagnostic concentration that kills all or nearly all susceptible larvae but few or no resistant larvae. Neonates were tested individually, with 24 neonates tested per concentration per replicate and three replicates for each concentration. After 21 d in darkness at 29 ± 1 °C, live fourth instars and pupae were scored as survivors. Each year, as an internal control, we used the same methods to test the susceptible strain QJ-S, which was started with insects collected from Qianjiang, Hubei province, in 2004 and reared in the laboratory without exposure to toxin. Results from 2005 to 2010 were reported previously (18) and are included here for comparison with the new data for 2011–2015.

Percentage of Cotton Seeds Containing Cry1Ac.

We used immunoassays to determine the percentage of cotton seeds producing Cry1Ac for cotton hybrids, Bt cotton varieties, and non-Bt cotton varieties (SI Appendix, Tables S3–S6). In each year from 2010 to 2015, we purchased 1,200–1,375 g of seeds of each of 20–30 cotton hybrids and varieties that were recommended that year by the local government. Each year, we randomly selected 100 seeds from each hybrid or variety and divided them into four groups with 25 seeds per replicate. We tested each seed individually for Cry1Ac in a total of 140 assays (total n = 14,000 seeds tested individually) of six varieties of Bt cotton, 10 varieties of non-Bt cotton, and 68 hybrids.

We used Cry1Ab/Cry1Ac immunoassay test strips (Aochuangjinbiao Biotech) according to the manufacturer’s instructions. Each replicate of 25 seeds had a separate positive and negative control. Each seed was stored in a 2-mL CryoPure tube (Sarstedt) with a 6-mm-diameter oil-free steel ball and 400 μL of leaching solution (Aochuangjinbiao Biotech). After centrifugation at 5,000 rpm for 18 seconds in a bead mill homogenizer (Precellys 24; Bertin Technologies), we added 600 more μL of leaching solution before analyzing each sample with a test strip. After 5 min, the appearance of a quality control line indicated the strip worked properly and the appearance of a second line indicated detection of Cry1Ac.

For each of the 68 hybrids, if the mean percentage of seeds producing Cry1Ac was >87.5% (mean of the expected 100% for F1 seeds and 75% for F2 seeds), the hybrid was classified as F1. If the mean percentage of seeds producing Cry1Ac was <87.5%, the hybrid was classified as F2. Based on this criterion, which aligns with a gap in the observed distribution (SI Appendix, Fig. S2), we classified 23 hybrids as F1 (n = 38 assays, mean = 96.3%, SE = 0.5%, range = 89–100%; SI Appendix, Table S5) and 45 hybrids as F2 (n = 82 assays, mean = 67.1%, SE = 1.4%, range = 39–84%; SI Appendix, Table S6).

Percentage of Cotton Planted to F1 and F2 Hybrid Cotton, Bt Cotton, and Non-Bt Cotton.

For each year from 2004 to 2015, we calculated the percentage of cotton hectares planted to hybrid cotton, Bt cotton, and non-Bt cotton using data from China’s Ministry of Agriculture for the six provinces in the Yangtze River Valley we studied (Anhui, Hubei, Hunan, Jiangsu, Jiangxi, and Sichuan). Because the Ministry’s data do not distinguish between F1 and F2 hybrids, we used two methods to calculate the proportion of hybrid cotton planted with each of these two hybrids. For 2004–2009, we used responses regarding planting of F1 and F2 hybrids from a telephone survey of 12 seed companies (SI Appendix, Table S8). For 2010–2015, we used the data from the percentage of seeds containing Cry1Ac for the 68 hybrids (SI Appendix, Tables S5 and S6) in conjunction with the data from Ministry of Agriculture on the hectares planted to each of the 68 hybrids (SI Appendix, Tables S9 and S10). For each year, we summed the hectares planted to the 23 F1 hybrids and, separately, to the 45 F2 hybrids. We calculated the proportion of hybrid cotton planted to F1 (or F2) hybrids in each year as the hectares planted to the F1 (or F2) hybrids divided by the total hectares planted to the F1 plus F2 hybrids. For 2010–2015, the mean percentage of total hectares planted to the 68 tested hybrids was 58.4% (SE = 6.3). For 1 y, 2010, we have the data on the percentage of hybrids consisting of F2 hybrids for both methods: 52% from the survey versus 77% based on analysis of Cry1Ac in seeds and hectares planted to each hybrid. This contrast suggests that the survey may underestimate the planting of F2 hybrids.

Percentage of Cotton Planted to Non-Bt Cotton.

To calculate the percentage of cotton hectares planted to non-Bt cotton plants in each year, we multiplied the proportion of non-Bt cotton seeds (1 minus the proportion containing Cry1Ac based on the assays described above) for each of the four types of cotton (F1 and F2 hybrid cotton, Bt cotton, and non-Bt cotton) by the percentage of cotton hectares planted of the corresponding type of cotton in that year (Fig. 2A), and then summed the four resulting percentages. For each of the 23 F1 hybrids and 45 F2 hybrids, we calculated the mean proportion of non-Bt seeds for 2010–2015. For each year from 2010 to 2015, we calculated the hectares of non-Bt cotton plants for each F1 hybrid by multiplying the mean proportion of non-Bt cotton seeds for that hybrid by the hectares planted to that hybrid in that year. We calculated the percentage of non-Bt cotton plants for all F1 hybrids for each year from 2010 to 2015 as 100-fold the total hectares of non-Bt cotton plants for all F1 hybrids divided by the total hectares of all F1 hybrids. We used the same approach to calculate the percentage of non-Bt cotton for all F2 hybrids for each year from 2010 to 2015. For 2004–2009, when seeds were not tested for Cry1Ac, we used the mean proportion of non-Bt cotton seeds based on all assays from 2010 to 2015 for F1 hybrids and separately for F2 hybrids. Relative to the extensive data on Cry1Ac in seeds for F1 hybrids (38 assays) and F2 hybrids (82 assays), which enabled calculation of the proportion of non-Bt cotton seeds separately for each year from 2010 to 2015, we had fewer assays for varieties of Bt cotton (nine assays) and non-Bt cotton (11 assays). Accordingly, we used the mean proportion of non-Bt cotton seeds from 2010 to 2015 to estimate the percentage of non-Bt cotton plants for all years for Bt cotton varieties (mean = 0.013, SE = 0.006, n = 9 assays of six varieties; SI Appendix, Table S3) and non-Bt cotton varieties (mean = 0.77, SE = 0.02, n = 11 assays of 10 varieties; SI Appendix, Table S4).

Computer Simulations.

To assess the potential effects of F2 hybrid cotton on the evolution of pink bollworm resistance to Bt cotton in the Yangtze River Valley of China, we used a previously described deterministic population genetic model of resistance to a Bt crop (26, 29) with some minor modifications. We chose this simple, previously described population genetic model for several reasons: to make clear the link between the model and the hypothesized increase in refuge percentage associated with planting F2 hybrids, to enable incorporation of realistic biological parameters for pink bollworm in the Yangtze River Valley, to examine projected outcomes of different assumptions about refuges and fitness values using the same basic model, and to make the modeling results readily verifiable by readers. Furthermore, as reported previously (26), the results from this model match precisely with the results of Gould (30) and are similar to results from much more complex models, such as the results of Gustafson et al. (31). The standard values for parameters and the alternative values used in sensitivity analyses are listed in SI Appendix, Table S12.

We assumed that resistance to Cry1Ac is controlled by two alleles at a single autosomal locus: a recessive allele r conferring resistance and an allele s conferring susceptibility. This assumption is based on previous analyses of pink bollworm resistance to Cry1Ac in many laboratory-selected strains from Arizona and in a strain derived from three field-selected populations in India (3235). Data reported here for a laboratory-selected resistant strain (AQ47) of pink bollworm derived from the Yangtze River Valley also demonstrate autosomal recessive inheritance of resistance to Cry1Ac in bolls of Bt cotton, F1 hybrid cotton, and F2 hybrid cotton (SI Appendix, Table S18). In addition, preliminary data indicate that in several strains of pink bollworm established recently from populations in the Yangtze River Valley, resistance to Cry1Ac is conferred by mutations in the same cadherin gene that is associated with recessive resistance to Cry1Ac in pink bollworm from Arizona and India (3236). We used the data from China reported here and the extensive, previously published empirical data on pink bollworm tested in greenhouse or field experiments to estimate the relative fitness of each genotype on Bt cotton and non-Bt cotton (25) (SI Appendix, Tables S12 and S18). To reduce the influence of differences between strains unrelated to resistance, we estimated fitness on non-Bt cotton using data only from experiments with resistant and susceptible strains that shared a common genetic background (i.e., the resistant strain was derived from the susceptible strain via laboratory selection).

We assumed random mating among all adults and projected the change in r allele frequency from one generation to the next as follows:

Δp=pq(p[WrrWrs]+q[WrsWss])/WM, [1]

where p and q are the frequencies of the r and s alleles, respectively; WM is the population mean fitness; and Wrr, Wrs, and Wss are the fitnesses of rr, rs, and ss, respectively. Fitness for each genotype was calculated as follows:

Wrr=BtWrrPBt+RefWrrPRef, [2]
Wrs=BtWrsPBt+RefWrsPRef, [3]
Wss=BtWssPBt+RefWssPRef, [4]

where PBt is the proportion of Bt cotton plants and PRef is the proportion of refuge (non-Bt cotton plants), and fitnesses for rr, rs, and ss, respectively, are BtWrr, BtWrs, and BtWss on Bt plants and RefWrr, RefWrs, and RefWss in refuges. Population mean fitness was calculated with the standard equation (37):

WM=p2Wrr+2pqWrs+q2Wss. [5]

We iterated Eq. 1 with a computer program to project changes in the r allele frequency over successive generations.

For each year, we simulated three generations, corresponding to the three annual generations of pink bollworm in the Yangtze River Valley (38). We modeled evolution of resistance from the end of the third generation of 2008 (end of the season) to the end of the third generation of 2015 (end of the season). We started with 2008 because 2008 is the first year that resistant individuals were detected in the Yangtze River Valley (18), which allowed us to use the monitoring data to set the percentage of resistant individuals (rr) at the end of 2008 as the observed value of 0.88% (18).

To make robust comparisons between predicted and observed outcomes, we used the means for parameters in the model (SI Appendix, Table S12) and for the observed percentage of resistant individuals in field populations based on survival at the diagnostic concentration of Cry1Ac (SI Appendix, Table S1). We also conducted sensitivity analyses with the model to assess the potential effects on resistance evolution of variation in the refuge percentage and variation in the fitness of rr on Bt cotton and non-Bt cotton, which yields variation in incomplete resistance and fitness cost (39), respectively. In sensitivity analyses, we modeled four assumptions about which cotton plants act as refuges: (i) hypothesis (a): all non-Bt cotton plants, including plants in fields of F2 hybrids, F1 hybrids, non-Bt cotton, and Bt cotton; (ii) hypothesis (b): only non-Bt cotton plants in fields of non-Bt cotton varieties; (iii) hypothesis (c): only non-Bt cotton plants in fields of F2 hybrid cotton; and (iv) hypothesis (d): non-Bt cotton plants in fields of F1 hybrids, non-Bt cotton varieties, and Bt cotton varieties (but not F2 hybrids) (Fig. 3 and SI Appendix, Figs. S3 and S4). Based on experimental results (25), the standard fitness of rr was 0.16 on Bt cotton and 0.46 on non-Bt cotton. In sensitivity analyses, we also multiplied the fitness of rr by 1.25 on Bt cotton (0.16 × 1.25 = 0.20) and on non-Bt cotton (0.46 × 1.25 = 0.58), which tends to make resistance evolve faster by reducing the negative effect of incomplete resistance on Bt cotton and by reducing the fitness cost on non-Bt cotton. In a complementary sensitivity analysis, we divided the fitness of rr by 1.25 on Bt cotton (0.16/1.25 = 0.13) and on non-Bt cotton (0.46/1.25 = 0.37), which tends to slow evolution of resistance by increasing the negative effect of incomplete resistance on Bt cotton and by increasing the fitness cost on non-Bt cotton.

Host Plant Experiments: Larval Feeding in the Laboratory.

We conducted larval feeding experiments to test a non-Bt variety of cotton (Simian3) and six other crops common in the Yangtze River Valley as potential pink bollworm host plants: okra, tomato, eggplant, cucumber, kidney bean, and apple. The following procedures were followed for all crops except apple. The crops were planted at the Wuhan experimental farm of the Hubei Academy of Agricultural Sciences in mid-April 2014. No insecticides were used throughout the growing season. In mid-August, for each crop, we collected >50 fruits (i.e., bolls, pods, other fruits), with one to three fruits collected per plant. We bought apples from a supermarket and soaked them in water for 2 h to reduce potential pesticide residues. For the fruits used in experiments, the approximate diameter was 2 cm for cotton, and it was 7 cm for tomato and apple; the approximate length was 12 cm for okra, 10 cm for eggplant, 25 cm for cucumber, and 15 cm for kidney beans. All fruits were rinsed with water in the laboratory, air-dried, and placed in plastic boxes (43 × 32 × 22 cm). The number of fruits per box was 34 for cotton and okra; 54 for tomato, eggplant, cucumber, and apple; and 108 for kidney bean. For each of the seven crops, we tested three replicates with one replicate per box.

We used a soft fine brush to put neonates (less than 1 h old) of the susceptible QJ-S strain of pink bollworm on the fruits. The number of neonates per fruit was five for cotton, okra, and kidney bean and 10 for the others. After inoculation, the boxes were closed and held at 29 ± 1 °C, 60 ± 10% relative humidity (RH), and 13 h light/11 h dark. One day later, we counted the number of entrance holes under a stereoscopic microscope. Five days after inoculation, we put soft paper into each box to facilitate pupation of larvae emerging from fruit. Larvae emerged and pupated over a period of more than 2 wk. We stopped tracking pupation after no larvae had emerged for 3 consecutive days from any fruit.

Host Plant Experiments: Larval Infestation in the Field.

We measured infestation of non-Bt cotton (Simian3), okra, and tomato by pink bollworm larvae in field plots at the Wuhan experimental farm of the Hubei Academy of Agricultural Sciences in 2014 and 2015. We planted the crops in a total of six fields, consisting of the following three pairs of crops in adjacent fields: cotton and okra, cotton and tomato, and okra and tomato. Within each pair, each crop was planted on 1.8 × 36 m and separated from the other crop in the pair by 0.5 m. The distance between paired fields was 2 m. Crops were planted in mid-April, and fruits were sampled six times: early, middle, and late August and September. Each sample consisted of 100 randomly collected fruits of each crop. The diameter was ∼2 cm for cotton bolls and 10 cm for tomatoes, and okra length was ∼12 cm. All samples were stored in plastic boxes (43 × 32 × 22 cm) covered with 100-mesh gauze, and held in the laboratory at 29 ± 1 °C, 60 ± 10% RH, and 13 h light/11 h dark. After 10 d, we checked the fruits for emergence holes, and then opened the fruits to check for live larvae. We calculated the total number of pink bollworm larvae per fruit as the number of emergence holes plus the number of live larvae per fruit.

Host Plant Experiments: Adult Males Caught in Pheromone Traps.

We used traps baited with pink bollworm female sex pheromone (Pherobio Technology Co.) to catch male pink bollworm moths at field sites in four counties (provinces): Anqing (Anhui), Anxiang (Hunan), Qianjiang (Hubei), and Wuxue (Hubei). Cotton, okra, and tomato were planted in each county, with roughly 50 km separating fields of the three different crops within a county (four counties × three field sites per county = total of 12 field sites). At each of the 12 sites, three fields of the same crop were planted with one trap in each field (total of three traps per site × 12 sites = 36 traps). Males caught in each trap were counted daily from June through September. For data analysis, the moths caught were totaled for each of the 12 traps per crop for each of the 4 mo of the field study.

Pink Bollworm Population Density.

To evaluate pink bollworm population density, we used the number of first-generation pink bollworm eggs per 100 cotton plants. A total of 150 cotton plants per county were sampled every 4 d in 12–24 counties of six provinces of the Yangtze River Valley (Anhui, Hubei, Hunan, Jiangsu, Jiangxi, and Sichuan) during June and July each year from 1995 to 2015. For each sampled plant, a thorough visual whole-plant survey was conducted to count the pink bollworm eggs, and each egg found was removed. Pink bollworm females do not distinguish between Bt and non-Bt cotton for oviposition (40), and the type of cotton plant (Bt or non-Bt) was not recorded.

Insecticide Sprays Targeting Pink Bollworm.

We obtained the data on insecticide sprays targeting pink bollworm from the National Agricultural Technology Extension and Service Center, which has a network of plant protection stations that report these data. The number of sprays targeting pink bollworm in each of 14–16 counties per year is a mean based on approximately 10 cotton fields per county.

Dominance of Resistance to Cotton Bolls Producing Cry1Ac in a Pink Bollworm Strain from the Yangtze River Valley.

We determined the dominance of pink bollworm resistance to Cry1Ac-producing bolls by testing four types of larvae derived from the Yangtze River Valley: resistant (AQ47 strain), susceptible (QJ-S strain), F1 progeny from resistant females × susceptible males, and F1 progeny from susceptible females × resistant males (additional details are provided in SI Appendix). We used survival on bolls containing Cry1Ac of the resistant strain, the susceptible strain, and their F1 progeny to calculate dominance (h) of resistance (which ranges from 0 for completely recessive to 1 for completely dominant) as described previously (41).

Data Analysis.

We analyzed diet bioassay data with probit regression (42) to determine LC50 values and their 95% fiducial limits, as well as slopes of the concentration-mortality lines. We calculated the resistance ratio as the LC50 for a strain divided by the LC50 for the susceptible QJ-S strain tested in the same year. We used Fisher’s exact test to determine if the proportion of populations with one or more survivors at the diagnostic concentration differed between 2008–2010 and 2011–2015. We used the Mann–Whitney U test to determine if percentage survival at the diagnostic concentration differed between 2008–2010 and 2011–2015.

Supplementary Material

Supplementary File

Acknowledgments

We thank Yidong Wu for his thoughtful comments. This work was supported by National Natural Science Foundation of China Grants 31210103921 and 31321004, China’s Key Project for Breeding Genetically Modified Organisms Grant 2016ZX08012-004, and US Department of Agriculture Biotechnology Risk Assessment Grant 2014-33522-22214.

Footnotes

Conflict of interest statement: B.E.T. is a coauthor of a patent on modified Bacillus thuringiensis toxins, “Suppression of Resistance in Insects to Bacillus thuringiensis Cry Toxins, Using Toxins that Do Not Require the Cadherin Receptor” (patent nos. CA2690188A1, CN101730712A, EP2184293A2, EP2184293A4, EP2184293B1, WO2008150150A2, and WO2008150150A3). Bayer CropScience, Dow AgroSciences, DuPont Pioneer, Monsanto, and Syngenta did not provide funding to support this work, but may be affected financially by publication of this paper and have funded other work by B.E.T. Y.C. has received funding from DuPont Pioneer, but DuPont Pioneer did not provide funding for this work. Syngenta, Dupont, Bayer Crop Science, FMC, and Gowan have funded other work by X.L.

This article is a PNAS Direct Submission.

This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.1073/pnas.1700396114/-/DCSupplemental.

References

  • 1.James C. 2015. Twentieth Anniversary (1996 to 2015) of the Global Commercialization of Biotech Crops and Biotech Crop Highlights in 2015, ISAAA Brief No. 51 (International Service for the Acquisition of Agri-Biotech Applications, Ithaca, NY)
  • 2.Downes S, Walsh T, Tay WT. Bt resistance in Australian insect pest species. Curr Opin Insect Sci. 2016;15:78–83. doi: 10.1016/j.cois.2016.04.002. [DOI] [PubMed] [Google Scholar]
  • 3.Hutchison WD, et al. Areawide suppression of European corn borer with Bt maize reaps savings to non-Bt maize growers. Science. 2010;330:222–225. doi: 10.1126/science.1190242. [DOI] [PubMed] [Google Scholar]
  • 4.Lu Y, Wu K, Jiang Y, Guo Y, Desneux N. Widespread adoption of Bt cotton and insecticide decrease promotes biocontrol services. Nature. 2012;487:362–365. doi: 10.1038/nature11153. [DOI] [PubMed] [Google Scholar]
  • 5.National Academies of Sciences, Engineering, and Medicine . Genetically Engineered Crops: Experiences and Prospects. National Academies Press; Washington, DC: 2016. [PubMed] [Google Scholar]
  • 6.Tabashnik BE, et al. Suppressing resistance to Bt cotton with sterile insect releases. Nat Biotechnol. 2010;28:1304–1307. doi: 10.1038/nbt.1704. [DOI] [PubMed] [Google Scholar]
  • 7.Wu KM, Lu YH, Feng HQ, Jiang YY, Zhao JZ. Suppression of cotton bollworm in multiple crops in China in areas with Bt toxin-containing cotton. Science. 2008;321:1676–1678. doi: 10.1126/science.1160550. [DOI] [PubMed] [Google Scholar]
  • 8.Dhurua S, Gujar GT. Field-evolved resistance to Bt toxin Cry1Ac in the pink bollworm, Pectinophora gossypiella (Saunders) (Lepidoptera: Gelechiidae), from India. Pest Manag Sci. 2011;67:898–903. doi: 10.1002/ps.2127. [DOI] [PubMed] [Google Scholar]
  • 9.Tabashnik BE, Brévault T, Carrière Y. Insect resistance to Bt crops: Lessons from the first billion acres. Nat Biotechnol. 2013;31:510–521. doi: 10.1038/nbt.2597. [DOI] [PubMed] [Google Scholar]
  • 10.Gassmann AJ, et al. Field-evolved resistance by western corn rootworm to multiple Bacillus thuringiensis toxins in transgenic maize. Proc Natl Acad Sci USA. 2014;111:5141–5146. doi: 10.1073/pnas.1317179111. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Tabashnik BE. Tips for battling billion-dollar beetles. Science. 2016;354:552–553. doi: 10.1126/science.aag101. [DOI] [PubMed] [Google Scholar]
  • 12.Jin L, et al. Large-scale test of the natural refuge strategy for delaying insect resistance to transgenic Bt crops. Nat Biotechnol. 2015;33:169–174. doi: 10.1038/nbt.3100. [DOI] [PubMed] [Google Scholar]
  • 13.Burkness EC, Cira TM, Moser SE, Hutchison WD. Bt maize seed mixtures for Helicoverpa zea (Lepidoptera: Noctuidae): Larval movement, development, and survival on non-transgenic maize. J Econ Entomol. 2015;108:2761–2769. doi: 10.1093/jee/tov253. [DOI] [PubMed] [Google Scholar]
  • 14.Brévault T, Tabashnik BE, Carrière Y. A seed mixture increases dominance of resistance to Bt cotton in Helicoverpa zea. Sci Rep. 2015;5:9807. doi: 10.1038/srep09807. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Carrière Y, Fabrick JA, Tabashnik BE. Can pyramids and seed mixtures delay resistance to Bt crops? Trends Biotechnol. 2016;34:291–302. doi: 10.1016/j.tibtech.2015.12.011. [DOI] [PubMed] [Google Scholar]
  • 16.National Corn Growers Association 2016 Adoption of refuge in a bag simplifies grower compliance. Available at www.ncga.com/news-and-resources/news-stories/article/2016/02/adoption-of-refuge-in-a-bag-products-simplifies-grower-irm-compliance. Accessed December 20, 2016.
  • 17.Onstad DW, et al. Seeds of change: Corn seed mixtures for resistance management and integrated pest management. J Econ Entomol. 2011;104:343–352. doi: 10.1603/ec10388. [DOI] [PubMed] [Google Scholar]
  • 18.Wan P, et al. Increased frequency of pink bollworm resistance to Bt toxin Cry1Ac in China. PLoS One. 2012;7:e29975. doi: 10.1371/journal.pone.0029975. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Centre for Agriculture and Biosciences International (CABI) 2016 Invasive Species Compendium: Datasheet report for Pectinophora gossypiella (pink bollworm). Available at www.cabi.org/isc/datasheet/39417. Accessed December 7, 2016.
  • 20.Tabashnik BE, Croft BA. Managing pesticide resistance in crop-arthropod complexes: Interactions between biological and operational factors. Environ Entomol. 1982;11:1137–1144. [Google Scholar]
  • 21.Dong H, et al. Development of hybrid Bt cotton in China—A successful integration of transgenic technology and conventional techniques. Curr Sci. 2004;86:778–782. [Google Scholar]
  • 22.Feng G, et al. Comparisons of photosynthetic characteristics in relation to lint yield among F1 hybrids, their F2 descendants and parental lines of cotton. J Integr Agric. 2014;13:1909–1920. [Google Scholar]
  • 23.Zhang BH, et al. Inheritance and segregation of exogenous genes in transgenic cotton. J Genet. 2000;79:71–75. [Google Scholar]
  • 24.Heuberger S, et al. Effects of refuge contamination by transgenes on Bt resistance in pink bollworm (Lepidoptera: Gelechiidae) J Econ Entomol. 2008;101:504–514. doi: 10.1603/0022-0493(2008)101[504:eorcbt]2.0.co;2. [DOI] [PubMed] [Google Scholar]
  • 25.Tabashnik BE, Dennehy TJ, Carrière Y. Delayed resistance to transgenic cotton in pink bollworm. Proc Natl Acad Sci USA. 2005;102:15389–15393. doi: 10.1073/pnas.0507857102. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Tabashnik BE, Gassmann AJ, Crowder DW, Carrière Y. Insect resistance to Bt crops: Evidence versus theory. Nat Biotechnol. 2008;26:199–202. doi: 10.1038/nbt1382. [DOI] [PubMed] [Google Scholar]
  • 27.Ariela N, et al. 2011 Can tomato be a potential host plant for pink bollworm? World Cotton Research Conference-5, eds Kranthi KR, et al. (Excel India, New Delhi), pp 258–260. Available at www.icac.org/meetings/wcrc/wcrc5/Proceedings.pdf. Accessed July 31, 2016.
  • 28.Siegfried BD, et al. Ten years of Bt resistance monitoring in the European corn borer: What we know, what we don’t know, and what we can do better. Am Entomol. 2007;53:208–214. [Google Scholar]
  • 29.Tabashnik BE, Crowder DW, Carrière Y. Modeling evolution of insect resistance to genetically modified crops. Protocol Exchange. 2008 doi: 10.1038/nprot.2008.125. [DOI] [Google Scholar]
  • 30.Gould F. Sustainability of transgenic insecticidal cultivars: Integrating pest genetics and ecology. Annu Rev Entomol. 1998;43:701–726. doi: 10.1146/annurev.ento.43.1.701. [DOI] [PubMed] [Google Scholar]
  • 31.Gustafson DI, Head GP, Caprio MA. Modeling the impact of alternative hosts on Helicoverpa zea adaptation to Bollgard cotton. J Econ Entomol. 2006;99:2116–2124. doi: 10.1603/0022-0493-99.6.2116. [DOI] [PubMed] [Google Scholar]
  • 32.Morin S, et al. Three cadherin alleles associated with resistance to Bacillus thuringiensis in pink bollworm. Proc Natl Acad Sci USA. 2003;100:5004–5009. doi: 10.1073/pnas.0831036100. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Tabashnik BE, et al. Association between resistance to Bt cotton and cadherin genotype in pink bollworm. J Econ Entomol. 2005;98:635–644. doi: 10.1603/0022-0493-98.3.635. [DOI] [PubMed] [Google Scholar]
  • 34.Fabrick JA, Tabashnik BE. Similar genetic basis of resistance to Bt toxin Cry1Ac in boll-selected and diet-selected strains of pink bollworm. PLoS One. 2012;7:e35658. doi: 10.1371/journal.pone.0035658. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Nair R, Kamath SP, Mohan KS, Head G, Sumerford DV. Inheritance of field-relevant resistance to the Bacillus thuringiensis protein Cry1Ac in Pectinophora gossypiella (Lepidoptera: Gelechiidae) collected from India. Pest Manag Sci. 2016;72:558–565. doi: 10.1002/ps.4023. [DOI] [PubMed] [Google Scholar]
  • 36.Fabrick JA, et al. Alternative splicing and highly variable cadherin transcripts associated with field-evolved resistance of pink bollworm to bt cotton in India. PLoS One. 2014;9:e97900. doi: 10.1371/journal.pone.0097900. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Hartl DL, Clark AG. Principles of Population Genetics. 2nd Ed Sinauer; Sunderland, MA: 1989. [Google Scholar]
  • 38.Wan P, et al. Seasonal pattern of infestation by pink bollworm Pectinophora gossypiella (Saunders) in field plots of Bt transgenic cotton in the Yangtze River Valley of China. Crop Prot. 2004;23:463–467. [Google Scholar]
  • 39.Carrière Y, Tabashnik BE. Reversing insect adaptation to transgenic insecticidal plants. Proc Biol Sci. 2001;268:1475–1480. doi: 10.1098/rspb.2001.1689. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Liu YB, et al. Oviposition on and mining in bolls of Bt and non-Bt cotton by resistant and susceptible pink bollworm (Lepidoptera: Gelechiidae) J Econ Entomol. 2002;95:143–148. doi: 10.1603/0022-0493-95.1.143. [DOI] [PubMed] [Google Scholar]
  • 41.Liu Y, Tabashnik BE. Inheritance of resistance to Bacillus thuringiensis toxin Cry1C in the diamondback moth. Appl Environ Microbiol. 1997;63:2218–2223. doi: 10.1128/aem.63.6.2218-2223.1997. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Russell RM, et al. POLO: A new computer program for probit analysis. Bull Entomol Soc Am. 1977;23:209–213. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary File

Articles from Proceedings of the National Academy of Sciences of the United States of America are provided here courtesy of National Academy of Sciences

RESOURCES