Skip to main content
PMC home page
Journal of Insect Science logoLink to Journal of Insect Science
. 2022 Feb 7;22(1):17. doi: 10.1093/jisesa/ieab096

Impact of Honey Bee Migratory Management on Pathogen Loads and Immune Gene Expression is Affected by Complex Interactions With Environment, Worker Life History, and Season

Michael Simone-Finstrom 1,✉, Micheline K Strand 2, David R Tarpy 3,4,✉, Olav Rueppell 6,7
Editor: Margarita Lopez-Uribe
PMCID: PMC8825759  PMID: 35137136

Abstract

The effects of honey bee management, such as intensive migratory beekeeping, are part of the ongoing debate concerning causes of colony health problems. Even though comparisons of disease and pathogen loads among differently managed colonies indicate some effects, the direct impact of migratory practices on honey bee pathogens is poorly understood. To test long- and short-term impacts of managed migration on pathogen loads and immunity, experimental honey bee colonies were maintained with or without migratory movement. Individuals that experienced migration as juveniles (e.g., larval and pupal development), as adults, or both were compared to control colonies that remained stationary and therefore did not experience migratory relocation. Samples at different ages and life-history stages (hive bees or foragers), taken at the beginning and end of the active season, were analyzed for pathogen loads and physiological markers of health. Bees exposed to migratory management during adulthood had increased levels of the AKI virus complex (Acute bee paralysis, Kashmir bee, and Israeli acute bee paralysis viruses) and decreased levels of antiviral gene expression (dicer-like). However, those in stationary management as adults had elevated gut parasites (i.e. trypanosomes). Effects of environment during juvenile development were more complex and interacted with life-history stage and season. Age at collection, life-history stage, and season all influenced numerous factors from viral load to immune gene expression. Although the factors that we examined are not independent, the results illuminate potential factors in both migratory and nonmigratory beekeeping that are likely to contribute to colony stress, and also indicate potential mitigation measures.

Keywords: apiculture, deformed wing virus, acute bee paralysis virus, gene expression, migratory management

The health of the western honey bee (Apis mellifera Linnaeus [Hymenoptera: Apidae]) remains a major concern for the sustainability of apiculture and the associated pollination of agricultural and wild plants that rely upon them (Klein et al. 2007, Kulhanek et al. 2017, Gray et al. 2020). A number of potentially interacting factors may contribute to compromising honey bee health. These include pesticides, pathogens and pests, and poor nutrition (Vanbergen 2013, Goulson et al. 2015) with potentially synergistic effects among stressors (Pettis et al. 2012, Dolezal et al. 2019). Several of these stressors depend on the management and environment of honey bee colonies, and the apparent heterogeneity of causes of honey bee health declines may be related to the diversity of external conditions that honey bees experience.

The lives of bees in migratory versus stationary management differ significantly, which has implications for colony health (Simone-Finstrom et al. 2016, Traynor et al. 2016a, Steinhauer et al. 2021). Commercially managed honey bee colonies are transported throughout the country to fulfill pollination service contracts, being moved among different crops as needed in addition to different areas for honey production. Stationary colonies simply are maintained in a single location. Although differences between stationary and migratory hives are often reported in surveys of colony losses (Seitz et al. 2016) and pathogens (Traynor et al. 2016b, Bartlett et al. 2021), management practices, operation size, and landscape are not independent variables making it difficult to interpret these differences. Specifically, migratory practices lead to large-scale changes in the environment that bees experience, most notably exposure to differential forage quality, reduced forage diversity, and pesticides, which can impact colony health (Alaux et al. 2017, Colwell et al. 2017, Alburaki et al. 2018, Van Esch et al. 2020, St Clair et al. 2020). Similarly, migratory management is practiced by large-scale beekeepers that employ different management practices than smaller-scale, mostly stationary beekeepers (Welch et al. 2009, Zhu et al. 2014, Traynor et al. 2016b, Goodrich et al. 2019, Underwood et al. 2019, Bartlett et al. 2021). Consequently, the direct effects of migratory management on honey bee health in its own right remain relatively sparsely investigated and poorly understood (Ahn et al. 2012, Simone-Finstrom et al. 2016, Alger et al. 2018, Jara et al. 2021).

Previously, we reported on direct effects of different migration regimes on the level of oxidative stress and life expectancy of honey bee workers, with interactions between developmental and adult stages (Simone-Finstrom et al. 2016). In this prior work, we reported nuanced effects on the accumulation of oxidative stress in bees that were exposed to migratory management either during juvenile development or as adults. Importantly, migratory treatment effects differed, depending on whether migration was experienced during development or adulthood. Here, we expand this paradigm by using samples from the same experiments to test the hypothesis that immunity and pathogen levels are also impacted by migratory beekeeping. Pathogens in migratory honey bees are of particular concern because honey bees can share many pathogenic agents with nonmanaged pollinators (McMahon et al. 2015, Graystock et al. 2015, Manley et al. 2019), potentially facilitating long-range dispersal of pollinator pathogens. We assess the impact of migratory versus stationary conditions during the juvenile and adult phases of honey bee workers (hive bees and foragers) on five viral targets, four other pathogens, and nine stress response genes at two time points during a single season. We predicted that the environment that individuals experience as adults will have the most impact on pathogen loads, but that both rearing and adult experience would influence the health and stress response genes.

Material and Methods

To establish homogeneous experimental units, colonies were set up from randomized mixtures of unselected worker bees, brood, food stores, and sister queens that were allowed to mate naturally at the Lake Wheeler Farms Bee Research Facility at North Carolina State University, Raleigh, NC (Simone-Finstrom et al. 2016). Throughout the experiment, these colonies were maintained in standard Langstroth hives according to best management practices without pest control measures. Five colonies were kept stationary in an apiary bordered by a forested park and agricultural field, while five other colonies were subjected to a migratory treatment, moving colonies over more than 30 miles every 21 d (the developmental period of a worker honey bee) among three agricultural sites in North Carolina (see Simone-Finstrom et al. 2016 for site details). In a paired design, newly emerging workers from one stationary and migratory colony (n > 300 bees) were paint-marked with a colony-specific color and evenly introduced into their own colony and the paired colony of different treatment, resulting in four experimental groups based on their juvenile and adult environment: “migratory treatment during juvenile development/migratory treatment during adult life” (MM), “migratory/stationary” (MS), “stationary/migratory” (SM), and “stationary/stationary” (SS). After 14 and 28 d, marked workers were retrieved from brood frames (as hive bees) or from the outside of colony entrances (as foragers), representing young and old ages, respectively. Both hive bees and foragers were collected at both age groups since chronological age, behavioral task, and pathogen dynamics could all influence one another. To account for seasonal effects, two replicate samples were collected, one after the first migration in May (early season) and the second at the end of July (late season). Collected samples (Fig. 1) were immediately frozen and stored at –80 °C until further processing.

Fig. 1.

Fig. 1.

Open in a new tab

Experimental design of a systematic study of migratory beekeeping effects on pathogen loads. After developing either in a migratory or stationary hive, adults were marked upon emergence and either reintroduced into their native hive or transferred to a corresponding hive of the opposite treatment, creating four principal experimental groups: “SS” = stationary juvenile and adult environment, “MS” = migratory juvenile and stationary adult environment, ‘SM’ stationary juvenile and migratory adult environment, and “MM” = migratory juvenile and adult environment. After 14 and 28 d, respectively, hive bees and foragers were individually identified and sampled from each of the four treatment groups. This protocol was performed at the beginning and the end of the active season, resulting in 32 (2 seasons × 2 juvenile treatments × 2 adult treatments × 2 ages × 2 behavioral states) groups. For each group, five individuals were pooled into one hive replicate, resulting in 160 samples representing 800 bees.

Pools of five abdomens per each colony and treatment group combination (i.e. juvenile environment, adult environment, age, behavioral state, and season; see Fig. 1) were homogenized prior to RNA extraction, cDNA synthesis, and qPCR following established practices (López-Uribe et al. 2017, De Guzman et al. 2017, Simone-Finstrom et al. 2018). Briefly, RNA was extracted using the Maxwell system with the LEV simplyRNA tissue kit (Promega) with quantity and purity assessed spectrophotometrically (Nanodrop, ThermoFisher, USA). cDNA template was generated from 2 µg of total RNA using the QuantiTect Reverse Transcription Kit (Qiagen Inc.) following the manufacturer’s protocols. qPCR reactions were conducted in triplicate using a CFX96 Real-Time PCR (BioRad, Inc.). Amplification was performed in 10 μl volumes using PowerUP SYBR Green Master Mix (Applied Biosystems). The cDNAs were assessed by qPCR for the following viral targets: Deformed Wing Virus (DWV), Lake Sinai Virus (LSV), Black Queen Cell Virus (BQCV), and the AKI virus complex (Acute Bee Paralysis Virus + Israeli Acute Paralysis Virus [IAPV] + Kashmir Bee Virus [KBV]). IAPV and KBV were also analyzed separately to specify AKI results. In addition, the cDNA was used to quantify the pathogens Ascosphaera apis (fungal agent of Chalkbrood [CB]), Melissococcus plutonius (bacterial agent of European foulbrood [EFB]), the gut parasite Nosema (Nosema referring to its common name as taxonomically its genus is now Vairimorpha; Tokarev et al. 2020), trypanosomes (Lotmaria passim), according to previous studies (e.g. Dainat et al. 2012, Glenny et al. 2017), although a better understanding of the validity of this approach is needed. Finally, we assessed the expression of nine honey bee genes related to immunity and health (Supp. Table S1 [online only]), using β-actin as a reference, via qPCR. Honey bee genes of interest were chosen in order to cover those responding to a large swathe of different stressors (López-Uribe et al. 2020). Triplicates were averaged for relative quantification according to ∆Ct = Ctreference—Cttarget so that a high ∆Ct intuitively indicates a high target expression. Not all combinations of the two ages, two behavioral states, four treatment groups, and two seasons were sampled and processed successfully, so that only 115 of the 160 samples were included in the final analyses.

Correlations of relative abundances were assessed among all qPCR targets, using a FDR < 0.05 significance threshold. For the samples that overlapped with our previous study (Simone-Finstrom et al. 2016), we also calculated correlation coefficients between average transcript levels and oxidative stress levels across experimental groups but did not find any significant relationships.

For statistical analysis, first, a multivariate principal component (PC) analysis was conducted in JMP 12.2.0 to provide a descriptive analysis of the influence of juvenile treatment, adult treatment, age, behavioral state, and season on gene targets in the following three functional groups: viral loads, other pathogens (CB, EFB, Nosema and trypanosomes), and bee health and stress response genes. For PC1 and PC2 of each group, the most likely additive model of juvenile treatment, adult treatment, age, behavioral state, and season was selected in “R” (v3.5.1) using a backward linear model procedure to identify potential main effects (“lm” combined with “stepAIC”). When a main effect was identified, interactions among variables were examined with respect to juvenile treatment or adult treatment. Based on the results of the principal component analysis (PCA), we then used the same model selection procedure for each individual target to better elucidate the factors influencing particular targets of interest. Consistent with the PCA, any significant main effect was specifically tested for interactions with juvenile treatment or adult treatment. Model selection was followed by Tukey’s HSD posthoc analyses to determine differences between specific groups. Bonferroni correction was used to adjust α to conservatively account for multiple testing in each functional group (viral load: corrected α = 0.0125; other pathogens: corrected α = 0.0125; health response genes: corrected α = 0.0055). This enabled us to focus on the main question (i.e., the influence of rearing and adult environments on pathogen exposure and immune expression) while still identifying other potential effects and interactions.

Results

The ten colonies included in this study all survived the entire experimental period, and basic colony variables (e.g., colony size, brood, pollen stores, and Varroa infestation) did not significantly differ among treatment groups (Simone-Finstrom et al. 2016). Recovery of marked foragers for the late-season replicate was difficult in several hives, reducing our sample size for these experimental groups. Across all samples, most viruses were positively correlated with each other (Fig. 2). Viruses were also mostly positively correlated with European foulbrood (Melissococcus plutonius: EFB) and Chalkbrood (Ascosphaera apis: CB) but not with trypanosomes or Nosema. The quantities of various viruses had minimal relationship to gene expression levels: Deformed wing virus (DWV) with a positive relation to vitellogenin (VG) and defensin1 and a negative relation to lysozyme2 (Lys2); BQCV with a positive relation to HSP90; Lake Sinai virus (LSV) with a negative relation to VG; and Kashmir bee virus (KBV) and Israeli acute paralysis virus (IAPV) with positive relations to VG (Fig. 2). VG and Lys2 expression also exhibited significant correlations with the eukaryotic pathogens, while most of the other surveyed genes were only positively correlated with each other (Fig. 2).

Fig. 2.

Fig. 2.

Open in a new tab

Correlations among pathogens and expression levels of health-related genes across all samples. Correlation coefficients are indicated by color and size of the squares, significant (based on FDR<0.05) correlations are outlined in black. DWV, Deformed wing virus; BQCV, Black queen cell virus; LSV, Lake Sinai virus; AKI, AKI virus complex; IAPV, Israeli acute paralysis virus; KBV, Kashmir bee virus; EFB, European foulbrood (Melissococcus plutonius); CB, Chalkbrood (Ascosphaera apis); Nosema, Vairimorpha apis or V. ceranae (formerly Nosema spp.); Trypano, Lotmaria passim; Cyp9, Cyp9Q3 cytochrome P450; HSP90, heat shock protein 90; Hymen, hymenoptaecin; PGPR, peptidoglycan recognition protein; VG, vitellogenin; Lys2, lysozyme2.

Viral and Pathogen Levels

The first two PCs extracted from the virus quantification indicated a strong impact of the AKI complex on overall variation (PC1) and a moderate influence of DWV, BQCV, and LSV on PC2 (refer to Supp. Table S2 [online only] for eigenvectors and loading values). Based on model selection, adult environment and behavioral state were identified as significant factors influencing PC1 (adult environment: F(1,112) = 4.58, P = 0.034; behavioral state: (F(1,112) =5.85, P = 0.017; see Fig. 3A). While season was the only significant factor for PC2 F(1,112) = 86.75, P < 0.001).

Fig. 3.

Fig. 3.

Open in a new tab

The major descriptors of the variation for the principal components analysis for A) PC1 of viral load data, B) PC2 for nonviral pathogens, and C) PC1 for the health and stress response genes. For viral load (A) and health response genes (C), the adult environment (left side) and behavioral state (right side) were highly influential on their own, while the nonviral pathogens (C) were influenced by a three-way interaction between adult environment, behavioral state and season, largely driven by the hive bees. Groups in each panel were significantly different at p < 0.05 as indicated by the letters above each bar.

Examining each virus separately clarified the relationships indicated by the PCA. Viral infection levels were affected by migratory treatment, both alone and via interactions with other variables. Migration during adulthood increased AKI loads relative to the stationary treatment overall (F(1,112) = 7.4, P = 0.007). Independent of treatment, age, or season, hive bees tended to have higher AKI levels than foragers (F(1,112) = 6.3, P = 0.014), but this difference was nonsignificant based on the Bonferroni-adjusted α. When independently analyzing IAPV and KBV, the same differences were found; migration during adulthood significantly increased KBV (F(1,112) = 8.30, P = 0.005) and numerically, but nonsignificantly increased IAPV (F(1,112) = 5.71, P = 0.019), and hive bees had higher loads than foragers (IAPV: F(1,112) = 7.10, P = 0.009; KBV: F(1,112) = 7.34, P = 0.008). An influence of migration during adulthood on viral levels was also suggested for BQCV by an interaction between adult treatment and season (F(1,112) = 8.78, P = 0.0037), which was due to a higher BQCV in the migratory than the stationary treatment early in the season which reversed later in the year (Fig. 4). DWV increased over the experimental season (F(1,112) = 64.46, P < 0.0001), which also interacted separately with migratory treatment during development (F(1,112) = 9.10, P = 0.003). The seasonal increase in DWV was stronger in bees from a migratory juvenile environment than from a stationary one. LSV levels were only influenced by season, with lower levels seen later in the season (F(1,112) = 11.36, P = 0.001).

Fig. 4.

Fig. 4.

Open in a new tab

Effects on hive management on virus levels based on relative quantification according to ∆Ct. A) Levels of the AKI virus complex were increased by migratory conditions during adulthood. (P < 0.0125). B) Migration experienced during adulthood also influenced Black queen cell virus, which was most abundant in samples from the migratory treatment early in the season and least abundant in migratory samples late in the season, with stationary samples intermediate and exhibiting a less pronounced seasonal decrease. C) Deformed wing virus was influenced by developmental conditions; samples that developed in migratory hives exhibited the lowest levels early in the season, but the highest late in the season. Individuals that experienced stationary conditions during development were intermediate and exhibited a less pronounced seasonal increase. Different letters indicate significant differences when interactions occurred (P < 0.0125).

The PCA for the other pathogens also indicated multiple influencing factors in the data. PC1 correlated with all four pathogens and was only significantly affected by season (F(1,112) = 4.58, P = 0.034). PC2 represented mostly EFB and was impacted by a three-way interaction of adult environment, behavioral state, and season (F(1,112) = 4.75, P = 0.031; Fig. 3B), typifying the complex nature of effects of migratory management on these bee health metrics. In order to elucidate these factors, again each pathogen was analyzed separately. For trypanosomes, the treatment during adulthood and season interacted (F(1,112) = 31.42, P < 0.0001; Fig. 5); while bees from the adult migratory treatment had significantly higher levels early in the season versus later in the season, the corresponding samples from the stationary treatment experienced a less-pronounced seasonal change with intermediate levels. Overall, Nosema levels decreased from early to late season (F(1,112) = 34.82, P < 0.0001) and were numerically (but nonsignificantly) higher in bees experiencing the stationary treatment as adults relative to those from the migratory treatment (F(1,112) = 5.43, P = 0.021). No experimental treatment effects were found in the larval pathogens, but CB levels were higher in late season relative to early season (F(1,112) = 21.73, P < 0.0001) and EFB was found in higher levels in hive bees versus foragers (F(1,112) = 28.25, P < 0.0001).

Fig. 5.

Fig. 5.

Open in a new tab

Effects on hive management on nonviral pathogens based on relative quantification according to ∆Ct. A) Overall, Nosema did not significantly differ among the different treatment groups. B) Experimental treatment alone did not affect overall trypanosome levels. However, workers that were in migratory hives as adults showed high levels early in the season and low levels later, while this decrease in workers under stationary conditions was much smaller. Different letters indicate significant differences (P < 0.0125).

Immune and Health-Related Gene Expression

PCA indicated the major descriptors of variation for the immune and health-related gene targets were adult environment, behavioral state, and season. A main effect of adult environment (F(1,112) = 5.38, P = 0.022; Fig. 3C) along with an interaction between behavioral state and season (F(1,112) = 15.02, P = 0.0002) was significant for PC1, which represented numerous gene targets in similar fashion with the exception of vitellogenin (Vg). PC2 also represented numerous genes to varying degrees and was largely influenced by behavioral state (F(1,112) = 18.01, P < 0.0001).

Based on individual analysis, expression of target genes was influenced by migratory treatment (during juvenile and adult stages) and differed between in-hive workers and outside foragers. Two genes exhibited overall higher expression in bees that were in stationary hives as adults as compared to bees from the adult migratory environment (PGRP: F(1,112) = 9.15, P = 0.003; dicer-like: F(1,112) = 11.97, P = 0.0008; Fig. 6). The antimicrobial peptides defensin1 and hymenoptaecin were influenced by behavioral state, with both being more expressed in foragers than hive bees (defensin1: F(1,112) = 13.54, P = 0.0004; hymenoptaecin: F(1,112) = 14.70, P = 0.0002). Lysozyme2 was only influenced by season, exhibiting a decline over time (F(1,112) = 12.97, P = 0.0005). Abaecin, HSP90, and CypQ93 showed no significant differences in response to any treatment.

Fig. 6.

Fig. 6.

Open in a new tab

Management effects during adulthood on worker gene expression presented based on relative quantification according to ∆Ct. Multiple genes exhibited significantly increased expression as a result of the workers experiencing stationary compared to migratory conditions. A) PGRP showed low variability but significantly higher expression overall in adults that experienced stationary conditions. Workers that experienced a stationary adult environment also had higher expression of dicer-like (B), than workers from the migratory adult treatment but not HSP90 (C), or Cyp9Q3 (D). Within each panel, different letters indicate significant differences (P < 0.0055).

Vitellogenin expression patterns were complex, characterized by a significant 5-way interaction effect among all independent variables. To simplify, VG expression was thus analyzed separately for each season, based on previous studies that already documented seasonal effects on Vg expression (Amdam and Omholt 2002, Kunc et al. 2019). For the early-season samples, the adult treatment influenced Vg expression, interacting with age and behavioral state (F(1,58) = 8.68, P = 0.0051). In the stationary treatment group, old foragers had higher Vg expression than old hive bees and young foragers, whereas young bees in the stationary treatment group and all bees in the migratory treatment group exhibited the expected decrease in Vg expression in foragers relative to hive bees (Fig. 7). Later in the season, behavioral state was the only significant factor (F(1,54) = 11.73, P < 0.0001) with hive bees expressing higher Vg than foragers across all groups.

Fig. 7.

Fig. 7.

Open in a new tab

Variation in vitellogenin expression. VG expression was highly dynamic and treatment effects differed between seasons. A) Early in the season, old foragers exhibited the highest VG levels, but only when experiencing a stationary environment as adults, while all other groups followed the predicted pattern of lower VG levels in foragers than in hive bees. Different letters indicate significant differences (P < 0.0055). B) Later in the season, all groups demonstrated the same, predicted decline in VG expression in foragers compared to hive bees, although the magnitude of this effect varied slightly. Significance (P < 0.0055) is noted at the top.

Discussion

Despite its importance for pollinator health, the consequences of migratory beekeeping are still insufficiently studied. Here, we attempted to elucidate the direct effects of migratory hive movement on pathogen levels and the expression of honey bee genes that play roles in immunity and other physiological defenses. It was beyond the scope of this work to establish separate experiments to examine every variable associated with migratory beekeeping. Instead, we employed a complex experimental design to disentangle the effects of a migratory environment during development and adulthood on measures at different ages and behavioral phenotypes, while also accounting for potential seasonal variation. The results demonstrated that the effects of these five factors (juvenile treatment, adult treatment, age, behavioral state, and season) were variable and often interacted, highlighting the complexity and heterogeneity of honey bee health problems.

Similar to our previous findings on life expectancy and oxidative stress levels (Simone-Finstrom et al. 2016), there were no simple or general positive or negative effects of migratory management on pathogen levels or gene expression. Effects of migratory practices varied with season, age, and behavioral/life-history stage. The investigated pathogens are transmittable among bees, particularly when they were residing in the same hive. Thus, the transfer of samples as newly emerged bees may have negated migratory treatment effects on resident pathogens due to the potential introduction of new pathogens across pairs of colonies. Therefore, our results likely represent an underestimation of experimental effects. Conversely, any differences among experimental groups that were sampled as adults from the same hive occurred despite the common environment, indicating the persistence of developmental influences on individuals (Scofield and Mattila 2015), as was also shown previously with regard to oxidative stress and lifespan in these same samples (Simone-Finstrom et al. 2016). In the case of infectious pathogens, this persistence at the individual level is qualitatively different than previous findings of persistent effects of management at the colony level (Jara et al. 2021, Bartlett et al. 2021).

Our simultaneous assessment of pathogen levels and expression levels of relevant genes added an additional dimension to our data. Beyond correlations among viruses and some other pathogens (Cornman et al. 2012), as well as among immune and other health-related genes, we identified some correlations among pathogen levels and expression levels of a few genes, most notably vitellogenin (Vg). However, it is impossible to infer causality from these correlations among pathogen and gene expression data. For example, immune genes may be upregulated in response to an infection (Doublet et al. 2017), a pathogen can weaken immunity (Di Prisco et al. 2016) creating a negative correlation, or the level of a pathogen could be increased in response to an immune system that is downregulated due to other factors (Dolezal and Toth 2018). Several unexpected positive correlations between Vg and pathogens may be due to higher Vg expression in hive bees that nurse brood and are simultaneously more exposed to brood pathogens, such as chalkbrood or European foulbrood (Naug and Camazine 2002). Thus, a careful and nuanced interpretation of our findings is required.

Nevertheless, some broad trends were apparent (Fig. 8). Migratory versus stationary treatment affected most pathogens and the expression level of many genes, although some of these effects were interacting with other variables. Direct effects of the adult environment were more common than effects of developmental environment. The migratory stress experienced as an adult may thus trigger physiological changes, such as lower expression of some immune and stress resistance genes that enabled successful infections (e.g., increasing IAPV and KBV). Alternatively, the variable environment of the migratory colonies may have increased exposure to pathogens, leading to infection and subsequent consequences of disease on gene expression. It is impossible to determine causality but the increase in the relatively rare viruses of the AKI complex but not the common DWV or BQCV in migratory colonies supports the latter interpretation.

Fig. 8.

Fig. 8.

Open in a new tab

Overview of the experimental effects on pathogen loads and expression of genes related to health. Arrows indicate significant main effects in the indicated direction and crosses indicate interaction effects. “Develop” = Migratory relative to stationary management during development, “Adult” = Migratory relative to stationary management during adulthood, “Age” = 28-day old compared to 14-day old, “Behavior” = Forager relative to in-hive worker, “Season” = July relative to May. DWV, Deformed wing virus; BQCV, Black queen cell virus; LSV, Lake Sinai virus; AKI, AKI virus complex; IAPV, Israeli acute paralysis virus; KBV, Kashmir bee virus; EFB, European foulbrood (Melissococcus plutonius); CB, Chalkbrood (Ascosphaera apis); Nosema, Vairimorpha apis or V. ceranae (formerly Nosema spp.); Trypano, Cirithidia mellificae or Lotmaria passim; Cyp9, Cyp9Q3 cytochrome P450; HSP90, heat shock protein 90; Hymen, hymenoptaecin; PGPR, peptidoglycan recognition protein; VG, vitellogenin; Lys2, lysozyme.

Effects of stationary versus migratory management persisted from the developmental treatment period through metamorphosis into adulthood for DWV levels. This finding, along with our earlier work with these samples on oxidative stress (Simone-Finstrom et al. 2016), suggests that follow-up studies on brood-care of bees during migratory management are warranted. Compared to their counterparts, bees that developed under migratory conditions exhibited higher DWV loads later in the season but lower DWV loads earlier in the season. The migratory conditions in our experiments may have improved the diversity of pollen sources (see Simone-Finstrom et al. 2016), and thus nutritional benefits could explain lower DWV in the early season (DeGrandi-Hoffman et al. 2010, Dolezal and Toth 2018). The relatively greater increase in DWV from early to late season in bees that were exposed to the migratory environment during development could be reflective of increased stress (Kuster et al. 2014). However, the later increase in DWV may also be explained by a nonsignificant trend for higher levels of the ectoparasitic Varroa mite in the migratory colonies at the end of the season (Simone-Finstrom et al. 2016), as Varroa increases DWV abundance (Martin et al. 2012); this may have been enough to increase DWV levels seen here.

The more direct effects of adult hive management on pathogen loads varied; while both brood pathogens (chalkbrood and European foulbrood) and most viruses did not show significant differences between treatment groups, the AKI complex load was higher in migratory colonies. In contrast, trypanosomes (gut parasites) were significantly higher in stationary colonies. A number of physiological functions were decreased by migratory conditions during adulthood, including anti-viral defenses, which could explain the increased AKI loads (Brutscher and Flenniken 2015). Conversely, migratory management may have led to more exposure to these viruses in the environment, increasing virus loads in migratory colonies (Welch et al. 2009) and consequently decreasing physiological functions. This effect is particularly conceivable for viral pathogens that can readily spread through the environment (Singh et al. 2010, McMenamin et al. 2016). However, such transmission is also possible for gut parasites (Higes et al. 2008; Jara et al. 2021), making the higher levels of these organisms in stationary colonies difficult to explain. It is possible that potential robbing from nearby colonies resulted in exchange of these parasites among the stationary colonies (Fries and Camazine, 2001), as this experimental site did experience reduced forage availability late in the season (Simone-Finstrom et al. 2016).

Among the numerous effects that were detected in addition to management, the overall declines of IAPV and KBV viral loads and EFB in foragers compared to in-hive bees are noteworthy, although not unexpected when independently accounting for chronological age. In-hive bees are particularly exposed to brood pathogens (Roetschi et al. 2008), and their stronger contact with nestmates may have also led to increased levels of IAPV and KBV. It is also possible that the reduced IAPV in foragers was a sampling artifact, as higher viral titers can cause bees to forage precociously (Benaets et al. 2017) and impact homing ability (Li et al. 2013), both of which would result in the elimination of foraging bees with higher titers by the age of collection in the current study. Additionally, expression of some immune genes was upregulated in foragers compared to nurses (defensin1, hymenoptaecin). The age of sampled bees additionally affected Vg expression. Old foragers having similar Vg as young hive bees when in the stationary adult environment early season is contrary to expectation and may be an artifact of sample size. However, given the regulatory role of Vg in lifespan, oxidative stress resistance, and onset of foraging, it is possible that bees that forage later in life would have higher Vg titers as low Vg is associated with precocious foraging (Nelson et al. 2007). Furthermore, we detected several general seasonal effects in accordance with expectations, such as the increase in DWV (Faurot-Daniels et al. 2020) and decrease in Nosema (Traver et al. 2012).

Overall, the results of this study support previous work indicating both negative and positive impacts of migratory beekeeping. These results support our hypothesis that immunity and pathogen levels are impacted by migratory beekeeping. Our data also is consistent with predictions that adult environment is most impactful on pathogen loads. Adult treatment was also more impactful than juvenile treatment on the expression of health and stress response genes, contrary to our prediction. This result could be due to responses of these genes to pathogen levels, or could be a more general response to the stress of transport and/or environmental change. Our findings shed light on the complex interactions of migratory bee management, such as the relocation of colonies for commercial pollination services, on bee health, and the spread of pathogens. Previous work on these samples showed negative effects of migratory management on lifespan. But better access to nutrition during periods of dearth by relocating colonies to agricultural areas reduced the impacts of oxidative stress (Simone-Finstrom et al. 2016). Here, the finding that in some cases bees involved in migratory management (either during larval and pupal development or as adults) show increase prevalence of some viral infections, while gut parasite loads are reduced provides continued evidence that migratory management is a mix of costs and benefits (Alger et al. 2018). Additional work is needed to determine why the effects of migratory management are greater on adult bees; the identification of specific causal factors could elucidate ways to mitigate the negative effects of both migratory and nonmigratory beekeeping on bee health. Such work is vital not only for pollinator health but also for food security given the dependence of many major food crops on managed pollination.

Supplementary Material

ieab096_suppl_Supplementary_Table_S1
Click here for additional data file. (45.6KB, docx)
ieab096_suppl_Supplementary_Table_S2
Click here for additional data file. (93.6KB, docx)

Acknowledgments

The study was supported by grants W911NF-04-D-0003 and W911NF-15-2-0045 to D.R.T., O.R. and M.K.S. from the Army Research Laboratory, as well as United States Department of Agriculture-National Institute of Food & Agriculture (2017-68004-26321 to O.R.). Jennifer Keller helped with field experiments and sample collection and Phil Tokarz assisted with the lab work. Mention of trade names or commercial products in this publication is solely for the purpose of providing specific information and does not imply recommendation or endorsement by the U.S. Department of Agriculture. USDA is an equal opportunity provider and employer. Samples were collected and treated according to standard protocols, minimizing distress of individual bees. Ethics committee approval was not required.

Authors’ Contributions

M.S.-F., M.K.S., D.R.T., and O.R. designed the study, M.S.-F. conducted all field experiments and laboratory analyses. M.S.-F. and O.R. analyzed the data, interpreted the results, and wrote the first draft of the manuscript. All authors reviewed and edited the final manuscript.

References Cited

  1. Ahn, K., Xie X., Riddle J., Pettis J., and Huang Z. Y.. . 2012. Effects of long distance transportation on honey bee physiology. Psyche J. Entomol. 2012. [Google Scholar]
  2. Alaux, C., Allier F., Decourtye A., Odoux J. F., Tamic T., Chabirand M., Delestra E., Decugis F., Le Conte Y., and Henry M.. . 2017. A ‘Landscape physiology’ approach for assessing bee health highlights the benefits of floral landscape enrichment and semi-natural habitats. Sci. Rep. 7: 40568. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Alburaki, M., Chen D., Skinner J. A., Meikle W. G., Tarpy D. R., Adamczyk J., and Stewart S. D.. . 2018. Honey bee survival and pathogen prevalence: from the perspective of landscape and xposure to pesticides. Insects 9: 65. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Alger, S. A., Burnham P. A., Lamas Z. S., Brody A. K., and Richardson L. L.. . 2018. Home sick: impacts of migratory beekeeping on honey bee (Apis mellifera) pests, pathogens, and colony size. Peerj. 6: e5812. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Amdam, G. V., and Omholt S. W.. . 2002. The regulatory anatomy of honeybee lifespan. J. Theor. Biol. 216: 209–228. [DOI] [PubMed] [Google Scholar]
  6. Bartlett, L. J., Boots M., Brosi B. J., de Roode J. C., Delaplane K. S., Hernandez C. A., and Wilfert L.. . 2021. Persistent effects of management history on honeybee colony virus abundances. J. Invertebr. Pathol. 179: 107520. [DOI] [PubMed] [Google Scholar]
  7. Benaets, K., Van Geystelen A., Cardoen D., De Smet L., de Graaf D. C., Schoofs L., Larmuseau M. H., Brettell L. E., Martin S. J., and Wenseleers T.. . 2017. Covert deformed wing virus infections have long-term deleterious effects on honeybee foraging and survival. Proc. Royal Soc. B: Biol. Sci. 284: 20162149. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Brutscher, L. M., and Flenniken M. L.. . 2015. RNAi and antiviral defense in the honey bee. J. Immunol. Res. 2015: 941897. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Colwell, M. J., Williams G. R., Evans R. C., and Shutler D.. . 2017. Honey bee-collected pollen in agro-ecosystems reveals diet diversity, diet quality, and pesticide exposure. Ecol. Evol. 7: 7243–7253. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Cornman, R. S., Tarpy D. R., Chen Y., Jeffreys L., Lopez D., Pettis J. S., vanEngelsdorp D., and Evans J. D.. . 2012. Pathogen webs in collapsing honey bee colonies. Plos One. 7: e43562. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Dainat, B., Evans J. D., Chen Y. P., Gauthier L., and Neumann P.. . 2012. Predictive markers of honey bee colony collapse. Plos One. 7: e32151. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. De Guzman, L. I., Frake A. M., and Simone-Finstrom M.. . 2017. Comparative flight activities and pathogen load of two stocks of honey bees reared in gamma-irradiated combs. Insects 8: 127. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. DeGrandi-Hoffman, G., Chen Y., Huang E., and Huang M. H.. . 2010. The effect of diet on protein concentration, hypopharyngeal gland development and virus load in worker honey bees (Apis mellifera L.). J. Insect Physiol. 56: 1184–1191. [DOI] [PubMed] [Google Scholar]
  14. Di Prisco, G., Annoscia D., Margiotta M., Ferrara R., Varricchio P., Zanni V., Caprio E., Nazzi F., and Pennacchio F.. . 2016. A mutualistic symbiosis between a parasitic mite and a pathogenic virus undermines honey bee immunity and health. Proc. Natl. Acad. Sci. U. S. A. 113: 3203–3208. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Dolezal, A. G., and Toth A. L.. . 2018. Feedbacks between nutrition and disease in honey bee health. Curr. Opin. Insect Sci. 26: 114–119. [DOI] [PubMed] [Google Scholar]
  16. Dolezal, A. G., Carrillo-Tripp J., Judd T. M., Allen Miller W., Bonning B. C., and Toth A. L.. . 2019. Interacting stressors matter: diet quality and virus infection in honeybee health. R. Soc. Open Sci. 6: 181803. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Doublet, V., Poeschl Y., Gogol-Döring A., Alaux C., Annoscia D., Aurori C., Barribeau S. M., Bedoya-Reina O. C., Brown M. J., Bull J. C., . et al. 2017. Unity in defence: honeybee workers exhibit conserved molecular responses to diverse pathogens. BMC Genomics. 18: 207. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Faurot-Daniels, C., Glenny W., Daughenbaugh K. F., McMenamin A. J., Burkle L. A., and Flenniken M. L.. . 2020. Longitudinal monitoring of honey bee colonies reveals dynamic nature of virus abundance and indicates a negative impact of Lake Sinai virus 2 on colony health. Plos One. 15: e0237544. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Fries, I., and Camazine S.. . 2001. Implications of horizontal and vertical pathogen transmission for honey bee epidemiology. Apidologie, 32: 199–214. [Google Scholar]
  20. Glenny, W., Cavigli I., Daughenbaugh K. F., Radford R., Kegley S. E., and Flenniken M. L.. . 2017. Honey bee (Apis mellifera) colony health and pathogen composition in migratory beekeeping operations involved in California almond pollination. Plos One. 12: e0182814. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Goodrich, B. K., Williams J. C., Goodhue R. E.. . 2019. The great bee migration: supply analysis of honey bee colony shipments into California for almond pollination Services. Am. J. Agri. Econ. 101: 1353–1372. [Google Scholar]
  22. Goulson, D., Nicholls E., Botías C., and Rotheray E. L.. . 2015. Bee declines driven by combined stress from parasites, pesticides, and lack of flowers. Science. 347: 1255957. [DOI] [PubMed] [Google Scholar]
  23. Gray, A., Adjlane N., Arab A., Ballis A., Brusbardis V., Charriere J.-D., Chlebo R., Coffey M. F., Cornelissen B., Costa C. A., . et al. 2020. Honey bee colony winter loss rates for 35 countries participating in the COLOSS survey for winter 2018–2019, and the effects of a new queen on the risk of colony winter loss. J. Apicult. Res. 59: 744–751. [Google Scholar]
  24. Graystock, P., Goulson D., and Hughes W. O.. . 2015. Parasites in bloom: flowers aid dispersal and transmission of pollinator parasites within and between bee species. Proc. Biol. Sci. 282: 20151371. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Higes, M., Martín-Hernández R., Garrido-Bailón E., García-Palencia P., and Meana A.. . 2008. Detection of infective Nosema ceranae (Microsporidia) spores in corbicular pollen of forager honeybees. J. Invertebr. Pathol. 97: 76–78. [DOI] [PubMed] [Google Scholar]
  26. Jara, L., Ruiz C., Martín-Hernández R., Muñoz I., Higes M., Serrano J., and De la Rúa P.. . 2021. The effect of migratory beekeeping on the infestation rate of parasites in honey bee (Apis mellifera) colonies and on their genetic variability. Microorganisms, 9: 22. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Klein, A. M., Vaissière B. E., Cane J. H., Steffan-Dewenter I., Cunningham S. A., Kremen C., and Tscharntke T.. . 2007. Importance of pollinators in changing landscapes for world crops. Proc. Biol. Sci. 274: 303–313. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Kulhanek, K., Steinhauer N., Rennich K., Caron D. M., Sagili R. R., Pettis J. S., Ellis J. D., Wilson M. E., Wilkes J. T., Tarpy D. R., . et al. 2017. A national survey of managed honey bee 2015–2016 annual colony losses in the USA. J. Apicult. Res. 56: 328–340. [Google Scholar]
  29. Kunc, M., Dobeš P., and Hurychová J., . et al. 2019. The year of the honey bee (Apis mellifera L.) with respect to Its physiology and immunity: a search for biochemical markers of longevity. Insects 10: 244. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Kuster, R. D., Boncristiani H. F., and Rueppell O.. . 2014. Immunogene and viral transcript dynamics during parasitic Varroa destructor mite infection of developing honey bee (Apis mellifera) pupae. J. Exp. Biol. 217: 1710–1718. [DOI] [PubMed] [Google Scholar]
  31. Li, Z., Chen Y., Zhang S., Chen S., Li W., Yan L., Shi L., Wu L., Sohr A., and Su S.. . 2013. Viral infection affects sucrose responsiveness and homing ability of forager honey bees, Apis mellifera L. Plos One. 8: e77354. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. López-Uribe, M. M., Fitzgerald A., and Simone-Finstrom M.. . 2017. Inducible versus constitutive social immunity: examining effects of colony infection on glucose oxidase and defensin-1 production in honeybees. R. Soc. Open Sci. 4: 170224. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. López-Uribe, M. M., Ricigliano V. A., and Simone-Finstrom M.. . 2020. Defining pollinator health: a holistic approach based on ecological, genetic, and physiological factors. Annu. Rev. Anim. Biosci. 8: 269–294. [DOI] [PubMed] [Google Scholar]
  34. Manley, R., Temperton B., Doyle T., Gates D., Hedges S., Boots M., and Wilfert L.. . 2019. Knock-on community impacts of a novel vector: spillover of emerging DWV-B from Varroa-infested honeybees to wild bumblebees. Ecol. Lett. 22: 1306–1315. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Martin, S. J., Highfield A. C., Brettell L., Villalobos E. M., Budge G. E., Powell M., Nikaido S., and Schroeder D. C.. . 2012. Global honey bee viral landscape altered by a parasitic mite. Science. 336: 1304–1306. [DOI] [PubMed] [Google Scholar]
  36. McMahon, D. P., Fürst M. A., Caspar J., Theodorou P., Brown M. J. F., and Paxton R. J.. . 2015. A sting in the spit: widespread cross-infection of multiple RNA viruses across wild and managed bees. J. Anim. Ecol. 84: 615–624. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. McMenamin, A. J., Brutscher L. M., Glenny W., and Flenniken M. L.. . 2016. Abiotic and biotic factors affecting the replication and pathogenicity of bee viruses. Curr. Opin. Insect Sci. 16: 14–21. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Naug, D., and Camazine S.. . 2002. The role of colony organization on pathogen transmission in social insects. J. Theor. Biol. 215: 427–439. [DOI] [PubMed] [Google Scholar]
  39. Nelson, C. M., Ihle K. E., Fondrk M. K., R. E.Page, Jr, and Amdam G. V.. . 2007. The gene vitellogenin has multiple coordinating effects on social organization. PLOS Biol. 5: e62. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Pettis, J. S., vanEngelsdorp D., Johnson J., and Dively G.. . 2012. Pesticide exposure in honey bees results in increased levels of the gut pathogen Nosema. Naturwissenschaften 99: 153–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Roetschi, A., Berthoud H., Kuhn R., and Imdorf A.. . 2008. Infection rate based on quantitative real-time PCR of Melissococcus plutonius, the causal agent of European foulbrood, in honeybee colonies before and after apiary sanitation. Apidologie 39: 362–371. [Google Scholar]
  42. Scofield, H. N., and Mattila H. R.. . 2015. Honey bee workers that are pollen stressed as larvae become poor foragers and waggle dancers as adults. Plos One. 10: e0121731. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Seitz, N., Traynor K. S., Steinhauer N., Rennich K., Wilson M. E., Ellis J. D., Rose R., Tarpy D. R., Sagili R. R., Caron D. M., . et al. 2016. A national survey of managed honey bee 2014–2015 annual colony losses in the USA. J. Apicult. Res. 54: 292–304. [Google Scholar]
  44. Simone-Finstrom, M., Li-Byarlay H., Huang M. H., Strand M. K., Rueppell O., and Tarpy D. R.. . 2016. Migratory management and environmental conditions affect lifespan and oxidative stress in honey bees. Sci. Rep. 6: 32023. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Simone-Finstrom, M., Aronstein K., Goblirsch M., Rinkevich F., and de Guzman L.. . 2018. Gamma irradiation inactivates honey bee fungal, microsporidian, and viral pathogens and parasites. J. Invertebr. Pathol. 153: 57–64. [DOI] [PubMed] [Google Scholar]
  46. Singh, R., Levitt A. L., Rajotte E. G., Holmes E. C., Ostiguy N., Vanengelsdorp D., Lipkin W. I., Depamphilis C. W., Toth A. L., and Cox-Foster D. L.. . 2010. RNA viruses in hymenopteran pollinators: evidence of inter-Taxa virus transmission via pollen and potential impact on non-Apis hymenopteran species. Plos One. 5: e14357. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. St Clair, A. L., Zhang G., Dolezal A. G., O’Neal M. E., and Toth A. L.. . 2020. Diversified farming in a monoculture landscape: effects on honey bee health and wild bee communities. Environ. Entomol. 49: 753–764. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Steinhauer, N., vanEngelsdorp D., and Saegerman C.. . 2021. Prioritizing changes in management practices associated with reduced winter honey bee colony losses for US beekeepers. Sci. Total Environ. 753: 141629. [DOI] [PubMed] [Google Scholar]
  49. Tokarev, Y. S., Huang W. F., Solter L. F., Malysh J. M., Becnel J. J., and Vossbrinck C. R.. . 2020. A formal redefinition of the genera Nosema and Vairimorpha (Microsporidia: Nosematidae) and reassignment of species based on molecular phylogenetics. J. Invertebr. Pathol. 169: 107279. [DOI] [PubMed] [Google Scholar]
  50. Traver, B. E., Williams M. R., and Fell R. D.. . 2012. Comparison of within hive sampling and seasonal activity of Nosema ceranae in honey bee colonies. J. Invertebr. Pathol. 109: 187–193. [DOI] [PubMed] [Google Scholar]
  51. Traynor, K. S., Pettis J. S., Tarpy D. R., Mullin C. A., Frazier J. L., Frazier M., and vanEngelsdorp D.. . 2016a. In-hive pesticide exposome: assessing risks to migratory honey bees from in-hive pesticide contamination in the Eastern United States. Sci. Rep. 6: 33207. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Traynor, K. S., Rennich K., Forsgren E., Rose R., Kunkel G., Madella S., Evans J., Lopez D., and vanEngelsdorp D.. . 2016b. Multiyear survey targeting disease incidence in US honey bees. Apidologie 47: 325–347. [Google Scholar]
  53. Underwood, R. M., Traver B. E., and López-Uribe M. M.. . 2019. Beekeeping management practices are associated with operation size and beekeepers’ philosophy towards in-hive chemicals. Insects. 10: 10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Van Esch, L., De Kok J. -L., Janssen L., Buelens B., De Smet L., de Graaf D. C., and Engelen G.. . 2020. Multivariate landscape analysis of honey bee winter mortality in Wallonia, Belgium. Environ. Model. Assess. 25: 441–452. [Google Scholar]
  55. Vanbergen, A. J. 2013. Threats to an ecosystem service: pressures on pollinators. Front. Ecol. Environ. 11: 251–259. [Google Scholar]
  56. Welch, A., Drummond F., Tewari S., Averill A., and Burand J. P.. . 2009. Presence and prevalence of viruses in local and migratory honeybees (Apis mellifera) in Massachusetts. Appl. Environ. Microbiol. 75: 7862–7865. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Zhu, X., Zhou S., and Huang Z. Y.. . 2014. Transportation and pollination service increase abundance and prevalence of Nosema ceranae in honey bees (Apis mellifera). J. Apicult. Res. 53: 469–471. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

ieab096_suppl_Supplementary_Table_S1
Click here for additional data file. (45.6KB, docx)
ieab096_suppl_Supplementary_Table_S2
Click here for additional data file. (93.6KB, docx)

Articles from Journal of Insect Science are provided here courtesy of University of Wisconsin Libraries

RESOURCES