Skip to main content
PLOS ONE logoLink to PLOS ONE
. 2020 Feb 25;15(2):e0228161. doi: 10.1371/journal.pone.0228161

Conspecific and interspecific stimuli reduce initial performance in an aversive learning task in honey bees (Apis mellifera)

Christopher A Varnon 1,*, Christopher W Dinges 2, Adam J Vest 2, Charles I Abramson 2
Editor: James C Nieh3
PMCID: PMC7041878  PMID: 32097420

Abstract

The purpose of this experiment was to investigate whether honey bees (Apis mellifera) are able to use social discriminative stimuli in a spatial aversive conditioning paradigm. We tested bees’ ability to avoid shock in a shuttle box apparatus across multiple groups when either shock, or the absence of shock, was associated with a live hive mate, a dead hive mate, a live Polistes exclamans wasp or a dead wasp. Additionally, we used several control groups common to bee shuttle box research where shock was only associated with spatial cues, or where shock was associated with a blue or yellow color. While bees were able to learn the aversive task in a simple spatial discrimination, the presence of any other stimuli (color, another bee, or a wasp) reduced initial performance. While the color biases we discovered are in line with other experiments, the finding that the presence of another animal reduces performance is novel. Generally, it appears that the use of bees or wasps as stimuli initially causes an increase in overall activity that interferes with early performance in the spatial task. During the course of the experiment, the bees habituate to the insect stimuli (bee or wasp), and begin learning the aversive task. Additionally, we found that experimental subject bees did not discriminate between bees or wasps used as stimulus animals, nor did they discriminate between live or dead stimulus animals. This may occur, in part, due to the specialized nature of the worker honey bee. Results are discussed with implications for continual research on honey bees as models of aversive learning, as well as research on insect social learning in general.

Introduction

In this paper, we investigate whether honey bees (Apis mellifera) are able to associate conspecific or interspecific cues with shock in an aversive conditioning paradigm. Honey bees have long been a popular species of study in fields such as behavioral ecology, comparative psychology, and neurophysiology. Colonies are often considered superorganism units [13] and thus a substantial body of literature investigates the rich social repertoire of honey bees. Despite possessing a relatively small nervous system of around one million neurons [4, 5], honey bees are able to direct hive mates to food sources [6, 7], use collaborative decision-making to select new hive locations [8], discriminate between related and unrelated individuals [912], and maintain a hygienic living area by removing dead or diseased individuals [13, 14]. These and other tasks are coordinated across a somewhat flexible, age-based, division of labor [1517].

Another major area of research examines the learning ability of bees, focusing primarily on appetitive conditioning. In appetitive experiments, bees either learn to associate neutral stimuli with desirable events, or learn to alter their behavior to produce a pleasant outcome. The proboscis extension response conditioning procedure has long been used to study appetitive conditioning in controlled laboratory conditions [1821], while other methods use free-flying, foraging bees when ecological validity is of greater importance [22, 23]. Research using these methods has produced a number of findings related to topics such as visual and olfactory sensation [2426], perception of time [27], conditioned taste aversion [28], learning of abstract concepts [29, 30], quantity discrimination [31, 32], and the neurophysiology of memory [3335].

Substantially less attention has been given to aversive conditioning, where bees learn that aversive stimuli are presented in association with other stimuli, or as a consequence for behavior [36, 37]. While reviews specifically comparing use of appetite and aversive techniques are not available, aversive literature is somewhat less common. A growing body of research investigates aversive conditioning in a shuttle box, where unrestrained bees learn to alter their behavior to reduce shock or other aversive stimuli in a small runway. Research using this method has explored areas such as learning differences between drones and workers [38], learned helplessness [39], visual discrimination [40], modulation of phototaxis [41], detection of narcotics [42], and the roles of the dopamine, octopamine and the mushroom body in aversive learning [41, 43, 44].

A relatively unexplored area of research is the intersection of social behavior and aversive learning. While a number of studies show that foraging and appetitive learning can be affected by social stimuli [21, 4547], we are aware of only two studies that investigate aversive learning in relation to a social stimulus. The first investigation showed that low levels of exposure to isopentyl acetate, the principal component of alarm pheromone, improves learning in a shuttle box procedure while exposure to geraniol, the main component of the social homing pheromone, did not have any effect on learning [48]. The second study found that isopentyl acetate increased sensitivity to shock in a sting extension response paradigm, while geraniol decreased aversive responsiveness [49]. These two studies are the first to demonstrate that social stimuli can affect aversive learning, however, as they only investigates the effect of the primary components of pheromones, not the complex visual, tactile, and olfactory stimuli related to interactions with an intact animal, the study of social stimuli in aversive conditioning remains an open area.

The following experiment employs a group design with 14 groups to address the relative lack of aversive conditioning research when compared to appetitive procedures, as well as an almost complete absence of social aversive conditioning studies. In this experiment, we use a shuttle box method to investigate spatial aversive learning using intact animals as social stimuli. Subjects are thus provided with a complex range of social stimuli, similar to what they may encounter in natural interactions. Across several groups, bees learn that either shock or safety is associated with a hive mate. As comparisons, we also investigate whether bees are able to distinguish between a conspecific hive mate, or an interspecific paper wasp, as well as whether the bees respond differently if the stimulus animals are alive or deceased. Additional groups are included as standard and social controls.

Materials and methods

Subjects

Experimentally naïve worker honey bees (Apis mellifera) were collected from a single hive in Stillwater, Oklahoma (36.1156° N, -97.0584° W) for use as subjects in this experiment (n = 268). Subjects were ultimately divided into 14 groups, with approximately 19 subjects per group. We used this sample and group size as this group size is extremely common in honey bee learning research, and as this experimental protocol is relatively novel, the literature was insufficient to permit a power analysis that would suggest a specific sample size. Additionally, our previous research [38, 39] with this general method suggested this sample size was adequate. Foraging bees were collected from a feeder containing 50% sucrose solution (weight/volume) approximately 50 meters away from the hive. All bees were collected from this single hive so that the experiment could test the ability of worker bees to use hive mates as a discriminative stimuli while avoiding issues related to antagonistic behaviors between bees from different hives. After collection, the bees were housed communally in a wire mesh carrier and were fed a 50% sucrose (weight/volume) solution. Bees were moved to a separate mesh carrier after acting as subject. Any bees that were complexly immobile once placed in the apparatus were discarded from the experiment. As Oklahoma State University does not require an ethics institutional review for non-threatened invertebrates, no specific review or permits were required to collect subjects for the present study.

Social discriminative stimuli

Bees used as conspecific social stimuli, hereby referred to as stimulus bees, were collected and maintained in the same manner as subject bees. The stimulus bees were moved to a separate mesh carrier after acting as a stimulus and were not reused as subjects. The interspecific social stimuli were paper wasps (Polistes exclamans), hereby referred to as stimulus wasps, and were collected along with their nests from several locations on Oklahoma State University campus during fall. Colonies of stimulus wasps were collected from high traffic areas where they are frequently removed by staff. As colonies of North American paper wasps survive for only a single year, with the fall marking the end of a colony’s life cycle, we expect collection had minimal impact on the campus wasp population. Each nest of wasps was maintained separately in a small plastic container, and the wasps’ natural nests were glued to the top of the container. Wasps were provided with water, 50% sugar sucrose solution (w/v), and small portions of cricket and mealworm meat ad libitum. Stimulus wasps were returned to their nest after acting as a stimulus in an experimental session. As Oklahoma State University does not require an ethics institutional review for non-threatened invertebrates, no specific review or permits were required to collect stimulus animals for the present study.

Apparatus

The apparatus consisted of two shuttle boxes and a control unit (see Fig 1). Two independent subjects were run simultaneously, one in each shuttle box, during each session. Subjects were run in pairs for the sake of efficient data collection, and were not able to interact to any degree. The general design of the shuttle boxes was modeled after those employed in [38, 39, 43]. The walls of the shuttle boxes were constructed from bars of HDPE plastic while the ends of the shuttle boxes were constructed from 3D printed ABS plastic. The ends served as both doors to permit subject entry and as stimulus chambers. The stimulus chambers were separated from the subjects’ area in shuttle box with a metal mesh. The mesh permitted odor transfer and tactile interaction, including allowing antennae to protrude through the mesh, but did not permit subjects or stimulus animals to cross the barrier. The inner dimensions of the stimulus chambers were 50 x 50 x 20 mm (length x width x height). During initial testing of the apparatus we were able to observe that the stimulus animals could rotate in the chamber, but could not move closer to, or farther from, the subjects’ area.

Fig 1. Shuttle box apparatus.

Fig 1

The top image shows a single shuttle box, disconnected from the electronics unit for clarity. Here, the shuttle box is set up for a blue/yellow discrimination. The bottom image is taken from the subject’s perspective inside the shuttle box and shows a stimulus bee in the stimulus chamber.

The ceiling and floor of the shuttle boxes consisted of a series of brass rods wired to form a shock grid. The rods were spaced 5.0 mm apart (center to center) with 3.5 mm gaps between the rods. Alternating rods were connected such that a bee could be shocked when it touched two adjacent rods. Clear acrylic strips were placed above the ceiling shock grid and below the floor shock grid to ensure that smaller subjects could not escape through the small gaps between the rods. The inner dimensions of the shuttle boxes measured 145 x 50 x 20 mm (length x width x height), ensuring that bees could move with little restriction but were always in contact with either the ceiling or floor shock grid. A pair of 38 kHz modulated infrared beams in the center of each shuttle box was used to detect the location of the subjects. We used modulated infrared beams as they are resistant to ambient infrared interference from artificial lights and sunlight.

The control unit consisted of a Propeller Experiment Controller [50, 51], and a series of LEDs and switches to allow users to control the parameters of each session. During an experiment, the Propeller Experiment Controller read the experimental parameters from the user interface, generated the 38 kHz infrared beam signal, detected the subjects’ locations, and provided 9 VDC shock to the entire shock grid through electromechanical relays based on the subjects’ locations. This level of shock was based on a number of previous experiments [e.g., 38, 39] and ensured that subjects were not physically impaired and did not sting or vomit, which could cause an increase in conductivity and shock received. After each session was completed, the Propeller Experiment Controller created a data spreadsheet on a microSD card containing detailed information on the instances when subjects broke the infrared beams and the duration that subjects spent on each side of the shuttle boxes. The data spreadsheets were transferred to a computer at the end of each day after sessions were completed, though the computer was not required to conduct experimental sessions.

Procedure

Each experimental session lasted nine minutes, and was divided into 18 30-second bins for sake of data analysis. Previous research indicates that this session length is adequate for observing learning, and changes in behavior are often apparent after a single minute [38]. During sessions, the shock grid activated if the subject was on the incorrect side of the shuttle box, as determined by group. Two subjects were run simultaneously in each shuttle box.

Subjects were randomly assigned to one of 14 groups. For all groups, the location of shock and stimuli were counterbalanced. See Table 1 for an overview of the group-design. The first set of groups were considered standard controls. These groups were used to replicate some existing findings of the honey bee aversive conditioning literature. In the spatial group, the shuttle boxes were placed on a plain grey background without stimulus animals present in the stimulus chambers. Shock was presented on either the front or the rear side of the shuttle box, with shock location counterbalanced across subjects. This group was used to assess spatial learning in the absence of other explicit cues. In the shock on blue and shock on yellow groups, the shuttle boxes were placed on a colored background where the color under the one half (either the front or the rear) of the shuttle box was blue, and the color under the other half of the shuttle box was yellow. Shock was presented on the blue side of the shuttle box for the shock on blue group, while shock was presented on the yellow side of the shuttle box for the shock on yellow group. Shock location was counterbalanced across subjects. We used these color discrimination groups to see whether bees could learn a non-social cue to avoid shock.

Table 1. Experimental design.

Group Category Group n
Standard Control Spatial 20
Standard Control Shock on blue 19
Standard Control Shock on yellow 19
Social Control Bee baseline 17
Social Control Wasp baseline 18
Social Control Bee and wasp baseline 20
Conspecific Safe by live bee 20
Conspecific Shock by live bee 20
Conspecific Safe by dead bee 18
Conspecific Shock by dead bee 18
Interspecific Safe by live wasp 20
Interspecific Shock by live wasp 20
Interspecific Safe by dead wasp 20
Interspecific Shock by dead wasp 19

The second set of groups acted as social controls and investigated responses to hive mates or wasps in the absence of shock. In the bee baseline group, one live stimulus bee was placed in a stimulus chamber, but no shock was delivered. Similarly, the wasp baseline group used one live wasp in a stimulus chamber, and did not deliver shock. The bee and wasp baseline group used a bee in one stimulus chamber, a wasp in another stimulus chamber, and did not deliver shock.

The third set of groups investigated whether bees could use a conspecific as discriminative stimuli. These groups included a live or dead stimulus bee from the same hive as the subject. All procedures with conspecific stimuli were run with the shuttle boxes on the grey background to investigate the effects of the social cues without adding additional visual cues. In the safe by live bee group, subjects received shock when on the opposite side of the apparatus as the stimulus bee, while in the shock by live bee group, subjects received shock when they were on the same side as the stimulus bee. The safe by dead bee and shock by dead bee groups functioned similarly, except that the stimulus bee was deceased. All dead stimulus bees were killed by freezing for several hours and were thawed before the experimental session began. Killing bees in this manner is consistent with other experiments that study both social and necrophoresis responses [13, 5254]. Neither live nor dead stimulus bees were used for more than one day.

The final set of groups investigated whether bees could use an interspecific, in this case the wasp P. exclamans, as discriminative stimuli. These groups included a live or dead stimulus wasp. As with the conspecific groups, shuttle boxes were placed on the grey background. The safe by live wasp, shock by live wasp, safe by dead wasp, and shock by dead wasp groups all mirrored their conspecific group counterparts, except that a stimulus wasp was used instead of a stimulus bee. All dead stimulus wasp were killed by freezing for several hours and were thawed before the experimental session began. Neither live nor dead stimulus wasps were used for more than one day.

Measurement and analysis

Measurement

We analyzed two measures. The first, correct-compartment restriction (CCR), refers to the duration the subjects spent in the correct compartment of the shuttle box. Honey bees constantly move inside the shuttle box, and thus CCR refers to the time their movement is restricted to the correct compartment, with an emphasis that they still may be exploring and pacing in that compartment. We report CCR in terms of percent, thus 50% CCR is a chance level of response indicating that a bee’s behavior is similar on both sides of the shuttle box. Our second measure, activity, concerned the overall activity of the subject, as measured by the number of times the infrared beams in the center of the apparatus were broken. Higher activity levels indicate more movement near the center of the apparatus, but not necessarily side-crossing.

Regression analysis

We first analyzed CCR and activity using a repeated measures linear regression via generalized estimating equations [e.g., 28, 5557]. All regression analysis were conducted through the StatsModels package [58] included in the Anaconda distribution of Python [59], a free scientific analysis distribution of the Python programming language (python.org). To analyze CCR, we first subtracted 50 from all scores so that the chance level of CCR (50, or 50%) became 0. Positive CCR scores thus refer to an improvement above chance, while negative CCR scores refer to a decrease below chance. This adjustment allows regression parameters to easily refer to difference between observed CCR and expected chance level of CCR. As activity level does not have an intuitive chance level of response, we analyzed z scores of activity. The z scores were created with respect to the mean of activity scores for the entire experiment, including all groups and bins. This allows regression parameters for activity to refer to an increase or decrease compared to the mean of all activity scores.

For all regression analyses, as the subjects’ behavior may change across time as a function of learning and we sought to investigate differences across groups, we included group, bin, and the group x bin interaction in all models. We used interceptless regressions that considered each level of the group variable to be a mutually exclusive categorical variable. The regression results thus compare the dependent variable to 0 (chance for CCR, or average for activity), as well as compare each groups’ bin effect to 0 (no change across bin).

This interceptless regression method can be obtained by a number of manners including recoding the group variable and running the regression multiple times so that each group acts as the reference level once, by use of intercept removal commands in statistical software packages, or by manually recalculating regression parameters. Given the speed of modern computers we chose the first method of rerunning the regression. After obtaining initial regression parameters, our analysis next provided pairwise comparisons between each groups and between each group x bin interaction. In regressions with standard intercepts, pairwise comparisons between two parameters can be seen when the first parameter is included in the intercept. The estimate, standard error, confidence intervals and p-value of the second parameter represent the difference between the parameters. We simply saved these comparisons when rerunning the regression. Pairwise comparisons can also be calculated manually by finding a difference between parameter estimates, then creating a z score by dividing the difference between the estimates by the square root of the sum of the squared standard errors of the estimates [60, 61]. Regardless of method, the result is the same.

We found our interceptless method to be highly interpretable. For example, consider regression with a three-group categorical variable as the only parameter. The standard form of displaying the regression results would display: 1) a comparison to zero for the first group, 2) a comparison between the first and second groups, and 3) a comparison between the first and third groups. Our method displays a more complete set of results: 1) a comparison to zero for the first group, 2) a comparison to zero for the second group, 3) a comparison to zero for the third group, 4) a comparison between the first and second groups, 5) a comparison between the first and third groups, and 6) a comparison between the second and third groups.

Given the somewhat arbitrary nature of p-values and multiple comparison correction techniques, we did not include a single, specific manner to adjust the significance threshold when making multiple comparisons. Instead, we encourage the reader to select their own initial threshold and adjustment criteria. Then, the reader is free to interpret our analysis in the manner they believe is the most appropriate. One simple and common approach would be adjusting the traditional alpha value of 0.05 using a Bonferroni correction. This would involve dividing the alpha value by the number of pairwise comparisons being made. For example, in the bottom three rows of Table 2 we compare the learning rates (group x bin interactions) of the spatial, shock on blue, and shock on yellow groups. The provided p-values could either be compared to the traditional alpha value of 0.05, or a Bonferroni corrected threshold of 0.0167 (0.05 divided by three comparisons). A wide number of other of multiple comparison methods could also be used. Regardless of the interpretation of the p-values, other aspects of the analysis remain consistent. For the sake of simplicity, we will primarily focus on the parameter estimates, but when considering p-values we will compare them to the traditional alpha level of 0.05.

Table 2. Control groups CCR analysis.
Parameter Estimate Standard Error 95% Confidence Intervals p-value
Spatial 4.483 4.330 -4.003 12.969 0.300
Shock on blue -1.975 3.972 -9.761 5.810 0.619
Shock on yellow -9.162 3.343 -15.714 -2.609 0.006
Spatial x Bin 0.644 0.306 0.045 1.244 0.035
Shock on blue x Bin -0.403 0.300 -0.992 0.186 0.180
Shock on yellow x Bin 0.592 0.282 0.040 1.144 0.036
Pairwise Comparison Difference Standard Error p-value
Spatial: Shock on blue 6.458 5.876 0.272
Spatial: Shock on yellow 13.645 5.470 0.013
Shock on blue: Shock on yellow 7.186 5.192 0.166
Spatial x Bin: Shock on blue x Bin 1.048 0.429 0.015
Spatial x Bin: Shock on yellow x Bin 0.053 0.416 0.899
Shock on blue x Bin: Shock on yellow x Bin -0.995 0.412 0.016

Ordinal analysis

In addition to the regression analysis, we also analyzed CCR with an ordinal analysis technique following the overall method outlined by James Grice [62]. For detailed discussion and examples see [23, 38, 6366]. This technique does not consider aggregate descriptive statistics or rely on comparisons to standardized distributions. Instead, all observations are compared directly, and comparisons between observed and randomized data are used in place of comparing a test statistic to a standard distribution. This approach provides an intuitive way to analyze data that is not subject to the many assumptions and interdisciplinary criticisms of traditional null-hypothesis significance testing [6771].

In our ordinal analysis of CCR, we analyzed each group separately. For each subject in a group, we conducted a pairwise comparison of CCR for each bin. We used this method to test three models: 1) CCR increases across bin, with earlier bins having CCR less than or equal to subsequent bins; 2) CCR decreases across bin, with earlier bins having CCR greater than or equal to subsequent bins; and 3) CCR does not change across bin, with earlier bins having CCR equal to subsequent bins. For each model, a percent correct classified (PCC) value was obtained. The PCC value represents the percent of observations that fit a model prediction. The PCC value can thus be considered similar to measure of model fit.

After obtaining a PCC value, the data for a group was then randomized (the entire group’s data set was shuffled across subject and bin), and the ordinal analysis was run again with the randomized data. This time a random PCC value was recorded. This process was repeated 1,000 times. The PCC value for the observed data and the randomized PCC values were compared to produce a chance value (c-value). The c-value represents the probability of the randomized PCC values being equal to or greater than the observed PCC value. In some ways, the c-value can be interpreted similarly to a p-value, in that a low c-value suggests a PCC value was very unlikely to occur by chance. However, unlike a p-value, no population assumptions are needed to interpret a c-value, nor is an arbitrary significance point given. Overall, interpretation of this method relies on 1) considering whether the model is valid, 2) considering the PCC value for the model, and 3) considering the c-value for the model. While this and other techniques are freely available in the Observation Oriented Modeling software [72], we conducted the analysis as described above in Python (S1 Ordinal analysis code).

Results

Standard control groups

Fig 2 shows correct-compartment restriction (CCR) and activity levels in the spatial, shock on blue, and shock on yellow groups. While bees in the spatial group quickly improved above the 50% chance level of CCR, bees in the color discrimination groups seldom performed above chance. Generally, it appears that the presence of color decreased the bees’ ability to reduce shock. For the shock on blue group, CCR begins at nearly chance, but decreases somewhat across the experiment. For the shock on yellow group, however, the bees appear to have a preference for yellow, initially responding below chance. However, this bias is less apparent later in the experiment as CCR improves above the chance level, likely due to learning. In terms of activity level, bees in the spatial group were somewhat less active in the center of the shuttle box than bees in the color discrimination groups, but no other clear tendencies were observed.

Fig 2. Correct compartment restriction (CCR) and activity levels in the spatial, shock on blue, and shock on yellow groups.

Fig 2

The left plot shows average CCR values with error bars indicating standard error of the mean. Note the axis is truncated between 20% and 75% to provide a clearer view of the data. The dotted grey line in the center of the graph indicates the 50% chance level of response. The right plot shows average and standard error of activity levels, as defined as the total number of times a bee broke either infrared beam in the center of the shuttle box.

S1, S2 and S3 Figs in the supporting material show heat map plots of CCR and activity levels of the spatial, shock on blue, and shock on yellow groups that illustrate the behavior of each individual bee. While variability across groups was high, a few interesting tendencies can be observed. Most notably is that several subjects performed exceptionally well in the spatial group, with frequent CCRs of nearly 100% often accompanied by very low activity near the center of the apparatus. In the color discrimination groups, low levels of CCR are also often accompanied by low activity levels. Generally, we observed low level of activity consisted of pacing within a small area, not immobility.

Table 2 shows the results of a regression analysis of CRR including the spatial, shock on blue, and shock on yellow groups, as well as interactions between these groups and bin. Generally, these analyses support earlier inspection of the graphs. The differences between the spatial, shock on blue, and shock on yellow groups’ initial CCR can be seen both in the parameter estimate indicating difference from the 50% chance level of CCR, and the pairwise comparison difference scores showing the disparity in CCR between the groups. The interactions of group and bin show the learning effects, and comparison between learning effects, for each group. As seen in Fig 2, initial CCR for the shock on yellow group is significantly below zero, and both the spatial and shock on yellow groups show significant learning effects. Table 3 shows a similar regression for z scores of activity levels, however, as seen in the graphs, few clear tendencies are observed. Note that for the pairwise comparisons shown in Tables 2 and 3, a multiple comparison correction to the significance threshold could be made. For example, a Bonferroni correction would suggest a threshold of 0.0167 (0.05 divided by three comparisons), for the pairwise comparisons of overall effects (three groups) and for the pairwise comparisons of learning effects (three group x bin interactions).

Table 3. Control groups activity analysis.

Parameter Estimate Standard Error 95% Confidence Intervals p-value
Spatial -0.146 0.183 -0.505 0.213 0.425
Shock on blue 0.014 0.188 -0.354 0.382 0.940
Shock on yellow 0.078 0.188 -0.290 0.447 0.676
Spatial x Bin -0.007 0.007 -0.022 0.007 0.319
Shock on blue x Bin 0.002 0.008 -0.013 0.017 0.783
Shock on yellow x Bin -0.006 0.008 -0.021 0.009 0.403
Pairwise Comparison Difference Standard Error p-value
Spatial: Shock on blue -0.160 0.262 0.541
Spatial: Shock on yellow -0.224 0.262 0.392
Shock on blue: Shock on yellow -0.064 0.266 0.809
Spatial x Bin: Shock on blue x Bin -0.010 0.011 0.372
Spatial x Bin: Shock on yellow x Bin -0.001 0.011 0.922
Shock on blue x Bin: Shock on yellow x Bin 0.008 0.011 0.432

Table 4 shows an ordinal analysis of CCR across bins for all groups. For the spatial group, the increasing across bin prediction shows the highest percent correct classified (PCC) value and a very low c-value, indicating this prediction is the best fit for the data and that this level of PCC cannot easily occur by chance. The decreasing across bin prediction has a very high c-value, indicating that the PCC of 48.1 can very easily be obtained by random data. The equal prediction has a very low PCC value, but also a low c-value, indicating that CCR was rarely equal between two bins, but that this level of equality is very unlikely to be observed in randomized data. A careful inspection of the individual data (see S1 Fig) shows that once subjects reach 100% CCR, the CCR often remains at 100%. This may account for the low PCC value and c-value of the equal prediction. It is also important to note that the increasing across bin prediction includes cases where CCR is equal across two bins (i.e., the comparison is ≤ not <). Subtracting the equal PCC from the increase PCC, thus shows the percent of comparisons where a CCR from a bin is less than, but not equal to, a subsequent bin’s CCR. For the spatial group, this is still higher than the decreasing prediction CCR. Similar results can be observed for the shock on yellow group. However, the shock on blue group appears to be a better fit for the decreasing prediction. Overall, the ordinal analysis supports the visual and regression analyses: the spatial group outperforms the other groups, and the shock on blue group performs the worst.

Table 4. Ordinal analysis of change in CCR across bin.

Group Increasing Decreasing Equal
PCC c PCC c PCC c
Spatial 58.5 0.00 48.1 0.94 06.5 0.00
Shock on blue 51.8 0.29 55.2 0.01 06.9 0.00
Shock on yellow 57.7 0.00 49.8 0.73 07.5 0.00
Bee baseline 47.6 0.90 52.7 0.10 00.3 0.01
Wasp baseline 54.0 0.02 46.4 0.97 00.4 0.00
Bee and wasp baseline 50.9 0.36 50.1 0.50 01.0 0.00
Safe by live bee 57.4 0.00 45.1 0.99 02.5 0.00
Shock by live bee 53.1 0.05 49.5 0.66 02.7 0.00
Safe by dead bee 60.0 0.00 45.4 0.99 05.4 0.00
Shock by dead bee 59.3 0.00 47.2 0.97 06.5 0.00
Safe by live wasp 54.9 0.01 50.8 0.48 05.7 0.00
Shock by live wasp 55.5 0.00 47.5 0.93 03.0 0.00
Safe by dead wasp 59.4 0.00 45.5 0.99 04.9 0.00
Shock by dead wasp 55.3 0.00 47.0 0.95 02.2 0.00

Social control groups

Fig 3 shows CCR and activity levels in the bee baseline, wasp baseline, and bee and wasp baseline groups. For these groups, a side was considered “correct” if it contained a bee, did not contain a wasp, or both. Thus, CCR can be used to show preferences for these stimuli. Generally, it appears that the subject bees slightly preferred being near stimulus bees and away from stimulus wasps. This is somewhat observed in all three groups, but it is interesting that the bee and wasp baseline group does not show an additive effect, instead it shows the smallest effect. Subjects in the bee baseline group showed the highest activity level, while subjects in the wasp baseline group showed the lowest activity level. In this case, the activity level of the bee and wasp baseline is an intermediate of the other groups. Activity levels appear to slightly decrease across session for all groups. S4, S5 and S6 Figs in the supplemental material show heat maps illustrating CCR and activity of each individual bee.

Fig 3. Correct compartment restriction (CCR) and activity levels in the bee baseline, wasp baseline and bee and wasp baseline groups.

Fig 3

The left plot shows CCR values, with the side near the stimulus bee being considered correct, and the side near the stimulus wasp being considered incorrect. Error bars indicate standard error of the mean. Note the axis is truncated between 20% and 75% to provide a clearer view of the data. The dotted grey line in the center of the graph indicates the 50% chance level of response. The right plot shows average and standard error of activity levels, as defined as the total number of times a bee broke either infrared beam in the center of the shuttle box.

Table 5 shows the results of a regression analysis of CCR including the bee baseline, wasp baseline, and bee and wasp baseline groups, as well as interactions between these groups and bin. Generally, these analyses support earlier inspection of the graphs; only weak, non-significant effects are observed, and the bee and wasp baseline group clearly does not show an additive effect of the bee baseline and wasp baseline groups. Table 6 shows a similar regression for z scores of activity levels. Subjects appear to be more active when a stimulus bee is present, but this effect decreases slightly over time when a wasp is also present. As with previous analysis a multiple comparison correction could be applied to the significance threshold for the pairwise comparisons in Tables 5 and 6.

Table 5. Social control groups CCR analysis.

Parameter Estimate Standard Error 95% Confidence Intervals p-value
Bee 3.770 2.661 -1.446 8.986 0.157
Wasp 0.315 3.692 -6.921 7.552 0.932
Bee and wasp -0.043 3.392 -6.691 6.604 0.990
Bee x Bin -0.110 0.231 -0.564 0.343 0.633
Wasp x Bin 0.396 0.282 -0.158 0.949 0.161
Bee and Wasp x Bin 0.098 0.336 -0.561 0.756 0.772
Pairwise Comparison Difference Standard Error p-value
Bee: Wasp 3.455 4.551 0.448
Bee: Bee and Wasp 3.813 4.311 0.376
Wasp: Bee and Wasp 0.358 5.014 0.943
Bee x Bin: Wasp x Bin -0.506 0.365 0.166
Bee x Bin: Bee and wasp x Bin -0.208 0.408 0.610
Wasp x Bin: Bee and wasp x Bin 0.298 0.439 0.497

Bee baseline, wasp baseline and bee and wasp baseline groups are abbreviated as bee, wasp, and bee and wasp, respectively.

Table 6. Social control groups activity analysis.

Parameter Estimate Standard Error 95% Confidence Intervals p-value
Bee 0.374 0.137 0.105 0.642 0.006
Wasp 0.060 0.173 -0.279 0.398 0.731
Bee and wasp 0.351 0.089 0.176 0.526 0.000
Bee x Bin -0.010 0.007 -0.023 0.004 0.155
Wasp x Bin -0.007 0.010 -0.027 0.012 0.475
Bee and Wasp x Bin -0.030 0.009 -0.046 -0.013 0.001
Pairwise Comparison Difference Standard Error p-value
Bee: Wasp 0.314 0.221 0.154
Bee: Bee and Wasp 0.023 0.164 0.890
Wasp: Bee and Wasp -0.292 0.195 0.134
Bee x Bin: Wasp x Bin -0.002 0.012 0.836
Bee x Bin: Bee and wasp x Bin 0.020 0.011 0.066
Wasp x Bin: Bee and wasp x Bin 0.022 0.013 0.086

Bee baseline, wasp baseline and bee and wasp baseline groups are abbreviated as bee, wasp, and bee and wasp, respectively.

An ordinal analysis of CCR for the social control groups can be seen in Table 4. The results here generally support the findings of visual and regression analysis. Any minor preferences observed in the bee baseline decrease across bin, while preferences in the wasp baseline increase across bin. It appears that neither an increasing nor decreasing pattern is a good fit for the bee and wasp baseline group.

Overall, it appears that subject bees may have a very minor preference for being near another bee. A slight preference for being away from the stimulus wasp is also present, and this is the only preference that seems to increase over time. A post-hoc single sample t test of total CCR (bins summed; one measurement per subject) with all groups pooled, also found a small and nonsignificant tendency for the subject to be near the stimulus bee and away from the stimulus wasp: t(54) = 1.804, p = 0.0768, d = 0.243. Activity was also increased in the presence of another bee but decreases over time when a wasp is present. As all these effects appear to be very minor, they do not hinder investigations of using stimulus bees or stimulus wasps as cues, as would strong, consistent biases.

Conspecific and interspecific groups

Fig 4 shows CCR and activity levels in the safe by live bee, shock by live bee, safe by dead bee, and shock by dead bee groups. The spatial group was also included as a comparison. Generally, subjects in the conspecific groups initially perform below chance, then gradually improve across the session. Surprisingly, even though some minor preference for being near stimulus bees was observed in the bee baseline group, subjects do not appear to perform better when a stimulus bee predicts safety than when a stimulus bee predicts shock. It also does not appear that subjects make a distinction between live or dead stimulus bees. Instead, it appears that presence of any stimulus bee initially reduces performance, especially when compared to the spatial group. Activity levels also do not show a clear distinction between groups, but do show a slight overall decrease throughout the session, similar to what was observed for the social control groups (see Fig 3). S7, S8, S9 and S10 Figs in the supplemental material show heat maps of CCR and activity levels for the conspecific groups that illustrate the behavior of each individual bee. Generally, the same tendencies are observed as in Fig 4, but it can also be seen that extreme CCR levels are occasionally accompanied by low activity levels.

Fig 4. Correct compartment restriction (CCR) and activity levels in groups that used another bee as a stimulus.

Fig 4

The left plot shows average CCR values with error bars indicating standard error of the mean. Note the axis is truncated between 20% and 75% to provide a clearer view of the data. The dotted grey line in the center of the graph indicates the 50% chance level of response. The right plot shows average and standard error of activity levels, as defined as the total number of times a bee broke either infrared beam in the center of the shuttle box.

Fig 5 shows CCR and activity levels in the safe by live wasp, shock by live wasp, safe by dead wasp, and shock by dead wasp groups. The spatial group was also included as a comparison. Although the data appears more variable, similar tendencies are observed as with the conspecific groups. Subjects in the interspecific groups show initially low CCR levels, followed by gradual improvement, and this tendency is not distinct across group. Activity levels for the interspecific groups appear both higher and more variable than those of the conspecific groups. S11, S12, S13 and S14 Figs in the supplemental material show heat maps of CCR and activity levels for the interspecific groups that illustrate the behavior of each individual bee. As with the color discrimination and conspecific groups, high CCR levels are occasionally accompanied by low activity levels.

Fig 5. Correct compartment restriction (CCR) and activity levels in groups that used a wasp as a stimulus.

Fig 5

The left plot shows average CCR values with error bars indicating standard error of the mean. Note the axis is truncated between 20% and 75% to provide a clearer view of the data. The dotted grey line in the center of the graph indicates the 50% chance level of response. The right plot shows average and standard error of activity levels, as defined as the total number of times a bee broke either infrared beam in the center of the shuttle box.

Tables 7 and 8 show the results of a regression analyses of CCR and activity levels, respectively. These analyses include all conspecific and interspecific groups, but also include the spatial group for comparison. Unlike previous tables, pairwise comparisons were not included. As the list of pairwise comparisons is rather lengthy, they are instead included in the supporting material as S5, S6, S7 and S8 Tables. For CCR, while the bees in the spatial group initially respond above chance, bees in the conspecific and interspecific groups initially respond below chance with the exception of the shock by live bee group. While this is fairly consistent, with only the shock by live bee group presenting an exception, these effects are not large or significant. The learning effects for all groups, however, were consistently positive, and also significant with the exception of the three weakest effects (shock by live bee x bin, safe by live wasp x bin, and shock by dead wasp x bin). It is worth noting that the regression does not show any interesting similarities between groups. For example, the parameter estimates for groups where the stimulus animal predicts safety do not differ much from those that predict shock, nor do groups with live stimulus animals differ from those with dead stimulus animals, nor do groups with stimulus bees from those with stimulus wasps. The same general finding is true for the activity levels; most groups are similar and no overall differences between safe/shock, live/dead, and bee/wasp, are observed. The pairwise comparisons of CCR and activity level also show similar tendencies, and no substantial thematic difference between the various conspecific and interspecific groups emerge. The ordinal analysis in Table 4 support what the visual and regression analysis suggest about CCR. All conspecific and interspecific groups show an increase in CCR across bin in a manner that appears similar to the spatial group.

Table 7. Experimental groups CRR analysis.

Parameter Estimate Standard Error 95% Confidence Intervals p-value
Spatial 4.483 4.330 -4.003 12.969 0.300
Safe by live bee -6.678 3.543 -13.622 0.265 0.059
Shock by live bee 2.501 3.794 -4.935 9.937 0.510
Safe by dead bee -6.366 4.472 -15.130 2.398 0.155
Shock by dead bee -5.862 5.527 -16.694 4.970 0.289
Safe by live wasp -0.206 4.545 -9.113 8.702 0.964
Shock by live wasp -1.298 4.726 -10.561 7.965 0.784
Safe by dead wasp -6.172 3.663 -13.352 1.007 0.092
Shock by dead wasp -5.244 4.284 -13.641 3.154 0.221
Spatial x Bin 0.644 0.306 0.045 1.244 0.035
Safe by live bee x Bin 0.846 0.321 0.218 1.475 0.008
Shock by live bee x Bin 0.364 0.367 -0.355 1.082 0.321
Safe by dead bee x Bin 0.918 0.375 0.182 1.654 0.014
Shock by dead bee x Bin 1.052 0.431 0.207 1.897 0.015
Safe by live wasp x Bin 0.114 0.309 -0.491 0.719 0.711
Shock by live wasp x Bin 0.710 0.340 0.045 1.376 0.037
Safe by dead wasp x Bin 0.874 0.237 0.409 1.339 0.000
Shock by dead wasp x Bin 0.641 0.418 -0.178 1.459 0.125

Table 8. Experimental groups activity analysis.

Parameter Estimate Standard Error 95% Confidence Intervals p-value
Spatial -0.238 0.168 -0.568 0.092 0.157
Safe by live bee 0.080 0.163 -0.239 0.399 0.625
Shock by live bee -0.053 0.141 -0.329 0.224 0.708
Safe by dead bee 0.180 0.232 -0.275 0.635 0.439
Shock by dead bee 0.201 0.276 -0.339 0.741 0.466
Safe by live wasp 0.349 0.266 -0.171 0.870 0.189
Shock by live wasp -0.208 0.170 -0.540 0.125 0.221
Safe by dead wasp 0.195 0.179 -0.157 0.546 0.278
Shock by dead wasp 0.115 0.228 -0.332 0.562 0.614
Spatial x Bin -0.007 0.009 -0.025 0.010 0.414
Safe by live bee x Bin -0.010 0.012 -0.034 0.014 0.414
Shock by live bee x Bin 0.000 0.010 -0.018 0.019 0.970
Safe by dead bee x Bin -0.011 0.016 -0.043 0.020 0.479
Shock by dead bee x Bin -0.020 0.019 -0.057 0.016 0.278
Safe by live wasp x Bin -0.007 0.016 -0.037 0.024 0.674
Shock by live wasp x Bin 0.010 0.012 -0.013 0.033 0.402
Safe by dead wasp x Bin -0.017 0.014 -0.045 0.010 0.209
Shock by dead wasp x Bin 0.007 0.015 -0.023 0.038 0.632

Taken together, it is clear that subjects in the conspecific and interspecific groups show similar improvement in CCR. However, although they are learning, the presence of conspecific or interspecific cues do not seem to aid subjects. Instead, any cue in addition to the spatial task initially reduces correct responding. Surprisingly, the bees in this experiment do not appear capable of using additional information to improve performance, and this is consistent across conspecific and interspecific groups.

Discussion

Overview

Generally, our results indicate that while bees in the spatial group display a rapid improvement in performance, the addition of any other stimuli (color, bee, or wasp) initially reduces performance. Bees in the spatial group showed a final level of performance of around 61% to 66% CCR (estimates derived the averaged data and from the analysis in Table 2, respectively). While only a few studies use comparable automated measurements to study aversive learning in honey bees, similar performance is consistently reported. Across four other papers, many including multiple experiments and groups, we found that bees in groups that responded above chance had an average CCR of around 61% [3840, 44]. It appears that bees in our spatial group showed a typical response.

Although bees in the shock on blue and shock on yellow groups did not perform as well as bees in the spatial group, this is not surprising given the prevalence of color biases in bee research. Such biases are common in appetitive and foraging experiments, and have been observed in honey bees [73, 74], stingless bees [75], and bumble bees [76]. In an earlier shuttle box experiment using a previous version of our apparatus and a similar protocol, Dinges et al. [38] found that bees performed better when shock was associated with blue than when shock was associated with yellow. Similarly, Kirkerud, Schlegel, and Galizia [40] found that bees easily learned that safety was associated with green, but had trouble learning that green was associated with shock. Black et al. [77] explained this by demonstrating that biases acquired during natural foraging can persist in aversive conditioning procedures. Given that bees in our shock on blue and shock on yellow groups performed worse than bees in the spatial group, it is likely the bees had a history of feeding on both blue and yellow flowers. As bees in the shock on blue group responded worse throughout the experiment while bees shock on yellow group improved, our bees also likely had more experience feeding on blue than on yellow flowers.

While bees in the standard control groups responded similarly to findings reported in other literature, the low performance of the bees in the conspecific and interspecific groups may seem surprising. Unfortunately, there is no literature suitable for direct comparison. Generally, bees in conspecific and interspecific groups initially responded below chance, then slowly increased in performance throughout the experiment. This effect occurred consistently without respect to the predictive role, living status, or species of the stimulus animal. Our findings suggest that bees have difficulty using other bees or wasps as cues in this spatial aversive conditioning task and perform better overall when these stimuli are not available. This leads to two questions. First, why did the presence of another animal reduce initial performance compared to the spatial group and how was this overcome? Second, why were no differences observed in response to different types of animal stimuli?

Reduced performance in the conspecific and interspecific groups

The reduced initial performance of the conspecific and interspecific groups compared to the spatial group suggests that the bees can detect, and are affected by, the stimuli animals. A major possibility for this difference is that the shuttling behavior of bees may initially sensitize, then habituate to the presence of other insects in the apparatus. Rapid shuttling behavior is common in this type of procedure [e.g., 40, 44], and the relationship between shuttling and reduced learning has been observed in other shuttle box experiments. For example, Dinges et al. [39] found that the shuttling behavior of bees prohibited use of traditional negative reinforcement (escape/avoidance) methods to study learned helplessness, and instead required a positive punishment method similar to the one used in the present experiment (i.e., shock is delivered contingent on entering a side). In the current experiment, the presence of other animals may cause increase shuttling as the bee investigates the situation, searches for better access to a hive mate, or attempts to an escape an apparatus containing wasp. This increased shuttling may be somewhat incompatible with learned performance, as has been observed in previous studies, and could cause a nearly chance level of correct response. Additionally, although we used a mild shock, the act of being shocked may physically inhibit a bee’s ability to shuttle back to the correct side, resulting in more time spent on the incorrect side and thus a lower than chance level of correct response. Over time, the impact of the animal stimuli may decrease through habituation, leading to less overall movement, and facilitating behavior that is more directed by the aversive continencies.

Some evidence for this interpretation can be seen in the present experiment. The spatial group showed the highest average CCR and the lowest average activity of all groups. Conversely, all 11 groups with stimulus animals, even those without shock, showed much higher activity scores, and these scores decreased across the experiment with only three exceptions (shock by live bee–no change in activity; shock by live wasp and safe by life wasp–increase in activity). For groups with stimulus animals and shock, CCR scores were initially low and increased across the experiment, likely as the shuttling response habituated to the stimulus animal. Though we observed similar tendencies in earlier work using this method [39], the only other research using social stimuli in a spatial aversive conditioning paradigm found a different effect from our study. In a series of experiments, Avalos et al. [48] showed that exposure to components of alarm pheromone, but not homing pheromone, improves aversive spatial learning. It is possible that the general, holistic stimuli used in our experiment promotes increased activity that may be incompatible with the shuttle box task, while the isolated alarm pheromone used by Avalos et al. [48] causes a different response that enhances performance.

Failure to discriminate between types of stimulus animals

We did not observe any fundamental distinctions in responses of subjects that received bees as stimuli compared to those that received wasps as stimuli, nor were clear differences observed when the stimulus animals were alive or dead. Both of these findings may be accounted, in part, by the specialized nature of the worker honey bee. It is possible that the bees we collected are not well suited to produce distinct behavior in response to nestmates compared to wasps. While bees and wasps often have antagonistic interactions and bees guarding the hive are known to make such distinctions [78, 79], the guard bee is a specialized role of worker bee that is unlikely to be present in our sample of bees that were collected at feeders. Forager workers may instead ignore the presence of other non-predator insects while collecting nectar and pollen. Alternatively, the presence of any pollinator insect (including other bees and paper wasps) may facilitate foraging at a particular location, as has been shown with bumble bees [5254].

It is also possible that our sample of bees was also not well suited to discriminate between live and dead individuals. This discrimination may be reserved for undertaker bees, another specialized role that worker bees may take before becoming foragers [14, 80]. Social foraging research with bumble bees shows that dead conspecifics are indeed adequate cues to facilitate foraging [5254]. As the traditional method of collecting worker bees captures bees while foraging, this sampling approach may also prevent undertaker bees from being collected.

Future directions and conclusions

Our work contributes to a growing body of research on aversive learning in honey bees. Additionally, we present one of the only experiments investigating the relationship between social stimuli and aversive learning. The findings of our experiment suggest that, as with previous research using this method [39], careful consideration needs to be given to not only the learned performance of bees, but also to their tendency to rapidly pace inside a shuttle box. Our findings contribute to a body of research showing that while bees will learn the task, they are also affected by colors that may be related to their foraging history, and we provide a new findings suggesting that their tendency to pace may be enhanced in the presence of other insects, to the extent that it reduces initial performance.

The fact the bees in our experiment did not discriminate between live and dead stimulus animals also suggests that experiments investigating topics such as observational learning or social facilitation in insects should carefully consider what stimuli their subjects are actually responding to, and may benefit from using deceased stimulus animals as a control. Indeed, research on social facilitation of foraging in bumble bees has already shown that foragers prefer to select flowers occupied by another bee, regardless of whether the other bee is alive or dead [5254]. Our research suggests similar effects may be observed in honey bees, and research on social learning in other taxa, such as Drosophila, may also benefit from considering this possibility.

Additional research will be required to determine specifically which social stimuli affect learning, and what manner these stimuli affect behavior. For example, could an isolated substance, such as the fatty acids found in comb wax that aids in hive mate recognition [10] evoke the reduced performance effect observed in this experiment? Combining such social substances with live, dead, or even dummy bees could help identify the cause of the effects we observed.

Future work should also consider using refined position and activity measurements such as the grid of infrared beams used by Kirkerud et al. [40] and Nouvian and Galizia [81], or a computer vision approach similar to that used by Marchal et al. [41] and Kimura et al. [82] to precisely measure the pacing behavior observed in shuttle box experiments. Additionally, investigations on the habituation to social stimuli discussed in this paper could also consider the many principles of habituation and sensitization outlined by Thompson and Spencer [83], Groves and Thompson [84], and Rankin et al. [85]. For example, the stimulus generalization principle suggests that habituation may to one stimulus may generalize to similar stimuli. In the case of social stimuli such as those used in the present experiment, habituation to one species of Polistes paper wasp may also cause habituation to other species of Polistes. These three papers [8385] discuss a number of similar, relatively simple, but important principles that could be considered.

Finally, as honey bees are specialized not only by caste, but also by the current role of the worker bee, future research should consider explorations of multiple castes and worker roles. A growing body of research is already outlining learning differences between worker and drone castes [38, 48], but similar work is also needed to compare worker specializations and ages. For example, bees that have matured into foragers but have not yet foraged may not display the color biases that are commonly observed, while newly emerged bees may show a completely different manner of responding than our subjects. We hope that our findings and suggestions will help facilitate continual research on honey bees as models of aversive learning.

Supporting information

S1 Table. Experimental groups CRR analysis—Group effects.

(DOCX)

S2 Table. Experimental groups CRR analysis—Bin interactions.

(DOCX)

S3 Table. Experimental groups activity analysis—Group effects.

(DOCX)

S4 Table. Experimental activity CRR analysis—Bin interactions.

(DOCX)

S5 Table. Experimental groups CRR analysis—Group effects.

(DOCX)

S6 Table. Experimental groups CRR analysis—Bin interactions.

(DOCX)

S7 Table. Experimental groups activity analysis—Group effects.

(DOCX)

S8 Table. Experimental activity CRR analysis—Bin interactions.

(DOCX)

S1 Fig. Heat map of CCR and activity levels of the spatial group.

(DOCX)

S2 Fig. Heat map of CCR and activity levels of the shock on blue group.

(DOCX)

S3 Fig. Heat map of CCR and activity levels of the shock on yellow group.

(DOCX)

S4 Fig. Heat map of CCR and activity levels of the bee baseline group.

(DOCX)

S5 Fig. Heat map of CCR and activity levels of the wasp baseline group.

(DOCX)

S6 Fig. Heat map of CCR and activity levels of the bee and wasp baseline group.

(DOCX)

S7 Fig. Heat map of CCR and activity levels of the safe by live bee group.

Note that one extreme activity score was truncated to 25 for sake of consistency across graphs.

(DOCX)

S8 Fig. Heat map of CCR and activity levels of the shock by live bee group.

Note that one extreme activity score was truncated to 25 for sake of consistency across graphs.

(DOCX)

S9 Fig. Heat map of CCR and activity levels of the safe by dead bee group.

(DOCX)

S10 Fig. Heat map of CCR and activity levels of the shock by dead bee group.

(DOCX)

S11 Fig. Heat map of CCR and activity levels of the safe by live wasp group.

Note that five extreme activity scores were truncated to 25 for sake of consistency across graphs.

(DOCX)

S12 Fig. Heat map of CCR and activity levels of the shock by live wasp group.

(DOCX)

S13 Fig. Heat map of CCR and activity levels of the safe by dead wasp group.

(DOCX)

S14 Fig. Heat map of CCR and activity levels of the shock by dead wasp group.

Note that one extreme activity score was truncated to 25 for sake of consistency across graphs.

(DOCX)

S1 Data

(CSV)

S1 Ordinal analysis code

(PY)

Acknowledgments

We are grateful for the useful comments of Paul Marchal and one anonymous reviewer. We would also like to thank Noelle Vallely and Charlie Beheler for their assistance with reviewing literature.

Data Availability

All relevant data are within the manuscript and its Supporting Information files.

Funding Statement

This material was based upon work which was supported in part by National Science Foundation grants DBI 0552717, DBI 1263327, OISE 1545803, GRF 1144467 (received by CIA), and by National Institutes of Health grant P20GM103499 (received by CV). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

  • 1.Seeley TD. The honey bee colony as a superorganism. Am Sci. 1989; 77: 546–553. [Google Scholar]
  • 2.Seeley TD. Honey bee colonies are group-level adaptive units. Am Nat. 1997; 150: S22–S23. 10.1086/286048 [DOI] [PubMed] [Google Scholar]
  • 3.Southwick EE. The honey bee cluster as a homeothermic superorganism. Comp Biochem Physiol A Physiol.1983; 75: 641–645. [Google Scholar]
  • 4.Giurfa M, Menzel R. Cognitive architecture of a mini-brain: the honeybee. Trends Cogn Sci. 2001. 5: 62–71. 10.1016/s1364-6613(00)01601-6 [DOI] [PubMed] [Google Scholar]
  • 5.Witthöft W. Absolute anzahl und verteilung der zellen im hirn der honigbiene. Zoomorphology. 1967; 61: 160–184. [Google Scholar]
  • 6.Gould JL. Honey bee recruitment: The dance-language controversy. Science. 1975; 189: (4204), 685–693 10.1126/science.1154023 [DOI] [PubMed] [Google Scholar]
  • 7.von Frisch K. The Dance Language and Orientation of Bees. Cambridge, MA: Harvard University Press; 1967. [Google Scholar]
  • 8.Visscher PK, Seeley T, Passino KM. Group decision making in honey bee swarms. Am Sci. 2006; 94: 220–229. [Google Scholar]
  • 9.Breed MD. Nestmate recognition in honey bees. Anim Behav. 1983; 31: 89–91. [Google Scholar]
  • 10.Breed MD. Recognition pheromones of the honey bee. BioScience. 1998; 48: 463–470. [Google Scholar]
  • 11.Breed MD, Leger EA, Pearce AN, Wang YJ. Comb wax effects on the ontogeny of honey bee nestmate recognition. Anim Behav. 1998; 55: 13–20. 10.1006/anbe.1997.0581 [DOI] [PubMed] [Google Scholar]
  • 12.Dani FR, Jones GR, Corsi S, Beard R, Pradella D, Turillazzi S. Nestmate recognition cues in the honey bee: Differential importance of cuticular alkanes and alkenes. Chem Senses. 2005; 30: 477–489. 10.1093/chemse/bji040 [DOI] [PubMed] [Google Scholar]
  • 13.Visscher PK. The honey bee way of death: Necrophoric behaviour in Apis mellifera colonies. Anim Behav. 1983; 31(4): 1070–1076. [Google Scholar]
  • 14.McAfee A, Chapman A, Iovinella I, Gallagher-Kurtz Y, Collins TF, Higo H, et al. A death pheromone, oleic acid, triggers hygienic behavior in honey bees (Apis mellifera L.) Sci Rep. 2018; 8: 1–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Haung Z, Robinson G. Regulation of honey bee division of labor by colony age demography. Behav Ecol Sociobiol. 1996; 39: 147–158 [Google Scholar]
  • 16.Schulz DJ, Robinson GE. Octopamine influences division of labor in honey bee colonies. J. Comp Physiol A. 2001; 187: 53–61. 10.1007/s003590000177 [DOI] [PubMed] [Google Scholar]
  • 17.Johnson B. Organization of work in the honeybee: A compromise between division of labour and behavioral flexibility. Proc R Soc B. 2003; 270: 147–152. 10.1098/rspb.2002.2207 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Takeda K. Classical conditioned response in the honey bee. J Insect Physiol. 1961; 6: 168–179. [Google Scholar]
  • 19.Bitterman MR, Menzel R, Fietz A, Shäfer S. Classical conditioning of proboscis extension in honey bees (Apis mellifera). J Comp Psycho. 1983; 97(2): 107–119. [PubMed] [Google Scholar]
  • 20.Abramson CI, Boyd BJ. An automated apparatus for conditioning proboscis extension in honey bees (Apis mellifera L.). J of Entomol Sci. 2001; 36: 78–92. [Google Scholar]
  • 21.Matsumoto Y, Menzel R, Sandoz JC, Giurfa M. Revisiting olfactory classical conditioning of the proboscis extension response in honey bees: A step toward standardized procedures. J Neurosci Methods. 2012; 211: 159–167. 10.1016/j.jneumeth.2012.08.018 [DOI] [PubMed] [Google Scholar]
  • 22.Abramson CI, Dinges CW, Wells H. Operant conditioning in honey bees (Apis mellifera L.): The cap pushing response. PLoS ONE. 2016; 11: e0162347 10.1371/journal.pone.0162347 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Craig DPA, Grice JW, Varnon CA, Gibson B, Sokolowski MBC, Abramson CI. Social reinforcement delays in free-flying honey bees (Apis mellifera L.). PloS One. 2012; 7, e46729 10.1371/journal.pone.0046729 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Hori S, Takeuchi H, Arikawa K, Kinoshita M, Ichikawa N, Sasak M, et al. Associative visual learning, color discrimination, and chromatic adaptation in the harnessed honeybee (Apis mellifera L). J Comp Physiol A. 2006; 192: 691–700. [DOI] [PubMed] [Google Scholar]
  • 25.Wright GA, Carlton M, Smith BH. A honeybee’s ability to learn, recognize, and discriminate odors depends upon odor sampling time and concentration. Behav Neurosci. 2009; 123: 36–43. 10.1037/a0014040 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Avarguès-Weber A, Mota T. Advances and limitations of visual conditioning protocols in harnessed bees. J Physiol Paris. 2016; 110: 107–118. 10.1016/j.jphysparis.2016.12.006 [DOI] [PubMed] [Google Scholar]
  • 27.Craig DPA, Varnon CA., Sokolowski MBC, Wells H, Abramson CI. An assessment of fixed interval timing in free-flying honey bees (Apis mellifera ligustica): An analysis of individual performance. PloS One. 2014; 9, e101262 10.1371/journal.pone.0101262 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Varnon CA, Dinges CW, Black TE, Wells H, Abramson CI. Failure to find ethanol-induced conditioned taste aversion in honey bees (Apis mellifera L.). Alcohol Clin Exp Res. 2018; 42(7), 1260–1270. 10.1111/acer.13761 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Giurfa M, Eichmann B, Menzel R. Symmetry perception in an insect. Nature. 1996; 382: 458–461. 10.1038/382458a0 [DOI] [PubMed] [Google Scholar]
  • 30.Giurfa M, Zhang S, Jenett A, Menzel R, Srinivasan M. V. The concepts of ‘sameness’ and ‘difference’ in an insect. Nature. 2001; 410: 930–933. 10.1038/35073582 [DOI] [PubMed] [Google Scholar]
  • 31.Gross HJ, Pahl M, Si A, Zhu H, Tautz J, Zhang S. Number-based visual generalisation in the honeybee. PLoS ONE. 2009; 4: e4263 10.1371/journal.pone.0004263 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Howard SR, Avarguès-Weber A, Garcia JE, Greentree AD, Dyer AG. Numerical cognition in honeybees enables addition and subtraction. Science Advances. 2019; 5: eaav0961 10.1126/sciadv.aav0961 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Hammer M, Menzel R. Learning and memory in the honeybee. J Neurosci. 1998; 15: 1617–1630. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Müller U. The molecular signaling processes underlying olfactory learning and memory formation in honeybees. Apidologie. 2012; 43: 322–333. [Google Scholar]
  • 35.Eisenhardt D. Molecular mechanisms underlying formation of long-term reward memories and extinction memories in the honeybee (Apis mellifera). Lean Mem. 2014; 21: 534–542. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Abramson CI. Aversive conditioning in honey bees (Apis mellifera). J Comp Psychol. 1986; 100(2): 108–116. [DOI] [PubMed] [Google Scholar]
  • 37.Vergoz V, Roussel A, Sandoz J-C, Giurfa M. Aversive learning in honey bees revealed by the olfactory conditioning of the sting extension reflex. PloS ONE. 2007; 2(3): e288 10.1371/journal.pone.0000288 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Dinges CW, Avalos A, Abramson CI, Craig DPA, Austin ZM, Varnon CA, et al. Aversive conditioning in honey bees (Apis mellifera anatolica): A comparison of drones and workers. J Exp Biol. 2013; 216, 4124–4134. 10.1242/jeb.090100 [DOI] [PubMed] [Google Scholar]
  • 39.Dinges CW, Varnon CA, Cota LD, Slykerman SC, Abramson CI. Studies of learned helplessness in honey bees (Apis mellifera L.). J Exp Psychol: Animal Learn Cogn. 2017; 43: 147–158. [DOI] [PubMed] [Google Scholar]
  • 40.Kirkerud NH, Schlegel U, Galizia CG. Aversive learning of colored lights in walking honeybees. Front Behav Neurosci. 2017; 11: 1–17. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Marchal P, Villar ME, Geng H, Arrufat P, Combe M, Viola H, et al. Inhibitory learning of phototaxis by honeybees in a passive-avoidance task. Learn Mem. 2019; 26: 412–423. 10.1101/lm.050120.119 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Schott M, Klein B, Vilcinskas A. Detection of illicit drugs by trained honeybees (Apis mellifera). PLoS ONE. 2015; 10(6): e0128528 10.1371/journal.pone.0128528 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Agarwal M, Giannoni-Guzmán M, Morales-Matos C, Del Valle Díaz RA, Abramson CI, Giray Tugril. Dopamine and octopamine influence avoidance learning of honey bees in a place preference assay. PLoS ONE. 2011; 6(9): e25371 10.1371/journal.pone.0025371 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Plath JA, Entler BV, Kirkerud NH, Schlegel U, Galizia CG, Barron AB. Different roles for honey bee mushroom bodies and the central complex in visual learning of colored lights in an aversive conditioning assay. Front Behav Neurosci. 2017; 11: 1–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Urlacher E, Francés B, Giurfa M, Devaud J. An alarm pheromone modulates appetitive olfactory learning in the honeybee (Apis mellifera). Front Behav Neurosci. 2010; 4: (157), 1–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Ichikawa N, Sasaki M. Importance of social stimuli for the development of learning capability in honeybees. Appl Entomol Zool. 2003; 38: 203–209. [Google Scholar]
  • 47.Farina WM, Grüter C, Díaz PC. Social learning of floral odours inside the honeybee hive. Proc R Soc Lond [Biol]. 2005; 272: 1923–1928. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Avalos A, Pérez E, Vallejo L, Pérez ME, Abramson CI, Giray T. Social signals and aversive learning in honey bee drones and workers. Biol Open. 2017; 6: 41–49. 10.1242/bio.021543 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Rossi N, d’Ettorre P, Giurfa M. Phermones modulate responsiveness to a noxious stimulus in honey bees. J Exp Biol. 2018; 221: jeb172270. [DOI] [PubMed] [Google Scholar]
  • 50.Varnon CA, Abramson CI. The propeller experiment controller: Automation for the comparative analysis of behavior in research and teaching. J Mind Behav. 2018; 39 (1–2): 1–153. [Google Scholar]
  • 51.Varnon CA, Abramson CI. The propeller experiment controller: low-cost automation for classroom experiments in learning and behavior. Comprehensive Psychology. 2013; 2, 1–18. [Google Scholar]
  • 52.Leadbeater E, Chittka L. The dynamics of social learning in an insect model, the bumblebee (Bombus terrestris). Behav Ecol Sociobiol. 2007; 61: 1789–1796. [Google Scholar]
  • 53.Kawaguchi LG, Ohashi K, Toquenaga Y. Contrasting responses of bumble-bees to feeding conspecifics on their familiar and unfamiliar flowers. Proc R Soc B. 2007; 274: 2661–2667. 10.1098/rspb.2007.0860 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Kawaguchi LG, Ohashi K, Toquenaga Y. Do bumble bees save time when choosing novel flowers by following conspecifics? Funct Ecol. 2006; 20: 239–244. [Google Scholar]
  • 55.Hardin J, Hilbe JM. Generalized Estimating Equations. Boca Raton, FL: Chapman & Hall; 2003. [Google Scholar]
  • 56.Mustard JA, Edgar EA, Mazade RE, Wu C, Lillvis JF, Wright A. Acute ethanol ingestion impairs appetitive olfactory learning and odor discrimination in the honey bee. Neurobiol Learn Mem. 2008; 90: 633–643. 10.1016/j.nlm.2008.07.017 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Riddell CE, Mallon EB. Insect psychoneuroimmunology: immune response reduces learning in protein starved bumblebees (Bombus terrestris). Brain Behav Immune. 2005; 20: 135–138. [DOI] [PubMed] [Google Scholar]
  • 58.Perktold J, Seabold S, Taylor J. StatsModels. Version 0.9.0 [Computer software]. 2018; http://www.statsmodels.org.
  • 59.Anaconda. Anaconda Software Distribution. Version 4.6.14 [Computer software]. 2019; http://www.anaconda.com.
  • 60.Clogg CC, Petkova E, Haritou A. Statistical methods for comparing regression coefficients between models. Am J Sociol. 1995; 100: 1261–1293. [Google Scholar]
  • 61.Paternoster R, Brame R., Mazerolle P, Piquero A. Using the correct statistical test for the equality of regression coefficients. Criminology. 1998; 36(4): 859–866. [Google Scholar]
  • 62.Grice JW. Observation oriented modeling: Analysis of cause in the behavioral sciences. Cambridge, MA: Academic Press; 2011. [Google Scholar]
  • 63.Craig DPA, Abramson CI. Ordinal pattern analysis in comparative psychology—A flexible alternative to null hypothesis significance testing using an observation oriented modeling paradigm. J Comp Psychol. 2018; 31: 1–21. [Google Scholar]
  • 64.Grice JW. From means and variances to persons and patterns. Front Physiol. 2015; 6: 1–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 65.Grice JW, Barrett PT, Schlimgen LA, Abramson CI. Toward a brighter future for psychology as an observation oriented science. Behav Sci. 2012; 2: 1–22. 10.3390/bs2010001 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 66.Grice JW, Yepez M, Wilson NL, Shoda Y. Observation-oriented modeling: going beyond “Is it all a matter of chance”? Educ Psychol Meas. 2016; 77: 855–867. 10.1177/0013164416667985 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 67.Beghetto RA, Toward avoiding an empirical march to nowhere. Psychol Aesthet Creat Arts 2014; 8: 18–20. [Google Scholar]
  • 68.Fidler F, Burgman MA, Cumming G, Buttrose R, Thomason N. Impact of criticism of null‐hypothesis significance testing on statistical reporting practices in conservation biology. Conserv Biol. 2006. 20: 1539–1544. 10.1111/j.1523-1739.2006.00525.x [DOI] [PubMed] [Google Scholar]
  • 69.Gigerenzer G. Mindless statistics. J Soc Econ. 2004; 33: 587–606. [Google Scholar]
  • 70.Skinner BF. Cumulative record. Appleton-Century- Crofts, New York; 1972. [Google Scholar]
  • 71.Woodside AG. Diffusion and adoption of good science: Overcoming the dominant logic of NHST and the reporting of rubbish. J Bus Mark Manag. 2016; 23: 327–333. [Google Scholar]
  • 72.Grice JW. Observation Oriented Modeling [Computer Software]. 2011; http://www.idiogrid.com/OOM.
  • 73.Amaya-Márquez M, Abramson C, Wells H. Use of flower color-cue memory by honey bee foragers continues when rewards no longer differ between flower colors. J Insect Behav. 2017; 30: 728–740. [Google Scholar]
  • 74.Morawetz L, Svoboda A, Spaethe J, Dyer AG. Blue colour preference in honeybees distracts visual attention for learning closed shapes. J Comp Physiol A. 2013; 199: 817–827. [DOI] [PubMed] [Google Scholar]
  • 75.Scienza L, Pinheiro de Carvalho M, Machado A, Moreno AM, Biscassi N, de Souza Gracas. Simple discrimination in stingless bees (Melipona quadrifasciata): Probing for select–and reject–stimulus control. J Exp Anal Behav. 2019; 112, 1–14. [DOI] [PubMed] [Google Scholar]
  • 76.Riveros J, Gronenberg W. Decision-making and associative color learning in harnessed bumblebees (Bombs impatiens). Anim Cogn. 2012; 15: 1183–1193. 10.1007/s10071-012-0542-6 [DOI] [PubMed] [Google Scholar]
  • 77.Black TE, Fofah O. Giray T, Wells H, Le Conte Y, Abramson CI. Influence of environmental experience on aversive conditioning in honey bees (Apis mellifera L.). Apidologie. 2018; 49: 5, 647–659. [Google Scholar]
  • 78.Couvillon MJ, Roy GF, Ratnieks FLW. Recognition errors by honey bee (Apis mellifera) guards demonstrate overlapping cues in conspecific recognition. J Apicult Re. 2009; 48(4): 225–232 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 79.Markwell TJ, Kelly D, Duncan KW. Compeition between honey bees (Apis mellifera) and wasps (Vespula spp.) in honeydew beech (Nothofagus solandri var. solandri) forest. New Zealand Ecological Society. 1993; 17: 85–93. [Google Scholar]
  • 80.Sun Q. Zhou X. Corpse management in social insects. Int J Biol Sci. 2013; 9(3): 313–321. 10.7150/ijbs.5781 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 81.Nouvian M, Galizia CG. Aversive training of honey bees in an automated Y-maze. Front Physiol. 2019;10: 678 10.3389/fphys.2019.00678 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 82.Kimura T, Ohashi M, Crailsheim K, Schmickl T, Okada R, et al. Development of a new method to track multiple honey bees with complex behaviors on a flat laboratory arena. PLoS ONE. 2014; 9: e84656 10.1371/journal.pone.0084656 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 83.Thompson RF, Spencer WA. Habituation: A model phenomenon for the study of neuronal substrates of behavior. Psychol Rev. 1966; 73: 16–43. 10.1037/h0022681 [DOI] [PubMed] [Google Scholar]
  • 84.Groves PM, Thompson RF. Habituation: A dual-process theory. Psychol Rev. 1970; 77: 419–450. 10.1037/h0029810 [DOI] [PubMed] [Google Scholar]
  • 85.Rankin CH, Abrams T, Barry RJ, Bhatnager S, Claytong D, Colombo J, et al. Habituation revisited: An updated and revised description of the behavioral characteristics of habituation. Neurobiol Learn Mem. 2009; 92: 135–138. 10.1016/j.nlm.2008.09.012 [DOI] [PMC free article] [PubMed] [Google Scholar]

Decision Letter 0

James C Nieh

28 Oct 2019

PONE-D-19-27434

Conspecific and intraspecific stimuli reduce aversive learning in honey bees (Apis mellifera)

PLOS ONE

Dear Dr. Varnon,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

In particular, please address the statistical concerns of the reviewers, especially the concerns of reviewer 1 on sample size. 

We would appreciate receiving your revised manuscript by Dec 12 2019 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

James C. Nieh, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

1. 

When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2.  In your Methods section, please provide additional location information of the collections site, including geographic coordinates for the data set if available.

3.  In your Methods section, please provide additional information regarding the permits you obtained for the work. Please ensure you have included the full name of the authority that approved the collection site access and, if no permits were required, a brief statement explaining why.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: General points and major concerns:

This is a generally well-conducted, and thoroughly analyzed study that addresses a gap in the literature on honey bee aversive conditioning -- in particular, the ability of honey bees to use members of their own species or another species as a cue to facilitate learning. Given that honey bees are eusocial, it is reasonable to predict that individual bees should be attentive to the presence of a conspecific, and should therefore be capable of using this type of stimulus as a cue; however, what the authors show is somewhat more complex. They claim that, although the stimulus organism was noticed, the presence of another organism seems to have decreased performance, especially at the beginning of a trial, when subjects often performed below chance level. This is puzzling, given that what constituted "correct" was different across conditions -- so would have to mean that sometimes, the subject got closer to the stimulus animal, sometimes, further away (sometimes one side of the box, sometimes the opposite side), but that they seem to have been drawn toward making an incorrect choice in all cases.

The biggest limitation of the present study is the large number of groups (14) and time bins (18) employed, which would necessitate a larger sample size than the one the authors used -- particularly if the expected effect size is small. I am also concerned that the authors seem to have failed to control for the multiple comparisons they conducted. Since any effects observed appear to be rather small, it is understandable that the authors would not have wished to further diminish the probability of obtaining a significant effect; however, what comes out of the various analyses is often difficult to interpret, seeming to fluctuate randomly. Personally, I would need further assurances from the authors that any patterns seen in the present study are truly indistinguishable from noise.

Specific points:

Page 4, line 9, I think you meant "single" instead of signal.

Page 5, line 123, I am not sure I undestand. Do you mean that there are two bees simultaneoulsy in the same chamber? If so, would you not expect that to interact with whatever effect the conspecifics and interspecific animals had?

Another question about the shuttle box: you mention that the subjects were able to physically interact with the stimulus organisms on the other side of the mesh, but did you observe them doing this? How close were the stimuli, especially the dead stimuli, to the mesh? Is it possible

Page 23, line 439, I think a word is missing in the header.

You explain you small and lack of effects in the context of biases established during foraging. What would you predict would happen if you used newly-emerged bees instead of experienced foragers?

Throughout: I would avoid using the term "trend" unless you are referring to an actual trend analysis.

Reviewer #2: General comments:

In the manuscript entitled “Conspecific and intraspecific stimuli reduce aversive learning in honey bees (Apis mellifera)” (PONE-D-19-27434), the authors describe a series of experiments assessing if of honey bees can use conspecific (a bee from the same hive) or interspecific (a wasp) as cues for a place preference assay in a shuttle box. For these experiments the authors measured two variables: the “correct compartment restriction (CCR), indicating the preference of the bees, at a given time, for one or the other side of the shuttle box, and the “activity”, which indicates the number of time a bee passes from one side to the other of the shuttle box. In particular, the authors performed several experimental and control groups to test several possible effects of conspecific and interspecific cues on an aversive place preference learning task. The data are presented in a clear graphs and supplementary material figures present a graphical representation of the raw data. Along the paper, the authors proceed to a systematic description of results that makes it easy to understand the many analyses they performed. Addionally, the analysis of each experiment in three steps (visual description, regression statistics, ordinal statistics) is appreciated and allows the authors to not be constrained to only discuss the results in terms of statistical difference and p-values. Finally, the statistics used in this study are extensively explained in the material and methods section, which allows for a good understanding of the analyses.

However, some general and more specific aspects of the manuscript could be improved. On a general level, the bibliography cited in the manuscript is in several occasions rather dated or incomplete (see specific comments). The graphs could be improved with colors and error bars (the error bars could represent a confidence interval already computed in the “bootstrapping” ordinal analysis) for better visualization of the experimental data. Also, and most importantly, the authors seem to confuse the preference for a side or another (CCR) at a given time, and the evolution of the CCR value across time (change that can be a proxy of learning the conditioning rules of the experiment). This confusion is present at several places in the manuscript and should be remedied (see specific comments).

On the whole, despite some corrections needed to improve or correct the manuscript, this study has enough strength to be accepted for publication.

Specific comments:

- Throughout the manuscript, the interspecific group is sometimes mistakenly labelled intraspecific (title, introduction, etc…)

- Line 26, authors could mention that the experiment consists in a spatial conditioning, as other studies of the same kind in honey bees usually use visual or olfactive conditioning instead.

- In this study authors conclude that learning is similar in conspecific, interspecific and spatial groups (see Table S2; Spatial X bin : Other groups X Bin). However, in the title and abstract, they state several times that social cues reduced learning. In fact, social cue only generally reduced CCR, but doesn’t impair its improvement (learning). This should absolutely be remedied.

- Line 51, citations for nestmate and non-nestmate recognition and discrimination studies in the honey bee seem a bit dated. Maybe some other studies were published more recently and could be added.

- Line 58, the most recent review on the PER conditioning protocol is lacking from the citations and should be added (Matsumoto et al., 2012, J Neuro Methods)

- Line 61, a review article on visual conditioning in harnessed honey bees has been published recently and could be cited to illustrate the point (Avarguès-Weber and Mota 2016, J Physiol Paris)

- Line 62, recent publication by Howard and colleagues (science) have been published on numerical abilities in honey bees that would illustrate perfectly the point on quantity discrimination.

- Line 62, the most recent and up to date citations for the neurophysiology of memory in honey bees are (Müller 2012, Apidologie; Eisenhardt 2014, Learn & Mem).

- Line 65, to illustrate a broader spectrum of aversive conditioning laboratory studies, two citations could be added (Kirkerud et al., 2017; Marchal et al., 2019, Learn & Mem).

- Line 70, the list of the themes tackled using an aversive shuttle box kind of experimental setup could be widened citing Marchal et al., 2019 with the study of modulation of phototaxis.

- Line 71, Marchal et al., 2019 also qualify as a reference on the role of dopamine, octopamine and mushroom bodies in aversive learning.

- Line 72-81, the authors elaborate on the lack of studies on the modulation of aversive learning by social cues. This section could be improved by adding a statement concerning the studies that exist on shock responsiveness modulation by pheromones in honey bees, which might be interesting to discuss in the framework of this manuscript (Rossi et al., 2018, JEB).

- Line 123, the authors describe the setup as being able to perform the conditioning of two independent animals in parallel. How was this capacity used? Were animals of the same groups performed in parallel? Or on the opposite, were animals of different groups performed in parallel?

- The Apparatus section of the Material and Methods could benefit from a visual representation of the apparatus (that seem to be slightly different from the one used in previously published studies) to help understanding.

- Line 148, typo: Controlled should be replaced by Controller.

- Line 169-170, the description of the apparatus using the terms top half and bottom half is misleading as the terms top and bottom were previously used to describe what could be called floor and ceiling of the apparatus. This highlights again the improvement that a visual on the apparatus could ease understanding.

- Line 170, the text seems to suggest that a side of the apparatus was always associated to yellow and the other side to blue, indicating that the sides were not balanced with respect to the color stimuli. If this is the case, it might imply a spatial bias in the experiment.

- Line 227, contrasting with the rest of the statistical analysis description, the part explaining the z-score of activity might be missing clarity. It is not clear with respect to what mean value the z-scores are computed. Is it to the mean of all groups of a same experiment at a given bin? Is it to the mean of all bins of a single group?

- Line 296, the authors state that “the bias appears to dissipate” because the CCR is improving across bins. However, the increase is probably due to learning, while the bias is still present.

- Line 314, the low CCR bees in the color discrimination groups show low activity too, which means that the bees stayed in the shocked compartment (keeping receiving shocks), which might indicate that those bees were not feeling the shock, or behaviorally impaired. This possibility could be discussed.

- Line 367, “effect” could be replaced by “activity”.

- Line 368, “illustrate” should be replaced by “illustrating”.

- Line 382, the study does not seem to specifically test a general preference (all bins pulled) towards stimulus bee or stimulus wasp which could be interesting to assess (as a post hoc analysis for example).

- Line 435, “High” should be added before “CCR”.

- Line 448, as the conspecific and interspecific groups are compared to the spatial group, it might improve understanding of the data to make the spatial group appear on all conspecific and interspecific graphs for visual comparison.

- Line 488, what does the expression “initially reduces learning” mean? Learning being a change in behavior across time, the term initial learning does not seem to fit. This could be replaced by “reduces the overall CCR level”.

- Line 489, the authors state that the spatial group reached a final level of 66%. However, to be precise, this value is extracted from the model that fitted the data, not the measured behavior of the animals. The final CCR level that the spatial group reached is slightly lower and should be the one reported here.

- Line 498, stating that the study has been performed with the same experimental set up and protocol would strengthen the argument.

- line 503, the authors argue that the low CCR (compared to the spatial group) of bees in the shock on blue and shock on yellow groups is likely due to prior experience of bees on colored flower. However, if both groups were performed in parallel, and given the fact that they were exposed to blue and yellow cues at the same time in both groups, a bias would logically only make them learn better in one case or in the other (with a constant bias towards one of the colors). However, the experiment shows a lower CCR of both those groups compared to the spatial group. Hence, the bias resulting from prior experience is not likely to explain the low CCR. However, authors also demonstrate that the spatial and shock on yellow groups learn equally (see fig 2: Spatial X bin : Shock on yellow X bin) while shock on blue group do not learn. This might be due to a bias resulting from more experience feeding on blue than yellow flowers, hence difficulties to associate the blue color to an aversive stimulus.

- Line 523, the statement about sensitization and habituation is not clear.

- Line 526, are paper wasps known antagonists of honey bees? If not, there might not be any reason to expect an antagonistic behavior with paper wasps.

- Line 526, syntax.

- Line 540, the authors describe an alternative punishment method without much precision. This could be elaborated further for better understanding.

- Line 565, “collecting” should be replaced by “collected”.

- Line 576, authors should replace “overall learning” for “preference level”. The learning performances have been demonstrated to be similar.

- Line 587, as the intended negative controls (dead animal stimuli, compared to live ones) did show the same reduction of overall CCR compared to spatial group, this study would have benefitted from the use of additional controls to isolate the parameter that induces the reduction of CCR. For example, some studies use dummies to control for the possible chemical detection of the stimulus animal. In absence of such control (demonstrating no effect on behavior compared to the spatial group), one could argue that the measured effect (decrease in CCR levels) could be due to unintended experimental bias. This could be discussed somehow.

- Line 591, the earlier version of the visual APIS setup (Kirkerud et al., 2017) already used multiple IR beams to assess position and velocity of bees in the apparatus.

- Line 592, Marchal and colleagues, 2019 also used a video analysis method for precise honey bee tracking inside a visual aversive shuttle box.

- Line 592-594, the idea developed by the authors in this sentence would benefit from a more explicit statement in order to be understandable without having to read the cited paper.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Paul Marchal

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2020 Feb 25;15(2):e0228161. doi: 10.1371/journal.pone.0228161.r002

Author response to Decision Letter 0


11 Dec 2019

We are grateful for the insightful comments of the reviewers (see our acknowledgements). We have substantially revised our manuscript based on the comments of the editor and reviewers. We address specific comments in our response to reviewers document.

Attachment

Submitted filename: Response to Reviewers.docx

Decision Letter 1

James C Nieh

30 Dec 2019

PONE-D-19-27434R1

Conspecific and interspecific stimuli reduce initial performance in an aversive learning task in honey bees (Apis mellifera)

PLOS ONE

Dear Dr. Varnon,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please address the statistical concerns of Reviewer 1.

We would appreciate receiving your revised manuscript by Feb 13 2020 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

James C. Nieh, Ph.D.

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Although I appreciate the re-analysis of the data, and the fact that the authors have responded to several of my comments, I do not feel that some of the more important ones have been addressed sufficiently. For example, it should not be up to the reader to decide if and how to correct for multiple comparisons. However, this is a relatively minor quibble. My more major concern is that you are arguing for a particular interpretation of the data, and you should therefore back it up. The graphs alone, especially lacking any sort of error bar, do not tell a compelling enough story. Outside the difference between the spatial group and all other groups, and the general learning curve, I cannot see patterns clear enough in these data to warrant drawing definitive conclusions or dispensing with p-values, arbitrary as they may be. And the statistics you have provided do not strongly support your interpretation, either, in my opinion.

In my previous review, I said: "They claim that, although the stimulus organism was noticed, the presence of another organism seems to have decreased performance, especially at the beginning of a trial, when subjects often performed below chance level. This is puzzling, given that what constituted "correct" was different across conditions -- so would have to mean that sometimes, the subject got closer to the stimulus animal, sometimes, further away (sometimes one side of the box, sometimes the opposite side), but that they seem to have been drawn toward making an incorrect choice in all cases." I was unable to find a place in the text or the response that dealt with this issue.

Ultimately, as I said in my first review, I think this was a well-conducted (and no doubt time-consuming) study; however, I do not believe the data, as they are presented in their current form, are strong enough to justify publication.

Reviewer #2: The revised version of the manuscript is, in my opinion, totally satisfactory. The authors addressed all major and minor comments and suggestions from both reviewers in a comprehensive and positive manner that led to the improvement of the manuscript.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Paul Marchal

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2020 Feb 25;15(2):e0228161. doi: 10.1371/journal.pone.0228161.r004

Author response to Decision Letter 1


7 Jan 2020

We appreciate the efforts of the reviewers, especially in regard to speed. We have revised the manuscript and we respond to specific comments below in the response to reviewers document.

Attachment

Submitted filename: Response to Reviewers.docx

Decision Letter 2

James C Nieh

9 Jan 2020

Conspecific and interspecific stimuli reduce initial performance in an aversive learning task in honey bees (Apis mellifera)

PONE-D-19-27434R2

Dear Dr. Varnon,

We are pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it complies with all outstanding technical requirements.

Within one week, you will receive an e-mail containing information on the amendments required prior to publication. When all required modifications have been addressed, you will receive a formal acceptance letter and your manuscript will proceed to our production department and be scheduled for publication.

Shortly after the formal acceptance letter is sent, an invoice for payment will follow. To ensure an efficient production and billing process, please log into Editorial Manager at https://www.editorialmanager.com/pone/, click the "Update My Information" link at the top of the page, and update your user information. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, you must inform our press team as soon as possible and no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

With kind regards,

James C. Nieh, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Acceptance letter

James C Nieh

15 Jan 2020

PONE-D-19-27434R2

Conspecific and interspecific stimuli reduce initial performance in an aversive learning task in honey bees (Apis mellifera)

Dear Dr. Varnon:

I am pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

For any other questions or concerns, please email plosone@plos.org.

Thank you for submitting your work to PLOS ONE.

With kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. James C. Nieh

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 Table. Experimental groups CRR analysis—Group effects.

    (DOCX)

    S2 Table. Experimental groups CRR analysis—Bin interactions.

    (DOCX)

    S3 Table. Experimental groups activity analysis—Group effects.

    (DOCX)

    S4 Table. Experimental activity CRR analysis—Bin interactions.

    (DOCX)

    S5 Table. Experimental groups CRR analysis—Group effects.

    (DOCX)

    S6 Table. Experimental groups CRR analysis—Bin interactions.

    (DOCX)

    S7 Table. Experimental groups activity analysis—Group effects.

    (DOCX)

    S8 Table. Experimental activity CRR analysis—Bin interactions.

    (DOCX)

    S1 Fig. Heat map of CCR and activity levels of the spatial group.

    (DOCX)

    S2 Fig. Heat map of CCR and activity levels of the shock on blue group.

    (DOCX)

    S3 Fig. Heat map of CCR and activity levels of the shock on yellow group.

    (DOCX)

    S4 Fig. Heat map of CCR and activity levels of the bee baseline group.

    (DOCX)

    S5 Fig. Heat map of CCR and activity levels of the wasp baseline group.

    (DOCX)

    S6 Fig. Heat map of CCR and activity levels of the bee and wasp baseline group.

    (DOCX)

    S7 Fig. Heat map of CCR and activity levels of the safe by live bee group.

    Note that one extreme activity score was truncated to 25 for sake of consistency across graphs.

    (DOCX)

    S8 Fig. Heat map of CCR and activity levels of the shock by live bee group.

    Note that one extreme activity score was truncated to 25 for sake of consistency across graphs.

    (DOCX)

    S9 Fig. Heat map of CCR and activity levels of the safe by dead bee group.

    (DOCX)

    S10 Fig. Heat map of CCR and activity levels of the shock by dead bee group.

    (DOCX)

    S11 Fig. Heat map of CCR and activity levels of the safe by live wasp group.

    Note that five extreme activity scores were truncated to 25 for sake of consistency across graphs.

    (DOCX)

    S12 Fig. Heat map of CCR and activity levels of the shock by live wasp group.

    (DOCX)

    S13 Fig. Heat map of CCR and activity levels of the safe by dead wasp group.

    (DOCX)

    S14 Fig. Heat map of CCR and activity levels of the shock by dead wasp group.

    Note that one extreme activity score was truncated to 25 for sake of consistency across graphs.

    (DOCX)

    S1 Data

    (CSV)

    S1 Ordinal analysis code

    (PY)

    Attachment

    Submitted filename: Response to Reviewers.docx

    Attachment

    Submitted filename: Response to Reviewers.docx

    Data Availability Statement

    All relevant data are within the manuscript and its Supporting Information files.


    Articles from PLoS ONE are provided here courtesy of PLOS

    RESOURCES