Neural divergence and hybrid disruption between ecologically isolated Heliconius butterflies

Significance

Although previous studies highlight the importance of ecological variation in driving divergence in brain morphology, whether these changes accumulate after ecological transitions or play a significant role in facilitating them is unclear. We show that divergent selection associated with microhabitat partitioning drove heritable divergence in brain composition and neural gene expression between sister clades of Heliconius butterflies. Neuroanatomical divergence is restricted to visual structures and reflect adaptations to contrasting sensory niches. We show how divergent neural traits can act as barriers to gene flow by demonstrating that hybrids have intermediate traits and by presenting evidence that genes with adaptive patterns of expression divergence are more likely to occur in regions of the genome that are resistant to interspecific gene flow.

Abstract

The importance of behavioral evolution during speciation is well established, but we know little about how this is manifest in sensory and neural systems. A handful of studies have linked specific neural changes to divergence in host or mate preferences associated with speciation. However, the degree to which brains are adapted to local environmental conditions, and whether this contributes to reproductive isolation between close relatives that have diverged in ecology, remains unknown. Here, we examine divergence in brain morphology and neural gene expression between closely related, but ecologically distinct, Heliconius butterflies. Despite ongoing gene flow, sympatric species pairs within the melpomene–cydno complex are consistently separated across a gradient of open to closed forest and decreasing light intensity. By generating quantitative neuroanatomical data for 107 butterflies, we show that Heliconius melpomene and Heliconius cydno clades have substantial shifts in brain morphology across their geographic range, with divergent structures clustered in the visual system. These neuroanatomical differences are mirrored by extensive divergence in neural gene expression. Differences in both neural morphology and gene expression are heritable, exceed expected rates of neutral divergence, and result in intermediate traits in first-generation hybrid offspring. Strong evidence of divergent selection implies local adaptation to distinct selective optima in each parental microhabitat, suggesting the intermediate traits of hybrids are poorly matched to either condition. Neural traits may therefore contribute to coincident barriers to gene flow, thereby helping to facilitate speciation.

Ecological adaptation is a major force driving the evolution of new species (1, 2). Although it is well established that divergent selection can influence behavioral traits and promote speciation (3), there are few empirical examples of how divergent selection acts on the underlying sensory and neural systems. For example, existing studies on adaptation across divergent light regimes have largely focused on the peripheral sensory systems, often in the context of divergent mate preference (4, 5). However, sensory perception is only the first of many mechanisms within the nervous system that may experience divergent selection, and mating preferences are only one of many behaviors that can be affected by the environment, and only one of many behaviors that may contribute to reproductive isolation. Behavioral challenges imposed by novel environmental conditions can instead be met by changes in how sensory information is processed, often reflected in differential investment in brain components that refine the sensitivity, acuity to, or integration of, different stimuli.

The intimate relationship between brain structure and ecology is apparent in many comparative studies of neuroanatomy. For example, the expansion of visual pathways in primates (6), the cerebellar expansion and refinement of the exterolateral nucleus in electric fish (7–9), the contrasting adaptations of diurnal and nocturnal lifestyles in hawk moths (10), and changes in sensory brain investment during independent colonizations of cave systems that underlie the radiation in Mexican cavefish (11), all indicate the importance of neuroanatomical adaptations to contrasting ecological needs. However, with some exceptions (SI Appendix, Table S1), comparative studies of brain evolution generally focus on phylogenetic comparisons across relatively distantly related species that encompass a large range of morphological and ecological variation. More recently, several studies have also investigated intraspecific variation in neural traits across populations, or between eco-morphs, and highlight the potential for plasticity in brain development to optimize brain structure and function to local conditions (12–14).

Between these population and phylogenetic levels there is a scarcity of information about the role brains play in facilitating speciation across environmental gradients, either through developmental plasticity or the accumulation of heritable changes during ecological divergence (SI Appendix, Table S1). Hence, whether evolutionary changes in neural systems contribute to the evolution of reproductive isolation during ecological divergence (15), or accumulate over longer time frames after reproductive isolation is complete, is unknown. A handful of insect studies have linked specific changes in neural processing to the evolution of reproductive isolation among close relatives; however, these specifically focus on divergent host preferences and the detection of host cues (16–22). Whether brains respond to changes in broader features of the environment, such as luminance or habitat structure, at a similar timescale is yet to be established. Recently, studies of closely related populations on the path to speciation have begun to address this question (11, 13, 23, 24) (summarized in SI Appendix, Table S1). Importantly, however, these studies are often unable to disentangle the effects of drift and selection, and have not determined whether hybrids between ecologically distinct populations show intermediate brain morphologies that may reflect major fitness deficits, which would support a more causative role for divergence in neural systems during the incipient stages of speciation.

Here, we investigate the role of heritable divergence in neuroanatomy and gene expression in a clade of closely related Heliconius butterflies. Heliconius are well known for their bright warning patterns and Müllerian mimicry (25, 26). Speciation within Heliconius is also often associated with ecological transitions (27–30), and habitat partitioning among sister taxa is generally required for complete speciation (28, 31). In particular, within the melpomene–cydno clades, coexisting species are often found in “mosaic sympatry,” with sister taxa inhabiting relatively open forest edge, or closed canopy forest, respectively (31–33). These species are ecologically distinct and mate assortatively, but at a population level continue to exchange alleles, implying speciation is incomplete (34). These environmental differences are associated with changes in light environment, with cydno clade taxa occupying forest with lower light levels (34). Wild caught Heliconius melpomene/cydno show evidence of divergence in peripheral eye structure and light sensitivity (35, 36), suggesting a potential response to the contrasting light environments. Indeed, across a range of taxa, transitions between dark/light conditions are consistently associated with changes in brain morphology that are assumed to be adaptive (6, 10, 11). We therefore hypothesized that differences in habitat use between melpomene and cydno clade taxa have imposed distinct sensory challenges throughout the evolution of reproductive isolation, leading to consistent, divergent changes in brain structure and function.

To test this hypothesis, we collected data to examine the evolution of brain morphology at two levels of divergence: population-level variation among melpomene and cydno clade taxa, and divergence between these two clades. Using these data, we test the following predictions that are expected under adaptive divergence: 1) there will be a consistent signal of divergent brain morphologies between melpomene and cydno clade taxa; 2) these differences should be heritable, not induced by the environment; and 3) divergence should exceed rates expected by neutral evolution. We then complement these data with neural transcriptomes from common-garden–reared individuals to test whether divergence has also occurred in processes that cannot be captured by volumetric data, and whether this can also be explained by divergent selection. Finally, we replicated these data in first-generation hybrids to test the hypothesis that hybrid offspring show intermediate traits and expression profiles. Combined with evidence of divergent selection, which would suggest that melpomene and cydno brains are adapted to distinct fitness optima, intermediacy in hybrid neural traits would imply that they fall outside both parental optima, likely being maladaptive.

Results and Discussion

Divergence in Neuroanatomy in the H. Melpomene–Cydno Complex.

To investigate the effects of ecological divergence on brain morphology within the melpomene–cydno complex, we sampled butterflies from Costa Rica, Panama, Peru, and French Guiana (Fig. 1). Where members of the melpomene and cydno clades are sympatric, the species boundary is maintained by ecological divergence and disruptive selection against hybrids, which now occur at low frequencies (37).

Fig. 1.

Population sampling and ecological divergence. (A) Outline map of Central and South America showing the range of the cydno clade (blue), the melpomene clade (red), and their overlap (brown). Circles indicate sampled populations in Costa Rica, Panama, Peru, and French Guiana. In the Andes, the cydno clade species, H. timareta, is restricted to high elevations but overlaps with H. melpomene at its lower margins. Green Passiflora sp. leaves indicate oligophagous populations that are host-plant specialists; orange leaves indicate polyphagous populations that lay on multiple Passiflora species. Populations included in the common-garden experiments are shown in bold. (B) Illustration of niche partitioning between melpomene (red; open forest, forest edge) and cydno (blue; closed forest).

Across the two clades, while controlling for population effects, the average volume of the combined optic lobe neuropils (OL) is significantly larger in cydno clade species (including H. cydno, Heliconius pachinus, and Heliconius timareta) than H. melpomene (n = 77, Χ² = 17.354, P < 0.001). This difference is not explained by allometric scaling (y axis shift in OL ∼ unsegmented central brain neuropil [rCBR]: Χ² = 12.260, P < 0.001; Fig. 2C). Five of the six optic lobe neuropils are significantly larger in the cydno clade (Fig. 2 C, D, and I and SI Appendix, Table S4), with the sole exception being the lobula. For a given brain size, these five neuropils are between 13 and 27% larger in cydno, suggesting altered patterns of investment are unequal across structures. However, in each case, the increase is associated with grade shifts in allometric scaling (SI Appendix, Table S5). Across insects, these structures are vital for summation and parallelization of photoreceptor signals (38–40), and a diverse range of visual processes including color vision (41–43), shape and motion detection, maneuverability in flight (44, 45), and circadian rhythms (46). The ventral lobula (vLOB), which is only present in some butterflies (47–50), also acts as a relay center sending visual information to the mushroom body (50), a major site of insect learning and memory.

Fig. 2.

Divergence in brain morphology between H. melpomene and H. cydno clades. (A) Three-dimensional volumetric models of a Heliconius brain showing segmented neuropils from anterior (A) and posterior (A′) views; visual neuropils in yellows–oranges, antennal lobe in blue, the central complex in green, the mushroom bodies in red, and the unsegmented rCBR is transparent. Visual neuropils discussed in the main text are labeled as follows: aME, accessory medulla; AOTU, anterior optic tubercle; LAM, lamina; LOB, lobula; LOP, lobula plate; ME, medulla; vLOB, ventral lobula. (B) Three-dimensional volumetric models of a Heliconius brain showing segmented neuropils from anterior (A) and posterior (A′) views where blue neuropils are significantly different in size between H. melpomene and H. cydno clades, based on the lme4 results including all populations, with darker neuropils indicating higher significance. (Scale: in A/B, 500 μm.) (C) Grade shift in the scaling relationship between optic lobe (OL) and central brain (rCBR) volume between melpomene (red points) and cydno (blue points) clade taxa. (D–I) Nonallometric shifts in the size of individual visual neuropils between melpomene and cydno clade taxa; lamina (LAM), medulla (ME), accessory medulla (aME), ventral lobula (vLOB), lobula plate (LOP), and anterior optic tubercule (AOTu). C–I show results of the SMATR analysis, including all populations. α-shift indicates the significance of a grade shift in neuropil scaling, where ***P < 0.001.

The anterior optic tubercle (AOTU) is also 23% larger in the cydno-clade populations (Χ² = 10.050, P < 0.001). The AOTU is the most prominent optic glomerulus in the central brain and is involved in processing sky light and spectral cues, as well as polarized light, which is less abundant in inner forests (51–53). The finding that this structure is expanded in the cydno clade is consistent with forest-type–dependent variation in light regimes affecting visual processing pathways. Contrary to claims that there is a trade-off between investment in major insect visual and olfactory neuropils (54), we find no evidence of volumetric shifts in the antennal lobe (Χ² = 0.615, P = 0.615). Excluding the AOTU, no other central brain neuropil shows robust evidence for nonallometric expansion (SI Appendix, Tables S4 and S5). Divergence in brain structure is therefore restricted to neuropils associated with visual processing. As melpomene and cydno clade taxa occupy forest of different light intensities and physical structure (25, 31–33), differential investment in these neuropils likely reflects contrasting demands on visual processing. Consistent with this interpretation, previous data suggest H. cydno responds to lower intensities of light than H. melpomene (35).

Distinct Patterns of Intraclade Variation Reveal a Consistent Role of Ecology in Shaping Brain Morphology.

To further understand the origins of differential investment in visual neuropil, we next considered variation within the H. cydno and H. melpomene clades. Despite evidence of genetic substructuring (55), brain morphology was highly consistent across the four cydno-clade populations we sampled, with no neuropil showing significant geographic variation (SI Appendix, Table S4B). In contrast, we do find evidence of variation across populations of H. melpomene, both in total optic lobe volume (Χ² = 9.917, P = 0.007) and for several of the individual visual neuropils that differentiate the H. cydno and H. melpomene clades (SI Appendix, Tables S4B and S5B). These include the largest visual neuropil, the medulla (Χ² = 11.161, P = 0.004), and the AOTU (Χ² = 9.647, P = 0.008) (SI Appendix, Fig. S1). Post hoc analysis reveals that these results are not driven solely by a single divergent population (SI Appendix, Table S5C), raising the possibility that H. melpomene may occupy more visually heterogeneous habitats than H. cydno, and may be exposed to a greater range of local sensory conditions.

Despite greater variability within H. melpomene, comparisons between sympatric species pairs suggest a consistent pattern of divergent investment between melpomene and cydno clade populations. In Panama, H. m. rosina and H. c. chioneus are differentiated by total optic lobe volume (Χ² = 12.708, P < 0.001), with five of seven visual neuropils having larger volumes in H. cydno (SI Appendix, Table S4A). Similarly, in Peru, H. m. amaryllis and H. timareta thelxinoe vary in total optic lobe volume (Χ² = 6.773, P = 0.009) and the two largest visual neuropils, the medulla and lamina (SI Appendix, Table S4A). Given H. m. amaryllis and H. t. thelxinoe are comimics and do not appear to distinguish conspecifics using visual cues (56), the shift in visual investment is unlikely to be related to mate choice. In contrast, H. c. galanthus and H. pachinus, which are ecologically equivalent but geographically isolated across Costa Rica’s central valley (31, 57, 58), show no evidence of neuroanatomical divergence despite strong visual mate preferences (58), supporting the causative role of divergent ecologies (SI Appendix, Table S4A). Comparisons between H. m. melpomene, which is allopatric with respect to H. cydno, to all cydno populations also detect evidence of divergence in OL volume (Χ² = 4.974, P = 0.026) with levels of phenotypic divergence comparable to other melpomene populations (SI Appendix, Table S4A). This suggests an absence of strong character displacement for this trait.

Neuroanatomical Differences Are Heritable.

We next reared H. melpomene rosina and H. cydno chioneus under common-garden conditions to determine whether the variation we observe is heritable. As in our comparisons between wild-caught individuals, we observed a nonallometric expansion of the optic lobe in insectary-reared H. cydno relative to insectary-reared H. melpomene (33%; n = 20, Χ² = 11.363, P = 0.001; SI Appendix, Table S6). This was driven by volumetric increases ranging from 24 to 57% across specific visual neuropils in H. cydno, including five of the six structures that differed between wild-caught individuals (SI Appendix, Table S6). The most pronounced shifts were found in the lamina (57% larger, Χ² = 13.702, P < 0.001), vLOB (49%, Χ² = 6.359, P < 0.001), and AOTU (40%, Χ² = 21.749, P < 0.001). We found no evidence that the extent of divergence between H. cydno and H. melpomene was higher in wild-caught than common-garden individuals for any individual neuropil (SI Appendix, Table S6C). Differences in brain morphology therefore appear to have a substantial heritable component and are not the product of environmentally induced plasticity during development.

Neuroanatomical Divergence Is Likely Driven by Natural Selection.

Heritable trait divergence could be explained by neutral genetic drift or divergent selection. Allometric scaling among traits, where component sizes vary consistently with total size, is evidence for constraint on trait evolution (59–62), including on brain structure (63, 64). This suggests populations evolving under genetic drift should follow conserved allometric scaling relationships, as is typical among recently diverged taxa (61). In contrast, our observation of nonallometric variation of brain components, among both wild-caught and common-garden–reared individuals, strongly implicates divergent natural selection.

To explicitly test for selection, we calculated PST for variation in neuropil volumes in Panamanian H. m. rosina and H. c. chioneus raised under common-garden conditions. PST is a direct phenotypic analog of FST, which measures population differentiation relative to the total variance across populations (65). Comparisons between PST and FST can therefore be used as a formal test of divergent selection, where PST values that exceed genome-wide FST suggest greater phenotypic divergence than expected by neutral genetic divergence (65). After accounting for allometric effects, PST significantly exceeds genomewide FST (34) for total optic lobe size (adjusted P = 0.011), lamina (adjusted P = 0.006), medulla (adjusted P = 0.020), lobula (adjusted P = 0.016), vLOB (adjusted P = 0.005), and AOTU (adjusted P = 0.005), consistent with the action of divergent natural selection. Although inferences made from PST can be vulnerable to underlying assumptions regarding trait heritability (65), our results are robust across a broad range of quantitative genetic scenarios (SI Appendix, Table S8).

As a further test for selection acting across the melpomene–cydno complex, we performed a partial-Mantel test to assess whether pairwise divergence in brain morphology between wild populations is predicted by levels of neutral genetic divergence (FST). Here, we expect that the presence of divergent selection would erode the relationship between genetic distance and phenotypic divergence (66). In contrast, if selection is absent, we expect a linear relationship between trait divergence and genetic divergence, consistent with a strong phylogenetic signal. After allometric correction, only two neuropils, the antennal lobe and lobula, show patterns of divergence consistent with neutral expectations (SI Appendix, Table S8). The lack of association for any neuropils with divergent volumes between H. melpomene and H. cydno again implies our results are not explained by drift.

Together, evidence of 1) nonallometric divergence in brain structure, 2) between-species variation that significantly exceed neutral predictions under controlled environmental conditions, and 3) a lack of an association between phenotypic and genetic divergence across the melpomene–cydno complex, strongly implicates natural selection as the driving force behind the observed differences in neuroanatomical structures.

Neuroanatomical Evolution Is Mirrored by Shifts in Neural Gene Expression.

Volumetric changes in neuroanatomy are likely the result of differences in cell number or size, which may in turn reflect replicated or divergent neural circuitry. Shifts in neural physiology or activity are also behaviorally important but may not be captured in morphometric data. These differences, however, can potentially be captured in patterns of differential gene expression between species. We therefore also examined gene expression between age-matched, adult H. m. rosina and H. c. chioneus brain tissue, from individuals raised under common-garden conditions to control for environmental effects. After accounting for the influence of tissue composition (67), we still detect significant levels of interspecific divergence in expression profiles for age and environment-matched individuals, suggesting a degree of divergence in neural gene expression that is independent of divergence in brain morphology (see Fig. 4A and SI Appendix, Figs. S3–S5). This pattern is consistent across two independently collected datasets. Differentially expressed genes are enriched for molecular functions linked to cytoskeletal and transmembrane channel activities (SI Appendix, Table S10), consistent with changes in brain physiology being achieved through alterations of neuronal wiring or activity.

Fig. 3.

Intermediate brain morphology in H. m. rosina × H. c. chioneus F1 hybrids. (A) Variation in H. m. rosina (red), H. c. chioneus (blue), and hybrid (gray) brain morphology in a principal-component analysis of all segmented neuropils and rCBR. (B–E) Examples of neuropils with intermediate volumes in hybrids (B and C), or melpomene-like volumes (D and E) in F1 hybrids; lamina (LAM), ventral lobula (vLOB), medulla (ME), and anterior optic tubercule (AOTu).

Differential expression between species could, again, be explained by genetic drift, rather than divergent selection. We therefore applied the same approach used to test for divergent selection in morphological traits. Estimated PST exceeds genome-wide FST for 18.5% (305 of 1,647) of differentially expressed genes, strongly implicating divergent selection in driving at least some shifts in neural gene expression. Divergent selection between melpomene and cydno implies that these species occupy distinct selective optima. This could create a reproductive barrier by selecting against individuals that possess allelic variation poorly matched to the local environment, thereby reducing gene flow between species around loci under divergent selection. Consistent with this hypothesis, f_d, a measure of shared allelic variation that is used to infer barriers to gene flow (68), is negatively correlated with values of PST for neural gene expression, even after accounting for variation in local recombination rate (Χ² = 179.0, P < < 0.001). Previous genome-wide analyses have highlighted a highly heterogeneous pattern of genetic divergence between H. m. rosina and H. c. chioneus, with selection against gene flow acting across the genome (34, 68). This suggests that the species barrier is determined by multiple, polygenic traits. Because f_d, and by extension PST, is not clustered across the genome (68), our data are consistent with this inference. We therefore suggest that divergence in neural traits is, at least in part, shaping the landscape of genetic differentiation between H. m. rosina and H. c. chioneus.

Hybrids Show Evidence of Intermediate Traits that Deviate from Both Parental Conditions.

Reproductive isolation can result from a mismatch between intermediate hybrid phenotypes and the environment, such that hybrids suffer lower fitness in either parental environment (1, 2). To explore whether divergent brain structures might contribute to fitness deficits in hybrids, we produced multiple crosses between H. m. rosina and H. c. chioneus. We focus on first generation (F1) individuals, which account for a major proportion of natural Heliconius hybrids (37). In F1 hybrids, strong dominance patterns at genes under divergent selection could mean that individuals predominantly appear like one parental species, suggesting they may have similar fitness. In contrast, if polygenic traits have incomplete or mixed dominance patterns, F1s may have intermediate traits. This could disrupt behavioral performance leading to fitness deficits, potentially contributing to the evolution of reproductive isolation. Multivariate analysis of the seven visual neuropils reveals that hybrids indeed show intermediate brain morphologies (Fig. 3A and SI Appendix, Table S7). This intermediate state is the product of variable dominance effects on specific neuropil (Fig. 3 B–E and SI Appendix, Table S7). Four of the seven neuropils are significantly larger in H. cydno than F1 hybrids, but are not significantly different between F1s and H. melpomene (SI Appendix, Table S7B), suggesting that these are largely influenced by loci with melpomene-dominant alleles. In contrast, two neuropils, the lamina and vLOB, are significantly different between F1s and both parental species (Fig. 3 B and C and SI Appendix, Table S7B), implying incomplete or mixed dominance across multiple loci. Importantly, this mosaic pattern also leads to disrupted scaling relationships between some visual neuropil, which may affect the flow and integration of visual information in the brain (SI Appendix, Table S7 C and D and Fig. S2).

We observed a similar pattern in our neural transcriptomic data, where hybrids show intermediate patterns of gene expression (Fig. 4 and SI Appendix, Figs. S4 and S5). Focusing on genes that are differentially expressed between H. cydno and H. melpomene, F1 hybrids cluster outside the range of both parental species (Fig. 4A). As inferred for the visual neuropils, the expression of individual genes shows variable patterns of dominance (SI Appendix, Fig. S6): 36% of differentially expressed genes are “melpomene-like” in F1 hybrids, 21% are “cydno-like,” and 43% are statistically intermediate. Again, consistent with divergent selection playing a role in gene expression evolution, genes with intermediate expression in F1 hybrids show increased levels of PST (Χ² = 5825.9, P < <0.001), with a greater proportion (23%) of intermediate genes showing PST values in excess of genome-wide FST, compared to genes with melpomene-like (9%) or cydno-like expression (7%). In contrast, only 0.01% of genes with consistent expression between H. cydno and H. melpomene, and no genes with transgressive expression in hybrids, show such signatures of selection (Fig. 5A). Again, these results are robust across a broad range of quantitative genetic scenarios (SI Appendix, Fig. S7). In addition, as expected given their enrichment for high PST values, genes with intermediate hybrid expression are more likely to coincide with regions of reduced gene flow than other differentially expressed genes (Χ² = 116.1, P < <0.001; Fig. 5C). In sum, genes with intermediate expression in hybrids are more likely to have evolved under divergent selection in H. melpomene and H. cydno, and are more likely to be associated with selection that removes alleles that introgress between species.

Fig. 4.

Divergence in gene expression between H. m. rosina and H. c. chioneus. (A) Principal-component analysis of neural gene expression for differentially expressed genes. H. c. chioneus samples are colored in blue, F1 hybrids in gray, and H. m. rosina in red. Sequencing year is denoted by dot shape: circular (2014); rhomboid (2019). Note that the batch effect of year may be due to technical aspects associated with library preparation and sequencing, rather than biological variation, but is orthogonal to the main group effect. Sequencing batch was included in statistical models testing for differential expression. (B) Examples of expression profiles for genes that fall into different gene categories defined by their expression in F1 hybrids relative to both parental species, with horizontal bars indicating the mean and boxplots delineating ±SD of the normalized gene counts (n indicates quantity, considering all genes), with a pie chart depicting the proportion of genes that fall into each category (red, H. melpomene-like; blue, H. cydno-like; dark gray, intermediate; light gray, not differentially expressed; yellow, transgressive).

Fig. 5.

Gene expression, selection, and gene flow between H. m. rosina and H. c. chioneus. (A) Medians, interquartile ranges, and distributions of FST and PST values for genes assigned to different categories based on their expression profiles in F1 hybrids (see also Fig. 4). (B) Median, interquartile range, and distributions of admixture proportions (f_d), estimated in 100-kb windows, between H. cydno and H. melpomene, for different gene categories. **P < 0.001; *P < 0.01; n.s., not significant, Kruskal–Wallis test with post hoc Dunn test, with Bonferroni correction (see SI Appendix, Fig. S8 for full comparisons). (C) Regression lines between f_d and PST for the four key categories, showing significant negative associations that imply loci under selection for gene expression level overlap with regions of low gene flow. This association is most pronounced for intermediate genes (slope = −0.336 [95% CIs, −0.219 to −0.453], P << 0.001; see main text). The relationships are also significant (P << 0.001), and similar between melpomene (slope = −0.221 [95% CIs −0.154 to −0.288]) and cydno-like genes (slope = −0.239 [95% CIs, −0.169 to −0.305]), but shallower for genes with no group differences (slope = −0.047 [95% CIs, −0.036 to −0.058]).

Our results therefore reveal both divergence of neural phenotypes between ecologically distinct populations, and the potential disruption of these phenotypes in F1 hybrids. We consider it highly unlikely that hybrids experience “hybrid vigor” due to 1) the lack of a distinct “intermediate” forest habitat where intermediate investment in visual processing could be advantageous; 2) evidence that introgressed alleles are removed from parental populations, which implies selection against immigrants and hybrids (that occur at very low frequencies in the wild); and 3) high levels of heterozygosity in both parental populations meaning inbreeding effects are unlikely. Instead, we suggest the intermediate neural phenotypes are likely to act as further barriers to gene flow. Divergence in gene expression is a major source of genetic incompatibilities between species (69–71), and causes abnormal development and reduces survival (72). Hybrid disruption of expression profiles has been reported in diverging species pairs (72–77), and some evidence points to the importance of divergence in neural gene expression during ecological divergence (78, 79). In Heliconius, phylogenetic comparisons of neural gene expression also provide some evidence of selection at deeper timescales (80). Our data add clear support for this hypothesis. More broadly, disruption in the scaling relationships between components of the sensory systems that coevolve within species, but are under divergent selection between species, likely alters the way in which environmental stimuli are perceived and processed. This occurs at anatomical and molecular levels and may lead to a mismatch between the visual system of hybrids and their sensory environment.

In summary, using a large sample of multiple, geographically disparate populations, we provide strong evidence that divergent selection has acted on brain composition and gene expression during the evolution of habitat partitioning between melpomene and cydno clades of Heliconius butterflies. These changes are heritable, significantly exceed expected rates of neutral divergence, and result in intermediate traits in F1 hybrids. Neuroanatomical divergence is restricted to the visual neuropils, strongly suggesting that adaptation to contrasting sensory niches contributes to hybrid fitness deficits. These data are consistent with known differences in ecology between the two clades (25, 31–33, 35, 81). We therefore hypothesize that the divergent traits we identify reflect adaptations to contrasting light levels and conditions, including differences in the availability of polarized light and sun cues. These changes would impact multiple behaviors that contribute to fitness, from foraging efficiency to mating behavior, implying neural evolution has an underappreciated role during ecological speciation. Although we have not directly demonstrated the fitness effects of divergent neural traits, our hypothesis is consistent with previous evidence that H. melpomene and H. cydno have different visual sensitivities (35, 36), and the evidence for divergent selection itself implies these fitness differences must be present.

While disruptive selection on color pattern has a major role in maintaining reproductive isolation between species (82–84), habitat divergence is thought to be critical to “complete” speciation in Heliconius (28, 31). Our results suggest environmentally dependent selection on neural traits contributes to this process. Whether local adaptation in neural traits is a primary driver of reproductive isolation, whether shifts in color pattern initiate speciation with other traits diverging only secondarily, or if speciation only occurs when divergence in these traits is coincident in time, is unclear. However, given that the quality of the aposematic signal is environment and community dependent (85–87), changes in microhabitat preference, and the corresponding neurobiological adaptations to local environmental conditions, likely occur at the early stages of divergence. Our results are consistent with this early role, as differences between melpomene and cydno clades are comparable across populations, implying they originated as these clades diverged. Together, divergent ecological selection on behavior, and their neural bases, in addition to disruptive selection on mimetic warning patterns, would ultimately provide strong, coincident barriers to gene flow (88), thereby facilitating continued divergence.

At a macroevolutionary scale, diverse studies, ranging from recent adaptive radiations in cichlid fish (89) to more ancient diversification of mammals (90), highlight the importance of ecological transitions in driving divergence in sensory regions of the brain. However, whether these changes in brain composition accumulate after ecological transitions or play a significant role in facilitating them is unclear. Our data provide evidence that brain evolution has a facultative role in ecological transitions. Our results mirror a previous analysis of divergence in brain morphology between Heliconius himera and Heliconius erato cyrbia (23), which are isolated across a steep ecological transition between dense lowland wet forest and more open higher altitude dry forest (30, 91). In this case, heritable shifts in investment are again most notable in sensory neuropils, including visual neuropils (23). Similar conclusions can be drawn from the evolution of several fish ecotypes (13, 14, 92–94); however, here environment-dependent plasticity often plays a dominant role in producing population differences (13, 14, 95). By demonstrating heritable divergence in brain composition, rates of neural gene expression that exceed neutral expectations, and hybrid disruption at both an anatomical and molecular level, our data provide a robust case for adaptive neural evolution. Given the prevalent role of niche separation and environmental gradients in many adaptive radiations, we suggest that local adaptation in brain and sensory systems may have an underappreciated role during ecological speciation.

Methods

Animals.

We sampled three pairs of species in Costa Rica, Panama, and Peru, and a population of H. m. melpomene from French Guiana (Fig. 1) with permission from local authorities (SI Appendix). All wild individuals (n = 77) were hand netted, and brain tissue was fixed in situ in a ZnCl₂-formalin solution (96) within a few hours of collection. Common-garden samples of H. c. chioneus and H. m. rosina were reared at the Smithsonian Tropical Research Institute’s Gamboa insectaries. Hybrids were produced from multiple H. c. chioneus × H. m. rosina crosses in 2013 and 2019. Insectary individuals were dissected at 2 to 3 wk for neuroanatomical (n = 30), and 9 to 15 d for gene expression samples (n = 49) (SI Appendix, Tables S2 and S3).

Immunohistochemistry and Imaging.

Brain structure was revealed using immunofluorescence staining against a vesicle-associated protein at presynaptic sites, synapsin (anti-SYNORF1; obtained from the Developmental Studies Hybridoma Bank, Department of Biological Sciences, University of Iowa, Iowa City, IA; RRID: AB_2315424) and Cy2-conjugated affinity-purified polyclonal goat anti-mouse IgG (H+L) antibody (Jackson ImmunoResearch Laboratories), obtained from Stratech Scientific Ltd. (Jackson ImmunoResearch; catalog no. 115-225-146; RRID: AB_2307343). All imaging was performed on a confocal laser-scanning microscope (Leica TCS SP5 or SP8; Leica Microsystem) using a 10× dry objective with a numerical aperture of 0.4 (Leica Material No. 11506511), a mechanical z-step of 2 μm and an x–y resolution of 512 × 512 pixels. Confocal scans were segmented using Amira 5.5 (Thermo Fisher Scientific) to produce estimates of neuropil volumes.

RNA Extraction and Sequencing.

Brains were dissected out of the head capsule in cold (4 °C) 0.01 M PBS. In 2019, total RNA was extracted using TRIzol Reagent (Thermo Fisher) and a PureLink RNA Mini Kit, with PureLink DNase digestion on column (Thermo Fisher). Illumina 150-bp paired-end RNA-seq libraries were prepared and sequenced at Novogene. In 2014, RNA was extracted using TRIzol Reagent and a RNeasy kit (Qiagen), samples were treated with DNase I (Ambion), and libraries were prepared and sequenced with 100-bp paired-end reads at Edinburgh Genomics. After trimming adaptor and low-quality bases from raw reads using TrimGalore v.0.4.4 (http://www.bioinformatics.babraham.ac.uk/projects), Illumina reads were mapped to the H. melpomene 2 genome (97)/H. melpomene 2.5 annotation (98) using STAR v.2.4.2a in two-pass mode (99). We kept only reads that mapped in “proper pairs,” using Samtools (100). The number of reads mapping to each gene was estimated with HTseq v. 0.9.1 (model = union) (101).

Statistical Analyses of Neuropil Volumes.

Nonallometric differences in brain component sizes were estimated using nested linear models in lme4 R (102). Linear models included each brain component as the dependent variable, the volume of rCBR, and taxonomic/experimental grouping as independent variables, with sex and country (where relevant) included as random factors. Including country as a random factor controls for the fact that populations within each clade are nonindependent; however, we also test the hypothesis that phylogenetic structure explains our results using Mantel tests (see below). The log-likelihoods of nested models were compared using likelihood ratio tests and a Χ² distribution, with 1 degree of freedom between nested models including/excluding focal factors, and sequential Bonferroni correction (103). For neuropils showing a significant clade/species effect, we subsequently explored the scaling parameters responsible for group differences using SMATR v.3.4–3 (104). Partial-Mantel tests were performed between pairwise differences in neuropil volumes and FST (55), controlling for rCBR, using ECODIST (105) with Pearson correlations and 1,000 permutations. We calculated PST using the PSTAT package (106) with a c/h² ratio of 1, and allometric correction with the res() function. The significance of PST was calculated as the proportion of the FST distribution (34) that was above each PST value. Finally, to identify intermediate traits in hybrids we also performed principal-component analysis and ANOVAs among parental and hybrid individuals, with post hoc Tukey tests to compare group means, using base R packages (107).

Statistical Analyses of Gene Expression Data.

Differential gene expression analyses were conducted in DESeq2 (108), including sex and sequencing batch as random factors, with a minimum fold change in expression of 2 to counter effects of tissue composition (67). We conducted a principal-component analysis on rlog-transformed gene count data (as implemented in DESeq2) to inspect clustering of expression profiles. ANOVAs on normalized gene expression counts of species and hybrids, with post hoc Tukey tests, were performed using base R packages (107). PST from normalized gene counts in H. m. rosina and H. c. chioneus was calculated following Uebbing et al. (109), with h² set to 0.5 and c to 1.0. Estimated admixture proportions (f_d) between H. m. rosina and H. c. chioneus, and population recombination rates (rho) were taken from Martin et al. (68). To test for an association between low gene flow and high PST, we fitted a linear mixed model: f_d ∼ rho + PST + (1|chromosome), with a Gaussian distribution, using 100-kb nonoverlapping windows of f_d. We used InterProScan v.5 (110) to retrieve gene ontology (GO) terms for the Hmel2.5 gene set and performed GO enrichment tests with the TopGO package in R (111), using the “elim” algorithm, which corrects for nonindependence among GO terms.

Data Availability.

Full descriptions of the methodology and the neuroanatomical dataset are available in SI Appendix and have been deposited on DataDryad, along with R code (https://doi.org/10.5061/dryad.7wm37pvs3). The majority of the raw reads from the genetic dataset have been already available in The European Nucleotide Archive (accession no. PRJEB39935), and the remaining have been deposited on The European Nucleotide Archive (accession no. PRJEB42387).

All study data are included in the article and/or supporting information.