Managing the tradeoff between reproduction and survival requires flexibility in behaviour and gene regulation in three-spined stickleback

Abstract

Predators exert a powerful selective force, however, predator avoidance can conflict with other important activities such as attracting mates. Decisions over whether to court mates versus avoiding predators are vital to fitness, yet the mechanistic underpinnings of how animals manage such tradeoffs are poorly understood. Here, we investigate the flexibility of behaviour and gene regulation in response to a tradeoff between avoiding predators (survival) and courting potential mates (reproduction) in three-spined stickleback (Gasterosteus aculeatus). We compared behavioural and transcriptomic responses of male sticklebacks faced with a courtship opportunity and cues of a predator simultaneously with the responses of males faced with a courtship opportunity or cues of a predator alone, and found that males behaviourally compromised courtship in favour of predator avoidance when faced with a tradeoff between them. The need to manage this tradeoff elicited dynamic changes in brain gene expression, and sets of functionally connected genes were organized into discrete modules based on co-expression. Additionally, we found that behavioural flexibility in response to tradeoffs corresponded to flexibility in gene regulatory network structure. Combined, these results uncover the coordinated response by the brain to a fundamental ecological tradeoff, providing insight into the structure and function of genetic networks underpinning how animals make fitness-influencing decisions.

1. Introduction

Predators exert a powerful selective force, shaping a diverse array of behavioural and morphological adaptations by prey animals to avoid capture [1]. Predation carries high fitness costs, however, predator avoidance can conflict with other important activities, such as foraging and reproductive activities [1,2]. Tradeoffs between predation risk and courtship are of particular interest, as they reflect a fundamental and widespread tradeoff between survival and reproduction [1,3]. For example, elaborate ornamentation and conspicuous courtship displays are often essential to attracting mates, but these traits can increase the risk of predation by attracting eavesdroppers [4,5]. Likewise, inactive or cryptic individuals may avoid the attention of predators, but fail to attract any potential mates [1]. There are many examples of how this tradeoff has been resolved over evolutionary time, for example, by shaping sexual signals or favouring alternative reproductive behaviours that reduce predation risk [6,7]. However, decisions over how to respond to predation risk also play out in real time because both the risk of predation and the availability of mates can vary across space and through time. While many animals are capable of flexibly adjusting their reproductive decisions depending on both their assessment of predation risk and their expected future reproductive opportunities [1,3,8,9], the fundamental mechanistic underpinnings of behavioural flexibility are not well-understood [9–11]. Identifying these mechanisms and how they relate to decision making is critical to our understanding of how animals manage multiple, competing demands, as well as how such behavioural flexibility evolves.

Studying the dynamic changes in brain gene expression can provide insight into decision making at the molecular level. Brain gene expression profiles often show remarkable concordance with behavioural responses to diverse environmental stimuli, suggesting that there is a coordinated response in the brain to behavioural decision making [12–14]. Network-based approaches for analysing whole transcriptome data [15,16] can organize differentially expressed genes (DEGs) into discrete modules based on co-expression and reveal the modular structure of gene co-expression networks and how they respond to decision making. Gene expression profiles in turn are coordinated by complex regulatory interactions between transcription factors (TFs) and their targets, which form gene regulatory networks (GRNs) [17,18]. Identifying the TFs associated with decisions during competing demands may therefore provide insight into how gene regulation relates to behavioural flexibility, opening the way for comparative studies into how these mechanisms evolve in response to selection on decision making.

At the network level, tradeoffs between competing demands such as courtship and predation risk may correspond to flexibility in GRN structure [18]. The structure of GRNs can be represented using a dynamical systems approach [18,19], which models how network structure relates to changing patterns of neural activity or gene expression, respectively, through time. This approach can then be used to find the features that contribute to network flexibility, which has generated the prediction that fewer TF–TF regulatory interactions correspond to a more flexible transcriptomic landscape, but at the expense of network stability [18,20]. Sinha et al. [18] proposed that the structure of GRNs associated with behaviour should be more flexible than developmental GRNs, considering the greater flexibility of behaviour compared to relatively stable developmental processes. Here, we extend this hypothesis to behavioural decision making. A recent study found that a distinct set of genes was recruited in the brains of male sticklebacks when presented with a tradeoff between courtship and territorial defense, but not a courtship opportunity or territorial challenge alone [21], suggesting that there is something unique about the need to manage competing demands simultaneously. We therefore hypothesize that the set of uniquely expressed genes associated with managing competing demands reflects selection on behavioural flexibility. When animals are faced with simultaneous competing demands, flexibility might be favoured, allowing an individual to transition more easily between behavioural states. However, when faced with only one demand, consistency might be favoured, allowing an individual to invest more effort into the relevant behaviour and maintain its behavioural state. We predict here, then, that the GRN when animals are balancing competing demands will be more flexible, characterized by fewer TF–TF interactions, compared to when animals are doing just one thing at a time.

Here, we explore the flexibility of behaviour and gene regulation in response to the fundamental tradeoff between reproduction and survival in three-spined stickleback (Gasterosteus aculeatus). Male sticklebacks are often confronted by multiple, competing demands during the breeding season: they must simultaneously defend their territory from intruders, court potential mates, provide paternal care and guard against predation threats towards themselves and their offspring [11,21]. Reproductive male sticklebacks develop bright nuptial colouration and court females with conspicuous courtship displays [22], making them particularly vulnerable to predation [23]. To cope with this elevated threat, males can assess predation risk using visual and chemical odor cues of a predator, as well as cues from injured conspecifics, and flexibly adjust their reproductive behaviour in response [24,25]. Candolin [26] demonstrated that male sticklebacks reduced courtship activity in the presence of a predator. However, the amount of risk a male is willing to take has been shown to depend on his current mating probability, suggesting that males are able to adjust their reproductive decisions according to expected future reproductive opportunities [24]. Previous studies in sticklebacks have shown that there are dramatic changes in brain gene expression in response to the simulated risk of predation [27] and the opportunity to court a female [21,28]. However, the mechanisms underlying decisions over courtship when simultaneously faced with the risk of predation have not been explored. Additionally, Barbasch et al. [21] identified a distinct set of genes associated with managing competing demands over courtship and territorial defense—another vital ecological tradeoff—raising novel questions about how the expression of these ‘tradeoff’ genes is coordinated and if they are common to tradeoffs in general.

In this study, we compared the behavioural and transcriptomic responses of male sticklebacks faced with a tradeoff between courtship and predation risk to males faced with a courtship opportunity or cues of a predator alone. We first identified sets of DEGs associated with courtship, predation risk and the tradeoff, measured in two gross brain regions, the diencephalon and telencephalon. To determine whether there are similarities in the identities of genes involved in managing different types of tradeoffs, we compared the set of genes associated with the tradeoff between courtship and predation risk with the set of genes identified in Barbasch et al. [21] as associated with the tradeoff between courtship and territorial defense. Next, we grouped sets of functionally connected genes into modules of co-expression, and asked how the expression profiles and regulators of these modules relate to individual behavioural decisions over courtship and predator avoidance. Finally, we investigated patterns of transcriptional regulation and tested the a priori hypothesis that behavioural flexibility in response to tradeoffs corresponds to flexibility in GRN structure, measured as fewer TF–TF regulatory interactions. In addition to the number of TF–TF regulatory interactions, we measure several other network metrics to further probe the relationship between GRN structure and decision making. Combined, these results uncover the coordinated response by the brain to a fundamental ecological tradeoff, providing insight into the structure and function of the genetic networks underlying how animals make vital, fitness-influencing decisions.

2. Methods

(a). Study population

Adult male and female stickleback were collected using minnow traps from South Rolly Lake, AK (61.667 N, 150.138 W) on 1 June 2022 and shipped to the University of Illinois Urbana Champaign. Upon arrival, fish were placed in groups of 15 or 30 in 10 and 25 gallon tanks, respectively, and quarantined for one week. Fish were kept on a 16 : 8 hours light/dark photoperiod to mimic the breeding season photoperiod [24] and water temperatures were maintained at 62–68°C. Housing tanks were given 25% water changes every other day and all fish were fed ad libitum once per day.

(b). Behavioural trials

Behavioural trials were conducted between 12 and 29 July 2022. Males were placed in experimental tanks (53 L cm × 33 W cm × 24 H cm) with sand and algae for nesting material. Once males built their nests, they were assigned to one of four treatments: courtship opportunity alone, predation risk alone, courtship opportunity and predation risk together (hereafter ‘tradeoff’) and a control. Stimuli were confined to cages made from clear plastic (7.5″ × 5″ × 2.5″) with 0.5 cm wire mesh at each end to allow olfactory cues to permeate. For each trial, a male was presented with two cages placed 10 cm from the nest at either end of the tank. In the courtship opportunity treatment, a live gravid female was confined to one of the cages. In the predation risk treatment, trout visual and olfactory cues (see below) were presented in one of the two cages. In the tradeoff treatment, a live gravid female was confined to one cage and trout cues were presented in the other. Finally, the control consisted of two empty cages. Visual and olfactory cues of a rainbow trout predator (Oncorhynchus mykiss) were used to simulate predation risk. Rainbow trout are a common predator of stickleback, and native to South Rolly Lake [29]. Trout visual cues consisted of a realistic 6″ rainbow trout fishing lure (Savage Gear, Denmark, EU) with the hook removed. To simulate movement, during the trial the front of the trout was lifted up and down approximately 2 cm once every 10 s using clear fishing line tied around the head (as in [30]). Trout odor cues consisted of water collected from a raceway housing several hundred rainbow trout at Jake Wolf Memorial Fish Hatchery. Injured conspecific cues were generated by macerating eight conspecifics and soaking them in 0.6 l of water (as in [25]). Both the trout odor cues and the injured conspecific cues were stored at −20°C in 100 ml aliquots and thawed immediately before use. The 100 ml olfactory cue consisted of 50 ml trout odor cues and 50 ml injured conspecific cues.

Each trial consisted of five min of acclimation, after which the cages with their corresponding stimuli were introduced and the behaviour of the focal male recorded. In the predation risk and tradeoff treatments, olfactory cues were poured gently above the predator cage. As a disturbance control, 100 ml of tank water were added simultaneously above the female cage and empty cage in the courtship opportunity treatment, above the female cage in the tradeoff treatment, and above each of the two empty cages in the control treatment. Predator avoidance was quantified as the number of retreats by the focal male, where the male would swim rapidly backwards with dorsal spines raised [30]. Male courtship was quantified by recording the number of ‘zigzag’ displays performed by the male, a common courtship display in sticklebacks [22] and the number of ‘pecks’ at the female cage (or a randomly assigned empty cage in the control). Because females were confined to cages, we assumed that the number of pecks at the female cage reflected courtship effort by the males. We focus on number of pecks because zigzags occurred at a low frequency relative to the number of pecks at the female cage, and rates of zigzags and pecks were positively correlated (Pearson’s correlation = 0.40, t = 2.318, d.f. = 29, p‐value = 0.028), suggesting that the number of pecks is a reasonable proxy for courtship behaviour.

(c). Behavioural analysis

All analyses were conducted in R v. 4.2.2 [31]. To investigate behavioural responses to the tradeoff between predation risk and courtship, two linear models were fit using the ‘lme4’ package [32]: (i) the number of pecks at the female (or empty control) cage and (ii) the number of retreats from the predator cage, with treatment as a fixed effect. Response variables were +1 ln-transformed to fit the assumptions of linear models. Tukey post hoc tests were performed on significant treatment effects using the ‘emmeans’ package [33].

(d). RNA extraction and sequencing

One hour after the trial, males were sacrificed (n = 8 after a courtship opportunity, n = 8 after predation risk, n = 8 after a tradeoff, n = 7 after a control) and their telencephalon and diencephalon rapidly dissected and deep frozen at −80°C until extraction [21]. RNA was extracted using Invitrogen PureLink RNA extraction kits (Invitrogen Corporation, Carlsbad, CA, USA) and quality was checked using Agilent Bioanalyzer chips. Library prep and sequencing were performed by the Functional Genomics Unit of the W.M. Keck Center (University of Illinois Urbana Champaign). Libraries were prepared using the ‘Kapa Hyper Stranded mRNA library kit’ (Roche) and pooled, quantitated by qPCR and sequenced on one S4 lane on a NovaSeq6000 to a depth of >30 M reads. Reads were aligned using STAR [34] to the G. aculeatus reference genome (Ensembl release 95; [35–37]). Gene transcript counts were generated using HTSeq [38]. Counts from the diencephalon and telencephalon samples were analysed separately. Out of 62 samples, we removed two samples from the telencephalon (one ‘Control’ and one ‘Predation Risk’) owing to overlap with the diencephalon samples on a PCA plot, potentially because not all of the diencephalon tissue was cut away while dissecting the smaller telencephalon.

(e). Differential gene expression

Genes were filtered to include those with >0.5 counts per million (CPM) in four or more samples, resulting in 18 302 and 18 588 genes in the telencephalon and diencephalon, respectively. Filtered read counts were TMM (timed mean of M-values) normalized using a tagwise dispersion estimate and used to perform a GLM to identify the set of DEGs in the three experimental treatments relative to the control, with an empirical false discovery rate (eFDR) p-value correction (as in [21]). Briefly, sample names were permuted 500 times to generate a null distribution of p-values and compared to the actual p-values. A cut-off of eFDR < 0.01 was used. DEGs were annotated with ‘biomaRt’ [39] and ‘TopGo’ [40] was used to identify genes and biological processes recruited in response to each treatment relative to the control. Here, we included only the DEGs unique to a specific treatment to test whether these distinct genes represent shared or distinct biological processes.

Next, to investigate the overlap between the unique DEGs associated with managing the tradeoff between courtship and predation risk (this study) and the unique DEGs associated with managing the tradeoff between courtship and territorial defense [21], we used a hypergeometric overlap test comparing gene lists between the two experiments in the diencephalon and telencephalon. Significantly enriched GO terms for the DEGs unique to each experimental treatment (courtship opportunity, predation risk and tradeoff) were identified using Fischer’s exact tests and the ‘parentchild’ algorithm [41], which takes into account parent–child relationships among terms. To gain insight into the expected number of genes and GO terms despite differences in experimental conditions and stickleback populations, we additionally compared gene lists and GO terms between the courtship opportunity treatment from both experiments.

(f). WGCNA

To identify modules of co-expressed genes and how they relate to individual behaviour and treatment, weighted co-expression networks were built using weighted gene co-expression network analysis (WGCNA) [15] with the genes that passed filtering in the two brain regions. Briefly, signed networks were built using a biweight midcorrelation function with a soft thresholding power chosen according to where the scale-free topology index curve flattened upon reaching 0.9 (power = 11 for the telencephalon, 5 for the diencephalon). The minimum block size was set to 30 and similar modules were merged using the dynamic tree cut method, which resulted in 23 modules in the telencephalon and 40 modules in the diencephalon.

To identify general patterns organizing functionally connected genes into co-expression modules, hypergeometric overlap tests were performed using the sets of DEGs from each experimental treatment (including overlapping genes) and the sets of genes composing each module. Overlapping genes were included to identify modules enriched for DEGs associated with multiple treatments. A Bonferroni-corrected p-value threshold of 0.1 was used to assess significance. To identify biological processes associated with the genes composing these modules, we performed GO analysis on the modules significantly enriched for the DEG lists as described above. Next, to find modules whose expression patterns differed between treatments, we fit linear models with module eigengene expression as the response variable and treatment as the independent variable. Empirical p-values were calculated as above, with sample names permuted 10 000 times. We used an eFDR threshold of 0.1 and Tukey post hoc tests were used to make pairwise comparisons for models with a significant treatment effect.

To understand the relationship between module expression and individual behaviour, linear models were fit with module eigengene expression as the response variable. To investigate female-directed behaviour, the number of pecks at the female cage in the courtship opportunity and tradeoff treatments was fit as the independent variable. We included treatment only if it significantly improved model fit, as determined using likelihood ratio tests. Similarly, to investigate predator avoidance behaviour, we used the number of retreats in the predation risk and tradeoff treatments, controlling for treatment where treatment significantly improved model fit. We used an eFDR threshold of 0.1 to control for multiple comparisons.

(g). Gene regulatory network analysis

To investigate patterns of transcriptional regulation, GRNs were inferred using GENIE3 [16]. GENIE3 was the best performer in several in silico benchmark challenges such as the DREAM4 Multifactorial Network Challenge [16]. Additionally, GENIE3 is easy to use and computationally tractable [16]. GENIE3 quantifies the strength of regulatory interactions using random forests to predict the expression pattern of each ‘target’ gene based on the expression of a set of ‘regulators’. As regulators, we used the putative TFs for stickleback from the Animal Transcription Factor Database [42]. A hard threshold of 0.017 was used to select the connections with the strongest importance values, which are determined based on GENIE3’s ranking of regulators according to their relevance for predicting target gene expression [16]. To determine whether results were robust to choice of threshold, we compared network metrics across a range of thresholds from 0.015 to 0.019, representing a more than twofold difference in the total number of network connections (18 721 connections at threshold 0.019 and 41 100 at threshold 0.015). Selecting the middle threshold of 0.017, the resulting network consisted of 1036 TFs, with 27 132 connections in the telencephalon and 1071 TFs with 13 390 connections in the diencephalon.

Transcription factors whose targets were significantly enriched for the unique DEGs from each experimental treatment were identified using Bonferroni-corrected hypergeometric overlap tests (false discovery rate (FDR) < 0.1), including TFs with at least five targets. Here, as with the GO analysis, we used the sets of DEGs unique to each treatment, allowing us to identify distinct regulators involved in the response to tradeoffs, as well as to test whether there are differences in the structure of gene regulation when animals are faced with multiple demands compared to only one. To investigate the role of gene regulation in the behavioural response to tradeoffs, we identified TFs whose targets were significantly enriched for genes composing WGCNA modules associated with treatment and modules associated with behaviour using Bonferroni-corrected hypergeometric tests (FDR < 0.1). Next, we tested whether treatment (for TFs with enrichment for treatment modules) or behaviour (for TFs enriched for behaviour modules) was associated with the expression of those TFs using linear models, as with the WGCNA analysis. We used an eFDR threshold of 0.1 to control for multiple comparisons and Tukey post hoc tests to make pairwise comparisons between treatments.

To test our main hypothesis that the need to manage tradeoffs is associated with greater flexibility in GRN structure, for each treatment we calculated the TF–TF edge ratio as the number of target genes for each TF that also encode TFs divided by the total number of target genes for that TF (as in [18]). We predicted that the tradeoff treatment would have a lower TF–TF edge ratio, corresponding to greater flexibility in behaviour and GRN structure, compared to the courtship and predation treatments. To understand network-level differences between our treatments, we used a p-value cutoff of 0.05. We compared the ratios across treatments using Kruskal–Wallis tests and post hoc comparisons were made using Dunn’s tests. Additionally, to gain insight into the level of organization at which tradeoffs are managed we determined whether treatments differed in other network metrics that might correspond to network flexibility [43]. Specifically, we calculated in-degree as the number of TFs that regulate a gene, out-degree as the number of genes a TF regulates and betweenness as how often a node lies on the shortest path between two nodes.

3. Results

(a). Males behaviourally prioritized predator avoidance over courtship when faced with a tradeoff between them

Treatment had a significant effect on the number of pecks (F-value (3,27) = 13.422, p‐value < 0.0001). While males occasionally pecked at the cages in the Control treatment, post hoc tests revealed that males pecked at a cage significantly more when a female was present (courtship opportunity – control: estimate = 1.475, d.f. = 27, t-ratio = 3.723, p‐value = 0.005; courship opportunity – predation risk: estimate = 2.284, d.f. = 27, t-ratio = 5.967, p‐value ≤ 0.0001). Additionally, males pecked at the female cage significantly less when there was a predator also present (courtship opportunity – tradeoff: estimate = 1.851, d.f. = 27, t-ratio = 4.835, p‐value = 0.0003; figure 1a). Treatment also had a significant effect on the number of retreats from the predator (F-value (3,27) = 27.64, p‐value < 0.0001). Males retreated significantly more when a predator was present compared to an empty cage (predation risk – control: estimate = 1.296, d.f. = 27, t-ratio = 5.787, p‐value ≤ 0.0001) or a gravid female (predation risk – courtship opportunity: estimate = 1.750, d.f. = 27, t-ratio = 8.088, p‐value ≤ 0.0001). However, males performed a similar number of retreats in the presence of a predator regardless of whether cues of predation risk were presented alone or together with a gravid female (predation risk – tradeoff: estimate = 0.365, d.f. = 27, t-ratio = 1.688, p‐value = 0.349; figure 1b). Males occasionally pecked at or retreated in the presence of the empty cage, but at relatively low frequency compared to treatments with cues of a female or predator, thus the empty cage control allowed us to isolate the response to the stimulus from the response to the cage holding it.

Figure 1.

Males prioritized predator avoidance over courtship when faced with the tradeoff between them. (a) Males presented with a courtship opportunity pecked at the female cage significantly more than males presented with cues of a predator or a control. In the predation risk treatment, pecks were recorded towards the empty cage. In the control treatment, pecks were recorded towards one of the two (randomly chosen) empty cages. (b) Males performed more retreats when in the presence of predator cues, regardless of whether a female was present. Letters represent statistical differences based on Tukey post hoc tests of linear models with a significant main effect of treatment. n = 7–8/Treatment.

(b). DEGs involved in managing tradeoffs were organized into co-expression modules

Analysis revealed the sets of DEGs in the three experimental treatments relative to the control in the two brain regions (electronic supplementary material, table S1). Several genes of interest were identified, including pomca in all three treatments in the telencephalon, several nuclear receptor subfamily genes in the predation risk treatment in the telencephalon and tradeoff treatments in the telencephalon and diencephalon, and tshba in the courtship and tradeoff treatments in the diencephalon. POMC is involved in stress response and energy metabolism [44,45], and thus may be important in the response to a variety of stimuli. Likewise, nuclear receptors regulate a variety of physiological processes, including reproduction, energy metabolism and circadian rhythm [46]. Thyroid hormones are implicated for their role in male reproduction in fishes, promoting spermatogenesis and regulating metabolism [47]. While in both regions there was some overlap of DEGs between treatments, a unique set of DEGs was identified when males were confronted by a tradeoff (figure 2a,b). GO analysis identified significantly enriched GO terms (electronic supplementary material, table S2). These sets of GO terms were largely non-overlapping across treatments as well as across brain regions (electronic supplementary material, figure S1), suggesting that these distinct genes also represent distinct biological processes.

Figure 2.

Distinct neurogenomic responses to the tradeoff between courtship and predation risk. Upset plots of DEGs in the three experimental treatments relative to the control in the (a) telencephalon and (b) diencephalon. Heatmaps show the GO enrichment for biological processes (rows) for each of the modules (columns) that were significantly enriched for DEGs from each experimental treatment in the (c) telencephalon and (d) diencephalon. Many of the processes were unique to a module. The coloured squares under each module (column) show significant enrichment for the corresponding DEG set(s), including shared genes between treatments as this allowed us to identify modules involved in the response to multiple treatments. GO terms were summarized for semantic similarity with REVIGO [48].

Next, we compared the unique tradeoff DEGs and their associated GO terms with an experiment that investigated another important ecological tradeoff, i.e. between courtship and territorial defense [21]. We found one gene in the telencephalon and four in the diencephalon that were overlapping between experiments in the tradeoff treatment (electronic supplementary material, figure S2a). To assess whether the overlap reflects differences between experiments or populations versus differences in the biological nature of different tradeoffs, we also compared the number of overlapping genes in the courtship opportunity treatments across experiments. We found that six genes overlapped in the telencephalon and 10 in the diencephalon, comparable to the overlap between tradeoff treatments (electronic supplementary material, figure S3). Additionally, the GO terms enriched in the telencephalon genes were entirely non-overlapping between the experiments, while there were three shared GO terms in the diencephalon (electronic supplementary material, figure S2b), an overlap that is greater than expected owing to chance (p‐value = 0.008). Again, these overlaps were comparable to the number of GO terms shared between courtship opportunity treatments across experiments, thus experimental and population differences might explain the relatively low number of overlapping relative to shared genes.

Next, we investigated modules of co-expression to gain insight into the coordinated response of the brain to tradeoffs. The WGCNA analysis constructed 23 modules consisting of 13 616 genes (74.3%) in the telencephalon and 40 modules consisting of 14 954 genes (80.4%) in the diencephalon. In both brain regions, several modules were significantly enriched for DEGs from the experimental treatments (electronic supplementary materials, figure S4 and table S3), including two modules in the telencephalon (T4 and T7) and two in the diencephalon (D3 and D34) enriched for DEGs associated with the tradeoff treatment but not the courtship opportunity or predation risk treatments. Interestingly, although there were modules associated with both the predation risk treatment and the tradeoff or courtship opportunity treatment, there were no modules significantly enriched in DEGs associated with the predation risk treatment alone. Summarizing GO terms associated with these modules revealed that each module was enriched for some biological processes that were non-overlapping with other modules, even between modules enriched for the same DEG lists, suggesting that these modules are distinct from each other and reflect distinct biological processes (figure 2c,d). For example, the ‘T4’ module associated with tradeoff DEGs is enriched for biological processes including sensory perception of mechanical stimulus and biological regulation, while module ‘T7’, also associated with tradeoff DEGs, is enriched for biological processes including sulfur compound metabolic process, which may indicate that the T4 module is involved in responding to the tradeoff stimulus itself, while module T7 is involved in the metabolic and cellular response. Similarly, in the diencephalon, the ‘D3’ module associated with the tradeoff is enriched for several biological processes related to a response to stimulus and signal transduction, while the ‘D34’ module that is also associated with the tradeoff is enriched for cellular modified amino acid metabolic process and regulation of muscle adaptation. We also identified five modules in the telencephalon and one in the diencephalon whose eigengene expression was significantly different among the four treatments (figure 3; electronic supplementary material, tables S45). Combined, these results indicate not only that there are distinct genes associated with managing this tradeoff, but that these genes are associated with each other according to their expression patterns and thus may be functionally connected at a higher level of organization.

Figure 3.

The expression patterns of modules of co-expressed genes are associated with treatment. (a) Five modules in the telencephalon and (b) one in the diencephalon showed a significant effect of treatment on module eigengene (ME) expression (p‐value < 0.1). Shown are the mean ± s.e. Points represent individual samples. For module D39 in (b), none of the post hoc comparisons were significant despite a significant effect of treatment overall. Coloured squares correspond to the enrichment for DEG lists of modules. Letters represent statistical differences based on Tukey post hoc tests of linear models with a significant main effect of treatment on eigengene expression (electronic supplementary material, table S5).

(c). Module expression patterns were correlated with individual behavioural responses in the telencephalon

To investigate the relationship between module eigengene expression and individual behaviour, we looked at the relationship between pecks at the female cage in treatments where a female was present (courtship opportunity and tradeoff treatments) and eigengene expression, as well as the effect of retreats in treatments where predator cues were present (predation risk and tradeoff treatments). We identified four modules significantly associated with number of pecks at the female cage (figure 4a), one of which was also significantly associated with the number of retreats (figure 4b), all in the telencephalon.

Figure 4.

Relationship between module eigengene expression and (a) pecks at the female cage when cues of a female were present (courtship and tradeoff treatments) and (b) retreats when cues of a predator were present (predation risk and tradeoff treatments). Shown are significant relationships (eFDR < 0.1) between behaviour and module eigengene expression. Points represent individuals, coloured by treatment. There were no significant correlations between ME expression and behaviour in the diencephalon. Coloured square corresponds to the enrichment of the T7 module for tradeoff DEGs.

(d). Transcriptional regulatory flexibility in response to a tradeoff in the telencephalon

We investigated transcriptional regulation by constructing GRNs to identify TFs associated with managing tradeoffs as well as test hypotheses about how GRN composition and structure relate to the need to manage tradeoffs. We identified TFs whose targets were significantly enriched for the unique DEGs from each of the experimental treatments, which likely represent distinct sets of regulators controlling the expression of genes involved in each treatment (electronic supplementary material, table S6). Next, we investigated the role of gene regulation in activating modules of co-expressed genes. We identified TFs whose targets were significantly enriched for WGCNA modules associated with treatment (electronic supplementary material, table S7). From this set, the expression of four TFs was significantly associated with treatment (electronic supplementary materials, tables S89 and figure S5). Similarly, we identified TFs associated with behaviour using TFs whose targets were significantly enriched for WGCNA modules associated with behaviour (electronic supplementary materials, table S10 and figure S6).

Finally, we tested the a priori hypothesis that the need to manage courtship and predator avoidance corresponds to a more flexible network structure with fewer TF–TF interactions. In support of this, Kruskall–Wallis tests found a significant effect of treatment on TF–TF edge ratio in the telencephalon (but no significant effect was detected in the diencephalon). Dunn’s tests comparing the TF–TF edge ratio for each pair of experimental treatments in the telencephalon revealed that the tradeoff treatment had fewer TF–TF edges on average than the courtship opportunity and predation risk treatments, while the courtship opportunity and predation risk treatments did not differ from each other, consistent with our predictions (figure 5). This pattern was robust to changes in edge weight threshold (0.015–0.019) when determining the GRN matrix. Network metrics in-degree, out-degree and betweenness did not differ between treatments in the telencephalon across thresholds. However, in the diencephalon there was a higher in-degree in the tradeoff compared to the courtship and predation treatments, and a higher out-degree and lower betweenness in the predation risk treatment compared to the tradeoff and courtship opportunity treatments (electronic supplementary material, table S11).

Figure 5.

Evidence for more flexible GRN network structure in the tradeoff treatment compared to the predation risk and courtship opportunity treatments in the (a) telencephalon, but not the (b) diencephalon. For each TF (whose targets were significantly enriched for DEGs unique to each experimental treatment), the TF–TF edge ratio was calculated by counting the number of its target genes that also encode TFs and taking the ratio of this count and the total number of target genes of that TF. The ratio was compared across treatments using Kruskall–Wallis tests and Dunn’s tests for post hoc comparisons. Asterisk denotes a significantly lower TF–TF edge ratio in the tradeoff compared to the courtship opportunity and predation treatments in the telencephalon.

4. Discussion

The presence of predators can have dramatic effects on behaviour and physiology, yet animals must frequently balance the risk of predation against other important activities such as foraging and seeking new mates [1,3]. Behavioural flexibility can allow for rapid and adaptive responses to changing contexts, yet we know little about how the underlying genomic architecture relates to behavioural decision making [11]. Here, we investigated flexibility in behaviour and gene regulation in response to a fundamental tradeoff between courtship and predation risk. We found that male sticklebacks compromised courtship in favour of predator avoidance when faced with a courtship opportunity and cues of a predator simultaneously. Next, we found that the need to manage courtship and predator avoidance elicits dynamic changes in gene expression, and these functionally connected genes are organized into discrete modules based on their expression patterns, providing insight into the coordinated response by the brain when animals are faced with competing demands. Finally, we investigated patterns of transcriptional regulation and found evidence that behavioural flexibility in response to tradeoffs corresponds to flexibility in GRN structure. These results raise novel questions about the evolution of GRNs and their role in decision making.

We found that, when faced with a tradeoff between courtship and predation risk, male sticklebacks compromised courtship activity and prioritized predator avoidance, consistent with other studies in sticklebacks [24,26]. Despite the behavioural response to predation risk being similar when predator cues were presented alone or alongside a courtship opportunity, brain transcriptomic profiles reflected a distinct and coordinated response to the tradeoff between courtship and predation risk. While there was some overlap between the DEGs in response to a courtship opportunity, predation risk and the tradeoff between courtship and predation risk, there was a set of genes differentially expressed only in the tradeoff treatment. A similar pattern was found in a recent study on tradeoffs between courtship and territorial defense in sticklebacks [21]. This raises the question of whether the response to these two tradeoffs involves some of the same genes; there might be something general about the response to tradeoffs across contexts, for example, if the need to manage multiple competing demands is stressful or involves a common behavioural response. In both experiments, sticklebacks were found to compromise courtship—here in favour of predator avoidance—and in Barbasch et al. [21] this response was in favour of territorial defense. Somewhat surprisingly, there were very few shared genes among the tradeoff DEGs in the two experiments. However, the overlap between the different tradeoff treatments was comparable to the overlap between courtship opportunity treatments across experiments, likely reflecting differences in experimental conditions, behavioural responses and stickleback populations in the two experiments. Still, there were fewer overlapping genes and GO terms between the tradeoff treatments compared to the courtship opportunity treatments, raising the intriguing possibility that different sets of genes are involved in the response to different tradeoffs, and thus these underlying mechanisms may evolve independently. Future studies examining other types of tradeoffs and in other taxa would begin to provide insight into the evolution of decision-making mechanisms.

Uncovering distinct sets of genes and regulators in response to tradeoffs raises further questions about how their expression is coordinated in the brain. To gain insight into how functionally connected genes associated with decision making are related at a higher level of organization, we identified modules of co-expressed genes that were significantly enriched for the unique sets of DEGs. We identified two modules in the telencephalon and two modules in the diencephalon that were significantly enriched for DEGs associated with the tradeoff treatment, but not the courtship opportunity or predation risk treatment alone. Additionally, we investigated regulatory relationships within these modules and identified TFs whose expression was associated with both module composition and behaviour. Investigating how the expression patterns of these modules and TFs relate to treatment and behaviour provides some insight into their functional significance. In the telencephalon, the ‘T7’ module was significantly enriched for Tradeoff DEGs and there was a significant negative relationship between courtship behaviour and eigengene expression, which corresponded to a higher average module expression in the tradeoff compared to the predation risk treatment. This module may therefore be related to the behavioural suppression of courtship behaviour and involved in the decision to compromise courtship under the risk of predation. The transcription factor creb5b, whose targets showed significant enrichment for the genes composing module T7, also showed an association with courtship behaviour and thus may be a key regulator involved in the suppression of courtship. Likewise, the ‘T6’ module, though not significantly enriched for any DEGs, showed a positive relationship with courtship behaviour, and thus may be related to promoting courtship when a courtship opportunity is present. The ‘T1’ module, on the other hand, was positively related to both courtship and predator avoidance behaviour, and therefore may play a role in multitasking or ‘switching’ between behaviours when faced with a tradeoff [11]. Intriguingly, we found that two regulators of the T1 module, sox19b and foxd1, were each associated with different behaviours: sox19b was negatively associated with courtship while foxd1 was negatively associated with predator avoidance. This suggests that the need to manage both courtship and predator avoidance simultaneously may involve differential regulation of the same set of co-expression genes.

While we identified several modules in the telencephalon whose expression patterns were significantly associated with courtship and predator avoidance behaviour, in the diencephalon we did not find evidence of module–behaviour relationships. The telencephalon contains several nodes within the vertebrate social decision-making network [10], thus the response to tradeoffs may be coordinated in part by this same set of deeply conserved brain regions involved in responses to social stimuli. The diencephalon, however, contains the optic tectum, a large region involved in the processing of visual signals [49], which are likely involved in the general response to any visual stimulus. A study by Bloch et al. [50] found that the neurogenomic response associated with mate preferences in female guppies exhibited a more diffuse network structure in the diencephalon compared to the telencephalon. Thus, perhaps the lack of a relationship between any of our modules and behaviour in the diencephalon suggests that the responses to courtship and predation risk are also more diffuse. There were also several overlapping genes and GO terms in response to a tradeoff between courtship and territorial defense [21] and between courtship and predation risk in the diencephalon, but not the telencephalon. Thus, the response in the diencephalon may be more general to tradeoffs, and this region may be involved in initiating a response that is processed differently in the telencephalon. Alternatively, the more diffuse response could be a result of the diencephalon being larger than the telencephalon, and thus having more cell types and regions activated in response to a tradeoff; greater spatial resolution of gene expression could be insightful.

In addition to distinct sets of genes, we identified distinct sets of TFs whose targets were significantly enriched for courtship and predation risk DEGs, and several TFs whose targets were significantly enriched for tradeoff DEGs. Moreover, the expression of TFs and modules of co-expressed genes showed associations with treatment and behaviour. This further corroborates other studies demonstrating robust associations between behavioural and transcriptomic states. Jones et al. [51], for example, found that changes in the expression of sets of TFs were strongly associated with behavioural phenotype in honeybees. Additionally, differences in TF expression are associated with the response to territorial aggression and parental care in sticklebacks [52]. The distinct sets of regulators in the tradeoff treatment suggest that a different GRN is recruited when there are competing demands [11]. In addition to differences in gene expression mediated by TF composition, GRN structure may also depend on behavioural state, reflecting the interplay between stability and flexibility [18]. By comparing the TF–TF connectivity in a behavioural GRN in mouse and a developmental GRN in fruitfly, Sinha et al. [18] found some support for the hypothesis that the structure of the behaviour-related GRN had relatively fewer TF–TF connections compared to the developmental GRN, which could reflect the greater flexibility of behaviour compared to developmental processes. Extending this hypothesis to behavioural flexibility, we found that in the telencephalon, the TFs whose targets were significantly enriched for tradeoff genes had fewer TF–TF connections relative to TFs in the courtship or predation risk treatments, consistent with our a priori predictions. We found no difference in GRN connectivity among treatments in the diencephalon, which again may reflect a more diffuse response or greater number of cell types in the diencephalon compared to the telencephalon. Our investigation into patterns of transcriptional regulation thus provided some support for our main hypothesis that the need to manage the competing demands of courtship and predator avoidance at the same time corresponds to flexibility in both behaviour and gene regulation. Comparison of other network metrics in-degree, out-degree and betweenness, revealed no consistent differences across the treatments in the telencephalon. However, some interesting patterns emerged in the diencephalon. The predation risk treatment had a lower out-degree but a higher betweenness compared to the tradeoff and courtship treatments, suggesting that TFs associated with the response to predation risk regulated fewer genes and were more connected, potentially reflecting the lower number of predation risk DEGs compared to the other treatments. Intriguingly, the tradeoff treatment exhibited higher in-degree compared to the courtship and predation risk treatments. MacNeil & Walhout [43] propose that networks with high in-degree could exhibit high robustness or high flexibility, depending on whether the high number of TFs reflects redundancy or noise owing to the greater number of regulators. A study of the yeast GRN found correlations between the plasticity of a gene and the number of TFs regulating it, suggesting that high in-degree may be indicative of flexibility [53]. Combined with our findings, this raises the question of what metrics should be used to investigate the interplay between robustness and flexibility, and highlights the need for modelling approaches to generate predictions as well as comparative studies using this framework to explore other contexts where flexibility is favoured.

Investigating gene regulation at the network level can help reveal general patterns about how selection on behavioural flexibility shapes underlying gene regulatory architecture, providing insight into how these mechanisms evolve. Moreover, GRNs are properties of single cells, while behaviour emerges through the coordinated action of many cells, yet how these spatial scales interact remains an open question. This study opens new avenues for future research into this question. Of particular interest are TFs sox19b and foxd1, which regulated the same set of co-expressed genes but were associated with different behaviours, revealing one way in which conflict between courtship and predator avoidance can be resolved. Future studies into where these TFs are expressed in the brain and what cell types they are associated with could provide insight into how GRNs integrate with neural networks. Additionally, localizing the expression of the distinct sets of genes and TFs associated with managing tradeoffs could reveal whether there are also distinct cell types and brain regions involved in managing tradeoffs, helping disentangle the level of organization at which tradeoffs are managed. Emerging technologies such as single cell and spatial transcriptomics therefore have great potential to integrate the neural and molecular mechanisms underpinning behavioural flexibility [18]. Combined, our results create a framework for linking gene regulatory dynamics and architecture to behaviour, opening the way for future investigations into behavioural flexibility and the evolution of decision-making mechanisms.

Ethics

All experiments were conducted with the approval of the Institutional Animal Care and Use Committee at the University of Illinois Urbana Champaign (protocol no. 21031). Field collection was conducted under ADFG permit #SF2022-09.

Data accessibility

Raw sequence and processed count data are available in the GEO repository (accession no. GSE271894). Behavioural data and code for all statistical methods are available from the Dryad Digital Repository [54].

Supplementary material is available online [55].

Declaration of AI use

We have not used AI-assisted technologies in creating this article.

Authors’ contributions

T.A.B.: conceptualization, data curation, formal analysis, funding acquisition, investigation, methodology, project administration, resources, software, supervision, validation, visualization, writing—original draft, writing—review and editing; V.I.A.: conceptualization, investigation, methodology; B.C.: conceptualization, investigation, methodology, writing—review and editing; A.M.B.: conceptualization, funding acquisition, resources, supervision, writing—review and editing.

All authors gave final approval for publication and agreed to be held accountable for the work performed therein.

Conflict of interest declaration

We declare we have no competing interests.

Funding

This work was supported by an NSF postdoctoral research fellowship to T.A.B. (Award ID: 2109619). This material is based upon work supported by the National Science Foundation under grant no. IOS 1645170. Research reported in this publication was supported by the National Institute for General Medical Sciences (NIGMS) of the National Institutes of Health under award number 1R35GM139597.