Abstract
The protein-folding chaperone Hsp90 has been proposed to buffer the phenotypic effects of mutations. The potential for Hsp90 and other putative buffers to increase robustness to mutation has had major impact on disease models, quantitative genetics, and evolutionary theory. But Hsp90 sometimes contradicts expectations for a buffer by potentiating rapid phenotypic changes that would otherwise not occur. Here, we quantify Hsp90’s ability to buffer or potentiate (i.e., diminish or enhance) the effects of genetic variation on single-cell morphological features in budding yeast. We corroborate reports that Hsp90 tends to buffer the effects of standing genetic variation in natural populations. However, we demonstrate that Hsp90 tends to have the opposite effect on genetic variation that has experienced reduced selection pressure. Specifically, Hsp90 tends to enhance, rather than diminish, the effects of spontaneous mutations and recombinations. This result implies that Hsp90 does not make phenotypes more robust to the effects of genetic perturbation. Instead, natural selection preferentially allows buffered alleles to persist and thereby creates the false impression that Hsp90 confers greater robustness.
Author Summary
Most biologists appreciate that natural selection filters new mutations (e.g., by eliminating deleterious ones), such that genetic variation in nature is biased. The idea that selection also skews the types of genetic interactions that exist in nature is less appreciated. For example, studies spanning diverse species have shown that the protein Hsp90, which helps other proteins to fold properly, tends to diminish the observable effects of genetic variation. This observation has led to the assumption that Hsp90 also buffers the effects of new mutations. This untested assumption has served as a rationale for cancer-treatment strategies and shaped our understanding of variation in complex traits. We measured the effects of new mutations on the shapes and sizes of individual yeast cells and found that Hsp90 does not tend to buffer these effects. Instead, Hsp90 interacts with new mutations in diverse ways, sometimes buffering, but more often enhancing mutational effects on cell shape and size. We conclude that selection preferentially allows buffered mutations to persist in natural populations. This result alters common perceptions about why cryptic (i.e., buffered) genetic variation exists and casts doubt on cancer-treatment strategies aiming to target presumed buffers of mutational effects.
Introduction
Previous work in diverse eukaryotes has demonstrated that inhibition of the protein-folding chaperone Hsp90 reveals previously hidden phenotypic effects of standing genetic variation (see “buffering” in Fig 1A) [1–6]. This result prompts an important question [7–10]: does Hsp90 increase an organism’s robustness to genetic perturbation? That is, by helping mutant proteins to fold, does Hsp90 buffer the phenotypic consequences of many new mutations that would otherwise have effects? An alternative possibility is that Hsp90-buffered mutations are rare, yet appear more prevalent in nature because stabilizing selection does not purge such mutations, whereas it efficiently purges other mutations that have immediate (non-buffered) effects on phenotype [7].
Understanding robustness is crucial for understanding evolutionary processes [10–12] as well as complex disease (i.e., why certain mutations are associated with disease states in some genetic backgrounds but not others) [13]. Furthermore, the potential to modulate robustness by inhibiting Hsp90 presents an intriguing strategy to treat cancer [14–16]; because tumors have increased mutation rates, they might be particularly reliant on proteins that buffer the effects of mutation [17,18]. Although discussions of mutational robustness (sometimes termed genetic canalization) often feature Hsp90 [5,15,19–24], the aforementioned hypothesis that Hsp90-buffered variation is rare but accumulates in natural populations due to selection [7–10] has not been excluded.
There is evidence that Hsp90 sensitizes cells to, rather than buffers, the effects of some genetic perturbations (see “potentiation” in Fig 1B) [6,25,26]. Genetic variants with effects that are more pronounced when Hsp90 is present are particularly common in studies of populations that recently underwent adaptive evolution [25,26], reinforcing the hypothesis that natural selection filters which types of Hsp90-by-genotype interactions exist in nature. To distinguish the “robustness” hypothesis from the “selection” hypothesis, we study how Hsp90 modifies the phenotypic effects not only of standing genetic variation but also of new mutations that have experienced highly reduced selection pressure.
Results
A Collection of Mutations That Experienced Reduced Selection Pressure
To study new mutations, we utilized 94 Saccharomyces cerevisiae mutation accumulation (MA) lines created previously [27]. These yeast lines were all founded by the same ancestral diploid strain. To produce each MA line, independent derivatives of this ancestral strain were bottlenecked 100 times each through single colonies that were randomly chosen—irrespective of colony size—in order to relax selection on growth rate and other fitness-related traits [27]. Because the lines were propagated asexually, recessive mutations were shielded as heterozygotes, further relaxing selection. During the repeated bottlenecking, which was carried out for an estimated 2,062 generations [27], each line accumulated on average ~4 single-nucleotide mutations per haploid genome as well as a greater number of mutations in simple-sequence repeats [28,29]. The single-nucleotide mutations do not show a signature of selection; for example, the ratio of synonymous to non-synonymous substitutions does not deviate from random expectation [28].
A previous study demonstrated that the MA lines contain mutations that affect the morphologies of individual cells [9]. As in the previous study, we used haploid derivatives of the MA lines to assay the effects of mutations absent any influence of dominance [9]. Our expectation was that many of these lines would contain mutations with Hsp90-dependent effects on single-cell morphology. Although Hsp90 mediates the folding of only a subset of “client” proteins [30], Hsp90 interactions may be indirect, mediated through other proteins connected to Hsp90 via a protein-protein network.
A Quantitative Technique for Detecting Buffering and Potentiation
We used high-throughput microscopy and automated image analysis [31] to quantify how Hsp90 inhibition changes variation in single-cell morphology between MA lines. We quantified differences in morphological variation using a statistical approach [9] that detects significant increases or decreases in phenotypic variance upon Hsp90 inhibition (corresponding, respectively, to buffering or potentiating roles of Hsp90). Detecting changes in phenotypic variance that are specifically due to Hsp90 inhibition requires an experimental design (Fig 2) that accounts for the other factors that might affect phenotypic variation. Measuring phenotypic variance is far less common than studying phenotypic averages and presents a unique set of challenges [32]. Therefore, we describe our approach in some detail.
We inhibited Hsp90 using geldanamycin (GdA), a small-molecule inhibitor that binds the ATP-binding site of Hsp90, rendering it unable to perform its cellular function [33]. This drug is commonly used to inhibit Hsp90 in diverse species [1–4], including yeast [6,34]; GdA and its analogs have also been administered in human clinical trials to treat cancers [14,35,36]. Previous studies of Hsp90-mediated buffering in yeast used GdA concentrations ranging from 5 μM [6] to 200 μM [34]. We performed the majority of our experiments at 8.5 μM GdA because this concentration has minimal effects on the growth rate and lag duration of the yeast strains we study (S1 Fig).
In each step (cell growth, cell staining, microscopy) of each replicate experiment in this study, cells grown in Hsp90-inhibited (8.5 μM GdA) and control conditions were studied side by side (Fig 2A). The effects of GdA are therefore not confounded with replicate effects, which we estimated and removed using linear modeling. In each replicate, cells were grown to saturation followed by exponential growth in 96-well plates for 6 h to a density of approximately 1 x 106 cells/mL. Then cells were fixed in 4% paraformaldehyde and stained with dyes that label cell-wall components and nuclear DNA. Stained cells grown in control or inhibited conditions were mounted side by side, in duplicate, on 384-well microscopy plates. High-throughput microscopy and morphometric analysis [31] typically yielded between 100 and 500 quantified cell images from each cell-cycle stage in each strain in each replicate experiment (S1 Fig).
In addition to variation between replicate experiments, there is another source of variation that must be considered when estimating phenotypic variance between yeast strains: cell-to-cell variation within each yeast strain. Hsp90 inhibition has been shown to increase nongenetic heterogeneity for some single-cell morphological features [34], so the common statistical assumption of homogeneous within-group variances is likely to be violated. Therefore, for each phenotype, we partitioned the within-strain variation and between-strain variation, as in previous work [9], with separate within-strain variance terms for Hsp90-inhibited and control conditions by using a Bayesian mixed-model approach based on Markov chain Monte Carlo (MCMC) sampling, implemented using the R package MCMCglmm [37].
Using the approach outlined above (and in Fig 2), we quantified between-strain variation in 29 principal components (PCs) of MA-line morphology that capture most of the variance of 132 single-cell traits. These 132 traits represent a high-quality group of morphometric features identified in a previous study as having lower variation across replicate experiments [38]. Each trait is specific to cells from a particular phase of cell growth, such that 6 PCs represent 83% of the variance in morphology among yeast cells without a bud, 9 PCs represent 85% of the variance among cells with a small bud, and 14 PCs represent 86% of the variance among cells with a large bud (S1 Table). Our ultimate goal was, for each of these PCs (hereafter “phenotypes”), to assess whether Hsp90 inhibition significantly increases or decreases the amount of phenotypic variation between MA lines (as schematized in Fig 1). But first, we asked more generally whether the effects of the mutations in the MA lines are influenced by Hsp90 (without asking whether such mutations are buffered or potentiated).
MA Lines Possess Mutations with Hsp90-Dependent Effects on Single-Cell Morphology
Several observations suggest that the MA lines possess mutations with effects on morphology that differ when cells are grown in the control versus the 8.5 μM GdA condition (Fig 3). Linear modeling detects significant condition-by-line interactions contributing to variation in 27/29 phenotypes (and 108/132 of the original traits) at p < 0.001 (S1 Table).
These statistically detected interactions are borne out by inspection of cell images (Fig 3A and 3B). For example, the second PC in unbudded cells is strongly influenced by traits that concern the position of the nucleus (see loadings in S1 Table). The average distance from the nucleus to the cell wall in the MA line ancestor does not significantly change upon GdA treatment (magenta line in Fig 3A). However, in some MA lines, the average distance from the nucleus to the cell wall decreases upon GdA treatment (e.g., yellow line and cell images in Fig 3A); in other MA lines, the average distance from the nucleus to the cell wall increases upon GdA treatment (e.g., orange line and cell images in Fig 3A). A similar example is presented by the third PC in large-budded cells, which is strongly influenced by the bud angle. Whereas the average bud angle in the MA line ancestor (magenta line in Fig 3B) and in several MA lines (e.g., olive line in Fig 3B) does not change significantly upon GdA treatment, the average bud angle in other MA lines does (e.g., cyan line in Fig 3B). Both examples make clear that mutations in the MA lines have effects that differ between the control and GdA treatments; results from other phenotypes are consistent with this conclusion as well (S2 Fig).
The significant interactions are not all driven by a handful of MA lines possessing mutations with particularly strong GdA-dependent effects. Rather, multiple MA lines demonstrate GdA-induced phenotypic changes that diverge from those observed in the MA ancestor (Fig 3A and 3B; S3 Fig), so much so that there are 17 distinct most-divergent MA lines across 29 PCs (Fig 3C).
Our finding that many MA lines undergo phenotypic changes upon GdA treatment that diverge from those of the MA ancestor suggests that the phenotypic effects of many mutations are influenced by Hsp90. It is unlikely that the few mutations per each MA line mostly fall in the coding sequences of Hsp90 clients. Further study of the mutations in MA lines with GdA responses that diverge from those of the ancestor could reveal whether Hsp90 often interacts directly with mutant versions of non-client proteins, or whether Hsp90’s influence on mutant phenotypes often manifests through indirect interactions percolating through protein networks [39,40]. Because most MA lines possess multiple mutations, we do not know which ones are responsible for the divergent GdA responses. Therefore, to gain insights about what types of mutations have GdA-dependent phenotypic effects, we focused on a subset of 20 MA lines for which whole genome sequencing identified only one single-nucleotide mutation each (S2 Table). Of these MA lines, those that possess mutations predicted to have severe effects on protein function (S2 Table) [28,41] tend to have responses to GdA treatment that diverge more from those of the MA ancestor (Fig 3D).
GdA-By-Genotype Interactions Are Dominated by Line-Crossing Epistasis
GdA-by-genotype interactions can be partitioned [9] into a component that captures changes in the amount of phenotypic variance between MA lines (Fig 4A; line spreading) and a component that captures changes in rank order of MA lines (Fig 4A; line crossing). Such partitioning quantifies the importance of buffering or potentiation (line spreading) relative to genetic interactions that do not alter the phenotypic variance among lines (line crossing). For all 27 PCs with a significant condition-by-line interaction term in linear models, line spreading comprises only a relatively small percentage of Hsp90’s effect on phenotypic variation between strains; instead, line crossing dominates (Fig 4B; for a PC demonstrating the typical ratio of line crossing to line spreading, see Fig 4C). The strong prevalence of line crossing across all PCs suggests that Hsp90 rarely acts specifically as a buffer or a potentiator, but often its relationship with spontaneous mutations can be described more accurately (and simply) as epistasis [10]. These observations, by suggesting that Hsp90 does not act exclusively—or even usually—as a buffer of spontaneous mutations, do not support the hypothesis that Hsp90 increases an organism’s robustness to genetic perturbation.
Assuming that GdA treatment is equivalent to inhibition of Hsp90, the above observations suggest there is epistasis between Hsp90 and many mutations that occur spontaneously in nature. This conclusion is consistent with Hsp90’s position as a hub of protein-protein interaction (because hub proteins, either directly or indirectly, likely influence the phenotypic effects of mutations in many other proteins). However, there might be other reasons that GdA treatment affects phenotypic variation, including reduction of growth. Although previous studies [1,2,6] concluded that the effects of low-level GdA on phenotypic variation result from Hsp90 inhibition and not a nonspecific effect of GdA on growth, and although 8.5 μM GdA has minimal effects on growth of the strains we study (S1 Fig), we sought to test this assumption as well.
We performed additional experiments that indeed support the conclusion that GdA treatment affects phenotypic variation through its inhibition of Hsp90. We find that the MA line responses to GdA and to 5.0 μM radicicol (another Hsp90 inhibitor that is structurally unrelated to GdA [42]) are highly correlated; the median Pearson correlation across 29 PCs (r) is 0.95 (Fig 4D; for all PCs, see S4 Fig). MA line morphological responses to altering the duration of exponential growth are not as correlated (median r = 0.76) (Fig 4D; S4 Fig). Furthermore, although altering exponential growth duration often has a significant effect on the parameter we are most interested in—the amount of between-strain variation (S5A Fig)—neither the magnitude nor the direction of this effect predicts which phenotypes have between-strain variances that are significantly influenced by GdA (S5B Fig). Finally, as noted below, restricting our dataset to traits for which variation is not significantly affected by growth duration does not alter the conclusions reported for the analyses of between-strain variation in the MA lines and other strain collections. These results suggest that GdA’s influence on between-strain phenotypic variation does not result generally from growth perturbation but rather is a specific consequence of Hsp90 inhibition.
Hsp90 Does Not Increase Mutational Robustness
Although GdA treatment impacts the phenotypes of individual MA lines more than it changes the overall amount of variance between lines (Fig 4B), we can still quantify whether such variance changes tend more toward buffering or potentiation. If yeast cells are robust to the effects of spontaneous mutations in a way that is compromised by Hsp90 inhibition, we would expect GdA treatment to increase phenotypic diversity between MA lines by revealing the previously buffered effects of mutation (as schematized in Fig 1A). Instead, for many phenotypes, GdA treatment decreases morphological variation between MA lines (compare ranges of blue and red lines in Fig 3A and 3B). To determine if these differences in variance are significant, we asked whether a 95% credible interval around each difference overlaps zero. Using this cutoff, several phenotypes (7 of 29 PCs; Fig 5A; S1 Table) demonstrate significant decreases in variance upon GdA treatment. Using the same cutoff, no phenotype displays a significant increase in between-strain variance. A 95% confidence interval surrounding the median of the variance differences across all 29 PCs (Fig 5A) falls entirely below zero, suggesting that inhibiting Hsp90 with GdA generally tends to decrease the phenotypic effects of mutations on MA-line morphologies. These observations imply that Hsp90, when not inhibited, tends to potentiate rather than buffer the phenotypic effects of new mutations. This conclusion is inconsistent with the “robustness” hypothesis described above.
The idea that Hsp90 increases robustness against the effects of new mutations has persisted [5,14,15,19–24] despite alternate models for the prevalence of buffered variation in natural populations [7,8], despite an absence of studies focusing on new mutations rather than standing genetic variation [7,43], and despite the previous observations that in some cases Hsp90 potentiates, rather than buffers, the effects of genetic perturbation [6,25,26]. Our finding that Hsp90 tends to potentiate the effects of new mutations more than to buffer them (and that neither potentiation nor buffering is nearly as salient as line-crossing epistasis) must be reconciled with the observation, made in species spanning eukaryotic diversity, that impairing Hsp90 reveals cryptic genetic variation [1–6]. We investigated the hypothesis that, although buffered mutations are less likely to arise than potentiated mutations, buffered mutations preferentially accumulate in populations over evolutionary time [7].
Hsp90 Inhibition Tends to Increase Variation between Natural Yeast Isolates
To test this selection hypothesis, we repeated all experiments on several additional collections of yeast strains isolated from natural environments (S3 Table). These four yeast collections, hereafter called the “Ale,” “Diverse,” “SPD,” and “SPH” strains, consist respectively of (1) 36 strains used in ale production; (2) 24 strains from a worldwide collection from diverse habitats [44,45]; (3) 18 strains of the closely related, nondomesticated species Saccharomyces paradoxus, isolated from one soil environment in England [44,45]; and (4) 18 haploid derivatives of the SPD strains [45].
Unlike the mutations in the MA lines, which accumulated under artificially reduced selection pressure, the standing genetic variation among natural yeast isolates has survived natural selection. Patterns of genetic variation in the collections of natural isolates that have been sequenced (Diverse, SPH, and SPD) reveal evidence of selection [44,46]. These studies also suggest that fermentation processes, like those used to cultivate strains in the Ale collection, impose selective pressures [44,46]. Further studies demonstrate that stabilizing selection has acted specifically on the morphological traits that we study, constraining morphological variation between strains in the Diverse collection [47]. Consistent with strong stabilizing selection, we find that levels of morphological variation in each of the four collections of natural isolates do not far exceed those observed in the MA lines (S6 Fig), even though their levels of genetic diversity are vastly greater [44]. These observations, plus the strong relationship between yeast morphology and critical processes such as growth and reproduction [47], provide compelling evidence that genetic variation contributing to single-cell morphology in the Ale, Diverse, SPD, and SPH strain collections has been constrained by natural selection.
The selection hypothesis predicts that the single-cell morphologies in yeast collections representing natural isolates will tend to become more divergent upon Hsp90 inhibition. Consistent with the selection hypothesis, Hsp90 inhibition tends to increase morphological diversity in all four such yeast collections (Fig 5B). The qualitatively different responses to GdA observed in the natural isolates versus the MA lines persist when the MA lines are subsampled (S5A Fig), when we eliminate phenotypes for which between-strain variation is influenced by growth (S5C Fig), and when we study separately those phenotypes specific to cells in different phases of the cell cycle (S5D Fig). The tendency toward potentiation in the MA lines also persists when we restrict our analysis to the 20 MA lines that each possess a single coding mutation (S5E Fig), arguing against the possibility that this tendency is somehow caused by the combined effects of multiple spontaneous mutations per line.
Hsp90 Inhibition Tends to Decrease Variation between Lines with Novel Genetic Recombinations
The distinction between the MA lines and natural isolates might support the hypothesis that natural selection acts as a filter of new mutations with epistatic interactions [7,8], skewing the relationship between Hsp90 and the rest of the genotype from one that tends toward potentiation to one that tends toward buffering. However, another possibility is that the ancestor of the MA lines possesses a specific genetic make-up that causes it to interact differently with GdA. Perhaps MA-line growth is uniquely responsive to GdA in a way we did not control for. Or perhaps the MA line ancestor possessed a faulty copy of a gene involved in protein homeostasis that causes Hsp90 inhibition to have unexpected effects. To rule out such possibilities, we repeated our experiments on an additional yeast strain collection that consists of 78 recombinant (Rec) progeny from a cross between two divergent yeast strains from the “Diverse” collection [48]. Although each individual mutation in these Rec lines has been exposed to selection, the unique combination of mutations in each line has not, because the mutations are largely private to one or the other lineage [44]. Therefore, studying these lines served two purposes: (1) testing an additional collection of lines each representing a unique genotype that has not experienced selection and (2) testing if Hsp90 buffers the effects of recombination, as it has been suggested that recombination exerts a selective force favoring genetic buffering [49,50].
Previous studies of Rec lines have produced mixed results. A study in yeast found roughly equal frequencies of buffering and potentiation [6]. In contrast, studies in plants concluded that genetic buffering by Hsp90 is common [2,4,20]. However, in the plant Rec inbred lines, line-specific phenotypes revealed by Hsp90 inhibition were also seen in those lines without Hsp90 inhibition, albeit less frequently and in more mild form [2,20]. The apparent increase in variance among lines can therefore be explained by Hsp90 buffering within-line rather than between-line variation. Our study avoids this problem by using statistical procedures that partition multiple contributions to variance.
Similarly to the MA lines, we found that GdA treatment tends to decrease phenotypic diversity among the Rec lines (Fig 5C). These differences between strain collections suggest that Hsp90’s influence on the amount of phenotypic variance between strains tends toward potentiation, but that natural selection tends to skew this genetic interaction toward one of buffering (Fig 5A–5D).
Stabilizing Selection Can Transform Patterns of Line-Crossing Epistasis into Patterns with More Line Spreading
To further characterize how the relationship between Hsp90 and genetic variation differs in the MA and Rec lines versus the natural isolates, we partitioned genetic interactions into line spreading versus line crossing (see definitions in Fig 4A) for those phenotypes that had a significant interaction term in linear models (p < 0.01; orange shading in S1 Table). In all collections of yeast studied here, line spreading comprises only a relatively small percentage of Hsp90’s effect on phenotypic variation between strains, and, instead, line crossing dominates; this is especially true in the MA and Rec lines (Fig 5E). The selection hypothesis [7] explains how stabilizing selection can re-purpose epistatic proteins as buffers of standing genetic variation (Fig 6), transforming patterns dominated by line crossing (Fig 5E: MA and Rec lines) into patterns with more line spreading in the direction of buffering (Fig 5E: Ale, Div, SPH, and SPD strains).
Discussion
For years, discussions of robustness and genetic canalization have featured Hsp90 [5,15,19–24]. The evidence that Hsp90 impairment reveals cryptic genetic variation fit a narrative in which mutationally robust genetic systems accumulate genetic variation that can be expressed when robustness is compromised [19,51]. Although the logic of this narrative is sound (decreasing robustness will reveal cryptic genetic variation), the converse (revelation of cryptic genetic variation means robustness has decreased) is not necessarily true. Indeed, as pointed out over a decade ago, revelation of cryptic genetic variation can occur even when robustness increases [7].
The persistence of the idea that Hsp90 is an agent of genetic canalization is especially unwarranted, because Hsp90 function has been shown in some cases to be required for new mutations to have effects [6,25,26]. As a result, two parallel descriptions of Hsp90 have existed: Hsp90 is a “capacitor” that buffers genetic variation until its function is impaired, and Hsp90 is a “potentiator” that enables new mutations to have effects [52,53]. When Hsp90 is viewed as both a capacitor and potentiator, the circularity of associating it with genetic canalization and mutational robustness should be even more apparent—Hsp90 buffers what it buffers (and does not buffer what it does not buffer).
Studies promoting the robustness hypothesis often reference Hsp90’s role as a protein-folding chaperone, painting a vivid picture of Hsp90 buffering mutational effects by helping proteins fold properly despite mutation [14,23]. But Hsp90 can influence mutant proteins indirectly, potentiating their effects by activating pathways that promote their expression [39] or activating stress response programs that allow mutant phenotypes to manifest without causing cell death [54]. Sometimes the contrasting roles of Hsp90 are not named as capacitance or potentiation, but rather are both attributed to Hsp90’s role in “buffering” mutant proteins, where the meaning of “buffering” is not limited to maintaining wild-type protein activity but also includes enabling new activity [26]. The confusion surrounding discussions of Hsp90's interactions with genetic variation could perhaps be cleared up by abandoning specialized terms such as "buffer" or "potentiator" in favor of descriptions of epistasis [10].
Our results provide direct evidence that Hsp90 does not tend to increase mutational robustness for the large set of cell-morphological traits we studied. Instead, the relationship between Hsp90 and new mutations is dominated by line-crossing epistasis. Therefore, on both theoretical and, now, empirical grounds, Hsp90 should not be described as an agent of genetic canalization. Using a very similar experimental design, except with deletion of the focal gene rather than pharmacological inhibition of its protein product, Richardson et al. (2013) reached the same conclusions about another candidate robustness-increasing factor, the histone variant H2A.Z. Together, the Hsp90 and H2A.Z results raise the question of whether any gene product increases robustness to mutation on a genome-wide scale [10].
Additionally, our results highlight that natural selection transforms the types of epistasis that are present in nature, leaving the false impression of robustness. Although it is appreciated that selection filters out certain types of new mutations (e.g., deleterious ones), the idea that natural selection also skews the types of epistasis that accumulate in genomes is much less appreciated. The false equality drawn between the genetic interactions common in nature and those that are common among new mutations has likely prevented a full and accurate understanding of natural variation in complex traits (including human disease traits).
To be clear, our results do not contradict the prevalence or importance of cryptic genetic variation in nature [55]. Our study confirms that natural genetic variation is enriched for buffered alleles. Previous work explains how the release of such variation, perhaps by stress that impairs Hsp90 function, may play a pivotal role in adaptation by revealing phenotypic diversity under novel or stressful conditions [1,2,5,56]. Revelation of cryptic genetic variation by environmental or genetic perturbation also likely contributes to complex human disease [13]. Our study contributes to previous work by clarifying why cryptic genetic variation exists, potentially shedding new light on the properties of cryptic alleles. For example, perhaps the biased subset of Hsp90-interacting variation that accumulates in nature tends to interact with Hsp90 via a different mechanism than most new mutations.
The nature of Hsp90’s interaction with new mutations is of high practical importance, because Hsp90 inhibitors are considered promising anticancer drugs. One rationale for inhibiting Hsp90 has been that Hsp90 protects tumor cells from the deleterious effects of their elevated mutation rates by suppressing the effects of many mutations [14]. Our work suggests that Hsp90 does not, on balance, protect cells from mutations’ effects and therefore challenges the validity of this rationale. An alternative and more straightforward rationale for inhibiting Hsp90 is that some Hsp90 client proteins are known to promote tumor development; targeting the chaperone therefore might work as a broad strategy for targeting signaling proteins to which cancer cells are “addicted” [15]. Although the rationale for this strategy does not depend on whether or not Hsp90 tends to suppress the effects of mutations, unfortunately, the strategy has not in general succeeded. The culprit appears to be that high doses of Hsp90 inhibitor are necessary, and these doses activate the stress-response factor HSF1, which itself promotes malignancy [15,26].
More recent work (upon which ongoing clinical trials are based) has focused on enhancing the efficacy of another anticancer drug by combining it with low-dose Hsp90 inhibition that does not activate the stress response [26]. The rationale in this case is that low-level inhibition is enough to prevent Hsp90 from potentiating the effects of adaptive mutations, thereby curtailing a tumor’s evolution of resistance to the other drug. A similar principle was proposed for the utility of Hsp90 inhibition in preventing resistance to antifungal drugs [25]. Results for estrogen receptor-positive breast cancer cells are promising [26]. It is possible that a bias toward potentiation of new mutations by Hsp90, as we observed in yeast, contributes to the success of such an approach. However, one might expect the contribution to success to be small, given the prevalence of line-crossing epistasis that we observed. Further investigation of the spectrum of effects of mutations that arise during adaptive evolution with and without Hsp90 inhibition, and ultimately the mechanism by which Hsp90 modifies the phenotypic effects of mutations, is warranted.
Materials and Methods
Yeast Growth, Staining, and Visualization
Yeast growth, cell staining, and microscopy were performed the same way for all six strain collections (for additional details about strain collections and sequencing of MA lines, see S1 Text as well as S3 Table). We prepared cells for microscopy using established protocols [9,38,57], with modifications (S1 Text). Briefly, yeast strains were grown from frozen stocks to saturation in rich media (YPD) in 96-well plates (48 h of growth). Each plate was used to inoculate a pair of new 96-well plates: a control plate containing synthetic complete media (SC) plus DMSO and an Hsp90-inhibited plate containing SC + 8.5 μM GdA solubilized in DMSO (Fig 2). Our goal was for everything about this pair of 96-well plates to be as similar as possible, including the concentration of DMSO, the identity of the yeast strain in each well, the orientation of plates during culturing, and the timing of all subsequent steps. This pair of 96-well plates (GdA+ and GdA−) was grown to saturation and then was used to inoculate a fresh pair of plates containing the same control or Hsp90-inhibited media (Fig 2A). This freshly inoculated pair of plates was grown for 6 h (to mid log), after which we removed growth media, added 4% paraformaldehyde, and fixed cells for 1 h. Cells were then stained overnight and mounted on 384-well glass bottom microscopy plates (Fig 2A). We performed epifluorescence microscopy using a Nikon Eclipse Ti automated microscope using a 40× objective. For each strain, typically between 500 and 1,000 cells were imaged per condition (S1 Fig); at least two complete biological replicate experiments were performed per strain. Imaged cells were analyzed for quantitative morphological traits using CalMorph software (Fig 2B) [31].
Pre-processing Cell Image Data
All data analysis was performed using the open-source R statistical computing package (http://www.r-project.org/), and analysis generally followed that done previously for similar morphology datasets [9,38,57], with modifications (S1 Text). Briefly, for each strain collection, each morphological trait was normalized to have a mean of zero and a standard deviation of one following Box-Cox transformation. Critically, this normalization allows the variances of GdA+ and GdA− conditions to differ, but standardizes the amount of variation across strain collections and phenotypes for downstream statistical analysis and visualization. Linear modeling was used to normalize data from replicate plates (S1 Text). As was done previously, we restricted our analysis to 132 high quality morphometric traits chosen for their lower replicate-to-replicate variation [38]. We eliminated redundancy among these traits by using PC analysis (PCA). The loadings of morphological traits onto PCs are fairly similar between strain collections (S1 Table).
Detecting Genotype-By-GdA Interactions and Line Crossing versus Line Spreading
We used a linear model, fit using maximum likelihood in the R package lme4 [58], to estimate the contribution of genotype and condition to variation in each phenotype (PC). When a likelihood ratio test indicated that linear models including a genotype-by-condition interaction term fit the data significantly better than those without (p < 0.01), we reported a significant genotype-by-GdA interaction for that phenotype in that strain collection (orange shading in S1 Table). Partitioning of this interaction variance into line-crossing and line-spreading components (Figs 4B and 5E) was only done for PCs with a significant genotype-by-GdA interaction and was performed as described previously [9].
Detecting Significant Differences in Between-Strain Variance upon Hsp90 Inhibition
For each PC, condition-specific strain means and between-strain variances were estimated from linear models using MCMC sampling with the R package MCMCglmm [37], following methodology outlined in a previous study (S1 Text) [9]. For each PC, the difference between the control and Hsp90-inhibited variances was called significant when the 95% highest posterior density interval of the difference obtained from the MCMC samples did not overlap zero. Although this analysis does not explicitly control for multiple hypothesis testing, the strong directionality of our results (i.e., that we detect seven and six PCs with decreased variance in MA and Rec lines, respectively, but none with increased variance) suggests that the trends we detect at this significance threshold are meaningful.
Testing Effects of Less Growth versus Hsp90 Inhibition
We performed a control experiment on a subset of the MA lines, chosen randomly (21 MA lines; S2 Table), plus the MA line ancestor in order to directly compare the effects of 8.5 μM GdA, 5.0 μM radicicol, and a shortened exponential growth period. All experiments were performed using procedures described previously (Fig 2), except that all conditions (GdA, Rad, less growth) were present on the same 96-well plate. This plate was removed from the 30°C incubator once after 4 h to harvest cells in the “less growth” condition, and again after 6 h to harvest cells in other conditions; cells from all conditions were prepared for microscopy and imaged together (see S1 Text).
Supporting Information
Acknowledgments
We are grateful to Eugene Plavskin, Christine Queitsch, Michael Dorrity, Daniel Jarosz, and Dmitri Petrov for providing comments on an earlier version of this manuscript, Daniel Jarosz and Stefan Millson for reagents, and Jess Caudill (Wyeast) and Douglas Amport for assistance acquiring ale strains.
Abbreviations
- GdA
geldanamycin
- MA
mutation accumulation
- MCMC
Markov chain Monte Carlo
- PC
principal component
- PCA
principal component analysis
- Rec
recombinant
Data Availability
All data files are available from the Open Science Framework database (DOI 10.17605/OSF.IO/TN24X).
Funding Statement
This work was supported by National Institutes of Health grants R01GM086673, R01GM097415 and R35GM118170 to MLS, and fellowship F32GM103166 to KAGS. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
References
- 1.Rutherford SL, Lindquist S. Hsp90 as a capacitor for morphological evolution. Nature. 1998;396: 336–342. 10.1038/24550 [DOI] [PubMed] [Google Scholar]
- 2.Queitsch C, Sangster TA, Lindquist S. Hsp90 as a capacitor of phenotypic variation. Nature. 2002;417: 618–624. 10.1038/nature749 [DOI] [PubMed] [Google Scholar]
- 3.Yeyati PL, Bancewicz RM, Maule J, van Heyningen V. Hsp90 selectively modulates phenotype in vertebrate development. PLoS Genet. Public Library of Science; 2007;3: e43 10.1371/journal.pgen.0030043 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Sangster TA, Salathia N, Lee HN, Watanabe E, Schellenberg K, Morneau K, et al. HSP90-buffered genetic variation is common in Arabidopsis thaliana. Proc Natl Acad Sci USA. National Acad Sciences; 2008;105: 2969–2974. 10.1073/pnas.0712210105 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Rohner N, Jarosz DF, Kowalko JE, Yoshizawa M, Jeffery WR, Borowsky RL, et al. Cryptic variation in morphological evolution: HSP90 as a capacitor for loss of eyes in cavefish. Science. American Association for the Advancement of Science; 2013;342: 1372–1375. 10.1126/science.1240276 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Jarosz DF, Lindquist S. Hsp90 and environmental stress transform the adaptive value of natural genetic variation. Science. American Association for the Advancement of Science; 2010;330: 1820–1824. 10.1126/science.1195487 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Hermisson J, Wagner GP. The population genetic theory of hidden variation and genetic robustness. Genetics. Genetics Society of America; 2004;168: 2271–2284. 10.1534/genetics.104.029173 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Rando OJ, Verstrepen KJ. Timescales of genetic and epigenetic inheritance. Cell. Elsevier; 2007;128: 655–668. 10.1016/j.cell.2007.01.023 [DOI] [PubMed] [Google Scholar]
- 9.Richardson JB, Uppendahl LD, Traficante MK, Levy SF, Siegal ML. Histone variant HTZ1 shows extensive epistasis with, but does not increase robustness to, new mutations. Petrov DA, editor. PLoS Genet. 2013;9: e1003733 10.1371/journal.pgen.1003733 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Siegal ML, Leu J-Y. On the Nature and Evolutionary Impact of Phenotypic Robustness Mechanisms. Annu Rev Ecol Evol Syst. 2014;45: 495–517. 10.1146/annurev-ecolsys-120213-091705 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Masel J, Siegal ML. Robustness: mechanisms and consequences. Trends Genet. 2009;25: 395–403. 10.1016/j.tig.2009.07.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Levy SF, Siegal ML. The robustness continuum. Adv Exp Med Biol. New York, NY: Springer New York; 2012;751: 431–452. 10.1007/978-1-4614-3567-9_20 [DOI] [PubMed] [Google Scholar]
- 13.Gibson G. Decanalization and the origin of complex disease. Nat Rev Genet. 2009;10: 134–140. 10.1038/nrg2502 [DOI] [PubMed] [Google Scholar]
- 14.Miyata Y, Nakamoto H, Neckers L. The therapeutic target Hsp90 and cancer hallmarks. Curr Pharm Des. 2013;19: 347–365. 10.2174/138161213804143725 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Whitesell L, Santagata S, Lin NU. Inhibiting HSP90 to treat cancer: a strategy in evolution. Curr Mol Med. 2012;12: 1108–1124. 10.2174/156652412803306657 [DOI] [PubMed] [Google Scholar]
- 16.Whitesell L, Lindquist SL. HSP90 and the chaperoning of cancer. Nat Rev Cancer. 2005;5: 761–772. 10.1038/nrc1716 [DOI] [PubMed] [Google Scholar]
- 17.McFarland CD, Mirny LA, Korolev KS. Tug-of-war between driver and passenger mutations in cancer and other adaptive processes. Proc Natl Acad Sci USA. National Acad Sciences; 2014;: 201404341 10.1073/pnas.1404341111 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.McFarland CD, Korolev KS, Kryukov GV, Sunyaev SR, Mirny LA. Impact of deleterious passenger mutations on cancer progression. Proc Natl Acad Sci USA. National Acad Sciences; 2013;110: 2910–2915. 10.1073/pnas.1213968110 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Rutherford SL. From genotype to phenotype: buffering mechanisms and the storage of genetic information. Bioessays. John Wiley & Sons, Inc; 2000;22: 1095–1105. [DOI] [PubMed] [Google Scholar]
- 20.Sangster TA, Salathia N, Undurraga S, Milo R, Schellenberg K, Lindquist S, et al. HSP90 affects the expression of genetic variation and developmental stability in quantitative traits. Proc Natl Acad Sci USA. National Acad Sciences; 2008;105: 2963–2968. 10.1073/pnas.0712200105 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Takahashi KH. Multiple capacitors for natural genetic variation in Drosophila melanogaster. Mol Ecol. 2013;22: 1356–1365. 10.1111/mec.12091 [DOI] [PubMed] [Google Scholar]
- 22.Peuß R, Eggert H, Armitage SAO, Kurtz J. Downregulation of the evolutionary capacitor Hsp90 is mediated by social cues. Proc Biol Sci. 2015;282: 20152041 10.1098/rspb.2015.2041 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Sabater-Muñoz B, Prats-Escriche M, Montagud-Martínez R, López-Cerdán A, Toft C, Aguilar-Rodríguez J, et al. Fitness Trade-Offs Determine the Role of the Molecular Chaperonin GroEL in Buffering Mutations. Mol Biol Evol. Oxford University Press; 2015;32: 2681–2693. 10.1093/molbev/msv144 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Rohner N. Selection Through Standing Genetic Variation Biology and Evolution of the Mexican Cavefish. Elsevier; 2016. pp. 137–152. 10.1016/B978-0-12-802148-4.00007-4 [DOI] [Google Scholar]
- 25.Cowen LE, Lindquist S. Hsp90 potentiates the rapid evolution of new traits: drug resistance in diverse fungi. Science. American Association for the Advancement of Science; 2005;309: 2185–2189. 10.1126/science.1118370 [DOI] [PubMed] [Google Scholar]
- 26.Whitesell L, Santagata S, Mendillo ML, Lin NU, Proia DA, Lindquist S. HSP90 empowers evolution of resistance to hormonal therapy in human breast cancer models. Proc Natl Acad Sci USA. National Acad Sciences; 2014;111: 18297–18302. 10.1073/pnas.1421323111 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Hall DW, Mahmoudizad R, Hurd AW, Joseph SB. Spontaneous mutations in diploid Saccharomyces cerevisiae: another thousand cell generations. Genet Res (Camb). Cambridge University Press; 2008;90: 229–241. 10.1017/S0016672308009324 [DOI] [PubMed] [Google Scholar]
- 28.Zhu YO, Siegal ML, Hall DW, Petrov DA. Precise estimates of mutation rate and spectrum in yeast. Proc Natl Acad Sci USA. National Acad Sciences; 2014;111: E2310–8. 10.1073/pnas.1323011111 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Lynch M, Sung W, Morris K, Coffey N, Landry CR, Dopman EB, et al. A genome-wide view of the spectrum of spontaneous mutations in yeast. Proc Natl Acad Sci USA. National Acad Sciences; 2008;105: 9272–9277. 10.1073/pnas.0803466105 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Citri A, Harari D, Shohat G, Ramakrishnan P, Gan J, Lavi S, et al. Hsp90 recognizes a common surface on client kinases. J Biol Chem. American Society for Biochemistry and Molecular Biology; 2006;281: 14361–14369. 10.1074/jbc.M512613200 [DOI] [PubMed] [Google Scholar]
- 31.Negishi T, Nogami S, Ohya Y. Multidimensional quantification of subcellular morphology of Saccharomyces cerevisiae using CalMorph, the high-throughput image-processing program. J Biotechnol. 2009;141: 109–117. 10.1016/j.jbiotec.2009.03.014 [DOI] [PubMed] [Google Scholar]
- 32.Geiler-Samerotte KA, Bauer CR, Li S, Ziv N, Gresham D, Siegal ML. The details in the distributions: why and how to study phenotypic variability. Curr Opin Biotechnol. 2013;24: 752–759. 10.1016/j.copbio.2013.03.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Whitesell L, Mimnaugh EG, De Costa B, Myers CE, Neckers LM. Inhibition of heat shock protein HSP90-pp60v-src heteroprotein complex formation by benzoquinone ansamycins: essential role for stress proteins in oncogenic transformation. Proc Natl Acad Sci USA. 1994;91: 8324–8328. 10.1073/pnas.91.18.8324 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Hsieh Y-Y, Hung P-H, Leu J-Y. Hsp90 regulates nongenetic variation in response to environmental stress. Mol Cell. 2013;50: 82–92. 10.1016/j.molcel.2013.01.026 [DOI] [PubMed] [Google Scholar]
- 35.Fukuyo Y, Hunt CR, Horikoshi N. Geldanamycin and its anti-cancer activities. Cancer Lett. Elsevier; 2010;290: 24–35. 10.1016/j.canlet.2009.07.010 [DOI] [PubMed] [Google Scholar]
- 36.Sidera K, Patsavoudi E. HSP90 inhibitors: current development and potential in cancer therapy. Recent Pat Anticancer Drug Discov. 2014;9: 1–20. 10.2174/15748928113089990031 [DOI] [PubMed] [Google Scholar]
- 37.Hadfield JD. MCMC methods for multi-response generalized linear mixed models: the MCMCglmm R package. Journal of Statistical Software. 2010. [Google Scholar]
- 38.Bauer CR, Li S, Siegal ML. Essential gene disruptions reveal complex relationships between phenotypic robustness, pleiotropy, and fitness. Mol Syst Biol. 2015;11: 773–773. 10.15252/msb.20145264 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Taipale M, Jarosz DF, Lindquist S. HSP90 at the hub of protein homeostasis: emerging mechanistic insights. Nat Rev Mol Cell Biol. 2010;11: 515–528. 10.1038/nrm2918 [DOI] [PubMed] [Google Scholar]
- 40.Taylor MB, Ehrenreich IM. Transcriptional Derepression Uncovers Cryptic Higher-Order Genetic Interactions. Wittkopp P, editor. PLoS Genet. Public Library of Science; 2015;11: e1005606 10.1371/journal.pgen.1005606 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Kumar P, Henikoff S, Ng PC. Predicting the effects of coding non-synonymous variants on protein function using the SIFT algorithm. Nat Protoc. 2009;4: 1073–1081. 10.1038/nprot.2009.86 [DOI] [PubMed] [Google Scholar]
- 42.Schulte TW, Akinaga S, Soga S, Sullivan W, Stensgard B, Toft D, et al. Antibiotic radicicol binds to the N-terminal domain of Hsp90 and shares important biologic activities with geldanamycin. Cell Stress Chaperones. 1998;3: 100–108. 10.1379/1466-1268(1998)003%3C0100:ARBTTN%3E2.3.CO;2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.de Visser JAGM, Hermisson J, Wagner GP, Ancel Meyers L, Bagheri-Chaichian H, Blanchard JL, et al. Perspective: Evolution and detection of genetic robustness. Evolution. 2003;57: 1959–1972. 10.1111/j.0014-3820.2003.tb00377.x [DOI] [PubMed] [Google Scholar]
- 44.Liti G, Carter DM, Moses AM, Warringer J, Parts L, James SA, et al. Population genomics of domestic and wild yeasts. Nature. 2009;458: 337–341. 10.1038/nature07743 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Cubillos FA, Louis EJ, Liti G. Generation of a large set of genetically tractable haploid and diploid Saccharomyces strains. FEMS Yeast Res. Blackwell Publishing Ltd; 2009;9: 1217–1225. 10.1111/j.1567-1364.2009.00583.x [DOI] [PubMed] [Google Scholar]
- 46.Elyashiv E, Bullaughey K, Sattath S, Rinott Y, Przeworski M, Sella G. Shifts in the intensity of purifying selection: an analysis of genome-wide polymorphism data from two closely related yeast species. Genome Res. Cold Spring Harbor Lab; 2010;20: 1558–1573. 10.1101/gr.108993.110 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Yang M, Ohnuki S, Ohya Y. Unveiling nonessential gene deletions that confer significant morphological phenotypes beyond natural yeast strains. BMC Genomics. 2014;15: 932 10.1186/1471-2164-15-932 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Gerke J, Lorenz K, Cohen B. Genetic interactions between transcription factors cause natural variation in yeast. Science. American Association for the Advancement of Science; 2009;323: 498–501. 10.1126/science.1166426 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Masel J, Trotter MV. Robustness and evolvability. Trends Genet. 2010;26: 406–414. 10.1016/j.tig.2010.06.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Azevedo RBR, Lohaus R, Srinivasan S, Dang KK, Burch CL. Sexual reproduction selects for robustness and negative epistasis in artificial gene networks. Nature. 2006;440: 87–90. 10.1038/nature04488 [DOI] [PubMed] [Google Scholar]
- 51.Bergman A, Siegal ML. Evolutionary capacitance as a general feature of complex gene networks. Nature. 2003;424: 549–552. 10.1038/nature01765 [DOI] [PubMed] [Google Scholar]
- 52.Jarosz D. Hsp90: A Global Regulator of the Genotype-to-Phenotype Map in Cancers. Adv Cancer Res. Elsevier; 2016;129: 225–247. 10.1016/bs.acr.2015.11.001 [DOI] [PubMed] [Google Scholar]
- 53.Lindquist S. Protein folding sculpting evolutionary change. Cold Spring Harb Symp Quant Biol. Cold Spring Harbor Laboratory Press; 2009;74: 103–108. 10.1101/sqb.2009.74.043 [DOI] [PubMed] [Google Scholar]
- 54.Cowen LE, Carpenter AE, Matangkasombut O, Fink GR, Lindquist S. Genetic architecture of Hsp90-dependent drug resistance. Eukaryotic Cell. American Society for Microbiology; 2006;5: 2184–2188. 10.1128/EC.00274-06 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Paaby AB, Rockman MV. Cryptic genetic variation: evolution's hidden substrate. Nat Rev Genet. 2014;15: 247–258. 10.1038/nrg3688 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Masel J. Evolutionary capacitance may be favored by natural selection. Genetics. Genetics Society of America; 2005;170: 1359–1371. 10.1534/genetics.105.040493 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Levy SF, Siegal ML. Network hubs buffer environmental variation in Saccharomyces cerevisiae. Levchenko A, editor. PLoS Biol. Public Library of Science; 2008;6: e264 10.1371/journal.pbio.0060264 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Bates D, Mächler M, Ben Bolker, Walker S. Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software. 2015;67: 1–48. 10.18637/jss.v067.i01 [DOI] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
All data files are available from the Open Science Framework database (DOI 10.17605/OSF.IO/TN24X).