Abstract
Vision and Change calls for increasing the quantitative skills of biology majors, which includes neuroscience majors. Accordingly, we have devised a module to give students practice at regression analyses, covariance, and ANOVA.
This module consists of a quantitative comparative neuroanatomy lab in which students explore the size of the hippocampus relative to the brain in 62 different mammalian species—from an anteater to a zebu. We utilize a digital image library (with appropriate metadata) allowing students to quantify the size of the hippocampus as well as obtain an index of the size of the brain in these various species.
Students then answer the following questions: (1) Do brains scale with body size? (2) Does the hippocampus scale with brain size? (3) If we control for body size, does the hippocampus still scale with brain size? (4) How does the hippocampus change as a proportion of brain size? (5) Is the proportional scaling of the hippocampus different among primates, carnivores, and other mammals? (6) Do the data provide evidence for mosaic or concerted evolution?
Measures of the pedagogical efficacy showed clear and significant gains on a PreTest vs PostTest assessment of material related to the module. An open ended qualitative measure revealed students’ perception of the purposes of the module, which were consistent with the learning goals.
This module utilizes open access digital resources and can be performed at any institution. All the materials or links to online resources can be found at https://mdcune.psych.ucla.edu/modules/cna.
Keywords: Quantitative Skills, Comparative Neuroanatomy, Neuroanatomy, Hippocampus, Digital Learning, Vision and Change, Pedagogy, STEM, Higher Education
The learning objectives of this module were to: (1) introduce students to comparative neuroanatomy, (2) expand students’ understanding of hippocampal structure and function, (3) enhance students’ communication skills, and most particularly (4) solidify and develop students’ understanding of data analyses, including linear modeling. The latter objective was inspired by Vision and Change (Ledbetter, 2012; Bauerle et al., 2013). Vision and Change is a handbook that outlines curricular considerations for the biological sciences in general. Vision and Change notes that biology is becoming more quantitative and that models of structure and function are enriched if they are based on the quantitative analysis of data. Our instruction must reflect these changes in our field by including more quantitative analyses. In our experience, data analyses are relegated to statistics classes, and students quickly forget these lessons, perhaps because they are so divorced from real data.
In this module, students need to use logarithmic transformations of data, covariance, ANOVAs, and linear regression. Linear regression and linear modeling are analytical tools that often get relatively neglected in undergraduate statistics courses in the curricula of neuroscience, biology, and psychology. Yet, linear techniques are being increasingly utilized not only in neuroscience, biology, and other fields for such diverse things as modeling spike trains (Gerwinn et al., 2010), fMRI analyses (Beckmann et al., 2003), and in education research (Freeman et al., 2007; White et al., 2017). Here we present an exercise that utilizes both linear regression and covariance as a means of controlling for extraneous variables. This latter point is noteworthy because ANCOVA is commonly used in studies with humans in which many variables cannot be experimentally controlled.
This module also provides an opportunity to teach comparative and quantitative neuroanatomy by allowing students to explore how aspects the brain have changed across the mammalian radiation. Students may be able to utilize the materials provided by this module to conceivably break new ground and have the best (if not the only) characterization of a given brain area in a given species. Thus, students could engage in genuine research that adds to the extant body of knowledge in the field, which is also consistent with Vision and Change. Further, this module considers the proportion of the brain devoted to a given structure, which is not often done in the literature, thus providing students a unique insight in contemplating evolution.
So far, we only had students quantify the size of the hippocampus, which is readily identifiable (except in the dolphin brain in which the hippocampus is exceptionally small and relatively obscure). Students initially predict how they would expect the absolute size of the hippocampus to change across mammalian brains from smaller to larger species. We also ask students to predict how the hippocampus will change proportionally to the rest of the brain as mammalian brains get larger across species. Typically, students predict that the hippocampus will get larger with larger brains and that the proportion will either not change or get larger.
METHODS
Methods for Conducting Student Labs
We initially orient the students to the task by making them aware of several different atlases, which can serve as exemplars for the many varied species that they may be encountering. Specifically, we direct them to the INCF Rhesus Atlas (see URL List #2), the Allen Brain mouse atlas (URL List #3), and atlases from Michigan State University of the sheep (URL List #4), and dolphin (URL List #5).
Students employ a set of free online images of 62 different mammalian species from 20 different orders, including 10 primates and 17 carnivores from the Comparative Mammalian Brain Collections (URL List #6; see also Fobbs and Johnson, 2011). We have made these images available via our website (URL List #1) along with the metadata (slice thickness, sampling interval, and scale factors), which are necessary for calculating volumes. On our website, we provide a tutorial guide that will enable other instructors to easily adapt this module at their home institutions. Further, our website includes a complete set of sample data (URL List #1—see Comparative Neuroanatomy Sample Data) against which instructors may check their students’ data (generally, we do not raise questions unless student values differ from the historical data by ten-fold, in which case there was probably a mistake in the scaling factors employed with ImageJ).
We use ImageJ (URL List #7) to quantify the sizes of the hippocampus and the volume of the sections in which the hippocampus can be found. ImageJ was selected because it yields values that are very close to methods such as the Cavalieri method (Acer et al., 2010), and students are familiar with drawing programs that function like ImageJ. The scale factors for each species are different, but we have provided these as a part of the metadata spreadsheet on the MDCUNE website (URL List #1—see Comparative Neuroanatomy Demographics). Specific instructions are also provided for quantifying the hippocampus (URL List #1—see Comparative Neuroanatomy Tutorial). Students are typically assigned two to four species. For each species, they quantify the hippocampus in one hemisphere then double this volume to approximate the whole structure (Figure 1). We do not have students quantify the size of the entire brain since this would be a tedious process. Rather, they quantify the volume of the brain in which the hippocampus occurred and use this measure as a proxy for brain size (URL List #1—see Comparative Neuroanatomy Tutorial). Using a proxy for brain size or even a part of the brain such as telencephalon is an accepted practice in the literature (Grisham et al., 2007). It usually takes students about an hour to process a given brain, which includes time for instruction and orientation. If instructors wish to have their students quantify the size of the entire brain, all sections are included in our image sets.
Figure 1.
Sample of coronal sections containing the hippocampus (outlined in magenta) in a small set of the species for which sections are available.
We have used the free data analysis package, JASP (URL List #8), for students to analyze their data. The JASP interface resembles SPSS—and is quite user friendly—but is actually running R. One caveat of JASP is that it does not calculate the standard error of the mean (SEM) properly. (All of its other values—when compared with SPSS—are correct). As a result, we ask students to use JASP’s standard deviation (SD) and sample size values to calculate the SEM using Excel. Figures 2–7 below are derived from a student-generated dataset.
Figure 2.
Example of how student data look when logarithmic scaling is not used; students can clearly see the necessity of a log transform of their data.
Figure 3.
log10 of the associated brain volume (the volume—in cubic millimeters—of only the sections containing the hippocampus) as a function of body weight (in kilograms). The trend line is the least squares/best fit/regression line. Clearly body weight predicts brain size well.
Figure 4.
Relationship of the log10 hippocampal volume as a function of the log10 associated brain volume.
Figure 5.
Statistical analysis shows that when body weight is covaried—and thus controlled for—there is still some unique variance that is predicted by brain volume (see highlighted p value). With more than one predictor, students should pay attention to the Adjusted R2.
Figure 6.
Student data scatter plot and regression lines of the proportion of the brain that is the hippocampus as a function of the log10 associated brain volume.
Figure 7.
Student data (means and SEMs) displaying the proportion of the brain that is devoted to hippocampus as a function of taxonomic group. A significantly smaller proportion of the carnivore and primate brain is devoted to hippocampus relative to the other orders in the sample, which were combined into one group due to the small sample size available for any individual order.
Methods for Measures of Pedagogical Efficacy
We obtained an IRB Exemption from UCLA (IRB#16-001489) for conducting research on the efficacy of this module. This module was taught across two summer sessions at UCLA, and scores were combined across sessions. We collected demographics on the students, which were as follows: 33 were female, 20 were male; 22 students were Asian, 12 Latinx, 13 white, and 6 other/multiple races. Subjects participating in the module consisted of 6 neuroscience majors, 6 psychology majors, 2 cognitive science majors, and the remaining 39 were psychobiology majors. We also obtained data from a comparison group during UCLA’s winter quarter that was not exposed to the module (and given a different module instead). The comparison group demographics were as follows: 11 women, 8 men; 6 Asian, 1 Latinx, 9 white, and 3 other/multiple races. All were neuroscience majors.
We administered a 16-item test in a Pre-Post format that measured the content of the module with particular emphasis on statistics, evolution, and the hippocampus (we ultimately scored responses on only 14 of the items because upon reconsideration, 2 questions did not have a clear correct answer). Along with the content items, we included an additional 4-item scale of critical thinking questions that did not relate to the content of the module. In the PostTest version for students who had experienced the module, we added a 13-item affective Likert scale questionnaire to measure student satisfaction with different aspects of the module. Finally, along with the PostTest, we included an open-ended qualitative item, “Please describe the purpose of the Comparative Neuroanatomy module from a learning standpoint.” The Pre-Post test (with the answers highlighted for both content and critical thinking items) is included as supplemental materials (Supplementary Material #1—note that this supplement also includes the affective and qualitative items we used).
RESULTS
Results of Mammalian Brain Measurements
We have students address a series of questions in their data analyses: (1) Do brains scale with body size? (2) Does the hippocampus scale with brain size? (3) If we control for body size, does the hippocampus still scale with brain size? (4) How does the hippocampus change as a proportion of brain size? (5) Is the proportional scaling of the hippocampus different among primates, carnivores, and other mammals? (6) Do the data provide evidence of mosaic or concerted evolution?
(1) Do brains scale with body size?
Since body weights and sex were not known for the individual brains, students looked up standard body weights for their given species on Wikipedia then calculated an average if that species is sexually dimorphic or has a range of adult sizes. We also have students discover why it is necessary to logarithmically transform the data—if they do not log transform the data, most of the points crash onto the Y-axis because there are such tremendous differences in body and brain sizes, for instance, between an elephant shrew and a polar bear (Figure 2). Students then see if there is a relationship between the log10 brain size as function of log10 body size (not surprisingly, there is—see Figure 3). Most published work in this field examines the direct measures, such as the log10 hippocampus volume as a function of the log10 brain volume (Finlay and Darlington, 1995; Reep et al., 2007).
(2) Does the hippocampus scale with brain size?
Not surprisingly, bigger brains come with bigger parts, so students find that the log10 of the hippocampus size scales with log10 brain size, and students can usually predict this outcome with confidence (Figure 4). What comes as a surprise is the proportion of brain devoted to hippocampus (see Question 4 below).
(3) If we control for body size, does the hippocampus still scale with brain size?
We then raise the question of whether or not all the variance is actually due to body size, or if there is something unique to brain growth (e.g., a specialized set of genes) that makes brains a better predictor of hippocampus size? To answer this question, we have students use body weight as a covariate. Students find that body size actually has very little impact on the relationship between hippocampal size and brain size—that is, there is still variance unique to brain size as a predictor of hippocampal size when covarying for body size (Figure 5). Students are often unfamiliar with using covariates to control for a given variable(s), so this is a good introduction to these ideas.
(4) How does the hippocampus change as a proportion of brain size?
When the proportion of brain that is hippocampus is plotted as a function of log10 brain size, students find a significant inverse relationship (Figure 6).
(5) Is the proportional scaling of the hippocampus different among primates, carnivores, and other mammals?
Mammals with larger brains devote a smaller proportion of their brain to hippocampus than do mammals with smaller brains (Figure 6). Since our sample of brains has a lot of primates and carnivores, we also have students examine these groups by running an ANOVA using taxonomic group as the independent variable. Typically, students find that primates have a significantly smaller proportional hippocampus (Figure 7). Students should also perform an ANCOVA and include brain size as a covariate to make sure that the result is not merely driven by brain size (it is not).
(6) Do the data provide evidence of mosaic or concerted evolution?
Concerted evolution holds that various brain components evolve together whereas mosaic evolution holds that various brain components evolve independently from each other. The University of Virginia hosts some websites that deal well with these topics (URL List #9-10; Finlay and Darlington, 1995; Striedter, 2005). Since the log10 data show a strong linear relationship, the data favor concerted evolution.
Results of Pedagogical Measures
We used a mixed design ANOVA that had one between-subjects factor with two levels (Exposed to module vs Unexposed) and one within-subjects factor with two levels (Pre vs Post) to examine the results of our content items relating to the module. We did not obtain a main effect of exposure to the module (F(1,70) = 1.39, p = 0.23), nor did we get a main effect of PreTest vs PostTest (F(1,70) = 2.54, p = 0.115), but we did find a highly significant interaction of exposure to module with PreTest vs PostTest (F(1,70) = 24.327, p < 0.001, η2 = 0.25; refer to Figure 8). Students exposed to the module showed clear gains on the PostTest relative to the PreTest on items relating to the module, t(29) = −7.692, p < 0.001 (Cohen’s d = −1.057, 95% CI = −20.56 to −12.05). The comparison group of students that was not exposed to the module actually decreased performance on the PostTest relative to the PreTest t(17) = 2.185, p < 0.05 (Cohen’s d = 0.515, 95% CI = 0.205 to 11.7).
Figure 8.
Percent correct (means and SEMs) on the content-item subscale of a quiz that was administered before and after exposure either to the module (Exposed) or to a different module (Unexposed).
We also analyzed the scores on the critical thinking items in a similar mixed design ANOVA that had one between-subjects factor with two levels (Exposed to module vs Unexposed) and one within-subjects factor with two levels (Pre vs Post). There was a marginally significant trend showing improvement on the critical thinking items (Pre vs Post F(1,70) = 3.839, p = 0.05, χ2 = 0.052; refer to Figure 9), but there was no differential change between the groups in Pre vs Post, so no interaction nor main effect of exposure to the module was found (both p > 0.25; refer to Figure 9).
Figure 9.
Percent correct (means and SEMs) on critical thinking-item subscale of a quiz that was administered before and after exposure either to the module (Exposed) or to a different module (Unexposed).
Affective/opinion data showed that students agreed that they had learned something about comparative neuroanatomy and statistics, that the module had been interesting, and that the instructional materials were clear (Figure 10; full data are available in Supplementary Material #2).
Figure 10.
Results of an affective Likert questionnaire. The graphs represent the following questions: (Evaluation Question 2) The Comparative Neuroanatomy lab tutorial was thorough and precise with its instructions. (Evaluation Question 7) I learned something about comparative neuroanatomy from the Comparative Neuroanatomy module. (Evaluation Question 11) I learned something about statistics from the Comparative Neuroanatomy module. (Evaluation Question 12) I found it interesting to compare the brain structures of different mammals in the Comparative Neuroanatomy module.
Finally, students were asked the open-ended question, “Please describe the purpose of the Comparative Neuroanatomy module from a learning standpoint.” The following seven categories were considered in analyzing this question: (1) Comparative measurement/Relationship of Hippocampal/Structure Size, (2) Brain Function/Evolution, (3) Quantification Methods, (4) Statistics/Data Analysis, (5) Research Methods, (6) Problem Solving, (7) Critical Thinking. Each response could (and often did) qualify in multiple categories, so a given response could be classed in more than one category. A “category tally” was kept for every response, and a constant “response tally” was calculated from the sum of all category tallies. Each category tally was converted to a percent of the response tally, and these results are summarized in the pie graph in Figure 11. Responses were fairly consistent with the learning objectives of the module.
Figure 11.
Percentages of responses that students gave to the following open-ended qualitative question: “Please describe the purpose of the Comparative Neuroanatomy module from a learning standpoint.” Note that each response could be quantified in multiple categories.
The students were assessed via an APA-style laboratory report. We used scaffolded instruction in teaching this module, which included having students write a draft of their introduction, which was then read and critiqued one-on-one with the student and either the instructor or a teaching assistant. This was an extremely low-stakes assignment, and the students overwhelmingly appreciated the feedback. Further, their introductions were better focused and the writing greatly improved by the final product.
DISCUSSION
The data produced by the students in this exercise were very robust: both summer session cohorts that were exposed to the Comparative Neuroanatomy module obtained significant and consistent results. Students appreciated getting significant results, and they seemed to enjoy the fact that they were among the first to quantify a brain region in a particular species.
We chose to quantify the hippocampus because students could readily recognize this neural structure (except for in dolphins, in which case referring to Jacobs et al., 1979, is recommended). Conceivably, any neural structure could be utilized as long as it is well delineated by Nissl or fiber stains. Indeed, students could be breaking new ground since a quantitative comparative description of many neural structures across mammalian species has never been undertaken.
Clear pedagogical gains produced by the Comparative Neuroanatomy module are shown by the significant increase in content items scores with large effect sizes for the Summer Session cohorts that experienced the module relative to the Winter Quarter cohort, which did not. Content assessment items are available in Supplementary Material #1, which includes questions about statistical analyses, brain evolution, and functions of the hippocampus. Thus, we are confident that the module provided students with a pedagogical experience of value.
Although the data showed significant gains on the content items, the overall performance on the PostTest was still rather low. Examination of the questions showed that the gains as well as their lack were fairly evenly distributed among items that touched on statistics, evolution, and the hippocampus. We were teaching seniors, so we assumed that they would have a certain level of retention from previous courses. This assumption may have been flawed, however. Better would have been to assume that the students had little background.
The Pre vs Post comparison on the critical thinking subscale suggests that critical thinking was enhanced whether or not participants were exposed to the module. This result was somewhat surprising since it is only a four-item subscale, and critical thinking scales with few items suffer from low reliability (Liu et al., 2014). Given that students’ critical thinking scores in both groups increased, there is a possibility that this handful of items is especially sensitive to score changes due to a pretest sensitization effect (Willson and Putnam, 1982). Regardless, gains in critical thinking cannot be attributable to exposure to the Comparative Neuroanatomy module.
Affective/opinion items invariably resulted in high ratings for the module overall (Figure 10; Supplementary Material #2). Thus, students seem to like the exercise. The open-ended question produced insightful statements on the purpose of the module that were mostly consistent with the learning goals.
Obstacles and challenges
Students seemed quite uninformed about phylogeny, despite having taken several biology courses. Students seemed particularly confused because phylogenic lineages do not necessarily follow the you-are-what-you-eat rule. For example, pandas almost never eat meat, but are molecularly classified as true bears, and therefore they are carnivores. Also, humans often eat meat but are primates rather than carnivores. We provided students with the mammalian order to which each of our species belong. In future presentations of this module, it would probably be more edifying for the students to obtain this information from their own Internet research. This would give them a better sense of what the animals actually looked like and what constitutes a carnivore versus a primate, versus none of the above.
Going forward, we also plan to have students further analyze the regression data by comparing the slopes of the regression lines for the various taxa to each other (Figure 6; templates for doing this are provided on the MDCUNE website, URL List #1—see Comparative Neuroanatomy Analysis of Slope Differences by Order). Comparing slopes would help expand their use and practice with linear modeling.
The first time we taught this module, we emphasized concerted versus mosaic evolution. Concerted evolution describes a circumstance in which changes in one body part correlate highly with changes in related body parts, say the size of the hippocampus and the size of the whole brain. Mosaic evolution, in contrast, describes a situation in which changes in one given body part is not predicted by the size of related parts. For instance, the size of the inferior colliculus in echo locating bats is substantially larger than would be predicted by the size of the rest of the brain (Covey, 2005). Students were asked which idea fit their data better, concerted or mosaic evolution. They seemed to struggle with making the case for either idea and interpreting their data in the light of theory. (The data are more consistent with concerted evolution as per Figure 4 above, despite the fact that the proportion of brain devoted to hippocampus declines as brains get larger.) Accordingly, during our second summer offering of this module, we downplayed interpreting the data in light of the two competing hypotheses. In retrospect, it is probably a stronger module if students have to use their data to support a theoretical conclusion.
CONCLUSIONS
Vision and Change suggests instructing students in skills of posing problems, generating hypotheses, designing experiments, observing nature, testing hypotheses, and interpreting and evaluating data to fathom their implications. Learning more and varied statistical analyses will prove to be valuable tools. These tools and skill sets would be valuable not only in biology and neuroscience but also in many other endeavors. As instructors, we need to focus on skill sets such as the statistical analyses presented by this module. Teaching skill sets will prove ultimately to be the more efficacious pedagogical approach, and one that will benefit our students regardless of their careers.
SUPPLEMENTARY MATERIALS
Comparative Neuroanatomy: PreTest and PostTest (Key)
Comparative Neuroanatomy: Affective/Opinion Data (Summer 2016 and Summer 2017)
APPENDIX. URL LIST
UCLA MDCUNE: Comparative Neuroanatomy (CNA) https://mdcune.psych.ucla.edu/modules/cna/
Scalable Brain Atlas: Macaque – NeuroMaps Atlas https://scalablebrainatlas.incf.org/macaque/DB09
Allen Mouse Brain Atlas http://mouse.brain-map.org/
Michigan State University Brain Biodiversity Bank: The Sheep Brain Atlas https://msu.edu/~brains/brains/sheep/
Michigan State University Brain Biodiversity Bank: The Dolphin Brain Atlas https://msu.edu/~brains/brains/dolphin/
Comparative Mammalian Brain Collections http://neurosciencelibrary.org/
ImageJ https://imagej.nih.gov/ij/
UVA Psychology 5559: Evolutionary Neuroscience (The Evolution of Brain Size: Mosaic & Concerted Evolution) https://pages.shanti.virginia.edu/PSYC_5559_Evol_Neurosci/the-evolution-of-brain-size-debates-and-metabolic-constraints/
UVA Psychology 5559: Evolutionary Neuroscience (Mosaic Evolution) https://pages.shanti.virginia.edu/PSYC_5559_Evol_Neurosci/the-evolution-of-brain-size-debates-and-metabolic-constraints/sub-page-1/
Footnotes
This article is dedicated to John (Jack) Johnson, Walley Welker, and Adrianne Noe, who built the Comparative Mammalian Brain Collections upon which this teaching module is based. Thanks also to Hope Schaaf-Grisham, who helped download the image dataset.
REFERENCES
- Acer N, Uğurlu N, Uysal DD, Unur E, Turgut M, Camurdanoğlu M. Comparison of two volumetric techniques for estimating volume of intracerebral ventricles using magnetic resonance imaging: a stereological study. Anat Sci Int. 2010;85(3):131–139. doi: 10.1007/s12565-009-0068-4. Epub 2010 Jan 22. [DOI] [PubMed] [Google Scholar]
- Bauerle C, DePass A, Lynn D, O’Connor C, Singer S, Withers M, Anderson CW, Donovan S, Drew S, Ebert-May D, Gross L, Hoskins SG, Labov J, Lopatto D, McClatchey W, Varma-Nelson P, Pelaez N, Poston M, Tanner K, Wessner D, White H, Wood W, Wubah D. In: Vision and change in undergraduate biology education: a call to action. Smith D, editor. Brewer, CA: 2013. Nov, Retrieved from http://visionandchange.org/files/2013/11/aaas-VISchange-web1113.pdf. [Google Scholar]
- Beckmann CF, Jenkinson M, Smith SM. General multilevel linear modeling for group analysis in FMRI. Neuroimage. 2003;20(2):1052–1063. doi: 10.1016/S1053-8119(03)00435-X. [DOI] [PubMed] [Google Scholar]
- Covey E. Neurobiological specializations in echolocating bats. Anat Rec A Discov Mol Cell Evol Biol. 2005;287(1):1103–1116. doi: 10.1002/ar.a.20254. [DOI] [PubMed] [Google Scholar]
- Finlay BL, Darlington RB. Linked regularities in the development and evolution of mammalian brains. Science. 1995;268(5217):1578–1584. doi: 10.1126/science.7777856. [DOI] [PubMed] [Google Scholar]
- Fobbs AJ, Johnson JI. Brain collections at the National Museum of Health and Medicine. Ann N Y Acad Sci. 2011;1225(Suppl 1):E20–29. doi: 10.1111/j.1749-6632.2011.06036.x. [DOI] [PubMed] [Google Scholar]
- Freeman S, O’Connor E, Parks JW, Cunningham M, Hurley D, Haak D, Dirks C, Wenderoth MP. Prescribed active learning increases performance in introductory biology. CBE Life Sci Educ. 2007;6(2):132–139. doi: 10.1187/cbe.06-09-0194. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gerwinn S, Macke JH, Bethge M. Bayesian inference for generalized linear models for spiking neurons. Front Comput Neurosci. 2010;4:12. doi: 10.3389/fncom.2010.00012. eCollection 2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Grisham W, Park SH, Hsia JK, Kim C, Leung MC, Kim L, Arnold AP. Effects of long-term flutamide treatment during development in zebra finches. Neurosci Lett. 2007;418:92–96. doi: 10.1016/j.neulet.2007.03.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jacobs MS, McFarland WL, Morgane PJ. The anatomy of the brain of the bottlenose dolphin (Tursiops truncatus). Rhinic lobe (Rhinencephalon): The archicortex. Brain Res Bull. 1979;4(Suppl 1):1–108. doi: 10.1016/0361-9230(79)90299-5. [DOI] [PubMed] [Google Scholar]
- Ledbetter ML. Vision and change in undergraduate biology education: a call to action presentation to Faculty for Undergraduate Neuroscience, July 2011. J Undergrad Neurosci Educ. 2012;11(1):A22–A26. [PMC free article] [PubMed] [Google Scholar]
- Liu OL, Lois F, Roohr KC. Research report ETS RR-14-10 assessing critical thinking in higher education: current state and directions for next-generation assessment. 2014. (ETS Research Report Series). [DOI] [Google Scholar]
- Reep RL, Finlay BL, Darlington RB. The limbic system in Mammalian brain evolution. Brain Behav Evol. 2007;70(1):57–70. doi: 10.1159/000101491. [DOI] [PubMed] [Google Scholar]
- Striedter GF. Principles of Brain Evolution. Sunderland, MA: Sinauer Associates, Inc; 2005. [Google Scholar]
- White AO, Javier LK, Goldberg NR, Boucquey V, Overman J, Ochaba J, Marsh S, Huffman D, Nicholas A. Front and back flipping for neurobiology! Developing a hybrid upper-division lab course. J Undergrad Neurosci Educ. 2017;16(1):A95–A101. eCollection: 2017 Fall. [PMC free article] [PubMed] [Google Scholar]
- Willson VL, Putnam RR. A meta-analysis of pretest sensitization effects in experimental design. Am Educ Res J. 1982;19(2):249–258. doi: 10.2307/1162568. [DOI] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Comparative Neuroanatomy: PreTest and PostTest (Key)
Comparative Neuroanatomy: Affective/Opinion Data (Summer 2016 and Summer 2017)