Discovery and Broad Relevance May Be Insignificant Components of Course-Based Undergraduate Research Experiences (CUREs) for Non-Biology Majors

Abstract

Course-based undergraduate research experiences (CUREs) are a type of laboratory learning environment associated with a science course, in which undergraduates participate in novel research. According to Auchincloss et al. (CBE Life Sci Educ 2104; 13:29–40), CUREs are distinct from other laboratory learning environments because they possess five core design components, and while national calls to improve STEM education have led to an increase in CURE programs nationally, less work has specifically focused on which core components are critical to achieving desired student outcomes. Here we use a backward elimination experimental design to test the importance of two CURE components for a population of non-biology majors: the experience of discovery and the production of data broadly relevant to the scientific or local community. We found nonsignificant impacts of either laboratory component on students’ academic performance, science self-efficacy, sense of project ownership, and perceived value of the laboratory experience. Our results challenge the assumption that all core components of CUREs are essential to achieve positive student outcomes when applied at scale.

INTRODUCTION

Engaging undergraduate science students in research experiences has a number of important benefits (1, 2). However, the traditional “apprenticeship” model of undergraduate research, in which a highly motivated student works as part of a faculty member’s research team, is typically restricted to a subset of developing scientists. A relatively recent approach for providing undergraduate students with opportunities to conduct research is the course-based undergraduate research experience (CURE) (3, 4). CUREs are scalable research experiences capable of engaging large numbers of students by involving the entire population of a course in a research question within the context of the course itself. This structure provides research experiences to students who would not otherwise participate in more traditional research, such as students in non-biology majors (hereafter “nonmajors”). For these students, a one-semester laboratory course may be the only formal scientific or research training they experience in college. CUREs offer opportunities for these students to gain valuable experience while also meeting course requirements (5, 6). CUREs can vary in their duration, setting, extent of mentoring, and cost depending upon the logistical restraints of the institution (7).

According to Auchincloss et al. (4), several core components define a CURE: 1) cycles of iterative experimentation followed by critical evaluation of data; 2) collaborative work with other students and/or the course instructor in order to address complex problems; 3) use of scientific practice through engagement with science investigations; 4) experience of discovery as students work on a novel question to arrive at a conclusion unknown to the students, instructor, and broader scientific community; 5) production of data broadly relevant to the scientific or local community (Fig. 1).

FIGURE 1.

Summary of differences and similarities among three laboratory learning environments: CURE, discovery-based inquiry, inquiry (see Ballen et al. [5]). Specifically, CUREs possesses all five core components; discovery-based labs lack broad relevance; inquiry labs lack discovery and broad relevance. We used a backward elimination experimental design to test the importance of one or more CURE components for student success. CURE = Course-based undergraduate research experience.

While these five core components provide a useful framework for thinking about the design and implementation of CUREs, there is little empirical evaluation of the importance of individual core components. In other words, we lack an understanding of whether each of the components relates to positive student outcomes, e.g., competencies, student attitudes, or retention in the discipline. Implementing all components simultaneously can be resource-intensive or difficult to facilitate and maintain in a classroom or laboratory setting over time, limiting the scalability of CUREs for some institutions. Therefore, it is essential that we justify the utility of these design features in a variety of contexts. Empirical validation of each component will allow for more efficient course design that maximizes the impact of course-based research for all students, and contributes new scientific knowledge to the scientific community (8).

Some of the core components highlighted in Auchincloss et al. (4) are fairly easy to understand, if not to implement. For example, the way students can experience the use of scientific practice (component 3) has been articulated by Lopatto (1) and echoed by Seymour et al. (9). They include understanding primary literature, designing experiments, collecting and interpreting data, and writing scientifically. Collaboration and iteration (components 2 and 1, respectively) are likewise unambiguous concepts. While the relevance of these hallmarks should also be critically explored, we chose to examine the importance of discovery and broad relevance (components 4 and 5, respectively) because we believe these are less tangible CURE components.

Discovery and broad relevance in CUREs

Discovery in science is the process of obtaining new knowledge that leads to new understanding of the natural world. In many laboratory experiments, students participate in a discovery exercise because the outcome of their investigation is new to them, but within a CURE, the outcome is unknown to both the students and the instructor. This “discovery with novelty” implies that students have the potential to contribute new knowledge to the field. Establishing this potential, via a careful understanding of the status of the field, is imperative. Arriving at this understanding requires a course facilitator (instructor or teaching assistant) with a solid grasp of the discipline and an awareness of the boundaries of knowledge. Developing the scientific novelty of the proposed work is relatively simple when the work involves an area in which the instructor is an expert. However, large courses with multiple lab sections are often taught by graduate students or undergraduates who are not experts in the discipline, and may be unable to judge the novelty of student research proposals. If novelty is in fact critical for obtaining the proclaimed benefits of a CURE, instructors will need to think seriously about creative ways to incorporate this aspect into their courses. In the current study, we collaborated with an expert in the discipline who could steer students toward novel questions.

Creating the opportunity for students’ work to be broadly relevant requires the involvement of one or more interested parties who exist beyond the classroom. Examples of interested parties include a research laboratory conducting work on a topic relevant to the CURE, a local community who benefits from the results of a CURE, or a publicly available database of student results that could further research in the field.

Questions and hypothesis

Of the CURE elements suggested by Auchincloss et al. (4), we found that both discovery and broad relevance require more logistical considerations than the other elements and are especially difficult to successfully execute in a large-enrollment course for nonmajors. This work is thus motivated by the overriding question: Do discovery and/or broad relevance matter in a laboratory experience geared toward nonmajors? In other words, do students who are working at the edge of scientific knowledge benefit from the novelty aspect of their work, or the fact that someone is interested in their findings?

We addressed these questions via a backward elimination experimental design, which involves some sections of a nonmajors’ biology course engaged in the course’s capstone CURE with all five core components, and testing the impact of eliminating one component at a time (Fig. 1). We hypothesized that experimental treatments would not influence students’ performance in the course, reported science self-efficacy and project ownership, and qualitative perceptions of the lab experience. We chose to examine self-efficacy because of its power to predict actual performance among students (10–12). We chose to measure self-reported project ownership because of prior demonstrated positive outcomes associated with independent research experiences for undergraduates (9, 13–15). Finally, we provided students with an opportunity to describe in open-ended responses their perceptions of the value of each laboratory experience, and we performed qualitative analyses on these responses. Our results have broad implications for the development of scalable CUREs in university curricula.

METHODS

Student population

Our student population included 412 students enrolled in an introductory biology course for nonmajors at the University of Minnesota in Minneapolis, MN. This course, Biology 1003: The Evolution and Biology of Sex, has the dubious distinction of being the favored course of the most science-phobic subset of the University’s student population (Cotner, unpublished data). Students come from a variety of academic backgrounds, range from incoming freshmen to graduating seniors, and are diverse with respect to age and racial/ethnic identity (Table 1). To control for the influence of instructor gender on any of the student outcome variables (e.g., 16), the two instructors involved in the courses were both women. The gender of teaching assistants (TAs) who guided labs varied across treatment groups (inquiry treatment TAs 75% women; discovery treatment TAs 100% women; CURE treatment TAs 50% women).

TABLE 1.

Student demographic information (%) across three laboratory treatments in introductory biology at the University of Minnesota.

	CURE (N = 115)	Discovery-based (N = 115)	Inquiry (N = 182)
Year in school
1st year	7.0	5.2	5.5
2nd year	43.5	47.8	54.9
3rd year	25.2	27.0	21.4
4th year	24.3	20.0	18.2
Race/ethnicity
American Indian	0.9	0.0	2.2
Asian American	6.1	10.4	7.1
African American	6.1	4.3	2.3
Hawaiian	1.7	0.0	0.5
Hispanic	3.5	4.3	4.4
International	16.5	14.0	14.3
White	65.2	67.0	69.2
Gender
Female	61.7	61.7	59.9
Male	38.3	38.3	40.1
College
Other STEM	8.7	6.9	7.1
Non-STEM	91.3	93.1	92.9

CURE = Course-based undergraduate research experience.

Experimental manipulation

This experiment included 18 laboratory sections across three large lecture sections of Biology 1003 in fall 2016. A significant portion of a student’s lab grade involved their work on a multi-week, collaborative research project examining an authentic dataset used in collaboration with the University of Minnesota’s Program in Human Sexuality (PHS). The laboratory activity, entitled Testing Hypotheses about Adolescent Sexual Behavior, occurs over five lab periods that take place once a week, and has students reading and discussing the literature about adolescent sexual behavior. For the learning exercise, we had students 1) observe and interpret a real, anonymized dataset; 2) develop a hypothesis to test using the dataset; 3) analyze the data to test their hypothesis; and 4) present the results of their research. The full exercise can be found in the course lab manual, the University of Minnesota’s The Evolution and Biology of Sex: Laboratory Investigations (17). Undergraduate or graduate-student TAs led the lab sessions of 20 to 24 students. We split students into one of three treatment groups and trained TAs to guide students through a CURE, a discovery-based inquiry, or an inquiry lab, as defined below (Fig. 1; see also Ballen et al. [5]):

1. The CURE treatment group (N = 115 students from 5 laboratory sections in lecture section 01) experienced all five core components of a CURE as defined by Auchincloss et al. (4): cycles of iterative experimentation, collaborative work, use of scientific practice, experience of discovery, and dissemination of data broadly relevant to the science community. Specifically, we required that students ask questions not previously addressed in the published literature after reviewing research already conducted with the PHS dataset, and after students presented their findings to the lab section, they emailed their presentations to a researcher at the Program in Human Sexuality (Newstrom). Prior to the onset of the CURE, Newstrom attended lecture section 01 to explain the importance of the research to students, and express his interest in student findings.

2. In the discovery-based inquiry treatment group (N = 115 students from 5 laboratory sections in lecture section 02), we required that students undertake four out of the five defining features of a CURE: cycles of iterative experimentation, collaborative work, use of scientific practice, and experience of discovery; we did not require they disseminate data broadly relevant to the science community. Specifically, students asked original questions after reviewing previous research conducted with the PHS dataset. Students presented these results to their lab section, but did not work with, or disseminate their results to, the researcher from the PHS.

3. In the inquiry treatment group (N = 182 students from 8 laboratory sections in lecture section 03), we required that students undertake three out of the five defining features of a CURE: cycles of iterative experimentation, collaborative work, and use of scientific practice; we did not facilitate an experience of discovery, nor require students to disseminate data broadly relevant to the science community. Specifically, students developed and pursued their own research questions about the PHS dataset (without any requirement to ask a novel question); furthermore, students did not interact with or disseminate their results to the researcher from the PHS. Students were assigned readings from the primary literature that highlighted research similar to that conducted on the PHS dataset.

Across treatments, students worked in groups to develop a hypothesis, learn basic statistical analyses (e.g., one-way ANOVA) using the statistical software JMP Pro 12 (SAS Institute Inc., Cary, NC, USA), analyze data, interpret their results, graphically depict their results, and prepare a written and oral presentation of their work. In all treatment groups, students presented their research to their lab section with a PowerPoint presentation, but we required that only the CURE treatment group send their presentations to Nicholas Newstrom at the PHS. The laboratory schedule and point allocation within laboratory and lecture can be found in Supplemental Table 1 and Supplemental Table 2 (Appendix 1).

DATA COLLECTION AND ANALYSIS

To test the importance of discovery and broad relevance in nonmajors’ CUREs, we conducted quantitative and qualitative analyses. First, we examined the effects of the treatments on student performance (lab grade), self-reported confidence in the ability to do science (“science self-efficacy”), and sense of project ownership. We addressed the following questions: 1) Does discovery or broad relevance improve student performance in lab, as compared with an inquiry laboratory that lacks these components? 2) How does discovery or broad relevance impact student science self-efficacy and sense of project ownership?

We performed all statistical analyses using IBM SPSS Statistics v.24. We first ran a post-hoc ANOVA to compare incoming student academic preparation among treatments. These only included students who finished the course. Then, we used general linear mixed models to compare student lab achievement (lab grades) and two affective metrics (science self-efficacy and project ownership) across the three treatment groups: CURE, discovery-based inquiry, and inquiry. We evaluated student performance based on total grade in lab because all laboratory sections are evaluated using the same manual and grading rubric. We did not use the research laboratory reports as a performance measure because laboratory teaching assistants graded them out of 8 points, and most students received full credit. All models include the same fixed and random variables, and we included a covariate in the performance model to address the variation in incoming academic preparation (student cumulative GPA). Fixed factors included laboratory treatment group, gender, underrepresented minority (URM) status, and age. We included the laboratory section as a random effect in all analyses. To fit the assumptions of the general linear model, we transformed students’ lab grades by taking the linear log of [120 – student grade]. For all Likert-scale analyses, we treated the dependent variables as continuous for ease of interpretation, given that nonparametric tests have yielded very similar results to the ones reported in this paper (18, 19). Prior to the analysis, we decided that it is unlikely that student characteristics (e.g., age, gender, race/ethnicity) would interact with treatments to influence the statistical outcomes, but we included them in all analyses because of their demonstrable effect on some performance outcomes (e.g., for confidence: 16, 20).

Using post-course surveys, we asked to what extent students felt confident comprehending, critically assessing, and communicating scientific concepts. Following Bandura’s (12) work on self-efficacy, we modified survey questions from an existing instrument (21) in which students rated confidence in their ability to complete course-relevant tasks. Responses were quantified on a four-point Likert scale: 1 = not confident; 2 = slightly confident; 3 = mostly confident; 4 = very confident (Appendix 2).

We conducted exploratory factor analyses on the 11 science self-efficacy survey items. We had adequate sampling to produce reliable results according to the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy for the whole dataset (KMO > 0.8). In order to test the presence of relationships between variables, we used Bartlett’s test of sphericity, which was significant (p < 0.001). Post-course surveys generated a single component that explained 58% of the total variance. We tested for internal consistency using Cronbach’s alpha, and found these survey items to be correlated (Cronbach’s alpha > 0.9). We then generated a single science self-efficacy response variable for each student using an additive scale. We re-ran the analysis after excluding seven outliers and the results were the same, so we include them here.

To examine student sense of ownership and perceptions of the laboratory experiences, we used five survey questions modified from Hanauer and Dolan (22); these responses were also quantified on a five-point Likert-type scale (Appendix 3). To test whether these data were suitable for factor reduction, we conducted an exploratory factor analysis. For project ownership, the KMO for the whole dataset was 0.833 and Bartlett’s test of sphericity was p < 0.001. The five survey items generated a single component that explained 61% of the total variance. We tested for internal consistency using Cronbach’s alpha, and found them to be correlated (Cronbach’s alpha > 0.8). In response to these results, we combined measures using an additive scale that represented a comprehensive project ownership score for analyses.

Students took the project ownership survey only once at the end of the course because it was designed to gauge student perceptions over the course of the laboratory experience. Figure 2 displays the student responses to the project ownership survey separately in order to illustrate nuanced results from the survey rather than a broader construct (which is more suitable for analyses with variable reduction).

FIGURE 2.

Mean scores (95% CI) reported by students on project-ownership survey items (Assessment 2) do not significantly differ across CURE (blue), discovery-based inquiry (yellow), and inquiry (orange) laboratory treatment groups (N = 302). The survey gauged to what extent students felt invested in the project (Investment), agreed that work on their project was broadly relevant beyond the classroom (Broad relevance), that there was the potential to discover something new (Discovery), that their research project was interesting (Interesting), and that they were responsible for the outcomes of the project (Independence). For all post hoc analyses of individual survey items, p > 0.15. CURE = Course-based undergraduate research experience.

Students were assured of anonymity during the course, confidentiality after the course, and the ability to omit any of the survey items. The surveys were approved by the University of Minnesota’s Institutional Review Board (#1405E50826). Of the 412 possible respondents, we secured post-course surveys from 302 students (73% response rate).

Qualitative analyses

Our second objective was to qualitatively explore, through open-ended responses submitted by students, perceptions of the value of each laboratory experience (CURE, discovery-based inquiry, or inquiry; N = 78 student responses). After asking a third party to collate and randomize student responses, two of the authors used inductive coding to generate six recurring themes: 1) real-world applications, 2) choice/ownership/discovery, 3) learned science process skills, 4) learned something new, 5) general interest in the topic, 6) required more guidance. From the 78 student responses, we coded 98 different statements into one of the six constructed themes (Table 2). We excluded six responses because they were unclear, addressed difficulty with the statistical software, or only expressed feelings about the lab TA. Any coding disputes were discussed and consensus reached before the analysis was done. We analyzed coding data by comparing the relative ratios of each coding theme as a percentage of the total number of coded responses within each treatment group (Cohen’s kappa > 70%).

TABLE 2.

Student-reported views in response to the open-ended questions, “Please comment on any aspect of you research project. Was it a valuable experience? What could your instructor or TA have done differently to help you make the most of your research experience?”

Response	Category Guide to Coding Responses	Examples
Real-World Applications	Words like “useful,” “outside connections,” “relevant,” “real world,” “relate,” and “helpful,” make connections from the project to the outside world	“This was my favorite experiment because it could be related to the real world and an overall big picture. It was also a topic that isn’t usually covered in a classroom setting so it was a new topic for almost everyone.”
Choice/Ownership/Discovery	Able to choose question/project/topic, express ownership of project/direction, or discover something for themselves	“I thought it was really cool to find our own relationship and think about the factors that contribute. I liked being able to pick what I wanted.”
Learn Science Process Skills (SPS)	Learning science process skills, or “how science is done”	“I think the final research project was an incredible way to cap off the semester, and we were able to use the things we learned throughout the course to come up with a hypothesis, test it, and make educated conclusions.”
Learn Something New	Learning something new; not related to science process skills	“It was fun looking through all of the information and learning about different aspects that affect adolescent sexual behavior.”
Learn Something Interesting	Mentions that the project was “interesting,” or wants to “know the answer” to their question	“It was an excellent learning experience and we discovered a lot of interesting data.”
More Guidance	Mentions needing more guidance on question/topic selection	“I should have chosen a more interesting subject. My subject we predicted and got it right easily.”

RESULTS

Quantitative results

We tested the effect of different laboratory environments on student performance, science self-efficacy, and sense of project ownership. First, an ANOVA comparing incoming student academic preparation among treatments revealed nonsignificant differences between student populations (cumulative GPA p = 0.155). Within our mixed models, we found no significant effect of laboratory treatment on laboratory grade (F_2,14.7 = 2.155, p = 0.151, N = 411), while cumulative GPA (F_1,397.5 = 168.350, p < 0.001) positively predicted laboratory grade, as did gender (F_1,402.9 = 4.950, p = 0.027), with male students outperforming female students. These results suggest that there are not statistically significant differences in laboratory performance among students who participate in an inquiry lab, discovery-based inquiry lab, or CURE.

Next, students’ post-course reported science self-efficacy did not differ based on laboratory treatment group (F_2,13.2 = 0.008, p = 0.992; N = 289 students completed the survey; Table 3), suggesting that belonging to different treatment groups did not impact students’ confidence in their skills related to conducting, communicating, and interpreting science. Note that we only measured science self-efficacy in one post-course survey and did not examine its change over the course of the semester. We assume that students in the three different treatment groups had roughly equivalent incoming measures.

TABLE 3.

Itemized means (SD) of science self-efficacy measures reveal no significant differences between treatment groups (all p > 0.15).

Please rate your level of confidence:	CURE (N = 84)	Discovery (N = 71)	Inquiry (N = 140)
Understand and evaluate scientific literature	3.04 (0.783)	3.06 (0.735)	2.99 (0.715)
Analyze a set of observations tables, or graphs to identify possible patterns	2.63 (0.788)	2.55 (0.713)	2.65 (0.719)
Pose questions about the observations that can be answered with an experiment	2.99 (0.799)	2.96 (0.726)	2.90 (0.733)
Develop a hypothesis related to a question that has been posed	2.96 (0.719)	3.01 (0.707)	2.94 (0.702)
Design a well-controlled experiment to test a hypothesis	3.05 (0.731)	3.14 (0.723)	3.06 (0.702)
Make predictions about the results I could get from an experiment	2.71 (0.769)	2.80 (0.786)	2.80 (0.741)
Collect, organize, and display the results of an experiment	3.11 (0.712)	3.13 (0.716)	3.06 (0.679)
Use statistics or other appropriate methods to analyze data	3.18 (0.779)	3.23 (0.778)	3.10 (0.720)
Draw conclusions about a hypothesis based on the results of the experiment	2.89 (0.870)	2.89 (0.854)	2.92 (0.720)
Explain an experiment, the results, and analysis orally	3.00 (0.760)	3.11 (0.747)	3.14 (0.637)
Explain an experiment, the results, and analysis in writing	3.01 (0.829)	3.07 (0.743)	3.03 (0.739)

CURE = Course-based undergraduate research experience.

Third, we carried out a similar analysis of student project ownership responses (Fig. 2). We found that being in the laboratory treatment group did not significantly affect students’ responses (F_2,15.2 = 0.023, p = 0.977; N = 302 students completed the survey). All other factors in the analyses were also nonsignificant (p > 0.15).

Qualitative results

We categorized 98 statements from 78 open-ended, post-course responses to survey questions that asked students to reflect on their laboratory experience. Student comments were categorized into one or more themes based on whether they mentioned the following in their open-ended response: real-world application, choice/ownership/discovery, learning science process skills, learning something new, or needing more guidance. Coding showed that students in the inquiry treatment commented on all six themes, whereas students in discovery and CURE treatment groups did not comment on learning science process skills or needing more guidance (Fig. 3; Table 4).

FIGURE 3.

Percentages of binned themes from open-ended responses by students about one of three laboratory experiences (CURE, discovery-based inquiry, or inquiry). We categorized responses based on whether students emphasized real-world application (dark blue), choice/ownership/discovery (orange), learning science process skills (SPS; purple), learning something new (yellow), or needing more guidance (green) in their answers. CURE = Course-based undergraduate research experience.

TABLE 4.

Percentages of binned themes from open-ended responses by students about one of three laboratory experiences (CURE, discovery-based inquiry, or inquiry).

	Real-World Applications	Choice/Ownership/Discovery	Learn SPS	Learning Something New	Learn Something Interesting	More Guidance	Statements in Survey Responses (N = 98)	Survey Responses (N = 78)
CURE	25.0%	16.7%	—	29.2%	29.2%	—	24	21
Discovery-based Inquiry	30.8%	23.1%	—	19.2%	26.9%	—	26	19
Inquiry	35.4%	10.4%	10.4%	14.6%	22.9%	6.3%	48	38

CURE = Course-based undergraduate research experience.

Overall, student comments from all the treatment groups were remarkably similar. This similarity suggests that there were not large differences in the overall student perceptions of their laboratory experience regardless of the treatment group they were assigned to. One difference was that only students in the inquiry treatment mentioned learning science process skills in their comments. This finding is consistent with other research in which students describe the inquiry lab as a “skill-building” opportunity (23). It is also interesting that students assigned to the inquiry laboratory commented on the need for more guidance in their labs. This feeling could be due to the limited role of the TA and primary literature in guiding their question and hypothesis creation, which may have made the inquiry exercise feel artificial to the students. However, we are reluctant to draw firm conclusions based on the percentages generated from these data because the total number of responses for the inquiry treatment was approximately double the number of responses for the discovery and CURE treatments (Table 4). Therefore, the lack of comments related to science process skills and guidance in the discovery and CURE groups could be due to limited sampling within those groups. In general, this analysis revealed little evidence of predominant themes in student comments that were unique to any one laboratory treatment group.

DISCUSSION

Our data show that discovery and broad relevance have insignificant effects on student performance, science self-efficacy, and sense of project ownership in our population of nonmajors. Instead, students across all laboratory treatment types found personal reliance to be important for determining the value of a research experience. We demonstrate that, for nonmajor students, inquiry approaches may be sufficient to achieve the measured outcomes for a laboratory learning environment. These findings should be relevant to instructors who desire course outcomes that mirror those we document here—course performance, science self-efficacy, and project ownership.

These results highlight the need to empirically evaluate the other design elements of CUREs in order to determine which contribute to positive student outcomes for both nonmajor and major student populations. For example, if students value the opportunity to choose their own research questions, such autonomy may be in direct conflict with instructors who seek high quality data through directed undergraduate CURE collaborations. It will also be important to test the generality of our results; for instance, a similar study examining an undergraduate biology majors population, or a graduate student population, may find stronger preferences for elements of discovery and broad relevance.

The limitations of this research also warrant consideration. CUREs are structured in a variety of ways, the most common of which are described on the CUREnet website (http://curenet.cns.utexas.edu/) and in the National Academies of Sciences, Engineering, and Medicine convocation report on discovery-based research courses (https://www.nap.edu/catalog/21851/integrating-discovery-based-research-into-the-undergraduate-curriculum-report-of). These describe more-traditional wet-bench research, while students in our lab worked with an established database. Our students also used data that related to human sexuality, which is a unique topic that might influence students’ responses. Finally, we chose three surveys that measure affective qualities and one measure of performance, but a number of different assessment tools would have allowed us to ask other specific questions to deliberately align teaching goals with practical outcomes. For example, the Test of Scientific Literacy Skills (TOSLS) quantifies student proficiency in using scientific concepts beyond the laboratory setting (24). In addition, the Test of Science-Related Attitudes (TOSRA) can be used to quantify favorable attitudes towards science and scientists (25). However, in the absence of results from these other measures, we conclude that there were no observable differences between laboratory treatment groups. Finally, our attempt to create treatment groups that reflected truly “broadly relevant” questions and provided students with a sense of “discovery with novelty” (investigating questions that are new to a field), while controlling for other factors that might influence student outcomes in other areas, were limited by logistical restraints inherent to working within three large introductory classrooms. For instance, it would have been ideal to provide each novice undergraduate with the opportunity to work directly with a principal investigator, to experience discovery as it is generated by the work conducted within the context of the researcher’s agenda. Students who experienced this type of hands-on research may have reported different attitudes at the end of the semester. These logistical constraints on our research are the same restraints that will limit sustained efforts to implement research experiences in large introductory courses, particularly at a large public institution.

Critically, it would be naïve to assume that all expert guidance is equal; few would claim that all primary instructors or teaching assistants are similarly skilled at facilitating inquiry. Thus, we assume that experts vary in their ability to communicate scientific facts and enthusiasm for the investigation, and to provide helpful feedback. In our CURE treatment, the expert’s involvement was minimal: meeting once with the students at the beginning of the project to introduce the research and convey interest in the students’ work, and being available to review ideas and final products. An expert who was involved with student projects on a weekly basis may have contributed differently to student outcomes. However, given our interest in scalability, the involvement we document is more practical for multiple sections of nonmajors introductory biology.

Our results may come as a relief to some instructors designing research experiences for nonmajors. Fully implanting the current CURE model—which requires incorporating expert input, providing students with unexplored data, and finding an audience that cares about the results—can be time consuming and impractical, especially for large courses with several laboratory sections. Furthermore, it may be difficult to find experts who are as enthused about working with nonmajors as they might be about working with developing scientists and potential future colleagues. Additionally, it is reasonable for instructors to have very different desired learning outcomes for nonmajor students than for major students, and when designing a laboratory experience a reverse design framework where the instructor uses the learning outcomes to determine appropriate student experiences should be applied (19). Instructors must think critically about whether the full CURE laboratory is the most appropriate way to achieve the desired learning outcomes for their students.

Overall, these findings indicate that instructor efforts to incorporate research into curricula may not require the additional—and often logistically difficult—steps of providing students with a sense of authentic discovery and broad relevance. Our results challenge the value of CUREs as they are currently defined, and support a call for a deeper understanding of why different laboratory environments are effective for both major and nonmajor student populations.