Open Inquiry versus Broadly Relevant Short-Term Research Experiences for Non-Biology Majors

Abstract

Undergraduate student participation in course-based research experiences results in many positive outcomes, but there is a lack of evidence demonstrating which elements of a research experience are necessary, especially for non-biology majors. Broad relevance is one element that can be logistically challenging to incorporate into research experiences in large-enrollment courses. We investigated the impacts of broad relevance in a short-term research experience in an introductory biology course for non-majors. Students either participated in an open-inquiry research experience (OI-RE), where they developed their own research question, or a broadly relevant research experience (BR-RE), where they investigated a question assigned to them that was relevant to an ongoing research project. We found a significant association between the type of research project experienced and students’ preference for an experience, with half of the students in the OI-RE group and nearly all students in the BR-RE group preferring a broadly relevant research experience. However, since science confidence increased over the course for both groups, these findings indicate that while students who participated in a BR-RE valued it, broadly relevant research experiences may not be necessary for positive outcomes for non-majors.

INTRODUCTION

There are many types of undergraduate research experiences, ranging from research apprenticeships to course-based research experiences. Research apprenticeships increase students’ understanding of the process of science, science confidence [we are using the term “confidence” to be consistent with our earlier work (1, 2), yet we see confidence as synonymous with self-efficacy, as defined by Marshman et al. (3), who defined self-efficacy as “...the belief in one’s capability to be successful in a particular task, course, or subject area”], and persistence in science, technology, engineering, and mathematics (STEM) (4–7), but opportunities for research apprenticeships are limited to relatively few students. In contrast, course-based research experiences provide research opportunities for many students, and student participation in course-based research experiences also results in many positive outcomes, including gains in knowledge, understanding of the process of science, process of science skills, science confidence, interest in science, and persistence in STEM (8–11).

One framework for understanding the positive impacts of course-based research experiences is provided by Self-Determination Theory (SDT) (12, 13), a theory of human motivation which posits that individuals thrive in contexts that are supportive of their autonomy, competence, and relatedness. In an educational context, autonomy refers to a person’s perception of having agency—the ability to make choices and express oneself—in the learning activity; competence means the person is able to master a skill or understand content; relatedness refers to an individual’s feeling supported by, and connected with, others. Given the variability inherent in these research experiences, it is logical to assume that these experiences would vary in how they leverage student autonomy, competence, and relatedness to achieve optimal student benefit. In the current work, we focus on short-term research experiences that differ in their relevance to the world beyond the classroom, and the degree to which students can direct their own research experience.

Short-term research experiences (SREs) are course-based laboratory experiences that involve students in a short-term, broadly relevant research project (14). In the context of an SRE, a short-term experience is two or more weeks, but less than the length of a full semester, and broadly relevant research projects involve work that is relevant beyond the classroom and important to the broader scientific community (14). Participating in real world, relevant, collaborative experiences that connect the classroom to the outside world can lead to meaningful learning (15–17). However, making these experiences relevant beyond the classroom is logistically challenging in a large-enrollment course: relevance beyond the classroom often involves a principal investigator (PI)-driven project and consequently the commitment of a PI (11, 18). For the work to have the potential to contribute to ongoing research, students must not deviate from established procedures, meaning that an in-person facilitator, familiar with the research, is typically necessary to ensure procedural fidelity (18, 19). In some cases, the work is costly and these extra costs may not be covered by the PI (20). Given these challenges, it is critical to determine whether the broadly relevant element of course-based research is necessary for the positive student outcomes sought for the course, especially for non-biology majors who may be less motivated by the broad relevance aspect of the work than majors.

Non-biology majors are different than biology majors in many ways, and these differences may influence which elements of research experiences are necessary for positive outcomes for each of these student populations. Non-majors have lower motivation, interest, and confidence, and less positive attitudes with respect to science than majors (2, 21–23). Additionally, course goals for these two populations are often different, with goals for non-majors often focused on encouraging positive perceptions of science and increased science literacy and goals for majors often focused on developing laboratory skills and persistence in their major (24). With most of the research showing that participation in course-based research experiences results in positive student outcomes focused on majors (8, 14, 25) and very few studies having investigated the impacts of research experiences on non-majors (1, 24, 26), additional research is needed to understand not only which elements of research experiences are necessary for positive student outcomes, but also which elements are necessary for specific student populations (i.e., biology majors or non-majors).

Our research questions were:

1. What are non-majors’ perceptions of broadly relevant research experiences? Do they differ from non-majors’ perceptions of open-inquiry research experiences?

2. Is broad relevance a necessary component of a research experience to achieve positive outcomes for non-biology majors?

To answer these questions, non-biology majors who were enrolled in an introductory biology course either participated in an open-inquiry research experience or a broadly relevant research experience. We analyzed students’ precourse, post–research experience (midcourse), and postcourse responses to surveys that asked about their perceptions of their research experiences, research experience preferences, and affective characteristics (e.g., science confidence).

METHODS

Participants

Participants were undergraduate students enrolled in an introductory biology course for non-biology majors at the University of Minnesota during the fall 2017, spring 2018, and fall 2018 semesters. Students completed pre-, mid-, and postcourse surveys, and their demographic information was collected from institutional data sources. The University of Minnesota Institutional Review Board reviewed the research plans and granted an exempt status (Study 1405E50826). All students consented to participate and were free to opt out of the study.

Research experiences

Students were enrolled in Environmental Biology: Science and Solutions, an introductory biology course for non-majors that explores the science behind environmental topics. There were four laboratory sections each semester with 20 to 24 students enrolled in each section. Students were assigned to one of two short-term research projects —an open-inquiry research experience (OI-RE) or a broadly relevant research experience (BR-RE)—based on the laboratory section number they were enrolled in. Half of the laboratory sections were assigned to each research experience each semester—laboratory section numbers 2 and 3 were assigned to the OI-RE group, and laboratory section numbers 4 and 5 were assigned to the BR-RE group.

OI-RE group

Students who participated in an OI-RE asked their own research questions using an established model system. There was no expectation that students ask novel questions, generate original results, or inform any community beyond the classroom. Specifically, students pursued a 3-week research project about Daphnia and their tolerance to various toxic chemicals. This work was aligned with the course curriculum because one of the four multiweek lecture units is focused on environmental toxicology. In preparation for the first week of the OI-RE, before coming to class students read a short introduction to toxicological testing and the Daphnia life cycle and one of four primary literature articles in which Daphnia were used in a toxicological bioassay. During the first week of the OI-RE, students participated in a primary literature jigsaw activity [similar to that described in (26)], observed Daphnia characteristics (e.g., heart rate, number of eggs), and worked in pairs to design their experiment. The following week, students set up their Daphnia experiment. The third week, students observed their Daphnia, recorded results and, as a homework assignment, wrote methods and results sections, including tables and figures, in the style of a peer-reviewed scientific paper.

This research experience was open inquiry in that students selected projects strictly based on personal preferences of each pair. For example, some students chose to test the effect of the agricultural herbicide atrazine, which acts as an endocrine disruptor, on Daphnia reproduction, while others investigated the effects of chemicals that may be found in urban aquatic ecosystems, such as sodium chloride from road salt runoff, on Daphnia mortality, reproduction, and other student-selected endpoints.

BR-RE group

Students who participated in a broadly relevant research experience worked on a cutting-edge research topic and contributed their findings to a University of Minnesota professor’s ongoing research (27–30). Students were assigned a novel research question, generated original results, and communicated their results outside of the classroom. Specifically, students pursued a 3-week research project investigating the optimization of the native weed pennycress for use as a cover and cash crop for local farmers (31, 32; see https://www.forevergreen.umn.edu/).

Prior to the first week of the BR-RE, students were asked to go to the Pennycress Project website, where they watched an introductory video explaining the project, its relevance to various communities, and how they would be contributing to the ongoing research. They were also instructed to explore the two research questions (described below) that they would be investigating during the project. This information was relayed via video and text on the project website. After exploring the site and learning more about the project, students were given a short assignment in which they created videos describing how the Pennycress Project, and their involvement in it, was meaningful and relevant to them. Students were instructed to watch at least four of their classmates’ videos and respond via video with their thoughts. This assignment used an interactive video application called Flipgrid (https://info.flipgrid.com/).

Once during the 3-week project, David Marks, the sponsoring researcher, visited each laboratory section to give a brief presentation and to answer any questions regarding his research program. He emphasized the importance of student contributions to his work and encouraged the class to ask questions.

Students investigated the first research question, where the goal was to select for pennycress plants with longer hypocotyls (the part of the stem of a germinating seedling below the seed leaves and above the root) that may grow better under field conditions. In groups of four, students selected for this desirable phenotype by planting seeds derived from mutagenized plants into trays containing potting mix. Two weeks later, students scanned the trays for mutants with longer hypocotyls (27). Mutants were found that had a significantly longer hypocotyl length as determined by the students when compared with wild-type pennycress plants. These new mutants were incorporated in the pennycress research program.

In the second and third weeks of the BR-RE, students investigated the second research question, for which the goal was to select for pennycress seeds that had a low level of the bitter-tasting compound glucosinolate. Glucosinolates are composed of a glucose molecule attached to a sulfur- and nitrogen-containing side chain. Using seeds harvested from mutant plants, students determined the level of released glucose as proxy for glucosinolate in biochemical assays and compared these levels with those in wild-type seeds. Any seeds with low levels of glucose, i.e., glucosinolate, were noted by the students and the information was given to the Marks lab for further testing.

The broad relevance of this research experience was conveyed to the students through the following activities: (i) during one of the laboratory meetings, the PI from the research laboratory pursuing this work visited the students and gave a short presentation about the current state of the pennycress research project; (ii) the Flipgrid assignment at the beginning of the experience called for students to reflect upon and share their thoughts about how they would personally benefit from contributing to this research experience; (iii) the project videos emphasized the importance of this research from a variety of perspectives.

Data collection

Survey data were collected using the online survey tool Qualtrics. The course instructor sent an email to students with a link to the survey the week before the semester began (precourse), immediately after the research experience (midcourse), and the last week of the semester (postcourse). The surveys took less than 10 minutes to complete and collected data about students’ perceptions of research experiences, preferences for OI-RE or BR-RE, and science confidence (see Appendix 1 for survey items). The items for the survey on perception of research experiences consisted of published (33) or self-drafted (by the authors) Likert-type items that asked about students’ interest, investment, and responsibility in their research projects, their perceptions of whether their work was “relevant beyond the classroom,” and whether there was “the potential to discover something new.” The survey item addressing preferences for OI-RE or BR-RE was self-drafted (by the authors) and asked students which of the following they would prefer: (1) I would prefer to choose my own research question for which the results are already known (I would not contribute new information with broad relevance to the scientific community) or (2) I would prefer to be assigned a research question for which the results are not known (I would contribute new information with broad relevance to the scientific community), followed by an open-ended survey item asking them to explain their preference. The science confidence items were based on similar published items (6, 34) and have been published previously (1, 2). Students were offered an incentive of one point of extra credit each (equal to ~0.15% of their total course grade) for completing the precourse and the postcourse surveys. The midcourse survey was done during laboratory time.

Data inclusion and exclusion criteria

Students’ demographic data were matched to their survey responses, and data were de-identified. Students who withdrew or were repeating the course were removed and were not included in the analysis. Of the 275 enrolled students, 98 (36%) completed full surveys for all three time points. Analysis was performed on these 98 complete cases only. The demographic composition (e.g., gender, ethnicity, first-generation student status, international student status) and ACT scores of the complete cases (students who completed all three surveys, n = 98) and the full survey data set (students who completed at least part of at least one survey, n = 270) were not significantly different as determined by chi-square tests of homogeneity (demographics) or a two-sample t-test (ACT scores).

Quantitative data analysis

All data analysis was performed using R version 3.6.0 (35). Statistical significance was defined as p < 0.05. Chi-square tests of homogeneity were calculated to compare the proportions of each demographic characteristic between the OI-RE and BR-RE groups. A two-sample t-test was calculated to compare ACT scores between the OI-RE and BR-RE groups. The data for students’ perceptions of research experiences are ordinal, and a Mann-Whitney test was calculated to compare individual survey item responses between the OI-RE and BR-RE groups. A chi-square test of independence and standardized residuals (z) were calculated to compare research experience preference between the OI-RE and BR-RE groups.

The science confidence construct consists of 11 Likert-type science confidence survey items (e.g., “design a well-controlled experiment to test a hypothesis”). An average science confidence score was calculated for each student by taking the mean of their responses to the 11 science confidence survey items, where “not confident” = 1 and “very confident” = 4. Exploratory factor analyses were conducted previously to establish validity for this construct and this population—specifically, non-majors at the University of Minnesota (1). Cronbach’s alpha was calculated to ensure high internal consistency (reliability) for each group (OI-RE and BR-RE) at each time point (pre-, mid-, and postcourse). The Cronbach’s alphas ranged between 0.87 and 0.94 for each group at each time point (Appendix 2), indicating good to excellent internal consistency (36).

To account for the nonindependence of students in a repeated-measures and nested study design (37), multilevel modeling was used so that fixed and random effects could be included in the model. Model selection was performed as described in Theobald (37). In all of the models the response variable was students’ average science confidence. Using the lme4 package (38), the random effects structure was determined first. The random effects tested were student, section, and semester. The random effects structure was determined using model selection based on Akaike information criterion (AIC) values, with the lowest AIC value indicating the best-fitting model. When models had ΔAIC < 2, the model with the fewest parameters was considered the best-fitting model. When all combinations of random effects were tested, the best-fitting model was the one that included student as a random effect (random intercept) but did not include section or semester as random effects. Using the model with the most appropriate random effects structure, the fixed effects that had a significant impact on the response variable were determined using automated model selection with the MuMIn package (39). The fixed effects tested were treatment (i.e., open inquiry vs. broadly relevant), time (i.e., pre-, mid-, and postcourse), and an interaction between treatment and time. The fixed effects structure was determined using model selection based on AIC_c values, ΔAIC_c values, and Akaike weights (w_i). AIC_c is AIC corrected for small sample size. ΔAIC_c is the difference in AIC_c values between each model and the best-fitting model; models with ΔAIC_c ≤ 2 have substantial support, between 4 and 7 have considerably less support, and > 10 have no support (40). Akaike weights (w_i) are conditional probabilities for each model (41). Model-averaged regression coefficients, standard errors, and p values were also calculated using all models with ΔAIC_c < 4. The initial full model was:

$$\text{Average\hspace{0.17em}science\hspace{0.17em}confidence}=\text{Treatment}+\text{Time}+\text{Treatment}*\text{Time}+(\text{I}\mid \text{Student})$$

Qualitative data analysis

Students’ responses to the open-ended survey item asking them to explain their choice for their research experience preference were categorized through multiple rounds of coding. Student responses from one semester were used for the first rounds of coding, where initial coding was used to capture all possible reasons for students’ research experience preferences (42). Two coders (S.H., J.E.B., or an undergraduate student) coded the responses independently, discussed codes, and grouped codes into categories with defined inclusion criteria (Appendix 3). The resulting categories were used to recode all previously coded responses and code the remaining responses. After coding all responses independently, the coders discussed any coding disagreements and came to a consensus. Initial coder percent agreement was 97%.

Qualitative data are reported as frequencies and percentages of each coding category. Coding category percentages were calculated as a percentage of responses that fell into that category, with each response having the potential to fall into more than one category.

RESULTS

Student demographics

Students participated in either an open-inquiry research experience (OI-RE, n = 55) or a broadly relevant research experience (BR-RE, n = 43). Student demographic characteristics are shown in Table 1. The majority of students in the OI-RE and BR-RE groups were male, white, continuing generation, and not international. Demographic characteristics were not significantly different between the OI-RE and BR-RE groups. Mean (± standard deviation) ACT scores were 29.3 (± 4.4) for the OI-RE group and 28.6 (± 4.3) for the BR-RE group and were not significantly different.

TABLE 1.

Student demographic characteristics.

	OI-RE [% (n)]	BR-RE [% (n)]
Gender
Female	40.7 (22)	39.5 (17)
Male	59.3 (32)	60.5 (26)
URM
Non-URM	88.9 (48)	93.0 (40)
URM	11.1 (6)	7.0 (3)
Student of color
Student of color	38.9 (21)	32.6 (14)
White	61.1 (33)	67.4 (29)
First generation
No	81.5 (44)	79.1 (34)
Yes	18.5 (10)	20.9 (9)
International
No	88.9 (48)	88.4 (38)
Yes	11.1 (6)	11.6 (5)

URM, underrepresented minority. Demographic data were not available for one student in the OI-RE group.

Students’ perceptions of their research experience

After participating in the OI-RE or BR-RE, students responded to five Likert-type survey items that asked about their perceptions of their research experiences (Appendix 1.1). When asked whether their work on their project was relevant beyond the classroom, students in the BR-RE group reported significantly higher levels of agreement than students in the OI-RE group (Fig. 1A, p < 0.01). Students in the BR-RE group also reported significantly higher levels of agreement than students in the OI-RE group when asked whether there was the potential for discovery with their project (Fig. 1B, p < 0.001). For both of these survey items, the median response for students in the BR-RE group was “Agree,” while the median response for students in the OI-RE group was “Neither agree nor disagree.” In contrast, students in both groups reported similar levels of agreement with statements that their research project was interesting (Fig. 1C) and that they were responsible for the outcomes of their research (Fig. 1D); the median response for both groups and both items was “Agree.” Students in both groups also reported similar levels of investment in their projects (Fig. 1E); the median response for both groups was “Somewhat invested.”

FIGURE 1.

(A to D) Box plots of project ownership survey items showing students’ level of agreement (1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, 5 = strongly agree). (E) Box plots of project ownership survey items showing students’ level of investment (1 = not at all invested, 2 = minimally invested, 3 = somewhat invested, 4 = very invested, 5 = extremely invested). **, p < 0.01; ***, p < 0.001.

Research experience preferences

Following their participation in either the OI-RE or BR-RE, students were asked whether they would prefer an open-inquiry research experience or a broadly relevant research experience and to explain their choice (Appendix 1.2). For students who participated in the OI-RE, 41.8% indicated they would prefer an open-inquiry research experience and 58.2% that they would prefer a broadly relevant research experience. However, for students who participated in the BR-RE, only 2.3% said they would prefer an open-inquiry research experience and 97.7% indicated they would prefer a broadly relevant research experience (Fig. 2). There was a significant association between research experience and research experience preference (χ² (1) = 20.35, p < 0.001). For students who participated in the OI-RE, significantly more students than expected preferred an open-inquiry research experience (z = 2.60) and as many students as expected preferred a broadly relevant research experience (z = −1.48). For students who participated in the BR-RE, significantly fewer students than expected preferred an open-inquiry research experience (z = −2.94) and as many students as expected preferred a broadly relevant research experience (z = 1.67).

FIGURE 2.

Percentages of students who prefer an open inquiry or broadly relevant research experience. Students who participated in an OI-RE or BR-RE are represented by blue and green bars, respectively.

A few themes emerged from students’ explanations for their preferred research experience. Of the 24 students who preferred an open-inquiry research experience, 22 (91.6%) provided an explanation. The two most common themes for students preferring an open-inquiry research experience were positive affect (e.g., interesting, fun, etc.) and confirmation of known results (Table 2). Within the positive affect theme, students’ responses focused on the autonomy inherent in an open-inquiry research experience. They responded that it is more interesting to ask their own research question or develop their own hypothesis or methodology (e.g., “I think it’s more interesting to research something you decide on.”). Within the confirmation of known results theme, students’ responses focused on wanting to know whether they did the experiment correctly (e.g., “I like to know if I did the experiment correctly or not”). Of the 74 students who preferred a broadly relevant research experience, 65 (87.8%) provided an explanation. The two most common themes for students preferring a broadly relevant research experience were positive affect (e.g., interesting, rewarding, enjoyable, exciting, etc.) and broad relevance (Table 3). Within the positive affect theme, many students’ responses focused on their interest in or enjoyment of the discovery aspect of broadly relevant research and the meaningfulness of the work (e.g., “I feel like contributing to new information would be more interesting and meaningful to the experiment”). Within the broad relevance theme, students’ responses focused on contributing to the scientific community, a sentiment which was often intertwined with the discovery aspect of broadly relevant research (e.g., “I liked knowing that I could contribute something new to the scientific community, no matter how small that contribution may be”).

TABLE 2.

Qualitative coding percentages and examples for students who preferred open-inquiry research experiences.

Coding Category	n (%)	Example Responses
Positive effect (e.g., interesting, fun)	10 (45.5)	“I think it’s more interesting to research something you decide on.” “It’s more fun to choose our own hypothesis.”
Confirmation of known results	10 (45.5)	“I like to know if I did the experiment correctly or not.” “I think it is always more productive to use known information as a goal to shoot for when experimenting. This way the end product is known and shows a well-conducted lab.”
Process of science	5 (22.7)	“It’s nice having that exposure of developing your own methodology and reasoning for that gap in knowledge.”
Straightforward	3 (13.6)	“It’s clear; keeps everyone on the same page.”
Doubts relevance of results	2 (9.1)	“I doubt any of the research done in lab will amount to any serious work.”

TABLE 3.

Qualitative coding percentages and examples for students who preferred broadly relevant research experiences.

Coding Category	n (%)	Example Responses
Positive effect (e.g., interesting, rewarding, enjoyable, exciting)	31 (47.7)	“I feel like contributing to new information would be more interesting and meaningful to the experiment.” “It is more interesting to see what is happening with actual research and get to contribute to that.”
Broad relevance	29 (44.6)	“I liked knowing that I could contribute something new to the scientific community, no matter how small that contribution may be.” “It gives a sense of fulfillment and achieving something and contributing to something that impacts the real world and is tangible.”
Discovery	17 (26.2)	“It motivates me to be able to research and explore new frontiers of science.” “I thought it was really cool that I could potentially find something that no one else had before.”
Straightforward	16 (24.6)	“It would be a clearer goal in mind than it is when we are tasked to create our own.”
Lack of science confidence	7 (10.8)	“I don’t think we have enough knowledge of the field to ask meaningful scientific questions.”
Realistic	4 (6.2)	“The option I selected seems more like real life than just a classroom activity.”
Process of science	3 (4.6)	“I think it is a lot easier to show the scientific method when there isn’t a “right” answer going into the experiment. It’s really easy to accidentally bias your results, especially when you know what you should get. It’s also frustrating when it doesn’t work out the way you want.”

Science confidence

The trend for students’ average science confidence levels was similar for both the open inquiry and broad relevance groups. Students’ average science confidence increased over time (Table 4), with midcourse confidence levels (immediately following their research experience) higher than precourse levels, and postcourse levels higher than midcourse levels.

TABLE 4.

Students’ average science confidence pre-, mid-, and postcourse.

	OI-RE (mean ± SD)	BR-RE (mean ± SD)
Precourse	2.93 ± 0.49	2.88 ± 0.59
Midcourse	3.12 ± 0.46	3.01 ± 0.65
Postcourse	3.18 ± 0.52	3.20 ± 0.52

The most plausible model for students’ science confidence included time as the only fixed effect with student as a random effect but did not include treatment or the interaction between treatment and time as fixed effects (Table 5). The best-fitting model was 2.5 times more likely than the next best-fitting model. All three reasonably supported models (ΔAIC_c < 10) contained time as a parameter, indicating a consistent impact of time on students’ science confidence. Time significantly and positively impacted students’ science confidence, but treatment did not have a significant impact (Table 6).

TABLE 5.

Candidate models and parameters

Candidate Model	K	LogL	AICc	ΔAICc	wi
Time	5	−185.8	381.7	0.00	0.657
Treatment + Time	6	−185.6	383.5	1.83	0.263
Treatment + Time + (Treatment * Time)	8	−184.7	385.9	4.19	0.081
Treatment	4	−200.5	409.2	27.51	0.000

Candidate models (i) for predicting average science confidence and calculated number of parameters (K), log-likelihood (LogL), AIC with small sample size adjustment (AIC_c), ΔAIC_c, and Akaike weights (w_i). All models include student as a random effect.

TABLE 6.

Model-averaged coefficients, standard errors, and p values.

Parameter	Estimate	SE	p
Intercept	2.913	0.059	<0.001
Time (relative to precourse)
Midcourse	0.165	0.050	<0.001
Postcourse	0.281	0.050	<0.001
Treatment (relative to OI-RE)
BR-RE	−0.013	0.053	0.806

Model includes student as a random effect.

DISCUSSION

While research experiences are generally considered to be beneficial for all students, traditional, apprentice-style research experiences are not typically available to everyone and tend to enlist the most confident and well-connected students. Thus, course-based research experiences [e.g., course-based undergraduate research experiences (CUREs) and SREs] are seen as a remedy to the problem of access: by integrating research into a formal curriculum, all students can develop science process skills in the context of meaningful inquiry.

Emerging literature suggests certain criteria for an experience to qualify as a course-based research experience (14, 43). However, science educators must still evaluate these recommendations in context to understand which elements of a research experience are necessary for desired outcomes. For example, Corwin et al. (44) demonstrate positive benefits of discovery, iteration, and collaboration in the context of course-based research. Adedokun et al. (45) found the benefits of undergraduate research increase with time spent in a research program. And initial findings of Ballen et al. (1) suggested that broadly relevant work may not matter much to a population of non-majors biology students.

At our institution, we were especially intrigued by the broad relevance aspect of CUREs and SREs, largely because it is the most difficult to incorporate at scale. We are also focused on a non-majors student population with many of our courses and understand that desired course outcomes—as well as the mechanisms for achieving them—may differ between non-majors and majors (2, 24). Further, given the criticisms of Corwin et al. (46) of our earlier work (1), we sought to improve on our experimental design while also focusing on short-term research experiences, rather than CUREs overall.

In our quasi-experimental manipulation of student exposure to course-based research experiences, we compared student outcomes (preferences and confidence) between an OI-RE and a BR-RE. Using survey response data from three semesters in one introductory biology course, we found evidence to suggest that (i) student preferences are, to some extent, influenced by exposure (i.e., students in the BR-RE expressed a strong preference for broadly relevant research participation), and (ii) student confidence improves during both open inquiry and broadly relevant research, and there is not a significant difference in gains between the two. Furthermore, in their open-ended comments about their research preferences, many students who preferred open inquiry cited either the autonomy inherent in the OI-RE experience or the enjoyment of designing their own experiments around topics that were personally interesting. Students who preferred broadly relevant work valued the potential, however small, to contribute to ongoing research.

We can interpret these student preferences under the lens of SDT (12, 13). Specifically, a student’s need for autonomy can be met by providing the sort of choices that constitute an open-inquiry experience. Research that is broadly relevant is likely to constrain student choice by necessity—in order for student data to be useful to a PI, students simply cannot deviate from an established question, experimental methodologies, or data collection procedures. Conversely, students can experience relatedness, or external validation, by collaborating with their peers and faculty on research that is meaningful (i.e., their work has the potential to contribute to ongoing research). Thus, a student’s need for relatedness may best be met in a broadly relevant research experience rather than one that involves open inquiry. Finally, students can easily experience competence, or mastery of content and skills, in either open inquiry or broadly relevant research.

In any course-based research experience, faculty seeking to maximize student motivation can leverage SDT by emphasizing autonomy, competence, and relatedness. Although open inquiry, by definition, naturally supports student autonomy via multiple opportunities to make decisions and direct their own research, there are avenues for facilitators of broadly relevant research experiences to emphasize autonomy as well. For example, students can exercise creativity by using a variety of mechanisms to communicate the results of their research—creating a presentation, a public service announcement or letter to the editor, or a video for the PI’s laboratory website. Similarly, relatedness can be achieved in open inquiry by encouraging students to work together and to focus on lines of inquiry that are personally meaningful.

In their open-ended comments, students who prefer open inquiry to broad relevance also cite an appreciation for confirmation; if the outcomes of their research are already well established, then students can use these established outcomes as a benchmark for their own work. For example, one student reported that, “I think it’s nice as someone with minimal biology experience to be able to explore a question that interests me that also has known answers that can help provide structure to my learning.” Conversely, students engaged in novel research may not feel confident enough in their own work to find meaning in their results. For example, another student responded, “I do not feel comfortable contributing new information to the scientific community because I feel inexperienced and am not sure if the data I generated in lab is correct or useful.” These sentiments suggest that, even in laboratories that emphasize novel research with unknown results, facilitators can provide initial opportunities for students to master skills and acquire results that are predictable. For example, prior to engaging in an SRE or CURE, students can first demonstrate mastery of relevant techniques using established procedures and outcomes. From that competency-supportive background, students can engage in broadly relevant research, with some confidence that their work will be reliable and useful.

Certain limitations constrain our conclusions. For example, our work not only has a single-institution focus, but also a single-course focus. Students in different courses at different institutions and institutional types may respond differently to similar variation in lab experiences. Also, we did not focus our assessment on the acquisition of specific skills; however, we assume different lab experiences will emphasize the development of different skill sets, regardless of whether the lab is open inquiry or broadly relevant. Finally, our work focused on the experiences and outcomes of students in single semesters. A longitudinal perspective [e.g., (34)] might shed further light on how different research experiences impact confidence after the conclusion of the course and, consequently, how these variable experiences could impact further STEM pursuits in different populations of students (e.g., biology majors).

When we designed this activity, we tried to optimize each learning experience for the students in each group. The two labs had distinct goals for student outcomes. For the open inquiry (Daphnia) lab, the emphasis was on the open inquiry aspect of it and the experimental design. To prepare for this, we wanted to give students an idea of what has been done with toxicology assays and expose students to the variety of ways these assays can be used. Thus, we used a jigsaw approach to expose the students to published literature that showed a variety of applications of the bioassay. The objective of this introductory activity was to pique student interest in designing their own experiment. For the broad relevance (pennycress) activity, the broad relevance was emphasized (experimental design was predetermined). This emphasis started in the pre-lab work in which we asked students to engage in the broader relevance and the contemporary nature of the ongoing research on campus. Thus, instead of reading published papers, students looked at the current laboratory webpages and watched videos of researchers currently doing the work. This broad relevance and the students’ thoughts about participating in this active research project was what we asked students to engage in as they made the Flipgrid videos. For the post-laboratory work, both groups created a data figure to communicate their results. The open inquiry group also created a “methods” section, as their individual methods were of their own design, so needed to be communicated with their results. The broad-relevance group was working from prescribed methods, so we did not feel like this was as relevant for their experience. We considered the idea of making the labs as identical as possible (same activities for introduction and conclusion) but felt strongly that would not give students what we thought were the optimal preparation and reflection for each activity. We felt that (i) this would be sacrificing the student experience for the sake of experimental design and (ii) this would not be our best effort at an OI or BR experience, and we wanted to compare our best efforts at both of the OI and BR experiences.

Regardless of limitations, we are able to shed light on the value of broad relevance in short-term research experiences in a non-majors biology course. We interpret our findings as indicative that although broadly relevant research is valued by students, it is probably not critical for a non-majors course-based research experience. Therefore, where including broad relevance is especially challenging, due to costs, PI commitment, etc., this aspect may be waived without sacrificing student gains in competence and motivation.