Introduction to Research: A Scalable, Online Badge Implemented in Conjunction with a Classroom-Based Undergraduate Research Experience (CURE) that Promotes Students Matriculation into Mentored Undergraduate Research

Abstract

The benefits of mentored undergraduate research to student success, retention, and persistence in science, technology, engineering, and mathematics (STEM) have long been identified. However, many students miss out on the opportunity to engage in research often due to unfamiliarity of various research opportunities or how to approach potential research mentors. To address this, we developed a scalable online badge, Introduction to Research, that draws on aspects of the Entering Research curriculum (Branchaw, Pfund, & Rediske, 2010) to help students explore and prepare for undergraduate research in the biomedical and behavioral sciences. Students in the BUILD Training Program, part of the larger STEM BUILD at UMBC Initiative, completed the badge in conjunction with a 3-week classroom-based undergraduate research experience (CURE) before the start of their second year of undergraduate study at the University of Maryland, Baltimore County (UMBC). We were interested in investigating how this intervention, online badge plus CURE, correlated to students engaging in undergraduate research before the end of their second year at UMBC. We did this through student self-report, comparing students who had participated in the online badge plus CURE (BTP) to those who participated in neither (Control). Our data demonstrate that students who participated in the Introduction to Research Badge and CURE entered into mentored research at a significantly higher rate than students who were exposed to neither. Further, previously validated instruments of students ‘research self-efficacy and science identity were used to compare how the Introduction to Research Badge and CURE may impact these two psycho-social variables. Students who participated in the Introduction to Research Badge and CURE had significantly higher gains in research self-efficacy compared to the control group. However, no change was observed in science identity for either group. Collectively, our results suggest that students who engage in the Introduction to Research Badge in combination with a CURE engage in mentored research within a year of completion at higher levels than students who engage in neither.

Introduction

Engaging undergraduates in mentored research is a high-impact intervention associated with academic, career, and persistence outcomes for students interested in the science, technology, engineering, and mathematics (STEM) fields (Barlow & Villarejo, 2004; Bauer & Bennett, 2003; Hunter, Laursen, & Seymour, 2007; David Lopatto, 2007; Seymour, et al., 2004; Zydney et al., 2002). The outcomes achieved through engaging undergraduates in research correlates to increased matriculation into STEM graduate programs (Barlow & Villarejo, 2004; Bauer & Bennett, 2003; Hathaway, 2002; Nagda et al.,1998; Osborn & Karukstis, 2009; Pender, Marcotte, Domingo, & Maton, 2010; Seymour et al., 2004; Zydney et al., 2002), which is attributed to mentored research promoting students ‘research self-efficacy, confidence as a scientist, and scientific identity (Chemers et al., 2011; Hunter et al., 2007; S. H. Russell, Hancock, & McCullough, 2007). Undergraduate students who are traditionally underrepresented in STEM fields are particularly impacted by participating in mentored research (Bauer & Bennett, 2003; Boyd & Wesemann, 2009; Eagan et al., 2013; David Lopatto, 2007; Pender et al., 2010), which suggests that undergraduate research is one intervention that can help to diversify the STEM field.

Given the correlation between engaging in undergraduate research and the positive student outcomes described above, national calls have been made to engage all STEM undergraduates in research (American Association for the Advancement of Science, 2011; Boyd & Wesemann, 2009; National Research Council, 2003; President’s Council of Advisors on Science and Technology, 2012). While numerous independent research experiences (e.g., National Science Foundation’s Research Experience for Undergraduates, or NSF REU) and research-intensive courses (i.e., course-based undergraduate research experiences or CUREs) have been developed to address this national call, many students still miss out on the opportunity to engage in mentored research as an undergraduate. Capacity is certainly to blame for this, as the number of traditional one mentor-to-one mentee research experiences are not meeting the demand of STEM undergraduate enrollments, with most mentored research opportunities being secured by upper-level (third and fourth year) students with high academic performance (Guertin & Esparragoza, 2009; S. H. Russell, 2005). Equally to blame, however, is that many undergraduates are unfamiliar with the process of entering into a mentored undergraduate research experience. Reasons for this include, but are not limited to, students ‘unfamiliarity with how to identify and contact potential research mentors, unfamiliarity with different research opportunities available to them, and a lack of understanding of the culture of research environments (Balster et al., 2010; Pyles & Levy, 2009). This unfamiliarity can be particularly profound for students who are not associated with scholars programs, where undergraduate research is an expectation and a support structure is in place to help scholars navigate the undergraduate research process (e.g., Carter, Mandell, & Maton, 2009; Maton & Hrabowski, 2004). This suggests that alternative approaches are needed to help familiarize lower-level students, or first- or second-year undergraduates, with the research process early in their undergraduate experience.

Digital Badges

Digital badges are a mechanism to provide credentialing to an individual who has demonstrated acquisition of newly formed knowledge or skill sets (Gibson et al., 2015). The badge itself allows the individual to share their newly acquired knowledge or skills in a manner analogous to how badges are awarded in youth or professional initiatives, just in a digital form. While most badges are co-curricular online learning experiences, they often have the same objectives as academic credit-bearing courses: to promote the attainment of an individual’s conceptual knowledge and skills related to a particular topical area. Besides learning management systems (e.g.,Blackboard or Canvas), specific badging platforms, such as Credly, Badgestack, and Mozilla Open Badges, have been created (Gibson et al., 2015).

Within the STEM field, badging has been used as a way for students to demonstrate their proficiency with laboratory techniques. For example, within introductory chemistry, badging has been used as a means for students to demonstrate their proficiency with laboratory equipment, such as pipetting and appropriate use of glassware (Hensiek et al., 2016; Hensiek et al., 2017; Towns et al., 2015). Implementation of these badges in introductory chemistry, where students had to upload videos of themselves using laboratory equipment common to a chemistry laboratory, correlated to student self-reported significant gains in their confidence with the laboratory equipment (Hensiek et al., 2016; Towns et al., 2015). Furthermore, the badge provided students the opportunity to receive constructive feedback on their technique and the instructors reported less damage to equipment with the implementation of the badge (Hensiek et al., 2017).

Course-based Undergraduate Research Experiences (CUREs)

Course-based undergraduate research experiences (CUREs) are implemented in a wide variety of formats with topics spanning across the STEM disciplines, with a common aim of providing students with an authentic research experience while in a larger group setting (Auchincloss et al., 2014; Bangera & Brownell, 2014; Rowland et al., 2012). CUREs have been established as an effective way of allowing undergraduate students to participate in research earlier in their academic careers by enabling many students to participate in a research experience simply by enrolling in a course (Auchincloss et al., 2014; Corwin et al., 2015). Students who enroll in CUREs have demonstrated gains in their perception of science as being a creative and process-based field (Auchincloss et al., 2014; Russell & Weaver, 2011). Studies have reported that students who complete CUREs show gains associated with involvement in undergraduate research internships, most notably gains in scientific self-efficacy, research and laboratory skills, academic success, and intention to persist in a STEM discipline/career (Auchincloss et al., 2014; Bangera & Brownell, 2014; Brownell et al., 2015; Caruso et al., 2016; Hanauer, et al., 2012; Harrison et al., 2011; Jordan et al., 2014; Lopatto et al., 2008; Olimpo, Fisher, & DeChenne-Peters, 2016; Rodenbusch et al., 2016; Rowland et al., 2012; Shaffer et al., 2010). When compared to traditional laboratory courses, CUREs provide more opportunities for students to repeat and troubleshoot scientific and research skills, work in a collaborative research environment, and have ownership over a project that has a “real-world” impact (Corwin et al., 2018; Corwin et al., 2015; Gin et al., 2018; Hanauer & Dolan, 2014; Thiry et al., 2012). The practice of focusing CURE content on relevant discoveries utilizing simple projects that have a high probability of student success has encouraged a sense of achievement and self-efficacy among participating students (Gin et al., 2018; Schunk & Pajares, 2009). Overall, CUREs have been established as an effective method to not only address the aforementioned capacity issue that the one-to-one research mentor-to-mentee model possesses, but also as a possible means of promoting a more inclusive and diverse scientific research community while increasing student gains (Auchincloss et al., 2014; Bangera & Brownell, 2014).

The Current Study

To help familiarize a broad demographic of lower-level undergraduates with how to search for research experiences, contact potential mentors, and explore the culture of a research laboratory, we developed the “Introduction to Research” digital badge. The badge is a zero credit, asynchronous adaptation of the Entering Research curriculum (Balster et al., 2010; Branchaw et al., 2010) that was established in the Spring of 2016 at the University of Maryland, Baltimore County (UMBC) as part of the interventions associated with the STEM BUILD at UMBC Initiative (LaCourse et al., 2017). The badge is implemented exclusively online through a learning management system (LMS), offering students the flexibility of completing the badge deliverables at their own pace. The intended audience of this badge is lower-level, or first- or second-year, undergraduates interested in pursuing research in the biomedical and behavioral sciences that have yet to apply for and/or engage in an undergraduate research experience.

The learning objectives (LOs) of the badge are as follows:

1. Identify personal motivation(s) for pursuing undergraduate research and how it will benefit long-term academic and professional goals

2. Compare and contrast different undergraduate research opportunities

3. Create a personal statement for an NSF REU opportunity

4. Draft an initial contact email to a potential research mentor on campus

5. Conduct an informational interview of a researcher

6. Evaluate different mentoring scenarios

7. Evaluate the culture of a research lab and how to resolve conflict

The intended purpose of this badge is to help lower-level students who have yet to engage in mentored research to explore and matriculate into undergraduate research in a manner that is both scalable and sustainable. The badge has the ultimate goal of providing students an opportunity to reflect on their motivations for pursuing undergraduate research and begin the process of searching and/or applying for opportunities in a structured and scaffolded manner, with feedback provided. Further, this badge was designed such that it allows students the opportunity to explore different undergraduate research experiences, identify the appropriate mentoring relationship for them, and develop and revise documents to help them matriculate into competitive undergraduate research opportunities.

While the Introduction to Research Badge draws on specific aspects of the Entering Research curriculum (Branchaw et al., 2010), such as reflecting on motivations for engaging in research, making initial contacts to research mentors, and exploring mentoring types and the culture of a research laboratory, there are clear distinctions between the two curriculums. First, students who participate in Entering Research enter research labs either before or at the start of the course, while the goal of Introduction to Research is to provide a framework for students to enter into mentored research after completion of the badge. Further, the Entering Research curriculum, which is designed to be a credit-bearing face-to-face course, focuses on advanced research skills such as documenting research results, formulating research questions and hypotheses, experimental design, drafting research proposals, and disseminating research findings. Introduction to Research, which is non-credit bearing and implemented on-line, focuses on exploring different research opportunities, making personal connections with campus researchers, and developing personal statements for competitive research opportunities on or off campus. The rationale for this is that the Introduction to Research Badge is designed to be a scalable mechanism that all lower-level students at an institution (i.e., first- or second-year undergraduates), regardless of their program affiliation, can participate in if they are interested in participating in undergraduate research. Further, the Introduction to Research Badge was made zero credit as a way to allow flexibility with completing assignments while providing a mechanism to help students build the necessary skills to find a research experience without impacting their institutional credits for degree completion.

Students who completed the Introduction to Research Badge were members of the BUILD Training Program (BTP), which is one element of the larger STEM BUILD at UMBC Initiative that was funded by the National Institutes of Health in 2014 (LaCourse et al., 2017). These students participate in a variety of interventions designed to promote their success and persistence into biomedical research careers. Students in the BTP receive financial support and supplemental academic advising, reside in a STEM Living and Learning Community on campus during their first year, and participate in annual Summer Bridge programming and program-specific academic year coursework. As part of this comprehensive program, students participate in the Introduction to Research Badge and a three-week CURE.

Three cohorts of BTP students are described in this study, where students in two of the three cohorts completed the Introduction to Research Badge in the summer between their first and second year, with concurrent enrollment in a three-week summer CURE. The remaining cohort completed the Introduction to Research Badge during the academic year of their first year, with enrollment in a CURE the subsequent summer (Figure 1). The purpose of implementing the badge in conjunction with a CURE is that the badge was designed to help students explore and prepare for matriculation into undergraduate research (Balster et al., 2010; Branchaw et al., 2010), while the CURE provides students with technical laboratory, data analysis, critical thinking, and quantitative and scientific reasoning competencies (Auchincloss et al., 2014; Bangera & Brownell, 2014; Bell et al., 2017; Harrison et al., 2011; Jordan et al., 2014; Shaffer et al., 2010) that may help them succeed in subsequent mentored undergraduate research environments. We hypothesized that these two activities, the digital badge and CURE, in combination will promote student matriculation into undergraduate research experiences and gains in their science identity and research self-efficacy. We explored this hypothesis as part of an overall evaluation of the STEM BUILD at UMBC Initiative (LaCourse et al., 2017) and examined the student self-report survey responses of the treatment group (those who completed the badge and CURE) and a control group (those who completed neither) to investigate the following questions:

1. Are there differences in research participation among students that participated in the Introduction to Research Badge and CURE (BTP) versus not (Control)?

2. Are there differences in research self-efficacy and science identity between the two groups?

Figure 1.

Timeline for BTP participation in the Introduction to Research Badge and CURE

Intervention

The Introduction to Research Badge was built in the Blackboard LMS and included five units, with each unit having 1–3 modules with specific learning objectives (LOs) and activities that were worth a set number of points (Table 1). Students in BTP Cohorts 1 and 2 were required to complete all units and modules to be awarded the badge. Students in BTP Cohort 3 had to complete three required assignments and earn 80% of the possible points for the remaining assignments to be awarded the badge. Based on formative feedback, we made the switch with Cohort 3 to allow them flexibility in prioritizing assignments that they felt were of most value to them. Further, by not requiring all of the assignments for Cohort 3, we were able to decrease the burden of the badge given other constraints on their time. A PhD-level molecular biologist with experience mentoring undergraduates in research served as the lead instructor for the Introduction to Research Badge. This individual developed the curriculum and worked with other members of the BTP staff, which included 2–3 upper-level undergraduate peer mentors, to provide students feedback on the badge deliverables. The undergraduate peer mentors provided feedback to the students enrolled in the Introduction to Research Badge, with peer mentors receiving guidance and oversight from the lead instructor on how to provide positive, constructive feedback to students enrolled in the badge.

Table 1.

Summary of the Introduction to Research Badge Curriculum

UNIT	MODULE	LEARNING OBJECTIVE	ACTIVITIES	POINTS
1	Undergraduate research readiness, expectations, and contributions	LO 1: Identify personal motivation(s) for pursuing undergraduate research and how it will benefit long-term academic and professional goals	Read short articles and respond to reflective writing prompts (Appendix I)	50
2	Explore different undergraduate research experiences	LO 2: Compare and contrast different undergraduate research opportunities	Conduct online search of different undergraduate research experiences and complete a worksheet (Appendix II).	120
	Personal statement for a research opportunity	LO3: Create a personal statement for an undergraduate research opportunity	Compare 4 different NSF REU programs and identify interest in the programs (Appendix III)	REQ
			Prepare a personal statement for a research opportunity (Appendix IV)	REQ
3	Email critique	LO4: Draft an initial contact email to a potential research mentor on campus	Review and critique emails that students sent to potential research mentors (Appendix V).	100
	Draft email to research mentor		Search for research mentors and draft an initial contact email (Appendix VI).	100
	Informational interviews	LO5: Conduct an informational interview of a researcher	Watch short video and critique two students’ approaches to conducting an informational interview (Appendix VII).	100
			Identify a researcher to interview, develop questions for interview, and schedule and execute interview. Feedback provided to students from the researcher (Appendix VIII).	REQ
4	Mentoring approaches	LO6: Evaluate different mentoring scenarios	Read article and watch video on mentoring relationships. Review mentoring approaches and identify advantages and disadvantages (Appendix IX)	100
	Culture of research lab	LO7: Evaluate the culture of a research lab and how to resolve conflict	Strategize how to approach a situation where there is a conflict between a student and mentor (Appendix IX).	100

For unit 1, which was adapted from the Entering Research curriculum (Branchaw et al., 2010), students completed reflective writing prompts after reading articles on the benefits of engaging in research as an undergraduate (Lizarraga, 2011; S. A. Webb, 2007), as well as different types of undergraduate research experiences and the process by which students enter into those experiences (Institute for Broadening Participation, n.d.; Slaughter, 2006b). The reflective prompts asked students about how engaging in research will help them achieve their educational and professional goals and included students expressing their excitement and concerns (Appendix I). Students also responded to questions that had them formulate expectations for the research experience and identify the contributions that undergraduates can make to a research team.

For unit 2, students compared various undergraduate research opportunities. These included a National Science Foundation Research Experience for Undergraduates (NSF REU), an internship at the NIH, research with a faculty member on campus, and an undergraduate research scholars program, such as Maximizing Access to Research Careers Student Training in Academic Research (MARC U-STAR) Program (Hall, Miklos, Oh, & Gaillard, 2016), of which UMBC is one of the MARC U-STAR institutions. Students also explored the competitive Undergraduate Research Award available at UMBC and completed a worksheet that required them to find information on the various research programs through online searches (Appendix II).

In addition to comparing various undergraduate research experiences, students explored four NSF REU sites and outlined why they were interested in the program, specific skills they hope to develop through their participation in the REU, and specific mentors and/or projects they wish to work on (Appendix III). They then took this outline and prepared a one-page personal statement for their preferred NSF REU opportunity, addressing three prompts (Appendix IV). Students received feedback on their personal statements from full-time BTP and non-BTP staff members in a blinded review structure (undergraduate peer mentors did not provide feedback on personal statements).

Unit 3 focused on searching for research mentors and making initial contacts via email. Students first reviewed content that was adapted from Branchaw et al. (2010) and watched a short video titled “Identifying an Undergraduate Research Mentor” (Center for Engaged Learning, 2014). Students then critiqued sample emails that students sent to research mentors about engaging in research in their lab (Appendix V), before searching departmental websites for potential research faculty who they were interested in working with and drafting an initial contact email to them. Students received feedback on the emails to potential research mentors using a rubric (Appendix VI).

Students also had to critique two students ‘approaches (Appendix VII) to an informational interview in unit 3 after watching a short video of them (Syracuse University College of Law, 2012). Afterwards, students had to schedule and conduct an informational interview with a faculty member, postdoc, or graduate student conducting research on campus (Appendix VIII). To prepare for the informational interview, students had to submit three questions that they intended to ask during the interview. Program staff provided formative feedback on the questions before their interview. While students were expected to schedule their own interview, BTP staff introduced students who were unable to schedule an interview on their own to researchers via email. Students had to send a link to a feedback form after the interview, requesting that the interviewee provide feedback to the student on their informational interview.

During unit 4 (Appendix IX), students explored different mentoring approaches and identified how to resolve conflict in a mentoring relationship. Students started this unit by reading an article on how to navigate mentoring relationships (Slaughter, 2006a) and watching a video entitled “Finding Your Research Home” (NIH Office of Intramural Training and Education, 2014). Students reviewed three mentoring scenarios that are published in Branchaw et al., (2010, pp. 65–66) and provided advantages and disadvantages of each mentoring type. Students also reviewed four scenarios where conflict may arise between a student and mentor and strategized how to handle each situation (Branchaw et al., 2010, pp. 88–90).

Description of BTP CUREs

Students in cohort 1 participated in a three-week adaptation of the first semester of the SEA PHAGES course (Hanauer et al., 2017; Hatfull et al., 2006; Jordan et al., 2014). Students did not receive academic credit for this CURE, but their participation was required as part of their participation in the BUILD Training Program. During the course, students isolated and characterized novel bacteriophage from soil Bacillus spp. using published SEA PHAGES protocols (seaphages.org) adapted for Bacillus, spp. (Sauder et al., 2016). Specifically, they isolated and purified Bacillus cereus group phage, stained the phage for transmission electron microscope (TEM) visualization, isolated and purified phage DNA, performed cluster analysis and host range testing, and evaluated the genomic quality of their phage DNA. At the conclusion of the 3-week course, students wrote an abstract and presented posters of their findings at an undergraduate research symposium on campus.

Students in cohorts 2 and 3 participated in a three-week Bioanalytical Instrumentation CURE. As with cohort 1, students did not receive academic credit for this CURE, but participation was a BTP requirement. Throughout the course, students worked towards the goal of identifying and characterizing antibiotic resistance genes from freshwater sources using a variety of molecular biology and analytical chemistry techniques. Students first isolated DNA from a freshwater source and used PCR and gel electrophoresis to identify antibiotic resistance genes. Students then used Gibson Assembly to clone their identified antibiotic resistance gene and attempted to express the associated protein in E. coli. Their purified protein (or a similar protein standard) was analyzed and identified via mass spectrometry. Additionally, cohort 3 students attempted to detect antibiotic compounds in their freshwater source using mass spectrometry to try to determine a correlation between the presence of antibiotic compounds and resistance genes in the water sources. As with cohort 1, students wrote an abstract and presented a poster of their findings at a campus research symposium.

Methods

Study Population

Students who completed the Introduction to Research Badge were members of The BUILD Training Program (BTP), which is part of the larger STEM BUILD at UMBC Initiative (LaCourse et al., 2017). BTP is a randomized control trial study that is investigating interventions that promote the success and retention of students in the biomedical and behavioral sciences. Each year, students who have not yet matriculated to UMBC are invited to apply to BTP based on admissions criteria (high school GPA 3.0 and math SAT 550), no other significant scholarship support (<$10,000), and declaration of one of the following majors: biology, biochemistry and molecular biology, bioinformatics, chemistry, chemical engineering, mathematics, mechanical engineering, psychology (B.S.), or statistics.

First-year BTP applicants are reviewed by program staff for meeting the application criteria and are chosen for eligibility based on responses to four short essay questions. A final approved applicant list is sent to the evaluation team who then conducts the randomized selection process and assigns applicants to one of three groups: BTP, STEM Living and Learning Community (LLC), and Comparison (COMP). Students in BTP are exposed to numerous curricular and co-curricular interventions, including supplemental advising and required residence in the STEM LLC their first year, and are also provided financial support in the form of partial tuition and a monthly stipend. Students in the STEM LLC group reside in the residential community and have access to supplemental tutoring from upper-level undergraduates and the opportunity to participate in enrichment activities. Finally, students in the comparison group receive standard support provided to all STEM students at UMBC. Prior to randomization, the applicant pool is stratified into categories based on gender and race/ethnicity in order to ensure that the resulting study populations are similar. Due to small sample sizes the LLC and COMP groups have been combined to create a control group for the BTP. Sample sizes and demographics of those students included in our analyses can be found in Table 2. There are no statistically significant differences between the groups with respect to gender or underrepresented minority status (URM). The URM classification includes students who identify as African-American, Latinx, and/or Native American.

Table 2:

Sample Size and Demographics of Randomly Assigned Treatment Groups

Group	Sample Size	Gender		URM status
		Women	Men	URM	non-URM
BTP - Badge and CURE	55	36 (65%)	19 (35%)	27 (49%)	28 (51%)
Control (LLC/COMP) - No Badge nor CURE	57	31 (54%)	26 (46%)	18 (32%)	39 (68%)

Three cohorts of students in BTP completed the Introduction to Research Badge (Figure 1). Cohort 1 completed the badge during the Spring of 2016 and subsequently completed a 3-week adaptation of the Phage Hunters CURE (David I. Hanauer et al., 2017; Hatfull et al., 2006; Jordan et al., 2014) during the following summer (2016). Cohort 2 completed the badge during the Summer of 2017, with concurrent participation in a three-week Bioanalytical Instrumentation CURE. Cohort 3 completed the badge during the Summer of 2018, with concurrent participation in the three-week Bioanalytical Instrumentation CURE. The badge was self-paced, with students allowed to complete the various units and/or modules at their own pace within a specified unit of time. Students in the LLC and COMP groups (referred to as the Control group) did not participate in neither the Introduction to Research Badge nor the CURE.

Procedure

The data for this study were collected via electronic surveys that were administered through Qualtrics, an on-line survey platform. At the time of the survey, students gave their consent to participate in the research and all survey questions and consent documents were reviewed and approved by the Institutional Review Board (IRB) at UMBC (protocol number Y15PR20053). Students in the BTP completed the baseline survey in the summer before their first year at UMBC, while the control (LLC/COMP) completed the baseline survey at the start of their first academic year. Both groups completed the follow up surveys each subsequent spring semester they were enrolled at UMBC. The surveys included formative questions about different programmatic components, students’ experiences on campus, and evaluation hallmarks measured by reliable and valid scales for research self-efficacy (Chemers et al., 2011) and science identity (Chemers et al., 2011; Estrada et al., 2011). All surveys took approximately 15–20 minutes to complete. Students in the BTP received no compensation for taking surveys, students in the STEM LLC (LLC) received $25, and students in the comparison (COMP) sample received $50 for their participation in each wave of the study. The survey incentive was added to their campus card and could be used like cash anywhere on the UMBC campus. The surveys used to evaluate the research questions of this study were baseline and those collected the spring of their second year at UMBC (end of year 2 or EOY2).

Student-Level Measures

Research participation

Participation in research was assessed using a five-item measure that asked students about which types of research experiences they had within the past academic year, including the previous summer. The question asked whether they had participated in hands-on research in a classroom setting, in a laboratory on their campus, at a different university, at a non-academic location, or had designed an independent research project. There was also an option to indicate no participation in research (Appendix X). The measure originated from the Diversity Program Consortium’s (DPC’s) Coordination and Evaluation Center (CEC) at the University of California, Los Angeles, which is responsible for the national evaluation of all BUILD programs. This measure was used to have comparable data to all BUILD sites and to assess the DPC’s Hallmark of “participation in undergraduate/ summer biomedical research training in labs or similar research environment” (McCreath et al., 2017). These data were recoded into a series of dichotomous variables. Any research participation was coded such that students with any research experience outside of the classroom was coded as 1, and no research or classroom research only was coded as 0. Furthermore, each item was individually recoded such that if students responded “yes” to that specific research experience, they were coded as 1 and if not, they were coded as 0.

Research self-efficacy

The research self-efficacy scale consists of 13 items, adapted from the original 14-item scale (Chemers et al., 2011; Estrada et al., 2011; Syed et al., 2018). Students were asked to rate their confidence in their ability to complete research tasks such as “generate a research question to answer,” “design a strategy to collect data for a study,” analyze data collected during an experiment,” and “develop theories (integrate and coordinate results from multiple studies).” Students respond using a scale of 1–5 where 1 is “not at all confident” and 5 is “absolutely confident” (Appendix XI). Responses to each item were averaged with higher scores signifying greater confidence. The adapted 13-item scale was reliable (α = .96). The DPC Hallmark “High Academic and Science Self-Efficacy” was assessed through this measure (McCreath et al., 2017).

Science Identity

The four-item science identity scale was adapted from the Chemers et al. (2011) original six item scale. Students indicated to what extent they perceive themselves to be a scientist on a 5-point scale where 1 indicates “strongly disagree” and 5 indicates “strongly agree” (Appendix XII). Items were averaged to create a scale score, where higher scores signified greater agreement with feeling like a scientist. The four-item scale was reliable (α = .92). This measure was used to assess the DPC’s Hallmark of “High Science/Researcher Identity” (McCreath et al., 2017).

Results

The first question was to determine whether there was any difference in research participation between students who did and did not receive the intervention (Table 3). To explore this, a series of chi-square tests of association were conducted between group (BTP versus control) and research participation outcomes, including 1) any research participation outside of class, 2) research in a UMBC lab only, and 3) conducting an independent research project, by the end of the second academic year. For each chi-squared test, all expected cell frequencies were greater than five.

Table 3:

BTP vs. Control participation in undergraduate research

Survey Item (* denotes significant difference, p≤.05)	BTP (% Yes)	Control (% Yes)
Any Research Experience*	60.0%	31.6%
Worked in laboratory at UMBC*	58.2%	22.8%
Worked in laboratory at another college/university	7.3%	10.5%
Worked in a non-academic location	5.5%	7.0%
Designed own research experience or project*	20.0%	7.0%

The initial chi-squared test examined whether there was a difference in any research participation outside of class between BTP and control groups. There was a statistically significant association between group and research participation (χ²(1) = 9.117, p=.003). The results indicated that the BTP students participated in research activities more than the students who were in the control group. During their sophomore year, 60% of the BTP students indicated that they had participated in undergraduate research while only 32% of the students from the control group indicated the same.

The next set of chi-square tests determined whether there was a difference between BTP and control students among individual research experiences. The association between group and research participation in a lab at UMBC was statistically significant (χ²(1) = 14.573, p<.001). The percentage of BTP students who reported conducting research in a UMBC research lab was more than twice that of the comparison students (58% vs 23%, respectively). Finally, the association between group and conducting an independent research project was statistically significant (χ²(1) = 4.067, p=.044). Twenty percent of BTP students report designing their own research experience or project as compared to only 7% of control students.

The second question was to evaluate whether there were differences in research self-efficacy and science identity among students who were exposed to the Introduction to Research Badge and CURE (BTP) compared to those who were not (control). We first evaluated the groups ‘change in research self-efficacy from baseline, or before matriculation to UMBC, to their Spring semester of their second year (EOY2). To assess this, two-way repeated measures ANOVAs were run to determine the change from baseline to EOY2 on research self-efficacy and science identity (Table 4).

Table 4:

BTP vs. Control research self-efficacy and science identity, baseline to end of year 2 (EOY2)

Measure	Group	Baseline; mean (standard deviation)	EOY2; mean (standard deviation)	p-value
Research Self-Efficacy	BTP (n=55)	3.52 (0.78)	3.83 (0.74)	<0.001
	Control (n=56)	3.66 (0.63)	3.34 (0.77)
Science Identity	BTP (n=55)	3.95 (0.67)	3.80 (0.96)	0.457
	Control (n=56)	4.06 (0.76)	3.79 (0.86)

Research self-efficacy was normally distributed, as assessed by Shapiro-Wilk’s test of normality on the studentized residuals (p > .05) and there were no outliers greater than ±3 standard deviations. The analysis revealed a statistically significant two-way interaction between group and time, F(1, 109) = 15.101, p < .001. To explore the significant interaction, simple main effects were run. At baseline, research self-efficacy was not statistically different among the BTP students (M = 3.52, SD = 0.78) and the control group (M = 3.66, SD = 0.63). However, at the end of their second year, research self-efficacy was higher for BTP students (M = 3.83, SD = 0.74) compared with the control group (M = 3.34, SD = 0.77), a mean difference of 0.48 (95% CI, 0.20 to 0.77).

Next, another two-way repeated measures ANOVA was run to determine changes over time on science identity. There was one outlier for baseline science identity, which had a studentized residual value of −3.11, and one outlier for end of year 2 science identity, which had a studentized residual value of −3.09. Science identity was not normally distributed at either baseline (p = .002) or at end of year 2 (p < .001). The analysis showed that there was no statistically significant two-way interaction between group and time, F(1, 109) = 0.558, p = .457. However, there was a statistically significant main effect of time, which showed significant differences in science identity between time points when BTP and control groups were collapsed, F(1, 109) = 8.105, p = .005. Descriptive data indicate that both groups reported a decrease in science identity from baseline to end of year 2.

Discussion

Herein, we describe implementation of the online Introduction to Research Badge designed to help lower-level students explore and prepare for mentored undergraduate research experiences in a scalable manner that was completed either before or during enrollment in a 3-week CURE. Students in the intervention group (BTP) participated in the Introduction to Research Badge and CURE before the Fall semester of their second year at UMBC while control students were exposed to neither the badge nor the CURE. Our data demonstrate that students who participated in the online badge and CURE before the start of their second year of undergraduate study participated in mentored undergraduate research by the end of their second year at UMBC at higher rates than students in the control group. These findings suggest that participation in the Introduction to Research Badge and CURE may promote students ‘ engagement in mentored undergraduate research. It is worth noting that neither the BTP or control group participated in research away from the UMBC camps in high levels. This may be attributed to the fact that most off-campus research programs, such as NSF REUs, target upper-level students and this intervention was designed for lower-level students.

The badge provides structure and a formative learning environment where lower-level students can explore research opportunities on campus and make direct connections with research mentors. The Introduction to Research Badge required students to identify campus research faculty with whom they were interested in working with, drafting initial contact emails to potential mentors, and conducting informational interviews with researchers on campus. These assignments, some of which were adapted from the Entering Research curriculum (Branchaw et al., 2010), were included as part of the badge to empower students to make professional connections with research mentors and helps address students ‘unfamiliarity with how to enter into mentored research as an undergraduate (Balster et al., 2010; Pyles & Levy, 2009).

The CURE provided the BTP students an opportunity to develop research-related skills, such as experimental design and formulating research questions/hypotheses, as well as development of technical laboratory skills and experience with laboratory equipment. This opportunity may have contributed to the significant gains in research self-efficacy that we observed in the BTP group. The CURE was implemented as part of our intervention as a way to expose lower-level students to the research process early on in their undergraduate study and our findings are supported by others (Auchincloss et al., 2014; Bangera & Brownell, 2014; Corwin et al., 2018; Harrison et al., 2011; Jordan et al., 2014; D. Lopatto et al., 2008; Rowland et al., 2012; Shaffer et al., 2010; Temple, Cresawn, & Monroe, 2010).

While students exposed to the Introduction to Research Badge and CURE reported significant gains in their research self-efficacy, no differences were found in either group for science identity. In fact, both the BTP and control groups reported decreases in science identity from the baseline to the end of year 2 measures. While it is not entirely clear what could be causing this decrease in science identity, we have two potential hypotheses. First, while the online badge and CURE appear to be impacting research self-efficacy, they may not be affecting students ‘scientific identity. This could be because the focus of this intervention is on exploring research opportunities and developing laboratory and research-related skills at a novice level. Another hypothesis is that the students could be experiencing the “Sophomore Slump,” where students struggle or become dissatisfied with the academic environment during their second year of the undergraduate experience (Lemons & Richmond, 1987; Pattengale & Schreiner, 2000; Webb & Cotton, 2019). Factors such as altered faculty contact hours, increased academic rigor of coursework, and lack of support structures that are often available to first year students have all been attributed to the “Sophomore Slump” effect (Gardner, 2000; Pattengale & Schreiner, 2000; Webb & Cotton, 2019). It could therefore be that the timing of the end of year 2 assessment implemented in this study was too distant from the intervention and/or the intervention was not strong enough to ameliorate the effects of the “Sophomore Slump.

Study Limitations

A major limitation of this study is that the students who participated in the Introduction to Research Badge and CURE (BTPs) are part of a larger student success initiative aimed at increasing the success and retention of undergraduate STEM majors (LaCourse et al., 2017). As part of this initiative, they receive multiple supports and interventions, such as financial support, program-specific course work, supplemental advising, and enriched summer experiences, which include the online badge and CURE described here. The analysis presented here is part of a larger evaluation investigating the impact of the BTP. Therefore, it is difficult to tease out individual components of the program and their effect on students. During BTP students ‘second year in the program, the interventions that they are exposed to include the Introduction to Research Badge, CURE, supplemental advising, a seminar-style course on responsible conduct of research, monthly community meetings with other students in the program, and continued financial support. However, students ‘second year in the program is less intense than the interventions that students are exposed to in their first year of the program, which include a Summer Bridge program, two academic year courses, and required residence in the STEM Living and Learning Community, in addition to the supplemental advising, community meetings, and financial support. The focus of students ‘first year in BTP is to establish a strong academic foundation and research readiness, rather than direct participation in research. Therefore, the Introduction to Research Badge and CURE are the major interventions that the students are exposed to during their second year in BTP with the objective that they will participate in undergraduate research soon thereafter.

Another important limitation of how we implemented the Introduction to Research Badge is that the badge was conducted either immediately before or concurrent with a CURE. While we cannot tease out whether the badge alone may impact students’ matriculation into mentored research, our data suggest that the badge plus the CURE act synergistically to promote students’ confidence in their research abilities, as demonstrated by gains in research self-efficacy, and inspire and prepare them to pursue mentored undergraduate research. Therefore, we think it may be of value to assess the impact of the Introduction to Research badge on student research outcomes independent of the CURE. We intend to do this by expanding the availability of the Introduction to Research Badge to other students across campus in the future and will assess outcomes.

Scalability of the Introduction to Research Badge

The online Introduction to Research Badge was designed with scalability in mind, as the objective was to develop a mechanism for a large population of undergraduates to explore research opportunities. This addresses one of the potential reasons why students don’t engage in research: unfamiliarity with different research experiences or how to establish a relationship with a research mentor (Balster et al., 2010). We specifically chose an online, asynchronous platform to accomplish this as it allowed large numbers of students to engage in the activities simultaneously while also allowing students flexibility in the completion of the various units. With the exception of a few assignments, we also allowed students flexibility in which badge activities they completed. This allowed them to select activities that most interested them and/or focus on areas where they had less familiarity.

Our assessment of the impact of the Introduction to Research Badge was with a student population that was required to complete the badge deliverables given participation in the BTP. While BTP students anecdotally mentioned the usefulness of the badge, in that it forced them to think about undergraduate research opportunities and make direct connections to potential research mentors, students buy-in to complete badge activities may not exist when the badge is not an institutional or course requirement. At UMBC, badging is an emerging mechanism to engage students in skill development across disciplines. It is therefore important that, when the Introduction to Research Badge is offered to the general student population, careful consideration is taken to communicate the benefits of the badge to students, to include outcomes of the badge assessment presented herein, and that student engagement in the badge is evaluated. Gaining the support of research faculty may help with this, as this could create an expectation where students have to complete the badge before entering into a faculty mentor’s lab. This would create a culture where the Introduction to Research Badge is of value to students, as it promotes students ‘exploration of and engagement in undergraduate research in a structured and formative environment that could correlate to positive academic and persistence outcomes.

One other thing to consider when scaling the Introduction to Research Badge is ensuring sufficient instructional staff to provide formative feedback to students. A single scientist and educator, who oversaw upper-level undergraduate peer mentors, provided feedback to the students who completed the badge in this study. The only exception to this was the personal statements (Unit 2), where BTP and non-BTP staff (not the peer mentors) provided feedback to students in a blinded review using a rubric. This arrangement was manageable with the cohort sizes reported in this study (approximately 20–25 students per cohort). However, for the badge to be expanded to larger student populations, consideration will need to be made to ensure that students receive formative feedback on assignments in a timely manner. This may include having an instructional team with each member of the team taking the lead on providing feedback on specific assignments. Members of the team do not have to be scientists; instead, they should have familiarity with providing students constructive feedback on professional documents and knowledge of the benefits of undergraduate research. Furthermore, upper-level students who had successfully completed the Introduction to Research Badge and engaged in mentored research could be recruited to provide feedback. This may be particularly powerful, as the upper-level students can provide very tailored responses to students enrolled in the online badge based on their own experiences searching for and navigating undergraduate research experiences (Bosselait & Maier, 2019; Weaver et al., 2009).

Adaptability of the Introduction to Research Badge Curriculum

Many CUREs are offered to undergraduates in their first and second year as a means to expose students to the scientific process early on and build critical thinking abilities. CUREs, however, typically focus on the research process itself as a means to develop technical laboratory and research-related skills, and develop conceptual knowledge related to a specific discipline. CUREs do not typically provide students an avenue to explore different research opportunities and begin their journey to mentored research. We therefore see the online badge complementing the objectives of CUREs, as it can be assigned as part of CURE requirements as a way to help students explore mentored research while engaged in authentic research in the context of the classroom.

While we hypothesize that combining a CURE with the Introduction to Research Badge is the most effective way to implement the program—as it allows for students to explore research experiences while engaged in authentic research— there are other ways in which the badge could be implemented. First is that the badge could be implemented through an institution’s office of undergraduate research. Most offices of undergraduate research provide programming to help students explore opportunities and the badge would provide a scalable mechanism to support this objective. Alternatively, students enrolled in a first-year experience program geared towards STEM fields, which are becoming common at most institutions, could complete the badge as part of the program requirements. Again, this could be implemented as a scalable approach to help a large number of students explore undergraduate research simultaneously.

In addition to the above-mentioned implementation strategies for our cohort of students interested in the biomedical and behavioral sciences, the curriculum of the Introduction to Research Badge could be adapted for other disciplines. These disciplines include the physical sciences, engineering, social sciences, economics, and the humanities. This would require that some of the units of the Introduction to Research badge, specifically units 2 and 4 that focus on exploring different opportunities and the culture of the research environment, should be adapted for these other disciplines. Further, the Introduction to Research Badge could be expanded to include more of the Entering Research curriculum, such as sessions on reading primary literature, roles of various personnel in a research laboratory, responsible conduct of research, and keeping a laboratory notebook. We purposely did not include these sessions as part of Introduction to Research Badge as they were covered in the CURE that students in our intervention participated in concurrently with the online badge and/or were discussed in other aspects of the BTP.

Conclusion

In conclusion, we described implementation of an online badge, Introduction to Research, designed to help lower-level students explore undergraduate research opportunities. The online badge was implemented in conjunction with a three-week CURE. Our findings demonstrate that students who participated in the Introduction to Research Badge and CURE matriculated into mentored research within a year of completion at higher levels than a control group. Further, students who engaged in the online badge and CURE had higher gains in research self-efficacy than students who participated in neither. Given that the badge is implemented online in a non-credit bearing format, it can be used as a scalable way for a large number of students to explore undergraduate research in a manner that is structured and provides formative feedback.