Instructional Models for Course-Based Research Experience (CRE) Teaching

Abstract

The course-based research experience (CRE) with its documented educational benefits is increasingly being implemented in science, technology, engineering, and mathematics education. This article reports on a study that was done over a period of 3 years to explicate the instructional processes involved in teaching an undergraduate CRE. One hundred and two instructors from the established and large multi-institutional SEA-PHAGES program were surveyed for their understanding of the aims and practices of CRE teaching. This was followed by large-scale feedback sessions with the cohort of instructors at the annual SEA Faculty Meeting and subsequently with a small focus group of expert CRE instructors. Using a qualitative content analysis approach, the survey data were analyzed for the aims of inquiry instruction and pedagogical practices used to achieve these goals. The results characterize CRE inquiry teaching as involving three instructional models: 1) being a scientist and generating data; 2) teaching procedural knowledge; and 3) fostering project ownership. Each of these models is explicated and visualized in terms of the specific pedagogical practices and their relationships. The models present a complex picture of the ways in which CRE instruction is conducted on a daily basis and can inform instructors and institutions new to CRE teaching.

INTRODUCTION

The movement to the course-based research experience (CRE), as an improvement to the traditional introductory laboratory course, has been ongoing for the last decade (American Association of University Women, 2010; American Association for the Advancement of Science, 2011; National Research Council [NRC], 2012; President’s Council of Advisors on Science and Technology [PCAST], 2012). Studies have shown that this educational approach, when compared with a traditional laboratory, increases inclusivity and persistence in science for a range of students with different demographic descriptors (Russell et al., 2007; Jordan et al., 2014; Hanauer et al., 2017; Hernandez et al., 2018) while reaching similar or better outcomes in terms of the development of procedural and content knowledge in science (Russell and Weaver, 2011; Jordan et al., 2014; Wolkow et al., 2014).

While there is broad agreement that CREs are a valuable alternative to the traditional laboratory, the instructional components of this form of inquiry teaching are not fully transparent and, to date, have not been adequately described. The Council on Undergraduate Research has recognized this situation in their recent MIRIC (Mentoring the Integration of Research into the Classroom) initiative designed to provide instructors with the “significant level of training” required to implement and execute a CRE (CUR, 2020). To further support the development and implementation of CREs, a more detailed description of the instructional processes involved in teaching a CRE is needed. It is this need that the current paper addresses.

The primary goal of this study is to develop a set of instructional models that can be used by science, technology, engineering, and mathematics (STEM) instructors to inform their in-lab CRE teaching. The concept of an instructional model as used here refers to organized sets of specific instructional practices used by CRE instructors to achieve particular CRE-related student educational outcomes. As such, the study presented here delineated a set of specific instructional practices used by active CRE instructors and modeled the relations between the use of these practices and specified outcomes. The resultant educational models should provide clarification of the ways in which the aims of CRE can be effectively achieved by instructors.

Traditional Laboratory

The traditional laboratory course has a long history of instruction and established methods of implementation. Its aim is to provide a structured approach to the learning of well-defined research skills and procedures. This is achieved through the provision of explicit direction for experiments that have predetermined and prescribed outcomes (Tamir, 1977; Tamir and Lunetta, 1978; Domin, 1999; Weaver et al., 2008; Auchincloss et al., 2014). The instructor presents the procedure and clarifies and exemplifies the processes involved, and the student follows both the verbal instructions and the descriptions that appear in the course manual (Tamir, 1977). Results of the experiments are already known, which means that students are not actually exposed to the mistakes and failures in science that are quite central to understanding the nature of the scientific process (Auchincloss et al., 2014). Within the context of the traditional laboratory course, it is possible for students to work with some of the components of scientific thinking, including observation, measurement, and documentation (Tamir and Lunetta, 1978). However, neither the student nor the instructor is faced with the unknown or the unpredictable aspects of science as part of the educational experience (Tamir, 1977; Tamir and Lunetta, 1978; Domin, 1999; Weaver et al., 2008; Auchincloss et al., 2014).

From the institution’s and the instructor’s perspectives, the traditional laboratory course does have some advantages. The course design is well controlled and suits itself to the institutional demands of clear objectives, planned timing, and explicit assessment. This approach allows a relatively large number of students to participate in a laboratory course with limited expenditures for time, space, equipment, and personnel (Domin, 1999). Students see advantages in this design, because they have exact guidelines on what they are supposed to be doing and the outcomes that are expected (Hanauer et al., 2018). Students consider the traditional laboratory course as helpful in developing foundational knowledge of skills and procedures and as involving low levels of pressure (Hanauer et al., 2018). However, the traditional laboratory course has been criticized because of its naïve exemplification of science, including avoidance of the failures and uncertainties of scientific work (Bencze and Hodson, 1999; Rahm et al., 2003; NRC, 2005) and because it may not promote persistence in the sciences (PCAST, 2012; Graham et al., 2013; Hanauer et al., 2017; Hernandez et al., 2018).

Course-Based Research Experience

While there are several different types of potential laboratory course designs, an important alternative to the traditional laboratory course is the CRE, which has authentic scientific research as its central component (Hanauer et al., 2006, 2012, 2016, 2017; Hanauer and Dolan, 2014; PCAST, 2012; Graham et al., 2013; Auchincloss et al., 2014; Hernandez et al., 2018). Authentic scientific research, as conceptualized within the CRE literature, refers to the requirement that the research and associated scientific output produced by the student have value and significance beyond the confines of the course itself (Hanauer et al., 2006, 2012, 2016, 2017; Hanauer and Dolan, 2014; Graham et al., 2013; Auchincloss et al., 2014; Brownell and Kloser, 2015; Rowland et al., 2016, Shortlidge et al., 2017). The data and analyses produced within the context of the CRE are used and disseminated to a wider group of scientists and researchers and are part of an ongoing scientific research agenda (Hanauer et al., 2006, 2012, 2016, 2017; Hanauer and Dolan, 2014; Graham et al., 2013; Auchincloss et al., 2014). This aspect of a CRE has direct ramifications for the way this type of educational model is implemented in laboratory courses. As with other scientific endeavors, results are not guaranteed and timing is not always predictable and may not fit into the timelines of the semester system (Auchincloss et al., 2014). Furthermore, students may face uncertainty and frustration from the presence of unexpected results and ambiguity (Auchincloss et al., 2014). The continuing stages following an unexpected result may be beyond what is on the syllabus of the course, involving new protocols and scientific processes. There is far more of the unknown within a CRE than in a traditional laboratory course (Auchincloss et al., 2014). Furthermore, because the data and analyses are going to be used beyond the classroom, there is a need for quality control over the scientific output. This adds extra levels of responsibility for the instructor and students involved. These features and the uncertainty of the of the protocols and processes make teaching a CRE different from teaching a traditional laboratory course (Auchincloss et al., 2014).

To address the need for enhanced clarity concerning CRE teaching, we worked with instructors who teach the Science Education Alliance–Phage Hunters Advancing Genomics and Evolutionary Science (SEA-PHAGES) CRE for undergraduate students. As with other CREs, SEA-PHAGES is designed to produce usable scientific outputs by teaching, directing, and mentoring students in the context of a laboratory course. The benefit of studying the SEA-PHAGES CRE is the advantages offered by its scale and administrative support, which are not found in many other CREs. First, SEA-PHAGES involves 293 instructors from 146 colleges and universities, all of whom deal with a shared set of scientific questions involving the discovery and genomic analysis of bacteriophages. Accordingly, the program provides access to a substantial number of instructors deeply involved in undergraduate education. Second, this is an established CRE with a history of continuous instruction for the last 11 years. As such, there is extensive expertise and experience in inquiry teaching across the instructors. And third, educational research within the SEA-PHAGES program has established that this program provides a quality education for a range of different student populations and is a promoter of persistence in the sciences (Hanauer et al., 2017).

While the SEA-PHAGES program is on a far larger scale than most CREs and has a supportive administrative structure not found in many other CREs, the central educational features of SEA-PHAGES are those of a CRE. The program is designed to facilitate usable scientific outputs by teaching, directing, and mentoring undergraduate students. The SEA-PHAGES program is a two-semester research experience that involves undergraduate students in the identification and characterization of novel bacteriophage viruses. The first semester involves bacteriophage isolation and DNA purification, which is followed by a second semester involving genome annotation and bioinformatic analyses. The student and faculty outcomes of this CRE involve submission of viral genome sequences to project and public databases such as PhagesDB (https://phagesdb.org) and GenBank (www.ncbi.nlm.nih.gov/genbank), making this output directly accessible to a wide range of researchers for the advancement of science and relevant for therapeutic application (Dedrick et al., 2019). Building upon the large number of SEA-PHAGES instructors with CRE teaching experience in the SEA-PHAGES program, we constructed instructional models that characterize CRE instruction.

METHODS

Overview

For the current study, the following systematic qualitative approach was employed to investigate, develop, and represent the instructional models of inquiry teaching in a CRE setting.

1. Defining Instructional Practices and Educational Aims for CRE Instruction: The aim of the first stage of this study was to define the aims and pedagogical practices of CRE instruction. A survey with open-ended questions asking about the main educational aims of instructors and how these were achieved pedagogically was sent to SEA-PHAGES instructors. Based on systematic approaches to qualitative data (Neuendorf, 2017), the verbal responses were coded for aims of CRE teaching and the instructional practices used by instructors.

2. Instructional Model Development: The aim of this stage was to model the instructional processes of CRE teaching based on systematic visualization methods (Miles et al., 2013; Neisser, 1967). Following the coding of the verbal data into instructional practices and aims in the first stage of this project, instructional practices were related to specific instructional aims. The relationships between instructional practices were visualized as a series of connecting lines within a specific model of instruction. The result was a network of the relations found between the instructional practices to achieve a particular instructional CRE aim. Both statistical relationships (frequencies of co-occurrence) and verbal explanations (explicit connections specified in the verbal data) inform the construction of the models. Through this process, three distinct models of CRE instructional aims emerged.

3. Community Validation of Instructional Models: The aim of this stage of the project was to evaluate, modify, and validate the emergent, exploratory models of CRE instruction defined in the previous stage. This aim was achieved through a process of large-scale community-member checking. In line with qualitative approaches to research, this approach involved informant feedback on proposed interpretations of qualitative data (Creswell and Miller, 2000). At the yearly Science Education Alliance (SEA) faculty meeting, each of the three models was presented in lecture format to all the instructors assembled at the faculty meeting. The instructors were then organized into small groups of four to six participants, given copies of the models, and asked to revise them according to their understandings of their own instructional practices. Revisions suggested by each group were either handwritten on the copies of the models that had been distributed or were made orally in front of all the assembled instructors at the faculty meeting. All copies of model modifications and comments from the groups were collected, and the oral session was fully recorded. Following the member checking session, the comments, revisions, and suggestions made both on the models and orally were analyzed, and revisions were made to the three models.

4. Expert CRE Instructor Validation of the Proposed Models: The aim of this stage was to validate with expert CRE instructors the revision of the instructional models made following initial large-scale member checking. A group of expert CRE instructors representing a range of institution types, genders, and ethnicities participated in a focus group. The revised models were presented, and oral responses were collected from the participants in the focus group. The session was video-recorded, and following this session, the models were revised and finalized.

Participants

Participation and completion of the initial survey involved 102 SEA-PHAGES instructors, which was an 85% response rate for the survey for instructors who participated in the student assessment outcomes process in 2018. Table 1 summarizes the demographic data collected on the respondents to the initial survey. As can be seen in Table 1, there is a range of faculty ranks represented in this data set, with a predominance of tenured faculty (63.4%). The sample of instructors predominantly identified as ethnically White (87.3%) and had a majority of female participants (56.4%). The proportions found within the SEA-PHAGES program are comparable to those of national averages with a predominance of white instructors (Heilig et al., 2019). The majority of the respondents had at least 11 years of science teaching (68.6%) including CRE and regular teaching. The respondents to the survey were relatively new to the SEA-PHAGES program, with 50.5% having been in the program for fewer than 3 years. SEA-PHAGES instructors who were invited to the annual SEA faculty meeting participated in the large-group feedback session. There were 96 participants at the SEA faculty meeting from a range of institution types, including research universities, 4-year schools, and community colleges. Travel and housing costs for the participants were paid for by the Howard Hughes Medical Institute. Approximately 90 participants were in the session representing the different institutions participating in the program. Seven instructors were selected for the expert validation focus group discussion. Two criteria were used for selecting the instructors for the focus group: instructors with student outcomes on the Persistence in the Sciences (PITS) survey consistently in the highest 10% and representation of a range a range of institution types (including community colleges, 4-year schools, and research universities), genders (four women and three men), and ethnicities (one African American, one Hispanic, and five White). The PITS assessment survey includes six psychosocial outcomes (Project Ownership Content, Project Ownership Emotion, Self-Efficacy, Science Identity, Scientific Community Values, and Networking) of a CRE research experience and has been shown to be reliable at differentiating CREs from other laboratory courses (Hanauer et al., 2016). As such, having student outcomes consistently in the upper 10% on the PITS survey is a measure of the ability of these instructors in teaching a CRE.

TABLE 1.

Demographic information on survey participants

Category	Frequency	Percentage
Rank
Part-time adjunct professor	3	3
Full-time adjunct professor	17	16.8
Full-time clinical professor	6	5.9
Tenure-track assistant professor	11	10.9
Tenured assistant professor	3	3
Tenured associate professor	24	23.8
Tenured full professor	37	36.6
Gender
Male	44	43.6
Female	57	56.4
Ethnic identification
Asian	4	3.9
Black or African American	3	2.9
Hispanic or Latino	5	4.9
White	89	87.3
Other	1	0.9
Years of teaching
1–2	3	2.9
3–5	12	11.8
6–10	17	16.7
11–15	20	19.6
16–20	15	14.7
21+	35	34.3
Years in SEA-PHAGES
1	16	15.5
2	21	20.4
3	15	14.6
4	7	6.8
5	4	3.9
6	8	7.8
7	7	6.8
8	7	6.8
9	8	7.8
10	7	6.8
11	3	2.9

Materials

A survey with quantitative and qualitative items was developed to collect data on instructor teaching practices and beliefs. Quantitative items were developed for this survey and consisted of the following instructor psychosocial variables: Self-Efficacy, Ownership of Educational Approach, Trust Relations, and Community Belonging. The Self-Efficacy as an Instructor scales were modified from Hanauer et al. (2016) and the Trust Relations scales from Cavanagh et al. (2018). The Ownership of Educational Approach and Community Belonging scales were developed specifically for this survey. Because the quantitative data involved both modified and new scales, a psychometric analysis of dimensionality was conducted. Following acceptable outcomes for the factor analysis of the dimensionality of each of these variables, scores were averaged for high-loading items on each variable (see Table 3 later in the article for full scales, factor loadings, and descriptive data). The qualitative variables address the instructors’ aims and activities. All responses on the survey to the qualitative data were written answers. The prompts for the qualitative data consisted of the following:

TABLE 3.

Descriptive statistics for instructor psychosocial variables

Instructor variables (7-point scale; 7 = strongly agree)	Item factor loadings	Mean	SD	Median	Minimum	Maximum
Self-efficacy as an instructor		6.68	0.45	7	5.25	7
I know how to explain scientific procedures relevant to this course.	−0.89
I know how to explain scientific concepts relevant to this course.	−0.82
I have adequate levels of scientific knowledge to be able to confidently teach this laboratory course.	−0.73
I know appropriate teaching strategies and approaches in order to teach this laboratory course.	−0.68
Ownership over educational approach	0	6.62	0.44	6.75	5	7
I think that students are positively engaged in this course.	0.99
I believe that the way this course is designed is most suitable for the aims of this course.	0.88
I am confident that the design of this course is of value to the student’s science education.	0.81
This course reflects what I learned about effective scientific inquiry teaching.	0.81
Trust relations with students	0	6.35	0.47	6.37	5.13	7
It’s important to me to understand what my students’ educational goals are.	0.68
I truly care about my students’ educational welfare.	0.61
I’m confident that my students would say that I “get” them.	0.43
Degree of community belonging	0	5.9	0.91	6.14	2	7
I feel that I belong in the SEA-PHAGES community.	0.78
I see myself as a phage-biologist I believe that my work contributes to the SEA-PHAGES community.	0.57
I believe that my work contributes to the SEA-PHAGES community.	0.58
The SEA-PHAGES community supports my pedagogical work.	0.58
Membership in the SEA-PHAGES community has added value to my professional identity.	0.62

Please take a few moments to think about your instruction in the SEA-PHAGES program. In the space below, please specify what you think are the main educational aims of the SEA-PHAGES program. What for you are the main educational aims of the SEA-PHAGES course?

In the space below, please explain how you achieve your educational aims in the SEA-PHAGES.

For the large-group member checking, once the small groups had been organized, the following prompts were provided for the evaluation of each of the models:

What does the model succeed in capturing? What relevant activities does the model miss? What additional relations need to be added? Is the model valuable in explicating instruction in the SEA-PHAGES program?

Participants were instructed to summarize their revisions and comments on the photocopied models provided. For the final focus group, each of the three models was presented in turn, and participants were asked to respond with any comments, modifications, and corrections they thought necessary.

Procedure

A survey link was sent to all SEA-PHAGES instructors who participated in the student outcomes assessment process using the online survey service Qualtrics (n = 120). The response rate from instructors was 85% (102 participants). Following online consent to research participation, participants responded to the survey. Feedback from SEA-PHAGES instructors on the emergent models was solicited at the annual SEA faculty meeting. All instructors at the SEA faculty meeting were invited to a group session. During the first 20 minutes of this session, the methodology and the analyzed models of instruction were presented to the whole group. Instructors were then divided into groups of four to six people and given a copy of the three models that had been presented. They were instructed to discuss the models as a group and address the prompt questions. Responses were written on the photocopies and collected at the end of the session. Each group presented an oral response to the models to the whole group and responded to questions and comments. The full session was video-recorded. For the final, expert focus group session, the modified versions of the three models were presented and discussed by the participants. The session was video-recorded. Data were collected according to the ethical guidelines of Indiana University of Pennsylvania Institutional Review Board (IRB no. 18-062).

Analytical Procedures

The analysis of the quantitative items on the survey followed standard statistical procedures. Descriptive statistics were used to understand the overall characteristics of the instructors in terms of the variables of Self-Efficacy as an Instructor, Ownership over the Educational Approach, Trust Relations with Students, and the Degree of SEA-PHAGES Community Belonging. Using standard qualitative analytical approaches, the verbal data on the survey were analyzed in the following stages:

1. Content Coding of Pedagogical Practices: The initial stage of analysis consisted of constructing a codebook of instructor responses to the open-ended items on the survey. Each of the instructors’ statements was analyzed in terms of the pedagogical practices specified and the aims that they were trying to achieve. Through a cyclical process of reading and note-taking, a provisional list of specific pedagogical practices and pedagogical aims was defined. Before the entire data set was coded, 20% of the utterances were coded by two researchers independently with an 87% agreement rate. All cases of disagreement were evaluated and either assigned to existing codes or a new code was developed. A second round of coding produced a 94% agreement rate, which was considered reliable for coding the data set for codes dealing with specific pedagogical practices. The list of specific pedagogical practice codes can be found in Table 2.

2. TURF (Total Unduplicated Reach and Frequency) Analysis of Aim: The frequency of occurrence for aims by instructor was calculated. A TURF analysis was performed to clarify the optimal agreement on the aims of the CRE across instructors ( Howell, 2016). The TURF algorithm identifies the optimal specified aims accepted by the largest number of participant instructors, which specifies which combination of aims is accepted by the largest number of instructors. This analysis produced a combination of three aims that were accepted by 95% of the instructors. The three aims were: 1) being a scientist and generating data; 2) teaching procedural knowledge; and 3) fostering project ownership. These three educational aims had the highest frequency of being mentioned across all instructors and, as such, were considered to best capture the educational aims underpinning the program as specified by instructors.

3. Pedagogical Practice Code Clustering: Once the data set had been analyzed for codes of specific practices, the codes used by each instructor were recorded. The sets of codes were organized initially by instructor and subsequently by the main educational aims specified by that instructor. In this way, codes were clustered by both instructor and aim.

4. Co-occurrence Code Analysis: The codes were analyzed within the verbal statements of each instructor for the co-occurrence of codes of specific pedagogical practices within an aim. The analysis was done in relation to the collection of statements from each instructor so as to not create artificial connections between codes. The analytical process of instructor-specific, co-occurrence coding created groupings of specific pedagogical practice codes found within each of the three aims specified in the TURF analysis. Specific pedagogical practices that had a similar pedagogical aim were grouped together into more general pedagogical practices, and high frequencies of co-occurrence in relation to particular educational aims were specified. Table 2 presents these groupings.

5. Model Specification: The result of previous stages of analysis consisted of the specification of three main educational aims of the program and a set of co-occurring pedagogical practices that facilitated this pedagogical aim. Three instructional models were defined based on this data. Related, co-occurring sequences of pedagogical practices were placed in relation to one another, and the full set of relations was graphically represented. Logical connections were made between the pedagogical practices, and instructor descriptions were rechecked. The result of this process was the specification of three models of instruction.

6. Member Checking: Instructor feedback and validation (member checking) was conducted to validate the emergent models from the analysis of the survey data. As specified in the METHODS above, the instructional models were presented to the collected audience of SEA-PHAGES instructors at the annual SEA faculty meeting, and through group work, specific modifications to the models were provided in written, visual, and oral form. All comments and suggestions from the instructor groups were analyzed. Repeated comments by at least two groups were considered important to address in revising the model, and revisions accompanied by a strong rationale were considered for inclusion. The following practices were added at this stage (Student Presentation, Scientific Output, Future Educational and Career Opportunities, and Ethical Understanding). The models were revised and updated with these additions.

7. Finalization of the Models: As a final stage of member checking, the models were presented to a focus group of expert instructors. The revised instruction models were presented and discussed with this group. Changes were not found to be necessary at this stage, and the models were finalized.

TABLE 2.

Codebook of pedagogical practices and associated educational aims

Pedagogical practice	Definition	Specific practices	Associated educational aim
Provide scientific content	A pedagogical practice in which background scientific information is provided to the student	Verbal explanation of scientific background for the technique being used Reading of primary literature Reference to lecture material	Procedure development
Explicit discussion of a CRE	A pedagogical practice in which the specific characteristics of a CRE (authentic research experience) are explained to the student	Differentiation of CRE classes from traditional labs Focus on the importance of generating authentic data Positioning students as researchers and scientists	Being a scientist and generating data
Facing ambiguity	A pedagogical practice in which students are made to face failure, uncertainty, and ambiguity in their scientific work	Allowing students to struggle with ambiguity and uncertainty Explaining that failure is a central part of science and is to be expected Holding back from providing quick answers	Being a scientist and generating data
Modeling scientific thinking	A pedagogical practice in which the instructor models verbally for the students the processes of thinking as a scientist in relation to the issues or problems they present in their ongoing research	Thinking through problems with students Modeling the design, planning, and methodological components of an experiment Critically thinking through and discussing the questions and issues posed by student work	Procedure development Being a scientist and generating data
Protocols and training	A pedagogical practice in which the instructor presents, trains, and supervises students following a new protocol	Provision and reading of the protocol Protocol practice and training Supervision of the following of a protocol	Procedure development Fostering ownership
Mentorship	A pedagogical practice in which the instructor or a peer mentors an individual student during the research process by providing individual guidance and problem solving	Mentoring of individual students by instructor or teaching assistant Peer mentoring and problem solving Provision of specific individualized guidance in writing	Being a scientist and generating data
Peer collaboration	A pedagogical practice that involves groups of students working together on their research	Team or small-group assignments Shared responsibility across the team Shared discussion and support for success and challenges	Fostering ownership
Encouraging independence in students	A pedagogical practice designed to enhance the ability of students to work independently as researchers	Directing students to think independently Asking students to problem solve themselves Positioning students as the decision makers in their scientific work Holding back from providing answers	Being a scientist and generating data Fostering ownership
Fostering personal responsibility	A pedagogical practice designed to enhance students’ understanding of their personal responsibility as researchers	Assigning specific work to be completed Explaining the responsibility of authentic scientific data Monitoring regular reporting of progress Requiring clear and consistent documentation of research Promoting personal reflection on the research that has been conducted	Fostering ownership
Encouraging enthusiasm	A pedagogical practice designed to recognize success and encourage positive enthusiasm toward research	Facilitating and recognizing the joy of discovery Instructor enthusiasm Course recognition of successes	Being a scientist and generating data
Documentation of scientific practice	A pedagogical practice involving the documentation of the research of the students	Lab notebook documentation Notebook discussion Notebook checking	Procedure development
Student presentation	A pedagogical practice involved in creating situations in which the students present their ongoing research	Lab notebook meeting with other students present Poster session for the presentation of outcomes Conference participation	Being a scientist and generating data Fostering ownership
Scientific output	This code relates to the actual scientific output of an authentic research activity.	Specific products of research (such as an isolated phage, electron microscopy, annotated genome)	Procedure development
Future educational and career opportunities	This code relates to the discussion of short- and long-term future educational and career opportunities.	Discussion of graduate school, internships, and related science careers Facilitating future options for scientific development through instructor network	Fostering ownership
Ethical understanding	This code relates to the discussion of ethical issues during course lab time with students	Discussion of ethics of research Discussion of integrity in research	Fostering ownership
Encouraging perseverance	This code relates to the encouragement a faculty member provided during lab work	Helping students overcome short-term frustrations Providing support during difficult periods	Being a scientist and generating data

RESULTS

Survey Participant Diversity and Alignment

To ensure validity and suitability of building instructional models with a range of instructors from the SEA-PHAGES program, we evaluated the degree to which these instructors were aligned and felt comfortable with the educational approach used in the program. On the initial survey to instructors, data were collected on the psychosocial variables of Self-Efficacy as an Instructor, Ownership over the Educational Approach, Trust Relations with Students, and the Degree of Community Belonging. As can be seen in Table 3, the instructors who completed the survey have very high levels of belief in their abilities as CRE instructors, strongly agree with the educational approach, have positive trust relations with their students, and have a sense of belonging to the educational community of the SEA-PHAGES program.

TABLE 4.

Mean, SD, and z-score for multiyear (2015–2020) SEA-PHAGES student outcomes (n = 22,492) and survey participant instructor student outcomes (n = 5564)

	Project ownership content	Project ownership emotion	Self-efficacy	Science identity	Scientific community values	Networking
Multiyear SEA-PHAGES student outcomes (2015–2020)	3.68 (1.04)	3.55 (1.11)	3.92 (1.06)	3.65 (1.14)	4.84 (1.41)	3.4 (1.19)
Student outcomes from instructor participants	3.62 (1.13)	3.48 (1.19)	3.85 (1.15)	3.61 (1.24)	4.76 (1.53)	3.4 (1.26)
z-score comparison	0.05	0.05	0.06	0.03	0.05	0.0

For evaluation of the representativeness of the set of instructors who participated in constructing the models of CRE instruction, the outcomes of their students on the PITS survey (Hanauer et al., 2016) was compared with the program-wide and multiyear data set of student outcomes. The z-score comparisons between the means of the multiyear sample (n = 22,492) and student outcomes from the instructors who participated in the survey (n = 5564) are between 0 and 0.06, indicating that the student outcomes for the participant instructors are very close to the averages for the SEA-PHAGES program.

Taken together, these comparisons suggest that instructors who participated in this study are valid informants for building models of instruction, because they are confident in their instructional abilities, are aligned with the program, and can produce student outcomes very close to program outcomes.

Aims and Pedagogical Practices of SEA-PHAGES CRE Instruction

The primary goal of this study was to describe the instructional practices of the SEA-PHAGES program. To accomplish this, we first determined the main educational aims of the SEA-PHAGES program as understood and accepted by the largest number of its instructors. Using TURF analysis of instructor responses to an open-ended survey, three aims that had the highest frequency of mention across all instructors and were shared by 95% of the instructors were identified. They consist of 1) being a scientist and generating data; 2) procedural knowledge; and 3) fostering ownership. These three aims are considered to best capture the underpinning aims of the SEA-PHAGES program as defined by its instructors.

To then uncover the set of pedagogical practices that instructors use to achieve their educational aims, we analyzed survey responses for the co-occurrence of pedagogical practices by both instructor and educational aim. Related, co-occurring pedagogical practices were then placed in relation to one another, and the full set of relations for each aim was graphically represented as a model of CRE instruction. Logical connections were drawn as lines between the pedagogical practices within each model, and these emergent models and connections were then checked, modified, and validated through an iterative process with SEA-PHAGES instructors. The three resulting models of CRE instruction and the relationships between pedagogical practices within each model are presented and described in the following sections.

Model 1: Being a Scientist and Generating Data

The pedagogical aim of supporting students to be a scientist and generate data is at the core of authentic research experiences as a STEM educational approach. As this educational approach is potentially different from other educational approaches to which many students may be accustomed, the pedagogical practices that support this aim are therefore centered around equipping students with the mindset and tools to engage productively with authentic research. These pedagogical practices, as described here and in Figure 1, can be grouped into three discernible stages.

FIGURE 1.

Being a scientist and generating data. Model presents co-occurrence relationships between coded verbal responses to survey items followed by member checking for validity. Lines represent co-occurring verbal codes within instructor statements concerning the instructional aim of being a scientist and generating data. The size of the box represents the number of related instructional practices. The model reveals the central role of ambiguity and uncertainty in CRE instruction and role of a series of instructor activities to alleviate, contextualize, and address this aspect of the CRE laboratory.

The first stage involves positioning students to be active participants in a research community. The model describes two pedagogical practices for doing so. The first involves equipping students with an understanding of their role and position as student-researchers within the context of a larger scientific community. This is achieved through explicit discussion with students about the importance of what they are doing, the ways in which their work interacts with the scientific community, and the resulting high bar required for the quality of their work (Table 2, “Explicit discussion of a CRE”).

The second approach involves providing students with the primary tools to begin conducting research. Here, students are trained to implement research protocols (Table 2, “Protocols and training”). Importantly, the learning and usage of specific protocols occur in response to the needs of their research at a given juncture. Thus, the learning of a protocol is not an abstract activity for future reference (as in some lab course designs), but rather is tied to a specific scientific output that the student needs to generate at a particular time during the course. These two pedagogical practices—explicit discussion and protocols and training—interact to support a deep understanding of the protocol and the need to perform the protocol appropriately so that students understand that the protocol is designed to produce authentic scientific data that will be used by other scientists. This is exemplified in the following instructor statements: “At the beginning of the semester, I explain the goals (as I see them) clearly and communicate the cutting-edge nature of this enterprise” and “I run my classes as ‘research groups,’ so that nearly everything we do is in service to increasing our understanding or moving forward of the research.”

As student-researchers begin to engage with research protocols, we enter the second and central stage of this instructional model, which deals directly with students facing ambiguous or unanticipated research outcomes inherent to authentic research (Table 2, “Facing ambiguity”). Instructors allow for situations in which a student is unsure of how to continue or faces an undesired research outcome, both natural aspects of science. However, given that this nature of science is not necessarily known or appreciated by a student-researcher and can therefore be difficult to contend with, the instructor contextualizes these outcomes and provides students the means to manage them.

As seen in Figure 1, SEA-PHAGES instructors describe two types of approaches for this. One approach involves a series of psychological dispositions and mindsets that include encouraging independence of thinking, perseverance, and engagement with and enthusiasm for their research. These are captured in the following quotes by instructors: “At the start of each lab class I ask the students to tell me what they are doing today, to encourage independence, rather than dependence on instructors” and “I praise and encourage every success, as well as their efforts so that they do not become discouraged.”

The second approach to help students manage ambiguity involves supportive and instructional frameworks. Here, instructors can promote peer collaborations so that students can share their expertise, skills, and knowledge to advance their research together, or the instructor can provide direct guidance and discussion for an individual student (Table 2, “Mentorship”). The latter is done by modeling the practice of scientific thinking (Table 2, “Modeling scientific thinking”). Modeling scientific thinking in the context of resolving an issue faced by students can promote their internalization of scientific thinking and thereby not only provide them the means to solve future issues but also the agency to move their ongoing research forward. Once students are able to manage the ambiguity they face, instructors can then engage students in the third stage of this model, in which scientific outputs that are valuable to the scientific community are actually produced. The specific scientific outputs will depend on the actual scientific inquiry being conducted. In the case of the SEA-PHAGES program, these include outcomes such as genome annotations submitted to GenBank.

Model 2: Teaching Procedural Knowledge

The pedagogical aim of teaching procedural knowledge is at the core of actualizing authentic research, because without procedural knowledge, science cannot be conducted. The pedagogical practices of this second model bolster an aspect of the first model—the teaching of protocols—so that a student can engage with protocols as a responsible, problem-solving, and decision-making student-researcher. The pedagogical practices for this aim, as described here and in Figure 2, can also be grouped into three discernible stages.

FIGURE 2.

A model of procedural knowledge instruction. Model presents co-occurrence relationships between coded verbal responses to survey items followed by member checking for validity. Lines represent co-occurring verbal codes within instructor statements concerning the aim of procedural knowledge instruction. The size of the box represents the number of related instructional practices. The model reveals the broader context within which protocols are taught in a CRE, specifying the importance of scientific background, modeling scientific thinking, and documentation in developing student understanding and use of scientific protocols.

The first stage involves the provision of scientific background for the experimental technique being taught (Table 2, Content Information). Importantly, this precedes teaching a student to use a protocol. The aim of this scaffolding is for students to be able to contextualize a procedure when it is taught and thereby avoid blindly following a protocol without really understanding what is happening from a scientific perspective. This is achieved through explicit instruction, discussion, and the reading of primary literature, and is exemplified by the following instructor statements: “I encourage my students to think through the methods for finding bacteriophages before I ever give them the protocols. This enhances their active learning by encouraging discussion and thought, rather than simply following instructions” and “My students also read primary scientific literature. I found a nice and relatively easy to read paper on phage therapy being used to treat coral disease. They follow the same basic protocols as the students carry out, so they’re not as intimidated by the Materials and Methods section.”

Once the student has sufficient content knowledge to understand a protocol, we enter the second stage, in which a student is taught to both use a protocol (Table 2, “Protocols and training”) and think as a scientist (Table 2, “Modeling scientific thinking”). The interaction between these three pedagogical practices—teaching content information, teaching protocols, and modeling scientific thinking—positions a student to be an independent researcher; a student will not only carry out a protocol, but will also be able to interpret the results, even when the results are ambiguous or unexpected; to problem solve; and to be the decision maker in relation the research that is being conducted. This positioning of students as independent researchers is exemplified by the way one instructor describes his interactions with students: “When they have questions, especially questions like, ‘What does this (result) mean?’ or ‘So now what do we do?’ I try to respond with my own questions, rather than answers. I try to get them to do the thinking, rather than me.”

In conjunction with promoting independence, being taught to properly document their research positions students to be responsible for their research (Table 2, “Documentation of scientific practice”). The usage of the notebook is described by one instructor in the following way: “We strive to help students understand that a notebook is far more than a collection of protocol steps. That it should, instead, tell the entire story of their experiment—explain their purpose and how the experiment works, record any modifications to the protocol, and thorough recording of primary data and their analyses.” This documentation process is therefore also an additional opportunity for students to exercise scientific thinking and to converse with the instructor and other peer student-researchers.

In the third stage, scientific output is generated that, as with other aspects of learning procedural knowledge, is contextualized in relation to the protocol, the science that underpins it, and the way a scientist would interpret the results.

Model 3: Fostering Project Ownership

Project ownership—the sense of the personal significance of the research being conducted—has been shown to be a feature that differentiates authentic research experiences from traditional laboratory courses (Hanauer and Dolan, 2014) and to promote the persistence of students in the sciences. The pedagogical practices for fostering project ownership are centered around promoting in students a sense of agency and personal responsibility to their research, to themselves, and to the larger scientific community. These pedagogical practices, as described here and in Figure 3, can be grouped into three different stages.

FIGURE 3.

A model of fostering project ownership. Model presents co-occurrence relationships between coded verbal responses to survey items followed by member checking for validity. Lines represent co-occurring verbal codes within instructor statements concerning the aim of fostering project ownership. The size of the box represents the number of related instructional practices. The model situates the development of project ownership within the learning of specific protocols followed by a series of instructor practices that encourage and support personal responsibility, peer collaboration, and shared research presentation.

The first stage of fostering project ownership begins with teaching a scientific protocol (Table 2, “Protocols and training”). This starting point is not arbitrary, because an actual ability to perform the science itself is required in order to be involved and engaged with, and to have ownership over, the research. This is principally an issue of self-efficacy and is exemplified by these instructor statements: “By having each student isolate a phage, characterize it, name it and see an image via electron microscopy, students get the feeling of ownership and contribution to the greater scientific community” and “Beyond all else, instilling beginning students with the confidence that they too can ‘do science’ establishes trust and mutual respect that leads to project ownership and success.”

Once a student has the means to actively engage in research, we enter the second stage, in which project ownership is fostered by making the students personally responsible for their research (Table 2, “Fostering personal responsibility”). One instructor stated the following: “I give the students as much control and ownership of their experiment as possible, and make them responsible for it as well. I tell them how to do the protocols, of course, and provide the materials, but then, essentially, we provide lab space for them for fourteen weeks, all day, every day, only interrupted by the scheduled meeting times of the class.” This responsibility is reinforced through several other pedagogical approaches. In one approach, the student’s responsibility is grounded through an understanding of the ethical aspects of conducting research such as the importance of integrity, honesty, and transparency (Table 2, “Ethical understanding”). In another approach, the student’s personal responsibility is bolstered through the explicit positioning of the student as an independent thinker (Table 2, “Encouraging independence in students”). Allowing and promoting the process of independent thought, problem solving, and decision making in relation to their research, gives students agency to be personally responsible for their research. One instructor stated: “I encourage them to evaluate their results, to discuss them with their peers, and then to present them to me. I don’t ever tell them what they should do next—instead I help them to work through the pros and cons of the various options before them, and encourage them to test out their ideas. When the results are in, I encourage them to evaluate them critically, and then to repeat the thought process.”

As students become more personally responsible for their own research, we then enter the third stage, in which a number of pedagogical practices are implemented to support students as they manage that responsibility. With regard to the more anxiety-ridden aspects of having personal responsibility over one’s research, the instructor can help by emphasizing the positive aspects of doing research (Table 2, Encouraging Engagement and Enthusiasm). One instructor stated: “I take time to get to know my students and provide them a tremendous amount of support and encouragement. I bring enthusiasm as well as compassion to my class/lab.” Instructors can also promote teamwork (Table 2, “Peer collaboration”) that requires shared and reciprocal responsibility of (or by) the team members. Personal responsibility can be further enhanced by relating the ongoing research process to future outcomes for a student’s own career. In the medium term, this can be in the form of presenting personal research to other people in the form of a final course poster session, data publication, research article participation, or conference participation (Table 2, “Student presentation”). In the longer term, participation in a research experience can provide important insight, skills, and experience that may be important for future career goals, such as graduate school and a science-related career (Table 2, “Future educational and career opportunities”). For students, having responsibility over their research also involves having responsibility over their educations and futures.

Summary of Relations between CRE Instructional Models

The research presented here describes CRE instruction in terms of three interrelated models. The first model—being a scientist and generating data—is focused on a set of instructional practices that support students while they handle the ambiguity and uncertainty of the scientific process in order to produce scientifically valuable output. The second and third models develop the first model by explicating the teaching of protocols in order to generate data and fostering underpinning aspects of being a scientist, respectively. While all three models can be seen as distinct instructional models fulfilling specific CRE-related educational aims, all three are connected to the larger aim of helping students to perform as novice scientists who produce usable scientific output.

DISCUSSION AND CONCLUSIONS

Prior research on CREs has tended to focus on the definitional aspects of a CRE (Hanauer et al., 2006, 2012, 2016, 2017; Hanauer and Dolan, 2014; Graham et al., 2013; Auchincloss et al., 2014; Hernandez et al., 2018) and student outcomes (Russell et al., 2007; Jordan et al., 2014; Hanauer et al., 2017; Hernandez et al., 2018). Less has been said about the characteristics of the pedagogy used in this educational approach. The aim of this paper is to characterize the pedagogy of CRE teaching through the modeling of instructors’ statements about how they teach. The results of this study suggest that three interrelated instructional models describe the specific components of CRE inquiry teaching. These models describe how to facilitate the process of being a scientist and generating scientific data and ways of teaching procedural knowledge and fostering project ownership.

A summary of the models and the data presented in this study suggests three underpinning features that characterize CRE pedagogy. The first feature is inherent to the design of a CRE and consists of the centrality of an active research agenda that directs the content and outcomes of the course. When the implementation of scientific research with needed scientific outputs directs the work of student-researchers, then many aspects of the laboratory course need to adapt to the procedural variability of authentic science. There is a degree of uncertainty related to timing, outcomes, and course development that needs to be addressed. The second feature that characterizes this form of pedagogy emerges from the first. Because the aim is to contribute to an active research agenda, and this causes a degree of uncertainty, a range of additional pedagogical supportive practices need to be implemented by the instructor. Instructors need to provide encouragement, foster perseverance, support peer work, model scientific thinking, and mentor students as they work through the research they are doing. These interventions are ongoing and timed in response to contingencies that emerge in the scientific research itself. The third feature is tied to both the first and the second features summarized here and consists of a change in the relationship between the instructor and the student. Because both the student-researcher and instructor are involved in the same research agenda, their relationship is characterized by shared interest in producing scientifically valuable outputs. As such, instructors encourage and support students’ personal responsibility, independence, and ownership over the research and, at the same time, situate students’ work within their own research and that of the scientific community. Together, the students and the instructor move the field of science forward.

The pedagogy presented here reveals differences in the relations between instructor, student, and course work in a CRE compared with a traditional laboratory course. The traditional laboratory course is defined by a structured environment for both student and instructor, in which the expectations of course timing, course content, and procedural outcomes are predetermined (Tamir, 1977; Tamir and Lunetta, 1978; Domin, 1999; Weaver et al., 2008) and the pedagogy involves explicit instruction, demonstration, and hands-on implementation of a set of defined experiments with defined outcomes (Tamir, 1977). As a result, the core set of relations between instructor and student are those of expert and novice, and while the student’s work might be consequential from an educational perspective, it has no scientific value in itself. This is very different from what is described for CRE pedagogy in this study. The instructor is a mentor and a guide; the student is an independent responsible researcher; the research is real and reported to the scientific community. Teaching in a CRE requires a different set of positions for the instructor, student, and scientific output. We hypothesize that these different positions are central to the positive outcomes observed for students participating in CREs.

The pedagogy presented here has implications for instructors and institutions new to CREs. CRE pedagogy has specific qualities that may make it different from the pedagogy of other undergraduate science courses. The CRE approach counters more traditional beliefs in a structured knowledge hierarchy within which scientific research can only be conducted at higher educational levels. These beliefs often inform the design of early-career laboratory courses and create labs designed to help students learn procedures in a controlled way without facing the difficulties and uncertainties of the authentic scientific process. CRE pedagogy, on the other hand, parallels the pedagogy of graduate science education, in that both are rooted within the context of doing science and involve cognitive, conceptual, and emotional facilitation to manage the uncertainties of the scientific process and to be productive as a scientist. Undergraduate science educators are likely already familiar with this form of pedagogy from their own graduate educations, with a subset additionally engaged in teaching graduate students. Key, then, to supporting instructors as they adopt and implement CRE pedagogy, are an institutional or departmental mindset and culture that promote the integration of undergraduate teaching with research, so that the scientific process is both authentic and a fundamental component of undergraduate science education, and that encourage instructors to leverage their graduate training experience and their researcher identity to inform their teaching. The models and data presented here provide a framework to support discussions regarding how to best teach a CRE.

There are limitations to the current study. The core data set used to produce the visualizations and descriptions of the instructional models consisted of written responses to open-ended questions. While a systematic approach was used to analyze these responses, verbal data are polysemantic and can be understood in a variety of ways. To address this, the current study used two levels of member checking and feedback provision from the instructors active in teaching a large-scale CRE. However, these instructors did come from one program—the SEA-PHAGES program—and this also might be a limiting factor of what is described in this study. The nature of this program does have diversity in terms of the student bodies taught and institutions involved, but all participants came from the same program, and this could have influenced what is described in the models. The models are still exploratory, and further research will be needed to show the relationships between these models and student outcomes.

The instruction models presented in this report attempt to capture what is specific to CRE inquiry teaching. They present a complex picture of the ways in which instruction is conducted on a daily basis as understood by instructors who teach in a CRE. Being a CRE instructor requires a flexible richness in addressing individual students and the work they are doing. It requires the ability to honestly interact as a scientist with the work conducted, and it requires allowing a student to function independently while providing the right amount of support for that independence. More than anything else, being a CRE instructor requires seeing the students as researchers and facilitating the process so that the students see themselves as scientists and their work as part of the scientific community.