Training for Responsible and Ethical Management of Lab Notebooks in a Course-Based Undergraduate Research Experience

ABSTRACT

Course-based undergraduate research experiences (CUREs) represent an innovative educational strategy to engage more science, technology, engineering, and math undergraduates in authentic research experiences. Research shows that student participation in CUREs results in positive student outcomes similar to those for traditional research experiences. However, less is known about how the research focus of a CURE or the varied emphasis on certain CURE design elements can impact student outcomes. CUREs provide a unique opportunity to infuse training essential for future researchers. Although responsible and ethical conduct is an important component of research and scientific practice, limited attention has been paid to incorporation and assessment of responsible and ethical conduct of research (RECR) in CUREs. Here, we address the gap in CURE RECR training by presenting an activity that can be easily built into any CURE or inquiry-based lab to train students in RECR relative to data management, specifically, the lab notebook. In this activity, students are asked to replicate or execute an experiment with only the records of a previous student’s lab notebook. This previous student’s notebook is purposefully designed by the instructor to miss important information that might not seem obvious to students but would prevent a future researcher from replicating the experiment. The idea is to create an early understanding of delayed gratification for students when it comes to responsible and ethical maintenance of lab notebooks. This activity is paired with a pre- and postactivity lecture and debriefing to instruct, guide, and reflect with students on RECR surrounding lab notebooks as well as iterative practice and assessment of lab notebooks throughout the semester.

INTRODUCTION

Course-based undergraduate research experiences (CUREs) have gained traction over the last decade in undergraduate biology curricula across the country for their ability to provide large numbers of undergraduates the academic and professional benefits of an authentic research experience. Through a CURE, students learn and practice skills used by scientists on a daily basis. CUREs model scientific research by incorporating five core elements into their curricula: discovery, a focus on broadly relevant or important work, collaboration, iteration, and the use of scientific practices (1). Some scientific practices commonly integrated into CURE curricula are hypothesis development, method selection, peer review, identifying meaningful variation, navigating the messiness of real world data, communicating findings, etc. One scientific practice that is critical to the integrity of the scientific process, but is often underaddressed in CUREs, is the ability to conduct research in an ethically responsible manner (2–4). The importance of this scientific practice is evidenced by the emphasis that has been placed on training in responsible and ethical conduct of research (RECR) by institutions, national agencies, and global groups. Training in RECR is a highly emphasized component of training for graduate students and early career researchers (5–9). Consequently, undergraduate RECR training has been mostly ignored, and the limited existing efforts have focused on training through apprentice-style mentorship or one-off seminars (10–12).

CUREs have allowed increasingly more undergraduates to participate in research, and they provide a unique and high-throughput opportunity to teach RECR content to undergraduates and foster a student’s ability to apply this content to their research through iteration and practice (2–4). Research has shown that undergraduate researchers bring a decent understanding of certain “mainstream” RECR topics, like fabrication, falsification, plagiarism, and confidentiality, but lack understanding of other important RECR practices and cannot necessarily apply their understanding in the lab (13). While participation in workshops can help students’ understanding of RECR, it cannot always impact application of that knowledge. CUREs provide an optimal environment to impact application of RECR knowledge through the broad relevance and importance of the research and the focus on iteration (1). If made relevant and purposeful for students, explicit discussion and practice of RECR decision-making as part of the day-to-day research process in CUREs may improve undergraduates’ application of RECR knowledge.

RECR encompasses a wide range of concepts grouped into nine core components: mentor and trainee responsibilities; publication practices and responsible authorship; peer review; collaborative science; human subjects; research involving animals; conflict of interest and commitment; research misconduct; and data acquisition, management, sharing, and ownership (14). Because CUREs are constrained by their course-based nature, not all aspects of RECR can be trained and covered while meeting the CURE’s main learning objectives. Thus, it is crucial to select an RECR area with relevant focus in order to ensure effective training. Within CUREs, data management is used on a daily or weekly basis, as students are frequently collecting data to support their research.

One RECR activity which falls under data management is maintenance of a lab notebook. This process of scientific documentation represents activities that researchers do on a daily basis to communicate and record their data, interpretations, theories, and findings over the course of an experimental series or research project. These records serve as the foundation for future publication and are therefore critical to the rigor of the overall scientific process. However, lab notebooks and record keeping are common areas of issue in RECR. At its most extreme, record keeping misconduct can include things like data falsification or fabrication. However, it can be argued that ethical concerns regarding record keeping are not typically egregious or malicious in nature. Misconduct in day-to-day management of lab notebooks and record keeping is easy to accumulate. This accumulation of little errors or poor record keeping can become a large problem that can eventually prevent research from being disseminated. Because CUREs are aimed to engage undergraduates in the scientific process and lab notebooks are an integral part of this day-to-day process, it makes this RECR focus more relevant in a CURE setting. In considering the distinction between extreme record keeping misconduct and “sloppy science” for undergraduate RECR training, we chose to focus on the latter for several reasons. Primarily, these day-to-day errors and omissions are a struggle for all early researchers, because they have yet to comprehend the accumulated impact of these small actions. Additionally, undergraduates have a better grasp on ethics surrounding large misconduct issues, like falsification, fabrication, and omission of data (13) (see Table 2, below). Furthermore, many large RECR decisions are made by stakeholders higher up in the decision-making chain of command (principal investigator, department chair, ethics committees, etc.). As a result, RECR training has been primarily provided for higher-level scientists responsible for making large and important ethical decisions. It is crucial, however, that undergraduates and entry-level scientists receive RECR training not just for large-scale decision making but also for smaller day-to-day actions and decisions in which undergraduates regularly participate.

TABLE 2.

FRI-specific case study questions on lab notebooks and data management

Additionally, limited studies on lab notebook abilities suggest that research scientists are traditionally unprepared by their undergraduate training for how to keep responsible and ethical records (15–17). Analyses show that undergraduate training is often inadequate, because the lab activities do not actually require the use of a lab notebook to document, due to experiments having known outcomes, being one-off, and being scripted or “cookbook style” (16). Training is also impacted by poor guidelines and lack of feedback by instructors. Most importantly, no matter how much the importance of doing something is understood, unless it has a personal importance or relevance, it is hard to turn understanding into practice. This is the problem of delayed gratification (17). Unless students can feel the importance of good record keeping and feel the negative impact of having to retrace poorly documented steps to analyze, present, or publish their data, they may never fully build responsible and ethical abilities in scientific documentation.

As such, we integrated an activity with scaffolded instruction and semester-long feedback into our CURE to help students realize the difficulty of delayed gratification before it became a problem for them in the lab. Our goal was to help students feel a personal impact of poor lab notebook maintenance in their own research by creating a similar moment of frustration felt by many researchers when trying to replicate experiments or publish work from previous lab notebooks. These moments of frustration are formative experiences where researchers truly understand not just the importance but also the impact and outcome of poor scientific documentation. To recreate this feeling, we provided students with a previous student’s lab notebook and instructed students to attempt to replicate the student’s work without class demos, class instruction, and step-by-step guidance from the instructor and peer mentors. Students were left to rely on the scientific legacy of previous students. In this case, the provided lab notebook was purposefully riddled with issues that would prevent a downstream user, like the students in the CURE, from being able to fully replicate the work. Here, we explain what we termed the previous student’s notebook activity (PSNA) and the corresponding lectures, rubrics, and assessments that were scaffolded around the activity to bring RECR training, specific to lab notebook maintenance, to the CURE classroom.

Intended audience, learning time, and prerequisite student knowledge

This laboratory notebook training program was developed for an introductory-level CURE for first-year biology majors, but any CURE or research experience could use some or all of this training module. In this module, students participated in an introductory workshop on lab notebooks and data management (1 h), a lab activity (3 h), and a debriefing discussion on the RECR surrounding lab notebooks and data management (1 h). Additionally, students iteratively continued to practice these learned skills regarding lab notebooks and data management throughout the semester as they maintained their own team lab notebook for their research (approximately 2 lab notebook entries each week of the 15-week semester). Qualitative and rubric-scored quantitative feedback on notebooks was provided for each entry by the instructor and peer mentors. Prior to execution of the lab activity, students should receive lab safety and basic lab skills training such that they can safely and successfully navigate the lab environment and execute the basic functions of the experimental protocol or method being used in the lab activity.

Learning objectives

At the end of this training module students will be able to achieve the following:

1. Create and maintain lab notebooks for primary research that meet scientific standards for experimental replication (i.e., precision, accuracy, completeness, organization).

2. Assess and revise lab notebooks so that they better meet scientific standards for experimental replication.

3. Describe the importance and ethical responsibilities of record keeping with regards to the impacts it can have on downstream users.

4. Predict the future consequences for themselves and broader communities if the daily professional responsibilities of record keeping are neglected.

Data presented in here were determined to be exempt (STUDY00002790) and were collected in accordance with Binghamton University’s Institutional Review Board rules and guidelines.

PROCEDURE

Materials

An outline for the activity and its associated lectures and discussions is provided in Table 1. In the example activity provided in this paper, students carried out a bacterial growth curve experiment following only the documentation from a modeled version of a previous student’s example notebook entry, which was devised intentionally by the authors to include some of the most common and impactful lab notebook mistakes (see Appendix 1 in the supplemental material). However, this activity can be executed for any experimental protocol or technique of your choosing. Materials needed for this activity will depend upon the experiment being used but would minimally require lab space, lab equipment, and lab supplies required to carry out the user’s desired experiment. The lab notebook could be a traditional bound composition book or an electronic lab notebook, as described for this activity (see Appendix 2 in the supplemental material) if space and resources permit the safe use of technology in the lab.

TABLE 1.

Timeline for the activity and corresponding pre- and postactivity lectures and discussions

Student instructions

1. Print out and bring a copy of the example notebook entry to lab (see Appendix 1 in the supplemental material).

2. Following general whole-lab instructions and announcements, work with your lab partner to replicate the experiment described in said notebook entry with the strains and conditions provided during the whole-lab instructions.

3. Attempt to execute this experiment with limited help from the instructor and peer mentors, asking only for help when you have made all reasonable attempts to proceed with the information provided in the notebook entry.

4. Throughout the experiment, document what you and your lab partner are doing, so that you can create a lab notebook entry for yourselves following execution of the experiment.

5. Using time remaining in the lab period and any additional time required outside of class, create a lab notebook entry for your experiment following the requirements outlined in the notebook instructions (see Appendix 2) and self-assessing your work using the notebook rubric (see Appendix 3).

6. Once you have created and assessed your lab notebook entry and you are ready to submit your work, export your electronic lab notebook entry as a PDF from Microsoft OneNote and submit to the corresponding assignment on the learning management system.

Prior to this lab activity, students reviewed and assessed the previous student’s notebook entry as part of a prelab lecture on how to maintain lab notebooks. Following the lab activity, students reassessed the previous student’s notebook entry and discussed the challenges associated with using this notebook entry in a postlab lecture on the consequences of poor notebook maintenance.

The PSNA was singular in its intent to elicit feelings associated with repeating work from a less-than-perfect lab notebook. However, the general structure of the labs and the requirement to maintain and submit a lab notebook entry as described above were consistent throughout the semester. Iteration and assessment were intended to grow students’ abilities to maintain an effective and ethical lab notebook that could be used by other students in the future.

Faculty instructions

Prior to the PSNA, instructors should dedicate lecture time (1 h suggested) to discuss what lab notebooks are, how to create and maintain lab notebooks, and RECR surrounding lab notebooks and data management (17–22). One of the most important topics to discuss in this preassignment lecture should be why we keep notebooks and why they are important in the CURE setting. Additionally, instructors should introduce the lab notebook requirements and rubric by doing an active learning activity where students use the instructions and notebook rubric (Appendices 2 and 3) to evaluate the example notebook entry for the PSNA (see Appendix 1 in the supplemental material). This familiarizes the students with the lab notebook entry prior to lab and with the notebook rubric and requirements. Additionally, this preassignment assessment will provide a point of comparison for discussions following the in-lab activity. Before moving onto the PSNA, instructors should also be sure to cover content, concepts, and skills required to understand, apply, and execute the particular lab protocol or experiment detailed in the example lab notebook entry.

Prior to the in-lab portion of this activity, instructors should ensure students receive lab safety training required to work in the lab and execute the particular experiment. During the in-lab portion of this activity, instructors should review any relevant content or safety information required for the experiment being conducted as part of the PSNA. Instructors should then connect the PSNA to the preassignment lecture by setting the scene for students about how many research projects get handed down in labs from one student to the next through their notebook records. Not always are students or faculty able to train the incoming students in a hands-on manner. Incoming students must be able to pick up the records of the previous student and apply them to the current problem or continue to replicate the experiment as previously done in the lab. The instructor should then instruct students to conduct the day’s experiment via the example notebook entry created by the instructor (see Appendix 1) as outlined above. All students receive the same example notebook entry to work from. Suggestions to consider when adapting or building an instructor’s own PSNA for other experiments or techniques relevant to the lab or CURE are included in Appendix 4 in the supplemental material.

Following the in-lab activity, instructors should plan to debrief the PSNA and have students discuss how their beliefs, attitudes, and behaviors may have changed regarding the importance and implication of poor lab notebook management and scientific documentation. This debrief could be part of a lab or lecture, but we suggest an hour for this debrief. Some prompts for discussion are included in Appendix 5 in the supplemental material. Additionally, instructors should have students reevaluate the lab notebook example using the rubric and compare it to their initial assessments. Instructors can use students’ pre- and postassignment evaluations as talking points to unpack what might have caused them to evaluate it differently after having gone through the PSNA and what realizations they may have come to in hindsight (17). Instructors should continue to use the same notebook instructions and notebook rubric (see Appendices 2 and 3) throughout the semester to help students practice applying the lessons learned from the PSNA.

Suggestions for determining student learning

In an attempt to assess student learning, questions on knowledge, attitudes, and decision-making regarding RECR and lab notebooks were included in a pre- and postsemester survey (see Appendix 6). Student responses to in-class discussion prompts (Zoom recording transcript and chat transcript) were also used to provide context to student learning and insight into their thought processes. To assess students’ ability in maintaining a research lab notebook that conforms to RECR practices, the notebook rubric (see Appendix 3) was created and used to assess students' weekly lab notebook entries. Lastly, student reflection essays were used to see if students discussed learning and growth in lab notebook maintenance. Focused interviews and discussion groups can also be used to attain qualitative data on students’ thoughts following the activity or semester of training.

Safety issues

For this activity, safety concerns and controls should be adapted to meet the requirements of the experiment being conducted with students. For instance, in the provided example, when working with microorganisms, faculty should follow all ASM guidelines for biosafety in teaching laboratories (23). When designing the PSNA, careful detail was put into creating the student example to ensure that no important details were left out that would jeopardize the safety of students involved.

DISCUSSION

Field testing

The activity presented here was piloted in the second CURE of a three-CURE sequence on Microbial Biofilms in Human Health within Binghamton University’s First-year Research Immersion (FRI) Program (24). This second-semester CURE is designed to train students in the basic research methods of microbiology and molecular biology through a class research project on bacterial biofilms and their mechanisms of antimicrobial tolerance. Additionally, students read literature and develop team research proposals for their own research project on microbial biofilms that they will execute in the third semester of the program with the learned research skills and proposed experimental design from this second-semester CURE. There were 30 students in the first cohort in which this activity was field tested.

Evidence of student learning

As part of the preactivity lecture, students were asked to evaluate the example notebook entry (see Appendix 1) using the notebook rubric (see Appendix 3) and discuss their reasonings for their evaluation. Following the PSNA, students were asked to reevaluate the previous student’s notebook entry. In Fig. 1 you can see students’ scores for pre- and postactivity evaluation, broken out by rubric categories. Overall, the students scored the notebook as a 15.2 out of 18 in the preassignment evaluation, followed by a significant decrease in score (P < 0.005) to 13.3 during the postassignment evaluation. Although nearly all sections showed a reduction in score from pre- to postassignment, the sections that showed the most change in score were the “Background, Purpose, Anticipated Outcomes, and Hypothesis” and “Protocol” sections. Interestingly, when we compared the students’ scores to scores generated by four different instructors from the FRI program, we saw that initially students scored the example higher than instructors, but that following the lab activity, students’ scores more closely resembled the instructors’ scores. During the postactivity lecture, students were instructed to compare their pre- and postassignment evaluations and discuss why there may or may not have been differences in scores. Some of the themes heard from students included a realization, following the lab activity, on what level of detail was truly needed for a lab notebook to be replicable by another user and what level of project understanding was required to know the “why” and “how” of the experimental protocol. For example, one student said, “We understand what it's like to be very confused in lab and to prevent others from having that confusion.” There were shared sentiments of frustration, confusion, stress, and anxiety felt by students during the lab activity, and students shared that these feelings helped them to better understand the importance of the lab notebook as well. Although we cannot conclude that the PSNA was alone responsible for student growth, students did show a significant improvement in lab notebook grades over the course of the semester (Fig. 2). It is likely that a combination of the PSNA with the iterative practice and feedback with the lab notebook rubric all contributed to improved student abilities in maintaining a lab notebook that would meet RECR standards.

FIG 1.

Instructor versus student pre- and postassignment evaluations of the example notebook entry. Data show average evaluations of the example notebook entry (see Appendix 1 in the supplemental material) using the notebook rubric (see Appendix 3) by students (n = 30) and instructors (n = 4) in the FRI program. Student participants were asked to evaluate the example notebook entry prior to and after the PSNA, as part of the pre- and postassignment PSNA lecture periods. Significant differences are indicated with brackets and asterisks (*, P < 0.05; **, P < 0.005; ***, P < 0.0005; ****, P < 0.0001) and were calculated with a one-way analysis of variance and Tukey’s post hoc tests.

FIG 2.

Class average scores on notebooks throughout the semester. Data represent average class scores, out of 18, on the lab notebooks, which were graded with the notebook rubric in Appendix 3 in the supplemental material. The x axis represents scored, individual lab notebook entries in chronological order. Data showed an improvement in student lab notebook management abilities, with a statistically significant increase in scoring from the beginning (notebook 1) to the end of the semester (notebook 14), calculated with Student's t test (P < 0.0001).

When attempting to look at students' understanding of, and attitudes related to, RECR in data management and lab notebooks using a pre- and postassignment survey created for this activity, we saw a statistically significant increase in score (P < 0.05), based on analysis using a paired, two-tailed Student's t test. The student overall scores on the postassignment survey (mean, 56.00/60.00) were significantly different from their overall preassignment survey scores (mean, 53.47/60.00) with a medium Cohen’s d effect size (0.588). This indicated that students had a meaningful change in their understanding and application of RECR as it related to laboratory notebooks and data management. Figure 3 depicts the individual pre- and postassignment survey questions and summarized class responses to each. Questions with the most variation and change seemed to be related to how RECR did or did not apply to research being conducted in a CURE like the one in which they were enrolled. This is interesting and worth further exploration. This may indicate that students have a good compass for what is right and wrong for a hypothetical researcher (Table 2) but are less clear on when those behaviors and ethical decisions are their responsibilities. This was further demonstrated by results of case studies, which were also a part of the pre- and postassignment surveys (Table 2). Case studies also showed some change in student understanding when it came to lab notebooks but not when it came to ethical data handling. Again, this highlighted a disconnect between understanding of rules and norms and application of these rules and norms in a CURE setting.

FIG 3.

Pre- and postassignment survey student responses to questions on lab notebooks and RECR. Data show student responses to questions on a 5-point Likert scale regarding concepts, behaviors, and attitudes on lab notebooks and data management prior to and at the end of the semester. A total of 30 students responded to both the pre- and postassignment surveys.

One final area where we saw evidence of student learning was in their end-of-semester reflection essays. At the end of each semester in the FRI program, all students are asked to write 300 to 500 words reflecting on their personal and professional growth throughout the semester. Of the 30 reflections, 12 made mention of growth related to skills or understanding of lab notebooks or data management. For example, one student reflected, “While writing the lab notebooks…seemed tedious to keep up with sometimes, I must admit that in retrospect it was a very important component of this course that would help me to perform experiments that other people can replicate.” In another example one student said, “Speaking of directions, actually writing reliable protocols was very helpful in itself; by doing this, I feel that I have gained a much deeper appreciation for writing down effective protocols. As I have painfully learned, redoing experiments revealed the flaws of written protocols since it is much harder to execute the protocol in the lab than it is planning to do it. Small details like shaking vs non-shaking incubators, or even locations of supplies suddenly felt essential.” In these comments, we can gain perspective on students’ understanding of the value of lab notebooks. Five of the 30 reflections also mentioned data management, discussing the value of being organized in research efficiency and success. For example, one student stated “FRI had prepared me a lot for doing professional lab works and how to record our data and keeping a professional lab notebook.” It is interesting that nearly 45% of the students chose to write about their personal and professional growth relative to lab notebooks or data management when given the freedom to focus on any aspect of personal or professional development throughout the semester. We feel this is further evidence of student learning generated through the lectures, activity, and assessment of lab notebooks discussed in this curriculum piece.

In the future, we hope to better assess student outcomes as a direct result of the PSNA as well as the iterative practice and feedback on lab notebooks through this CURE and its subsequent, connected CURE. We would like to refine and build upon the survey instrument used in this activity such that it can be used to generally assess knowledge and perceptions on lab notebooks regardless of the semester, CURE, institution, etc. Because development of lab notebook management skills requires time, we think it is important to measure skill development with notebook writing and changes in perceptions or attitudes with regard to notebooks over time. This includes during the third semester of the program, when notebooks continue to be a critical component but instruction is less activity based and more feedback oriented. We would also like to explore further ways in which we can build upon this activity to impact application and practice of these abilities in students’ undergraduate education and beyond.

Possible modifications

This activity could be modified to any discipline or research project, as all research disciplines require lab notebooks of some form or another. The central idea is to provide a teachable, dissonance moment where students can come to their own personal realization of the importance of their lab notebooks. By providing a dissonance moment, students can challenge their own previously held thoughts and beliefs on the importance of lab notebooks, and hopefully this moment will create positive behavioral change that encourages them to maintain more thorough and ethically responsible lab notebooks. In this activity, all students received the same previous student’s notebook entry with the same group of omissions, designed to reflect mistakes most often observed in student work. This example and activity could be modified with the use of multiple versions of the previous student’s notebook entry, each highlighting one main omission or error. At the end of the activity, groups could share their experience and what they have learned through class discussion. With any iteration or modification of this activity, the goal is to build better notebook and data management habits earlier in research training through incorporation into CUREs, so that we can better prepare and train the next generation of researchers to make responsible and ethical choices.