Responsible and ethical conduct of research (RECR) diagnostic survey using case scenarios from biology course-based undergraduate research experiences (CUREs)

ABSTRACT

Course-based undergraduate research experiences (CUREs) are increasingly becoming the first, and perhaps only, research experience for many biology students. Responsible and ethical conduct of research (RECR) is crucial for the integrity of scientific research and essential for students to have an understanding of the scientific process at any academic level. However, there is a current lack of RECR education in biology CUREs. To understand the level of RECR knowledge and skills in undergraduate students, we created a diagnostic survey that uses case scenarios designed to illustrate RECR issues in the CURE classroom. Analysis of students’ responses indicated that the overall percentage of students who are able to effectively use RECR terminology and identify the impact of RECR violations on science integrity and ultimately on society is low. Furthermore, some students equated RECR violations to academic dishonesty, indicating difficulties separating the research and academic aspects of CUREs. This diagnostic tool can aid instructors in identifying gaps in student RECR knowledge for the subsequent development of RECR educational interventions, particularly to ensure the integrity of the research performed in CURE settings.

INTRODUCTION

Course-based undergraduate research experiences (CUREs) have been associated with a diversity of impacts such as increased content knowledge (1); increased scientific abilities (e.g., increased ability to analyze data and increased familiarity with the research process) (2); attitudinal and motivational benefits (e.g., gains in scientific identity and gains in self-sufficiency) (3, 4) and long-term academic performance (e.g., increased graduation and increased cumulative GPA) (5). Additionally, a national survey analyzing data on 534 courses in the biological sciences reported that 77% of the courses devoted at least some time to research and 44% devoted more than a quarter of the course time to research, as defined by the respondents (6). Although the survey did not evaluate whether these courses were CUREs or not, it suggests that research-based experiences in biology courses are both beneficial and prevalent.

In order for scientific research to achieve its role as an activity that serves and benefits the public, it must be conducted with integrity. The importance of responsible and ethical conduct of research (RECR) to the research endeavor is highlighted, for example, by the emphasis that US federal agencies place on the requirement to offer RECR instruction for all research trainees involved in a research project (7–9). These mandates have focused on RECR instruction for research trainees who are conducting research under a traditional apprenticeship model, not those conducting research in the context of CUREs. Since many CUREs immerse students in authentic research, one important component of the experience is for the students to learn how to perform research according to RECR standards. However, a national survey of CURE instructors revealed a widespread lack of formal RECR instruction in CUREs (10).

In order for instructors to design effective RECR educational interventions, we need first to understand the current knowledge of RECR students bring to the classroom. With this aim, we designed a diagnostic survey to assess students’ understanding of research misconduct and questionable research practices (QRPs), two types of RECR violations. Research misconduct, defined as falsification, fabrication, or plagiarism, was chosen because the severity of its effects makes it a salient topic to measure students’ understanding of RECR. QRPs, on the other hand, are less severe violations than research misconduct, but are much more prevalent in research. A meta-analysis of researcher surveys found that ~12.5% of researchers admit to committing QRPs compared to only 2.9% of researchers who admit committing research misconduct (11) and a systematic review that triangulated data from surveys and observations indicates that estimates based on surveys are likely underestimating the widespread incidence of QRPs among researchers (12). Yet, despite QRPs being more common than research misconduct, many RECR education efforts focus mainly on research misconduct.

One of the commonly used tools in RECR education is discussion of case studies (13, 14). However, available case studies rarely feature undergraduate students as researchers and typically focus on complex RECR issues that might not be encountered in the CURE classroom. To be able to assess undergraduate students’ preparedness for RECR issues they might encounter in their CUREs we developed a survey using case studies developed in-house that feature undergraduate students confronting potential RECR issues that could occur in the context of a Biology CURE.

Intended audience

The intended audience of this diagnostic survey is the instructors and students in undergraduate biology courses, particularly those in CUREs. While this diagnostic survey would likely be most useful for instructors who wish to assess students’ perceptions of RECR topics, discussing the results of the survey with the students may also be useful for students to reflect on their own RECR reasoning abilities and decision-making. The case studies in this diagnostic tool are designed to present scenarios that an undergraduate student may encounter in a biology CURE focused on bacteriophage research, but could reasonably be adapted to suit other research topics.

Learning time

The approximate time required for students to take this survey is 20 min. Analysis of the quantitative data generated by the survey takes approximately 30 min. Qualitative analysis of student responses to the open-ended questions will take a variable amount of time depending on the number of students, the length of student responses, and the approach of the analysis.

Prerequisite student knowledge

As this diagnostic survey is designed to understand students’ perceptions of RECR, there is no prerequisite knowledge or skill requirements for the students. The diagnostic survey was designed for undergraduate biology students and was successfully deployed in both a first-year biology CURE and a senior biology course. The case studies used in the survey are set in a CURE lab where students conduct bacteriophage research and therefore refer to some of the methods used in this type of research. However, the cases were specifically designed to be understood by first-year biology students who have not taken the CURE yet.

Learning objectives

The objectives for this activity are:

1. Assess students’ ability to identify RECR violations of different severities and the consequences of such violations.

2. Analyze student responses to RECR violations to identify gaps in RECR knowledge/skills to be addressed.

PROCEDURE

Material development

Our goal was to develop a diagnostic survey using case studies that were appropriate for undergraduate students taking first-year biology courses, the cases were developed in-house by two undergraduate students who had already taken the bacteriophage lab (SB and JI), and modified by LADM to ensure conciseness and readability. The survey was distributed electronically using Qualtrics. It began with a consent form that described the project and informed potential subjects that the survey was voluntary and only open to those 18 years old or older. Those subjects who consented to participate were taken to the full survey. Participants who declined to participate or who were not at least 18 years old were directed to a thank you page. The survey consisted of 32 items divided into four sections. The first section included questions regarding demographic and academic information, such as gender, academic major, and year of studies. The second section asked the students to rank how strongly they identify with their major and as a STEM professional using a single-item STEM professional identity measure (15). The third section presented a case study on a QRP in which the characters failed to record the exact location where their environmental samples were taken in their laboratory notebook. After reading the case, students responded by ranking the ethicality of the QRP on a numerical scale from 0 to 10 where 0 was “very unethical” and 10 was “very ethical”, and were asked to provide a written explanation for their ranking. The fourth section contained the same setup as the first case study but featured a case study regarding research misconduct in which the characters falsified the results of an experiment. The full survey with the case studies is provided in appendix 1.

Materials required

The diagnostic survey can be administered electronically or on paper. The instructor will need access to quantitative analysis software (e.g., Excel and JMP). Qualitative analysis of open-ended responses can be performed using specialized software (e.g., QDA-Miner) or manually.

Student instructions

The only instructions needed for this activity are for the instructor to introduce the activity and provide the survey (electronically or on paper), other instructions are provided in the survey itself.

Faculty instructions

The full survey containing the two case scenarios and all demographic questions is provided in appendix 1. The survey can be customized to fit the instructor’s needs. For example, if the survey is going to be used for in-house assessment of RECR it can be shortened to include only the questions related to the case studies (questions 23–32). The codebooks we generated based on the student responses we received are provided in appendix 2 (Tables S1 to S3). The codebooks were generated by analyzing the responses to open-ended questions using a descriptive interpretative approach (16). A series of cycles of iterative inductive coding were performed in order to generate a codebook for each open-ended question. These codebooks can potentially be used by instructors to analyze student responses. However, the codes might vary in different student populations; thus, instructors are encouraged to identify the codes and generate their own codebook based on their students’ responses in a similar fashion.

Suggestions for determining student learning

The diagnostic tool can be used as a pre-post survey to assess changes in student perceptions after implementation of a RECR intervention.

Sample data

Sample data are provided in the discussion under the Evidence of student learning section.

Safety issues

There are no safety concerns associated with this activity. This project was reviewed and approved by the Institutional Review Board (IRB; Protocol# 2101DIABIO).

DISCUSSION

Field testing

The participants selected for this survey were biology majors enrolled in either an introductory course or an upper-division biology course. The introductory course selected was a CURE modeled after the SEA-PHAGES program (17). Twelve CURE course sections were invited to participate. Six were virtual in Spring 2021, and six were in person in Fall 2021. Classes were visited within the first 2 weeks of their corresponding semesters to minimize participant exposure to research prior to taking the survey. The introductory biology CURE is a service course and therefore many of the respondents were not Biology majors. The focus of our project was on biology majors and thus this was the subset of students studied, but the diagnostic survey can also be used to assess students from other majors who are participating in a Biology CURE. The upper division biology course selected is required for all biology majors (two sections in the Spring of 2021 and two sections in the Fall of 2021). Students enrolled in this course were required to take the CURE course as a prerequisite. As a result, all participants were enrolled in or have had a CURE experience prior to this survey. Respondents (N = 262) were surveyed by visiting these classes remotely via zoom during the Spring of 2021 (n = 138) and in-person during the Fall of 2021 (n = 124). Seventeen responses were removed because the survey was less than 50% complete or the subjects did not complete the open-response questions. From the subset of the complete responses (n = 245) the responses from students who indicated belonging to a biology major were selected (n = 136). Students were informed about the research project and then asked to voluntarily take part in the survey. The researcher remained in the room throughout the duration of the survey for any questions that arose. Participation for all subjects was voluntary and none of the participants received any compensation for their participation in this study.

Evidence of student learning

The objectives of this project were (i) to assess students’ ability to identify RECR violations of different severities and the consequences of such violations and (ii) to identify gaps in RECR knowledge/skills to be addressed. Students first read a case study on a QRP in which the characters failed to record the exact location where their environmental samples were taken in their laboratory notebook. After reading the case, students responded by scoring the ethicality of the characters’ actions on a numerical scale from 0 to 10 where 0 was “very unethical” and 10 was “very ethical,” and were asked to provide a written explanation for their ranking. The students then read a case study regarding research misconduct in which the characters, Ariel and Jordan, falsified the results of an experiment and were similarly asked to provide an ethicality score and reasoning for their score. As expected, students ranked the research misconduct as a highly unethical action (Fig. 1A, light bars), while the QRP case had a broader distribution of scores with a mode score of 5 (Fig. 1A, dark bars). These results indicate that the students were able to identify both cases as violations of RECR standards and score the ethicality according to the severity of the violation.

Fig 1.

Ethicality rating and major themes in undergraduate biology student responses to two RECR cases. (A) Ethical rating score given by undergraduate biology students to a case study involving a questionable research practice (QRP, dark gray) and a case study on research misconduct (light gray). The ethical scores ranged from 0 = very unethical to 10 = very ethical. (B) Quantitative analysis of the main themes identified in the open-ended responses describing the reasoning for the ethical rating given by the students for the QRP case (dark gray) and the misconduct case (light gray). Bars represent the percentage of subjects who included each theme in their response. (C) Quantitative analysis of the main themes identified in the open-ended responses by students when asked what the characters should do next in the research misconduct case. (D) Quantitative analysis of the main themes identified in the open-ended responses by students when asked who could potentially be affected by the actions of the characters in the research misconduct case.

Analysis of the student responses and codes revealed that one of the main themes in students’ responses was attempts to justify the characters’ actions. A majority of students (55.1%) tended to justify the characters’ actions in the QRP case by focusing on the fact that the characters made an effort to rectify the issue, even if that effort resulted in recording information that might not have been accurate. Another 25.7% of students justified the characters’ actions by minimizing the severity of the issue. In contrast, the tendency to justify the characters’ actions was almost absent in the research misconduct case. Sample comments to exemplify the students’ reasoning for their score in response to the QRP case are provided in Table 1 and sample comments in response to the misconduct case are provided in Table 2. Other themes identified in the students’ responses to both cases were attempts to classify or recognize the type of violation being presented in the cases (theme “Recognizes as” in Fig. 1B) and mentioning potential consequences or actions (theme “Consequences or actions” in Fig. 1B). Analysis of the student responses revealed that some of the students (47.8%) were able to identify the falsification case correctly as a case of research misconduct and specifically used this terminology, indicating some awareness of the correct RECR terminology. In addition, although none of the students used the term QRP, some students (41.9%) were able to articulate the idea that this was an RECR issue. In contrast, 19.9% of students mistakenly identify the falsification case as academic dishonesty rather than as a RECR violation.

TABLE 1.

Major themes identified in student responses to the question “Briefly describe your reasoning for selecting that score” when asked about the ethicality of the QRP case study

Themes and codes	Number and percentage of cases coded n = 136
Major Theme 1: Justification—Comments that the subjects made to justify the actions of the characters or minimize the severity of the issue.
Good Effort	75 (55.1%)
Sample Response: “I think they put forth a solid effort to retrace their steps remotely.”
No Malicious Intent	8 (5.9%)
Sample Response: “I don’t think this was an intentional lie.”
External Pressures	2 (1.5%)
Sample Response: “It does not seem completely correct to do that, but if not they will have to completely restart the lab.”
Purpose of the Data	6 (4.4%)
Sample Response: “For the scope of the assignment I would not classify their actions as ‘unethical.’”
Not a Severe Issue	35 (25.7%)
Sample Response: “The study is not a life or death situation.”
Major Theme 2: Recognizing ethical issues—Subject comments on the nature of the violation.
Recognizing as QRP	57 (41.9%)
Sample Response: “Those coordinates were not accurate.”
Recognizing as Research Misconduct	18 (13.2%)
Sample Response: “The students have to one degree or another, falsified data.”
Recognizing as Academic Honesty Issue	3 (2.2%)
Sample Response: “They are cheating by definition.”
Recognizing as Ethical Issue but not Research/Academic	10 (7.4%)
Sample Response: “By giving the specific coordinates it is unethical and not exactly truthful.”
Major Theme 3: Identifying Potential Solutions and Consequences
Communicating to Instructor	33 (24.3%)
Sample Response: “They did not inform the professor of the mistake.”
Unbiased Reporting of Procedure	32 (23.5%)
Sample Response: “They could mark in their notebook that they went back to find the coordinates.”
Repeat Experiment	8 (5.9%)
Sample Response: “They should have redone the experiment completely.”
Identifying consequences to science integrity/process of science	30 (22.1%)
Sample Response: “It is definitely bad science, and puts the legitimacy of their research into question.”
Major Theme 4: Nature of Science
Nature of Science	6 (4.4%)
Sample Response: “I think most of science are [is] predictions and approximations.”

TABLE 2.

Student responses to the question “Briefly describe your reasoning for selecting that score” when asked about the ethicality of the misconduct case study

Themes and codes	Number and percentage of cases coded n = 136
Major Theme 1: Justification—Comments that the subjects made to justify the actions of the characters or minimize the severity of the issue.
Good Effort	5 (3.7%)
Sample Response: “They were using the best information they had in place of the failed third purification.”
No Malicious Intent	2 (1.5%)
Sample Response: “…the students may not have been intentionally trying to cheat.”
Purpose of the Data	1 (0.7%)
Sample Response: “Since this is a school project it is not hurting anyone that much.”
External Pressures	18 (13.2%)
Sample Response: “The motivation, while wrong, is somewhat understandable in an academic environment that focuses so heavily on grades.”
Not a Severe Issue	10 (7.4%)
Sample Response: “No one is hurt in this situation.”
Major Theme 2: Recognizing ethical issues—Subject comments on specific actions or issues.
Recognizing as QRP	10 (7.4%)
Sample Response: “…they did not correctly follow the instructions.”
Recognizing as Research Misconduct	65 (47.8%)
Sample Response: “They falsified their results and therefore it is not ethical.”
Recognizing as Academic Dishonesty	27 (19.9%)
Sample Response: “I think that this [is] in clear violation of GU’s code of ethics on cheating.”
Recognizing as Ethical Issue but not Research/Academic	14 (10.3%)
Sample Response: “… just genuinely is dishonest in their approach to the problem period”
Major Theme 3: Identifying Potential Actions and Consequences
Communicating to Instructor	42 (30.9%)
Sample Response: “They should have informed the professor of their situation.”
Unbiased Reporting of Procedure	27 (19.9%)
Sample Response: “I feel that they could have stated that their third purification did not work, and instead labelled everything correctly.”
Repeat Experiment	11 (8.1%)
Sample Response: “They should have redone the purification.”
Identifying consequences to science integrity/process of science	29 (21.3%)
Sample Response: “This behavior undermines the validity of their experiment and it [is] a horrible practice to have in scientific discovery.”
Major Theme 4: Nature of Science
Nature of Science	26 (19.1%)
Sample Response: “Failure of experiments is a very common thing with research and is normal.”

After scoring the ethicality of the falsification case and providing their rationale for the score, the students were asked “What do you think Ariel & Jordan should do next?” The most frequent action listed by the students (64% of students) was that the characters should communicate with their instructor (Fig. 1C; Table 3) and 47.8% commented on the potential consequences to science due to the falsification. When specifically prompted to identify who could be affected by the RECR violation, a majority of the students (70.6%) identified other researchers as stakeholders but only a minority identified the potential consequences to other stakeholders and to society as a whole (Fig. 1D; Table 4).

TABLE 3.

Student responses to the question “What do you think Ariel and Jordan Should do next? Explain your reasoning,” in the context of research misconduct

Themes and codes	Number and percentage of cases coded n = 136
Major Theme 1: Actions—Comments on what Ariel and Jordan should do next.
Communicate with Instructor/Institution	87 (64.0%)
Sample Response: “They should tell their professor that the data they submitted was not genuine.”
Withdraw Sample	15 (11.0%)
Sample Response: “...get their samples terminated from the database.”
Acknowledge Issue (No Teacher, In General)	38 (27.9%)
Sample Response: “Ariel and Jordan should be honest about their mistakes.”
Major Theme 2: Recognizing Ethical Issues
Addresses Issue as QRP	27 (19.9%)
Sample Response: “...say their data are incorrect.”
Addresses Issue as Ethical Issue	34 (25.0%)
Sample Response: “... ethical duty to come clean.”
Addresses Issue as Misconduct	27 (19.9%)
Sample Response: “They falsified their results.”
Addresses Issue as Mistake/Blameless	37 (27.2%)
Sample Response: “Due to mistakes made during the research.”
Major Theme 3: Identifying Consequences—Identifying possible consequences of Ariel and Jordan’s actions.
Potential Consequences to Science	65 (47.8%)
Sample Response: “...could have implications on how researchers continue to do their research.”
Scale of Effect	19 (14.0%)
Sample Response: “The importance of these results has been amplified from the previous example.”

TABLE 4.

Student Responses to the question of “^a[...]who, if anyone, could potentially be affected by their decision?”

Themes and codes	Number and percentage of cases coded n = 136
Major Theme 1: Identifying Affected Groups—Comments made by the subjects that identify the groups affected by the ethical issue.
Instructor	46 (33.8%)
Home Institution	53 (39.0%)
Classmates	9 (6.6%)
Themselves	36 (26.5%)
Receiving Institution	41 (30.1%)
Other Researchers/Students	96 (70.6%)
Society	38 (27.9%)
Major Theme 2: How They are Affected—Comments made by the subjects that identify the ramifications for the affected groups if the students are not truthful about the ethical issue.
Future Research	41 (30.1%)
Sample Response: “If researchers assume the results to be accurate, their research could be endangered.”
Using Bad Data	47 (34.6%)
Sample Response: “They gave faulty evidence towards the presence of phages, and therefore are affecting everyone that depends on that information being accurate.”
Unaffected	4 (2.9%)
Sample Response: “...it won’t be used in the science, and doesn't harm anyone.”
Reputation	32 (23.5%)
Sample Response: “They would look incompetent, and dishonest.”
Liability	12 (8.8%)
Sample Response: “...face consequences such as possible expulsion from their departments for falsifying their information.”
Health/Wellbeing of Society	14 (10.3%)
Sample Response: “If their phages are selected for medical use, it could be an issue for anyone who needs the phages for whatever medical reason.”

^a The full question reads: “Assume that Ariel and Jordan decide to do nothing. Who, if anyone, could potentially be affected by their decision? List as many individuals/groups that could be affected as you can think of, and briefly explain how each individual/group could be affected”

This analysis allowed us to determine that some biology undergraduates are already aware of some RECR concepts such as falsification and research misconduct, but that education on other RECR terminology (e.g., QRPs) and a deeper understanding of the importance of RECR for research integrity is needed. Importantly, our survey showed that a fraction of students equates research integrity with academic honesty. The reason for this is unclear. It could be because academic honesty is a concept they are more familiar with or perhaps because the research in the case scenario occurred in a laboratory classroom. Being able to distinguish between RECR and academic honesty is particularly important in the context of CUREs to ensure that students do not conflate research results with course outcomes as the focus on grades could be converted into an additional pressure leading to RECR violations if the students do not distinguish between the research and academic aspects of the course. Overall, the diagnostic survey allowed us to achieve our goals of assessing students’ ability to identify RECR violations of different severities and to identify gaps in RECR knowledge/skills that need to be addressed.

Possible modifications

The use of case scenarios in a diagnostic survey allowed us to obtain rich information about students’ RECR knowledge and reasoning in response to two specific RECR violations. The cases can be modified to address specific RECR violations that the instructors are interested in assessing. In addition, the survey can also be used as a tool to assess the effectiveness of RECR educational interventions if it is administered in a pre-post fashion.

SUPPLEMENTAL MATERIAL

The following material is available online at https://doi.org/10.1128/jmbe.00119-23.

Appendices. jmbe.00119-23-s0001.docx.

Appendices 1 and 2 containing the full survey and code tables.

ASM does not own the copyrights to Supplemental Material that may be linked to, or accessed through, an article. The authors have granted ASM a non-exclusive, world-wide license to publish the Supplemental Material files. Please contact the corresponding author directly for reuse.