Improving Scientific Research and Writing Skills through Peer Review and Empirical Group Learning

Abstract

Here we describe a semester-long, multipart activity called “Read and wRite to reveal the Research process” (R³) that was designed to teach students the elements of a scientific research paper. We implemented R³ in an advanced immunology course. In R³, we paralleled the activities of reading, discussion, and presentation of relevant immunology work from primary research papers with student writing, discussion, and presentation of their own lab findings. We used reading, discussing, and writing activities to introduce students to the rationale for basic components of a scientific research paper, the method of composing a scientific paper, and the applications of course content to scientific research. As a final part of R³, students worked collaboratively to construct a Group Research Paper that reported on a hypothesis-driven research project, followed by a peer review activity that mimicked the last stage of the scientific publishing process. Assessment of student learning revealed a statistically significant gain in student performance on writing in the style of a research paper from the start of the semester to the end of the semester.

INTRODUCTION

It is important for students majoring in science to develop basic skills in reading and writing scientific literature and to have a meaningful understanding of the research process as practiced by scientists. The aim of this semester-long, multipart activity, “Read and wRite to reveal the Research process” (R³), was to engage students in reading primary scientific literature, writing scientific reports, and conducting research with the goal of increased understanding of the scientific research process in an authentic context. We linked the reading of primary research articles with scientific writing, data reporting, and the research conducting process, and then connected the students’ course experience to the scientific research process that addresses authentic research questions. R³ was delivered in three sequential stages: Stage 1: Understanding research paper components, Stage 2: Learning to write a research paper, and Stage 3: Practicing scientific research and writing (Fig. 1). The activities inherent in the design allowed students to develop and practice combined skills in scientific literature reading, research conducting, data processing and presenting, scientific writing, peer review, and working in groups.

FIG. 1.

A flow chart for the three stages of R³.

The R³ could be applied to any science course where the goal is to increase student understanding of the research process and improve students’ writing skills through reading and discussion of primary literature. Here we present the application of R³ to an immunology lab course.

Intended audience

R³ is appropriate for application in an advanced undergraduate science course. Here we describe the application to an upper-level immunology laboratory course that was designed for microbiology and biology majors.

Learning and preparation time

In the immunology course, R³ was implemented over a semester (14 weeks). Lab sessions met twice each week for two hours. R³ was implemented in three stages (Fig. 1):

• Stage 1 (Week 1-3): Understanding research paper components

• Stage 2 (Week 4-9): Learning to write a research paper

• Stage 3 (Week 9-14): Practicing scientific research and writing

During Stage 1 and Stage 2, approximately twenty minutes was allowed in each class period for discussion of research papers and student presentations (Instructor Version). In Stage 3, all in-class time was devoted to practicing scientific research and writing; students planned, implemented group research projects, and wrote their final Group Research Paper. The last full class meeting was devoted to peer review.

Students also were required to complete work out of class for each stage. In Stage 1, students read Paper#1 in stages, composed responses to questions and wrote individual “Introduction,” “Methods/Results,” and “Discussion” sections (“Guide and Questions” for “Introduction” section, “Methods and Results” section, and “Discussion” Section in Appendix 2, 3, and 4). In Stage 2, students read Paper #2, prepared for discussion of “Introduction” section, analysis and oral presentation of selected data from Research Paper #2 (Appendix 5), discussion of “Discussion” section, and wrote four complete lab reports. In Stage 3, students worked individually to prepare for the Group Research Project and then with their groups to complete the Group Research Paper (Appendix 6).

Instructor preparation

We have provided research papers, student materials, lab project scenarios, and grading rubrics used in the implementation of R³ in an immunology course. Project scenarios for Stage 3 were derived from primary research papers and were written based on the “Introduction” sections of these research articles (12). To apply R³ to another course, it will be necessary for an instructor to choose research papers, compose discussion questions, and develop lab project scenarios, based on the course subject. Using our work as a guide, we expect this will take about one week of effort.

Assigning students to groups

Allow approximately one hour to assign students into groups using information from student-completed Group Building Questionnaires.

Grading

One graduate student teaching assistant (TA) was responsible for grading of all assignments for one lab section of students (18–20 students). Using specifically designed rubrics (Materials), grading of student assignments required 1–4 hours, depending on the length of the assignment. For courses with an enrollment of 20 students or less, it is expected that the course instructor should be able to manage the grading.

Prerequisite student knowledge

For our application to an immunology lab course, students were expected to have general knowledge of microbiology, cell biology, and genetics, and required to have an immunology lecture course as a pre- or corequisite.

Learning objectives

R³ was part of a larger project involving the development of research-oriented learning activities (ROLA) in Host-Pathogen Interaction undergraduate courses (NSF DUE 0837515). ROLA development involved collaboration between faculty members of the Host Pathogen Interactions (HPI) teaching community (7, 8) and selected research-active graduate students serving as HPI teaching fellows. For each ROLA, the faculty research was used as the inspiration or model system for the design. Development was approached using the Backward Design method (4, 13), where learning goals and assessments were first established and then activities developed to meet the goals. The design of these activities was meant to help students develop higher-order thinking (1, 9), a meaningful understanding of the process and the relevance of science (14), and cultivate skills in research and communication. In order to accomplish these goals, R³ established the following learning goals. At the completion of the R³ course students will:

1. Understand scientific research as illustrated in scientific research papers.

2. Understand the steps for writing and publishing research results.

3. Be able to process and present scientific data in an appropriate scientific fashion.

4. Be able to articulate the role of collaboration in the research process.

5. Understand how scientific techniques are employed in scientific research.

6. Be able to present scientific information in oral and written fashion using appropriate terminology.

We have set forth learning outcomes (Table 1) to measure the progress that student make toward these goals.

TABLE 1.

Use of Backward Design to develop R³. Learning outcomes were established to meet the overall goals of the course design, followed by the development of assessment measures and student activities. Activities engaged students in reading and writing to reveal the scientific research process.

Learning Outcomes Students will be able to:	Learning Goals	Activities	Assessment
Explain the purpose of each section of a published research paper and relate the design to that of a lab report.	1, 2	-Guided Reading -Guided Discussion -Guided Writing of each section of a research paper using Research Paper #1	Introduction Rubric Methods/Results Rubric Discussion Rubric ROLA post-assessment
Read a scientific research paper and report on the value of each section in achieving the overall goal of the paper.	1, 2, 3, 6	-Research paper #2 In-class discussion -Oral Presentations of Research paper#2 results section	Oral Presentation Rubric
Process raw data (generated in lab or provided) into graphs and figures appropriate for presentation.	3, 6	- Discussion of how to process raw data - Process and present data generated in lab in lab reports and Group Research paper as graphs and figures.	Lab Report Rubric Group Research Paper Rubric
Write lab reports in the same format as published research papers.	2, 3, 6	- Writing Lab reports 1–4 - Writing Group research paper	Lab Report Rubric Group Research Paper Rubric
Identify a research question and propose a hypothesis and appropriate experiments to address the research question.	2, 5	-Group research project	Lab Report Rubric Group Research Paper Rubric
Write a group research paper in the style of a scientific research article.	1,2,3,4,5,6	-Group Research Paper	Group Research Paper Rubric
Work collaboratively.	4	- Writing of the Group Research Paper -Group discussions in class	ROLA post-assessment
Peer review a student research paper.	2, 6	- Peer Review Assignment	Peer Review Rubric

PROCEDURE

R³ used primary scientific literature to engage students in learning scientific writing and research skills. Students completed reading, discussion (within a group of 4–5 and within a lab section of 20 students), and individual and collaborative writing activities connected to the primary research articles while learning immunology concepts from lectures and lab exercises. The student activities were completed in three stages where the scope and depth of the student work increased incrementally over the semester: Stage 1: Understanding research paper components; Stage 2: Learning to write a research paper; and Stage 3: Practicing scientific research and writing (Table 2).

TABLE 2.

The organization of R³. R³ consists of three stages with paired reading and writing assignments, which are coordinated with in-class and in-group discussion and lab activities. See text in “Three Stages of Implementation” (Appendix 19) for description of Stages and Parts.

Period	Out of Class Activities	In-class Activities	Lab Activities
Stage 1. Understanding research paper components.
Week 1	- Read the introduction of the research paper #1 by Krishnamurthy, et al. (6) (Stage 1 Part a) - Write Introduction section for Lab 2 (Stage 1 Part c)	- Guided Discussion of Introduction Section (Stage 1 Part b)	- Lab 1. Identification of Immune Organs and Cells - Lab 2. Antibody-Antigen precipitation (Ouchterlony Test)
Week 2	- Read the methods and result sections of the paper 1 - Write the methods and result sections for Lab 3	- Guided Discussion of Methods/Results Sections	- Lab 3. Agglutination
Week 3	- Read the discussion section of the paper 1 - Write the result and discussion sections for Lab 4	- Guided Discussion of Discussion Section	- Lab 4. Immunofluorescence
Stage 2. Learn to write research paper.
Week 4–6	- Read the introduction of the research paper #2 by Paccani et al. (11) - Write a full report for Labs 5–6 (Stage 1 Part e)	- Identify the research questions in research paper #2 (Stage 2 Part a)	- Lab 5. Immunoprecipitation - Lab 6. Western blot
Week 7–8	- Read the result section of the research paper 2 - Write full reports for Labs 7 and 8 (Stage 1 Part e)	- Guided Analysis and Data Presentation (Stage 2 Part b) - Discussion on how to process and present ELISA data in scientific context (Stage 2 Part d)	- Lab 7. ELISA - Lab 8. Microbial killing by macrophages
Week 9	- Read the discussion section of the paper 2 - Write a full lab report for Lab 9 (Stage 1 Part e)	- In class discussion to summarize the results and conclusions of research paper #2 (Stage 2 part c) - Discussion on how to present flow cytometry data in scientific context (Stage 2 Part d)	- Lab 9. Flow cytometry
Stage 3. Practice scientific research and writing.
Week 9	- Propose a scientific question and a hypothesis based on group scenario chosen - Identify research methods to address the question (Stage 3 Part a)	- Read research project scenarios and choose project for group
Week 10	- Propose experimental procedures (Stage 3 Part b)	- Discuss the scientific question, hypothesis, and experimental approaches in groups - Plan experiments in groups	- Lab 10. Complement mediated cytolysis
Week 11–13	- Write Group Research Report (Stage 3 Part a)	- Write Group Research Report in groups	- Run experiments
Week 14	- Modify Group Research Report based on comments from peer review	- Peer review of Group Research Reports (Stage 3 Part c) - Peer review of contribution by individuals in a group
Week 14	- Submit Group Research Report

Materials

Sources of primary literature

All open access immunology research journals, particularly Infection & Immunity and Journal of Experimental Medicine.

Instruction documents

Stage 1:

• Group Building Questionnaire (Appendix 1)

• Guide and Questions: Introduction section (Appendix 2)

• Guide and Questions: Methods and results (MR) section (Appendix 3)

• Guide and Questions: Discussion section (Appendix 4)

Stage 2:

• Guide for Analysis and Presentation of Research Results (Appendix 5)

Stage 3:

• Guide for Group Research Paper (Appendix 6)

Assessment measures

Stage 1:

• Introduction Rubric (Appendix 7)

• MR Rubric (Appendix 8)

• Discussion Rubric (Appendix 9)

Stage 2:

• Lab Report Rubric (Appendix 10)

• Analysis and Presentation of Research Results Rubric (Appendix 11)

Stage 3:

• Peer Review Rubric of Group Research Paper (Appendix 12)

• Peer Review of individual contribution in a group (Appendix 13)

• Group Research Paper Rubric (Appendix 14)

• Assessments of student perceptions on the course:

• ROLA pre-assessment (Appendix 15)

• ROLA post-assessment (Appendix 16)

Student instructions

R³ was carried out in three stages of learning in the immunology lab course as depicted in Table 2. Written instructions are found in Appendix 2–6.

Instructor version

Distribution of materials

For distributing course materials, we used the Online University Learning Management System. Alternatively, materials could be emailed to students or distributed in class.

Group design

R³ incorporated group work into student activities. We divided students into groups of 4–5 individuals. This size was selected as it falls in the range of effective group size for collaborative learning (2, 5) and was related to the total number of students in each section of the class. To maximize diversity, students were asked to complete the Group Building Questionnaire (Appendix 1), and were divided into groups with diversity in their majors, previously completed courses, previous research experience, and career goals. A participation grade was built into the overall semester grade. Students were reminded throughout the semester about this grade and that poor participation would negatively impact their grade. Throughout the semester, TAs monitored student participation during class discussions and observed groups during lab activities. TAs documented evidence of students not participating in discussions or not contributing to the lab activities; they tracked students who consistently did not pay attention or were disruptive during discussions. At the end of the semester, TAs ranked each student’s participation as “Exceptional,” “Acceptable,” or “Poor.” Also at the end of the semester, students anonymously rated fellow group members on contributions to the group research paper and to lab activities (Appendix 13) on a scale of 1–10, with 10 indicating a significant contribution and 1 indicating little to no contribution to the group. When determining the final participation grade, we decided that students receiving a combination score of less than 5 from their group members and a TA participation grade of “Poor” received a 20% reduction in their overall participation grade (See Table 3).

TABLE 3.

Grading. The assignments in R³ are worth a total of 300 points. The assignments in Stage 1 were designed to be “low stakes” assignments while the assignments in Stage 2 and 3 are considered more “high stakes” assignments. “Type of assignment” refers to individual or group work. In the case of group work, all members of the group received the same grade.

Assignment	Type of Assignment	Total Points
STAGE 1 (“low stakes” assignments)
Introduction Section: Guided Reading Assignment	Individual	5
Introduction Section: Guided Writing Assignment	Individual	10
MR Section: Guided Reading Assignment	Individual	5
MR Section: Guided Writing Assignment	Individual	10
Discussion Section: Guided Reading Assignment	Individual	5
Discussion Section: Guided Writing Assignment	Individual	10
STAGE 2
Analysis and Presentation of Research Results Oral Presentation	Group	20
Lab Report #1	Individual	50
Lab Report #2	Individual	25
Lab Report #3	Individual	25
Lab Report #4	Individual	25
STAGE 3
Group Research Project: Identify Hypothesis and Experimental Methods	Individual	10
Group Research Project: Outline Experimental Procedures	Individual	30
Group Research Paper	Group	40
Peer Review	Group	10
Participation (over entire semester)	Individual	20
TOTAL		300

Pre- and postassessment

In the first and last class periods, students completed ROLA pre- and ROLA postassessments (Appendix 15 and 16).

Three stages of implementation

For a more detailed description of each stage of R³, see Appendix 19 and Table 2.

Stage 1: Understanding research paper components. Research Paper #1 sections of “Introduction,” “Methods and Results,” and “Discussion” were introduced separately and sequentially using the same three-step teaching strategy: guided reading, guided discussion, and guided writing.

• Part a: Guided Reading of Research Paper #1 “Introduction”

• Part b: Guided Discussion of Research Paper #1 “Introduction”

• Part c: Guided Writing of Introduction Section

Stage 2: Learning to write a research paper. In Stage 2 students read Research Paper #2 and applied insights from Stage 1. The “Results” section of Research Paper #2 was analyzed and discussed in detail. Furthermore, students worked directly on data processing and presenting techniques using their lab generated data and then reported their work by writing four lab reports.

• Part a: Guided Discussion of Research Paper #2 “Introduction”

• Part b: Guided Analysis and Presentation of Research Results

• Part c: Guided Discussion of Research Paper #2 “Discussion”

• Part d: Guided Data Processing and Presentation

• Part e: Writing a Full Lab Report

Stage 3: Practicing scientific research and writing. In the third stage of R³, student groups performed a Group Research Project that culminated in writing the Group Research Paper using the skills developed in Stage 1 and Stage 2.

• Part a: Performing Hypothesis-based Group Research Project

• Part b: Writing the Group Research Paper

• Part c: Guided Peer Review of Group Research Papers

Grading

Instructors and TAs graded written assignments using rubrics (Appendix 7–10 and 14). When a major deficiency was discovered during grading, it was discussed during a TA meeting, and an instructional strategy was proposed and carried out to address the deficiency in the following week. All of the assignments in the R³ application discussed here were worth a total of 300 points. The breakdown of points awarded for each assignment is described in Table 3.

Suggestions for determining student learning

Students’ improvements in writing were assessed during each of the three stages using the “Introduction Rubric” (Appendix 7), “MR Rubric” (Appendix 8), “Discussion Rubric” (Appendix 9), “Lab Report Rubric” (Appendix 10), and “Group Research Paper Rubric” (Appendix 14). Student learning can be further assessed by the analysis and presentation of research results (Appendix 11) and the ROLA pre- and postassessments (Appendix 15 and 16).

Sample data

An example of one “Group Research Paper” has been included, with instructor comments (Appendix 17). Peer reviews of the first draft of this “Group Research Paper” are also included (Appendix 18).

Safety issues

Safety issues relate to the specific lab protocols are not relevant to the course design as we describe here.

DISCUSSION

Field testing

We implemented R³ in an immunology lab course, which was a 400-level, two-credit course that required junior or senior standing. The immunology lab course was comprised of 35 seniors (18 females, 17 males) and split between two lab sections. The average GPA was 3.2, and all students were science majors.

A faculty member who was active in the immunological research was the instructor of the course. Graduate students who worked in immunology and immunology-related fields served as Teaching Assistants for the lab sessions. TAs met with the faculty instructor each week to review and discuss upcoming goals, assignments, grading, and lab activities for the week.

Evidence of student learning

Student learning was assessed at each stage of the project via a set of rubrics (Appendix 7–11, 14) and ROLA pre- and postassessments (Appendix 15, 16).

Goals 1-6 all related to students’ gains in understanding of the research process. The analysis of student work on the “Group Research Paper” and the peer review of that paper revealed student gains in all six learning goals. Assessment using the “Group Research Paper Rubric” revealed student’s understanding of the content and context of each section of a research paper indicating an understanding of scientific research as illustrated in scientific research papers (Goal 1). The success in completion of the Group Research Project and Report demonstrates enhanced ability of students to collaborate (Goal 4), to write scientifically (Goal 6), to apply scientific techniques to the scientific research process (Goal 5), and to process and present data (Goal 3). Students gained an understanding of the steps for writing and publishing research results as they constructed their own paper, peer reviewed other student’s papers, and then used the insight from the peer review process and comments from other students to rewrite their own final paper (Goal 6). Student’s success in completing the final group research paper was revealed in student scores as assessed by the Lab Report Rubric (Table 4). The average grade on the Group Research Paper was 37.96/40, or 96%. An example of a “Group Research Paper” is included (Appendix 17). The grade breakdown for this paper is “Group 1” in Table 4. Instructor’s comments during the grading of this paper are included in the document.

TABLE 4.

Assessment of Group Research Reports. Each group of 4-5 students wrote a Group Research Report based the hypothesis-driven research project. The reports were graded using the rubric (Appendix 14). All the major sections of the report and total points that can be earned in each section are listed in the top row. The total points and the points of individual sections for each group are listed.

Group	Title 1 Point	Abstract 3 Points	Introduction 9 Points	Methods 6 Points	Results 10 Points	Discussion 8 Points	References 3 Points	Total 40 Points
1	1.00	3.00	7.25	6.00	10.00	8.00	3.00	38.25
2	1.00	3.00	9.00	6.00	10.00	8.00	2.00	39.00
3	1.00	3.00	9.00	6.00	8.00	8.00	3.00	38.00
4	1.00	2.00	8.50	6.00	6.00	6.00	3.00	33.50
5	1.00	3.00	9.00	6.00	10.00	8.00	3.00	40.00
6	1.00	3.00	8.00	6.00	10.00	8.00	3.00	39.00
Average	1.00	2.83	8.45	6.00	9.00	7.66	2.83	37.96
SD	0.00	0.37	0.71	0.00	1.53	0.81	0.37	2.09

Student learning related to Goals 1–6 was attributed to R³. We compared student’s scores on Stage 1 writing assignments (that is, assignments prior to writing full lab reports) to scores on the final “Group Research Paper.” We compared student performance on the “Introduction,” “Methods/Results,” and “Discussion” writing activities with parallel sections of the “Group Research Paper.” All writing activities were assessed using the same rubric (Appendix 7–9 and 14). We found that student grades in the “Introduction” (p < 0.001) and “Discussion” (p < 0.05) sections in Stage 3 of the “Group Research Papers” were significantly higher than grades on Stage 1 writing activities (Fig. 2). This suggests that students made significant improvements in the writing of the “Introduction” (average grade increased from 87% to 96%), and the “Discussion” section (average grade increased from 85% to 94%), while maintaining a consistent grade on the “Methods/Results” section (average grade increased from 92% to 93%). The “Group Research Project” was more challenging than Stage 1 or Stage 2 activities, as students worked without instructor guidance to perform a set of experiments necessary to address real-world research questions. This was in contrast to Stage 1 and Stage 2 lab assignments where students completed projects that involved one experiment with instructor guidance. Furthermore, the “Group Research Project” required students to work collaboratively in their groups to analyze and interpret information from multiple sets of data, and to synthesize the information into one whole for presentation in a group authored report. The analysis of student learning portrayed in Fig. 2 does not account for the increase in the rigor of learning activities from Stage 1 to Stage 3 and, as such, we suggest this may reflect a minimal level of student development from the R³ experience.

FIG. 2.

Comparison of student writing during Stage 1 to student writing in Stage 3. Student performance (based upon Rubrics Appendix 7-9) on Stage 1 writing assignments: “Introduction,” “Methods/Results,” and “Discussion” were compared with student performance on the similar sections of the Stage 3 Group Research Report graded according to the same rubric (Appendix 14). Shown are average values and standard deviation (n = 27). The increase in student performance from Stage 1 activity to Stage 3 activity was statistically significant for the Introduction section (p < 0.001) and Discussion section (p < 0.05).

One of the major goals for R³ was to engage students in higher-order thinking. Higher-order thinking, defined as the use of cognitive skills, was evaluated using Bloom’s Taxonomy (1). Writing assignments as assessed by the “Lab Report Rubric” required students to collect and organize information into appropriate sections, summarize and appropriately present data in tables and charts, interpret and evaluate data, assess significance of findings, and synthesize information into a readable text meeting the format of a research publication. The “Analysis and Presentation of Research Results” activity as assessed by the “Analysis and Presentation of Research Results Rubric” required students to formulate a presentation using PowerPoint presentation software that illustrated their assessment of the “results” section of a research article. In the peer review process, students evaluated the clarity, presentation, and appropriateness of work presented in students’ group research papers. Each of these activities falls within Bloom’s levels 4–6 that are considered higher-order thinking (3). As indicated by the scores on the end of semester “Group Research Paper,” students were successfully engaged in higher-level thinking.

Student perceptions

At the end of the semester, students completed a postassessment survey (Appendix 16), which allowed them to report by using written prompts on their experience in the course. Our analysis of student perceptions is included in Appendix 20 (10).

Possible modifications

Based on our application of R³ in the context of an immunology lab, we suggest that R³ can be applied to any science course (lab or lecture). Research articles in any scientific field may be used in similar ways to introduce the scientific research process, scientific research paper writing, data analysis, and applications of course content to real-world questions. For a lecture course, we suggest that literature-based research reports may be used to substitute experiment-based lab reports and research reports. Current scientific scenarios and questions could be used as the topics of student writing assignments. Published scientific articles could be modeled for these writing assignments. We recommend that considerable time should be given to the process of choosing appropriate primary research papers prior to instituting this course design. We selected articles that were well-written, described a clear experimental design, and addressed specific immunology content and methods.

We suggest that based on the level and learning goals of a course, any one of the three stages of R³ could be incorporated into a course as an individual module. It is our belief that using research articles to teach scientific reading and writing gives a course an authenticity that allows students to link the classroom experience with research science.

SUPPLEMENTAL MATERIALS

Appendix 1: Group Building Questionnaire

Appendix 2: Guide and Questions: Introduction section

Appendix 3: Guide and Questions: Methods and Results (MR) section

Appendix 4: Guide and Questions: Discussion section

Appendix 5: Guide for Analysis and Presentation of Research Results

Appendix 6: Guide for Group Research Paper

Appendix 7: Introduction Rubric

Appendix 8: MR Rubric

Appendix 9: Discussion Rubric

Appendix 10: Lab Report Rubric

Appendix 11: Analysis and Presentation of Research Results Rubric

Appendix 12: Peer Review Rubric of Group Research Paper

Appendix 13: Peer Review of individual contribution in a group

Appendix 14: Group Research Paper Rubric

Appendix 15: ROLA pre-assessment survey

Appendix 16: ROLA post-assessment survey

Appendix 17: Examples of Group Research Papers

Appendix 18: Examples of Peer Review Assignments

Appendix 19: Instructor Version of Procedure: Three Stages of Implementation

Appendix 20: Analysis of students’ perceptions