Alternative Assessment Strategy and Its Impact on Student Comprehension in an Undergraduate Microbiology Course

Abstract

Medical Microbiology is a content-intensive course that requires a large time commitment from the students. Students are typically biology or prenursing majors, including students headed for professional schools, such as medical school and pharmacy school. This group is somewhat diverse in terms of background science coursework, so it can be difficult to teach in a way that benefits all the students. Numerous changes have been implemented in our microbiology curriculum to address the different abilities of our students by altering assessment and teaching strategies. It was hypothesized that changing the assessment strategy from the traditional scheme of two or three exams and one final to a new model of seven or eight shorter exams would have a positive impact on student comprehension and retention. The quantity of material taught or expected of the students to learn did not change, but there was definitely an impact on them. Although 30.0% of students routinely did not pass microbiology in previous semesters, the new method of assessment resulted in only 9.63% not completing the semester successfully, as determined by earning a grade of C or better. There is some evidence from conversations and interviews with students that indicates a positive impact of this methodology on student attitude. Implementation of these changes in other courses and their current effectiveness will be examined in the future, with an eye towards more broadly applicable successful teaching techniques in the sciences, especially for nonmajors.

Medical Microbiology is the main microbiology class currently offered at Towson University due to budget, space, time, and faculty constraints. For this reason, content must be tailored not only to biology majors, but also to those in the College of Health Professions (especially prenursing and nursing students) who are required to learn the basics of microbiology. The diversity of educational backgrounds of the students makes teaching and assessing student comprehension a difficult proposition. Material cannot be covered in exceptional molecular detail, which would cater to the advanced biology students at the expense of confusing the other students; nor can it be overly simplified, sacrificing the interest of the biology majors for the comprehension of the nonbiology majors. Teaching appropriate content to the audience is critical to learning (1, 4). Therefore, content is directed somewhere between the two extremes. This teaching strategy used to be accompanied by the types of assessment Towson University’s biology majors are trained to expect; that is, the primary means of assessing comprehension were based on three exams and one final exam, plus a single laboratory practical exam at the end of the semester (Table 1a). Unfortunately, this assessment approach frequently, in the instructors’ experience, led to poor grades for nonscience majors.

TABLE 1a.

Previous assessment strategy (before Spring 2003)

Assessment	Frequency	Percentage of final grade
Pretest (assessment of prerequisite knowledge)	Once	5.5
Lecture exams	Once per month (three per semester)	11.1 each; 33.3 total
Final exam	Once, at end of semester	11.1
Streak plate	Needs to be accomplished once during semester	1.1
Lab quizzes (short answer)	At least 6; only count best 5; random pop quizzes	11.1
Unknown ID project	Execution throughout semester; writing final paper during 2-week period	11.1
Lab practical exam	Final exam at end of semester	22.2
Lab citizenship and attendance	Continuous throughout semester	4.4

There is a growing trend toward a reexamination of assessment practices. In 1996, the National Science Education Standards (NSES) discussed their standards of assessments. The NSES, in Teaching Standard C, state that teachers should use multiple methods of assessment and that assessment tasks need to be valid and authentic (4). NSES Teaching Standard C also states that students should have adequate opportunity to demonstrate their achievement and that assessment tasks should lead to similar results if given at different times. The American Association for Higher Education (AAHE) has also looked at assessment and has outlined “Nine Principles of Good Practice for Assessing Student Learning” (http://www.aahe.org/assessment/principl.htm). Two important ideas from this list include developing assessments that measure important and valuable information and assessing student performance as an ongoing process where student growth can be shown. A bulletin from AAHE also discusses fair assessment practices (7). One of the steps indicated here for achieving fair assessment practices is the use of “many different measures and many different kinds of measures” (7, emphasis in original). This sentiment is also echoed by Heady, who states that using different types of assessments is central to how we learn and how we should teach (2). Walvoord and Anderson emphasize that the most important focus for assessment should be on student learning (9).

With these ideas in mind, numerous changes have been implemented in the microbiology curriculum to improve students’: (i) comprehension of the material, (ii) performance on a daily basis and over the course of an entire semester, and (iii) retention of the information in microbiology beyond the time it is examined in class. Primarily, the focus is on improving the learning environment by changing assessment and teaching strategies. Many science courses, especially the upper-level ones, rely heavily on a traditional assessment scheme of two or three exams and one final. A modified assessment strategy was implemented to include seven or eight shorter exams, with the expectation that there would be a positive impact on student comprehension and retention. The present study was started in the Spring 2003 semester to improve student outcomes in Medical Microbiology. The pattern of assessment was changed from the traditional format to one that has more diverse and frequent assessment opportunities. Expectations were that the alterations made in assessment (Table 1b) would positively impact (i) comprehension throughout the class, (ii) long-term retention of the material, and as a result (iii) overall grades in the class.

TABLE 1b.

Current assessment strategy (starting Spring 2003)

Assessment	Frequency	Percentage of final grade
Pretesta	Omitted	0
Unit quizzesb	Every 1 to 2 weeks; only count best 7 of 8	3.6 each; total of 25
Microbe card examsc	Take home midterm and final	4.5 each; total of 8.9
Final exam	Once, at end of semester	8.9
Unit outlinesc	Every 1 to 2 weeks	0.9 each; 8.9 total
Streak plate	Needs to be accomplished once during semester	0.9
Lab rules quizc	Once, at beginning of semester	1.8
Lab quizzes (short answer)	At least 6; only count best 5; random pop quizzes	8.9
Unknown ID project	Execution throughout semester; writing final paper over the course of 1 month	10.7
Virtual unknown ID projectc	Once, 1 month before unknown ID project at lab bench	2.7
Lab practical examsb	Twice, at midterm and final	8.9 each; 17.8 total
Lab citizenship and attendance	Continuous throughout semester	4.4

^aThe pretest, which was used to determine overall student familiarity with prerequisite material, was omitted in the current assessment strategy with no apparent detrimental effect.

^bThese curriculum changes are modifications to previously existing assessment components; these items were administered more frequently throughout the semester with less value per implementation.

^cThese curriculum changes are additional assessment items added to the curriculum to allocate more points throughout the semester with less value per item and also to provide in-depth independent instruction outside classroom time.

METHODS

Paramount to the improved methodology was an increase in the number of exams without sacrificing lecture time, while still maintaining or improving student learning. All previous forms of assessment are summarized in Table 1a. These included a standard set of three lecture exams and one final, one pretest (to assess preparation for the class in basic biologic concepts), a streak plate (to assess students’ ability to separate different species), random short-answer lab quizzes (to assess students’ preparation for labs), a semester-long project in identifying an unknown sample, one lab practical exam, and lab citizenship.

The plan included offering seven or eight small exams (now called unit quizzes), geared towards covering 1 or 2 weeks of lecture, instead of the traditional three large exams, which tended to cover a full month of lectures. The laboratory practical exam was split into a midterm and final. The final format (Table 1b) includes a combination of modified assessment features (the more frequent, shorter unit quizzes and the lab practical exams) and traditional features (final exam, unknown project) as well as shorter, less standard teaching and assessment techniques (requiring students to outline the reading before each lecture set, quizzing students on the laboratory rules, and giving two short take-home exams with Peppler’s Microbe Cards(5)).

The first author taught the course with the new design during the Spring 2003 semester. The second author implemented the same changes during the Summer 2003 session. During the Fall 2003 semester, the second author observed some lectures given by the first author and conducted student interviews for use as qualitative supporting data. Grades were collected from past semesters of Medical Microbiology from classes taught by both authors and were compared to the grades from each pilot semester, starting in Spring 2003. Grades were also culled for each student in his or her introductory biology class for statistical analysis. In addition to these quantitative data, qualitative data were also collected in the form of student comments gathered in personal interviews (Fig. 1). The qualitative methods used were drawn from a grounded theory perspective, which allows themes to emerge from the gathered data (6). The interviews were tape recorded and then transcribed.

FIG. 1.

Comments from student interviews. Interviews took place during a pilot semester with the new methodology. The comments, separated by headings, illustrate the impact on student attitude. There were no negative comments from the entire transcript of the two 30-minute interviews. This may be an artifact of the low number of interviews conducted.

Due to strongly nonnormal distributions, a nonparametric analysis of variance (ANOVA; a Kruskal-Wallis one-way analysis of variance) was utilized to determine the statistical significance of the increase in student performance, as outlined in the text, with an alpha level of P = 0.05. A contingency table analysis was used to determine the significance of the decrease in failure rates, as reflected by the relevant P values.

RESULTS

The experimental design was undertaken to improve the undergraduate experience in a microbiology class composed of primarily biology and health science majors. Experience with previous class formats showed that traditional assessment methods resulted in a large number of students, especially in the health sciences, who were not successfully completing the class for credit (Table 2, D/F/FX/W columns). Therefore, the assessment strategy was changed as outlined in Table 1. Results of the changes are illustrated in Table 2. Grades were weighted on a linear four-point scale (A = 4.0, B = 3.0, C = 2.0, D = 1.0, F = 0.0). Grades were separated and analyzed by major. In addition, the grades were separated and analyzed by instructor to determine if there were any instructor-specific effects. Data from students who were not biology or health science majors were not considered in the statistical analyses due to a relatively insignificant sample size (n = 32). Instead, the focus was concentrated on the grade effect for the two major groups: health science majors and biology majors (n = 219).

TABLE 2.

Summary of student outcomes with altered assessment strategy^a

	Biologyb			Health sciencesc
	Grade averagesd	D/F/FX/We	n	Grade averagesd	D/F/FX/We	n
Traditional assessment	3.20 ± 0.99	11%	44	2.14 ± 1.08	34%	47
Modified assessment	3.36 ± 0.92	7.2%	69	3.08 ± 0.89	12%	59
P value of improvement	0.849, 0.531f	0.598	N/Ag	0.021, <0.001f	0.025	N/Ag

^aFinal grades for each student were weighted on a linear scale, with a value of 4.0 for an A, 3.0 for a B, 2.0 for a C, 1.0 for a D, and 0.0 for an F.

^bBiology undergraduate major includes: (i) Biology, (ii) Environmental Sciences, and (iii) Molecular Biology, Biochemistry, and Bioinformatics.

^cHealth Sciences undergraduate major includes: (i) Occupational Therapy, (ii) Prenursing, (iii) Nursing, (iv) Exercise Science, (v) Deaf Studies, and (vi) Healthcare Management. Note that statistical analyses were limited to biology and health science majors because the sample sizes from other majors were too small to conduct meaningful analyses.

^dGrade averages and standard deviations of these values, based on the 0.0 to 4.0 point scale, were calculated for all students within each category.

^eAfter the registration period was complete, the final tally of students who would not receive credit for the class (D and F are failing grades; FX represents students who stopped attending class; W is a noncredit withdrawal) was determined to assay student retention and success rates in the class. These values indicate all the students in the respective category who did not complete the class successfully and do not indicate an average.

^fP values are presented for the data as they are related to each instructor.

^gNot applicable.

Grade averages overall improved (Table 2). More importantly, although there was not a significant increase in the grades of biology majors (Table 2) from a 3.20 (the equivalent of a B) to a 3.36 (also a B; P = 0.849 and P = 0.531 for each instructor), there was a significant improvement (Table 2) for the health science majors from a 2.14 (the equivalent of a C-) to a 3.08 (a B-), as determined by a nonparametric ANOVA (P = 0.021 and P < 0.001 for each instructor). Since both instructors taught sections of mixed majors which produced data with equivalent statistical significances (Table 2, bottom row), there was no instructor-specific skewing of the increase in health science majors’ performance.

The data were also analyzed on a Pass-No credit basis (Table 2). Again, there was not a significant difference in passing rates for biology majors (P = 0.598). However, there was a substantial increase in passing rates for health science majors (P = 0.025).

In addition to these quantitative data, supporting qualitative data from student interviews was also collected (Fig. 1). There were no negative comments from an entire transcript of the 30-minute interviews. These data show that the students, with the new assessment strategies, feel less pressure and feel positive about their learning. Testing situations can be very tense and nerve-racking (8). Science classes often confer a high level of stress, as stated by a student in Medical Microbiology:

“[I]n science, of course, especially as you get up to the three and four hundred-level classes, there is tons and tons of information and I have had instructors in the past who have had 10,000 pieces of information and three exams in the whole semester. It’s just so much information on one exam; a lot weighs on one exam. Me personally, I’m working on it but I’m not a good test taker, meaning that I get very stressed out, because I really want to do well, always looking for As, and I get stressed out. I’m afraid I’m going to forget something. I have a lot of test anxiety, so it seems like when I’m overwhelmed with a lot more information, and this particular exam is going to make or break my grade, and things like that, it makes me even more anxious. And the potential is that it can make me do worse.” (Interview, 10/27/03)

This stress and anxiety seems to be common for undergraduates, and changing the way students are assessed has had a positive impact on the attitude of the students.

DISCUSSION

By changing the frequency and types of assessment in Medical Microbiology without sacrificing content, the students’ experience in the class has improved, most markedly with health science majors. The students are much happier not having to retain extensive amounts of information for each assessment and seem to comprehend the material much more readily (Fig. 1). The final grades (especially of health science majors) have improved almost an entire letter grade, and class grade values have increased (Table 2).

There seems to be little effect of this treatment on the grades of biology majors. The reasons behind this are not clear. There may be an effect from the cap of possible grades (i.e., 4.0 is the highest possible numerical rank provided by an A), creating an artificial ceiling that alters the highest end of the numerical data for those students who achieve at a higher level. Perhaps biology majors are trained differently, in that they expect a three- or four-exam format. Also, classes within the major may be teaching them some content from Medical Microbiology in lower levels, leading to the students easily achieving high grades in Medical Microbiology by thorough comprehension during a second or third presentation of old material.

The data were also analyzed to ensure that the increase in performance for the students was caused solely by the change in assessment and not by potential outside confounding factors. The first, that different instructors would see different effects because of dissimilarities in teaching style, was refuted by the nonparametric ANOVA; both instructors saw a statistically significant increase in performance of health science majors, while relatively little effect on the outcomes of biology majors was observed. In addition, there was a possibility that the students in the group of better performers were those who initially performed better in their introductory biology class. To resolve this issue, grades from the prerequisite biology course taken by these students were analyzed. Excluding transfer students, who do not receive a letter grade for their previous experience, it was found that biology majors in Medical Microbiology using the previous assessment strategy attained an average of 3.2, while those enrolled in our experimental group achieved an average of 3.1 (both solid Bs) in their preparative biology class. Similarly, health science majors averaged a 2.8, whether those students were within our first cohort or our second. These observations indicate that the students in our second group were not necessarily more intelligent or better prepared than our first group, and that the statistically significant increase in grade and passing performance was not an artifact of previous performance.

This study is continuing each semester in an effort to improve the class with each iteration, creating an atmosphere that is more conducive to learning and to success. Continued application of this assessment format is expected to generate a group of students eager to proceed through their clinical training as health science majors without a fear of the basic science behind their profession.