Abstract
A basic bacteriology course was offered in two successive academic years, first in a conventional format and subsequently as a “hybrid” course. The latter combined (i) online presentation of content, (ii) an emphasis on online resources, (iii) thrice-weekly, face-to-face conversations to advance understanding, and (iv) frequent student postings on an electronic discussion board. We compared the two courses through statistical analysis of student performances on the final examinations and the course overall and student assessment of teaching. The data indicated that there was no statistical difference in performance on the final examinations or the course overall. Responses on an instrument of evaluation revealed that students less strongly affirmed the following measures in the hybrid course: (i) The amount of work was appropriate for the credit received, (ii) Interactions between students and instructor were positive, (iii) I learned a great deal in this course, and (iv) I would recommend this course to other students. We recommend clear direction about active learning tasks and relevant feedback to enhance learning in a hybrid course.
Baccalaureate education is a dynamic process affected by resources and expectations. Computers, in general, and the Internet, specifically, have become increasingly common resources. Current students expect that computers will be used in instruction (surveys performed in 2004 by Instructional Technology Leaders show that 72% of Lehigh students favored web-based course management); paralleling this expectation is a sense that lectures (despite the prospect that lectures might be both efficient and elegant) are often not an appealing mode of instruction.
Recent interest in the importance of learning tasks supports the notion that interactivity among students fosters active roles and is an effective way to engage students in learning and to enhance performance (5, 6, 10, 13). Creating learning tasks for students which involve an online discussion can be particularly effective. By creating an online community of learners, students are able to learn from one another by receiving peer-to-peer feedback as their ideas about the subject matter develop (1, 2). In discussions, students learn from each other as they struggle to resolve differences in opinion. This tension results in a deeper level of understanding. “Students learn from one another because in their discussions of the content, cognitive conflicts arise, inadequate reasoning can be exposed, disequilibration will occur, and higher-quality understandings will emerge” (12). Current research also supports the notion that using web-based activities can enhance students’ preparation for class and thereby allow the classroom discussion to be more worthwhile. Students directed simply to read material before class are less likely to do the required reading than students who are directed to read the material and also complete a task related to the reading (such as answer a question, discuss an issue, prepare a summary statement, etc). Requiring this type of preparation from students can evoke a change in the style of the class time from a lecture format, used to introduce new content, to a “workshop” format that allows the class time to be used for discussion of the content and an active learning experience described as a type of “illumination of the content” (3). This approach has been described as “changing the time and space for learning” where students and faculty reverse the traditional thinking about the role of class work and the role of homework (3, 10, 11).
Bacteriology seems an especially suitable subject for this instructional format. Students have access to a series of striking electronic resources including the Microbe Library sponsored by the American Society for Microbiology (http://www.microbelibrary.org); the Microbial Literacy Collaborative sponsored by the American Society for Microbiology, the National Science Foundation, and the Department of Energy (http://www.microbeworld.org); the more than 200 complete genomic sequences of prokaryotes presented by The National Library of Medicine (http://www.ncbi.nlm.nih.gov/genomes/MICROBES/Complete.html); access to microarray technology for determining the presence and expression of myriad genes; a detailed and highly extensive rendering of prokaryotic phylogeny presented, again, by The National Library of Medicine (http://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?name=Eubacteria); an exhaustive online manual of germ warfare maintained by the United States Army Medical Research Institute of Infectious Diseases (http://www.usamriid.army.mil/education/bluebook.htm), and so forth.
With these premises and resources in mind, the corresponding author first offered a traditional three-credit lecture course and subsequently a three-credit “hybrid” bacteriology course at Lehigh University. The hybrid course combined (i) online presentation of content, (ii) an emphasis on online resources, (iii) frequent student postings on an electronic discussion board, and (iv) thrice-weekly, face-to-face conversations to discuss postings in order to advance understanding.
The design of the hybrid course required students to exhibit sustained, active participation. We expected that students would be more deeply invested in the course and enjoy their more active roles. We anticipated that students would have an enhanced mastery of the course content, greater development of verbal and analytical skills, and a stronger sense of accomplishment. In our retrospective analysis, we compared the two courses through statistical analysis of student assessment of teaching and student performances on the final examination and the course overall.
METHODS
The course. An exhaustive syllabus served as the principal guide; the course content was divided into ten modules. The pace of the course was established by moving to a new module every seven class days in the14-week-long semester. The topics of the modules are presented in Table 1A; an outline of the content of a module is presented in Table 1B.
TABLE 1.
Course content
| A. The modules | B. An example in outline form of module content |
|---|---|
| I. The ubiquity of bacteria | VI. Growth
|
| II. Pathogenicity | |
| III. Defenses against pathogens | |
| IV. The prokaryotic surface and external structures | |
| V. The interior of a bacterium | |
| VI. Growth | |
| VII. Energy transformations and intermediary metabolism | |
| VIII. The biogeochemical cycles | |
| IX. Prokaryotic genomics and prokaryotic sex | |
| X. The wonder of it all |
Postings. The principal active learning task required students to post commentaries about each module. Students individually or in pairs were to post accurate, substantive, literate, and interesting commentaries pertaining to the current module. The students were free to post alone or to select a partner; with pairs, it was assumed that the students had equal responsibility. No restrictions whatsoever were placed on the extent or character of the commentaries. Postings were signed by the authors. The postings served as the basis of a second active learning task: in-class discussion. After the session of active face-to-face learning, the instructor sent an e-mail appraisal of the postings to the student author(s) commenting on substance (i.e., sufficiency to sustain discussion), accuracy, and literate expression (i.e., adherence to standard usage and observance of conventions, e.g., that the singular of bacteria is bacterium.)
When the period of time for a module was completed, a 20-minute quiz (comprised of short-answer objective questions plus an essay, problem, or diagram) covering both the module and the student commentaries was administered during the scheduled meeting time.
The student cohorts. The academic affiliations of students in the traditional and hybrid courses are listed in Table 2. The students came from a variety of academic backgrounds, and the hybrid course had fewer Bachelors of Science and more Bachelors of Arts students. Despite these differences, the overall academic performances of the two cohorts were equivalent. Specifically, the mean cumulative grade point average for students entering the traditional course was 3.30; the mean for students entering the hybrid course was 3.05; the difference is not significant (t(62) = 0.46). All students, n = 34 in the traditional course and n = 30 in the hybrid course, who rostered the courses completed them.
TABLE 2.
Student affiliations
| Curriculum | Traditional | Hybrid |
|---|---|---|
| Bachelor of Science in Biology | 13 | 10 |
| Bachelor of Science in Molecular Biology | 14 | 6 |
| Bachelor of Science in Biochemistry | 4 | 5 |
| Bachelor of Science in Behavioral Neuroscience | 0 | 3 |
| Bachelor of Arts in Biology | 3 | 4 |
| Bachelor of Arts in English | 0 | 1 |
| Undeclared major | 0 | 1 |
Student performance. Assessment of performance in the traditional course was based on performance on 10 quizzes and a comprehensive final. Assessment of performance in the hybrid course was based on performance on 10 similar quizzes, the quality of the 10 postings, and performance on a comprehensive final which was identical to the one used in the traditional course. The format of the quizzes was one page of short-answer questions and one page of essays. The format of the final examination was seven pages of short answer questions and seven pages of essays. Two examples of short-answer questions appear in Table 3A; an example of an essay question appears in Table 3B.
TABLE 3.
Examples of quiz questions
| A. Short answer | B. Essay |
|---|---|
| Among bacteria, not all organisms use molecular oxygen as a terminal electron acceptor. Provide examples of two compounds used in anaerobic respiration. |
|
| Organisms (likely prokaryotes) transmitted from one planet (or moon) to another would need to be metabolically inert or in a state of suspended animation. What word is used to describe an organism in which the water content has sublimed? |
To assess the extent of learning, we compared the performances on the final examination and the final grades for the two classes, despite the inclusion of postings in the hybrid course and their absence in the traditional course.
Assessment of teaching. Faculty at Lehigh University use a standard set of measures for evaluation of the course and the instructor by students. The evaluation instrument is modeled on the Instructor and Course Evaluation System (ICES) developed by the Division of Measurement and Evaluation of the Office of Instructional Resources at the University of Illinois (9). The 21 measures are broadly representative of areas deemed to be important in assessing student response to both the course and the instructor; these categories are: (i) global questions about the effectiveness of teaching and the overall quality of the course, (ii) course management, (iii) student outcomes of instruction, (iv) instructor characteristics and style, and (v) instructional environment.
The instrument asked students to respond to a series of statements on a 5-point scale, where A is Agree Strongly and E is Disagree Strongly. The statements appear in Table 4.
TABLE 4.
Statements used in student evaluation of course and instruction
|
The assessment was performed approximately 7 to 10 days before the conclusion of the semester during the first 15 minutes of a scheduled class. The instructor was absent from the room while students recorded their responses. The bubble-sheet response forms were collected by a student volunteer and mailed to Lehigh’s Department of Psychology. The machine-graded results were sent to the instructor after submission of grades and the conclusion of the semester. Twenty-six students submitted evaluations in the traditional course; 24 in the hybrid course.
Institutional Review Board (IRB) permission. The United States Code of Federal Regulations Title 45 exempts “(i) research conducted in established or commonly accepted educational settings, involving normal educational practices; and (ii) research involving the use of educational tests (cognitive, diagnostic, aptitude, achievement), survey procedures, interview procedures or observation of public behavior” (http://squid.law.cornell.edu/cgi-bin/get-cfr.cgi?TITLE=45&PART=46&SECTION=116&TYPE=TEXT) from informed consent practices. The Lehigh University IRB has reviewed the procedures used in acquiring the data for this study and has concluded that the “protocol has been approved by the Institutional Review Board for Human Subjects Research under paragraph #2 in the expedited review category.”
RESULTS
Performance on the final examination and in the course overall. We took the distribution of grades for the final examination and the distribution of the assigned grades for each semester, and assigned 4.0 to A, 3.7 to A−, etc. For each of the two measures, we performed a t test for a difference between means for traditional and hybrid courses (8). The mean grade on the final examination was 2.51 in the traditional course and 2.13 in the hybrid course. These two means did not differ significantly, t(56) = 0.41. The mean assigned grade was 2.87 in the traditional course and 3.03 in the hybrid course; these two means did not differ significantly, t(61) = 0.19.
Assessment of teaching. For each measure from the student evaluations, we performed a chi-square test to determine whether the frequency distribution of responses differed between the hybrid and conventional courses. We assigned the responses to three categories: “strongly agree” (response A), “agree” (response B) and “other.” (There were so few responses in categories C, D, and E that we combined these three responses into “other” so that the assumptions of the chi-square test would be met.) Four measures of student response were statistically significant, viz., “The amount of work was appropriate for the credit received” (P < 0.01), “Interactions between students and instructor were positive” (P < 0.01), “I learned a great deal in this course” (P < 0.05), and “I would recommend this course to other students” (P< 0.05). The data for these four measures are presented, respectively, in Figures 1A, 1B, 1C, and 1D.
FIG. 1.
Distributions of statistically significant student responses in the conventional and hybrid courses. (A) The amount of work was appropriate for the credit received. (B) Interactions between students and instructor were positive. (C) I learned a great deal in this course. (D) I would recommend this course to other students.
For each question on the student evaluation, the university computes a mean response. Higher mean scores for a measure indicate that students more strongly affirm that measure. Notably, only one measure (“Assignments aided learning”) received a higher score in the hybrid course while two (“Knew subject well” and “Was well prepared”) showed no difference. The remaining eighteen measures received lower scores. We applied the sign test (9) to this distribution of diminished scores and found a significant z = 3.27, P < 0.001, which indicates less satisfaction with the hybrid course as compared to the traditional course.
DISCUSSION
Active learning tasks in a hybrid bacteriology course generated a sustained and intense exposure to the course content. But student dissatisfaction is clearly manifest in response to the measure, “The amount of work was appropriate for the credit received.” On the one hand, students indicated in the course assessment that the assignments were beneficial and aided learning. On the other hand, anecdotal statements indicated that the students did not think that the work was too little; rather, the workload was perceived as too much. Indeed, this sentiment may have been the defining circumstance for the course from the students’ perspective. The feeling of having had to work too hard may have affected other perceptions. For example, response to the measure, “I learned a great deal in this course,” was significantly lower in the hybrid course compared to the conventional format. Yet, performances on the final examination and the course overall (which should be measures of learning) did not differ between the two versions of the course. (A reviewer suggested that “through the online and face-to-face discussion [students] came to realize that they didn’t know as much as they thought. Students in the conventional course may not have had the opportunity to come to this realization.”) Also, improving performance among highly capable students while maintaining stringent grading standards may not be readily achievable.
The measure, “Interactions between students and instructor were positive,” was also significantly lower in the hybrid course. Yet the measure, “The instructor treated students with respect,” was not significantly different. The juxtaposition of these two measures suggests that the interaction that was adversely affected in the hybrid course was pedagogical and not civil.
Collectively, students in the hybrid course seemed ambivalent about recommending the course, whereas students in the course with the conventional format appeared inclined to recommend the course. The evaluations were anonymous so an inclination to recommend cannot be correlated with grades. But the disposition to recommend can be correlated with both the feelings of how much effort was expended and how much was learned. We speculated that students who found the amount of work appropriate for the credit received would recommend the course to others. Contrariwise, students who found the amount of work inappropriate for the credit received would not recommend the course to others. We confirmed this hypothesis by analyzing the data as follows: for the statements on the student evaluations we assigned 1 to “strongly disagree,” 2 to “disagree,” etc. We then computed the correlation (8) between the response to the statement “Appropriate amount of work” and the statement “Recommend the course.” We obtained r = 0.45, which was significant beyond the 0.05 level. Students who were more inclined to recommend the course to others were more likely to think that the workload was appropriate to a three-credit course.
Two reasons are suggested to explain why the measure of satisfaction between “interaction between students and instructor” was lower in the hybrid course were that students (i) lacked clear direction about the required task and (ii) lacked sufficient or relevant feedback from the instructor to let them know if they were on track with their responses. These interpretations suggest that the e-mail responses of the instructor were inadequate in some fashion.
Students require clear direction about the task that they are required to do (2, 4, 7). The students were told to post “accurate, substantive, literate and interesting commentaries pertaining to the current module.” The instructor assumed that interactivity among students and their more active roles would engage the students in learning and enhance performances. But, research has established that such a result will not be achieved without explicit direction from the instructor about (i) the purpose of the online discussion and (ii) how to participate effectively (1, 2, 4).
The second suggested reason for students’ dissatisfaction is that they lacked sufficient or appropriate feedback from the instructor to guide their discussions. At each stage, the instructor can provide online feedback (instructor to students) to guide them through the discussion. For example, if the students have difficulties generating evidence related to the hypotheses suggested or generating alternate solutions, the instructor can provide online questions to model an appropriate line of reasoning.
Feedback can also be provided by peers if an online group is established that has well-defined requirements for the members. The University of Waterloo (UW) has developed a model based on student and instructor feedback from many online courses (http://LT3.uwaterloo.ca/faculty). The UW model assigns students to groups of four (10). Group members have an online discussion about a defined topic. Participants must make an initial posting about the topic before accessing the submissions of the other members. This requirement ensures that each member participates and takes a stand on the issue before he or she is influenced by reading the opinions of others. Also, by reducing the number of participants in the group to four (rather than the full class), students are less likely to feel overwhelmed by the large numbers of postings. Group members take turns as the leader who is responsible for posting a summary of the group’s comments to a posting that will be read by the full class.
In a second rendering of the hybrid course, we asked students a series of open-ended questions, e. g., “What was the most valuable feature of this course? ...the freedom to develop your understanding in any way that you wished? ...the lack of regimentation? ... the affirmation of student ability? ...the intended emphasis on principles rather than rote fact? ...something else?” The responses to such questions were plainly positive. Since similar questions were not asked in the traditional course, comparisons are not possible. Nonetheless, these responses indicate that the intense hybrid course did have positive outcomes and that the form of assessment is an important variable.
A second outcome which has yet to be assessed is a difference in long-term recall between the two forms of instruction. A longitudinal study may be informative and significant.
CONCLUSION
Web-based baccalaureate education provides an immense opportunity for learning bacteriology (and other subjects.) Paradoxically, that immensity can be a limitation. The experiences of students in Lehigh’s hybrid bacteriology course demonstrate conclusively that the role of the professor in managing student participation is of paramount importance. Similarly, providing freedom to students in their approach to a formal body of knowledge also requires diligent guidance by the professor. These conclusions are manifest in the model developed at the University of Waterloo that optimizes student performance in an online or hybrid course by specifying roles in group activities and offering appropriate supervision.
Acknowledgments
SK was the recipient of a “Ventures” grant from the Provost’s Office at Lehigh University to develop the hybrid course and a travel grant from the Faculty Development and Learning Innovations Program at Lehigh University to attend the “Faculty Development for Teaching, Learning and Technology Summer Institute 2002” at Portland State University.
REFERENCES
- 1.Bonk CJ, King KS. Electronic collabor-ators: learner centered technologies for literacy, apprenticeship, and discourse. Lawrence Erlbaum; Mahwah, N.J: 1998. p. 4. [Google Scholar]
- 2.Bonk CJ, Cunningham DJ. Searching for learner-centered, constructivist and sociocultural components of collaborative educational learning tools. In: Bonk CJ, King KS, editors. Electronic collaborators: learner-centered technologies for literacy, apprenticeship and discourse. Lawrence Erlbaum Associates; Mahwah, N.J: 1998. p. 32.p. 43. [Google Scholar]
- 3.Cennamo K, Ross J, Rogers C. Evolution of a web-enhanced course: incorporating strategies for self-regulation. Educause Quarterly. 2002;25:28–33. [Online.] http://www.educause.edu/ir/library/pdf/eqm0214.pdf. [Google Scholar]
- 4.Duffy T, Dueber B, Hawley C. Critical thinking in a distributed environment: a pedagogical base for the design of conferencing systems. In: Bonk CJ, King KS, editors. Electronic collaborators: learner-centered technologies for literacy, apprenticeship and discourse. Lawrence Erlbaum Associates; Mahwah, N.J: 1998. p. 61. [Google Scholar]
- 5.Harrigan K, Carey T, Salter D. An instructional design model for learning object re-use. 2002. Merlot International Conference, Academic Approaches to Technology: Content, Collaboration, Collections & Community, Atlanta, Georgia. [Online.] http://conference.merlot.org/conference/2002/program/detail_program.php#MonConc1.
- 6.Herrington J, Oliver R, Stoney S. Engaging learners in complex, authentic contexts: instructional design for the web. In: Wallace M, Ellis A, Newton D, editors. Proceedings of the moving on-line conference Southern Cross University; Lismore, New South Wales, Australia. 2000. pp. 85–96. [Google Scholar]
- 7.Light RJ. Making the most of college: students speak their minds. Harvard University Press; Cambridge, Mass: 2001. p. 242. [Google Scholar]
- 8.Mendenhall W, Beaver J, Beaver M. Introductions to probability and statistics. 10th ed. Brooks/Cole Publishing Company; Pacific Grove, Calif: 1999. p. 766. [Google Scholar]
- 9.Ory J. Instructor and course evaluation system catalog. University of Illinois at Urbana-Champaign; 2001. p. 28. [Google Scholar]
- 10.Salter D, Richards L, Carey T. The “T5” design model: an instructional model and learning environment to support the integration of on-line and campus based courses. Educ Media Int. 2004;41:207–217. doi: 10.1080/09523980410001680824. [DOI] [Google Scholar]
- 11.Small D. Calculus after high school calculus: confronting the core curriculum. In: Dossey JA, editor. Considering change in the undergraduate mathematics major; Proceedings of the Mathematical Association of America (notes 45), Mathematical Association of America; Washington, D.C.. 1998. pp. 47–54. [Google Scholar]
- 12.Slavin RE. Cooperative learning: theory, research, and practice. 2nd ed. Allyn and Bacon; Boston: 1995. p. 194. [Google Scholar]
- 13.Vella J. Taking learning to task: creative strategies for teaching adults. Jossey Bass; San Francisco: 2000. p. 151. [Google Scholar]