Abstract
Recent recommendations for educational research encourage empirically tested, theory-based, completely transparent, and broadly applicable studies. In light of these recommendations, we call for a research standard and community of practice in the evaluation of technology use in the undergraduate life science classroom. We outline appropriate research methodology, review and critique the past research on technology usage and, lastly, suggest a new and improved focus for research on emerging technologies.
INTRODUCTION
In an article written shortly after and influenced by the No Child Left Behind Act of 2001 and a subsequent report released by the National Research Council (38) on scientific research in education, Feuer et al. (19) put forth a list of attributes that all educational research should include, which we have summarized here:
a focus on significant issues that can be empirically tested
a base in scientific theory
the use of meaningful and scientifically sound methods that allow for direct and appropriate investigation of causal questions
the full disclosure of these methods
the production of results that are broadly applicable and consistently replicable
an allowance for professional review and critique of data and methodology
Feuer et al. (19) go on to outline many important features that should define educational research and the research community. We would like to extend this call for an established research standard and evaluate each of these attributes as it applies to the evaluation of technology use in undergraduate biology education.
A focus on significant issues that can be empirically tested
In the midst of our currently technologically saturated world, the use of technology in the classroom is a natural consequence and sometimes inescapable expectation. Many technologies are introduced as ‘new innovations in teaching’ even though their effect on learning has yet to be assessed. This seems especially true in the science classroom as the disciplinary boundaries between science and technology are so often blurred. Whether teachers prescribe it or students voluntarily implement it, technology is bound to make an appearance in most science classrooms, particularly at the undergraduate level, at some point in the immediate future.
Novak (32), however, pointed out that a significant roadblock to the implementation of potentially beneficial technologies in the classroom is the cost of hardware and software. Indeed, Carle et al. (8) further explain that the resistance is due to the paucity of scientifically based studies examining the effectiveness of technology, explicating the mechanism through which the benefits arise, and providing the evidence to justify the expense. On a national level, the US Department of Education ran the program Preparing Tomorrow’s Teachers to Use Technology (PT3) to encourage the use of technology in the classroom. Findings from funded projects indicated that implementation of technology was expensive, while the benefits were mixed (32). Policymakers who determine school and research funding rely on educational research findings to justify expenditures (19, 36).
The natural question that follows is, “How does technology affect the educational experience?” It is a question with which all educators and policymakers should be concerned. In our opinion, this is a significant issue facing current educational practices that can be empirically tested, given the right tools, a sense of appropriate scientific research practices, and collaborative efforts amongst researchers. We reiterate Roblyer and Knezek’s (37) agenda to ensure that studies on technology “look at technologies not as delivery systems, but as components of solutions to educational problems, and that research questions be stated in a way that the contributions of methods can be examined and tested.” This leads us to the second attribute of educational research.
A base in scientific theory
As is true in all research disciplines, hypotheses should be firmly based within a theoretical framework. For example, if we are interested in how the use of clickers influences student learning of pertinent concepts, we should consider learning theories that would support the idea that formative assessment opportunities encourage self-regulation and strengthen connections in long-term memory. We should consider also the effects of assessing lower Bloom’s levels versus higher Bloom’s levels (13, 27, 30).
Assessing technology use in education should follow the same rigorous protocol demanded by scientific research in all disciplines. As Roblyer and Knezek (37) emphasized, educational research should focus on an educational problem. In the case of technology research, we should focus on educational problems where technology can offer a possible solution (e.g., virtual laboratory experiences to accommodate large-enrollment courses in beginning biology). Then, the question should be not “if” virtual laboratory experiences increase student learning, but “how” the benefits are gained.
Hypotheses should then be derived based on careful consideration of the relevant learning theories. For example, if virtual laboratory experiences offer the authentic (although simulated) exploration of biological phenomena that would otherwise be lacking, then, based on constructivist theory, the virtual laboratory experiences in addition to lecture should lead to greater gains in conceptual understanding than a lecture-based course without labs. The question, then, pertains to how one might go about accurately assessing such gains.
The use of meaningful and scientifically sound methods that allow for direct and appropriate investigation of causal questions
Educational research has received a lot of criticism for its lack of rigor and divergence from traditional scientific methodology (5, 33). Some have claimed that, based on the nature of educational research, it cannot be held to the same standards (17). We would rather agree with the position of Feuer et al. (19): “The bottom line is that experimentation has been shown to be feasible in education and related fields and is still the single best methodological route to ferreting out systematic relations between actions and outcomes.”
The evaluation of technology use in science education can, and should, follow the same rigor and format of other scientific pursuits, including establishing controls, measuring dependent variables, isolating independent variables, recognizing and controlling for outside variables, and generating and testing theory-based hypotheses. We do, however, agree with Erickson and Gutierrez (17), as well as Berliner (5), that due to the complex nature of educational research with its dynamic social context, research results should be presented and received with a healthy level of skepticism and an acute awareness of the limits of these approaches.
The full disclosure of these methods
As a consequence of the complex nature of these studies, it is imperative that our methodologies as educational researchers be explicitly stated and fully disclosed. Strudler (39), in response to Robyler and Knezek’s article, emphasized that we need to pay particular attention to the surrounding circumstances of the technology implementation that worked in order to elucidate the causal mechanisms behind it. In addition, in order to broaden the applicability of our results, it is necessary to understand the dynamics of the context in which the technology was implemented.
This concept allows for individualized, context-specific studies to extend a general benefit to the broader scientific community. Some would claim that small-scale studies testing a specific technology in a particular setting are of little to no use in establishing the benefits of a given technology to educational practices (25). However, we posit that with contextual details clearly defined, these small-scale studies can lead to the formation of new testable hypotheses concerning the causal mechanisms behind the success of certain technological implementations, given an appropriate mechanism for sharing these results.
The production of results that are broadly applicable and consistently replicable
The other solution, of course, is to design studies that produce broadly applicable results. The best way to do this, as was discussed, is to test hypotheses the support of which would directly confirm generalized theories of learning. For example, if we can show that the use of computer animations of cellular respiration lead to greater conceptual understanding, as well as transfer of the knowledge, to diagnose metabolic disease compared to the effect of students simply reading about it in the textbook, there is not necessarily a direct transfer to animations of meiosis, for example, and its transference to understanding of genetic diseases. However, if we can show that animations of cellular respiration lead to greater conceptual understanding and knowledge transfer because they appeal to more than one learning modality (i.e., both visual and auditory) directly relating to the theory of modality (3, 35), then perhaps if the meiotic animations also appealed to both learning modalities, we could assume that these would lead to greater improvements in student understanding, as well.
In addition, studies should be run in several different settings (e.g., community colleges, liberal arts colleges, large universities) to ensure the general applicability of results. This can be facilitated by encouraging collaborations between researchers with access to different student bodies.
An allowance for professional review and critique of data and methodology
As Feuer et al. (19) noted, the challenge of educational researchers is bringing together diverse communities to share findings across domains, cultures, and methods. We may be interpreting this attribute in a slightly different way than the original intent, but we feel it is no less important. We would like to emphasize the need for a ‘community of practice’ (42) to be established for the assessment of technology use in education. Due to the complexities of educational research design, it is unlikely that any single study can produce entirely generalizable results. However, with the ability to share methodologies, validate assessment instruments, compare data, and communicate with one another, we can further our understanding of the impact of these technologies. Our goals, as researchers, are shared. We are in pursuit of an answer to the question posed previously: “How does technology affect the educational experience?” Together, we can establish scientific traditions and standards for our field thus making our scientific pursuits more rigorous, more aligned, and more generally applicable.
Research on technology in education
The past to the present
Our call for a community of practice will be further explicated in a moment. But, first we would like to review what has been and is being done in the research regarding technology use in undergraduate science education.
As learning theories have shifted, so have the research studies based upon them. Up until the early 1980s, Behaviorist Theory dominated the literature. Studies were aimed at assessing the effects of educational implementations (including technology) on student behavior. Commonly called “media comparison” studies, research was aimed at comparing one mode of instruction (usually one utilizing technology) to another mode (one that does not utilize technology). The defining characteristic of this type of study that has changed over time is the focus on the actual medium, rather than on the design of instruction, from which its major criticism has stemmed (40). In fact, Clark (9, 10, 11), one of the major criticizers of this method, claimed that the media, itself, is just a “delivery truck.” As the theory of constructivism began to take root in education, researchers began to shift their focus away from the media and more toward the method of instruction (see 37 and 40, for detailed reviews).
In 2003, Roblyer and Knezek (37) issued a call for a new agenda on technology in education, focusing more on the “why” behind the success of technology and looking at technology as a means to solving educational problems and as part of hypotheses firmly based in learning theory. We reiterate Roblyer and Knezek’s call and expand upon it. We agree that researchers should “look at technologies not as delivery systems, but as components of solutions to educational problems, and that research questions be stated in a way that the contributions of methods can be examined and tested.” But we further call for effective assessment strategies that match a common set of desired outcomes and provide generalizable data that support the usefulness of these technologies, and make a case for the expense and effort of their implementation. In Roblyer and Knezek’s meta-analysis, they searched the Journal of Research on Technology in Education (JRTE) and characterized the types of studies they found. The majority of the studies (64%) focused on evaluations or descriptions of specific implementations, programs, or teacher training to enhance technology use. Very few studies focused on gathering data that would support and justify the implementation of technology.
We have done a similar search, but broadened the search parameters to include all peer-reviewed publications catalogued in the ERIC database (www.eric.ed.gov) since 2000 involving the implementation of technology into the undergraduate science classroom. We have found similar trends, despite the passing of more than seven years (i.e., a distinct paucity of research studies that use legitimate assessment to gather data on the effectiveness of the technological implementation). Our first search parameters included any articles including the words “biology,” “technology,” and “assess.” This search brought up only 29 articles, of which only six were studies specifically assessing the effects of technology in teaching biology. Of these six, only three were at the undergraduate level. We decided to expand our search parameters to include any articles using just the words “biology” and “technology” in combination. Since the year 2000, over 600 articles have been written addressing biology and technology. We did a comprehensive review of each article that specifically focused on the implementation of a technology in an undergraduate, biology-related course. Roughly 15% (92 articles; see Appendix 1) of the articles used legitimate and obvious assessment to show the effectiveness of the technology in question. Table 1 lists a representative sample of those articles organized by their hypothesis, methodology, assessment instruments, and results (the complete list can be found in Appendix 1). What is truly troubling, however, is that of these 92 articles in which an attempt to assess effectiveness was made, only four (5%) of them reported a reliability measure for the assessment instrument used. In addition, 32 of the articles (35%) assessed only student self-reported opinions of the technology used without giving any consideration to the effect of the technology on student learning.
TABLE 1.
Representative sample of literature searcha.
| Authors | Technology Assessed | Major Finding | Assessment Instrument (Reliability) |
|---|---|---|---|
| Akpan and Strayer (1) | Simulated frog dissection vs. conventional dissection | Students dissecting frogs with a simulated program preformed better on dissection-related quizzes than students who dissected frogs conventionally | Course exams |
| Carle, Jaffee, Miller, et al. (8) | Technology-enhanced classroom (TEC) | TEC promotes higher engagement and achievement | Achievement scores on assignments and course exams; CLASSE, NSSE (0.85, 0.90) |
| Fancovicova, Prokop, Usak (18) | Website: Information and Communication Technology (ICT) | No difference in student achievement between traditional and ICT instruction | Opinion surveys; Course exams; Follow-up exams |
| Deniz and Cakir (15) | Computer-assisted instruction (CAI) for histology | Students enjoyed CAI but felt it should be done in conjunction with regular instruction and observations | Student interviews |
| Kiboss, Ndirangu, Wekesa (22) | Computer-mediated simulations of cell division | Simulations led to better achievement and attitudes | Biology Achievement Test (BAT, 0.81); Biology Classroom Environment Questionnaire (BCEQ, 0.84); Pupil Attitude Questionnaire (PAQ, 0.78) |
| Lyles, Robertson, Mangino, et al. (28) | Audio podcasts lectures | Positively received by students, negative impact on class attendance and note taking | Opinion surveys and interviews |
| Preszler, Dawe, Shuster, et al. (34) | Clickers | Clickers improved exam scores and student attitudes toward clickers | Course exams; Opinion surveys |
| Traver, Kalshery, Diwan, et al. (41) | Web technology, group work | Web technology when combined with group work led to an increase in learning | Opinion Surveys American Chemical Society 1992 Biochemistry standardized exam (0.75); Course grade |
| Zumbach, Schmitt, Reimann, et al. (43) | Lifelab® | Students learn content knowledge despite prior knowledge | Course exam and course grades |
Using “biology,” “technology,” and “assess” as search terms, including only articles with legitimate assessment.
As Roblyer and Knezek (37) found, a majority of the 600 articles (85%) simply describe the implementation of a technology or outline the training necessary to use such technologies, with no attempt at assessment. Table 2 lists some representative studies that fit this classification. Clearly there is a need for a community of practice, a way to align our strategies and compare our results across contexts. The financial strain of implementing technology versus the prevalence and student expectation of its use demands justification that it is effective.
TABLE 2.
Studies describing new technologies with no assessment component.
| Authors | New Technology Described |
|---|---|
| Barber and Njus (4) | Describes the strengths and weaknesses of six different clicker systems |
| Bradley, Stutz, Towill (6) | Describes the creation of a plant biology website that can be used as a stand-alone Internet-based course or as a supplemental component to a regular course |
| Brickman (7) | Describes the use of case studies assigned through WebCT and the use of clickers to teach DNA fingerprinting |
| Cooper (12) | Describes the use of the Multimedia Educational Resources of Learning and Online Teaching (MERLOT) in biochemistry and molecular biology education |
| De Souza-Hart (14) | Describes the use of blogs as online journal clubs to supplement classroom learning |
| Fisher (20) | Reviews Automated Classroom Response Systems (ACRS) for teaching human sexuality |
| Kass and LaRoe (21) | Describes a web-based system for analyzing genetic relationships |
| Klymkowsky (23) | Describes a course using online materials to replace textbook usage |
| Kohorst and Cox (24) | Describes Elluminate Live ®, an online collaboration software |
| Latham and Scully (26) | Describes the use of a computer model to study evolution |
| Mickle and Aune (29) | Describes an on-line laboratory course for distance learners |
| Murray, Gibson, Ward (31) | Describes the use of web-based student activities to present real-time ocean data in the classroom |
The future
We suggest a new and improved focus for research on emerging technologies. We reiterate the common sense of Roblyer and Knezek’s 2003 agenda to engage in theory-based hypothesis testing and to focus our research questions on technologies as answers to educational problems, but we take it a step further to suggest and strongly encourage the establishment of a “community of practice.” The reasons for this are four-fold: 1) to establish a common set of desired outcomes, 2) to gather a common set of valid and reliable assessment tools, 3) to broaden the impact and applicability of our studies, and 4) to share our resources as we establish valuable collaborations.
According to Bruce Alberts (2), Editor-in-Chief of Science Magazine, the goals of science education in general are to “prepare students to know, use, and interpret scientific explanations of the natural world…to prepare students to generate and evaluate scientific evidence and explanations, to understand the nature and development of scientific knowledge, and to participate productively in scientific practices and discourse.” We suggest that any research aimed at testing the effectiveness of technology use in the biology classroom should focus on measuring these four objectives. For example, does this particular technology help students “interpret scientific explanations” or “evaluate scientific evidence” more effectively than teaching without this technology? Or, does this particular technology facilitate student participation in “scientific practices and discourse”? By having a common set of objectives, research findings could be categorized by effect and broadened to more general audiences. In other words, if an instructor wanted a teaching technique that will help students “understand the nature and development of scientific knowledge,” he/she could easily search the literature to find applicable technologies that might facilitate the achievement of this outcome.
In order to make accurate comparisons between studies of the same or similar technologies in different venues, it is important that methodologies for measurement are comparable. This would be greatly enhanced by the establishment of a common set of assessment tools for researchers to use in assessing emerging technologies. In addition, the use of these tools in multiple settings by multiple researchers will strengthen their validity and reliability. Then, research findings could be easily compared, in a meta-analytical fashion, between studies, to give our findings more strength and applicability. If we hope to justify the expenditure, in both resources and time, of implementing new technologies, we must provide sufficient data to justify the expense. By creating a means to amass data between researchers, we can strengthen our justifications more easily. Instruments that have been used and are currently being used in the research are listed with each study in Appendix 1. If reliability measures were taken, they are listed. Perhaps this is a place to start in the establishment of a common set of tools.
As was argued in the previous paragraph, establishing a common set of tools and a common set of desired outcomes will allow us to share data, and help to broaden the impact of our research and extend the applicability of these technologies to diverse settings. In addition to these benefits, establishing a community of practice will encourage active collaboration between researchers who otherwise may not communicate. With the advent of Facebook, LinkedIn, and other social networking sites, our ability to collaborate has been greatly increased. However, finding common ground is sometimes challenging, and the usefulness of such social networks is often lacking. Perhaps our community of practice can be established through one of these social networking sites. It would be a network specifically aimed at assessing emerging technologies in undergraduate life science education. Members would have the opportunity to collaboratively establish common goals, create and validate common assessment instruments, share research data, draw appropriate conclusions, discuss the implications of such findings, and establish the breadth at which the findings can be applied. A preliminary site has been established at http://biology.unt.edu/beta for this very purpose.
Erin Dolan (16), Editor-in-Chief of CBE-Life Science Education, observed, “The absence of any unified, systematic mechanism for cataloging or accessing this information (i.e., educational research) makes it nearly impossible for scientists to keep abreast of the literature on science teaching and learning, much less use it to inform their own work.” In fact, ERIC, one of the leading educational research databases does not even access several educational journals (e.g., Journal of Microbiology & Biology Education, Evolution: Education and Outreach, Biology Education Online). A community of practice would allow for the sharing of such information to encourage the translation of theory-driven-research into practice. After all, the point of educational research is not to philosophize over the nuances of educational theory, or pontificate about the glory of a particular implementation that we have developed, but rather to inform educators about what works and what doesn’t work. Our purpose is to improve educational practices, but the application of research findings to educational practice cannot occur if the findings are too narrowly applicable, unsubstantiated by adequate evidence, poorly assessed, and especially if the findings are hidden so deep in the technical literature that educators never see them. We must find a way to communicate our findings, correlate our methods, and collaborate with one another in solving educational problems using emerging technologies. A community of practice for this purpose is a necessity.
SUPPLEMENTAL MATERIALS
Appendix 1: Comprehensive Literature Search of Technology Use in Undergraduate Biology Education
ACKNOWLEDGMENTS
The authors declare that there are no conflicts of interest.
Footnotes
Supplementary materials available at http://jmbe.asm.org
REFERENCES
- 1.Akpan J, Strayer J. Which comes first: the use of computer simulation of frog dissection or conventional dissection as academic exercise? J. Comput. Math. Sci. Teach. 2010;29:113–138. [Google Scholar]
- 2.Alberts B. Redefining science education [editorial] Science. 2009;323:437. doi: 10.1126/science.1170933. [DOI] [PubMed] [Google Scholar]
- 3.Barbe WB, Milone MN., Jr Modality. Instructor. 1980;89:44–47. [Google Scholar]
- 4.Barber M, Njus D. Clicker evolution: seeking intelligent design. CBE–Life Sci. Educ. 2007;6:1–8. doi: 10.1187/cbe.06-12-0206. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Berliner DC. Educational research: the hardest science of all. Educ. Res. 2002;31:18–20. doi: 10.3102/0013189X031008018. [DOI] [Google Scholar]
- 6.Bradley LK, Stutz JC, Towill LR. Plant biology: from the classroom to the internet. J. Nat. Resourc. Life Sci. Educ. 2009;38:82–86. [Google Scholar]
- 7.Brickman P. The case of the Druid Dracula: a directed “clicker” case study on DNA fingerprinting. J. Coll. Sci. Teach. 2006;36:48–53. [Google Scholar]
- 8.Carle AC, Jaffee D, Miller D. Engaging college science students and changing academic achievement with technology: a quasi-experimental preliminary investigation. Comput. Educ. 2009;52:376–380. doi: 10.1016/j.compedu.2008.09.005. [DOI] [Google Scholar]
- 9.Clark RE. Reconsidering research on learning from media. Rev. Educ. Res. 1983;53:445–459. [Google Scholar]
- 10.Clark RE. Evidence for confounding in computer-based instruction studies: analyzing the meta-analyses. Educ. Technol. Res. Dev. 1985;33:249–262. [Google Scholar]
- 11.Clark RE. Media will never influence learning. Educ. Technol. Res. Dev. 1994;42:21–29. doi: 10.1007/BF02299088. [DOI] [Google Scholar]
- 12.Cooper S. The use of MERLOT in biochemistry and molecular biology education. Biochem. Molec. Biol. Educ. 2005;33:323–324. doi: 10.1002/bmb.2005.49403305323. [DOI] [PubMed] [Google Scholar]
- 13.Crowe A, Dirks C, Wenderoth MP. Biology in bloom: implementing Bloom’s taxonomy to enhance student learning in biology. CBE–Life Sci. Educ. 2008;7:368–381. doi: 10.1187/cbe.08-05-0024. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.De Souza-Hart JA. Biology blogs: an online journal club and assessment tool. Am. Biol. Teach. 2010;72:149. doi: 10.1525/abt.2010.72.3.4. [DOI] [Google Scholar]
- 15.Deniz H, Cakir H. Design principles for computer-assisted instruction in histology education: an exploratory study. J. Sci. Educ. Technol. 2006;15:399–408. doi: 10.1007/s10956-006-9031-5. [DOI] [Google Scholar]
- 16.Dolan EL. Grappling with the literature of education research and practice. CBE–Life Sci. Educ. 2007;6:289–296. doi: 10.1187/cbe.07-09-0087. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Erickson F, Gutierrez K. Culture, rigor, and science in educational research. Educ. Res. 2002;31:21–24. doi: 10.3102/0013189X031008021. [DOI] [Google Scholar]
- 18.Fancovicova J, Prokop P, Usak M. Web-site as an educational tool in biology education: a case of nutrition issue. Educ. Sci. 2010;10:907–921. [Google Scholar]
- 19.Feuer MJ, Towne L, Shavelson RJ. Scientific culture and educational research. Educ. Res. 2002;31:4–14. doi: 10.3102/0013189X031008004. [DOI] [Google Scholar]
- 20.Fisher CM. Automated classroom response systems: implications for sexuality education and research. Am. J. Sex. Educ. 2006;1:23–31. doi: 10.1300/J455v01n04_03. [DOI] [Google Scholar]
- 21.Kass DH, LaRoe R. Web-based analysis for student-generated complex genetic profiles. Biochem. Molec. Biol. Educ. 2007;35:404–409. doi: 10.1002/bmb.127. [DOI] [PubMed] [Google Scholar]
- 22.Kiboss JK, Ndirangu M, Wekesa EW. Effectiveness of a computer-mediated simulations program in school biology on pupils’ learning outcomes in cell theory. J. Sci. Educ. Technol. 2004;13:207–213. doi: 10.1023/B:JOST.0000031259.76872.f1. [DOI] [Google Scholar]
- 23.Klymkowsky MW. Teaching without a textbook: strategies to focus learning on fundamental concepts and scientific process. CBE-Life Sci. Educ. 2007;6:190–193. doi: 10.1187/cbe.07-06-0038. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Kohorst K, Cox JR. Virutal office hours using a tablet PC: e-illuminating biochemistry in an online environment. Biochem. Molec. Biol. Educ. 2007;35:193–197. doi: 10.1002/bmb.50. [DOI] [PubMed] [Google Scholar]
- 25.Kozma RB. Will media influence learning? Reframing the debate. Educ. Technol. Res. Dev. 1994;42:7–19. doi: 10.1007/BF02299087. [DOI] [Google Scholar]
- 26.Latham LG, II, Scully EP. Critters! A realistic simulation for teaching evolutionary biology. Am. Biol. Teach. 2008;70:30–33. doi: 10.1662/0002-7685(2008)70[30:CARSFT]2.0.CO;2. [DOI] [Google Scholar]
- 27.Lord T, Baviskar S. Moving students from information recitation to information understanding: exploiting Bloom’s taxonomy in creating science questions. J. Coll. Sci. Teach. 2007;36:40–44. [Google Scholar]
- 28.Lyles H, Robertson B, Mangino M, Cox JR. Audio podcasting in a tablet PC-enhanced biochemistry course. Biochem. Molec. Biol. Educ. 2007;35:456–461. doi: 10.1002/bmb.115. [DOI] [PubMed] [Google Scholar]
- 29.Mickle JE, Aune PM. Development of a laboratory course in nonmajors General Biology for distance education. J. Coll. Sci. Teach. 2008;37:35–39. [Google Scholar]
- 30.Modell HW, Michael J, Wenderoth MP. Helping the learner to learn: the role of uncovering misconceptions. Am. Biol. Teach. 2005;67:20–26. doi: 10.1662/0002-7685(2005)067[0020:HTLTLT]2.0.CO;2. [DOI] [Google Scholar]
- 31.Murray L, Gibson D, Ward A. Real-time ocean data in the classroom. Sci. Teach. 2008;75:44–48. [Google Scholar]
- 32.Novak JD. The promise of new ideas and new technology for improving teaching and learning. Cell Biol. Educ. 2003;2:122–132. doi: 10.1187/cbe.02-11-0059. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Pellegrino JW, Goldman SR. Be careful what you wish for—You may get it: educational research in the spotlight. Educ. Res. 2002;31:15–17. doi: 10.3102/0013189X031008015. [DOI] [Google Scholar]
- 34.Preszler RW, Dawe A, Shuster CB, Shuster M. Assessment of the effects of student response systems on student learning and attitudes over a broad range of biology courses. CBE–Life Sci. Educ. 2007;6:29–41. doi: 10.1187/cbe.06-09-0190. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Reiff JC. Learning styles. National Education Association; Washington, DC: 1992. [Google Scholar]
- 36.Ringstaff C, Kelley L. The learning return on our educational technology investment: a review of findings from research. WestEd RTEC; San Francisco, CA: 2002. http://www.wested.org/cs/we/view/rs/619. [Google Scholar]
- 37.Roblyer MD, Knezek GA. New millennium research for educational technology: a call for a national research agenda. J. Res. Technol. Educ. 2003;36:60–71. [Google Scholar]
- 38.Shavelson RJ, Towne L, editors. Committee on Scientific Principles for Educational Research. National Academy Press; Washington, DC: 2002. Scientific research in education. [Google Scholar]
- 39.Strudler N. Answering the call: a response to Roblyer and Knezek. J. Res. Technol. Educ. 2003;36:72–76. [Google Scholar]
- 40.Surry DW, Ensminger D. What’s wrong with media comparison studies? Educ. Technol. 2001;41:32–35. [Google Scholar]
- 41.Traver HA, Kalshery MJ, Diwan JJ, Warden J. Student reactions and learning: evaluation of a biochemistry course that uses web technology and student collaboration. Biochem. Molec. Biol. Educ. 2001;29:50–53. [Google Scholar]
- 42.Wenger E. Communities of practice: learning, meaning, and identity. Cambridge University Press; Cambridge, UK: 1998. [Google Scholar]
- 43.Zumbach J, Schmitt S, Reimann P, Starkloff P. Learning life sciences: design and development of a virtual molecular biology learning lab. J. Comput. Math. Sci. Teach. 2006;25:281–300. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Appendix 1: Comprehensive Literature Search of Technology Use in Undergraduate Biology Education