Abstract
This project addresses the need to provide a visual context to teach the practical applications of genome sequencing and bioinformatics. Present-day research relies on indirect visualization techniques (e.g., fluorescence-labeling of DNA in sequencing reactions) and sophisticated computer analysis. Such methods are impractical and prohibitively expensive for laboratory classes. More importantly, there is a need for curriculum resources that visually demonstrate the application of genome sequence information rather than the DNA sequencing methodology itself. This project is a computer-based lesson plan that engages students in collaborative, problem-based learning. The specific example focuses on approaches to Human Immunodeficiency Virus-1 (HIV-1) vaccine design based on HIV-1 genome sequences using a case study. Students performed comparative alignments of variant HIV-1 sequences available from a public database. Students then examined the consequences of HIV-1 mutations by applying the alignments to three-dimensional images of the HIV-1 envelope protein structure, thus visualizing the implications for applications such as vaccine design. The lesson enhances problem solving through the application of one type of information (genomic or protein sequence) into concrete visual conceptualizations. Assessment of student comprehension and problem-solving ability revealed marked improvement after the computer tutorial. Furthermore, contextual presentation of these concepts within a case study resulted in student responses that demonstrated higher levels of cognitive ability than was expected by the instructor.
The completion of the human genome sequence was celebrated not only amongst the scientific community but was indeed a globally newsworthy event. Yet, exactly how one utilizes genomic sequence information in a practical context is beyond the comprehension not only of the average citizen but also eludes many undergraduate students. How can educators effectively teach the various applications of a genome sequence? Furthermore, how can students employ bioinformatics, which has become a prevalent word in the public vernacular, in a relevant context?
Many educators and scientists have witnessed the positive effects on student learning of problem-based and collaborative learning approaches. Charlin and colleagues emphasize a learner-centered approach towards problem solving as being of key importance and define four principles related to its effect on learning (3):
Learners are active processors of information;
Prior knowledge is activated and new knowledge is built on it;
Knowledge is acquired in a meaningful context;
Learners have opportunities for elaboration and organization of knowledge.
These principles are indeed reflective of the importance of contextual relevance for students who are presented with abstract concepts (4, 32). Furthermore, the creation of active-learning opportunities has been suggested to facilitate the development of critical thinking skills in biology students (7, 9, 10, 12, 23). This project is a collaborative computer-based laboratory lesson that enables students to analyze information that may be perceived as relatively challenging and put it into a practical context. Its versatility makes it adaptable to most courses in the biological sciences. A case study examining mutations in HIV-1 genome sequences obtained from a cohort of HIV-seropositive patients was presented to 54 third-year virology students. Students applied information obtained from bioinformatics to analyze the high genomic sequence mutation rates of HIV-1 with respect to epidemiology, molecular structure, and therapeutic design strategy. Based on these analyses, students developed three-dimensional models of the viral protein structures that allowed visual mapping of sequence evolution. This approach facilitated students’ appreciation for the application of genomic sequence information into a practical context.
Students collaborated to identify a region of the genome that mutated less frequently and would therefore be a target candidate for vaccine development. This active problem-solving approach enabled students to gain an appreciation for the challenge of developing vaccines or therapeutic drugs for a virus such as HIV-1, which mutates so frequently. The lesson was developed to facilitate critical thinking through the application of one type of information (DNA or protein sequence) into three-dimensional models.
One of the key criteria for an authentic learning experience is fidelity of context (11, 22, 28, 34). The goal of this project was to engage students in an open-ended learning experience based in a relevant context. In addition to strengthening students’ general inquiry-based research skills, the presentation of areas such as bioinformatics and genomics as a case study allows students to reflect on the particular relevance of highly analytical subjects.
METHODS
The total time devoted to the lesson was two 3-hour computer laboratory sessions, preceded by a 1-hour tutorial. The key concepts upon which the lesson is based are:
HIV-1 exhibits a high mutation rate during genome replication due to the low fidelity of the virus-specific enzyme, reverse transcriptase.
A population of HIV-1-infected individuals may display wide genetic heterogeneity of viral subtype nucleotide sequences; these patterns can change over time.
Bioinformatics enables analysis of specific patterns of changes in genomic sequences that can reveal crucial information about the molecular biology and epidemiology of HIV-1.
The changes in viral gene sequence may significantly affect the structure of viral proteins and hence present challenging implications with regard to targets for vaccine and/or therapeutic design.
The lesson is based on a published study (19) in which the pattern of HIV-1 evolution was compared to CD4+ T cell decline in 15 subjects. In the study, the subjects were followed from seroconversion for up to 4 years (at 6-month intervals). Mutations of clonal variants of HIV-1 (clade B) were analyzed via PCR amplification of a 285 bp fragment from the hypervariable V3 loop region of the env gene, encoding the viral glycoprotein gp120.
The background information was presented to the students as a case study scenario:
Background, HIV-1 evolution. HIV-1, the causative agent of AIDS, exhibits high mutation and replication rates that facilitate its adaptation to changes in the host environment as well as its eventual resistance to certain antiretroviral therapies. Like other retroviruses, HIV-1 has a much higher mutation rate than is typically found in organisms whose genomic replication does not involve reverse transcription. The in vivo forward mutation rate in HIV-1 has been estimated to be 3.4 x 10−5 mutations/base pair/replication cycle (5, 18, 20, 27).
HIV-1 infected individuals may display wide genetic heterogeneity (diversity) of viral subtype nucleotide sequences. The host environment can affect the genetic composition of a given virus pool. For example, instability could be generated by a dynamic host immune response or by differential display of coreceptors. If the destabilising force selected randomly against the broad range of existing variants, diversity would most likely be reduced to those variants that were initially most numerous. On the other hand, if selective forces such as the immune response targeted primarily those variants that were most abundant (frequency-dependent selection), overall viral load would be reduced but genetic diversity would be maintained, since the less frequent viral strains would still be present.
By examining patterns of diversity during HIV-1 evolution, we can observe the type and efficiency of selection forces influencing viral evolution, as well as how the virus adapts to those forces. Furthermore, the analysis of frequency and diversity of viral mutation provides valuable information for the framework of potential drug therapy-vaccine development for a given population.
Case study. A study has been conducted on the relationships between the pattern of HIV-1 evolution in 15 seroconverting patients and the rate of CD4+ T cell decline. CD4+ T cell depletion is a characteristic of HIV-1 infection, as the virus infects CD4+ T cells. The changes were monitored at frequent intervals over a period of up to 4 years. The individuals selected for the study were followed from the point of HIV-1 seroconversion and had attained different levels of CD4+ T cells.
Genetic sequence variation was analyzed for a 285 base pair region around the third hypervariable (V3) domain of the viral env gene. The env gene product, membrane protein gp120, binds to the CD4 receptor site on T lymphocytes and is involved with viral entry into the cell. This region was therefore chosen for analysis because it is an important site of host-virus interaction and is known to tolerate frequent mutations. Blood samples were collected from the 15 subjects at 6-month intervals for up to 4 years and analyzed for virologic and immunologic studies. PCR was used to amplify the 285 bp region from peripheral blood mono-nuclear cells. The PCR-amplified DNA fragments were cloned into a plasmid and sequenced. The results for the variant clones are tabulated on the following pages.
The students, working in pairs, were assigned one subject for their case study. Each pair of students was presented with a tabulated summary of data tracking all subjects’ disease progression (represented as CD4+ T cell counts) and number of HIV-1 clones detected in each subject at specific visits (sample of data shown in Table 1). Each pair of students also received the Genbank accession numbers for all of the HIV-1 env sequences from each visit for their subject (available at http://www.microbelibrary.org/Journal/TakayamaGenbankAccession.pdf).
TABLE 1.
Summary of data on subjects analysed for HIV-1 mutation and CD4+ cell count*

| Subject | Total number of visits | Total number of clones | Visit number | Number of clones | CD4+count |
|---|---|---|---|---|---|
| 1 | 3 | 42 | 1 | 13 | 464 |
| 2 | 16 | 305 | |||
| 5 | 13 | 15 | |||
| 2 | 3 | 24 | 1 | 6 | 715 |
| 3 | 9 | 825 | |||
| 4 | 9 | 830 | |||
| 3 | 5 | 39 | 1 | 4 | 819 |
| 3 | 10 | 375 | |||
| 4 | 9 | 265 | |||
| 5 | 10 | 100 | |||
| 6 | 6 | 45 | |||
| 4 | 4 | 47 | 1 | 3 | 1028 |
| 2 | 13 | 710 | |||
| 3 | 18 | 470 | |||
| 4 | 13 | 135 | |||
| 5 | 5 | 43 | 1 | 8 | 749 |
| 2 | 12 | 770 | |||
| 3 | 11 | 650 | |||
| 4 | 7 | 550 | |||
| 5 | 5 | 700 | |||
| 6 | 7 | 54 | 1 | 3 | 405 |
| 2 | 3 | 225 | |||
| 3 | 9 | 350 | |||
| 4 | 12 | 390 | |||
| 5 | 9 | 475 | |||
| 7 | 9 | 400 | |||
| 9 | 9 | 560 | |||
| 7 | 5 | 43 | 1 | 10 | 1072 |
| 2 | 7 | 735 | |||
| 3 | 8 | 330 | |||
| 4 | 9 | 270 | |||
| 5 | 9 | 310 | |||
| 8 | 7 | 49 | 1 | 5 | 538 |
| 2 | 5 | 800 | |||
| 3 | 7 | 605 | |||
| 4 | 6 | 510 | |||
| 5 | 6 | 625 | |||
| 6 | 10 | 515 | |||
| 7 | 10 | 250 | |||
| 9 | 8 | 64 | 1 | 5 | 489 |
| 2 | 5 | 485 | |||
| 3 | 8 | 440 | |||
| 4 | 11 | 370 | |||
| 5 | 9 | 365 | |||
| 6 | 8 | 665 | |||
| 7 | 10 | 555 | |||
| 8 | 8 | 270 | |||
| 10 | 5 | 49 | 1 | 7 | 833 |
| 2 | 6 | 850 | |||
| 4 | 16 | 420 | |||
| 5 | 10 | 150 | |||
| 6 | 10 | 15 | |||
| 11 | 4 | 32 | 1 | 7 | 753 |
| 2 | 6 | 600 | |||
| 3 | 10 | 270 | |||
| 4 | 9 | 175 | |||
| 12 | 6 | 37 | 1 | 4 | 772 |
| 2 | 4 | 780 | |||
| 3 | 5 | 1285 | |||
| 4 | 6 | 1030 | |||
| 5 | 10 | 1395 | |||
| 8 | 8 | 850 | |||
| 13 | 5 | 26 | 1 | 4 | 671 |
| 2 | 2 | 825 | |||
| 3 | 7 | 835 | |||
| 4 | 7 | 770 | |||
| 5 | 6 | 975 | |||
| 14 | 9 | 77 | 1 | 6 | 523 |
| 2 | 6 | 580 | |||
| 3 | 6 | 570 | |||
| 4 | 10 | 595 | |||
| 5 | 7 | 460 | |||
| 6 | 11 | 420 | |||
| 7 | 10 | 460 | |||
| 8 | 9 | 450 | |||
| 9 | 12 | 350 | |||
| 15 | 4 | 40 | 1 | 12 | 707 |
| 2 | 9 | 250 | |||
| 3 | 9 | 75 | |||
| 4 | 10 | 15 |
Some of the visit numbers are not sequential. In all cases visit 1 represents the first time the subject was evaluated. The subsequent time points represent six-month intervals from the initial visit. Thus, if a subject missed their six-month appointment their visits would be numbered 1, 3, 4, etc. (Reference: 19).
The students are then presented with the following challenge:
Research project. You are working with molecular virologists, clinical virologists, and physicians to develop an improved drug therapy regimen for HIV-1 patients. The env sequences analyzed in this study encode a protein that binds to a receptor on the T-cell surface, allowing the virus to enter and ultimately destroy the cell (and consequently the body’s immune capabilities). Developing drugs that block this binding have had limited short-term success, the major impediment being the rapid mutation rate of the env gene, which renders highly specific blockers ineffective. You will examine patterns in the mutations occurring in these env sequences over time.
Each pair of students initially discussed the data for all subjects to examine and critique the available information with regard to potential patterns of correlations. The students then performed genomic sequence alignment analyses (for the 285 bp V3 region) of all viral clones for their subject at the nucleotide (Fig. 1) and protein sequence levels (Fig. 2). The sequences are available on the Genbank public database, and students employed the open source web-based program Biology Workbench (http://workbench.sdsc.edu/) to perform their analyses.
FIG. 1.
Biology Workbench nucleotide sequence alignment demonstrating subject 1 HIV-1 clone diversity at visit number 1. The alignment was obtained using the boxshade function in Biology Workbench.
FIG. 2.
Biology Workbench protein sequence alignment demonstrating subject 1 HIV-1 clone diversity at visit number 1. The alignment was obtained using the boxshade function in Biology Workbench.
Students engaged in collaborative discussions to analyze their sequence alignments for their subjects using the following questions as a guide:
Questions.
Upon analysis of nucleotide mutations over time for each of the clones in your subject, are there specific nucleotide positions that mutate more frequently than others?
Do they mutate in predictable ways (have the same change) over time?
How do the mutation patterns of the protein sequences compare to those of the nucleotide sequences?
Based on your analyses, which viral clones are most closely related evolutionarily? Which clones are least related evolutionarily?
Discuss within your laboratory group and then as a whole class: if you were to design a drug or vaccine to target a specific portion(s) of the HIV-1 env glycoprotein, which region would you target based on your sequence analyses? Why?
In the second half of the lesson the open source web-based program, Protein Explorer (21), was utilized to model protein sequence alignments and test the students’ predictions. Protein Explorer is a powerful program that enables three-dimensional (3-D) modeling and visualization of protein structures. Multiple sequence alignments were mapped using Protein Explorer and ConSurf (8) to identify amino acid residue positions that remained highly conserved in comparison to those that mutated significantly (Fig. 3). Students compared protein sequence alignments of their subjects’ gp120 sequences against the sequence of one of several gp120 protein structures available in the Protein Data Bank (PDB) (1) (e.g., PDB accession number 1G9M (15, 16)). The interactivity of Protein Explorer allows students to rotate the 3-D structure of their gp120 alignment, zoom in and out, and highlight specific polypeptide chains as well as specific amino acid residues. Furthermore, the PDB structures of the gp120 chain are represented together with the CD4 receptor and an epitope-specific antibody, enabling students to examine potential steric interactions and interceptions between specific exposed regions of gp120.
FIG. 3.
3-D structure summarizing data from multiple sequence alignment of subject 1, visit 1 HIV-1 protein sequences modeled on HIV-1 gp120 core complexed with the CD4 receptor and a neutralizing human antibody. Magenta (spacefill) residues represent highly conserved regions; blue (spacefill) residues represent variable regions of gp120. The CD4 receptor, antibody light chain, and antibody heavy chain are indicated.
OUTCOMES AND EVALUATION
Students were able to examine the patterns of HIV-1 evolution in patients by analyzing the nucleotide sequences from a region of the env gene. They made predictions about which regions of the corresponding amino acid sequence may serve as potential targets for drug or vaccine design strategy. However, comparative sequence analysis on its own cannot provide structural information that is critical for scientists involved in pharmaceutical design. Furthermore, for the student to fully appreciate the consequences of the high mutational frequencies of HIV-1, the visualization of these events in three dimensions provides an invaluable comparative resource. Students were able to appreciate the functional consequences of mutations in specific regions of the HIV-1 genome by visualizing the structural effects in 3-D representations of gp120. Indeed, other educators have also reported that molecular visualizations are effective in facilitating student comprehension of protein structure and function (6, 29, 30, 33, 36).
Learning outcomes were determined by defining specific learning objectives for the lesson and analyzing students’ responses to conceptual questions designed to assess achievement of those objectives. Performance assessments judge student abilities to use specific knowledge and research skills to solve a problem or make an analysis (17, 31, 35). Student comprehension and problem-solving ability was assessed using the performance assessment questions. Questions were graded by the instructor. A summary is presented in Table 2.
TABLE 2.
Student performance objectives and assessment questions.
| Objective | Conceptual Question |
|---|---|
| A. To understand the biological relevance of the env gene | A. One primary target in HIV-1 that comes to mind when developing a drug or vaccine is the env gene, which encodes gp120. Why? |
| B. To understand how bioinformatics is utilized for genomic analysis. | B. What, in your understanding, does bioinformatics reveal to us about a genomic sequence of interest? |
| C. To apply conceptual knowledge towards developing a computer-based method to solve a bioinformatics problem. | C. If you were using bioinformatics to determine the most appropriate target for vaccine development against a region of gp120, draw a flowchart depicting your approach. Assume you have all the necessary bioinformatics analysis software and databases at your disposal. |
| D. To understand that comparative analyses of gene and protein sequences alone are insufficient for providing comprehensive biological functional information; i.e., structural information is also needed. | D. One of your colleagues has performed sequence alignments of the env gene sequence, and the gp120 protein sequence. She has decided on a vaccine target region based on this information. Please provide your critique on this. |
Students were asked to answer these questions before and after the computer laboratory sessions. The questions were designed to assess different levels of cognitive ability, focusing on the central theme of the application of genomics. Student responses were scored on a scale of 1 to 4, using the following rubric for classifying comprehension: 1, little or no understanding of the topic, failed to meet objective; 2, some understanding of topic, answer had some relevance towards meeting objective; 3, sufficient understanding of topic, answer is generally correct but lacks strategic or innovative detail; 4, outstanding understanding of topic, answer indicative of further reflection and application (transfer) of concepts. The outcomes are presented in Table 3.
TABLE 3.
Student outcomes of performance assessment
| Level of student cognitive ability1 | ||||
|---|---|---|---|---|
| 1 | 2 | 3 | 4 | |
| Objective A | ||||
| Scores before computer session2 | 13 | 22 | 16 | 2 |
| Scores upon completion of computer session3 | 0 | 0 | 34 | 17 |
| Level of significance (χ2 test): P< 0.001 | ||||
| Objective B | ||||
| Scores before computer session2 | 44 | 44 | 6 | 6 |
| Scores upon completion of computer session3 | 0 | 31 | 44 | 25 |
| Level of significance (χ2 test): P< 0.001 | ||||
| Objective C | ||||
| Scores before computer session2 | 50 | 29 | 18 | 3 |
| Scores upon completion of computer session3 | 0 | 0 | 69 | 31 |
| Level of significance (χ2 test): P< 0.001 | ||||
| Objective D | ||||
| Scores before computer session2 | 50 | 35 | 12 | 3 |
| Scores upon completion of computer session3 | 10 | 0 | 55 | 35 |
| Level of significance (χ2 test): P< 0.001 | ||||
Student responses were scored on a scale of 1 – 4 (refer to text for rubric definitions); values indicate the number of students whose responses to a specific objective question corresponded to that cognitive category.
A total of 53 students completed the assessment questions before the computer session.
A total of 51 students completed the assessment questions after the computer session.
For each objective, student performance improved markedly following the computer tutorial. The outcomes indicate that student comprehension of the application of bioinformatics is enhanced by this investigative approach. Furthermore, because the objectives were presented within a case study, emphasizing relevance and context, some answers extended beyond the expectations of the instructor. For example, for question D, a student replied, “In order to ensure that the potential vaccine target region is present on an exposed surface where it can be recognized by the immune system, protein modeling software may be used to predict the protein structure. Further in vitro and in vivo testing must be conducted to ensure that the target region can be expressed in a functional folded form that closely resembles the gp120 epitope and the target region is immunogenic.”
The value of 3-D computer modeling to examine the consequences of the gp120 mutations was appreciated by students. The students were able to explore the structure and crucial regions involved in receptor binding and recognition. Furthermore, the collaborative framework of the learning experience enhanced the students’ problem-solving strategies. Other studies have also demonstrated the successful outcomes of collaborative and problem-based learning mediated by technology (13, 24, 25, 26). The computer project described here can certainly be used in conjunction with comprehensive supplementary introductions to the areas of bioinformatics and functional genomics (such as those described in 2, 14) for further in-depth exploration of specific protein families or evolutionary and functional relationships among conserved proteins.
Acknowledgments
I wish to acknowledge the support of the University of New South Wales Innovative Teaching and Educational Technology Fellowship program, which allowed me to develop this project. The idea for this lesson plan evolved from a tutorial, “Exploring HIV Evolution,” presented in June 2001 by Sam Donavan at the Bioquest workshop, “Microbes Count: Problem Posing, Problem Solving, and Persuading Peers in Microbiology Education.” I am grateful to Bioquest for the opportunity to participate in the workshop and to Eric Martz, University of Massachusetts—Amherst, for introducing me to Protein Explorer.
REFERENCES
- 1.Berman HM, Westbrook J, Feng Z, Gilliland G, Bhat TN, Weissig H, Shindyalov IN, Bourne PE. The protein data bank. Nucleic Acids Res. 2000;28:235–242. doi: 10.1093/nar/28.1.235. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Campbell AM, Heyer LJ. Discovering genomics, proteomics, and bioinformatics. Benjamin Cummings (Pearson Education); San Francisco, Calif: 2003. [Google Scholar]
- 3.Charlin B, Mann K, Hansen P. The many faces of problem-based learning: a framework for understanding and comparison. Med Teacher. 1998;20:323–330. doi: 10.1080/01421599880742. [DOI] [Google Scholar]
- 4.Chinn CA, Malhotra BA. Epistemologically authentic inquiry in schools: a theoretical framework for evaluating inquiry tasks. Sci Educ. 2002;86:175–218. doi: 10.1002/sce.10001. [DOI] [Google Scholar]
- 5.Coffin JM. HIV population dynamics in vivo: implications for genetic variation, pathogenesis, and therapy. Science. 1995;267:483–489. doi: 10.1126/science.7824947. [DOI] [PubMed] [Google Scholar]
- 6.Dori YJ, Barak M. Virtual and physical molecular modeling: fostering model perception and spatial understanding. Educ Technol Society. 2001;4:61–74. [Google Scholar]
- 7.Douthright J. Undergraduate microbiology curriculum recommendations. ASM News. 1994;60:460–461. [Google Scholar]
- 8.Glaser F, Pupko T, Paz I, Bell RE, Bechor-Shental D, Martz E, Ben-Tal N. ConSurf: identification of functional regions in proteins by surface mapping of phylogenetic information. Bioinformatics. 2003;19:163–164. doi: 10.1093/bioinformatics/19.1.163. [Online.] http://consurf.tau.ac.il/ [DOI] [PubMed] [Google Scholar]
- 9.Goodwin L, Miller JE, Cheetham RD. Teaching freshmen to think—does active learning work? BioScience. 1991;41:719–722. doi: 10.2307/1311767. [DOI] [Google Scholar]
- 10.Harland T. Vygotsky’s zone of proximal development and problem-based learning: linking a theoretical concept with practice through action research. Teaching Higher Educ. 2003;8:263–272. doi: 10.1080/1356251032000052483. [DOI] [Google Scholar]
- 11.Herrington J, Herrington A. Authentic assessment and multimedia: how university students respond to a model of authentic assessment. Higher Educ Res Dev. 1998;17:305–322. doi: 10.1080/0729436980170304. [DOI] [Google Scholar]
- 12.Hoffman EA. Successful application of active learning techniques to introductory microbiology. Microbiol Educ. 2001;2:5–11. doi: 10.1128/me.2.1.5-11.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Hollingsworth RW, McLoughlin C. Developing science students’ metacognitive problem solving skills online. Australian J Educ Technol. 2001;17:50–63. [Google Scholar]
- 14.Krane DE, Raymer ML. Fundamental concepts of bioinformatics. Benjamin Cummings (Pearson Education); San Francisco, Calif: 2003. [Google Scholar]
- 15.Kwong PD, Wyatt R, Majeed S, Robinson J, Sweet RW, Sodroski J, Hendrickson WA. Structures of HIV-1 Gp120 envelope glycoproteins from laboratory-adapted and primary isolates. Structure. 2000;8:1329. doi: 10.1016/S0969-2126(00)00547-5. [DOI] [PubMed] [Google Scholar]
- 16.Kwong PD, Wyatt R, Robinson J, Sweet RW, Sodroski J, Hendrickson WA. Structure of an HIV gp120 envelope glycoprotein in complex with the CD4 receptor and a neutralizing human antibody. Nature. 1998;393:648–659. doi: 10.1038/31405. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Linn RL, Baker EL, Dunbar SB. Complex, performance-based assessment: expectations and validation criteria. Educ Res. 1991;20:15–21. [Google Scholar]
- 18.Mansky LM, Temin HM. Lower in vivo mutation rate of human immunodeficiency virus type 1 than that predicted from the fidelity of purified reverse transcriptase. J Virol. 1995;69:5087–5094. doi: 10.1128/jvi.69.8.5087-5094.1995. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Markham RB, Wang W-C, Weisstein AE, Wang Z, Munoz A, Templeton A, Margolick J, Vlahov D, Quinn T, Farzadegan H, Yu X-F. Patterns of HIV-1 evolution in individuals with differing rates of CD4 T cell decline. Proc Natl Acad Sci USA. 1998;95:12568–12573. doi: 10.1073/pnas.95.21.12568. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Martinez MA, Sala M, Vartanian JP, Wain-Hobson S. Reverse transcriptase and substrate dependence of the RNA hypermutagenesis reaction. Nucleic Acids Res. 1995;14:2573–2578. doi: 10.1093/nar/23.14.2573. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Martz E. Protein Explorer: easy yet powerful macromolecular visualization. Trends Biochem Sci. 2002;27:107–109. doi: 10.1016/S0968-0004(01)02008-4. [Online.] http://molvis.sdsc.edu/protexpl/index.htm/ [DOI] [PubMed] [Google Scholar]
- 22.Meyer CA. What’s the difference between authentic and performance assessment? Educ Leadership. 1992;49:39–40. [Google Scholar]
- 23.Miller JE, Cheetham RD. Teaching freshmen to think—active learning in introductory biology. BioScience. 1990;40:388–391. doi: 10.2307/1311217. [DOI] [Google Scholar]
- 24.Mulder I, Swaak J, Kessels J. Assessing group learning and shared understanding in technology-mediated interaction. Educ Technol Society. 2002;5:35–47. [Google Scholar]
- 25.Oliver R, Omari A. Using online technologies to support problem-based learning: learners’ responses and perceptions. Australian J Educ Technol. 1999;15:58–79. [Google Scholar]
- 26.Palloff RM, Pratt K. Building learning communities in cyberspace: effective strategies for the online classroom. Jossey-Bass; San Francisco, Calif: 1999. pp. 110–128. [Google Scholar]
- 27.Preston BD, Dougherty JP. Mechanisms of retroviral mutation. Trends Microbiol. 1996;4:16–21. doi: 10.1016/0966-842X(96)81500-9. [DOI] [PubMed] [Google Scholar]
- 28.Reeves TC, Okey JR. Alternative assessment for constructivist learning environments. In: Wilson BG, editor. Constructivist learning environments: case studies in instructional design. Educational Technology Publications; Englewood Cliffs, N.J: 1996. pp. 191–202. [Google Scholar]
- 29.Richardson DC, Richardson JS. Teaching molecular 3-D literacy. Biochem Mol Biol Educ. 2002;30:21–26. doi: 10.1002/bmb.2002.494030010005. [DOI] [Google Scholar]
- 30.Sears DW. Using inquiry-based exercises and interactive visuals to teach protein structure/function relationships. Biochem Mol Biol Educ. 2002;30:208. doi: 10.1002/bmb.2002.494030030081. [DOI] [Google Scholar]
- 31.Shavelson RJ, Baxter GP, Pine J. Performance assessment in science. Appl Measurement Educ. 1991;4:347. doi: 10.1207/s15324818ame0404_7. [DOI] [Google Scholar]
- 32.Tobias S, Hake RR. Professors as physics students: what can they teach us? Am J Phys. 1988;56:786–794. doi: 10.1119/1.15486. [DOI] [Google Scholar]
- 33.White B, Kim S, Sherman K, Weber N. Evaluation of molecular visualization software for teaching protein structure: differing outcomes from lecture and lab. Biochem Mol Biol Educ. 2002;30:130–136. doi: 10.1002/bmb.2002.494030020026. [DOI] [Google Scholar]
- 34.Wiggins G. Assessing student performance: exploring the purpose and limits of testing. Jossey-Bass; San Francisco, Calif: 1993. [Google Scholar]
- 35.Wiggins G. A true test: toward a more authentic and equitable assessment. Phi Delta Kappan. 1989;70:703. [Google Scholar]
- 36.Willian KR. Using three-dimensional imaging of proteins: examples of class activities and subsequent assessments. Biochem Mol Biol Educ. 2002;30:209–210. doi: 10.1002/bmb.2002.494030030080. [DOI] [Google Scholar]



