Use of a wiki as an interactive teaching tool in pathology residency education: Experience with a genomics, research, and informatics in pathology course

Abstract

Background:

The need for informatics and genomics training in pathology is critical, yet limited resources for such training are available. In this study we sought to critically test the hypothesis that the incorporation of a wiki (a collaborative writing and publication tool with roots in “Web 2.0”) in a combined informatics and genomics course could both (1) serve as an interactive, collaborative educational resource and reference and (2) actively engage trainees by requiring the creation and sharing of educational materials.

Materials and Methods:

A 2-week full-time course at our institution covering genomics, research, and pathology informatics (GRIP) was taught by 36 faculty to 18 second- and third-year pathology residents. The course content included didactic lectures and hands-on demonstrations of technology (e.g., whole-slide scanning, telepathology, and statistics software). Attendees were given pre- and posttests. Residents were trained to use wiki technology (MediaWiki) and requested to construct a wiki about the GRIP course by writing comprehensive online review articles on assigned lectures. To gauge effectiveness, pretest and posttest scores for our course were compared with scores from the previous 7 years from the predecessor course (limited to informatics) given at our institution that did not utilize wikis.

Results:

Residents constructed 59 peer-reviewed collaborative wiki articles. This group showed a 25% improvement (standard deviation 12%) in test scores, which was greater than the 16% delta recorded in the prior 7 years of our predecessor course (P = 0.006).

Conclusions:

Our use of wiki technology provided a wiki containing high-quality content that will form the basis of future pathology informatics and genomics courses and proved to be an effective teaching tool, as evidenced by the significant rise in our resident posttest scores. Data from this project provide support for the notion that active participation in content creation is an effective mechanism for mastery of content. Future residents taking this course will continue to build on this wiki, keeping content current, and thereby benefit from this collaborative teaching tool.

BACKGROUND

Advances in computing power[1] have permitted concomitant advances in educational techniques such as computerized simulation of patients,[2] delivery of lectures (e-Education) to handheld devices via streaming video servers,[3] computerized standardized testing,[4] creation of digital atlases and virtual slide teaching sets,[5] and the development of whole slide image search maps that are able to track and tutor trainees.[6]

As we enter the era of “Web 2.0,”[7] we have seen an increasing tendency toward collaborative web editing and publication technologies such as wiki[8] and blogging software.[9] These technologies have changed the way that users interact with content. While in the past the vast majority of published content came from professionals using technologies that were both expensive and extremely complex (for instance, the average content consumer of 1980 would not have had the money to buy or the expertise to operate the powerful but technically cumbersome dedicated video editing machines of the day), the current abundance of affordable computing power has allowed for a democratization of content creation. This democratization has led to widespread availability of user-created content online – a phenomenon that has thrilled some observers[10] and terrified others.[11]

Wiki software is of particular interest to medical education because of its emphasis on large-scale collaboration and publication of articles (as opposed to the more personal, single-source nature of blogs). There are many different kinds of wiki software, each built for a different purpose (MediaWiki, for instance, targets large encyclopedia projects, whereas dokuWiki is focused on small-team collaboration). While the specifics of each wiki software package may differ, at the foundation of all wiki software is a “stack” of software that includes, at a bare minimum, an operating system (OS), web server software, a database management system (DBMS), and a server-side scripting language. While it is beyond the scope of this article to discuss the technical implementation of such stacks, it is important to have some idea of how a stack's components interact with one another. The OS is closest to the hardware; its application programming interfaces (APIs) are the basis of all interaction between the rest of the software stack and the computer hardware the stack runs on. The web server software is installed atop the OS, and in its purest form implements the Hypertext Transfer Protocol (HTTP) to allow web presentation of pages written in Hypertext Markup Language (HTML).[12] The DBMS is likewise installed atop the OS, and provides facilities for permanent storage – and easy retrieval – of discrete data elements (e.g., the server-side file locations of media files presented by the wiki).[13] The web server software and DBMS interact with each other (and the user) through the server-side scripting language, which is (a) installed atop the web server software and (b) used to implement the wiki software. The user then accesses and edits the wiki through a web browser.

Each element of the stack (and indeed any interaction in between the elements of the stack) is technically complex, and is usually beyond the capability of the average user to maintain or directly program. The triumph of wiki software lies in its ability to abstract this technical complexity away from the user, presenting the user instead with something closer to a what-you-see-is-what-you-get (WYSIWYG) approach to content generation and editing. In a wiki, the user is presented with a web page written in HTML and a server-side scripting language (e.g., PHP), yet the user does not need to have any knowledge of HTML or experience with a scripting language in order to create or edit a page. Instead, the user utilizes a simplified markup language often called “wikitext” or “wiki markup.” The specifics of wikitext can differ greatly depending on the underlying wiki software, but in general wikitext is much simpler to use than HTML and presents the user with human-readable source text that often has high visual similarity to the way the final page will look in the web browser. Efforts to standardize wikitext (e.g., Creole) exist, but many popular wiki software packages (e.g., MediaWiki) currently have no plans to switch over to these nascent standards. Wikitext has facilities for all commonly used hierarchical text structures (e.g., bullet points, number lists, tables), and also allows the user to create links to pages both internal and external to the wiki. Whenever a user inserts a link to a wiki page that does not yet exist, the link to this page shows up in a different color (red by default) than a link to an already-existing page (blue by default). When a user clicks on such a link, s/he is presented with the option of creating content for the page; if the user opts to do so and saves his/her edits, the page is automatically generated by the wiki software utilizing the user-generated content and the color for all links to this page is changed to indicate that the page now exists. Figure 1 demonstrates the differences between wikitext, HTML, and final onscreen presentation.

Figure 1.

Comparison between wikitext, HTML, and final on-screen presentation. The wikitext variant used here is MediaWiki's. Note that the nonexistent page shows up as a red link in the final presentation – this indicates that this page has not yet been created

While currently the most common use of wiki technology among medical professionals revolves around the consumption and filtering of information provided by the popular web encyclopedia Wikipedia,[14,15] there have also been some efforts to utilize the collaborative nature of wiki software as a teaching tool. These efforts can be divided into three types:[16]

1. Creation of specialized wikis by instructors for student consumption.

2. Student creation or editing of wikis for evaluation by instructors.

3. Wikis as a collaborative learning space for students.

Of these, the third type of effort has been the most prevalent, as exemplified by the University of Minnesota Medical Student Wiki – which has evolved from an online repository of class notes into a collaborative textbook of the preclinical curriculum at the University of Minnesota Medical School[17] – and the Beth Israel Deaconess Internal Medicine Wiki – which has become such a popular resource for administrative and educational content that it has made obsolete the administration-provided Internal Medicine resident intranet page.[18] This is not surprising, as these are tasks for which the egalitarian information-sharing nature of wikis is ideal. Wikis of the first and second types are rare, possibly related to the fact that both demand significant involvement from faculty who may not have sufficient time or technical expertise to invest in such endeavors.

As of 2009, there were at least 69 public medical wikis in existence, ranging in scope from the very broad – e.g., the all-inclusive AskDrWiki – to the very narrow – e.g., the Flu Wiki, which focuses on the symptomatology, pathology, and epidemiology of influenza.[19] These wikis are in various stages of completion, have various amounts of content, and have wildly differing editorial policies – some follow the Wikipedia model and allow anyone to edit anything, and others require contributors to be licensed medical professionals. Our own field of pathology has several wikis [Table 1], but it is telling that the single most successful and comprehensive free online resource written in English for the practice of pathology – Pathology Outlines – utilizes an expert- and peer-review-based, rather than a crowd-sourced, approach to content generation.

Table 1.

Selected pathology informatics-centric wikis

In Pathology, the fields of informatics and genomics are rapidly advancing and consequently creating demands for pathologists with expertise in these areas, as well as a need to train pathology residents and fellows in these novel subspecialties. Recent efforts on the part of the American Medical Informatics Association have culminated in the announcement of the first-ever board certification examinations for the newly recognized subspecialty of Clinical Informatics.[20] The need for informatics and genomics training for pathology residents is now critical, and although existing education projects in these areas – such as the University of Pittsburgh Medical Center's Virtual Rotation in Pathology Informatics,[21] Massachusetts General Hospital's Pathology Informatics Wiki,[22] and Beth Israel Deaconess Medical Center's genomics curriculum[23] – are an impressive start, we have yet to reach a point where reliable, comprehensive and current resources for informatics and/or genomics education are available online. While we at the University of Pittsburgh Medical Center (UPMC) have offered our residents a didactic course in pathology informatics since 2000, it has undergone many changes as the field has rapidly advanced.[21] In 2011 we incorporated content about genomics and related emerging technologies as well as research methodology to create a 2-week course entitled Genomics, Research, and Informatics in Pathology (GRIP). As part of this course, we decided to train our students to use wiki technology, and to task them with creating and maintaining a wiki documenting the GRIP course. The final wiki was to be peer reviewed by the instructors of the course. The dual goal of this effort was (a) to evaluate whether this interactive Web 2.0 mechanism of engaging trainees would help in their education and (b) to create a current and organized database of the course contents that could be offered online and updated annually by future trainees in our program. We hypothesized that “using informatics to teach informatics” would not only encourage active participation from the residents, but would also lead to better retention of course material compared to prior years where this course offered mainly didactic lectures.

MATERIALS AND METHODS

GRIP Course

The inaugural GRIP class consisted of 18 residents from the second and third years of our pathology residency program. One resident was training in Clinical Pathology only, and the others were in the combined Anatomic and Clinical Pathology track. A total of 36 faculty members taught a 56-part curriculum which included didactic lectures, live demonstrations, and hands-on activities [Table 2]. Each resident was assigned to 4 lectures; the expectation was that the residents would be responsible for writing detailed review wiki articles on those lectures with collaborative help from their colleagues. Course instructors were available for questions during and after lectures, with one instructor sitting through the entire course with the residents at all times. Copies of John Sinard's Practical Pathology Informatics: Demystifying Informatics for the Practicing Anatomic Pathologist were lent to all residents, and residents were encouraged – but not required – to use this textbook as a reference. A pretest was given on the first day of the course, and a posttest was given on the final day of the course. The results of these tests were collated and the delta (% improvement) was compared to that obtained by former residents in the prior 7 years who were given similar tests during the informatics course given at our institution (an extremely close virtual analog to this course – complete with lecture videos from the actual course – can be found at https://secure.opi.upmc.edu/VRPI/). This predecessor course did not utilize wikis.

Table 2.

GRIP course topics. Each topic was associated with one or more educationalactivities, including didactic lectures, live demonstrations, and hands-on laboratory sessions

Once the course began, each lecture was recorded using Microsoft Powerpoint's built-in narration recording functionality. At the end of each day, the lectures of that day were converted into Windows Media Video (WMV) files using Microsoft Powerpoint's built-in WMV export functionality. The recorded WMV files were converted to Flash Video (FLV) files using the FFMPEG (http://ffmpeg.org/) video conversion command-line tool. These FLV files were uploaded into the GRIP Wiki by means of MediaWiki's upload file functionality Figure 2 and embedded into individual wiki pages utilizing the MediaWikiPlayer extension and JWPlayer. These recordings were made available to the residents as reference material for their wiki pages, and some lecturers also gave permission for the recordings of their lectures to be included as content in the final public wiki itself Figure 3.

Figure 2.

File upload utility of MediaWiki. The MediaWiki configuration files had to be manually adjusted to allow the large-sized lecture video files to be uploaded

Figure 3.

Wiki page for a lecture with integrated lecture video, link to lecture PDF, and student notes

Wiki Creation

Prior to the beginning of the course, the Bitnami MediaWiki Stack (http://www.bitnami.com) was installed on an HP dc5700 workstation (2.33 GHz Core 2 Duo E6400, 2GB DDR2 SDRAM, 80GB SATA HDD, Microsoft Windows XP SP3). This installation was configured for UPMC intranet use only. The MediaWikiPlayer extension (http://www.mediawiki.org/wiki/Extension:MediawikiPlayer) and JWPlayer (http://www.longtailvideo.com/) were installed to allow embedding of lecture videos in the wiki. A table of contents with links to each lecture of the GRIP course curriculum was generated, as shown in Figures 4 and 5. Residents were all trained to use this wiki technology. They were given a month to complete their wiki contributions. Their contributions were continuously monitored and peer reviewed by course instructors, and collaborative editing sessions were encouraged. Usage of external reference material – including PubMed and Digital Object Identifier (DOI) links to research articles – was allowed and encouraged. Once the resultant articles passed peer review, they were finalized for the purposes of internal use by residents participating in this course in future years. The entire wiki was duplicated onto a different server at this point; this wiki “fork” is under intense development and revision for the purpose of eventual public release.

Figure 4.

Table of Contents page of the GRIP Wiki. Note that all the links are in red – this indicates a preinitialized state in which the individual linked pages have not yet been created

Figure 5.

Markup code of the Table of Contents page. This is the code in which the residents were taught to write for their articles

RESULTS

The residents generated 59 peer-reviewed wiki articles comprising over 3000 collaborative editing sessions. The contributions ranged in length from 296 to 3244 words for a mean of 1191 words. Wiki pages ranged from comprehensive review articles to bullet point lists. Four residents were unable to satisfactorily finish their pages within the 1-month time limit, but these pages were finished within the next month. The server for the project suffered a catastrophic crash once during the course; fortunately no backed up data were ultimately lost and the server was reverted to its precrash state the next day. The group of residents showed a 25% improvement (standard deviation 12%) in their test scores, which was greater than the 16% delta from the prior 7 years of our predecessor course, which did not employ wikis (P = 0.006) [Figure 6]. When informatics content alone was considered, the residents showed a 29.4% improvement (standard deviation 13.7%) in their test scores, which was again greater than the 16% delta from the prior 7 years of our predecessor course (P = 0.04) [Figure 7].

Figure 6.

Comparison of pre-/posttest score deltas between the GRIP residents who utilized a wiki and the pre-GRIP residents who did not utilize a wiki. P = 0.006

Figure 7.

Comparison of pre-/posttest score deltas (informatics only) between the GRIP residents who utilized a wiki and the pre- GRIP residents who did not utilize a wiki. P = 0.04

CONCLUSION AND DISCUSSION

The GRIP Wiki

Our use of a wiki accomplished what it set out to do: it has (a) provided high-quality, peer-reviewed content that includes hyperlinks to key papers in the field and select videos of lectures that will form the basis of future pathology informatics and genomics courses and (b) it proved to be an effective teaching tool, as evidenced by the significant rise in our resident posttest scores and feedback received from our trainees. This paradigm solves many – if not all – of the reliability problems inherent in wikis, and it guarantees a collaborative environment free of any antiacademic bias (see subsection: The Reliability of Wikis and the Wikipedia Controversy). Future iterations of our course will continue to improve on this wiki, continuously adding new content. Our intention is to ultimately derive a public version of this wiki that will be made freely available on the World Wide Web.

However, feedback from course participants revealed that building this wiki took them a significant amount of time. Initially the residents were required to create their wiki pages as their “homework,” but in postcourse evaluations all 18 residents indicated a strong preference for protected time built into the course to work on the wiki. Most residents would have preferred more training on the use of wiki technology than what they received (as this was the first time the majority of them had been exposed to editing a wiki), and over half the residents indicated that they enjoyed the hands-on lab components of the course – for which no wiki articles were made – more than they enjoyed some of the didactic lectures.

That being said, it is clear from the rise in posttest scores that the residents’ hard work was worth the effort – by building the wiki, they learned more in the short term and retained more in the long term as compared to the residents who took our predecessor course without wikis. Residents assigned to didactic lectures would often contact the lecturers for long discussions on the topic material, many times staying after hours to do further research on and refinement of their wiki pages. This suggests that by engaging the lecture material in this novel, collaborative way, the residents – at least for the lectures they were assigned – shifted from passive absorption to active discussion of what was often remarkably complex course material.

Finally, while the initial generation of content for this wiki was rapid, the peer review and final editing process for publication to the Web has been much slower. This relates to a lesson already learned by expert-based collaborative knowledgebase projects such Nupedia and Citizendium: that proper generation of peer-reviewed material is often a laborious and time-consuming process.[24,25] It is our belief that as laborious and time consuming as it may be, this pursuit is worthwhile, and will result in a substantial expansion of the informatics and genomics training resources available for both future students and educators.

Comparison to Other Medical Education Wiki Projects

To the best of our knowledge, our specific usage model for a wiki in medical education is unique. The project that comes closest in both subject material and intended audience is the Massachusetts General Hospital Pathology Informatics Wiki (http://pathinformatics.wikispaces.com/), but our project utilizes an expert, peer-review-based content generation model linked tightly to a full-fledged offline didactic course (indeed, the editors of our project were the participants in this course), whereas the Massachusetts General Hospital Pathology Informatics Wiki focuses largely on culling usable content from Wikipedia.[22] Another wiki that attempts to define a pathology curriculum is the American Society for Clinical Pathology's consensus curriculum for laboratory management training (http://wiki.ascp.org/wikka.php?wakka=Curriculum); this wiki is much like the Massachusetts General Hospital Pathology Informatics Wiki in that it culls and links usable content from sources like Wikipedia and cdc.gov.[26] All of these approaches are valid and are likely to provide useful sources of publically available pathology informatics didactic material.

Wikis that focus on collaborative generation of “residency survival guide” style content have been extremely successful, precisely because these play to the strengths of wikis as collaborative publication platforms. Kohli and Bradshaw reported on one such wiki they developed for radiology resident education: this wiki included “phone numbers, dictation templates, rotation/call information, educational content, schedules, and others (useful links, policies, etc.).” According to this paper, the vast majority of the time residents accessed this wiki (> 80%), they did so for purely administrative purposes – for call schedules, phone numbers, dictation templates, and the like. Educational material was accessed less than 20% of the time.[27] These data corroborate the experience of Crotty, Mostaghini, and Reynolds, who reported on their 3-year experience with the Beth Israel Deaconess Internal Medicine Wiki. This wiki – likewise filled with administrative and other “survival guide” style content – was reported as having a large positive impact on workflow, but only a small impact on education.[18] As successful as these wikis have been, their usage model is very different from ours: we focused completely on resident-led generation of educational material, and demonstrated a large educational impact as a result.

There is also a growing class of wikis in medical education that serve primarily as an intelligence-sharing mechanism between multiple collaborators. Papakonstantinou et al. make a proposal for the usage of such a wiki to serve as a “collective memory” containing material pertinent to user training in healthcare process management, but to the best of our knowledge this has not yet grown beyond the proposal stage.[28] Kardong-Edgren et al. describe the use of another such “collective memory” wiki – this time for tracking of information and data in a nursing CPR education trial – but the group's final report on the results of the trial does not comment on how efficacious their use of wiki technology was.[29,30] Perhaps most relevant to us are two papers describing wikis as collaborative online repositories of medical school lecture notes. Thompson et al. describe one such wiki written and maintained by the students of the University of Minnesota Medical School. This project has had remarkable viewership and participation, and serves as validation of the idea that amazing content creation can occur when the incentives are high and the barriers to participation are low.[17] It would be interesting to see if the usage of this wiki (starting in 2007) has resulted in a collective rise in educational metrics (e.g., test scores) at the University of Minnesota Medical School, but to the best of our knowledge such an analysis has not yet been done. The second such wiki is that of Jalali et al. at the University of Ottawa, but this wiki has met with discouraging results primarily related to the difficulty students had in accessing the wiki and the scarcity of initial content.[31]

Perhaps the project that comes closest in spirit and implementation to ours is Cobus’ experience in using blogs and wikis in a graduate public health course. As part of this course, Cobus’ students used a wiki to collaboratively generate a peer-reviewed interactive bibliography of public health resources. During this activity, students were expected to find, annotate, and critique the reliability of resources – both online and from traditional academic sources – relating to assigned subdisciplines of public health. Their findings were continually peer-reviewed by course supervisors, who graded the students based on the quality and quantity of their contributions.[32] The differences in between Cobus’ approach and ours relate primarily to the difference between a library sciences course and an informatics course: while Cobus is rightfully more concerned with the fine-grained process of retrieving and appraising the reliability of information, we are more focused on imparting information we know to be reliable to our students. Furthermore, while the next iteration of Cobus’ course will see its students constructing a new wiki from scratch, our wiki will be continually updated over time. Finally, we had the ability to study the effect of a wiki on a preexisting course, whereas Cobus’ course has integrated wiki technology from the very beginning (and as a result there is no point of comparison).

Our project is therefore novel in its hybrid focus: its usage as both a teaching tool that provides a measurable increase in student outcomes and as the source material for a publically available pathology informatics didactic resource. To this end, we adopted an expert-based peer-review model for content generation, which stands in stark contrast to the more egalitarian model of Wikipedia. We felt this to be an elegant solution to the issue of reliability – both real and perceived – that has traditionally plagued Wikipedia itself.

The Reliability of Wikis and the Wikipedia Controversy

Wikipedia is perhaps the best-known poster child of the Web 2.0 age. Initially created as a feeder project for a more traditional expert-edited web encyclopedia project known as Nupedia, it quickly eclipsed its parent project and is today one of the most widely read encyclopedias in the world. The English edition of Wikipedia alone boasts over 3.8 million articles at the time of this writing, many of which were created and edited by anonymous contributors without academic credentials.[33] The open nature of Wikipedia – more specifically the fact that anyone can make alterations to it and the complete lack of credentialed academic oversight – has resulted in a great deal of controversy, best exemplified by the publication of (and the reception to) a paper in nature in 2005 that showed relatively little difference in accuracy and error rate between Wikipedia and Encyclopedia Britannica.[34] In addition, there is growing evidence of a strong antiacademic bias among key Wikipedia editors.[35] More recently, the historian Timothy Messer-Kruse presented his experience of trying to edit a Wikipedia article on his area of expertise, the Haymarket Riot of 1886.[36] His edits, even when backed up by a wealth of reliable references, were deleted by anonymous Wikipedia editors who confronted Messer-Kruse in a hostile manner. Only after Messer-Kruse published his article – generating significant negative publicity for Wikipedia – did Wikipedia see fit to incorporate his changes.[37] On the other hand, other efforts to create online encyclopedias (e.g., Nupedia, Citizendium, and Google Knol) with explicit participation by experts have thus far been relatively unsuccessful in their efforts.[38–40] Theories on why these projects have been less than successful range from “a low number of consistent participants” to “infighting amongst the expert editors” to (ironically) “lack of reliable content.”[41,42]

The field of medical education is in a difficult situation regarding wikis such as Wikipedia. On the one hand, there is reason for caution, largely due to the perceived risk of unreliable information. On the other hand, it is undeniable that physicians – especially those younger than 30 years of age – are utilizing Wikipedia and other Web 2.0 resources at a rate of at least once per week, often out of sheer expediency.[14,15] Some studies comment on the relative sparseness of medical content on Wikipedia,[43] while others comment on its lack of reliability in certain medical fields.[44] There have been calls for the medical community to commit to contributing to Wikipedia on a large scale,[45] although Wikipedia's current antiacademic bias makes it difficult to gauge how Wikipedia editors would receive such contributions.[36,37] There is growing awareness that students and young doctors themselves are aware of the issues surrounding the use of Wikipedia as a resource:[46] we have evidence that suggests that young doctors are likely to consult Wikipedia and other Web 2.0 resources so they can mitigate the effects of the possibly unreliable information found on those sites on the patient.[14,15] Finally, there is the fact that many people do not perceive Web 2.0 resources to be reliable, regardless of the actual reliability of the content. As a case in point, on amazon.com, two of the five reviewers of the book Medical Informatics: Practical Guide for Healthcare and Information Technology Professionals (4^th Edition) mention the book's extensive utilization of Wikipedia as an indicator of the unreliability of the book as a whole.[47] While we do not necessarily agree with the opinions expressed in those reviews, we recognize that they point to the fact that there is a common perception – justified or not – that Web 2.0 resources are not to be trusted as reliable sources of mission-critical information.

Web 2.0 and Future Pathology Education

Even so, as we enter the “digital decade” of medicine, the possibilities for Web 2.0 technologies in medical education seem endless. These technologies have enabled a broad democratization of online publication, allowing anyone to publish anything online. Although we have focused – rather narrowly – on wikis in this paper, we should note that interactive Web 2.0 technologies have been instrumental in projects as diverse as collaborative virtual microscopy in undergraduate medical education,[48] annotation and markup of radiology images on tablet devices,[49] and beyond. As practitioners of the discipline that arguably generates the vast majority of structured data found in medical records, we have an unprecedented opportunity to use these interactive technologies to educate not only our trainees but also our colleagues on topics of crucial interest to them. Collaborative virtual microscopy could revolutionize how surgical pathologists interact with both residents and clinicians; this is a logical next step given the fact that we are seeing more and more clinicians asking for digital images to be included in final pathology reports. If we choose to embrace – rather than regard with suspicion – the unique opportunities that Web 2.0 technologies are continuing to unlock, we will only be better off for it.