Adoption of a wiki within a large internal medicine residency program: a 3-year experience

Abstract

Objective

To describe the creation and evaluate the use of a wiki by medical residents, and to determine if a wiki would be a useful tool for improving the experience, efficiency, and education of housestaff.

Materials and methods

In 2008, a team of medical residents built a wiki containing institutional knowledge and reference information using Microsoft SharePoint. We tracked visit data for 3 years, and performed an audit of page views and updates in the second year. We evaluated the attitudes of medical residents toward the wiki using a survey.

Results

Users accessed the wiki 23 218, 35 094, and 40 545 times in each of three successive academic years from 2008 to 2011. In the year two audit, 85 users made a total of 1082 updates to 176 pages and of these, 91 were new page creations by 17 users. Forty-eight percent of residents edited a page. All housestaff felt the wiki improved their ability to complete tasks, and 90%, 89%, and 57% reported that the wiki improved their experience, efficiency, and education, respectively, when surveyed in academic year 2009–2010.

Discussion

A wiki is a useful and popular tool for organizing administrative and educational content for residents. Housestaff felt strongly that the wiki improved their workflow, but a smaller educational impact was observed. Nearly half of the housestaff edited the wiki, suggesting broad buy-in among the residents.

Conclusion

A wiki is a feasible and useful tool for improving information retrieval for house officers.

Background and significance

The increase in the use of digital resources for patient care and medical education has outpaced trainees' ability to organize information. The proliferation of electronic data has resulted in fragmentation of resources across multiple websites, emails, and online forums. Given the pace of innovation, published guides and materials quickly become outdated with no mechanism for updating them in real time.

The use of online technologies that facilitate information sharing and collaboration may provide a solution to these problems. Commonly referred to as ‘Web 2.0,’ wikis, blogs, and microblogs have vastly decreased barriers for web users wishing to share information.^1–3 The term ‘wiki,’ coined in 1995 and taken from the Hawaiian word for ‘fast,’ today refers to a set of associated web pages that allow collective authorship through a web browser.⁴

Although wikis have been previously used in medical education, the effect of these interventions on the overall educational experience of residents has not been studied.^5–12 Previous survey data have already shown that the current generation of new physicians routinely use wikis and other Web 2.0 modalities to make clinical decisions as well as for medical education.¹³ Furthermore, although physicians frequently make use of online resources for patient-specific questions, there are still unmet information needs that may be met by online information retrieval systems.^{14 15} In this context, we sought to develop a wiki-based online platform aimed at improving resident workflow and education.

We postulated that the creation of a wiki for housestaff would improve residents' experiences, efficiency, and education by centralizing administrative, clinical, and educational resources in a format that enables real-time updating and collaborative authorship. This might be particularly valuable to internal medicine residents who were accustomed to carrying several pocket references of clinical and institutional information around with them. We anticipated that the ability of any resident to update content would make a wiki preferable to static web pages that are updated centrally by an administrator or webmaster, and that residents would gravitate toward their own editable site. While the wiki may not be preferable to professionally produced reference material for clinical content, it could offer value by providing brief introductions to clinical topics with links to curated references that have been chosen by the resident's peers.

Objectives

The primary objective of this study was to describe the creation and evaluate the use of a wiki by medical residents, and to determine if a wiki would be a useful tool for improving the experience, efficiency, and education of housestaff.

Materials and methods

Setting

The Beth Israel Deaconess Medical Center (BIDMC) is a 621-bed tertiary care center in Boston, MA, and a principal teaching hospital of Harvard Medical School. The medical center has a long history of using computers in medicine, with electronic information systems providing clinical information and support in use since the late 1970s.¹⁶ The department of medicine has 60 interns and 98 residents who are accustomed to using computers for all order entry, and for accessing clinical information and reference materials. This wiki was foremost an information improvement project within the internal medicine training program and current internal medicine residents were selected as a convenience sample. The study was reviewed and granted an exception by the Beth Israel Deaconess Medical Center Institutional Review Board as a quality improvement project.

Process

The development team built the wiki on a Microsoft SharePoint V.3.0 platform hosted on a server behind the hospital firewall. Microsoft SharePoint was selected primarily because it had been purchased and installed by our institution previously and could be used at no additional cost. SharePoint's integration with our hospital's user directory allowed residents and faculty to use existing login credentials, eliminating the barrier of a separate username and password for authentication. Access was restricted to individuals with a hospital login, with editing restricted to users within the Department of Medicine.

We aimed to establish ‘buy-in’ by educating housestaff on how to edit the wiki and on the potential value of wiki content. After the first year, an introduction to the wiki was incorporated into intern orientation and an educational conference reviewing how to edit the wiki was held at the midpoint of each academic year.

To access the wiki, users would visit the wiki login page (http://www.bidwiki.org) that redirected them to the SharePoint server. Users could link directly to the wiki from any workstation on campus, or use the virtual private network for off-site access. While page views could be anonymous, we required user authentication to make any edits. Because a user may make several edits in one session, and so as to not take away from the rapidity of updates, updates were made live after authenticated by the submitter. Each edit automatically generated a notification email that was sent to the wiki editors-in-chief (BC, AM) and resident section editors for retrospective review.

Content design

Editors organized wiki content around three domains: (1) a residency portal, including quick links to common intranet destinations, policies, and forms (figure 1); (2) shared institutional knowledge, such as a telephone directory and conference calendar; and (3) educational content, including the curriculum and guides to key clinical topics built around the literature. Teams of residents and faculty built the initial wiki pages based upon the residency curriculum. The editors provided a standard format for each page that included a brief introduction, links to specific hospital protocols, and links to articles and electronic resources chosen by residents, editors, and supervising faculty. For clinical topics and official institutional policies, the wiki editors placed an emphasis on linking externally to fully edited and/or approved content.

Figure 1.

Screenshot of the wiki main page, which outlines various functions including a guide to clinical problems and the literature, quick references, residency and medical student curricula, and commonly used links.

The wiki became the de facto residency intranet site, hosting most of the program's guides and policies. Administrative staff within the training program's office, accustomed to emailing a weekly conference schedule, migrated to posting all events on the wiki. Presentations given during clinical conferences were uploaded by the speakers and made available to users to review and download.

Evaluation

We counted visits to the site using Google Analytics (http://www.google.com/analytics) installed on the login page with a timeout (the time lapsed between page refresh) of 30 min. Data were collected and analyzed from July 1, 2008 to May 13, 2011.

We performed a more detailed audit for the second academic year (AY 2009–2010) at the individual page and user levels using DocAve for SharePoint (AvePoint, Jersey City, New Jersey, USA). The second year was selected for the audit as we had a complete academic year of data at the time of evaluation. We categorized users by postgraduate year, chief medical resident, faculty, or administrator. The categorization of pages is given in table 1.

Table 1.

Breakdown of pages, views, and edits by content category in academic year 2009–2010

Page category	Description	% Of pages	% Of views	% Of edits
		n=512	n=13 903	n=1082
Wiki homepage	Wiki main page, including calendar and announcement functionality	0.20%	*	14.23%
Administrative information
Administrative	Institutional knowledge, phone numbers, policies	7.81%	43.56%	7.39%
Residency recruitment	Residency recruitment materials and scheduling	3.32%	3.98%	15.90%
Clinical information
Clinical pages	Specific clinical conditions and curated resources	59.77%	27.50%	36.41%
Protocols	Institution-specific clinical protocols	1.76%	2.50%	6.47%
Educational information
Intern guide	Insider information about clinical rotations from past interns	5.86%	7.52%	5.18%
Education/curriculum	Educational objectives for core rotations or electives	15.23%	11.36%	7.76%
Other
Miscellaneous	Social information, research, surveys, and miscellaneous	6.05%	3.59%	6.65%

^*Wiki homepage views (n=9190) removed from column 4 to reflect viewership of sought content.

To understand the perceived impact of the wiki on resident experience, efficiency, and education, interns, junior assistant residents, and senior assistant residents completed surveys at the end of AY 2010–2011.

Results

Usage data indicate that the wiki was accessed 96 857 times between July 1, 2008 and May 31, 2011, with usage trends increasing over time, with monthly averages of 1935, 2925, and 3686 in each of the three successive academic years. There was an annual nadir of views and updates in the spring. Of all visits, 73.6% occurred from within the BIDMC network, 1.6% from affiliated clinical sites, and 24.8% from off-site locations. We recorded access from mobile devices for AY 2010–2011 and found that 2.6% of all visits were from mobile devices.

At the end of the second year, the wiki contained 512 informational pages (table 1). While clinical pages accounted for the largest category of pages and updates, administrative pages accounted for the largest percentage of views. The most updated clinical pages were on medical consultation, primary care, and pre-operative risk stratification, each with a large emphasis on local procedures.

The majority of page editors were senior assistant residents (table 2). Most of these edits were made to clinical pages or to the wiki home page itself, which was often edited to make general announcements or to post new links to new pages. Postgraduate year 2 (PGY2) residents made the fewest number of edits, most of which were related to residency recruitment planning. Interns made some clinical and administrative edits, but the majority of their page edits were made at the end of the academic year as they updated the intern-to-intern guide for the incoming class.

Table 2.

Page edits by contributor status and content category in academic year 2009–2010

Editors/class size (% of class)	PGY1	PGY2	PGY3	Chief medical resident, faculty, or administrator	Totals
	n=17/62 (27%)	n=26/49 (53%)	n=29/46 (63%)	n=12	n=101
Edits per year by contributor status
Wiki homepage	7	6	118	23	154
Administrative information
Administrative	14	3	37	26	80
Residency recruitment	20	60	77	15	172
Clinical information
Clinical pages	30	0	187	177	394
Protocols	0	0	13	57	70
Educational information
Intern guide	34	1	18	3	56
Education/curriculum	2	0	28	54	84
Other
Miscellaneous	0	0	19	53	72
Totals	107	70	497	408	1082

PGY, postgraduate year.

A small number of faculty members (n=6) accounted for 21% of updates. These updates were mostly related to institutional protocols and clinical content pertaining to hospital medicine curricula for residents. We (AM, BC) did not see any instances of faculty directly editing a resident's post to correct content. On average, contributors made between 50 and 100 edits per month. None of the wiki edits to date have contained individually identifiable patient information or other professional violations.

When surveyed in 2011 (n=91, 58% response rate), interns and residents self-reported high levels of wiki access, with 74% of housestaff accessing the wiki at least once daily on inpatient rotations and 31% on outpatient rotations (figure 2). There were substantially fewer self-reported contributions (creating or editing a page), with 32% of residents having contributed to the wiki. Housestaff in general favored the wiki over the institution's intranet (63% vs 32%) as their primary source for administrative information, but used the intranet for official treatment protocols (78% vs 8%). The wiki was the primary source of telephone numbers for 84% of residents.

Figure 2.

Average self-reported use of the wiki in academic year 2010–2011 by type of rotation (n=91 respondents).

The overall impression of the wiki was very positive, with 90% and 89% of housestaff reporting that the wiki improved their experience and efficiency, respectively. A majority of housestaff (57%) felt that the wiki improved their education. Ninety-two percent of residents felt that the wiki made a positive contribution to the program, and 88% wanted to keep the housestaff wiki. Analysis of the AY 2010–2011 data by postgraduate year revealed no differences among classes (table 3).

Table 3.

Survey of residents (n=91, response rate of 58%) in academic year 2010–2011 on utility and experience with the wiki

Utility	Not useful	Useful
How useful was the wiki in guiding your education? (%)	8	92
How useful was the wiki in helping you complete tasks? (%)	0	100

Experience	Not agree	Agree
The wiki improved my experience at the BIDMC (%)	10	90
The wiki improved my efficiency at BIDMC (%)	11	89
The wiki improved my education at BIDMC (%)	43	57

BIDMC, The Beth Israel Deaconess Medical Center.

Discussion

Our experience demonstrates that a wiki is an effective and popular tool for organizing administrative and educational content for house officers. Usage and survey data indicate that the wiki has become an essential and increasingly utilized resource on the wards.

Housestaff unanimously felt that the wiki helped them complete tasks, and 90% and 89% agreed that the wiki improved their experience and efficiency, respectively. However, although 92% of housestaff felt that the wiki helped guide their education, fewer (57%) reported that the wiki improved their education. In combination with the usage data, this suggests that the success of the wiki has been primarily in improving administrative and workflow processes. While approximately 20% of visits to the wiki were for educational content, the wiki has not displaced existing forms of informational access and retrieval.

These results are in line with the findings of Currie et al who reported on the types of information needs of those conducting clinical work.¹⁴ Currie's group identified that although the majority of information needs of housestaff were related to specific patient data queries, 22% of information gaps were institution related, and 29% were related to clinical knowledge. It is possible that the satisfaction with the wiki is rooted in the institutional knowledge provided, even though residents likely used other resources to bridge clinical information gaps.

A few limitations both in terms of the wiki creation and evaluation bear discussion. The wiki was created in a setting where comfort with technology was already high, and where residents use computer workstations for most tasks. To replicate this experience would require several initial champions from within a housestaff program and support from administrators. This wiki was deployed among a large medicine housestaff, and smaller programs may not have the sufficient number of residents required to obtain a ‘critical mass’ of editors to enable a wiki to thrive. Other institutions may have different levels of baseline use of technology as part of their workflow and may not be as comfortable with the use of Web 2.0 technologies.¹⁷

Furthermore, because of significant concerns regarding contamination, it was not possible to study the effects of the wiki on education or efficiency by using a true control group for comparison. However, we feel that the responses received were honest estimations, and that the usage data indicate the successful adoption of the intervention. Our audit software was not capable of distinguishing major from minor changes, but the number of edits likely indicates which areas were more dynamic. Additionally, website views to edit pages are indistinguishable from visits to obtain information. The relative effect of these visits on overall website ‘usage’ is small given the limited number of edits relative to the large number of page views.

It is likely that these findings can be generalized to other postgraduate training programs in other specialties. Similar self-reported usage rates and satisfaction scores were identified in the use of a wiki within a radiology program.¹² Medical students may have different information needs and usage patterns than interns and residents. Wikis have been used in undergraduate medical education, but often in a different fashion. One common example is use with problem-based learning tutorials where students work together through a case. Here the wiki can act as a platform for ‘connectivism’ where students synthesize information from various resources and then rework the knowledge and apply it to a case.^{17 18}

Perhaps the most significant contribution of this wiki was the fact that it enabled housestaff to share useful administrative and educational information with each other in an efficient manner. In AY 2009–2010, the year in which the user-level audit was performed, 48% of house officers made logged page changes. As priorities shift or needs change, a wiki platform can adapt accordingly, maximizing utility and minimizing outdated web pages. Many programs and schools are considering how tools such as wikis can be used for medical education. While our experience with a large-scale wiki is nascent, several lessons are worth sharing.

1. The primary challenge we faced was to establish ‘buy-in’ from the targeted population; without users, a wiki is simply an empty scaffold. We tailored our wiki toward interns by prioritizing logistical and administrative content, and the wiki quickly become part of housestaff workflow. As the wiki developed into a central resource, housestaff began to experiment with the platform for other uses, including the development of evidence based clinical pathways and links to key literature topics.

2. Plans for oversight and maintenance should be created and implemented from the very beginning. We created a multi-layered system of oversight to eliminate the posting of inaccurate information to the wiki by designating resident editors for each clinical topic section, with dedicated faculty members within each subspecialty division serving as consultants. This oversight, combined with a charge to minimize the creation of original clinical content, was an effective strategy that afforded no notable instances of inaccurate information being published. As such, our model of self-publishing followed by editor review instead of holding publication until a post has been reviewed, appears to be effective for maintaining ease of use without sacrificing quality.

3. Creating a wiki entails two types of costs: (1) technological support for server space or a software package and (2) labor and opportunity costs for those who add and monitor content. It is often possible to use resources already offered at the local institution, or open source software to reduce costs. The manpower required to drive the wiki can be subdivided into creation and maintenance. The initial scaffolding was performed by the wiki editors (AM, BC) after which residents who had specific interests were asked to populate content and serve as section editors to maintain the site.

Conclusions

As technology improves and online collaboration becomes mainstream, the use of wikis in medicine has tremendous potential for expansion. Wikis allow for low cost publishing and may reduce the amount of outdated information. Our experience has shown that with modest resources, initiative, and buy-in from constituents within a residency program, a large-scale wiki can be created and grow in content and usage. Although we anticipated that knowledge content would predominate, the wiki provided unanticipated value in multiple other areas as well. While the wiki provided modest direct educational benefit, residents almost universally acknowledged the resource as an asset to the program.