Abstract
Scutwork.com is an online, peer-based residency review system. We report preliminary results of an online survey designed to investigate the impact of Scutwork on the residency application process. Overall, 68% of respondents believe that the reviews influenced their decision-making and 91% would use Scutwork again. These results and others reported below suggest that Scutwork may play a significant role in the residency selection process.
Background
For most applicants, the residency program selection process comprises many difficult personal and professional decisions. Residency programs have used paper and Internet-based evaluation systems, but these systems are designed primarily for internal assessment purposes and the results are often not made available to applicants. Scutwork allows medical students and house officers to submit reviews of their respective training programs and permits registered users to view these editorials. All reviews displayed are filtered and structured based on predefined criteria [1]. Since its deployment on January 1, 2000, we have collected over 4000 program reviews and currently average over half a million unique users per month. While it is clear that Scutwork has gained increasing popularity since its inception, we implemented an online survey to investigate further how Scutwork has influenced the residency selection process. In this report, we look specifically at how this system influenced current residents in their program selection process.
Methods
Our sample population consists of registered users of Scutwork. We solicited participation using the group e-mail feature available through the Scutwork application. We developed a survey containing a series of multiple-choice and free-text-entry questions and conducted it via the Internet using an online survey tool developed by SurveyMonkey.com. Responses were tabulated and percentages were calculated for each response. We report results from surveys returned in the first week of study.
Results
Over 28,000 emails were sent to registered users—1812 (6.5%) completed the online survey in the first week of study. Of the returned surveys, current residents who used Scutwork while applying to residency completed 464 (25.6%) of the surveys. Among them, 321 (70.4%) used Scutwork to help pick which residency program to apply. 297 (65.1%) used Scutwork to help decide which program at which to interview. Residents found Scutwork to be most useful at helping them:
▪ Decide where to apply (68.9%),
▪ Decide where to interview (31.1%) and
▪ Prepare a rank-order for the match (29.2%).
235 (51.5%) used Scutwork to help them ask questions during interviews and found the reviews to be useful. Among the 336 (78.2%) residents who read reviews of their own program prior to their applications, 316 (90.3%) found them to be accurate. 414 (91%) would use Scutwork to help research residency programs if they had to go through the match process again. Overall, 309 (67.7%) believed the reviews at Scutwork influenced their residency selection process.
Conclusions
Applicants are turning to web-based residency review systems for information during critical phases of the application process. In addition to the formal survey results, we received an overwhelming response in the free-text portion of the survey to further develop Scutwork. These encouragements along with the preliminary findings suggest not only that the reviews help applicants with their residency application process, but also that applicants feel a continued and vested interest in the expansion of this effort.
Acknowledgements
This research was partly supported by the National Library of Medicine fellowship training grant number 5T15LM07092-13.
References
- 1.Zai A. Developing a Web-based Review System on Residency Training Program. 2004 IMIA Proceedings.