Radiology Teaching Files: an Assessment of Their Role and Desired Features Based on a National Survey

Abstract

A radiology teaching file (TF) is a system containing a collection of cases with teaching value. Given the wide variety of TF solutions available, we conducted a national survey to better understand the need for TFs, TF features desired by users and their current implementation. A 28-question survey was created which explored TF implementation, utilization, and preferences among respondents. The survey was emailed to residents and faculty throughout the USA, with a request for program coordinators to forward the survey to their departments. The survey was completed by 396 respondents from 115 different institutions. These respondents included 60 % residents, 21 % attendings (non-program directors), 12 % program directors, 5 % fellows, and 1 % medical students. TFs were assigned to one of three categories: personal TFs, shared in-house TFs, and public TFs. Seventy-six percent of respondents kept a personal TF using a variety of media, and 67 % used a shared in-house TF. Of the public TFs used, the most popular were those requiring paid subscriptions. The features respondents valued most provided efficient querying of cases, simulated basic PACS functionality, enabled self-directed learning, and facilitated case submissions. There is a trend toward utilizing electronic media for TFs. The media utilized should be understood and reviewed to ensure PHI is properly secured. Contemporary users demand a high degree of functionality from TF solutions, and use both in-house and commercial products to meet their needs.

Introduction

Radiology education is a life-long process, occurring at medical student, resident, and attending levels. A teaching file (TF), defined as a system containing a collection of cases with teaching value, can serve multiple functions, including the following: (a) a refresher of important findings not to be missed, (b) a repository of cases for clinical follow-up, and (c) a reference in understanding the spectrum of a disease. Originally taking the form of printed films with hand-written diagnoses, TFs have evolved to take advantage of digital technologies, including PACS, file sharing services, and web-based collections [1–3].

TFs can be divided into three separate types: personal TFs, shared in-house TFs, and public TFs. Personal TFs are meant for the general use of the TF owner, who may have a specific interest in a set of cases in order to practice quality control, review personal work, or for the purposes of teaching and follow up. Personal TFs can be migrated into a shared TF environment, whereby the owner can add additional content to a case and make it available for viewing with their colleagues, institution, or beyond. Public TFs build on the shared TF model, but often with more comprehensive content that may undergo a formal review before “publication”. Public TFs sometimes charge a subscription fee.

TF repositories may take on a variety of forms. The simplest form may be a list of cases in a notebook or spreadsheet, while more sophisticated implementations include a key image case log, as previously described [4].

Given their importance and the broad range of options in establishing TFs, we conducted a national survey to better understand the desired features of the contemporary user and how TFs are implemented and utilized throughout the country. This study was supported by the Radiology Society of North America (RSNA) and conducted, in part, by members of the RSNA Medical Imaging Resource Center (MIRC) subcommittee.

Materials/Methods

An online, web-based survey was created using a commercially available tool (Survey Force Deluxe, Belitsoft, Minsk, Belarus). The 28 questions (see Appendix) explored the demographics of respondents, if and how respondents use TFs, technologies available for creating TFs, and specific features that users desire. Requests to complete the survey were sent to approximately 700 email addresses obtained from a mailing list of radiology residents and faculty throughout the USA, including 188 radiology program directors, with a request that it be forwarded to radiologists in their departments. The survey was additionally advertised at the 2012 Association of University Radiologists Annual Meeting. Respondents were offered the chance to win one of three $50 gift certificates to an online retail store, purchased and distributed by RSNA. At the time of data analysis, email addresses of respondents were removed from the survey data. The systematic analysis and publication of the de-identified data was determined by our Institutional Review Board to not constitute research involving human subjects, and therefore not requiring IRB approval.

Results

The survey was completed by 396 respondents from 115 different institutions within the USA. These respondents included 238 (60 %) residents, 83 (21 %) attendings (non-program directors), 48 (12 %) program directors, 20 (5 %) fellows, and 4 (1 %) medical students. The majority of respondents (87 %) were affiliated with university-based programs.

Eighty-nine percent of respondents use some form of TF. Seventy-six percent of respondents kept a personal TF (not made accessible to others), while 67 % used an in-house TF (shared within their institution’s radiology department). Respondents store their personal TFs using a variety of media, including: paper notebooks, thumb drives, portable and desktop computing devices, internet, PACS, and PowerPoint presentations (Fig. 1). The most popular means for storing personal TFs was through the institutional PACS (54 % of personal TF users), followed by PowerPoint presentation (39 %) and storage of images to personal computers (37 %). Twenty-one percent of personal TF users stored TF cases on the internet. Six percent of respondents also included “Other” in their response. The most common fill-in for “Other” was saving of cases to a University or Hospital server (5 respondents). Several respondents listed additional case-logging programs, including the following: MIRC, OsiriX, RaceTrack, and DropBox.

Fig. 1.

Percentage of respondents utilizing various media for storage of their personal TF. Only those respondents who completed the survey and utilized personal TFs were included (308 respondents). Respondents could check more than one box, and write in their own response (“Other”)

Respondents were asked to rate the importance of TF features: very important, somewhat important, or not important (Table 1). The two features rated “very important” by the largest percentage of respondents (77 % each) were as follows: (a) viewing cases while hiding the diagnosis and (b) categorizing and searching for cases by diagnosis, subspecialty, modality, or body part. Features rated as very important by 50 to 74 % of respondents were as follows: (a) the inclusion of supplementary information such as history, findings, diagnosis, and discussion; (b) the ability to scroll through images; and (c) the ability to author cases directly from PACS. The two features that received the fewest “Very Important” and greatest “Not Important” ratings were as follows: (a) storing patients’ names and medical record numbers and (b) rating and commenting on cases.

Table 1.

Percentage of respondents rating various TF features as “Very Important,” “Somewhat Important,” or “Not Important”

	Very important (%)	Somewhat important (%)	Not important (%)
View cases while hiding the diagnosis	77	21	2
Categorize and search for cases by diagnosis, subspecialty, modality or body part	77	22	1
Include supplementary information such as history, findings, diagnosis and discussion	72	25	3
Scroll through images	62	31	7
Author cases directly from PACS	57	33	10
Support JPEG/PNG/TIF formats	48	39	13
Support DICOM images	43	42	15
Include test questions	42	44	14
Use on handheld device	42	40	18
Show movie files	29	46	25
Share cases over the web	29	48	23
Show a case of the day	27	50	23
Store patient name and medical record number	16	27	57
Rate and comment on cases	16	45	39

Among respondents who use shared in-house TFs, few regularly submit their own cases for inclusion in the TF. Respondents were asked to evaluate the ease of submitting a case and how often they submitted a case to their shared in-house TF (Fig. 2). Thirty percent of in-house TF users never submit a case. Forty-two percent of users submit 1–5 cases per month. Of users who submit >10 cases per month, 100 % found the submission process very easy or somewhat easy, compared to 84 % of users who submit 1–5 cases per month. Of users who never submit cases to their in-house TF, 55 % did not know the difficulty of the submission process.

Fig. 2.

The frequency of case submission to respondents’ shared in-house TF, categorized by ease of submission. Only those respondents who completed the survey and had a shared in-house TF were included (267 respondents)

Respondents were asked to rate a variety of public TFs. The list included three TFs that required paid subscriptions for use. These were StatDx (Amirsys, Salt Lake City, UT), RadPrimer (Amirsys, Salt Lake City, UT) and ACR Learning Files (American College of Radiology, Reston, VA). Among all public TFs, whether free or paid, StatDx was regarded as “Very Valuable” by the greatest number (71 %), followed by RadPrimer (50 %) and ACR Case in Point (35 %) (Fig. 3).

Fig. 3.

Percentage of respondents rating various public TFs as “Very Valuable,” “Somewhat Valuable,” or “Not Valuable.” Percentages reflect only those users who have access to and utilize the specified public TF

Among the 115 institutions represented in this study, 83 institutions subscribed to paid TFs. StatDx was the most popular subscription service, utilized exclusively by 27 institutions, while an additional 54 institutions maintained a StatDx subscription with subscriptions to other TF services. Institutions with two paid TF subscriptions most commonly subscribed to both StatDx and RadPrimer, and institutions with three subscriptions most commonly subscribed to each of StatDx, RadPrimer, and ACR Learning Files (Fig. 4).

Fig. 4.

Distribution of paid TF subscriptions within the 83 institutions that offered at least 1 paid subscription to their staff

Discussion

Teaching files remain valuable components in radiology education, with their form and distribution evolving to take advantage of new technologies. Early TFs were largely film-based and stored in the office of a senior radiologist or departmental file room. Interest in converting these analog TFs to digital format and placing them online was described in the early days of the internet [5]. Although some early studies described the lack of completeness of digital TFs and their high cost of maintenance [6], this study demonstrates that there continues to be a trend toward using various digital media for the creation of robust radiology teaching content. Likewise, there are new avenues for distribution of this content, whether in a shared in-house TF or in a public TF that is available to the internet community at-large.

Whereas the earliest forms of digital TFs were simply the presentation of analog films in digital format with attached text, the modern user demands more functionality from present-day TF solutions (Table 1). Several publications have described the value of such features, including dynamic quizzes [7] and interactive training to adjust for a user’s knowledge level [8]. Furthermore, the same TFs that have been used for radiology resident education are being utilized for other areas of teaching, such as medical student anatomy education [9].

A majority of respondents (76 %) maintained a personal TF. While a very small percentage of survey respondents still used paper notebooks and hard copy films as forms of personal TFs, a majority of respondents used various forms of digital media including thumb drives, computing devices (both portable and desktop), and tools on the internet, including photo sharing websites, to organize their educational content (Fig. 1). A large percentage of respondents stored their personal TFs within their own PACS. PACSs that support TFs typically supply this feature in the form of a teaching folder, either for the institution in general or for each individual user. With a few clicks, the user can add selected images or an entire study for storage in his/her folder or even a disease-specific folder.

Storage of TFs in a digital media promotes the ability to share the case. Indeed, the simplest form of a shared in-house TF could be a list of names and medical record numbers on an electronic spreadsheet made accessible on a network hard-drive or departmental intranet. Depending on the abilities of an individual PACS, the PACS TF folders may be shared with other users or the entire department to create a shared in-house TF. Photo sharing websites can likewise be configured to share text and images with groups. More organized and commercially available in-house TF solutions include MyPACS (McKesson Technologies, Inc., San Francisco, CA) and MIRC (Radiology Society of North America, Oakbrook, IL).

In choosing a personal or in-house TF solution, it is important to understand the users’ needs and to address important patient privacy concerns. The features assessed in this survey included those that promote self-study, facilitate submission of new cases, increase accessibility, and incorporate stylistic features. Features that promoted self-study included the ability to “hide the diagnoses” and “include test questions”. The ability to “categorize and search for cases by diagnosis, subspecialty, modality, or body part” was one of the most desired features for users, likely because it allows efficient querying. The inclusion of supplementary information, such as history, findings, diagnosis, and discussion, was also highly valued. These details contribute to the understanding of a case.

TF features which facilitate submission of new cases included authoring cases directly from PACS and the support of common formats such as JPG, PNG, and TIF (“Very important” to 57 and 48 % of respondents, respectively). Authoring a case directly from PACS allows a radiologist to send images directly from the PACS to the in-house TF, thereby incorporating the creation of TF cases within his/her routine clinical workflow. This feature further enables the use of radiographic images in native DICOM format, which, with certain in-house TF solutions, offers the ability to change window width and window level [10]. The ability to add images in multiple file formats facilitates the inclusion of non-radiologic data such as pathology slides, and expands the educational value of the TF solution. The frequency of submission of cases to in-house TFs was associated with ease of the submission process, as expected (Fig. 2). Developers of shared, in-house TFs stand to heighten the usage of their products by including features such as integration with PACS, as described above. Thirty percent of users never attempted to upload cases to their shared in-house TF, the majority of whom answered “I don’t know” to the question of how easy it is to use their in-house TF. This suggests that familiarity and training are obstacles to usage.

The choice of a teaching file solution may impose certain limitations. Although convenient, keeping cases in PACS-based folders can be notably problematic. First, this solution does not allow the storage of non-DICOM images. Additionally, users may not have the ability to search for cases by key parameters such as history or diagnosis, and users may lack the ability to readily share cases with colleagues. Finally, storing of cases in the PACS environment makes preparing presentations and moving data difficult, particularly when a user leaves his/her institution, or when an institution desires to change its PACS. New PACS systems usually cannot import TF folders from other PACS, even if the new PACS is a product of the same vendor as the old.

The choice of a TF solution also raises important concerns about patient privacy. A high percentage of respondents reported using notebooks, thumb drives, computing devices, and the internet to store their personal TFs. Housing TFs using these media typically involves recording a patient’s name, medical record number, and diagnosis, so that the images can be later reviewed. Identifying information may be needed in certain use-cases that include follow-up of a patient to make a definitive diagnosis or determine disease progression/treatment response. However, a large fraction of TF cases can likely exist without the need for any PHI whatsoever. Many of the TF solutions utilized by survey respondents are insecure and risk release of protected health information (PHI). In a December 2013 press release, the US Department of Health and Human Services announced a $150,000 settlement with a medical practice in Massachusetts for the loss of an unencrypted thumb drive containing the PHI of its patients [11]. The case highlights the very real possibilities of PHI loss, and their serious consequences. Institutions should carefully review how PHI is being stored in various media, whether for TFs or other purposes. Special attention should also be paid to how PHI may be embedded in radiographic images, such as ultrasound. A recent study found that 36 % of PowerPoint presentations available for download online contained PHI [12].

Public TF solutions have become increasingly popular, providing users with instant access to thousands of cases of varying quality [13], sometimes for a fee. The most popular of these solutions among respondents include StatDx, RadPrimer, and ACR Learning Files (Figs. 3 and 4), which provide large, diverse collections of peer-reviewed TF cases with well-developed supplementary information. A 1-year subscription to a commercial TF for an individual user can range from a few hundred to a few thousand dollars. While many public and commercial solutions are available, most do not permit a user to easily submit personal cases to their libraries. A study by Novak et al. in 2011 employed medical students to create interactive digital case-based radiology TFs, and found that, “The experience of creating the files served as an opportunity for hands-on learning for the student authors, both of the material and of the practice of teaching” [14]. Therefore, while the public and commercial TF solutions may have a more diverse collection of cases than in-house TF solutions, they may limit the educational value derived from personal case creation and sharing.

This study has a few limitations. There is a selection bias; respondents who completed the survey are probably more likely to use TFs than those who did not complete the survey. All data presented is comprised of completed surveys only; there were 544 attempts to start the survey, and 73 % were completed. Respondents consisted primarily of residents, followed by attendings (non-program directors). While TFs are most heavily utilized among the resident population, the views of fellows and medical students are not well represented by our data. Some institutions were also more heavily represented in our study. Institutions with the greatest number of respondents were as follows: New York Presbyterian Hospital (Cornell) (9 %; 37/396); Massachusetts General Hospital/ Harvard Medical School (5 %; 20/396); University of Maryland (3 %; 13/396); Stanford University (3 %; 13/396); and Drexel University College of Medicine/ Hahnemann University Hospital (3 %; 12/396). Nonetheless, surveys were completed by respondents affiliated with 115 different institutions across the USA, with no institution affiliated to more than 9 % of respondents.

In conclusion, TFs take on a number of forms and are a highly valued tool utilized by trainees and faculty, for a variety of purposes. Individuals and institutions should ensure that PHI stored in their TF solutions is secured properly. Future TF development should focus on features that provide efficient querying, simulate basic PACS functionality, enable self-directed and assessed learning, and that facilitate case submissions. Educating users on the availability and features of TFs also play an important role in their usage. Contemporary users utilize both in-house and commercial products to meet their TF needs.

Appendix

This survey is about teaching files, defined as a system containing a collection of cases or a list of cases that have teaching value. This can range from a notebook to hardcopy films to a web-based image server. They can be a very valuable resource, and we want to know more about your own use, what features matter to you, and anything more you would like to see in a teaching file.

At the end of the survey, you will have the option to enter your email address to win one of three $50 Amazon gift cards.

Thank you for your participation.