The Time Has Come: a Paradigm Shift in Diagnostic Radiology Education via Simulation Training

Abstract

Current radiology training for medical students and residents predominantly consists of reviewing teaching files, attending lectures, reading textbooks and online sources, as well as one-on-one teaching at the workstation. In the case of medical schools, radiology training is quite passive. In addition, the variety of important and high-yield cases that trainees are exposed to may be limited in scope. We utilized an open-source dcm4chee-based Picture Archiving and Communication System (PACS) named “Weasis” in order to simulate a radiologist’s practice in the real world, using anonymized report-free complete cases that could easily be uploaded live during read-outs for training purposes. MySQL was used for database management and JBOSS as application server. In addition, we integrated Weasis into a web-based reporting system through Java programming language using the MyEclipse development environment. A freeware, platform-independent, image database was established to simulate a real-world PACS. The sever was implemented on a dedicated non-workstation PC connected to the hospital secure network. As the client access is through a webpage, the cases can be viewed from any computer connected to the hospital network. The reporting system allows for evaluation purposes and providing feedback to the trainees. Brief survey results are available. Implementation of such a low-cost, versatile, and customizable tool provides a new opportunity for training programs in offering medical students with an active and more realistic radiology experience, junior radiology residents with potentially better preparation for independent call, and senior resident and fellows with the ability to fine-tune high-level specialty-level knowledge.

Introduction

Education is a constantly evolving endeavor, especially when it comes to medical education. As each specialty becomes more and more advanced, the way topics are taught should be enhanced for the best possible outcome. We have proposed a new approach to teaching radiology to both medical students taking their radiology elective and radiology residents during their training, given all of the limitations that currently exist in both.

Current Radiology Residency Training and Its Limitations

Understanding how radiology residents are currently trained in the USA, and the vital role they play in patient management, will make it readily apparent just how beneficial augmenting radiology education could be. Radiology residency programs in the USA start at the post-graduate year (PGY) 2 level after residents complete either a transitional year or a preliminary year in medicine or surgery. Traditionally, radiology residency training is achieved with a combination of case conferences and didactic lectures by program faculty, daily teaching at the workstation, and pre-call assessments. On a typical day, junior radiology residents will start picking up cases from the list of studies and attempt to dictate them. After saving a certain number of cases, or at specific times throughout the day, the residents will review the cases with the attending and making the necessary modifications to the report before an attending signs off the final report. This is in contrast to more senior radiology residents, who will typically dictate and independently sign off a report, which will be visible as a preliminary read, later to be reviewed with and signed off by an attending radiologist.

Halfway into their PGY2 year (1st year of radiology residency), most of them will start “buddy” call, where they take call with an upper level radiology resident. Beginning in their PGY3 year (2nd year of radiology residency), they will begin to take independent in-house call overnight, either with cases being reviewed by an attending radiologists the next day or with an attending or radiology fellow available for backup in-house or offsite. In both cases, the attending is rarely called for help, which makes the residents’ role vital in patient management. Once they become a PGY3, and after satisfactory completion of required educational modules and readiness tests, they can serve as the upper level resident to the PGY2 [1]. In many academic institutions, diagnostic radiology residents are often the only physicians responsible for night radiology coverage, generating preliminary reports of examinations that will not be reviewed and finalized until the next day by the attending radiologist. Most residents find call to be anxiety inducing, but very motivating, rewarding, and educational [2].

There are areas in radiology where accurate timely diagnosis is of utmost importance, such as in the setting of acute stroke or bowel ischemia. An example of this is illustrated in the recent DAWN trial findings, where interventional management can extend beyond 6 h, making the radiology resident’s ability to make a timely and accurate diagnosis even more paramount. Therefore, residents who are more prepared for independent call can play a more drastic role in patients’ outcomes. Back in 1970s, Rhea et al. performed a quality improvement study at Massachusetts General Hospital emergency radiology facility and found that staff radiologists indicated that they had changed 11% of the initial reports by residents in a significant or potentially significant way [3]. In 1990s, errors that delayed surgical treatment or misdirected management, resulting in a potentially life-threatening event, were reported to range from 1.1 to 6%, much of it being attributed to inexperience [4].

Relying simply on the random cases that are available during a typical rotation month may not be adequate or the best approach. Augmenting resident’s education with a collection of high-yield targeted educational images and pathologies, in a simulated PACS, in addition to reading the new daily cases, can be highly beneficial, and more precisely facilitate and accelerate the learning process. In addition, enhancing pre-call training before the second year on-call services can empower first year radiology residents in multiple ways, such as self-assessment skills, self-confidence, and familiarity with the on-call environment [5]. For these reasons, and more, there is a need for more innovative ways to better prepare residents. Many solutions have been proposed to tackle these issues, but not all are feasible. Sajedi et al. developed a cloud-based PACS viewer for interactive educational program, which provides a unique method of learning radiology and has become increasingly popular among trainees [6]. However, the cost of purchasing this system and the lack of versatility make options like this more difficult to initiate.

Current Medical School Radiology Training and Its Limitations

Medical students going into radiology, or students going into other specialties that will encounter image interpretation, are graduating with limited radiology skills. Radiology is typically taught in a limited scope during the preclinical years, often as still images of common pathologies. It is also usually taught in large group lectures, which may not be ideal for problem-solving skills required in radiology and other types of medical practices [7]. In recent years, complementary educational approaches that challenge clinical decision-makings are growing, such as flipped learning (online lectures and modules viewed home before the in-person lecture which then focuses on discussing and practicing the online material) [6, 8]. This also includes radiology lectures, but it is still typically very basic and passive, when compared with the real-world experience they are likely to face when needing to interpret a study prior to a radiology report becoming available.

For those medical students who want more exposure to radiology, a clinical radiology elective is typically available in their 3rd and 4th years. However, these rotations usually involve “shadowing” a radiologist by sitting behind them and watching them dictate studies, with the occasional teaching points provided, depending on the attending. For the majority of students, a radiology elective can be a passive course with little skills being developed that can be of tangible use in other rotations. The problem is twofold: (1) they do not develop the skills needed to navigate a PACS system with interactive images when they are on their clinical rotations, during residency, and as a future non-radiologist attending, and (2) they do not fully experience what it is like to be a radiologist, which can deter students from actually pursuing radiology as a career. In fact, many rotations in 3rd year of medical school, such as surgery, medicine, and emergency medicine, require that students be able to navigate a PACS system and to read their patients’ images at a basic level. However, most students are not only unable to do this effectively, but are also intimidated by the prospect, given their lack of direct exposure and experience.

Currently, there are simulation labs available in most medical schools for medicine and surgery training, but radiology simulation labs are not widely available. Some medical schools have attempted to tackle this problem. Restauri et al. established and evaluated The Beginning to Advanced Radiology Learning Laboratory (BAR Lab) in University of Colorado. They intended to create both a space and curriculum that prepared students to practice interpreting images using tools similar to a radiologist instead of purely being observers sitting in the back seats [9]. However, the problem with having a dedicated radiology lab is the high cost of implementation and continued technical and logistical support. This is simply not feasible for most medical schools operating on a tight budget, and a more novel approach is needed.

A Proposed Solution to the Limitations of Radiology Training

To tackle this challenge, we developed a tool in order to (1) create a more realistic, tailored, and active learning experience for medical students, to (2) create a more diverse case assortment for radiology residents, and to (3) help assess their decision-making capabilities in high-stress situations. Furthermore, subspecialty level hands-on workshops were developed for senior residents and fellows with a focus on themes such as intracranial tumors or advanced acute stroke imaging, allowing for high-yield exposure to real-life cases at our institution.

Materials and Methods

We established a new PACS teaching system by utilizing the open-source software “dcm4chee” to be used as our image database management tool and integrated Weasis to be used as our image viewing tool. We utilized MySQL, a programming software that uses SQL for the programming language, to establish the database for dcm4chee. We incorporated JBOSS, an application server that behaves like an extended virtual machine, for running Java programming language applications, transparently handling connections to the database on one side, and, often, connections to the Web client user on the other. MyEclipse, an integrated development software, was used for our customized portions, with Java as the programming language. The instructions for download/installation of the source code are provided via online supplemental material.

Dcm4chee was used for Digital Imaging and Communications in Medicine (DICOM) image retrieval and storage. The Web Access to DICOM Object (WADO) feature of Weasis was used to achieve web-client user DICOM image access. Weasis, a Java applet built for dcm4chee, serves as a DICOM viewer without special software required. All functions (window width and level, zoom, measurement, etc.) are provided within the Weasis viewer. All PACS systems utilized clinically at our institution were equipped with an anonymization feature. Complete DICOM studies were seamlessly anonymized and sent from the hospital PACS to our shared secure server dcm4chee on the hospital secure network and were accessible by any computer in the network. While Weasis was developed to be a DICOM viewer and not equipped with a reporting system for the trainees to write up their brief interpretation, it can still be used for educational purposes. We initially had trainees hand write their impression points and review one-on-one with the attending. However, this was found to be inconvenient. To address this, we built a customized educational user interface with Weasis viewer with a reporting feature for trainees to write brief reports or impression points and for attending to write comments and provide feedback.

Installing Image Manager and Hospital PACS Setup

The first step is to find a suitable computer, install a database programming software, then install the image manager. MySQL, a database programming software, has to be installed before dcm4chee, the image manager, and is available online [10]. Installation instructions for dcm4chee are also available online [11]. In our case, the DICOM image manager dcm4chee was installed on one computer in the hospital secure network and left on at all times. Any basic computer with network connection is able to run as the server. However, the amount of free disk space on the computer determines the number of cases that can be stored. In our case, a PC with a Xeon processor 3.2 GHZ, 12 GB RAM, and 2 TB hard drive was used as the server. Once this is done, dcm4chee can be used for DICOM storage and retrieval using the application entities (AE) configuration that is being set. The hospital IT staff will require the following three AE configurations to set up dcm4chee as one of the copying destinations from the main hospital PACS system: (1) IP address of the host computer that dcm4chee is installed on, (2) port number, and (3) AE title. The IP address is also used for web access of dcm4chee from other hospital secure network computers. Both port number and AE title can be found at the AE section of dcm4cheeadmin page, shown in Fig. 1.

Fig. 1.

Weasis AE (application entities) title settings for DICOM management

DICOM images of desired cases are stored in the hospital PACS system. While working at a PACS station, attending radiologist can seamlessly copy any interesting case and push them to any DICOM database that have been set up as one of the destination nodes. The hospital PACS IT staff assisted with setting our DICOM image storing system (dcm4chee) as destination “WEASIS” shown in Fig. 2 (example of a CT Head without contrast image with patient data anonymized prior sending to “WEASIS”). Of note, the dose report has to be manually unselected and there are no automated safety nets in place to confirm that no patient information has been transferred to Weasis, and thus, a manual check on Weasis is required after the transfer to assure no patient data has been transferred.

Fig. 2.

Hospital PACS set up, copying studies to dcm4chee. Note that the ‘Dose Report’ images have been excluded from transfer (due to presence of patient information), the ‘Weasis’ destination that is previously set up with the help of hospital IT is chosen from the destination drop-down down menu, and the ‘Anonymize Exam(s)’ check-box is selected

Case Organization and Anonymization

We are not aware of dcm4chee having capability of sorting and grouping studies, but it is able to perform searching. Therefore, we utilized the anonymization feature of the PACS, where we used the “Set Values…” function before copying studies. We manually set values of the patient identifier: patient ID, first name and last name, all of which would have been semi-randomized if done automatically. This would allow organization and grouping of each category, which would be easily searchable. We set the patient ID to be RTXXX for resident cases or MSXXX for medical student cases, last name of the patient to be the radiology subspecialty, and the first name of the patient to be the organ system as shown in Fig. 3. For example, we set the patient ID as “RT001”, last name as “NEURO,” and first name as “BRAIN” because this is a head CT case suitable for resident teaching. In body imaging, the “organ” is not used as case label. Instead, we use chest, abdomen, or pelvis to label cases, which is the same information that would be provided to the trainees in the title of a study on PACS. Each of these fields can then be searched on the client end side, also allowing for the use of an asterisk. For example, typing “RT0*” in the Patient ID field would result in 99 search results that include RT001 to RT099. Note that the order of the cases being listed is based on the time they were added to the database. The client end of Weasis, however, does not allow for sorting of the search results. Exam dates can be sorted on DCM4CHEE and have to be named properly to identify follow-up studies. However, on the later discussed customized platform, exam dates cannot be sorted, but can still be grouped based on how the studies are titled.

Fig. 3.

Hospital PACS manual anonymization, with the three fields of ‘Patient ID’ (case identifier), ‘Last Name’ (specialty), and ‘First Name’ (organ) filled in

Installing the Web-based DICOM-viewer Weasis

After successfully testing the storage and retrieval function of dcm4chee, the DICOM-viewer Weasis can be installed and the installation instruction is available online [12]. We used Weasis as an embedded web applet. While the Weasis developer does not recommend launching Weasis as an applet in the web browser, due to several browsers potentially blocking the Java plugin, we have demonstrated successful launch in both Internet Explorer and Chrome browsers. However, the Java version on the client user computers has to be higher than the Java version used on the host server end. After Weasis has been successfully installed, DICOM images uploaded to dcm4chee can be viewed on any computer connected to the hospital secure network. The combination of dcm4chee and Weasis can serve as a stand-alone PACS system for education purposes.

Running the Server

After installation, the server has to be running at all times to enable accessibility on the client side, i.e., trainee. In case the server gets shut down or restarted, all the attending needs to do is to type in the Command Prompt window “location of Weasis\run.bat” as shown in Fig. 4a. It will automatically restart the server and it will be fully functional in 1 to 2 min depending on the server processor speed as shown in Fig. 4b. We put the new shortcut of run.bat to the “startup folder” and renamed it to “Weasis.bat” for ease of use, so that the server automatically loads as Windows boots up each time, as shown in Fig. 4c. The “startup folder” is a folder that contains a list of programs that run automatically each time the computer boots up. For Windows Vista, 7, 8, and 10 users, the “startup folder” can be found by pressing Windows key + R at the to open the Run box and in the run box type the following command: “shell:startup”. We also made a desktop shortcut shown in Fig. 4d, so that just in case the server has accidentally been shut down, it can be restarted simply by double clicking.

Fig. 4.

Running on the server side, a (top) execution command in ‘Command Prompt’, b (middle) successful running of the server which in this case has taken 1 min and 15 s, c (bottom left), startup folder short cut for running the server as an easier alternative, d (bottom right) desktop shortcuts which include the server shortcut which has to be running first, the Weasis application, and the optional document of answers to cases

The open source availability of dcm4chee and Weasis allows for web-page incorporation of Weasis and dcm4chee. Since dcm4chee and Weasis are developed based on Java language, we were able to develop our customization based on Java using MyEclipse software. Detailed instruction of downloading and installing our customization is included in the Appendix. After our version of customization, Weasis was used as a web widget with preservation of all functions including the added features. Once the customization installation is completed, running the customized version is the same as the base Weasis, i.e., running the “run.bat” file.

Utilization of Weasis and Customization

As one of the sets of cases, a combination of 100 essential call-related neuroradiology cases were collected, including previously saved dedicated institutional PACS teaching cases, cases with significant missed diagnoses from the previous years, as well as a few normal cases. For medical students, 100 cases that cover 5 radiology subspecialties were collected (including chest radiology, musculoskeletal radiology, neuroradiology, body imaging, and women’s imaging), each highlighting 20 important differential diagnoses. Cases were accessible through a web browser by trainees on any computer in the hospital connected to the shared secure hospital network, obviating the need for occupying an expensive dedicated radiology workstation. Residents and medical students, even simultaneously, could write up a very brief report, which was able to be reviewed by an attending radiologist. The cases could be categorized into different subspecialties, difficulty levels, and imaging modalities.

Results

Weasis System Flow Chart

Once a patient undergoes imaging (i.e., CT or MRI), the images are temporarily stored in the control console of the machine. Images are immediately pushed to the hospital PACS system once processed by a technologist in the form of DICOM files for the trainee or radiology attending to view and interpret. As the attending reviews hospital PACS images, they can seamlessly anonymize and send desirable teaching cases to the web-based PACS server dcm4chee. There are two ways of viewing the educational cases on the client end. One way is searching individual cases on dcm4chee, viewing it via the Weasis viewer, and then typing the impression on a word document/hand writing the impression to be reviewed with the attending. The base Weasis version is shown in the following Base Weasis section. The other method is by using our customized teaching system which incorporates Weasis PACS viewer as its browser to run Java applets; so that when trainees or attending access the teaching system’s website, they can pull the DICOM image from dcm4chee and display those cases on the Weasis PACS viewer. Therefore, by using our teaching system website, the attending can tailor and assign cases to individual trainees who will be interpreting studies, writing reports, submitting their results, and receiving feedbacks all online, which is shown in Fig. 5. The reporting function is part of the download package made available with this manuscript. It is not part of the standard Weasis.

Fig. 5.

Weasis system flow chart

Base Weasis as Stand-alone PACS

After successful launch of the server, a trainee can go to any computer connected to the hospital secure network and login at “http://IP address of the server:8080/dcm4chee-web3/”. The account is managed by the administrator, which is further explained within the User Management in dcm4chee section. Then, trainee can search the cases by patient name, patient ID, or study date. If nothing is put in the search criteria, all the studies uploaded to dcm4chee will be shown. In our example case, we put “NEURO” as our subspecialty and “RT*” meaning all cases that start with the characters “RT”, so we are searching all the neuroradiology cases for residents. The cases are populated in a list as shown in Fig. 6a. Weasis viewer can then be launched by clicking the eye icon on the right side of each study, and the Weasis viewer interface will open as shown in Fig. 6b. Measurement of the mass was added using the tools provided by Weasis.

Fig. 6.

Base Weasis as stand-alone PACS, a (top) dcm4chee search result, b (bottom) Weasis viewer

User Management in dcm4chee

In our training program, there are two roles: trainee and teacher. The teacher first assigns available cases to the trainee, who then goes on to write up a brief report of the case as an impression point(s). Afterwards, the teacher views those reports and provides comments to the trainees as feedback. Here, in the dcm4chee set up page (Fig. 7a), roles can be added with different rights or privileges by the administrator, usually abbreviated as admin. Then, users can be added with passwords and assigned with different roles by admin as well (Fig. 7b), so that all medical students/residents in the current batch can have their own accounts to work on cases, and different attendings can assign different cases to individual trainees. Only the admin has the capability to delete cases, so that teachers or trainees would not accidentally delete cases.

Fig. 7.

User management in dcm4chee, a (top) managing roles such as administrator, teacher, and student, b (bottom) managing each of the users

Assigning Cases to Student on Teacher Site

Both trainees and teachers will log in through the same login page each with their own accounts and will have different page configurations due to their different roles. Figure 8a is a screenshot of the teacher’s page setup. In this screenshot, we have the list of all the DICOM images pushed to dcm4chee. Teachers can preview the images directly on this webpage with the help of Weasis viewer and assign cases to individual medical students or residents. While assigning cases, attendings can view the image as if he or she is using the hospital PACS system, with all the necessary features, such as window/level and measurement tool that is available on a regular PACS viewer. In Fig. 8b, the pop-up box for the teacher can assign a case with a title and also a sample answer only visible to him/herself, to be able to use it as a reference answer later while reviewing the reports written by the trainees. Clicking “save subject” then creates the assignment/homework. After the assignments have been created, the teacher can pick each individual case and assign them to different trainee groups as shown in Fig. 8c and d.

Fig. 8.

Assigning cases to student on teacher site, a (top) teacher site interface, b (middle left) creating homework pop-up box, c (bottom) list of homework, d (middle right) assigning homework to different individual trainees

Case Interpretation on Student Site

There are three statuses of any case assigned to the students (Fig. 9a); one status is when the case has not yet been initiated which will display as “init” with an actionable button of “To Finish”; another status is after a case has been completed by a trainee but not reviewed by an attending displayed as “finished”; and lastly once a case has been reviewed by the attending, it will be displayed as “marked” and actionable button of “Detail” to review the comment left by the attending. The student is able to click on “To Finish” to bring up the Weasis viewer to view the studies (Fig. 9b). The most recent assignment is noted at the top of the list as noted here being “Homework001” (Fig. 9a). After clicking the “answer” button, the answering pop-up box is shown (Fig. 9c) and the student will type in his/her impression points of the current study. Approximately 25 medical students, 20 radiology residents, and 6 fellows have used the system at the time of the writing of this paper.

Fig. 9.

Case interpretation on student site, a (top) student site interface, b (bottom main window) student viewing studies with Weasis, c (bottom small window) answering homework pop-up box

Teacher Scoring

On the teacher site, as shown in Fig. 10a, there is a main page with the list of homework cases, showing the average score of the homework cases, the number of students assigned to each homework case, the number of homework cases that have been reviewed, the number of homework cases that need to be reviewed, and the number of trainees who have not yet finished it. To review the homework cases, the teacher will click the “Homework Info” tab at the left side tool bar. Once the student has submitted their work, the teacher will be able to view their answers and parallel to it is the reference answer as shown in Fig. 10b. The teacher is able to provide a score and feedback.

Fig. 10.

Homework scoring on teacher site, a (top) homework list, b (bottom) reviewing homework, scoring, and providing feedback

Survey Results

A total of 20 medical students and 20 residents/fellows participated in an online survey after acquiring Institutional Review Board approval (IRB: NCR191835) (Appendix A and B). Out of the 20 medical students, 19 would like more active learning during the radiology rotation and 17 would like PACS practices. Eleven out of 20 of the students who responded to the survey had used Weasis. Out of those 11, 10 would recommend their peers to use Weasis and 9 believe that Weasis is superior to the traditional method (passive shadowing) of radiology training. All 20 residents and fellows surveyed had used Weasis and would recommend it to their peers. Eighteen out of 20 believe Weasis improved their image interpretation skills, 19/20 would like to continue using Weasis, only 1/20 found it difficult to navigate Weasis, but he/she would still like to continue to use Weasis. Fifteen out of 20 think Weasis should be incorporated into their residency/fellowship training curriculum. In summary, (1) most medical students wanted more hands-on (i.e., PACS) experience, (2) residents/fellows liked Weasis and think it improves their image interpretation skills, (3) neither medical students nor residents thought it was difficult to navigate Weasis, and (4) both medical students and residents wanted Weasis to be incorporated into the curriculum.

Discussion

An attempt was made to tackle one of the limitations of current radiology training by utilizing a “dcm4chee” DICOM toolkit integrated by a web-based image viewing tool for trainees at all levels. As discussed above, the current limitations for medical students include the lack of a truly interactive and immersive learning experience representative of what they may encounter on clinical rotations, especially with regards to getting comfortable viewing an interactive image on PACS. A perceived limitation for radiology residents is the lack of adequate exposure to a variety of call cases that they may not encounter on the particular month(s) they are on a service, especially during first year of residency (R1), which may translate to lack of adequate preparation or sufficient confidence for taking call.

For this reason, we believe a simulated PACS using real cases that attendings can effortlessly upload especially during daily readouts over time without causing disruption in workflow would be a great asset to any radiology training program. Similar to the “skills/simulation labs” surgery, residents utilize to practice for the real world, this simulated PACS uses real images that are easily uploaded live to the simulated Weasis PACS for the purpose of training.

Installation of the standard Weasis requires approximately 3–4 h if the instructions are followed correctly and hospital/PACS IT are available for providing network-specific information such as IP addresses and AE configuration. There is no cost associated with installing the software if a significant number of cases are not being uploaded to the server side’s hard-drive and few end users simultaneously log in. Any person with some basic familiarity with windows command prompt should be able to install Weasis. By integrating a reporting system into Weasis, both trainee practice and evaluation could be achieved. By implementing Weasis, with or without the add-on reporting system, a real-time and easy-access freeware, platform-independent, image database can be established to simulate a real-world PACS.

The advantages of using “Weasis” for training over other similar educational options are plentiful, especially when it comes to the low cost of implementation, versatility of its use, and the customizability of the imaging cases that can be utilized. Notably, it can be tailored to benefit both medical students taking part in the radiology clerkship, or radiology residents during their early training. More recently, we have also implemented Weasis for senior residents and fellows for tailored hands-on workshops in specific topics, which will be discussed further. In addition, careful consideration was made with regard to Health Insurance Portability and Accountability Act (HIPPA) compliance and ways to further enhance the system. For example, during the image transfer, not all the series in a study are selected, i.e., documents and dose report images which contain patient information are excluded. As an additional level of security, the Weasis studies are only accessible through password-protected computers connected to the hospital secure network and not from outside the hospital.

Highlights of Weasis and Its Advantages

While searching for an ideal educational PACS system, the goal was to find a PACS viewer that was robust, web-based (platform-independent), low-cost, and open-source (customizable) system. We wanted a cloud-based PACS system that would allow attendings to seamlessly send desired anonymized studies from the hospital PACS during read-out to a shared secure server on the hospital network in a hassle-free fashion and without disrupting daily workflow. Cases could subsequently be immediately accessed by trainees on any computer in the hospital connected to the shared network, even side-by-side, while the attending reviews and finalizes the official report for the study. There are plenty of PACS viewers that have similar features, but the majority of them do not meet all three criteria [13]. One of them in particular, despite being conveniently available on a website, required uploading DICOM images through a website, which eventually proved cumbersome. This means that the attending would be required to download anonymized DICOM files from hospital PACS system and upload to this cloud-based PACS viewer. “Weasis”, due to its open source availability, and Java web-based features, provided the ability to build a customized platform for teaching purposes. In addition, it served as a stand-alone PACS system without a reporting feature being necessary if the end user does not desire it.

The cost of implementing is low because no new hardware is needed beyond existing computers. Any program could implement the system with minimal IT guidance. This is crucial for medical schools and hospitals on a low budget. As mentioned in the Colorado BAR labs paper, building an educational system is like running a small business [9]. However, not every institution is fortunate enough to extend their budget to purchase an additional expensive PACS system only for teaching, or to hire a contractor to build a simulation lab. Our approach using “dcm4chee” and “Weasis” allows institutions to customize and establish their own free PACS teaching program. However, customization of the teaching program is not necessary as “Weasis” can be run as a stand-alone PACS as mentioned earlier, serving as a practical teaching case collector for institutions with limited IT resources.

The versatility of “Weasis” is vast, as it can be used as stand-alone PACS system or be incorporated into a comprehensive educational program, which is what we have demonstrated in this paper. The customizability is also unique in that teaching cases can easily be accumulated for learning, didactics, and evaluation purposes. The content of the teaching and the exams will be more versatile and similar to real-world clinical scenarios, taking the needs of the institution into account. This may be in the form of an emphasis on areas where there is a large number of cases, for example gunshots in a level I trauma center, or to highlight cases where upper level trainees may not get adequate exposure to, such as neurodegenerative disorders. Most importantly, an attending can send live cases during read-out in a fast and seamless manner, with little disruption to their workflow, significantly increasing the chance of the system to be used. Ultimately, each program can tailor the system to emphasize their strengths or address their weaknesses in order to help meet their educational goals.

Benefits to Medical Student

In the case of medical students, the benefits include having an immersive experience of that as a radiologist, becoming more comfortable browsing through images and operating a PACS which will help them in their future rotations and career, learning to arrive at a diagnose at a deeper level and potentially motivating them to pursue radiology as a future career. Prior to using this tool, medical students simply shadowed during the read-outs and were provided with at least six dedicated in-person lectures, in addition to attending the daily radiology resident lectures. They were also provided a textbook during the rotation and guided to an online set of interactive radiology tutorials. However, the majority of medical students, non-radiology residents, and even non-radiology attendings have expressed frustration with the lack of adequate radiology education, especially a hands-on experience. For those medical students who take part in the radiology clerkship, this educational PACS could be the first step in addressing this issue and can add tangible value to their education. Students interested in pursuing clinical or surgical specialties would benefit from being able to independently make a quick assessment prior to the radiology report becoming available. An educational PACS such as this can help address this lack of dedicated radiology education that exists in medical schools.

While using the system, students can simulate being a radiologist, independently formulating an opinion, writing up a brief set of impressions, without the need for occupying an expensive PACS workstation. Multiple students could use the system at the same time on different computers. The cases can be labeled such that they are categorized by different subspecialties, difficulty levels, and imaging modalities. When an interesting case comes up, the attending can quickly upload it to the educational PACS for students to view it. Normal cases were not included since the system was being used as a tool for teaching important diagnoses. In addition, although not done at our institution, this system can also be easily used for evaluation and examination purposes. Once attending radiologists log on to the customized platform, reports pending review are listed for them. This is a very quick and easy task for the attending radiologist because the associated answer is already provided. They do not need to review the imaging again. In addition, they will be able to detect any lack of knowledge for the trainees and provide feedback or incorporate necessary information in upcoming lecture series to address those weaknesses. While there is no email alert system at the moment, it is something that could be developed in the future.

Benefits to Junior Radiology Residents

In the case of junior radiology residents, the educational PACS could serve to supplement their array of cases on a particular rotation, help them better prepare for call, and even be evaluated as part of the pre-call Objective Structured Clinical Examinations (OSCEs) for first year residents. There are cases that are of high educational value that are commonly missed, yet may not arise while a radiology resident is on a particular rotation. Timely diagnosis of some of these pathologic conditions, while subtle on imaging, such as focal vertebral artery dissection, is of paramount importance. Other diagnoses such as herpes encephalitis or tuberculosis have significant important implications with respect to proper immediate notification of the clinical teams, either due to the need for immediate treatment or due to the potential risk of contagion. Many attendings try to keep track of these, but with potential HIPPA violations, and the chaotic nature of the daily workflow, many of these cases are lost in the abyss. With Weasis, an attending can upload these cases to the educational PACS during the readouts or separately at other times, in order to create a database of essential and difficult cases that all residents must review before taking their first call. For simulation of on-call experience by residents and fellows, a handful of normal cases were included to more accurately represent a real-life experience. The list can be augmented over time or cases can be redacted, to gradually generate a valuable set of cases. Resident can review these cases during their free time after hours or possibly on rotations where there is down time available, without slowing the department’s workflow. At our institution, this tool has been fully implemented in the neuroradiology division for all trainee levels and very well received, but is most popular among junior residents who are preparing for on-call duties. An attending radiologist reviews the cases with trainees on average for an hour during one or two afternoons per week during a 4-week rotation. During this time, the other attending radiologist on duty focuses on interpreting cases and triaging questions and protocols.

Traditional case conferences or multiple-choice questions to some extent artificially simplify a case, by actually showing the abnormality. However, in real-life cases, the resident has to not only be able to arrive at a differential diagnosis for an abnormality, but has to first detect it by searching through hundreds of images. Furthermore, the concept of satisfaction of search with respect to secondary diagnoses or major incidental findings are important considerations, such as pulmonary emboli on a body CT or pulmonary nodules on a CT head/neck angiography.

Benefits to Senior Radiology Residents and Fellows

Many small and intermediate-sized radiology residencies and neuroradiology fellowship programs may not have an abundance of cases highlighting certain topics such as intracranial tumors, high-level traumas, neurodegenerative disorders, advanced acute stoke imaging, etc. This may prevent trainees from gaining adequate exposure to certain types of cases depending on the institutional focus, size of hospitals, and outpatient centers as well as geographic location. Weasis can be implemented in order to create a tailored and active learning experience for trainees simulating a realistic exposure to cases as they would present in the day-to-day clinical setting.

For example, a hands-on workshop for adult intracranial tumors was established. Fifty cases of adult brain and skull base tumors, some containing multiple comparison studies, were chosen, including a various gamut of diagnoses ranging from normal or metastatic disease to rare cases, or ones with complex findings such as pseudoprogression or pseudoresponse. Considering the anonymization of each case including its date, in order to have comparison studies be associated with a main study, we used a character at the end of the ‘Patient ID’ field in both alphabetical order and chronologically. For example, for case BT001, the most recent comparison study was labeled as BT001a, the next ones as BT001b, BT001c, etc. This allowed for viewing all the studies from different dates with respect to a case simultaneously when the term “BT001” was searched. In addition, our PACS anonymization automatically assigns a random date of birth to each case such that the anonymized age is close to the original age of the patient within a couple of years, allowing this information to keep its role in arriving at a diagnosis.

Other Considerations for the Department

At least one radiologist and one individual from IT are required to maintain this program. On a daily basis, however, any individual can simply assure that the server is running in order to allow access from the client side by users. However, from time to time, IT support may become necessary whenever a system or network upgrade takes place, especially with respect to Java-advanced settings on the client side. This program may be affected and IT support should either help with adjusting the web browser settings or at least notify radiologists regarding the upgrade information. The maintenance instructions have been provided to the section chiefs who can disseminate the information if and when necessary. With respect to user accounts, it mainly applies to medical students and at any given 4-week rotation, as there are never more than 10 users (commonly 4–5 users). Therefore, the user profiles can be reset and recycled each month. For radiology residents, in-person review of cases is done with the attending by going over the notes taken down by the residents while reviewing the cases.

Patient health information is a major concern when applying DICOM images for education and research purposes. In the study done by Fetzer, David et al., they focused on HIPAA privacy rule and Protected Health Information when researchers use DICOM image for research. Several anonymizing application tools failed to remove all PHI, and the processed DICOM file could not be fully considered de-identified. Additionally, even if all the PHI was removed from the DICOM header, some imaging devices directly print patient information directly on the image, which would then require image editing for the DICOM file to be de-identified [14]. For example, most ultrasound images have the patient information essentially burned into the image itself and thus ultrasound cases were excluded in our project. Therefore, accidental breach of confidentiality remains a possibility. Thus, we believe keeping the cloud-based system within the hospital network allows us to keep patient information safe while educating students at the same time.

Of note, a patient’s history was generally not provided for our selected cases. A brief history could be added to the study “subject title”, as shown in Fig. 8, if needed. In high-volume emergency settings, inadequate or simple default histories are typically provided, such as “pain” or “altered awareness”, therefore requiring trainees to be vigilant in considering broad differential diagnoses. To some extent, the lack of patient information has impacted narrowing down the final differential diagnosis, or arriving at a definitive final diagnosis. We believed it has taught trainees to consider conditions that ordering clinicians may not be contemplating, such tuberculosis presenting with neck lymphadenopathy, in addition to the fact that by meticulously evaluating important structures, such as vertebral arteries, they will appreciate the concept of ‘satisfaction of search’. It has also allowed open-ended discussions about cases as to why certain diagnoses are favored versus others.

Consistency in labeling is very important when multiple users are uploading cases. For the nearly 350 cases, a total of 3 users took part in the transfers, with 1 super-user giving instructions, providing supervision, and assuring consistency in all of the labeling. The section heads and the clerkship director provided a catalogued list of cases chosen from their teaching files before the upload. While there are no upload or coordination limits, thus far, unplanned random on-the-go upload of cases has not been a staple of our approach.

Furthermore, the educational PACS can be used for testing residents as well. This finally gives resident programs an ability to test residents with an interactive system where they can evaluate their reports rather than simply being limited to a multiple choice or other simple testing systems. Additionally, implementation of voice recognition software may be explored given the customizable nature of the user interface or by having the trainee dictate the impression points separately in Microsoft Windows which has Dragon Medical Add-in enabled. Further additions may be implemented as well. Balint, Brad J. et al. in a single-center study has shown that during the process of image interpretation, on-call radiology residents are often times interrupted by pager and phone calls for in-person consultations, imaging protocol prescription, answering incoming telephone calls, and returning pages, which negatively impact their diagnostic accuracy [15]. These workflow disruptions are unscheduled, forcing the radiologist to multitask, and usually in between of image interpretation, causing residents to forget tasks. Incorporating pre-recorded phone call disruptions, from technologists and ordering clinicians, using mobile apps, is another feature to consider, to at least empower our on-call residents to be more prepared for this kind of scenario.

Conclusion

By implementing Weasis and an add-on reporting system, a real-time and easy-access freeware platform-independent image database can be established to simulate a real-world PACS. The advantages of using “Weasis” for training over other similar educational options are plentiful, especially when it comes to the low cost of implementation, versatility of its use, and the customizability of the imaging cases that can be utilized. The cost of implementing is low because no new hardware is needed beyond existing computers. The versatility of “Weasis” is vast, as it can be used as stand-alone PACS system or be incorporated into a comprehensive educational program, which is what we demonstrated in this paper. And, the customizability is unique in that teaching cases can easily be accumulated for learning, didactics, and evaluation purposes. The content of teaching and the exams will be more versatile and similar to real-world clinical scenarios. Furthermore, attendings can send live cases during read-out in a fast and seamless manner, with little disruption to their workflow.

Implementation of our proposal provides a new opportunity for medical schools and radiology residency programs in offering medical students a valuable and more realistic radiology experience and radiology residents with better preparation for independent call.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Abbreviations

AE

Application entities

DICOM

Digital imaging and communications in medicine

HIPPA

Health Insurance Portability and Accountability Act

PACS

Picture Archiving and Communication System

PYG

Post-graduate year

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.

Supplementary Information

The online version contains supplementary material available at (10.1007/s10278-020-00405-2).