Impact of PACS and Voice-Recognition Reporting on the Education of Radiology Residents

Abstract

Rationale and Objectives: The introduction of picture archiving and communication system (PACS) has decreased the time needed to interpret radiology examinations resulting in an increased workflow. Because of concerns that the increase in exam throughput and the use of voice recognition may have a negative impact upon radiology resident education, a survey was conducted to assess the impact of PACS and voice recognition. Materials and Methods: Residents at four diagnostic radiology training programs were surveyed. Survey topics included resident demographics, didactic and technical issues, and areas for improvement. Results: One hundred thirty-four residents were polled with 42 respondents (42/134, 31.3%). The majority have been using PACS for more than 1 year (29/41, 70.7%) to interpret 75–100% of cases (33/39, 84.6%). A majority believed PACS is a superior teaching tool to printed film (28/38, 73.7%). However, only a minority (9/40, 22.5%) indicated that PACS was always used to contain teaching files and to conduct departmental conferences (5/40, 12.5%). The majority of respondents believed PACS have decreased the time needed to interpret diagnostic examinations (33/41, 80.5%). A majority (80.6%, 25/31) indicated that voice recognition takes more time than the traditional dictation and transcription process, where 51.3% (20/39, 51.3%) felt that voice recognition works well less than 50% of the time. Conclusions: Residents believe that PACS has positively affected their learning experience but indicate that it can be better utilized for resident education. Residents believe that voice recognition is less reliable and more time consuming than the traditional dictation system.

Introduction

Digital imaging and the picture archiving and communication system (PACS) continue to replace conventional radiographic film [1–4]. The benefits derived from PACS are extensive and positively affect health care by reducing costs, improving efficiency, and enhancing patient care [5–8]. For the radiologist, PACS facilitates the interpretation of radiologic studies in less time than with analog imaging, thereby increasing work flow [1–5, 8, 9].

Resident education at the PACS workstation must compete with this increased work flow. To optimize learning and efficiency, residents must acquire PACS skills early and consistently throughout their training [4, 10–13]. However, many radiology departments may not provide the resources for effective training programs [4, 11–13]. Training in PACS should instruct attendings and residents on the basics of image and report retrieval and on the various workstation tools to enhance image interpretation and resident education [4, 11, 12].

Although the effect of PACS on the health care system has been extensively studied, the effect of PACS on radiology resident education has been minimally addressed. In 1992, the potential applications of PACS for education and research were described including the benefits of using PACS to create an automated digital teaching file [14]. In 1997, a prospective cohort study investigated the impact of changing from a film-based image interpretation system to one using PACS workstations on the training of radiology residents [15]. The results of this investigation indicated that there was a decrease in the autonomous and active participation by radiology residents in the image interpretation and reporting process following the change to digital image workstations [15].

In an investigation conducted in 1999, the effect of PACS on resident education was assessed by a survey of radiology residents [13]. In this investigation 50 radiology residents were polled from two university-based radiology residency programs with 20 respondents [13]. The conclusions of that examination found that residents believed PACS positively affected their learning experience and did not negatively affect the quality of resident education or patient care [13]. Many important action steps were identified from the survey, including the need for formal PACS training and the use of PACS to create a digital teaching file [13].

The current investigation serves to confirm and expand on the findings from the 1999 examination with the administration of an updated survey. The number of radiology residency programs that were included in this survey increased from two to four to increase the sample size and to evaluate the opinions of residents in programs with varying PACS utilization. The second survey examines more closely the image interpretation process with emphasis on the effect of increased workflow on resident education. The penetration and effect of the voice-recognition system on the dictation process were also examined.

As previously stated, a combination of increased work flow, referring physicians’ expectations, and introduction of new technologies created the need for new methods of radiology education and practice [13]. Because of concerns that PACS and voice recognition may negatively affect resident education, combined with the opportunity to improve radiology education, the following survey of resident perceptions and recommendations was undertaken.

Materials and Methods

An anonymous survey was administered to all diagnostic radiology residents at four large university-based diagnostic radiology residency programs, and the results were tabulated. The diagnostic radiology residency programs were chosen based on the number of residents and the degree of PACS penetration for image interpretation at each institution. The survey was distributed and collected through the residency program director’s office. The residents at radiology residency program A (n = 40), radiology residency program B (n = 38), radiology residency program C (n = 34), and radiology residency program D (n = 22) were surveyed.

The survey was conducted from February 2003 through March 2003, at which time the four residency programs had mainly converted to PACS for daily interpretation. Penetration of voice-recognition reporting was variable at the different radiology residency programs. At radiology residency program A, residents used Impax PACS (Agfa, Ridgefield Park, NJ) and Powerscribe (Dictaphone, Stratford, CT) for interpretation and reporting. At radiology residency program B, residents used Impax PACS (Agfa) and TalkStation (Talk Technology, Bensalem, PA) for interpretation and reporting. At radiology residency program C, residents used the Pathspeed PACS (General Electric Medical Systems, Waukesha, WI) and TalkStation (Talk Technology) for interpretation and reporting. And at radiology residency program D, residents used the Impax PACS (Agfa) and TalkStation (Talk Technology) for interpretation and reporting.

The survey was divided into two sections. Section 1 encompassed demographics, including year in residency, percent cases interpreted on PACS, length of time interpreting on PACS, and evaluation of their computer skills. Section 1 also covered the educational use of PACS, the image interpretation, review, and dictation process, and perception of the voice-recognition system. Possible responses were provided in section 1, and respondents were asked to write commentary where appropriate. Section 2 provided open-ended questions with occasional suggestions and yes-or-no direction to facilitate commentary and to provide suggestions to improve the educational experience of image review of digital images using PACS. Topics included the educational use of PACS, PACS effect on image interpretation, review, and dictation, and the overall effect of PACS on resident education. Resident response varied from question to question where not all respondents answered every question in the survey; however, there was no significant difference in resident response between sections 1 and 2 of the survey.

Results

Section 1

Demographics

Of the 42 total respondents, 10 were first-year radiology residents, 7 were second-year residents, 7 were third-year residents, and 4 were fourth-year residents, and 14 respondents did not reveal their year of training. (Results of section 1 are included in Table 1.) The majority of respondents (25) indicated that they interpret 75–100% of cases on PACS, eight residents responded that they interpret 100% of cases on PACS, and six residents reported that they currently interpret 50–75% of cases on PACS. Twenty-nine residents had been using PACS for at least 1 year, whereas 12 residents had been interpreting images from digital examinations for more than 7 months at the time of the survey. Six residents rated their computer skills as expert, 18 residents rated their computer skills as excellent, 16 residents rated their computer skills as good, and 1 resident rated his/her computer skills as adequate.

Table 1.

Results of Section 1

Year in residency?	First (10)	Second (7)	Third (7)	Fourth (4)			NR (14)
What percentage of cases do you interpret on PACS?	0% (0)	0–25% (0)	25–50% (0)	50–75% (6)	75–100% (25)	100% (8)	NR (3)
How long have you been interpreting studies on PACS?	Days (0)	Weeks (0)	Months (12)	1 Year (5)	Years (24)		NR (1)
How would you rate your computer skills?	Poor (0)	Adequate (1)	Good (16)	Excellent (18)	Expert (6)		NR (1)
What is the effect of PACS on the length of time of interpretation of a diagnostic evaluation?	Decreases (33)	No change (0)	Increases (2)	I don’t know (4)	Too little experience to compare (2)		NR (1)
How often is PACS used to contain teaching file cases in your divisions or department?	Never (1)	Infrequently (4)	Sometimes (26)	Always (9)			NR (2)
How often are divisional or departmental conferences conducted solely or primarily on PACS?	Never (0)	Infrequently (11)	Sometimes (24)	Always (5)			NR (2)
Do you have enough PACS workstations for resident use?	Insufficient (23)	Just right (6)	Sufficient (12)				NR (1)
What is the speed that attendings go through cases on PACS versus printed film?	Too fast (3)	Just right (28)	Too slow (2)	I don’t know (6)			NR (3)
Prior to your readout sessions with attendings, you preview what fraction of cases?	0–25% (3)	26–50% (5)	51–75% (6)	76–100% (15)	Very variable (11)		NR (2)
During readout, in what fraction of cases do you feel that you have actually visualized all of the findings that attendings ask you to put in the report?	0–25% (4)	26–50% (7)	51–75% (7)	76–100% (21)	Very variable (3)		NR (0)
How many cases (all modalities) do you review with the attending prior to final dictation?	0–5 (4)	6–10 (5)	11–15 (9)	16–20 (6)	20+ (12)		NR (6)
Do you view the exam as you dictate or use written notes taken during the review with staff?	Always view (8)	Mostly view, sometimes notes (17)	Sometimes view, mostly notes (12)	Never view (0)	Very variable (3)		NR (2)
What percentage of cases do you dictate on voice recognition?	0% (3)	0–25% (7)	25–50% (7)	50–75% (2)	75–100% (10)	100% (10)	NR (3)
If voice recognition is used, what fraction of cases do you usually predictate?	0–25% (6)	26–50% (5)	51–75% (7)	76–100% (14)	Very variable (6)	NA (1)	NR (3)
What percentage of the time does your voice-recognition system work well?	0% (1)	0–25% (8)	25–50% (11)	50–75% (13)	75–100% (6)	NA (1)	NR (2)
What percentage of the time does your PACS system work well?	0% (0)	0–25% (1)	25–50% (1)	50–75% (14)	75–100% (25)	NA (0)	NR (1)
If you have had previous experience with traditional dictation and transcription, how does the time you spend with the entire VR process compare?	VR takes less time (2)	VR take same amount of time (4)	VR takes more time (25)				NR (11)

Survey questions for section 1 are shown with provided responses and resident replies. Not all residents answered all questions. NA = not applicable, NR = no response.

Educational Use of PACS

When asked about the use of PACS to contain teaching files in their divisions or departments, 26 residents replied that their departments sometimes use PACS to contain teaching files, 9 replied that their departments always use PACS to contain teaching files, 4 responded that their departments infrequently use PACS to contain teaching files, and 1 stated that PACS is never used. Similarly, 24 residents replied that PACS is sometimes used primarily to conduct departmental conferences, 5 responded that it was always used, and 11 responded that PACS is used infrequently to conduct departmental conferences.

Image Interpretation, Review, and Dictation

Concerning the effect of PACS on the length of time of interpretation of a diagnostic evaluation, a majority (33 residents) responded that PACS decreases the amount of time, whereas 2 responded that it increases the length of time of interpretation. Four residents responded that they did not know, and two residents felt that they had too little experience to compare.

Prior to final review with attendings, 15 residents indicated that they preview 76–100% of the cases, 6 residents preview 51–75% of cases, 8 preview less than 50% of cases, and 11 indicated that the number of cases they preview is variable. When asked about the speed that attendings review cases on PACS versus those on printed film, a majority of residents (28) felt that the attendings speed was just right, 3 indicated it was too fast, 2 too slow, and 6 stated that they did not know. Twenty-one residents felt that they actually visualized all of the findings that attendings indicated to place in the report in 76–100% of cases, 7 stated they visualized all the findings in 51–75% of cases, 11 responded that they visualized all the findings in less than 50% of cases, and 8 residents felt it was variable.

Twelve residents responded that they review more than 20 cases with the attending in one readout session prior to final dictation, six indicated that they review 16 to 20, nine residents review 11 to 15, five residents review 6 to 10, and four residents responded that they review 0 to 5 cases with the attending prior to final dictation. Eight residents indicated they always view the exam as they dictate, 17 residents responded they mostly view the exam and sometimes use written notes as they dictate. Twelve residents replied that they dictate mostly by written notes, and sometimes by viewing the image directly, and three residents responded that it was variable.

Twenty-three residents felt that there were insufficient PACS workstations for resident use, whereas 6 responded there were just the right amount available, and 12 residents believed that there were sufficient PACS workstations. When asked about how well their PACS system worked, 25 residents felt that it worked well 75–100% of the time, 14 residents felt that it worked well 50–75% of the time, whereas only 2 residents felt that it worked well less than 50% of the time.

Method of Dictation

Twenty residents responded that they dictate 75–100% of the cases on a voice-recognition system. Two residents responded that they use voice recognition for 50–75% of cases, and 14 residents responded that they use voice recognition for less than 50% of the cases that they dictate. Three residents indicated that they never use voice recognition to dictate cases. Of those residents using voice recognition, when asked about the frequency of predictation, the process of dictating a preliminary report prior to reviewing the case with the attending, 14 residents indicated that they predictate cases 76–100% of the time. Seven predictate 51–75% of the time, and 11 predictate less than 50% of the time. Six residents indicated that it is variable. Twenty-five residents felt that voice recognition takes more time compared with traditional dictation and transcription. Four residents believed that voice recognition takes the same amount of time, and two indicated that voice recognition takes less time. When asked about the reliability of voice recognition, 13 residents believed that their voice recognition worked well 50–75% of the time, 6 felt that it worked well 75–100% of the time, 19 felt that it worked well 0–50% of the time, and 1 resident felt that it never worked well.

Section 2

Educational Use of PACS

In response to open-ended questions, 28 out of 38 residents felt that PACS is a superior teaching tool to printed film. Residents noted that PACS was superior because of image manipulation (scrolling, windowing, magnification, measurement), comparisons to prior studies, decrease in ancillary time required to find and hang films, interpretation of more cases, and more one-on-one teaching. Also, identification and storage of teaching cases for conferences, presentations, and teaching files all make PACS superior to printed film. There was some concern that PACS was superior for cross-sectional studies but not for plain films, that resolution was lost, and that at times findings were harder to see. One resident felt that it was a superior interpretational tool compared to film, but not necessarily a superior teaching tool. Six residents felt that they did not have adequate experience to compare.

Adequate training and familiarity are important to maximize the use of a new computer system. In regard to PACS, 30 out of 39 residents who responded were not formally trained but “learned on the job.” Most learned from watching others and indicated that it would be useful to have a formal training session. Seventeen out of 39 residents had some sort of PACS training but felt it was incomplete. Two residents did not receive training and felt it did not matter as it was easy to pick up and use.

As PACS continues to penetrate more academic medical centers, its use for radiology education will increase. Twenty-nine out of 39 residents responded that their departments do use PACS for teaching cases. Some reported that there are large PACS teaching file collections, but that it was department dependent. This may be the result of differences in examination volume between departments and the availability of storage space necessary to store teaching files within the PACS system. Many stated that although not widespread, the practice of maintaining teaching files and using PACS for conferences is increasing. Six residents said that their division or department currently does not use PACS for teaching cases. Four residents stated it was variable as their particular PACS is too slow and that it is attending dependent.

A benefit of having access to cases on PACS is the ability to download high-quality images for teaching files, conferences, and presentations. Twenty-three out of 36 residents who responded indicated that they have downloaded images from PACS. Many felt that the process is quick, easy, and with excellent image quality. However, some residents noted that the process was difficult with variable success. Some residents noted there was a lack of training in the methods of downloading images and indicated that their system crashes easily with frequent network errors. Thirteen residents noted that they have not downloaded cases from PACS but would like to learn this technique.

Image Interpretation, Review, and Dictation

Since the introduction of PACS, 16 out of 39 residents felt that PACS has increased viewing images directly when they dictate. Residents noted that PACS makes studies easily available and easier to view while dictating. With PACS they are able to review a large number of images without the extra time required to hang films. Residents noted that the ability to annotate images directly was helpful during dictation. Eleven residents indicated that there was no change, or that PACS had no effect on viewing images while dictating. Some residents noted that using voice-recognition software actually interferes with viewing images directly and slows interpretation as they have to look at a third monitor for dictation. Eleven residents stated that they had little or no experience to compare.

When asked if they took notes (hand written or typed) during readout sessions using PACS, 19 out of 39 residents stated that they do take notes and felt this is distracting. Many indicated that notes are essential and do help make attending readouts faster, but can be distracting if lengthy note taking is involved. Conversely, some residents felt dictations were less accurate and made learning more difficult. One resident stated that notes are taken because attendings go too fast, which results in residents writing and not looking at images. Thirteen of 39 residents did take notes but indicated that this was not distracting. They stated that notes are used as memory aides and are helpful to identify key points, findings, and measurements that otherwise would be missed in the final dictation. Some residents found it less distracting to type notes into the voice-recognition software than writing notes. Three residents take notes but did not comment if they were distracting, but stated that most of the time they take notes on the voice-recognition system. Three residents did not take notes during readout, but would preview with notes and confirm during readout. One resident felt that it was variable and that it depends on department and modality.

When asked if they viewed images directly when dictating, 30 out of 39 residents said that they do view the images while dictating. They mentioned that they look back and forth from screen to notes and always try to look at the exams if time allows. However, residents noted that at times it is difficult to view images because of time constraints, voice recognition difficulties, lack of PACS workstations, and increased work flow. Seven residents indicated that they sometimes look at the images while dictating and other times they do not. Again, they mentioned it was variable depending on the availability of PACS workstations, time constraints, and voice-recognition software difficulties. Two residents stated that they do not look at the images and mainly dictate from notes.

Twenty-two out of 42 residents responded as to why they performed dictations without viewing the images while dictating. Reasons given include large volume of studies to dictate, limited time, limited number of PACS workstations, straightforward cases, easier to dictate from memory than from PACS, and voice-recognition system difficulties. Two residents stated that they rarely or never dictate without looking at the images.

Thirty-two out of 39 residents responded that the attending usually controls the mouse during image interpretation. Six residents indicated that it is variable depending on the attending, and one resident noted that he/she always controls the mouse. Residents noted a number of benefits for having the attending control the mouse, including increased efficiency, decreased “missed” findings, and the ability for residents to access patient information. Residents noted that when the attending controls the mouse they learn how to more efficiently review cases. However, they indicated that by not controlling the mouse they do not have the opportunity to develop PACS hand/eye coordination and found it difficult to follow anatomic structures.

Twenty-one residents responded that they would change something about the readout process using PACS. Suggestions for improvements include more PACS workstations, more efficient PACS systems, and “better” monitors. Other suggestions included easier access to prior examinations, more time to preview and predictate cases, improving voice recognition, and the ability to construct, save, and organize work lists. Residents also noted that they would like attendings to give a detailed review of the preview process and have them clearly summarize findings at the end of each case. Residents would like attendings to dictate more cases to increase the time residents spend on reviewing cases. Eight residents indicated that they would not change anything about the readout sessions using PACS.

PACS Overall

Overall, 28 out of the 31 residents felt that PACS benefits radiology resident education. Residents believed that PACS improves efficiency by eliminating the ancillary time of hanging films, tracking down films, and minimizing lost films. It allows for viewing greater number of cases, one-on-one teaching, the creation of teaching files, and better preparation for boards. Residents felt that PACS improves diagnostic accuracy with the use of image manipulation, comparison to prior studies, and multimodality comparison. Recommendations for improving the use of PACS in resident education include increasing the number of PACS workstations, establishing an organized PACS teaching file, and providing formal PACS training sessions. Residents indicated that voice recognition for dictation needs to be improved, and that although voice recognition has many problems one resident felt that reports are better with it. Two residents indicated there was no need for change and one resident felt that the introduction of PACS has had no real impact on resident education. A few residents noted that a PACS-only training will not allow them to confidently read cases on printed film, which will limit future employment opportunities.

Discussion

The use of PACS and the voice-recognition system in medical centers is increasing [7, 9, 12, 16, 17]. Physicians currently train and practice in a filmless environment in a number of medical centers, which stresses the importance of adequate PACS training and continual evaluation of PACS on education. The number of programs that were included in this survey was increased from two to four in order to increase the sample size and to evaluate resident opinions in programs with varying PACS utilization. Residents in different years of training, with varying computer skills, and PACS experience were questioned regarding the use of PACS and its impact on image interpretation and education.

The results of this survey provide reassurance that although radiology examinations are interpreted in less time, residents feel that PACS has a positive impact on their education. Residents believe that PACS is a superior teaching tool to plain films with a vast teaching potential. This teaching potential has yet to be fully realized where there are many underexploited educational tools. The survey has identified possible action steps to help improve the use of PACS in resident education. One important action step is the need for formal and ongoing PACS instruction. Dedicated instructional sessions, written manuals, and training on how to download images would be useful. Designing a training program where faculty and residents can continuously teach each other and share PACS functions can enhance interpretation and education. Another area of improvement is the creation of a centralized, digital teaching file that is organized by organ system or American College of Radiology (ACR) collection coding system. This allows improved access to teaching cases for presentations, conferences, and for boards review.

While there are clearly benefits of PACS, there exist a minority of residents who express concerns that training in a “PACS-only” program will leave them without the proficiency in interpreting cases on analog film. As hospitals continue to use analog film they feel that this will limit their employment opportunities. However, with the continued shift to an all-digital radiology department this will not be a major problem in the near future. Other concerning points that were identified from the survey include residents dictating cases without the image in front of them, reviewing a large number of cases prior to final dictation, and not visualizing all findings during final readout, which were attributed to a combination of an increased work flow and limited PACS workstations.

Based on resident suggestions on improving the image interpretation, review, and dictation process, residents feel it would be beneficial to preview cases prior to final review with the attending. Residents would benefit from previewing cases because they would have the opportunity to control the mouse, develop their own interpretation technique, learn hand/eye coordination, and are able to initially identify key findings. During the preview process residents can take notes and annotate images to decrease the amount of note taking during the final review with attendings. If voice recognition is available residents could predictate their findings and thus eliminate the need for handwritten notes (Fig 1).

Fig 1.

Flow chart of the interpretation/dictation process with and without previewing and voice recognition. If voice recognition is used properly with adequate training and support, and with the use of its specialized functions such as customizable templates/macros, it may prove to be beneficial to resident education if incorporated into the preview, review, and dictation process.

Regarding improvements for the readout sessions, residents emphasized the importance of having attendings clearly and thoroughly explain all findings and summarize interpretations. Attendings should explicitly ask each resident if they have visualized all of the findings in an examination before moving on to another case. Through this, residents would have the opportunity to confirm their findings and visualize new findings with the attending. Residents learn technique, efficiency, and are taught to thoroughly interpret cases. Improvements for final dictation include having residents dictate cases after reviewing a manageable number of cases with the attending. More importantly, residents should always dictate while visualizing the image. This process is dependent on the availability of sufficient PACS workstations for resident use.

As technology continues to improve the voice-recognition system along with continued acceptance amongst radiologists, its use will continue to increase [17, 18]. Up to half the residents surveyed are now using voice recognition to dictate a majority of cases. However, a majority of residents feel that voice recognition takes longer than the traditional dictation process. A majority of those who use voice recognition also feel that it works well less than 50% of the time. These opinions are very concerning as some residents feel that voice recognition is a distraction to resident education. These views of voice recognition can be the result of lack of training, improper training, and the frustration of using new and unfamiliar software [17]. Voice recognition has been shown to decrease report turnaround times and reduce costs [17, 19]. With improvements in voice-recognition technology, the accuracy rates have increased to 98.5% [18, 19]. However, with the current difficulties of using voice recognition, many physicians feel that they need to be transcriptionists as well as radiologists and feel that it decreases their productivity [16, 20]. Further objective studies are needed to compare the traditional dictation process (dictation, transcription, editing, and signing) with voice-recognition reporting to examine accuracy and the time required to complete reports [19, 20]. It is probable that if voice recognition is used properly, with adequate training and support, and with the use of its specialized functions such as customizable templates/macros, it may prove to be beneficial to resident education if incorporated into the preview, review, and dictation process (Fig 1) [17].

This evaluation of PACS and voice recognition has limitations. The survey is based on subjective questioning without the power of objective data. The survey questions serve to elucidate residents’ subjective perceptions of PACS and voice recognition without definable standards. The number of residents who responded was only 42 and included a number of first-year residents. A larger sample size would provide better statistical power, convey a wider range of opinions, and decrease bias. Also, objective studies are necessary to evaluate the effect of PACS on resident education. As previously stated, association with objective measures might confirm the subjective opinions found in this survey and may provide further insight of PACS on radiology education [13].

Conclusions

This examination successfully accomplishes the goal of expanding the subjective questioning to a larger group of residents at four different radiology residency programs. Above all, the results provide reassurance that PACS benefits radiology resident education while providing recommendations for improvement. In conclusion, residents believe that PACS has positively affected their learning experience but feel that it can be better utilized for resident education. Residents also believe that voice recognition is less reliable and more time consuming than the traditional dictation system.

[2], [3], [6]