Abstract.
The goal of this research was to examine whether search pattern training for central line positioning on chest radiographs (CXRs) improves the ability of healthcare trainees and practitioners to identify malpositioned central venous catheters. Two sets of CXRs with central catheters were shown; half of the images contained catheters that were appropriately positioned, half that were malpositioned. Subjects were asked to: mark the tip of the catheter using the simulated radiology workstations, indicate their confidence in tip localization, and state whether the catheter was appropriately positioned or malpositioned. Subjects were also given a survey assessing their thoughts about the usefulness of search pattern training and the simulated radiology workstation. There was a significant improvement in subjects’ ability to classify a catheter as malpositioned after training, -value = 0.03. There was no significant difference in localization of the catheter tips or in the confidence for tip localization. Subjects’ responses to the questionnaire were significantly positive for all statements, indicating that they felt search pattern training using a simulated radiology workstation had a positive impact on their education. These results suggest that our knowledge of medical image perception may be useful for developing rational educational tools for image interpretation, and that simulated radiology workstations may be a helpful means of deploying these tools.
Keywords: perception, education, training, radiology, imaging
1. Introduction
Scientists have studied medical image perception for many years and have learned much about the ways humans perceive medical images.1,2 However, the translation of this knowledge into educational tools to facilitate training in image interpretation has lagged behind the science significantly.
Recent work has examined teaching search patterns as a complement to traditional educational methods in medical image interpretation. Specifically, medical trainees shown a search pattern for evaluation of the lungs showed improved ability to identify pulmonary nodules on chest radiographs (CXRs).3,4 This lung search pattern instructs the observer to sweep their eyes over each intercostal space (Fig. 1), thereby ensuring that the entire lungs were evaluated by the observer’s central field-of-view. For the study involving group search pattern training, the experimental group performance improved with a -value = 0.0012. The effectiveness of search pattern training for identification of pulmonary nodules suggests that search pattern training may be useful for additional perceptual tasks.
A common indication for chest radiography is the characterization of central venous catheter (CVC) positioning. CVCs are used to deliver fluids and medications to ill patients. CXR is one of the more common imaging methods used to evaluate for correct CVC positioning. Evaluation of CXRs is a critical skill, as complications during CVC placement may result in patent morbidity and/or mortality.5 Potential complications of CVC placement depend on the type of catheter and route of insertion, and include malpositioned catheter, where a catheter is not in a central vein (incidence 6); pneumothorax (incidence 7); and hemothorax (incidence 8).
In spite of the importance of skilled evaluation of post-CVC placement CXRs, errors are still made when evaluating such images. This is especially problematic for radiographs acquired overnight. For overnight studies, the medical provider caring for the patient will often evaluate a post-CVC placement CXR to confirm adequate positioning and act on their interpretation, before the final radiology report is issued. If their judgment is flawed, the patient may be harmed.
The goal of this study was to determine if teaching a CVC search pattern to nonradiologist healthcare trainees and providers at a simulated radiology workstation improves their ability to identify malpositioned CVCs on chest radiography.
2. Methods
2.1. Subjects
After review by our institutional review board (IRB), this study received an “Exemption of Human Subjects Research.” A group of 13 nurse practitioner students and 5 radiology technologists elected to voluntarily participate.
2.2. CXR Images
Sixty frontal CXRs with appropriately placed or malpositioned CVCs were selected by a board-certified chest radiologist with 7 years of experience (W.F.A.), 30 with adequately positioned catheters and 30 with malpositioned catheters. For this study, catheters with tips in the mid to low superior vena cava were considered adequately positioned (discussed further in the search pattern training section), and catheters with tips not in the superior vena cava were considered malpositioned. Catheters with positioning that would be considered on the borderline of acceptably positioned and malpositioned were not included in this study, as even experts may disagree on such cases. Types of malpositioned catheters included: catheters crossing midline, catheters coursing superiorly into the neck, and catheters coursing into a tributary vein (such as the azygous vein). Each set of cases presented to subjects included 15 images with acceptably positioned catheters and 15 images with malpositioned catheters. All images were anonymized prior to inclusion into this study.
2.3. Study Hardware and Software
A simulated radiology workstation was used to present images and collect subject responses. The computers had a monitor resolution of 2 mega-pixels. While this resolution is lower than many diagnostic radiology monitors, it is similar to the resolution of the monitors typically used clinically by nonradiologists when viewing medical images. The computer workstations were otherwise similar to typical image-viewing workstations.
The software used for image presentation and recording of subject responses was ViewDEX (Sahlgrenska University Hospital, Goteborg, Sweden).9 This software allowed for presentation of images with viewing controls similar to a clinical radiology workstation, including window/level and pan/zoom. The software also allows the user to mark the catheter tip and indicate their level of confidence regarding whether the catheter is acceptably positioned or malpositioned.
2.4. Experimental Design
This study used a counterbalanced design, as shown in Fig. 2, where the experimental and control groups each received both search pattern training and an attentional control activity at different time points. This allowed for assessment of an intervention effect from the search pattern training, while allowing both experimental and control groups to receive the educational benefit of search pattern training.
After a brief introduction to the study and normal venous anatomy using electronic slides, both groups were shown a set of 20 baseline cases with CVCs, half appropriately positioned, and half malpositioned. Subjects were asked to mark the tip of the CVC using the study software, indicate their confidence in catheter tip localization, and state whether the CVC was appropriately positioned or malpositioned.
Immediately after the first set of cases, the experimental group was shown a suggested search pattern (see below) for evaluation of central lines on CXR. The control group was shown an attentional control, a journal article on CXR imaging of intensive care unit patients.10 The attentional control paper on ICU imaging did not include any information on central line positioning beyond what was given in the study’s introductory materials. The subjects were then shown a second set of 20 cases with the same task. After the second set of cases, the conditions were reversed such that the control group received search pattern training, and the experimental group received the attentional control. The subjects were then given a third set of cases to review.
2.5. Search Pattern Training Algorithm
The following suggested search pattern for characterization of CVCs was developed through consultation with three experienced faculty chest radiologists: 1—Identify the presence of a CVC; 2—identify the peripheral most part of the catheter in the field-of-view; 3—follow the catheter from its peripheral most aspect to its tip and confirm the catheter course is acceptable, and 4—confirm the catheter tip is in an acceptable position.
The subjects were shown graphical examples of this search pattern (a representative example is shown in Fig. 3). A portable document format file containing 20 electronic slides describing this search pattern and giving visual examples was provided to trainees at the time of search pattern training. Search pattern training lasted about 10 min and the time for the control group to read their materials was equivalent.
2.6. Questionnaire
A questionnaire consisting of eight items was given to subjects after completion of the study. The items are given in Table 1, with a histogram of the responses. Subjects were asked to respond to questions using a five-point Likert response format (1 = strongly disagree to 5 = strongly agree), where a response of 3 was neutral. Survey results are displayed graphically using histograms, and statistical significance was determined using the Wilcoxon signed-rank test. Note that as described previously, all subjects received search pattern training and were all able to comment on their perceptions of the training using the questionnaire.
Table 1.
Q# | Questionnaire statements |
---|---|
1 | In my prior course-work and self study, I have not been shown specific eye movement patterns for evaluation of line/tube positioning on CXRs. |
2 | The search pattern training was helpful for learning the skills needed to evaluate line/tube positioning on CXR. |
3 | The search pattern training helped me feel more confident about my ability to identify line/tube positioning on CXR. |
4 | Search pattern training for other medically relevant abnormalities would be a helpful way to learn about additional topics in radiology. |
5 | The simulated radiology workstation (SRW) used for this study was helpful for learning the skills needed to evaluate line/tube positioning on CXR. |
6 | Compared with conventional learning materials (including printed/electronic textbooks and case files), the SRW provided a more effective way to develop the skills needed to evaluate line/tube positioning on CXR. |
7 | SRW would be a helpful way to learn about additional topics in radiology. |
8 | Participation in this study was an overall positive experience. |
2.7. Statistical Analysis
Improvements in subject performance between case sets 1 and 2 were evaluated for both control and experimental groups. Metrics evaluated include: the difference in fraction of catheter tips correctly localized (D-CorrLoc), the difference in confidence in catheter tip localization (D-ConfLoc), and the difference in fraction of times a catheter were correctly characterized as appropriately (safely) positioned versus malpositioned (D-SafePos). In order for a catheter tip to be considered correctly localized, the subject marking was required to be within a 1 cm radius of the catheter tip. For each measure, we computed the mean, 95% confidence interval and -value using the Wilcoxon signed-rank test.11 Note that as all images contained a central catheter and were positive for a target, receiver operating characteristic analysis could not be used. The type I error level used to determine statistical significance was 0.05, and -values less than 0.05 were considered significant.
3. Results
3.1. Catheter Localization and Positioning
The results for fraction of catheters tips correctly localized (D-CorrLoc), change in confidence in catheter tip localization (D-LocConf), and change in ability to identify malpositioned catheters (D-SafePos) are given in Table 2.
Table 2.
Measure | Mean | 2.5th %ile | 97.5th %ile | -value |
---|---|---|---|---|
D-CorrLoc | ||||
0 – Ctrl | 0.0278 | 0.0833 | 0.5000 | |
1 – Exp | 0.0667 | 0.0056 | 0.1333 | 0.1875 |
D-ConfLoc | ||||
0 – Ctrl | 0.1333 | 0.9453 | ||
1 – Exp | 0.2167 | 0.4667 | 0.2422 | |
D-SafePos | ||||
0 – Ctrl | 0.0222 | 0.0778 | 0.4453 | |
1 – Exp | 0.1000 | 0.0444 | 0.1556 | 0.0313 |
For the control group, there was no significant improvement in performance. For the experimental group, there was a statistically significant improvement in fraction of times a catheter was correctly characterized as appropriately (safely) positioned versus malpositioned (D-SafePos). There was also an improvement in the subjects’ ability to localize catheter tips (D-CorrLoc) and in their stated confidence (D-ConfLoc), but these were not statistically significant.
3.2. Questionnaire
Histograms of the survey results are shown in Fig. 4 and demonstrate a strong positive response for all questions. -values for all survey responses were less than or equal to 0.0001.
4. Discussion
The fundamental paradigm for training in medical image interpretation has changed little over the past several decades. Medical image perception and interpretation is typically taught on-the-job with a focus on case volume. The advances in medical image perception offer a new and exciting opportunity to develop rational new educational tools based on our knowledge about how humans perceive medical images. However, application of this knowledge to date has been limited.
Only recently has our knowledge of medical image perception begun to be translated into educational tools. Auffermann et al.3,4 successfully used search pattern training to improve trainees’ ability to identify pulmonary nodules on chest radiography. This showed the feasibility of using focused search pattern training to improve the ability of novices to identify medically relevant abnormalities. There are several additional high yield tasks that may benefit from perceptual training in general and search pattern training in particular. While some researchers have found that systematic search patterns did not perform better relative to nonsystematic patterns,12 it may be because the benefit of a search pattern may be related to the perceptual task performed.
Confirmation of central line positioning is a critical perceptual task on CXR that is often carried out by nonradiologists during nights and evenings. The radiologists involved in conducting this study have anecdotally noted that malpositioned catheters are sometimes not perceived by medical providers. Failure to identify a malpositioned catheter can have serious adverse medical consequences. For these reasons, central line positioning was considered a good task for search pattern training.
Of importance, there was a statistically significant improvement in fraction of times a catheter was correctly characterized as appropriately (safely) positioned versus malpositioned (D-SafePos). The lack of a significant treatment effect for the other two measures considered may be due to several factors. There were only 18 subjects in total, and a larger study may have been better powered to detect significant differences. The study also involved perceptual cuing that may not have been present if trainees were evaluating CXRs outside the study environment. That is, they were specifically told to evaluate the catheter and identify the catheter tip, which would likely not occur in a typical practice environment. In addition, the failure to achieve significant differences may have been due to the prior classroom and work experience with CXR imaging. Both the radiology technologists and the nurse practitioners had experience viewing and interpreting CXRs prior to this study, and may have had enough experience that the search pattern training did not significantly complement their existing knowledge. For example, all the radiology technologists had several years of experience viewing CXRs after acquisition to determine if further imaging is needed. While their training does not typically include central line positioning, they were very familiar with CXRs. This study consisted of a heterogeneous group of trainees and practitioners, which may mask more subtle behaviors and patterns in image perception.
It is also possible that the training did not significantly affect subject performance and behavior (i.e., even though it provided an exemplar pattern they may not have adopted it). However, the statistically significant improvement in subject performance at classifying catheters as adequately versus malpositioned suggests that there is at least some potential impact from the search pattern training.
Observer performance on case set 3 and how it relates to performance on case sets 1 and 2 may provide additional interesting information, but is felt to be beyond the scope of this paper. This would be an interesting topic for future study.
The results for the questionnaire show that: (1) subjects did not think they had been previously shown eye movement search patterns for evaluation of central lines on CXRs and (2) they felt the search pattern training used for this study was helpful. In clinical medicine, arriving at the correct answer is important. However, confidence in one’s decisions, even if the decision does not change, is also very important.13 Consequently, the trainees’ subjective impression of a positive effect of training has intrinsic value.
On questionnaire items 5 to 7, subjects indicated that they felt the simulated radiology workstation used for this study facilitated learning and would be a useful tool for learning about other radiology topics. It is important to note that while simulation has been used previously in radiology, it has been used mostly to teach procedural skills.14,15 The responses from questionnaire items 5 to 7 suggest that simulation may be a useful method of deploying perceptual training in particular and may be of greater use in radiology education in general.
5. Conclusions
After search pattern training, subjects’ ability to identify malpositioned CVCs improved. In addition, questionnaire responses show subjects felt the search pattern training was a useful adjunct to their medical training and made them more confident in their ability to identify malpositioned central catheters. Our knowledge of medical image perception may be used to develop rational educational tools to facilitate the more rapid acquisition of expertise related to the evaluation of medical images. Simulated radiology workstations may be helpful for deploying these perceptual training methods and may be useful in radiology education in general.
Acknowledgments
We would like to thank the radiology technologists, nurse practitioner students, and medical students, who volunteered their time for this study. Preliminary results from this research were presented at the Medical Image Perception Society (MIPS) 2017 meeting in Houston, Texas, USA.
Biographies
William F. Auffermann is a cardiothoracic radiologist and an associate professor of the Departments of Radiology and Imaging Sciences at the University of Utah School of Medicine. He received his MD/PhD degrees from the University of Minnesota, Twin Cities Campus (2004). His research interests include medical image perception, medical education, and the application of perception research to new educational methods in radiology.
Elizabeth A. Krupinski is a professor at Emory University in the Departments of Radiology & Imaging Sciences, where she is the vice chair for research. She received her PhD from Temple University in experimental psychology. She has conducted extensive research on image perception as it relates to several fields of medicine, including radiology, pathology, and dermatology.
Srini Tridandapani is a cardiothoracic radiologist and an associate professor of the Departments of Radiology and Imaging Sciences at Emory University School of Medicine. He received his MD from the University of Michigan, Ann Arbor (2001), and his PhD from the University of Washington, Seattle (1994). His research interests focus on developing engineering tools to improve the practice of radiology.
Disclosures
No conflicts of interest, financial or otherwise, are declared by the authors.
References
- 1.Krupinski E. A., “Current perspectives in medical image perception,” Atten. Percept. Psychophys. 72(5), 1205–1217 (2010).https://doi.org/10.3758/APP.72.5.1205 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Kundel H. L., “History of research in medical image perception,” J. Am. Coll. Radiol. 3(6), 402–408 (2006).https://doi.org/10.1016/j.jacr.2006.02.023 [DOI] [PubMed] [Google Scholar]
- 3.Auffermann W. F., et al. , “Simulation for teaching and assessment of nodule perception on chest radiography in nonradiology health care trainees,” J. Am. Coll. Radiol. 12(11), 1215–1222 (2015).https://doi.org/10.1016/j.jacr.2015.07.014 [DOI] [PubMed] [Google Scholar]
- 4.Auffermann W. F., Little B. P., Tridandapani S., “Teaching search patterns to medical trainees in an educational laboratory to improve perception of pulmonary nodules,” J. Med. Imaging 3(1), 011006 (2016).https://doi.org/10.1117/1.JMI.3.1.011006 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Paoletti F., et al. , “Central venous catheters. Observations on the implantation technique and its complications,” Minerva Anestesiol. 71(9), 555–560 (2005). [PubMed] [Google Scholar]
- 6.Pikwer A., et al. , “The incidence and risk of central venous catheter malpositioning: a prospective cohort study in 1619 patients,” Anaesth. Intensive Care 36(1), 30–37 (2008). [DOI] [PubMed] [Google Scholar]
- 7.Capaccioli L., et al. , “Insertion and management of long-term central venous devices: role of radiologic imaging techniques,” Radiol. Med. 96(4), 369–374 (1998). [PubMed] [Google Scholar]
- 8.McGee D. C., Gould M. K., “Preventing complications of central venous catheterization,” N. Engl. J. Med. 348(12), 1123–1133 (2003).https://doi.org/10.1056/NEJMra011883 [DOI] [PubMed] [Google Scholar]
- 9.Borjesson S., et al. , “A software tool for increased efficiency in observer performance studies in radiology,” Radiat. Prot. Dosimetry 114(1–3), 45–52 (2005).https://doi.org/10.1093/rpd/nch550 [DOI] [PubMed] [Google Scholar]
- 10.Bentz M. R., Primack S. L., “Intensive care unit imaging,” Clin. Chest Med. 36(2), 219–234, viii (2015).https://doi.org/10.1016/j.ccm.2015.02.006 [DOI] [PubMed] [Google Scholar]
- 11.Divine G., et al. , “Statistical grand rounds: a review of analysis and sample size calculation considerations for Wilcoxon tests,” Anesth. Analg. 117(3), 699–710 (2013).https://doi.org/10.1213/ANE.0b013e31827f53d7 [DOI] [PubMed] [Google Scholar]
- 12.van Geel K., et al. , “Teaching systematic viewing to final-year medical students improves systematicity but not coverage or detection of radiologic abnormalities,” J. Am. Coll. Radiol. 14(2), 235–241 (2017).https://doi.org/10.1016/j.jacr.2016.10.001 [DOI] [PubMed] [Google Scholar]
- 13.Elfenbein D. M., “Confidence crisis among general surgery residents: a systematic review and qualitative discourse analysis,” JAMA Surg. 151(12), 1166–1175 (2016).https://doi.org/10.1001/jamasurg.2016.2792 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Sarwani N., Tappouni R., Flemming D., “Use of a simulation laboratory to train radiology residents in the management of acute radiologic emergencies,” Am. J. Roentgenol. 199(2), 244–251 (2012)https://doi.org/10.2214/AJR.11.7892 [DOI] [PubMed] [Google Scholar]
- 15.Desser T. S., “Simulation-based training: the next revolution in radiology education?,” J. Am. Coll. Radiol. 4(11), 816–824 (2007).https://doi.org/10.1016/j.jacr.2007.07.013 [DOI] [PubMed] [Google Scholar]