Abstract
The goal of this study was to assess whether radiologists’ search paths for lung nodule detection in chest computed tomography (CT) between different rendering and display schemes have reliable properties that can be exploited as an indicator of ergonomic efficiency for the purpose of comparing different display paradigms. Eight radiologists retrospectively viewed 30 lung cancer screening CT exams, containing a total of 91 nodules, in each of three display modes [i.e., slice-by-slice, orthogonal maximum intensity projection (MIP) and stereoscopic] for the purpose of detecting and classifying lung nodules. Radiologists’ search patterns in the axial direction were recorded and analyzed along with the location, size, and shape for each detected feature, and the likelihood that the feature is an actual nodule. Nodule detection performance was analyzed by employing free-response receiver operating characteristic methods. Search paths were clearly different between slice-by-slice displays and volumetric displays but, aside from training and novelty effects, not between MIP and stereographic displays. Novelty and training effects were associated with the stereographic display mode, as evidenced by differences between the beginning and end of the study. The stereo display provided higher detection and classification performance with less interpretation time compared to other display modes tested in the study; however, the differences were not statistically significant. Our preliminary results indicate a potential role for the use of radiologists’ search paths in evaluating the relative ergonomic efficiencies of different display paradigms, but systematic training and practice is necessary to eliminate training curve and novelty effects before search strategies can be meaningfully compared.
Key words: Volumetric dataset, navigation, stereoscopic display, lung nodule detection
INTRODUCTION
Medical imaging is rapidly adopting three-dimensional (3D) acquisition and representation, and in the near future, it is very likely that 3D datasets from various imaging modalities will dominate medical imaging for diagnosis, treatment, and image-guided surgery.1,2,3 Whereas there are many advantages to interpreting 3D volumes as opposed to 2D projections, the trend has greatly increased the workload of radiologists. The main advantage of 3D imaging is that it describes anatomical structure and tissue composition in 3D space without tissue superposition, which is a major limitation for projection radiography. One big difference between 3D image dataset and 2D projection image is that resolution on each single 2D image in a 3D dataset is much lower than that on a single 2D projection image. Low resolution on each 2D image and information spreading in 3D dataset require radiologists to rely more on the information between images and mentally reconstruct 3D structure while viewing a set of images. Ever-increasing image volume is another challenge of 3D image interpretation, which forces radiologists to be involved more and more actively in computer-based procedures and operations for image interpretation.
Volumetric datasets are generally comprised of multiple 2D slices. The traditional method for viewing such a dataset is to view individual slices as 2D images, or to create thicker slabs from the individual slices and project them onto a 2D display. Typically, when viewing volumetric data in either the slice-by-slice mode or in the thick slab mode, radiologists need to roam locally within the 3D volume and mentally integrate information between multiple slices for the fact that objects are frequently depicted across multiple slices. Mentally reconstructing procedure may incorrectly perceive information during integration and easily cause fatigue. The algorithms for thicker slab, such as mostly used maximum intensity projection algorithm or surface rendering, may not give correct geometrical projection or internal structure separation.
Because both of these viewing methods are tedious and error prone, considerable effort has been devoted to facilitating the visualization of medical 3D datasets.4,5 Other rendering methods exist that can be applied for particular clinical tasks, but in general they have not been shown to be acceptable for primary diagnosis. Surface rendering, for example, is a commonly used rendering method for displaying external structures and object shapes.6,7 Volumetric rendering methods are more diagnostically relevant for revealing internal anatomical structures.8,9 One of the most commonly used volumetric rendering methods is maximum intensity projection (MIP), which maximizes contrast on a rendered image by assigning pixels the brightest voxel along projections through the 3D dataset.10
Various display workstations and user interfaces have also been developed to achieve better image perception, ease reader–computer interaction, and improve efficacy of the interpretation process.11–16 The stereo display tested in this study is one of these attempts.11,12 In cases where comparisons have previously been made between competing display paradigms, the main focus has been detection performance.
Because of the variety of rendering and display methods and information across 3D space, the interpretation of volumetric datasets involves significantly more interaction between readers and displays than is customary for projection imaging. Thus, in any comparison between volumetric display systems, it is necessary to consider efficiency, as well as detection performance. However, little effort has been devoted to quantitatively measuring the impact of different display paradigms on the ergonomic efficiency of performing 3D reading tasks.
The amount of user interaction can be considered, to some extent, to reflect the complexity of a given task. Specifically, in the slice-by-slice and thick slab presentations, it is reasonable to expect that appropriate measures of the amount of roaming in the axial direction can be used as an indicator of the complexity of the reading task.
The efficiency can also be expressed, to a certain extent, as search patterns. Extensive studies have been conducted of eye search patterns on projected radiographic images for lesion detection.17–23 The results indicate that the eye search characteristics depend largely on experience, but are influenced by image quality and can be correlated to the performance of detection and diagnosis tasks.17,19,20,22–25 These studies have helped to improve image quality, image representation, and visual inspection technique. However, most of these works have been focused on 2D radiographic images, and very little research has been done on human–computer interaction and searching behavior associated with volumetric datasets.
To understand radiologists’ search pattern during interpretation of 3D image datasets, we have collected and analyzed navigation data from a pilot study designed for free-response receiver operating characteristics (FROC)-type analysis of lung nodule detection on computed tomography (CT) images and characterized the patterns that are related to nodule detection and classification. It is expected that these search patterns will ultimately be of value in optimizing the ergonomic aspects of 3D displays for radiology.
MATERIALS AND METHODS
Data acquired in a pilot study of lung nodule detection and classification in chest CT was used for studying navigation patterns. Radiologists were asked to perform a detection and classification task in each of three display modes (conventional slice-by-slice display, orthogonal MIP display and stereoscopic display), and their search patterns in the axial direction, along with the characteristics of detected features, were recorded and compared between modes.
Chest CT Dataset
This is a pilot study of our ongoing research to explore and develop display mechanisms for 3D dataset. In this study, we chose a small set of cases to test the feasibility of the research and address some of the issues relating to the designing and developing display workstations for 3D images. Low-dose lung CT images for lung cancer screening were acquired from a multislice CT scanner (LightSpeed, GE, Milwaukee) at a reconstructed thickness of 2.5-mm per slice with pixel resolutions ranging from 0.69 × 0.69 to 0.94 × 0.94 mm2. A total of 30 cases were randomly selected from our lung cancer screening project, with an average of about 100 axial images for each case.
The gold standard for the existence of nodules was consensus. The nodule-like features pooled from eight radiologists’ interpretations, in the three display modes, were reviewed and verified by an experienced chest radiologist who did not participate in the study but had read and discussed the cases with other radiologists multiple times.
Readers
We recruited six experienced staff radiologists who have more than 3 years practice in chest imaging and two fellows who have more than 2 years in chest imaging training to interpret the images. The primary task was to detect and then classify any nodules equal to or larger than 3 mm in diameter in each of three display modes as described below.
Display Modes
All readers interpreted each case in three display modes, which included slice-by-slice, orthogonal MIP, and stereoscopic display. Raw CT images were first processed with the convolution kernel provided by GE standard reconstruction software to form reconstructed images that are optimal for viewing lung tissues. The reconstructed images were then rendered based on the specification of each display mode. All renderings were precalculated and stored on hard disk for real-time display. The detailed rendering methods were included in Wang et al.11,12 A brief description of three display modes is given as follows.
Slice-by-slice mode is the most common display method adopted by radiologists for CT image interpretation. As images are read one at a time in sequence, no further rendering process was applied after the raw images were reconstructed. Note that the slice-by-slice mode was included as a special case (thickness = 1 slice) in the other two modes because there is a consensus that raw data (i.e., single slices) will always need to be available to radiologists despite any other rendering that is performed.
In the orthogonal MIP mode, multiple sequential slices were orthogonally projected by finding the maximum voxel value along each ray in the axial direction and using it as the final display value for the corresponding pixel in the MIP image. Users could adjust the thickness (number of sequential slices) and axial position of the projected volume. The allowed thicknesses were 1, 3, 5, 7, 9, 13, or 15 individual CT slices, where a slice thickness of 1 corresponded to a slice-by-slice display mode.
In the stereo perspective projection mode, projection volumes were defined under user control, as above, and projected using perspective transformations corresponding to left- and right-eye views. The offset between left- and right-eye projections was set based on a typical interocular distance of 6.5 cm, and it was assumed that viewers were at a distance of 45 cm from the viewing screen. This arrangement provided an acceptable level of stereopsis for all viewers participating in the study.
Voxel resampling was performed in the axial direction (z direction) to approximate isotropic voxel spacing before performing stereoscopic rendering. Then, two raycasting methods were employed for the stereo images.11,12 One was distance-weighted MIP, in which the maximum voxel value along a perspective ray was further modified by a weight based on the voxel location in the ray. High contrast images produced by distance-weighted MIP were used for nodule detection. The other raycasting method was averaging to produce images that preserve local geometry for nodule classification. As for the stereo compositing, the two algorithms, MIP and averaging, worked as a unit to provide complementary information of the images.
Display System
The workstation upon which the display was implemented consisted of a 2.0 GHz ADM Athlon 64 3200+ central processor, 512 MB RAM, and a 128 MB NVIDIA Quadro FX 1100 graphics card. During stereo display, the stereo effect was achieved by use of shutterglasses (StereoGraphics, Corp.) controlled by frame-swap signals corresponding to left-eye and right-eye images on the graphics card. A 21.0″ (20.0″ viewable) PerfectFlat CRT monitor (ViewSonic® Graphics Series G220f) was used as the actual display. The monitor’s refresh rate was set to 144 Hz to produce the frame-swapped stereo view while avoiding flicker.
A user interface was implemented using the Microsoft Visual C++ API combined with OpenGL for image display and user interactivity tools. Case interpretation basically involved navigation/search activity for lung nodules by moving along the axial direction throughout the lung volume and nodule assessment for any detected nodules. All the navigation/search-related activities were controlled through a programmable keypad, which was dedicated to the specific requirements of this study. The function keys on the programmable keypad can be used for selecting image axial viewing position and viewing volume (slab thickness), changing window/level settings, switching between MIP rendering and averaging rendering during stereo display, and toggling the visibility of cursors on detected features.
An onscreen scoring form was designed and implemented for reporting findings. When a radiologist clicks on a detected feature, a scoring form with questions related to nodule characterization pops up. The feature’s location is derived directly from the cursor position within the 3D volume. The cursor incorporates a reticulated scale to assist readers in estimating nodule dimensions.
Study Design
This study was designed for FROC type of analysis. The specific task was to detect and mark the location of any nodule-like feature, and then to report on characteristics such as size, likelihood that it is an actual nodule, and likelihood of malignancy.
To avoid bias, case order and display modes were randomized. We also imposed a minimum 14-day interval between the times a reader can see any particular case in different display modes, which was intended to reduce the possibility that cases could be remembered between reading modes.
Data Collection
We recorded navigation patterns from four participating radiologists who were randomly selected to anonymize attributes associated with each individual. Navigation patterns during interpretation were collected by recording viewing volume (slab thickness) and viewing position at 250-ms intervals. For a detected nodule, its x, y, and z position was recoded along with the other parameters, such as the likelihood of an actual nodule, the likelihood of malignancy, nodule shape, degree of calcification, and nodule size. The interpretation time for each case was also recorded.
Data Analysis
Nodule detection performance was evaluated by performing FROC analysis using JAFROC software (JAFROC, Chakraborty and Berbaum, http://www.devchakraborty.com/). Figures of merit from FROC were calculated on a per-nodule basis.
Interpretation times for cases were calculated for the display modes. The navigation and nodule detection patters were visually analyzed and compared between the modes, and between the beginning and the end of the study for predicting learning process. The variation of viewing thickness and optimal viewing volumes were analytically described for volumetric displays from slab thickness recorded during case interpretation.
We compared performance and navigation patterns for missed nodules in the apical lung area to nodules in areas close to the diaphragm, between the three display modes. Since a single nodule could be detected as many as eight times (eight participating radiologists) in each display mode, it is more appropriate to use the number of detections instead of the number of nodules for comparison.
The structural characteristics of false detections were classified for three display modes.
RESULTS
Nodule Detection Performance
A total of 174 nodule-like features, at least 3-mm in diameter, have been found in the 30 cases, and 91 of these are considered to be true nodules. FROC analysis suggests that the stereo display mode resulted in higher detection performance than the orthogonal MIP display but was equivalent to the slice-based display, although no statistically significant difference were shown between the three display modes. The figures of merit from the JAFROC software were 0.57 (stereo display), 0.56 (slice-by-slice display), and 0.52 (orthogonal MIP display) for all participated radiologists. There were inter-reader variations on the performance in each display mode. The detection rate was ranged from 72 to 41%.
By averaging interpretation time over radiologists on each display mode, we have shown that the average time was significantly less for the stereo display (3.5 ± 0.7 min) than for the slice-by-slice display (4.5 ± 0.4 min), but there was no significant difference between the stereo display and the orthogonal MIP display (3.7 ± 0.5 min).
Navigation Pattern
The average viewing volume for the 3D displays was between three and five CT slices during navigation, and between 1 and 3 when inspecting a suspicious region. There was no apparent difference in the preference of viewing volume between the stereo display and the orthogonal MIP display. Typically, when a suspicious region or feature was detected, readers rapidly navigated back-and-forth across several slices on either side of the suspected nodule. This distinctive navigation pattern was more often seen in the slice-by-slice display mode and for nodules described as being non-solid or semi-solid. Radiologists typically navigated through the dataset axially between the top and the base of the lung several times. The average number of such repetitions for the stereo display, the orthogonal MIP display, and the slice-by-slice display were 3.5 ± 4.3, 4.5 ± 2.2, and 3.5 ± 1.7, respectively.
Navigation patterns at the beginning and end of the study were compared to detect the possibility of a learning curve or a novelty effect associated with each display mode. Despite variation search pattern between individuals, the results of this, which are shown in Figures 1, 2 and 3, reveal that such effects are apparent for the stereo display and, to some extent, for the orthogonal MIP display but not for the slice-by-slice display mode.
Fig 1.
Navigation patterns from four radiologists in stereo mode. Each graph is the navigation recorded from one case interpretation. Two graphs in each row are taken from one radiologist’s interpretations in the beginning of the study (left) and the end of the study (right). The dark solid line represents viewing positions referenced to the scale on the left axis, and the gray solid line represents viewing thickness referenced to the scale on the right axis, with time.
Fig 2.
Navigation patterns from four radiologists in orthogonal MIP mode. Each graph is the navigation recorded from one case interpretation. Two graphs in each row are taken from one radiologist’s interpretations in the beginning of the study (left) and the end of the study (right). The dark solid line represents viewing positions referenced to the scale on the left axis, and the gray solid line represents viewing thickness referenced to the scale on the right axis, with time.
Fig 3.
Navigation patterns from four radiologists in slice-by-slice mode. Each graph is the navigation recorded from one case interpretation. Two graphs in each row are taken from one radiologist’s interpretations in the beginning of the study (left) and the end of the study (right). The dark solid line represents viewing positions with the time.
The stereo (Fig. 1) and the orthogonal MIP (Fig. 2) display modes were more complicated and dynamic for cases reviewed at the beginning of the study as opposed to those reviewed at its end, where navigation patterns became much smoother and more stabilized in both the stereo display and the orthogonal MIP display.
The navigation patterns from the slice-by-slice display were more like a random search pattern than a learning process and showed little difference between the beginning and the end of the study (Fig. 3). Since case order was randomized at each interpretation session for each radiologist, the navigation patterns between radiologists shown in Figures 1, 2, and 3 were not taken from the same cases.
Characteristics of Missed Nodules
The total number of detections in the apical area and diaphragm area is 128 and 112, respectively. In the apical area, there was a higher rate of missed nodules for both the stereo display (55%) and the orthogonal MIP display (55%) than for the slice-by-slice display (42%). However, the difference was not so obvious in the lung area close to the diaphragm, where the missed detection rates were 36% for the stereo display, 38% for the orthogonal MIP display, and 35% for the slice-by-slice display.
A detailed analysis of search patterns revealed that some of the missed nodules actually received extra attention from radiologists, although they ultimately went unreported. Typical search patterns in the vicinity of missed nodules are shown in Figure 4a and b. In the slice-by-slice mode, 25% of missed detections received extra attention as opposed to 15% in the orthogonal MIP mode and 16% in the stereo mode.
Fig 4.
The diagrams a and b illustrate the missed nodules that received extra attention. 
Structural Characteristics of False Detections
Inspection of the CT data at locations of false positive reports indicated that most false positives can be attributed to the presence of scar tissues and vessels (Table 1), of which scar tissues occurred most frequently. Other structures that were falsely reported to be nodules include bronchiectasis, atelectasis, and soft tissues. In the vessel group, more false detections were found in the orthogonal MIP display mode than in the slice-based or the stereo display mode as shown in Table 1.
Table 1.
Distribution of false positive findings in different structural groups
| Stereo | Orthogonal MIP | Slice by slice | Total | |
|---|---|---|---|---|
| Vessel | 11 | 27 | 18 | 56 (32%) |
| Scar | 23 | 32 | 33 | 88 (51%) |
| Other | 10 | 7 | 12 | 29 (17%) |
| Total | 44 (26%) | 66 (38%) | 63 (36%) | 173 (100) |
DISCUSSION
Radiology is rapidly evolving from being primarily involved with 2D projection images to acquiring and interpreting 3D datasets. Historically, the interpretation of projection images, which were traditionally displayed on film, involved little interaction of radiologists beyond hanging films on a view box. With the introduction of digital imaging methods and soft displays, radiologists’ interaction expanded to include such activities as window/level control—but these interactions were similar between display systems and carried little weight in making comparisons between the systems.
For systems designed to display 3D datasets, there is considerably more variety with respect to how the data can be presented on a 2D soft display for interpretation. The most common method is to present sequential 2D slices of a 3D volume, but as the number of individual slices that must be viewed increases, this becomes impractical. Other rendering methods have been implemented, such as combining thin slices to form thicker slabs and projecting these onto the 2D display with MIP or some other raycasting method, but the optimal display paradigm will likely vary between tasks, and it is not known what the optimal display is for any particular task. In any event, displays for volumetric datasets involve significantly more user interaction than displays for 2D datasets, and any overall comparison of volumetric displays must consider ergonomic efficiency as well as detection performance.
For displays that present sequential 2D slices or projections of sequential slabs, the most common volumetric display methods, the amount of user initiated motion in the direction perpendicular to the slices or slabs may serve as a measure of the mental effort required to accommodate the third dimension of the dataset. If so, then it will likely be useful in comparing the relative ergonomic efficiencies between different volumetric display methods.
When lung nodules are neighbored with similar intensity of non-lung tissues in a thick viewing volume, they are likely to be missed because of camouflage effect. We have examined two places where lung tissue could be obscured by surrounding structures. One was the apical lung area, where lung tissues are closely surrounded by rib cage. The other one was the area close to the diaphragm. The results indicate that there were more missed detections with either the stereo or the orthogonal MIP display than with the slice-by-slice display in the apical area, whereas no such difference showed in the diaphragm area between the three displays. As obscuration can lower the conspicuity of the nodules, other factors, such as structure density and shape relationship, may also have effect on the detection as suggested by the different results from the two areas.
In this study, we have not implemented multiple reformations for different viewing angles (or planes) because of the complexity of preparing prestaged multiple reformations. Results from other researches and our current project of real-time rendering on programmable graphics units indicate that volumetric displays that allow multiple reformatted viewing angles by rotating images can help reduce ambiguity caused by some poorly differentiated spacial relationships, including tissue superimposition.26–28 The advantage of multiple reformations can be more appreciated by volumetric displays than single slice-based display. Multiple views for single slice are geometrically discontinuous because they lack the information of the third dimension and require intensive mental work on geometrical correlations between two viewing angles. When viewing volumetric data, volume can be smoothly transformed between two viewing angles by rotating objects in 3D space to produce natural continuation of views of the objects.
While this was a small tentative study, we found that the volumetric display modes (i.e., stereo and MIP) resulted in simpler search paths relative to slice-by-slice displays, particularly in the vicinity of suspicious features. These differences were clearer than the differences in detection performance, which were not significant between display modes in this small study. Although a larger study would likely have detected significant differences in performance, detection performance alone is not likely to be a particularly sensitive measure of the effort required to interpret volumetric datasets. This study suggests that search strategy is largely independent of detection performance and some measure of search strategy will likely be an important component of any overall assessment of the relative merits of different volumetric display systems.
The results presented herein were confounded by the novelty of the stereographic display mode and, to some extent, by the MIP mode. In both of these modes, a training curve was evident. If a reliable measure of ergonomic efficiency is to be derived from search strategies, it will be necessary to train viewers sufficiently that any novelty or training effects can be overcome.
CONCLUSIONS
Although more 3D imaging modalities are being employed for medical screening and diagnosis, slice-by-slice display is still predominantly being used as a primary viewing method for interpretation. Adopting volumetric displays, therefore, involves learning process that extends and transforms current 2D understanding of medical images to the knowledge of volumetric information discovery. Effective utilization of 3D display for medical volumetric data relies both on software design and user training. Our preliminary data from a pilot study for lung nodule detection on CT images indicate that current 3D displays can be further improved by understanding radiologists’ interpretation behavior and diagnostic performance.
Acknowledgment
This work is sponsored in part by the Department of Defense, US Army Medical Research Acquisition Center, 820 Chandler Street, Fort Detrick, MD 21702-5014 under Contract PR043488, and also by grant CA80836 from the National Cancer Institute, National Institutes of Health. The content of the contained information does not necessarily reflect the position or the policy of the government, and no official endorsement should be inferred.
References
- 1.Barish MA, Rocha TC. Multislice CT colonography: current status and limitations. Radiol Clin North Am. 2005;43:1049–1062. doi: 10.1016/j.rcl.2005.07.008. [DOI] [PubMed] [Google Scholar]
- 2.Rowe VL, Tucker SW., Jr Advances in vascular imaging. Surg Clin North Am. 2004;84:1189–1202. doi: 10.1016/j.suc.2004.05.002. [DOI] [PubMed] [Google Scholar]
- 3.Israel GM, Bosniak MA. Renal imaging for diagnosis and staging of renal cell carcinoma. Urol Clin North Am. 2003;30:499–514. doi: 10.1016/S0094-0143(03)00019-3. [DOI] [PubMed] [Google Scholar]
- 4.Aufort S, Charra L, Lesnik A, et al. Multidetector CT of bowel obstruction: value of post-processing. Eur Radiol. 2005;15:2323–2329. doi: 10.1007/s00330-005-2733-x. [DOI] [PubMed] [Google Scholar]
- 5.Remy J, Remy-Jardin M, Artaud D, et al. Multiplanar and three-dimensional reconstruction techniques in CT: impact on chest diseases. Eur Radiol. 1998;8:335–351. doi: 10.1007/s003300050391. [DOI] [PubMed] [Google Scholar]
- 6.Bomans M, Hohne KH, Tiede U, et al. 3-D segmentation of MR images of the head for 3-D display. IEEE Trans Med Imag. 1990;2:177–183. doi: 10.1109/42.56342. [DOI] [PubMed] [Google Scholar]
- 7.Hoehne TU, Bomans KH, Pommert M, et al. Investigation of medical 3D-rendering algorithms. IEEE Comput Graph Appl. 1990;2:41–53. [Google Scholar]
- 8.Calhoun PS, Kuszyk BS, Heath DG, et al. Three-dimensional volume rendering of spiral CT data: theory and method. Radiographics. 1999;19:745–764. doi: 10.1148/radiographics.19.3.g99ma14745. [DOI] [PubMed] [Google Scholar]
- 9.Kuszyk BS, Heath DG, Ney DR, et al. CT Angiography with volume rendering: imaging findings. AJR Am J Roentgenol. 1995;165:445–448. doi: 10.2214/ajr.165.2.7618574. [DOI] [PubMed] [Google Scholar]
- 10.Fishman EK, Ney DR, Heath DG, et al. Volume rendering versus maximum intensity projection in CT angiography: what works best, when, and why. RadioGraphics. 2006;26:905–922. doi: 10.1148/rg.263055186. [DOI] [PubMed] [Google Scholar]
- 11.Wang XH, Maitz GS, Leader JK, et al. Real-time stereographic display of volumetric datasets in radiology. SPIE Electronic Imaging. 2006;6055:1A-1–1A-6. [Google Scholar]
- 12.Wang XH, Walter FG, Fuhrman CR, et al. Stereo CT image compositing methods for lung nodule detection and characterization. Acad Radiol. 2005;12:1512–1520. doi: 10.1016/j.acra.2005.06.009. [DOI] [PubMed] [Google Scholar]
- 13.Demiralp C, Jackson CD, Karelitz DB, Zhang S, Laidlaw DH. CAVE and fishtank virtual-reality displays: a qualitative and quantitative comparison. IEEE Trans Vis Comput Graph. 2006;12(3):323–330. doi: 10.1109/TVCG.2006.42. [DOI] [PubMed] [Google Scholar]
- 14.Levin D, Aladl U, Germano G, et al. Techniques for efficient, real-time, 3D visualization of multi-modality cardiac data using consumer graphics hardware. Comput Med Imaging Graph. 2005;29:463–475. doi: 10.1016/j.compmedimag.2005.02.007. [DOI] [PubMed] [Google Scholar]
- 15.Yee DK, Lee W, Kim D, et al. RadGSP: a medical image display and user interface for UWGSP3. Proc SPIE. 1991;1444:292–305. doi: 10.1117/12.45181. [DOI] [Google Scholar]
- 16.Fuchs H, Levoy M, Pizer SM. Interactive visualization of 3D medical data. Computer. 1989;22:46–51. doi: 10.1109/2.35199. [DOI] [Google Scholar]
- 17.Krupinski EA. Visual search of mammographic images: influence of lesion subtlety. Acad Radiol. 2005;12:965–969. doi: 10.1016/j.acra.2005.03.071. [DOI] [PubMed] [Google Scholar]
- 18.Krupinski EA, Berger WG, Dallas WJ, et al. Searching for nodules: what features attract attention and influence detection. Acad Radiol. 2003;10:861–868. doi: 10.1016/S1076-6332(03)00055-2. [DOI] [PubMed] [Google Scholar]
- 19.Nodine CF, Mello-Thoms C, Kundel HL, et al. Time course of perception and decision making during mammographic interpretation. AJR Am J Roentgenol. 2002;179:917–923. doi: 10.2214/ajr.179.4.1790917. [DOI] [PubMed] [Google Scholar]
- 20.Kundel HL, Nodine CF, Toto L. Searching for lung nodules. The guidance of visual scanning. Invest Radiol. 1991;26:777–781. doi: 10.1097/00004424-199109000-00001. [DOI] [PubMed] [Google Scholar]
- 21.Kundel HL. Visual cues in the interpretation of medical images. J Clin Neurophysiol. 1990;7:472–483. doi: 10.1097/00004691-199010000-00003. [DOI] [PubMed] [Google Scholar]
- 22.Kundel HL, Nodine CF, Krupinski EA. Searching for lung nodules. Visual dwell indicates locations of false-positive and false-negative decisions. Invest Radiol. 1989;24:472–478. doi: 10.1097/00004424-198906000-00012. [DOI] [PubMed] [Google Scholar]
- 23.Kundel HL, Nodine CF, Thickman D, et al. Searching for lung nodules. A comparison of human performance with random and systematic scanning models. Invest Radiol. 1987;22:417–422. doi: 10.1097/00004424-198705000-00010. [DOI] [PubMed] [Google Scholar]
- 24.Krupinski EA, Roehrig H. The influence of a perceptually linearized display on observer performance and visual search. Acad Radiol. 2000;7:8–13. doi: 10.1016/S1076-6332(00)80437-7. [DOI] [PubMed] [Google Scholar]
- 25.Nodine CF, Kundel HL, Lauver SC, et al. Nature of expertise in searching mammograms for breast masses. Acad Radiol. 1996;3:1000–1006. doi: 10.1016/S1076-6332(96)80032-8. [DOI] [PubMed] [Google Scholar]
- 26.Salvolini L, Bichi Secchi E, Costarelli L, et al. Clinical applications of 2D and 3D CT imaging of the airways-a review. Eur J Radiol. 2000;34:9–25. doi: 10.1016/S0720-048X(00)00155-8. [DOI] [PubMed] [Google Scholar]
- 27.Ou P, Celermajer DS, Calcagni G, et al. Three-dimensional CT scanning: a novel diagnostic modality in congenital heart disease. Heart. 2006;93:908–913. doi: 10.1136/hrt.2006.101352. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Durkee NJ, Jacobson J, Jamadar D, et al. Classification of common acetabular fractures: radiographic and CT appearances. AJR Am J Roentgenol. 2006;187:915–925. doi: 10.2214/AJR.05.1269. [DOI] [PubMed] [Google Scholar]