Evaluation of Virtual Reality for Detection of Lung Nodules on Computed Tomography

Abstract

Virtual reality (VR) systems can offer benefits of improved ergonomics, but their resolution may currently be limited for the detection of small features. For detection of lung nodules, we compared the performance of VR versus standard picture archiving and communication system (PACS) monitor. Four radiologists and 1 novice radiologist reviewed axial computed tomography (CTs) of the thorax using standard PACS monitors (SM) and a VR system (HTC Vive, HTC). In this study, 3 radiologists evaluated axial lung-window CT images of a Lungman phantom. One radiologist and the novice radiologist reviewed axial lung-window patient CT thoracic images (32 patients). This HIPAA-compliant study was approved by the institutional review board. Detection of 227 lung nodules on patient CTs did not result in different sensitivity with SM compared with VR. Detection of 23 simulated Lungman phantom lung nodules on CT with SM resulted in statistically greater sensitivity (78.3%) than with VR (52.2%, P = .041) for 1 of 3 radiologists. The trend was similar but not significant for the other radiologists. There was no significant difference in the time spent by readers reviewing CT images with VR versus SM. These findings indicate that performance of a commercially available VR system for detection of lung nodules may be similar to traditional radiology monitors for assessment of small lung nodules on CTs of the thorax for most radiologists. These results, along with the potential of improving ergonomics for radiologists, are promising for the future development of VR in diagnostic radiology.

Introduction

Virtual reality (VR) is an emerging technological advancement that traces its origins to the establishment of stereoscopic viewing in the 1830s with the classic View-Master (Mattel Inc., Hawthorne, CA), which integrated rotating stills of stereoscopic 3-dimensional (3D) images (1). Advances in technology and successive iterations have now enabled users to be artificially immersed and interact with computer-simulated worlds that are customizable with editing software, which can be used to create application-specific environments. In radiology, stereoscopic assessments were first investigated in the late 19th and early 20th centuries (2–4). VR headset allows for the user's field of vision to be fully replaced with a digital image, where each eye looks through a different lens to create a stereoscopic 3D effect (5). VR technology today has become readily accessible through commercially available hardware such as the HTC Vive (HTC, New Taipei City, Taiwan), so that it no longer requires the use of expensive equipment and software (6, 7).

Owing to extensive utilization of imaging technology, radiology may be well suited for VR adoption and integration. For example, radiology has undergone rapid change of the image interpretation environment when picture archiving and communication system (PACS) was introduced. PACS allows multimodality images to be displayed on monitors, which do not need to be located in the same geographic area as the medical scanners (8). One of the disadvantages of PACS has been an increase in image complexity, resulting in increased physical and mental fatigue among radiologists, especially with repetitive movements.

VR presents new opportunities by allowing for the conversion of standard radiology environment into a virtual 3D workspace that can be extrapolated to room scale. In a VR environment, radiological studies can be viewed not only while sitting or standing but also when moving in an area of >100 square feet in size (Figure 1). VR may enable a radiologist to use a dynamic environment that is not constrained by the hardwired mouse and monitor of the PACS station. Furthermore, VR enhances user experience by providing an immersive setting in which the displayed data are not limited by the number of physical PACS monitors available on the desktop. The standard mouse may be replaced by physical controllers whose “real-world” location accurately correlates with their location in the virtual environment owing to real-time tracking by infrared cameras. With the potential to access these features, VR may extend the boundaries of traditional radiographic interpretation and further improve ergonomics in radiology.

Figure 1.

Established and experimental image interpretation positions radiology: sitting with standard monitor (A), standing with standard monitor (B), and roaming with virtual reality in roomscale environment (C).

Although VR technology has been used for educational purposes in diagnostic imaging (9), there are no studies that have investigated the utility of VR as a tool for the interpretation of tomographic images. Therefore, the aim of this study is to expand the options of the standard radiology image-interpretation environment by evaluating the VR system to allow the user to view and interpret images in varied postures. Small lung nodules provide an opportunity to clinically assess the display resolution limits of the current commercially available VR technology. Furthermore, prior reports have suggested higher detection and classification performance with stereoscopic viewing in lung nodule detection in computed tomography (CT) (10, 11). With stereoscopy built into the VR systems, this may provide an additional benefit. Therefore, in this project, radiologists evaluate the performance the VR environment for detection of lung nodules on standardized lung phantom image data and clinical data from patient CT images of the thorax.

Materials and Methods

Patients and Lung Phantom

A retrospective computer-based data search identified 33 646 patients who underwent routine CT imaging of the thorax between January 2011 and September 2016 and whose radiology reports contained the text “nodule” or “nodules.” To avoid an excess nodule count per patient and to examine the size detection limits of VR systems, the search was further refined by applying an age range of 18–40 years and by limiting scans to only those with radiology reports containing the words “few” and “sub 4 mm” in the text. A ≤4-mm-nodule threshold was chosen to ensure more challenging assessment than the 6-mm-size limit suggested by the Fleischner Society 2017 guidelines for incidentally discovered pulmonary nodules (12). This approach yielded 87 patients. A cardiothoracic radiologist, who did not participate as an experimental reader in this study, selected from among the 87 CTs a final test group of scans reviewed in consecutive order until an approximate target number of ∼200 nodules was identified to ensure adequate power for statistical analysis. The cardiothoracic radiologist had, at his disposal, zoom capability, coronal and sagittal reformations, and maximum intensity projection (MIP) images. Throughout this process, any patients found to have an active lung process such as pneumonia or extensive background lung disease such as cystic fibrosis were excluded from further consideration. Of those patients remaining, all discrete nodules measuring ≤4 mm, including perifissural and calcified nodules, were deemed acceptable for inclusion. Subpleural nodules with obtuse margination and ground-glass nodules were ignored. This HIPAA-compliant study was approved by the institutional review board. Furthermore, CT images of a Lungman Phantom (Kyoto Kagaku, Kyoto, Japan) with simulated lung nodules at known locations were used in the study.

CT Imaging Protocols

Patients participating in this study underwent helical CT scans with a single breath-hold and the following parameters: 100–200 mAs, peak tube voltage of 120 kV. CT images with 0.625-mm section thickness were transferred to picture archiving and communication system IMPAX (Agfa-Gevaert Group, Mortsel, Belgium).

Phantom CT Image Analysis

Technical specifications VR and SM systems are shown in Table 1. Cardiothoracic radiologist (R1, >10 years of experience) and 2 radiologists (R2, >10 years of experience, and R3, 3 years of experience) evaluated axial lung-window CT images of a Lungman phantom with SM and VR (HTC Vive at least 2 weeks apart. Steam (Valve Corporation, Bellevue, WA) software was used to create a virtual monitor in VR on which CT images were displayed using IMPAX. NVIDIA GeForce GTX1060 (NVidia Corporation, Santa Clara, CA) graphics card was installed on personal computer used with the VR system. The readers reported the axial image number and the lobe where the nodule was identified and findings were compared with known nodule locations.

Table 1.

VR System and Standard PACS Monitor Specifications

VR System	PACS Monitor
Brand: HTC	Brand: Barco
Type: Vive	Type: MDCC-6330
Resolution: 2160 × 1200 (1200 × 1080 per eye)	Resolution: 3280 × 2048
Physical Size: 19.0 × 12.7 × 8.9 cm, 563 g	Physical Size: 65.4 × 40.9 cm
Price: $400–$800	Price: $15,000
Display Type: Dual low-persistence Samsung AMOLED (Diamond PenTile subpixel matrix)	Display Function: DICOM GSDF
Display Size: 91.9 mm × 2447 ppi	White Point Luminance: Maximized Lifetime (449.77 Cd/m2)
Field of View: ∼110H × 113V-degrees at optimal 8 mm lens-to-eye distance	White point Chroma: Native White
Lens Type: Fresnel	Ambient Light: (AAPM) CT/MR/NM Reading Room (0.3 Cd/m2)
Lens Adjustment: IPD (60.8-74.6 mm), lens-to-eye distance (“eye-relief” adjustment)	Black Luminance: Native Black (0.5 Cd/m2)
Refresh Rate: 90 Hz	Uniformity Luminance Technology: Enabled
Sensors: Accelerometer, gyroscope	Auto-calibration: Twice a year
Tracking Technology: 6 DOF IR Laser-based 360-degree tracking using “Lighthouse” Base Stations	DICOM gray scale calibrated
Integrated Camera: Yes	Compliance Test: Monthly
Tracking area: 15 × 15 feet	Display Test: Daily
Requirements: NVIDIA GeForce GTX 970 /AMD Radeon RX 480 equivalent or greater	Visual Test: Yearly

Patient CT Image Analysis

Cardiothoracic radiologist (>10 years of experience), who did not participate as an experimental reader in the study, identified the location and size of all nodules on patient CT images that would need to be detected by the readers and had exclusive zoom, multiple angle and MIP viewing capabilities on SM. Subsequently, a radiologist (R4, 3 years of experience) and a novice radiologist (R5, <1 year of experience) reviewed only axial lung-window patient CT images of the thorax using VR system and SM (Barco, MDCC-6330, Barco Inc., Duluth, GA) at least 2 weeks apart. The 2 readers reported the axial image number and lobe where each nodule was detected and findings were compared with known nodule locations. The time spent by radiologist R4 and novice radiologist R5 using VR and SM while evaluating patient CT images was measured and compared.

VR Questionnaires

Questionnaires were administered to readers to assess reader experience and ergonomics. Readers were asked to report the presence of potential VR-related symptoms, including general discomfort, headache, stomach awareness, nausea, vomiting, pallor, sweating, fatigue, drowsiness, disorientation, and apathy, and rate their symptoms as mild, moderate, or severe.

Statistical Analysis

The McNemar test was used to compare sensitivities for each investigator in the detection of nodules using VR and SM with SPSS Statistics (IBM Corporation, Armonk, NY). Specificity was not evaluated. Paired t-test was calculated to compare the time spent between VR and SM in the evaluation of patient CTs using SPSS Statistics.

Results

Lung Nodules

On the basis of inclusion criteria, the final study group consisted of 32 patients (women, 14; men, 18; mean age, 30.3 years; range, 19–40 years). Among all patients, there were a total of 227 nodules: 1 mm (15), 2 mm (127), 3 mm (66), and 4 mm (19). Lungman phantom CT images included 23 nodules: 5 mm spherical (21), 6 mm spiculated (1), and 7 mm fish oil pill (1).

Sensitivities

Sensitivities for VR versus SM in detection of lung nodules on Lungman phantom CTs are shown in Table 2. Sensitivities for cardiothoracic radiologist R1 and radiologist R3 were not significantly different between VR and SM (P = .505) reviewing phantom CTs. Sensitivity was greater for only radiologist R2 when using SM compared with VR (P = .041). Sensitivities for VR versus SM in the detection of lung nodules on patient CTs are shown in Table 3. Sensitivities for radiologist R4 and novice radiologist R5 were not significantly different between VR and SM (P = .063 and 0.633 respectively) examining patient CTs.

Table 2.

Sensitivities (%) for Detecting Simulated Lung Nodules With VR and SM on Lungman Phantom CT Images

	VR	SM	P
R1 (Cardiothoracic Radiologist)
Sensitivity	56.5	69.6	.505
CI	76.8–36.3	88.4–50.8
R2 (Radiologist)
Sensitivity	52.2	78.3	.041
CI	72.6–31.8	95.1–61.4
R3 (Radiologist)
Sensitivity	60.1	73.9	.505
CI	80.8–40.9	91.9–56.0

Abbreviations: VR, virtual reality; SM, standard monitor; CI, confidence interval.

Table 3.

Sensitivities (%) for Detecting Lung Nodules With VR and SM on Patient CT Images of the Thorax

	VR	SM	P
R4 (Radiologist)
Sensitivity	66.4	72.3	.063
CI	72.5-60.2	78.1-66.4
R5 (Novice Radiologist)
Sensitivity	40	38.3	.633
CI	46.6-33.7	44.7-32.0

Abbreviations: VR, virtual reality; SM, standard monitor; CI, confidence interval.

Time

Radiologist R4 spent an average of 273(79) [mean(SD)] seconds on VR and 275(69) seconds on SM evaluating patient CT images. Novice radiologist R5 spent an average of 158(43) seconds on VR and 175(64) seconds on SM examining patient CT images. There was no statistically significant difference in the time spent by radiologist R4 or novice radiologist R5 between VR and SM (P = .891 and 0.100, respectively) reviewing patient CT images.

User Feedback

One user reported mild headache, 1 mild stomach awareness, and 1 mild nausea with VR. No users reported headache, stomach awareness, or nausea with SM.

Discussion

This study compared the sensitivity of a commercially available VR system and traditional radiology monitors for detection of clinical lung nodules and standardized lung lesions on CTs of the thorax. VR and SM exhibited no statistically significant differences in detection sensitivities for the 277 clinical lung nodules identified by an experienced radiologist and a novice radiologist. Furthermore, for the 23 standardized lung nodules in the phantom, both VR and SM exhibited no statistically significant differences in detection sensitivities for all radiologists except 1. Stereoscopic applications in radiology were first reported in the late 19th and early 20th century (2–4). Wang et al. compared stereo display with orthogonal MIP display and section-based display across 91 lung nodules on chest CT and showed that stereo display had a slight advantage in detection rate, although not reaching statistical significance (10). In a related study, Wang et al. compared stereographic display with 2 different monoscopic display schemes across 647 lung nodules on chest CT and showed that stereo display had higher detection performance with a shorter time spent, although again the result was not statistically significant (11). Although no similar studies with VR stereoscopy have been carried out in diagnostic radiology for comparison, Farahani et al. explored the use of the Oculus Rift VR headset for examining digital pathology slides in a VR environment (13). In this work, 3 pathologists reviewed 20 randomly selected digital lymph node slides, first on a 27-inch 5K display and 2 weeks later using the Oculus Rift VR system (Facebook, Menlo Park, CA), to categorize lymph nodes as either benign or malignant. There was a 90% diagnostic concordance between the traditional method of reviewing whole slide imaging on a flat computer monitor and the VR method. Furthermore, the pathologists unanimously confirmed that the digital pathology slides were easily viewable in a VR environment using the Oculus Rift VR headset. Within interventional radiology, Devcic et al. has recently reported no difference in standard rendering when compared to VR (P = .14) in preoperative planning in splenic artery aneurysm repair using CT angiography. With the ability to analyze and manipulate 3D images in VR, operators had improved confidence (14). These results of diagnostic equivalence between VR and standard display within pathology and interventional radiology, combined with our findings of no significant difference in sensitivity in lung nodule detection for majority of diagnostic radiologists, are promising for the future development of VR in medical imaging.

In this study, we have chosen the detection of small lung nodules to test the sensitivity of the VR system, as it represents one of the more challenging radiographic tasks and is clinically important because lung nodules may potentially represent primary or metastatic malignancy. Specificity was not assessed because of the disproportionately large number of true negatives, which would be represented by any left or right portions of each CT section viewed in the study, which did not contain a lung nodule. Furthermore, in this initial study, we are primarily interested in the positive detection of nodules. A 2-week interval was imposed between VR and SM readings to avoid recall bias, as multiple consecutive readings have been shown to improve lung nodule identification (15). Furthermore, CT modality was chosen over chest radiography for this work, as several studies have indicated the mode of chest imaging dictates sensitivity of detection, with CT showing superiority over traditional chest radiography (16, 17).

With future enhancements in VR technology display and tracking systems, VR may offer benefits of improved ergonomics in radiology. Although the introduction of the PACS system has allowed for accelerated image viewing, it has resulted in higher image volume and complexity, causing physical fatigue, both through constrained posture and repetitive hand movements. Suggestions to change the current seated work station (Figure 1A) have intended to address positional ergonomic deficits through the use of standing desks (Figure 1B) and exercise desks that incorporate treadmills and ellipticals (18). Fidler et al. showed no difference of diagnostic accuracy of 2 radiologists (P = .0003 and P < .0001) when comparing a mobile workstation with a desk-confined workstation (19). Because a commercially available VR system may allow for room-scale mobility (Figure 1C), all radiologists in this study used the VR system as a portable VR PACS workstation that allows for freedom of movement and axial unloading without being confined to a chair. All radiologists using VR were able to roam in a room-scale environment and mobilization while interpreting medical images did not affect detection sensitivities for the majority of radiologists. In this study 1 user reported mild headache, 1 mild stomach awareness, and 1 mild nausea with VR. It is not known if the use of VR in radiology would reduce physical fatigue or affect eye strain, and future studies may be performed to evaluate the presence of physical and mental stress with VR.

VR immerses the user into the image interpretation experience and reduces distractions present within the open traditional reading environment. The displayed data are also not limited by the number of physical PACS monitors on the desktop, which may allow for significant cost reduction as a one-time investment in a VR system can produce unlimited screens while increasing the number of physical SM incurs multiplicative costs. The costs comparing the SM and VR systems are shown in Table 1.

One of the limitations of this study was the small number of nodules in the lung phantom, which may have caused underestimation of sensitivity differences for cardiothoracic radiologist R1 and radiologist R3. Furthermore, with only 32 patients enrolled in the study, it is more difficult to detect statistically significant sensitivity differences for readers R4 and R5. Nevertheless, the study was adequately powered with >200 nodules. The HTC Vive model used in this study has an inherent lower combined resolution of 2160 × 1200, compared the PACS monitor resolution of 3280 × 2048 (Table 1), which may decrease its sensitivity for detecting sub-5-mm modules. Nevertheless, the resolution of commercially available VR systems continues to improve, for example, with increased combined resolution of 2880 × 1600 pixels of the newly released HTC Vive Pro. Increase in resolution and sampling rate of VR systems has been reported to decrease rare VR-related symptoms such as nausea and headache. Furthermore, development of smaller headsets and wireless VR systems is rapidly advancing.

In conclusion, this study shows that the detection of lung nodules on CT images by commercially available VR systems is similar to traditional radiology monitors for assessment of small lung nodules for most radiologists. These results, along with the potential of improving ergonomics for radiologists, are promising for the future development of VR in medical imaging. This technology may have the potential to reduce the constraints of physical monitors to allow the radiologist more freedom of motion, particularly with the development of smaller headsets and wireless systems.