Abstract
The study investigates the performance of a dedicated whole-body magnetic resonance imaging (MRI) interpretation software with regard to diagnostic efficiency using quantitative and qualitative parameters. Forty-eight oncologic patients underwent whole-body computed tomography (WB-CT) and whole-body magnetic resonance imaging (WB-MRI). In a quantitative analysis, the times needed for interpretation of the CT and MRI datasets were measured. The MRI studies were read using a standard workstation and the whole-body MRI interpretation software, respectively. In the qualitative analysis, the numbers of metastases were separately recorded for 13 organ systems, again interpreting the MRI images on the standard workstation and with the dedicated software. Moreover, user friendliness and system usability were evaluated using a standardized questionnaire. Use of the whole-body MRI interpretation software significantly reduced the MRI interpretation time compared with the standard workstation. There was no significant difference between interpretation time of WB-CT and interpretation time of WB-MRI using the dedicated software. Comparison with WB-CT as the reference method demonstrated no significant difference between the whole-body MRI interpretation software prototype and the standard interpretation software in the number of metastases detected. In conclusion, the use of the dedicated whole-body reading software improves the interpretation process of WB-MRI studies with respect to time efficiency and system usability.
Key words: Whole-body imaging, image interpretation software, diagnostic efficiency, graphical user interface
Introduction
Major innovations, in particular, regarding receiver coil technology were necessary before a comprehensive examination of the total body by magnetic resonance imaging (MRI) became practical. The dedicated whole-body surface coil systems available today allow imaging from head to toe without the need for patient repositioning in an acceptable imaging time and with high image quality.1 Current areas of application for whole-body MRI (WB-MRI) comprise cardiovascular2–4 and oncologic indications1,5,6 as well as evaluation for inflammatory disease throughout the body.7 WB-MRI is expected to be increasingly used because it has some decisive advantages over computed tomography (CT), primarily its high soft-tissue contrast. Because of this advantage, WB-MRI was found to be equal or even superior to WB-CT in detecting disease processes in some preliminary studies.8,9
A major disadvantage of WB-MRI is closely related to issues on transforming the radiological interpretation process (TRIP), as unsolved problems regarding the management and effective reading of large numbers of sequences and images throughout the interpretation process are inherent.1,10,11 Conventionally, available workstations, in particular their graphical user interfaces (GUI), provide inadequate features for optimized workflow and rapid anatomical identification of individual sequences when reading large whole-body MRI datasets. These limitations might be overcome by a dedicated whole-body interpretation software with a GUI designed especially for whole-body MRI.
The aim of this study was to evaluate a prototype of a whole-body MRI interpretation software in terms of temporal efficiency, user-friendliness, and system usability.
Materials and Methods
Patients
Forty-eight oncologic patients (29 men, 19 women; mean age of 61.3 years) who were followed up by whole-body computed tomography (WB-CT) and whole-body magnetic resonance imaging (WB-MRI) were included in the analysis. Twenty-one patients had malignant melanoma and 27 patients had renal cell carcinoma.
Whole-body imaging (MRI, CT)
With both imaging modalities, the head, neck, chest, abdomen, and pelvis were examined. Upper and lower legs were imaged by MRI only. The mean interval between CT and MRI was 1.5 days (range 1–6 days). The CT and MRI examinations were approved by the local ethics committee, and all patients gave written informed consent.
MRI was performed on a 1.5-Tesla MR scanner with 32 independent receiver channels and a dedicated whole-body surface coil system with up to 76 coil elements (Magnetom Avanto with total imaging matrix (TIM) system, Siemens Medical Solutions, Erlangen, Germany). Eight localizer scans were obtained followed by acquisition of 26 sequences with a total of 1,305 images. The contrast-enhanced images were obtained after intraveneous (IV) injection of a gadolinium-based unspecific extracellular contrast medium with a gadolinium concentration of 0.5 mmol/ml at a body-weight-adjusted dose (Magnevist, Bayer-Schering-Pharma AG, Berlin, Germany).
The CT examinations were performed on a 64-slice CT scanner (Sensation 64, Siemens Medical Solutions, Forchheim, Germany) with 64*0.625 mm collimation and X-ray dose modulation (Care-Dose 4D). After oral administration of 1,000 ml contrast medium over 60 min and IV bolus administration of 120 ml of an iodine-based contrast medium with a concentration of 370 mg iodine/ml (Ultravist 370, Bayer-Schering-Pharma AG, Berlin, Germany), the chest, abdomen, and pelvis were consecutively scanned with a delay of 50 s. Subsequently, the neck was scanned after a second IV injection of 50 ml contrast medium. All acquisitions were reconstructed with a slice thickness of 1 mm to enable individually defined multi-planar reconstructions. The head was scanned last using an incremental technique (sequential slices, slice thickness of 4 or 5 mm).
Standard image interpretation workstation
The standard workstation (Magic View 1000, Siemens Medical Solutions, Erlangen, Germany) was equipped with a GUI that provides all regular features used in daily practice at our institution. The worklist with the patient names allows to switch between a numerical mode (displaying the total number of images) and a thumbnail mode that gives a quick overview on the respective image stack (Figure 1). Measurement tools, hanging protocols (configurable by the user) with either stack view or consecutive viewing mode were available (Figure 1 shows the stack mode that was used in this study). Three-dimensional (3D) multi-planar reconstructions (MPR) could be generated from each stack of images (Figure 2).
Fig 1.
Graphical user interface of the standard workstation: worklist. The worklist is displayed in the thumbnail mode, the MR images are shown in the stack-view mode and can be chosen by drag-and-drop from the respective thumbnail. The icons for selecting further features (e.g. measurements, changing of the hanging protocol) are displayed in the window on the left.
Fig 2.
Graphical user interface of the standard workstation: multi-planar reconstructions. Interpretation of a stack of thin-slice CT images using multi-planar-reconstructions (MPR).
Whole-Body MRI Interpretation Software
The whole-body MRI interpretation software was a prototype provided by the vendor (Siemens Medical Solutions, Erlangen, Germany). The structure of the GUI and the icons are based on a platform the vendor uses for many scanner- and workstation interfaces (SYNGO®). In addition to the features of the standard interpretation workstation mentioned above, the whole-body MRI interpretation prototype has several further features (Figure 3). At first, the software automatically assigns the different MR sequences to individually predetermined anatomical regions. This sorting is performed on the basis of a substring search of the series names. The filtering substrings are configurable, so the rules can be adapted to the series-naming convention of the individual user. In addition to series names, further DICOM attributes (e.g., slice position) are taken into account. The GUI allows the user to individually adjust which anatomic regions he prefers (Figure 4). In the present study, the MRI sequences were assigned to the following anatomic regions: head, neck, chest, abdomen, pelvis, and lower extremity. Localizer scans were automatically assigned to a category of their own, clearly separating them from the diagnostic sequences.
Fig 3.
Graphical user interface of the whole-body MRI interpretation software prototype. The anatomic region can be chosen in the upper left window by clicking on the respective body part. The window in the lower left part of the GUI allows to select the anatomic region by selecting the respective region from a list (shown here for the thorax). Images are displayed in the stack mode. The icons to select further features (e.g., measurements) are displayed in the window at the bottom of the GUI.
Fig 4.
Graphical user interface of the whole-body MRI interpretation software. Magnification of the window that allows to select the anatomic body region. By clicking on the respective body part, all MR sequences of this region are displayed in the thumbnail mode (shown here for the head, 4 sequences).
Furthermore, the GUI allows to define interpretation steps (e.g., organs–soft tissues–bony structures) to more clearly structure the interpretation process (Figure 4). Hanging layout of the sequences of the respective body regions was defined beforehand, and the hanging protocols can be personalized (either stack view or consecutive viewing mode). Because the software recognizes all series based on the analysis of their characteristic DICOM attributes, rather than by a fixed series name or series number, the hanging definitions are robust and easily transferable among patients.
Finally, the user can mark a lesion in any sequence, and the software then automatically transfers the marking to all other sequences of the same anatomic region. All marks are being collected in a findings list with bookmark functionality. The user can quickly jump to any of the findings by simply clicking it in the list. All findings can be summarized in a simple HTML-based report including automatically created key images for each finding.
All software settings (especially the sequence of reading steps and their hanging layouts) can be stored in a workflow template, which can be called up any time and applied to subsequent patients. This makes it easy to configure a characteristic reading-workflow based on one whole body case and then to reuse it for future cases (site-specific standardized reading).
Image Interpretation
The 48 whole-body MRI and CT datasets were read by a radiologist with 8 years of experience in the interpretation of oncologic MRI examinations using the standard diagnostic workstation. After 3 months, a second reading of the MRI datasets was performed using the whole-body MRI interpretation software prototype. Patient identification was anonymized for all readings.
Quantitative Analysis
The times needed for interpretation of the whole-body datasets were recorded for both software types and for the CT datasets. The interpretation time of each case was defined as from selecting the patient from the worklist to closing the case. The results were tested for significance with the Wilcoxon test assuming a probability of error of 5%.
Qualitative Analysis
Dividing the total body into 13 different organ systems/anatomical regions, the number of metastases detected with each of the two types of software was recorded and tested for significant differences using the t test (probability of error of 5%). The number of metastases detected with each system was compared with the respective number detected on CT, using the latter as the gold standard, as CT serves as the method of choice in metastasis screening in daily clinical practice. The differences in comparison to CT were presented graphically.
In addition, four radiologists who tested the software prototype evaluated the user friendliness using a five-point grading scale ranging from excellent (1) to deficient (5). They furthermore evaluated the prototype using the standardized System Usability Scale (SUS), a questionnaire comprising ten items and yielding a score of 0 to 100.12 Two of the radiologists were familiar with the GUI of the vendor used for the whole-body software prototype, and two were not.
Results
Quantitative Analysis
The mean time needed for interpretation of a whole-body MRI dataset was significantly shorter using the whole-body MRI interpretation software (p < .05; Wilcoxon test) compared to the standard software. The differences in mean interpretation time between CT and the whole-body MRI interpretation software were not significant (p > .05, Wilcoxon test) with CT interpretation being on average 1 min and 42 s faster. The results are listed in Table 1.
Table 1.
Interpretation Times
| n = 48 | MRI Interpretation Time: Whole-Body MRI Interpretation Software | MRI Interpretation Time: Standard Workstation | CT Interpretation Time: Standard Workstation |
|---|---|---|---|
| Mean | 12.60 | 14.20 | 10.90 |
| Median | 12.90 | 13.95 | 11.90 |
| Standard deviation | 3.45 | 3.85 | 3.65 |
All values are given in minutes.
Qualitative Analysis
No significant differences between the standard software and the whole-body MRI interpretation software were found regarding the number of detected metastases (p > .05, t test). The differences in the number of metastases detected by CT and MRI using the two types of software were presented graphically (Figure 5). The user friendliness was rated highly positive by both radiologists familiar with the vendor-specific GUI and by both radiologists not familiar with it (Table 2). The SUS scores were 90 and 87.5 for the radiologists familiar with it and 77.5 and 82.5 for the radiologists not familiar with it.
Fig 5.
Detection of metastases: comparison of standard software and whole-body MRI interpretation software with reference to CT. The number of metastasis is displayed on the y-axis, the 13 investigated anatomic regions/organ systems are displayed on the x-axis. Note that the numbers were calculated by subtracting the number of metastasis detected on CT from the number of metastasis detected on MRI (standard workstation and whole-body MRI interpretation software, respectively); LN: lymph nodes.
Table 2.
Results of the Evaluation of User Friendliness
| Parameter | 1st Radiologist | 2nd Radiologist | 1st Radiologist | 2nd Radiologist | Mean |
|---|---|---|---|---|---|
| Familiar with the GUI of the Vendor | Not Familiar with the GUI of the Vendor | ||||
| Clarity of interface | 1 | 1 | 1 | 2 | 1.25 |
| Intelligibility of icons | 1 | 1 | 3 | 3 | 2 |
| Suitability of the interface for radiological case presentations | 1 | 1 | 1 | 1 | 1 |
| Option for defining anatomical regions | 1 | 1 | 1 | 1 | 1 |
| Option for defining hanging of the sequences | 1 | 1 | 1 | 1 | 1 |
| Switching between anatomical regions | 2 | 2 | 1 | 1 | 1.5 |
| Speed of the software | 2 | 2 | 2 | 3 | 2.25 |
| Stability in the test phase | 1 | 1 | 1 | 2 | 1.25 |
| Option for documentation of findings by marking lesions | 2 | 2 | 2 | 1 | 1.75 |
| Image export | 3 | 3 | 3 | 3 | 3 |
Discussion
The expected future increase in whole-body MR examinations necessitates an efficient diagnostic software to adequately deal with the large number of images, which is currently one of the major limitations of whole-body MRI.10 Dedicated postprocessing software, especially in terms of multiplanar- and 3D reconstruction, has become available for CT to manage the “flood of images” resulting from the developments in multi-slice technology in recent years.13 However, there is still major ongoing effort to advance these applications which are constantly improved. A radiological workstation should enable efficient image interpretation in a minimum of time.14 For these reasons, newly developed software tools that allow individual user-defined configuration and are tailored to the specific requirements of whole-body imaging are highly welcome.
The whole-body MRI interpretation software prototype differs from the state of the art interpretation software in some crucial respects: the automatic assignment of the MR sequences to anatomic regions makes the interpretation process better organized, e.g., because the switching between different anatomic regions when reviewing pre- and postcontrast images is no longer necessary. Furthermore, predefined interpretation steps guide the radiologist through the reading process, eliminating potential errors arising from skipped sequences or organs. These errors could especially happen in whole-body image interpretation, when the order of the acquisition differs from the order of reading the images. In addition, the possibility to mark lesions and track these marks in other sequences potentially avoids double-counting of lesions that appear on multiple sequences with slightly different table positions. This is especially important for evaluating organs that are usually not captured on one single stack of slices due to the size of the organ (e.g., the lungs).
The results shown here indicate that the evaluated whole-body MRI interpretation software prototype improves the performance of whole-body MRI reading with respect to many issues. At first, the advantages of the software prototype over the standard software are primarily reflected by a significantly shorter image interpretation time and the high acceptance of the user interface by the radiologists. These results are in agreement with an earlier study that also found a significantly shorter interpretation time.10 A comparison with CT, the current method of choice for metastasis screening in daily clinical practice, has not been carried out so far. Moreover, earlier studies did not investigate whether the use of a dedicated whole-body interpretation software tool also has effects on the number of lesions detected. Our study did not reveal significant differences between the dedicated and the standard software in this respect in patients with metastatic malignant melanoma and renal cell carcinoma. These two tumor entities were chosen because they are characterized by a diffuse pattern of metastatic spread that makes metastasis detection a difficult diagnostic problem. In organs with large numbers of metastases (lungs and mediastinum), even fewer metastases were diagnosed using the software prototype, which may possibly be attributable to more accurate counting of lesions using the dedicated software tool because it allows marking of lesions, as discussed earlier. A limitation of this evaluation is the fact that CT was considered the gold-standard, although MRI is known to be more sensitive for lesion detection in some of the organs (e.g., brain, liver). The reason for using CT as gold standard was a practical point of view, as CT still serves as the method of choice for metastasis screening in probably the most radiological sites.
There were no significant differences in image interpretation time between CT and MRI using the dedicated whole-body software prototype. This is an important finding because CT still serves as the clinical method of choice for metastasis screening and therefore can be regarded to represent the benchmark in terms of image interpretation time. An imaging modality with markedly longer interpretation times than CT would therefore be less cost efficient and thus difficult to establish as a routine clinical imaging modality.
Many workstations that differ in terms of the graphical user interface are in clinical use, and a completely new designed GUI for whole-body MRI might be disadvantageous for some users because it would confront them with yet another GUI. In our opinion, the fact that the software prototype investigated here uses a known look and feel is an advantage, which is also reflected in the high ratings for user friendliness assigned by the radiologists.
There are several limitations regarding this study. At first, the fact that whole-body CT datasets were evaluated using a technology that is established in clinical routine but does not represent the highest state of the art in CT interpretation. The standard workstation used allows multiplanar- and 3D interpretation, but important new developments such as volume-rendering techniques and segmentation algorithms were not available. This workstation was chosen as a reference because it allows for interpretation of both CT and MRI and is still widely used at our department in the routine daily practice. A further limitation refers to the fact that the radiologist who read the studies was aware that the interpretation time was recorded, and he therefore might have changed the reading process leading to a systematic bias. A further study where the software automatically records the reading times would overcome this problem.
In conclusion, the results shown here indicate that the whole-body interpretation software prototype presented here improves the interpretation process of whole-body MRI studies with respect to time efficiency and system usability.
References
- 1.Schlemmer HP, Schafer J, Pfannenberg C, et al. Fast whole-body assessment of metastatic disease using a novel magnetic resonance imaging system: initial experiences. Invest Radiol. 2005;40:64–71. doi: 10.1097/01.rli.0000149250.37033.7c. [DOI] [PubMed] [Google Scholar]
- 2.Fenchel M, Scheule AM, Stauder NI, et al. Atherosclerotic disease: whole-body cardiovascular imaging with MR system with 32 receiver channels and total-body surface coil technology—initial experiences. Radiology. 2006;238:280–291. doi: 10.1148/radiol.2381041532. [DOI] [PubMed] [Google Scholar]
- 3.Klessen C, Asbach P, Hein PA, et al. Whole-body MR angiography: comparison of two protocols for contrast media injection. Rofo. 2006;178:484–490. doi: 10.1055/s-2005-858964. [DOI] [PubMed] [Google Scholar]
- 4.Kramer H, Schoenberg SO, Nikolaou K, et al. Cardiovascular screening with parallel imaging techniques and a whole-body MR imager. Radiology. 2005;236:300–310. doi: 10.1148/radiol.2361040609. [DOI] [PubMed] [Google Scholar]
- 5.Schaefer JF, Schlemmer HP. Total-body MR-imaging in oncology. Eur Radiol. 2006;16:2000–2015. doi: 10.1007/s00330-006-0199-0. [DOI] [PubMed] [Google Scholar]
- 6.Schmidt GP, Haug AR, Schoenberg SO, Reiser MF. Whole-body MRI and PET-CT in the management of cancer patients. Eur Radiol. 2006;16:1216–1225. doi: 10.1007/s00330-006-0183-8. [DOI] [PubMed] [Google Scholar]
- 7.Althoff CE, Appel H, Rudwaleit M, et al. Whole-body MRI as a new screening tool for detecting axial and peripheral manifestations of spondyloarthritis. Ann Rheum Dis. 2007;66:983–985. doi: 10.1136/ard.2007.069948. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Lauenstein TC, Goehde SC, Herborn CU, et al. Whole-body MR imaging: evaluation of patients with metastases. Radiology. 2004;233:139–148. doi: 10.1148/radiol.2331030777. [DOI] [PubMed] [Google Scholar]
- 9.Schmidt GP, Schoenberg SO, Schmid R, et al. Screening for bone metastases: whole-body MRI using a 32-channel system versus dual-modality PET-CT. Eur Radiol. 2007;17:939–949. doi: 10.1007/s00330-006-0361-8. [DOI] [PubMed] [Google Scholar]
- 10.Muller-Horvat C, Plathow C, Ludescher B, et al. Generating statements at whole-body imaging with a workflow-optimized software tool—first experiences with multireader analysis. Rofo. 2007;179:721–727. doi: 10.1055/s-2007-963077. [DOI] [PubMed] [Google Scholar]
- 11.Andriole KP, Morin RL, Arenson RL, et al. Addressing the coming radiology crisis—the Society for Computer Applications in Radiology transforming the radiological interpretation process (TRIP) initiative. J Digit Imaging. 2004;17:235–243. doi: 10.1007/s10278-004-1027-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Brooke J. System usability score (SUS) In: Jordan PW, Thomas B, Weerdmeester BA, McClelland AL, editors. Usability evaluation in Industry. London: Taylor and Francis; 1996. [Google Scholar]
- 13.Teistler M, Breiman RS, Lison T, et al: Simplifying the exploration of volumetric images, development of a 3D user interface for the radiologist’s workplace. J Digit Imaging (in press) 2008 (Mar 27,) [DOI] [PMC free article] [PubMed]
- 14.Moise A, Atkins MS. Designing better radiology workstations: impact of two user interfaces on interpretation errors and user satisfaction. J Digit Imaging. 2007;18:109–115. doi: 10.1007/s10278-004-1906-5. [DOI] [PMC free article] [PubMed] [Google Scholar]