Abstract
Objective
The aim of this study was to demonstrate the principle of supporting radiologists by using a computer algorithm to quantitatively analyse MRI morphological features used by radiologists to predict the presence or absence of metastatic disease in local lymph nodes in rectal cancer.
Methods
A computer algorithm was developed to extract and quantify the following morphological features from MR images: chemical shift artefact; relative mean signal intensity; signal heterogeneity; and nodal size (volume or maximum diameter). Computed predictions on nodal involvement were generated using quantified features in isolation or in combinations. Accuracies of the predictions were assessed against a set of 43 lymph nodes, determined by radiologists as benign (20 nodes) or malignant (23 nodes).
Results
Predictions using combinations of quantified features were more accurate than predictions using individual features (0.67–0.86 vs 0.58–0.77, respectively). The algorithm was more accurate when three-dimensional images were used (0.58–0.86) than when only middle image slices (two-dimensional) were used (0.47–0.72). Maximum node diameter was more accurate than node volume in representing the nodal size feature; combinations including maximum node diameter gave accuracies up to 0.91.
Conclusion
We have developed a computer algorithm that can support radiologists by quantitatively analysing morphological features of lymph nodes on MRI in the context of rectal cancer nodal staging. We have shown that this algorithm can combine these quantitative indices to generate computed predictions of nodal status which closely match radiological assessment. This study provides support for the feasibility of computer-assisted reading in nodal staging, but requires further refinement and validation with larger data sets.
Rectal cancer is a common and major cause of mortality in the Western world. The treatment of rectal cancer has seen significant progress over recent years with the development of new adjuvant and neoadjuvant therapies [1,2], and new surgical techniques such as transanal endoscopic microsurgery (TEM). These advances have made the accurate nodal staging of rectal cancer increasingly important in guiding the choice of treatment options for each individual patient. Confident exclusion of nodal involvement could enable patients with early-stage (T1) rectal cancer to undergo local excision such as TEM, with its advantage of reduced peri and post-operative morbidity compared with conventional surgical techniques [3-5]. Moreover, the accurate identification of nodes involved by tumour may identify patients who would benefit from either neoadjuvant chemoradiotherapy prior to surgery or adjuvant chemotherapy post-neoadjuvant chemoradiotherapy and resection where pre-operative treatment prevents accurate identification of pre-treatment disease status. Currently, rectal cancer staging is performed with MRI, which has established itself as the reference standard [6-8].
Different techniques have been evaluated to address the relatively low accuracy of nodal staging. Brown et al [9] described the use of morphological features of surgically resected lymph nodes to predict nodal status, and showed that 91% of nodes >4 mm in diameter with mixed signal intensity relative to the primary tumour were malignant; 92% of nodes with an irregular border contour were malignant; and that just 6% of nodes with smooth borders contained metastases. They also found that combining heterogeneous signal intensity and an irregular border gave a sensitivity of 85% and specificity of 95% for metastatic nodal disease [9]. However, this high per-lymph-node accuracy achieved by Brown et al [9] has not been replicated by others, and the published data indicate that there is a considerable variation in reported sensitivities and specificities achieved on a per-patient basis [10-12].
Computer-assisted reading (CAR) using feature extraction software has been shown to improve radiologist performance, particularly for less experienced readers such as those working in less subspecialised units, with a reduction in interobserver variability reported in CT colonography [13-15] and CT pulmonary angiography [13-15] using CAR. At present, there is no CAR software available to assist nodal reporting.
This proof-of-concept study aims to show the feasibility of a computer algorithm to quantitatively analyse the various morphological features used by radiologists to predict the presence or absence of metastatic disease in mesorectal nodes in rectal cancer.
Methods and materials
Patient and nodal selection
Patients at Oxford University Hospitals, UK, with rectal cancer who underwent pre-operative staging with MRI followed by primary excision or neoadjuvant therapy were identified from the local cancer registry. The initial staging MR studies were anonymised and reviewed by one of two radiologists (EMA or FVG) who had more than 5 and 15 years' experience, respectively, in interpreting rectal MR images. Examples of radiologically benign or malignant nodes were identified on high spatial resolution images from the study. The benign nodes were selected from patients with N0 surgical specimens on histopathology, and the malignant nodes from patients with node-positive disease (N1 or N2) on histopathology, whereby a true-positive node could be readily identified by matching with histology. A total of 43 nodes from 17 patients were selected. 20 nodes from 10 patients were benign, and 23 nodes from 7 patients were malignant.
MR technique
MRI was performed with a 1.5 T system (Signa® HDX; GE Medical Systems, Milwaukee, WI) with a pelvic phased-array surface coil. As part of the standardised rectal staging protocol, high spatial resolution T2 weighted fast spin echo images were acquired (repetition time, 3500 ms; echo time, 92 ms, echo train length, 16; slice thickness, 3 mm; field of view, 20 cm; 256×256 matrix; number of excitations, 3) in the coronal and oblique axial planes, oriented perpendicular to the long axis of the segment of rectum containing cancer.
Morphological predictors considered for this study
Based on the observations made by Brown et al [9], we included four morphological predictors for this study. Three of these were signal heterogeneity, relative mean signal intensity and node size. In their original work, Brown et al also considered the maximum diameter on an image slice as a descriptor of the node size. In this work, we also considered the volume, or area (in the case of slice-by-slice analysis), as an alternative descriptor of the node size. Enlarged nodes with uniform high signal intensity were considered benign [9]. Because the chemical shift artefact can be seen in nodes with uniform high signal intensity, this was included as a fourth morphological predictor.
Description of the computer algorithm
We devised a semi-automatic computer-aided method to both extract the morphological properties mentioned above and quantify them. We assumed that the location of a lymph node is known in a given image volume. We have previously published semi-automatic computer-aided procedures [17,18] to detect locations of possible lymph node candidates in a given MRI image volume. The first step of the computer algorithm is image segmentation. We used segmentation to divide the image into two different regions of interest, lymph node and non-lymph node, using the random walker segmentation algorithm proposed by Grady [19], who also provided the relevant computer codes. We have since completed an implementation of the random walker that is fast to compute, even for large image volumes. This method required initial seed points to be placed inside and outside the lymph node, although this can largely be automated. We used our own program to support this user interaction and then to perform other image-processing tasks specific to our analysis. Post-processing operations were performed to remove areas containing partial volume regions and to make the segmented region spatially continuous. The outcome of this phase of the procedure was a three-dimensional (3D) binary mask for the lymph node. This mask was then used to extract the following morphological properties of the lymph node and to quantify them.
Chemical shift artefact detection
Image morphological operations were used to search for bright signal voxels on the periphery of the segmented lymph node. A sample set of 11 nodes were assessed manually and, where present, the bright pixels representing the chemical shift artefact were estimated to represent at least one-eighth of the peripheral pixels. If the count of such bright pixels was more than one-eighth of the number of pixels on the periphery, chemical shift artefact was declared to be present. The chemical shift artefact “variable” could take only two possible numerical values: 1 (present) and 0 (absent).
Low signal voxels may appear on the periphery of the segmented lymph node owing to either the lymph node capsule or chemical shift artefact. In these incidences, we removed the voxels from further analysis because they may have been confused by the computer algorithm as part of the tumour which may also exhibit a low T2 signal intensity. A procedure similar to that used to detect chemical shift artefact was used for this purpose.
Relative mean signal intensity
For this purpose we selected the middle slice of the segmented lymph node, which was also expected to contain the largest number of voxels. We excluded all those voxels that were detected as a part of the chemical shift artefact and dark annular rim. We then calculated mean signal intensity of the remaining inside voxels, referred to as “m_inside”. We also calculated mean signal intensity of the outside voxels of the segmented lymph node using the initial seed points. This was referred to as “m_outside”. The relative mean signal intensity was obtained as m_inside divided by m_outside. The relative mean signal intensity “variable” could have any positive real value; however, in our experiment, it had values between 0.5 and 0.9.
Signal heterogeneity
As mentioned above, we selected only the middle slice of the segmented lymph node and excluded the chemical shift artefact and the dark annular rim voxels. We then calculated the information theoretic entropy (H) of the selected region, which quantified the spread of intensity values within the region; essentially, how dissimilar signal values were. This was calculated using the following formula:
 |
(1) |
where P(i) is the histogram of intensity values inside the selected region [20]. The information theoretic entropy has relatively low values if the histogram is sharply peaked, indicating that the signal is homogeneous, whereas it has relatively high values if the histogram is more widely spread, indicating that the signal is heterogeneous. The signal heterogeneity “variable” could have any positive real value; however, in our experiment, it had values between 2 and 6.
Node size (volume or area)
The segmented lymph node mask can be directly used to estimate either volume (in the case of 3D analysis) or area (in the case of slice-by-slice analysis). We also calculated the maximum diameter of the lymph node automatically from its segmented mask. The node volume/area “variable” could have any positive real value.
After the above features were quantified, the final step was classification: for each lymph node, quantitative indices of all morphological properties were grouped together to form a “feature vector”. A previously published machine learning technique known as the “random forest classifier” was used to classify these feature vectors into two classes: benign and malignant. A publicly available computer code for this classifier was used [21]. We also devised a two-dimensional (2D) version of the algorithm based on the same principles, but in which only the middle image slice containing the node was used.
Training and evaluation of the computer algorithm
For each lymph node assessed by the algorithm, the clinical status was identified as either benign or malignant. This clinical status was used as the reference standard and used to train the classifier. Our computer-aided procedure was evaluated using the “leave one out” procedure, which is commonly used in assessing machine learning when the size of the available data set is small. In each case, of the total 43 nodes, we used 42 to train the classifier and tested the remaining node. We then changed the test lymph node turn by turn and trained the classifier with the remaining 42 nodes, resulting in 43 test results, from which we calculated the final classification accuracy.
We performed two sets of experiments. In the first set of experiments, we evaluated the performance of the computer algorithm with 2D and 3D methods. In the second set of experiments, we evaluated the performance of the morphological predictors (signal heterogeneity; relative mean signal intensity; nodal volume or area; chemical shift artefact) individually and in combinations. The maximum node diameter, often used by radiologists to describe size, was also evaluated as an alternative criterion individually and in combination with other morphological predictors.
Comparison against manual assessment
We also validated our computer algorithm against manual assessment by one of the collaborating radiologists (DMLT). For each node, the radiologist scored each of the morphological properties in the following manner: signal heterogeneity (1=yes, 0=no); mean signal intensity (on the scale of 1–5, with 1 as high and 5 as low); chemical shift artefact (1=yes, 0=no); and maximum diameter (in millimetres). In order to compare the performance of the computer algorithm with manual assessment, we fed the manual assessment data to the classifier described previously.
Results
Table 1 summarises the accuracies of the computer algorithm against the reference standard when using various morphological features individually or in combination. Examples of nodes assessed by the algorithm are shown in Figures 1 and 2.
Table 1. Accuracy of prediction of lymph node status, as compared with the reference standard, using the proposed computer-aided algorithm.
| Combination of featuresa | Accuracy of classification (3D)b | Accuracy of classification (2D)c |
| 1 | 0.6977 | 0.5814 |
| 2 | 0.5814 | 0.4651 |
| 3 | 0.7674 | 0.5814 |
| 4 | 0.6744 | 0.4651 |
| 1, 2 | 0.8605 | 0.5814 |
| 1, 3 | 0.8372 | 0.6047 |
| 1, 4 | 0.7209 | 0.5814 |
| 2, 3 | 0.7674 | 0.6977 |
| 2, 4 | 0.6744 | 0.5814 |
| 3, 4 | 0.7907 | 0.6777 |
| 1, 2, 3 | 0.8372 | 0.5581 |
| 1, 2, 4 | 0.8605 | 0.5814 |
| 1, 3, 4 | 0.8605 | 0.6512 |
| 2, 3, 4 | 0.8372 | 0.5581 |
| 1, 2, 3, 4 | 0.8607 | 0.7209 |
3D, three-dimensional algorithm; 2D, two-dimensional algorithm.
aDifferent combination of morphological features used: 1, signal heterogeneity; 2, relative mean signal intensity; 3, node size (volume for 3D and area for 2D); 4, chemical shift artefact.
bAccuracy of classification for the 3D algorithm.
cAccuracy of classification for the 2D algorithm.
Figure 1.
Radiologically benign lymph nodes classified by the computer algorithm using all four morphological features. (a) A lymph node with a low signal heterogeneity score and chemical shift classified correctly by the algorithm as benign; (b) a small lymph node with a high signal heterogeneity score but no chemical shift artefact detected, and misclassified by the algorithm as malignant.
Figure 2.
Radiologically malignant lymph nodes classified by the computer algorithm using all four morphological features. (a) A large lymph node with low mean signal intensity and no detected chemical shift classified correctly by the algorithm as malignant; (b) a smaller lymph node with no detected chemical shift, but determined to have low signal heterogeneity score and high mean signal intensity, misclassified by the algorithm as benign.
Features in isolation and in combination
When each of the four morphological features analysed—signal heterogeneity (1), relative mean signal intensity (2), node volume or area (3) and chemical shift artefact (4)—were in turn used as the only predictor feature in the computer algorithm, the accuracies ranged between 0.58 and 0.77 for the 3D algorithm and between 0.47 and 0.58 for the 2D algorithm. When these features were combined in the algorithm, the accuracies of the algorithms were generally higher, ranging between 0.67 and 0.86 for the 3D and between 0.55 and 0.70 for the 2D algorithm. Using all 4 morphological features in the computer algorithm gave the highest accuracy for both the 3D and the 2D algorithms, with 37 out of 43 nodes correctly classified (0.86) for the 3D algorithm, and 31 out of 43 nodes correctly classified (0.72) for the 2D algorithm.
The accuracy of the combination of signal heterogeneity and mean signal intensity was higher than other combinations of two features, and identical to that of all four features together. The addition of the chemical shift artefact quantification into the algorithm (1, 2 and 4) did not improve diagnostic accuracy.
Performance of three-dimensional vs two-dimensional algorithms
In all algorithms using the various combinations of morphological features, the 3D version of the algorithm showed a higher accuracy than the 2D version. The range of accuracies using the whole volume of data (3D) available ranged from 0.58 to 0.86, while for the 2D algorithm using only the middle image slice the accuracies ranged from 0.47 to 0.72. When mean signal intensity (2) or chemical shift artefact (4) was used as the only morphological predictor in the algorithm, the accuracy was below 0.5 (Table 1).
Maximum node diameter
The maximum node diameter (5) was used as an independent feature. The results are shown in Table 2. When used alone as the morphological predictor in the algorithm, a high accuracy of 0.79 was achieved. Replacing the nodal size variable (3) with maximum node diameter in the various combinations used produced algorithms with a higher accuracy; for example, the combination of signal heterogeneity and maximum diameter (1 and 5) had an accuracy of 0.86 compared with an accuracy of 0.83 for signal heterogeneity and node volume (1 and 3). When all features were used but with node size replaced with maximum node diameter (1, 2, 4 and 5), the accuracy of the algorithm was increased from 0.86 to 0.91: equivalent to 39 out of 43 nodes correctly classified.
Table 2. Accuracy of prediction of lymph node status, as compared with the reference standard, using the proposed computer-aided algorithm.
| Combination of featuresa | Accuracy of classificationb |
| 5 | 0.7907 |
| 1, 5 | 0.8605 |
| 2, 5 | 0.8140 |
| 4, 5 | 0.8372 |
| 1, 2, 5 | 0.8837 |
| 1, 4, 5 | 0.8837 |
| 2, 4, 5 | 0.8837 |
| 1, 2, 4, 5 | 0.9070 |
aThe maximum nodal diameter (referred to as “5” in the left column) was used in the algorithm in combination with other morphological features.
bThe three-dimensional algorithm was used.
The results obtained from comparison with manual assessment are shown in Table 3. During the manual assessment, we also evaluated the signal heterogeneity property on the scale of 1–5. However, the results were generally low, so they have not been presented here. It can be noted from Tables 1–3 that the classification rates obtained while using each morphological property individually were generally higher for manual assessment except for the maximum diameter, where the computer algorithm showed better results. The classification rates with combination of morphological properties were very much comparable for both the methods.
Table 3. Accuracy of prediction of lymph node status with each individual property assessed manually.
| Combination of featuresa | Accuracy of classificationb |
| 1 | 0.7907 |
| 2 | 0.8142 |
| 4 | 0.8372 |
| 5 | 0.7442 |
| 1, 2 | 0.8140 |
| 1, 4 | 0.8372 |
| 1, 5 | 0.8837 |
| 2, 4 | 0.7674 |
| 2, 5 | 0.8605 |
| 4, 5 | 0.8140 |
| 1, 2, 4 | 0.8140 |
| 1, 2, 5 | 0.9302 |
| 1, 4, 5 | 0.8605 |
| 2, 4, 5 | 0.8602 |
| 1, 2, 4, 5 | 0.9302 |
aDifferent combination of morphological features used: 1, signal heterogeneity; 2, mean signal intensity; 4, chemical shift artefact; 5, maximum diameter.
bThe three-dimensional algorithm was used.
Discussion
In this study, we have shown that it is possible to use a computer algorithm to analyse high-resolution MR images from staging studies to quantitatively assess morphological features of nodes, including size as represented by 3D volume or maximum 2D diameter, signal heterogeneity, mean signal intensity relative to that in the immediate neighbourhood of the node and the presence or absence of chemical shift artefact. We have demonstrated that combinations of these quantitative indices can be used to generate a computed prediction of whether a node is benign or malignant, and that certain combinations produce predictions that closely match the radiological diagnosis for each node and perform better than each of the indices on their own.
MRI is currently accepted as the best technique for both nodal identification and categorisation in rectal cancer. While size as represented by the maximum 2D diameter is the most widely used feature for predicting nodal status, it has long been recognised as inadequate [9,22,23]. Size criteria fail to accurately predict nodal involvement, owing to the high rate of small involved lymph nodes in rectal cancer; mesorectal lymph nodes <5 mm in diameter still had a prevalence of lymph nodes that were positive for cancer of 15% [9]. A cut-off at 5 mm has an accuracy of 75%, comparable to our results.
Additional criteria have been sought to characterise lymph nodes, and the morphology of the nodes has proved to be a highly specific discriminator between benign and malignant nodes. Both Kim et al [24] and Brown et al [9] identified heterogeneous signal intensity nodes, on T2 weighted imaging, as being significantly more likely to be malignant. The sensitivity and specificity of signal heterogeneity for involvement was 49% and 99%, respectively, implying an accuracy of 88%, which is similar to our findings.
Lymph nodes with T2 signal intensity higher than the primary tumour are rarely involved [9], but there is considerable overlap between involved and uninvolved nodes whose signal intensity is similar to or less than the primary tumour. We found that on its own signal intensity did not accurately discriminate between lymph nodes that were considered involved or not. However, an unexplained finding was of an increase in classification accuracy when relative mean signal intensity was combined with heterogeneity of signal or nodal size.
Chemical shift is seen in enlarged, hyperintense signal nodes, which rarely contain tumours. As a single determinant of nodal involvement, it performed slightly better in our model than signal intensity did, but again increased the accuracy of nodal classification when combined with other morphological features.
Having established the importance of morphological features for the classification of nodal involvement, it is disappointing that their reliable identification has proven difficult, with other authors failing to produce similar results [25,26]. Other imaging strategies have been used to improve nodal characterisation, including ultra-small iron oxide particles [25] and diffusion-weighted imaging; however, these have also faced limitations [26].
CAR using feature extraction algorithms has been applied in several imaging settings such as detection of breast cancer at mammography, pulmonary nodules in CT and colonic polyps in CT colonography. CAR in these settings has been reported to show improvement in radiologist performance with a reduction in interobserver variability [13-15]. For inexperienced users, the benefit from CAR appears to be greater, although it may not provide a substitute for adequate training. At present, there is no CAR software available to assist nodal reporting; however, some of the underlying features we have used in our software such as size and intensity are also analysed by CAR software used in the other settings [13,15].
There were limitations to our study. We did not have a node-by-node correlation with histopathology for our reference standard; rather, all the benign nodes were chosen from patients who were node-negative at surgery, and the malignant nodes were identified from node-positive patients whereby the malignant node on MRI was judged to have matched that described at histology. Thus, the malignant nodes may demonstrate more extreme morphological characteristics than may generally be the case. Our relatively small sample size also meant that it was not appropriate to carry out statistical analysis to compare the performances of different combinations of features. However, the focus of this pilot study was to demonstrate the feasibility of such an algorithm in quantifying the features of the MR images already in use by radiologists when categorising lymph nodes as benign or malignant on MRI, rather than to test its clinical utility.
We have not yet considered the nodal contour. This has been demonstrated as an important morphological feature [9,24]. However, we felt that this morphological predictor could not be satisfactorily quantified with currently available computer-aided algorithms, not least because of the partial volume effect and chemical shift artefacts.
The comparatively poorer performance of the 2D algorithm can be explained by the reduced number of voxels available for analysis, even when the middle image slice containing the node was used, and the fact that the 2D images were not sectioned according to orientation of the nodes. This suggests that high-resolution, 3D volumetric data sets may allow improved performance from the computer algorithm.
In conclusion, we have developed a computer algorithm to quantitatively analyse features of nodes on staging MRI of rectal cancer patients that are currently used by radiologists when determining whether a node is benign or malignant, and have generated a calculated prediction of nodal status. This technique needs to be validated in surgical specimens with histologically identified nodes, and also alongside reporting radiologists to determine if it can aid the identification of malignant nodes in clinical practice. The addition of contour analysis potentially using thinner MRI data sets is likely to further refine and improve the algorithm, and this work is in progress.
Acknowledgments
The authors acknowledge support from Microsoft Research and from the Cancer Research UK–Engineering and Physical Sciences Research Council-funded Oxford Cancer Imaging Centre for the development of the algorithm.
References
- 1.Meredith KL, Hoffe SE, Shibata D. The multidisciplinary management of rectal cancer. Surg Clin North Am 2009;89:177–215 [DOI] [PubMed] [Google Scholar]
- 2.Tytherleigh MG, Warren BF, Mortensen NJM. Management of early rectal cancer. Br J Surg 2008;95:409–23 [DOI] [PubMed] [Google Scholar]
- 3.Buess G, Hutterer F, Theiss J, Böbel M, Isselhard W, Pichlmaier H. A system for a transanal endoscopic rectum operation. [In German.] Chirurg 1984;55:677–80 [PubMed] [Google Scholar]
- 4.Buess G, Kipfmuller K, Ibald R, Heintz A, Hack D, Braunstein S, et al. Clinical results of transanal endoscopic microsurgery. Surg Endosc 1988;2:245–50 [DOI] [PubMed] [Google Scholar]
- 5.Suppiah A, Maslekar S, Alabi A, Hartley JE, Monson JR. Transanal endoscopic microsurgery in early rectal cancer: time for a trial? Colorectal Dis 2008;10:314–27 [DOI] [PubMed] [Google Scholar]
- 6.Burton S, Brown G, Daniels IR, Norman AR, Mason B, Cunningham D, et al. MRI directed multidisciplinary team preoperative treatment strategy: the way to eliminate positive circumferential margins? Br J Cancer 2006;94:351–7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Glimelius B, Oliveira J. Rectal cancer: ESMO clinical recommendations for diagnosis, treatment and follow-up. Ann Oncol 2008;9Suppl. 2:ii31–ii32 [DOI] [PubMed] [Google Scholar]
- 8.The Association of Coloproctography of Great Britain and Ireland Guidelines for the management of colorectal cancer. London, UK; 2007 [Google Scholar]
- 9.Brown G, Richards CJ, Bourne MW, Newcombe RG, Radcliffe AG, Dallimore NS, et al. Morphologic predictors of lymph node status in rectal cancer with use of high-spatial-resolution MR imaging with histopathologic comparison. Radiology 2003;227:371–7 [DOI] [PubMed] [Google Scholar]
- 10.Halefoglu AM, Yildirim S, Avlanmis O, Sakiz D, Baykan A. Endorectal ultrasonography versus phased-array magnetic resonance imaging for preoperative staging of rectal cancer. World J Gastroenterol 2008;14:3504–10 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Chun HK, Choi D, Kim MJ, Lee J, Yun SH, Kim SH, et al. Preoperative staging of rectal cancer: comparison of 3-T high-field MRI and endorectal sonography. AJR Am J Roentgenol 2006;187:1557–62 [DOI] [PubMed] [Google Scholar]
- 12.Akasu T, Iinuma G, Takawa M, Yamamoto S, Muramatsu Y, Moriyama N. Accuracy of high-resolution magnetic resonance imaging in preoperative staging of rectal cancer. Ann Surg Oncol 2009;16:2787–94 [DOI] [PubMed] [Google Scholar]
- 13.Engelke C, Schmidt S, Bakai A, Auer F, Marten K. Computer-assisted detection of pulmonary embolism: performance evaluation in consensus with experienced and inexperienced chest radiologists. Eur Radiol 2008;18:298–307 [DOI] [PubMed] [Google Scholar]
- 14.Halligan S, Altman DG, Mallett S, Taylor SA, Burling D, Roddie M, et al. Computed tomographic colonography: assessment of radiologist performance with and without computer-aided detection. Gastroenterology 2006;131:1690–9 [DOI] [PubMed] [Google Scholar]
- 15.Baker ME, Bogoni L, Obuchowski NA, Dass C, Kendzierski RM, Remer EM, et al. Computer-aided detection of colorectal polyps: can it improve sensitivity of less-experienced readers? Preliminary findings. Radiology 2007;245:140–9 [DOI] [PubMed] [Google Scholar]
- 16.Mani A, Napel S, Paik DS, Jeffrey RB, Jr, Yee J, Olcott EW, et al. Computed tomography colonography: feasilbility of computer-aided polyp detection in a “first reader” paradigm. J Comput Assist Tomogr 2004;28:318–26 [DOI] [PubMed] [Google Scholar]
- 17.Joshi N, Bond S, Brady M. The segmentation of colorectal MRI images. Med Image Anal 2010;14:494–509 [DOI] [PubMed] [Google Scholar]
- 18.Bond S, Brady M, Gleeson F, Mortensen N. Image analysis of patient management in colorectal cancer. Computer Assisted Radiology and Surgery (CARS) 2005;1281:278–83 [Google Scholar]
- 19.Grady L. Random walks for image segmentation. IEEE Trans Pattern Anal Mach Intell 2006;28:1768–83 [DOI] [PubMed] [Google Scholar]
- 20.Bishop CM. Pattern Recognition and Machine Learning. New York, NY: Springer, 2006 [Google Scholar]
- 21.Random Forest [computer program] Open source. Available from: http://code.google.com/p/randomforest-matlab/
- 22.Vogl TJ, Pegios W, Mack MG, Hünerbein M, Hintze R, Adler A, et al. Accuracy of staging rectal tumors with contrast-enhanced transrectal MR imaging. AJR Am J Roentgenol 1997;168:1427–34 [DOI] [PubMed] [Google Scholar]
- 23.Zerhouni EA, Rutter C, Hamilton SR, Balfe DM, Megibow AJ, Francis IR, et al. CT and MR imaging in the staging of colorectal carcinoma: report of the Radiology Diagnostic Oncology Group II. Radiology 1996;200:443–51 [DOI] [PubMed] [Google Scholar]
- 24.Kim JR, Beets GL, Kim MJ, Kessels AG, Beets-Tan RG. High-resolution MR imaging for nodal staging in rectal cancer: are there any criteria in addition to the size? Eur J Radiol 2004;52:78–83 [DOI] [PubMed] [Google Scholar]
- 25.Koh DM, George C, Temple L, Collins DJ, Toomey P, Raja A, et al. Diagnostic accuracy of nodal enhancement pattern of rectal cancer at MRI enhanced with ultrasmall superparamagnetic iron oxide: findings in pathologically matched mesorectal lymph nodes. AJR Am J Roentgenol 2010;194:W505–13 [DOI] [PubMed] [Google Scholar]
- 26.Lambregts EM, Maas M, Riedl RG, Bakers FC, Verwoerd JL, Kessels AG, et al. Value of ADC measurements for nodal staging after chemoradiation in locally advanced rectal cancer – a per lesion validation study. Eur Radiol 2011;21:265–73 [DOI] [PMC free article] [PubMed] [Google Scholar]