Abstract
Objectives
Computer-aided detection (CAD) for CT colonography (CTC) has been developed to detect benign polyps in asymptomatic patients. We aimed to determine whether such a CAD system can also detect cancer in symptomatic patients.
Methods
CTC data from 137 symptomatic patients subsequently proven to have colorectal cancer were analysed by a CAD system at 4 different sphericity settings: 0, 50, 75 and 100. CAD prompts were classified by an observer as either true-positive if overlapping a cancer or false-positive if elsewhere. Colonoscopic data were used to aid matching.
Results
Of 137 cancers, CAD identified 124 (90.5%), 122 (89.1%), 119 (86.9%) and 102 (74.5%) at a sphericity of 0, 50, 75 and 100, respectively. A substantial proportion of cancers were detected on either the prone or supine acquisition alone. Of 125 patients with prone and supine acquisitions, 39.3%, 38.3%, 43.2% and 50.5% of cancers were detected on a single acquisition at a sphericity of 0, 50, 75 and 100, respectively. CAD detected three cancers missed by radiologists at the original clinical interpretation. False-positive prompts decreased with increasing sphericity value (median 65, 57, 45, 24 per patient at values of 0, 50, 75, 100, respectively) but many patients were poorly prepared.
Conclusion
CAD can detect symptomatic colorectal cancer but must be applied to both prone and supine acquisitions for best performance.
CT colonography (CTC) is increasingly used as a relatively non-invasive method of colonic investigation both for colorectal cancer screening [1,2] and in patients with symptoms suggestive of colorectal cancer [3-6]. Despite the advent of modern visualisation workstations, accurate interpretation of CTC is known to be difficult and requires substantial observer training [7-10]. It has been suggested that computer-aided detection (CAD) may enhance reader performance and perhaps also diminish the learning curve for CTC [11,12]. Recent studies confirm that reader sensitivity is increased when interpretation is supplemented by CAD [13-16].
Most claims for the potential benefits of CAD have been made in the context of colorectal cancer screening [12,17], which targets both early cancer and its precursor, the adenomatous polyp. In asymptomatic individuals, the prevalence of adenomas vastly exceeds that of early cancer and so CAD systems have naturally been optimised for polyp detection. However, CTC is also advocated for symptomatic patients, in whom the prevalence of established carcinoma is much higher [18]. Even then, symptoms suggesting cancer are both non-specific and common in the general population, with the result that most patients investigated do not actually have cancer [18], and CAD to assist diagnosis of colorectal cancer is therefore an attractive concept. Cancers, like polyps, are usually characterised by soft-tissue protrusion into the bowel lumen. It is therefore possible that CAD systems may serendipitously identify cancers. We aimed to determine whether a CAD system developed to detect colorectal polyps at CTC might also identify colorectal cancer in symptomatic patients.
Methods and materials
Local ethics committee approval was sought and a waiver obtained for retrospective analysis of CT data collected as part of standard care.
137 adult patients (58 male, 79 female; mean age 67.7 years, range 17–93 years) who had been investigated in 4 public hospitals for symptoms suggestive of colorectal cancer, and who were subsequently proved to have the disease, were identified retrospectively by searching the pathology database and matching this with the radiology database of those having CT colonography. Consecutive patients were identified at each centre and all were ultimately proven to have colorectal adenocarcinoma by histological examination of a colonoscopic biopsy specimen, by histological examination of the surgically resected tumour or by post-mortem examination in those patients unfit for attempted surgical resection.
CTC had been performed in each patient according to international guidelines for good practice [19]. Multidetector machines were used in all cases (Table 1). Full bowel purgation was attempted in all patients. Automated insufflation of carbon dioxide was used in 85 and room air in 52 (Table 1). Both prone and supine acquisitions were obtained in 125 patients; 12 patients underwent a single supine acquisition only, because of infirmity. CT data were acquired from the 4 centres between 2004 and 2006 inclusive; 2 from the UK (contributing 85 and 22 patients, respectively), 1 from Japan (contributing 21 patients) and 1 from China (contributing 9 patients). Tumour location broadly followed that expected in an unselected series of patients with colorectal cancer as follows: rectal, 50; sigmoid, 49; descending colon, 8; splenic flexure, 2; transverse colon, 2; hepatic flexure, 2; ascending colon, 9; caecal, 15.
Table 1. Details of 137 patients with proven colorectal cancer.
| Centre | No. of patients | Number of CT detector rows | Collimation (mm) | Insufflation gas |
| UK 1 | 85 | 16 and 64 | 2.5 | Carbon dioxide |
| UK 2 | 22 | 4 | 3.2 | Room air |
| Japan | 21 | 16 | 1.0 | Room air |
| China | 9 | 8 | 2.5 | Room air |
The CTC DICOM data from all 137 patients were transferred to a PC workstation loaded with commercially available CTC visualisation software (Innerview GI, Vital Images Inc., Minnetonka, MN) into which a proprietary CAD algorithm had been integrated (ColonCAD, Medicsight Plc, Hammersmith, UK). None of these data had been used previously to develop the CAD algorithm. Each individual case was interrogated by a single observer (CR), with prior experience of approximately 300 CTC examinations. The prone and supine data sets (if both were available) from each patient were opened with the visualisation software and the known colorectal cancer identified by the observer, on both series where possible. Colonoscopic data were available for each case to aid localisation and identification if the cancer was not immediately obvious. The original radiological report was also available if necessary. The maximal longitudinal extent of each cancer was estimated in centimetres using software calipers applied to the multiplanar reformat that best demonstrated this dimension.
The CAD system was then applied to each data set at each of 4 different sphericity settings: 0, 50, 75 and 100. Lower sphericity settings increased algorithm sensitivity for colorectal lesions projecting into the lumen at the expense of specificity (i.e. false-positive marks), whereas higher sphericity values had the opposite effect. CAD marks were displayed in both two-dimensional and three-dimensional visualisation modes. At each sphericity value the observer counted the number of CAD marks present on each individual acquisition, dividing these into true-positive and false-positive prompts. A true-positive prompt was defined as a CAD mark directly overlapping any part of the visible tumour outline. A false-positive prompt was defined as any other CAD mark within the colon, including those related to polyps (since the primary outcome measure for the present study was cancer). Any uncertainty was resolved by face-to-face discussion with the chief investigator (a radiologist with prior experience of >1000 CTC interpretations). The observer also made a simple subjective assessment of bowel cleansing: excellent, adequate or poor. “Excellent” was used when there was no residue, “poor” when the observer considered a repeat or alternative test would be necessary if polyps of 1 cm or larger were the primary diagnostic target, and “adequate” for those lying between these two definitions.
Raw frequencies for true- and false-positive prompts at each sphericity value were calculated, split by acquisition (prone/supine), and counts determined.
Results
Overall, CAD correctly identified 124 (90.5%) of the 137 cancers at a sphericity of 0; 122 (89.1%) at a sphericity of 50; 119 (86.9%) at a sphericity of 75; and 102 (74.5%) at a sphericity of 100 (Figure 1).
Figure 1.
Graph displaying the number of true-positive (TP) computer-aided detection (CAD) prompts per patient in 137 patients with known colorectal cancer when applied at 4 different CAD sphericity values (0, 50, 75, 100). Horizontal bars represent mean values.
The number of cancers detected by CAD on the 125 patients with paired prone and supine data sets is shown in Table 2. As expected, the sensitivity for cancer detection increased when the algorithm sphericity was decreased, ranging from 93.6 at a sphericity value of 0 to 76.0 at a sphericity value of 100. The mean number of CAD prompts per cancer detected for both the prone and supine studies combined was 8.4 (range 1–46) at a sphericity of 0; 8.4 (range 1–46) at a sphericity of 50; 7.0 (range 1–41) at a sphericity of 75; and 5.6 (range 1–34) at a sphericity of 100. The median length of cancers by CTC was 38 mm (mean 38.6 mm), range 7–100 mm. Measurements from cancers detected by CAD (mean 40.6 mm, range 8–100 mm) and not detected (mean 42.6 mm, range 7–75 mm) were similar.
Table 2. Number (percentage) of 125 cancers correctly identified by computer-aided detection (CAD) on both the prone and supine CT colonography acquisition, or on either acquisition alone.
| CAD sphericity value |
||||
| Cancer detection | 0 | 50 | 75 | 100 |
| Detected on prone and supine acquisition | 71 (56.8) | 71 (56.8) | 63 (50.4) | 47 (37.6) |
| Detected on prone acquisition only | 33 (26.4) | 34 (27.2) | 38 (30.4) | 37 (29.6) |
| Detected on supine acquisition only | 13 (10.4) | 10 (8.0) | 10 (8.0) | 11 (8.8) |
| Not detected on either acquisition | 8 (6.4) | 10 (8.0) | 14 (11.2) | 30 (24.0) |
For patients with paired prone and supine acquisitions, a substantial proportion of cancers were detected by CAD on only one of these. Indeed, at a sphericity setting of 100, the majority of cancers detected were only prompted on a single acquisition, emphasising the need for combined prone and supine series to optimise detection. Of the cancers detected by CAD, 39.3% (46 of 117) were detected on either the prone or supine series at a sphericity of 0; 38.3% (44 of 115) were detected at a sphericity of 50; 43.2% (48 of 111) were detected at a sphericity of 75; and 50.5% (48 of 95) were detected at a sphericity of 100. The majority of those cancers detected by CAD on a single series were identified on the prone rather than the supine acquisition; 71.7%, 77.3%, 79.2% and 77.1% at a sphericity of 0, 50, 75 and 100, respectively (Table 2).
7 (58.3%) cancers were detected in the 12 patients who had undergone a single supine acquisition. All of these were detected at all sphericity values. The mean number and range of CAD prompts per cancer detected was 4.9 (range 2–11) at a sphericity of 0; 4.6 (range 2–11) at a sphericity of 50; 4.1 (range 2–9) at a sphericity of 75; and 3.1 (range 1–5) at a sphericity of 100.
The mean number of false-positive prompts for the 125 patients with matched prone and supine acquisitions was heavily dependent on both the CAD sphericity value and the perceived quality of the bowel preparation, with lower sphericity and poorer preparation both accounting for increased numbers of false-positive results (Table 3).
Table 3. Mean number (range) of false-positive computer-aided detection (CAD) prompts in 125 patients with both prone and supine acquisitions according to the CAD sphericity value and subjective assessment of bowel preparation.
| CAD sphericity value |
||||
| Bowel preparation | 0 | 50 | 75 | 100 |
| Good (43 patients) | 49.4 (14–115) | 40 (9–101) | 25.5 (2–69) | 12.4 (0–37) |
| Adequate (72 patients) | 70.4 (11–541) | 62.4 (9–513) | 47.8 (7–485) | 26.6 (2–287) |
| Poor (10 patients) | 156.7 (53–380) | 142.8 (48–348) | 100.8 (29–245) | 46.2 (7–115) |
Five of the cancers had been missed by the reporting radiologist at the time of the original clinical report. 3 (60%) of these were detected by CAD at all sphericity settings (Figure 2). The remaining two cancers were not identified by CAD at any sphericity value.
Figure 2.
65-year-old male patient with subsequently proven colorectal cancer. The cancer was missed by the radiologist observer on the initial interpretation of the CT colonography examination but was detected by computer-aided detection (CAD) on both the prone (a) and supine (b) acquisitions. Images show two-dimensional and three-dimensional rendering with CAD applied at a sphericity of 75 and prompts displayed as red squares.
Discussion
At the time of writing very little has been published that specifically investigates the utility of CAD for the detection of colorectal cancer. Näppi and co-workers [20] examined the technical problems posed by cancer detection as opposed to polyps, and developed an algorithm based on fuzzy-merging and wall thickness analysis that detected 13 of 14 cancers successfully. They used an internal validation paradigm to assess the algorithm, i.e. the algorithm was developed and validated using the same data. This approach presents a relatively weak challenge to the software and external validation paradigms that are generally preferred [17,21]. Taylor and colleagues [22] used such an external validation paradigm to investigate CAD performance in 24 patients with early (T1) colorectal cancers that were morphologically flat, finding that CAD identified 20 (83.3%) of the tumours. However, flat cancers are not typical of cancers encountered in day-to-day symptomatic practice, which are usually raised.
Our study investigated CAD performance in a consecutive, unselected series of patients with colorectal cancer, in order to replicate as closely as possible the morphology of cancers encountered in general symptomatic practice. None of the data had been used to develop the algorithm previously so that CAD performance is likely to reflect that achievable in day-to-day clinical practice. We found that CAD was able to detect 86.9% (119 of 137) of cancers at the manufacturer's generally recommended (for polyps) sphericity setting of 75. In common with all CAD algorithms currently available for CTC, the product we tested had not been developed specifically to detect colorectal cancer, so these results are encouraging but serendipitous. We achieved sensitivity as high as 93.6 using a sphericity setting of 0 in patients in whom both prone and supine series had been obtained.
However, the false-positive rate is likely to be problematic at such sensitive settings. While recent work has shown that low specificity is not a problem in low-prevalence screening populations, because experienced observers are able to dismiss CAD false-positive results quickly [23], a similar situation cannot be assumed for symptomatic work. For example, symptomatic patients are generally older and may be frail, factors that may increase the prevalence of suboptimal bowel preparation. Supporting this hypothesis, we found that bowel preparation was judged by our observer as “good” in only 43 (31.4%) of the patients studied. Such patients generated a mean number of 25.5 false-positive prompts for the prone and supine acquisitions combined, whereas those with preparation judged only to be “adequate” generated 47.8 per patient. It could be argued that suboptimal bowel preparation is less of a problem when the primary diagnostic target is established cancer, because cancers are generally much larger than polyps and less likely to be masked by retained residue. However, the utility of CAD may be diminished by a high false-positive rate. We observed the largest numbers of false-positive prompts in patients whose preparation was judged to be adequate but in whom there were large amounts of retained fine particulate matter. Such residue does not necessarily impair visualisation of cancers by observers but does pose considerable problems for CAD algorithms.
It should be noted that for the purposes of this study we chose to classify CAD prompts on adenomatous polyps as false-positive since identification of cancer was our primary end point. Approximately 30% of patients aged 60 years or older will have adenomatous polyps (the vast majority of which will be small), and such polyps do not cause symptoms [18]. Their detection in older symptomatic patients has different management implications from detection in younger, asymptomatic individuals. For example, older patients have more comorbidity and are more at risk from colonoscopy-related adverse events and perforation during polypectomy. Also, adenomas have less opportunity to become malignant since older patients have a reduced life span. The result is that adenomas may not need be removed when detected in older patients. In contrast, colorectal cancer is always treated, even if only palliatively.
An important outcome of our study is the finding that a substantial proportion of cancers were not detected by CAD on one of the two acquisitions in which both prone and supine studies had been performed. This finding emphasises again that both prone and supine acquisitions should be acquired to optimise detection [24], and our data strongly suggest that this applies as much to CAD as to the unaided observer. When cancer was detected by CAD on a single acquisition, this was usually the prone study. Some centres scan frail patients in the supine position only, since this is most comfortable. Our data suggest that this may be a disadvantage with CAD: only 7 of 12 cancers were detected by CAD in patients who had undergone a single supine series. The majority of the cancers studied were either rectal or sigmoid, reflecting the normal distribution of colorectal cancer, and both of these segments are optimally distended with the patient prone.
Our study does have limitations. Our methodology has been used extensively in prior studies of CAD [12,17,21] but can only indirectly assess the impact that CAD might have on radiologist interpretation. We identified three cancers that had been missed during the original clinical interpretation but which were detected by CAD subsequently. While it is tempting to conclude that these cancers would have been diagnosed originally had CAD been used, there is no guarantee that readers will react to CAD prompts appropriately [13], and studies that directly investigate the effect on readers' interpretation in the context of cancer are needed. This type of study is beginning to emerge for polyps [13-16]. It might also be argued that CAD is less valuable for cancer detection because tumours are large and thus easier for unaided observers to detect than polyps. Our data tend to support this hypothesis – only 5 of the 137 cancers were missed by the original reporting radiologist. However, because the clinical consequences of a missed cancer far outweigh those of a missed polyp, and because patients with symptoms of colorectal cancer are common in clinical practice [18], we would argue that CAD is likely to have a positive contribution. A study of consecutive patients with proven colorectal cancer that was powered to detect the difference between assisted and unassisted interpretation using cancers that had been missed initially would be difficult since cancers missed by CTC are uncommon and such a study would have to draw from an unfeasibly large database. However, we would stress that the devastating consequences of a missed cancer argue for any supplementation that might enhance diagnosis until such data are available. For example, in mammography, patients value small gains in sensitivity vastly more than far larger falls in specificity [25]. We chose not to investigate the effect of cancer morphology on CAD detection. Classifying cancers into different morphological groups, “sessile” versus “polypoid” for example, is very difficult in the context of a research study since many tumours share elements of both and interobserver agreement is poor. Nor did we collect data on other tests performed prior to CT colonography or the frailty of individual patients, whose median age was 78 years. It is plausible that CAD performance for cancer would be improved in younger, fitter patients; bowel preparation was judged only “adequate” in most of our study group. Also, the advent of faecal tagging regimes sometimes preferred in the frail will negatively affect CAD performance at present but software developers are currently addressing these issues. Finally, we tested a single CAD vendor, and other systems may behave differently with respect to cancers, although we would expect that the performance of different algorithms is likely to be outweighed by the difference between using and not using CAD.
Conclusion
The CAD algorithm tested was able to detect colorectal cancer in a high proportion of consecutive unselected symptomatic patients referred for CT colonography but must be applied to both the prone and supine acquisitions for optimal results.
Conflict of Interest
S Halligan, G Iinuma and SA Taylor are clinical consultants for the software company whose product is tested in the article (i.e. Medicsight Plc).
Acknowledgments
A proportion of this work was undertaken at UCLH/UCL, which receives funding from the Department of Health's NIHR Comprehensive Biomedical Research Centre funding scheme. The views expressed in this publication are those of the authors and not necessarily those of the UK Department of Health.
References
- 1.Johnson CD, Chen MH, Toledano AY, Heiken JP, Dachman A, Kuo MD, et al. Accuracy of CT colonography for detection of large adenomas and cancers. N Engl J Med 2008;359:1207–17 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Pickhardt PJ, Choi JR, Hwang I, Butler JA, Puckett ML, Hildebrandt HA, et al. Computed tomographic virtual colonoscopy to screen for colorectal neoplasia in asymptomatic adults. N Engl J Med 2003;349:2191–200 [DOI] [PubMed] [Google Scholar]
- 3.Fenlon HM, Nunes DP, Schroy PC, 3rd, Barish MA, Clarke PD, Ferrucci JT. A comparison of virtual and conventional colonoscopy for the detection of colorectal polyps. N Engl J Med 1999;341:1496–503 [DOI] [PubMed] [Google Scholar]
- 4.Reuterskiöld MH, Lasson A, Svensson E, Kilander A, Stotzer PO, Hellström M. Diagnostic performance of computed tomography colonography in symptomatic patients and in patients with increased risk for colorectal disease. Acta Radiol 2006;47:888–9 [DOI] [PubMed] [Google Scholar]
- 5.Laghi A, Iannaccone R, Carbone I, Catalano C, Di Giulio E, Schillaci A, et al. Detection of colorectal lesions with virtual computed tomographic colonography. Am J Surg 2002;183:124–31 [DOI] [PubMed] [Google Scholar]
- 6.Munikrishnan V, Gillams AR, Lees WR, Vaizey CJ, Boulos PB. Prospective study comparing multislice CT colonography with colonoscopy in the detection of colorectal cancer and polyps. Dis Colon Rectum 2003;46:1384–9 [DOI] [PubMed] [Google Scholar]
- 7.Fidler JL, Fletcher JG, Johnson CD, Huprich JE, Barlow JM, Earnest F, 4th, et al. Understanding interpretive errors in radiologists learning computed tomography colonography. Acad Radiol 2004;11:750–6 [DOI] [PubMed] [Google Scholar]
- 8.Dachman AH, Kelly KB, Zintsmaster MP, Rana R, Khankari S, Novak JD, et al. Formative evaluation of standardized training for CT colonographic image interpretation by novice readers. Radiology 2008;249:167–77 [DOI] [PubMed] [Google Scholar]
- 9.Slater A, Taylor SA, Tam E, Gartner L, Scarth J, Peiris C, et al. Reader error during CT colonography: causes and implications for training. Eur Radiol 2006;16:2275–83 [DOI] [PubMed] [Google Scholar]
- 10.European SocietyofGastrointestinalandAbdominalRadiology CT., Colonography Group Investigators Effect of directed training on reader performance for CT colonography: multicentre study. Radiology 2007;242:152–6 [DOI] [PubMed] [Google Scholar]
- 11.Yoshida H, Dachman AH. CAD techniques, challenges, and controversies in computed tomographic colonography. Abdom Imaging 2005;30:26–41 [DOI] [PubMed] [Google Scholar]
- 12.Summers RM, Yao J, Pickhardt PJ, Franaszek M, Bitter I, Brickman D, et al. Computed tomographic virtual colonoscopy computer-aided polyp detection in a screening population. Gastroenterology 2005;129:1832–4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Halligan S, Altman DG, Mallett S, Taylor SA, Burling D, Roddie M, et al. Computed tomographic colonography: assessment of radiologist performance with and without computer-aided detection. Gastroenterology 2006;131:1690–9 [DOI] [PubMed] [Google Scholar]
- 14.Taylor SA, Charman SC, Lefere P, McFarland EG, Paulson EK, Yee J, et al. CT colonography: investigation of the optimum reader paradigm by using computer-aided detection software. Radiology 2008;246:463–71 [DOI] [PubMed] [Google Scholar]
- 15.Mang T, Peloschek P, Plank C, Maier A, Graser A, Weber M, et al. Effect of computer-aided detection as a second reader in multidetector row CT colonography. Eur Radiol 2007;17:2598–607 [DOI] [PubMed] [Google Scholar]
- 16.Petrick N, Haider M, Summers RM, Yeshwant SC, Brown L, Iuliano EM, et al. CT colonography with computer-aided detection as a second reader: observer performance study. Radiology 2008;246:148–56 [DOI] [PubMed] [Google Scholar]
- 17.Summers RM, Handwerker LR, Pickhardt PJ, Van Uitert RL, Deshpande KK, Yeshwant S, et al. Performance of a previously validated CT colonography computer-aided detection system in a new patient population. AJR Am J Roentgenol 2008;191:168–74 [DOI] [PubMed] [Google Scholar]
- 18.Halligan S, Lilford RJ, Wardle J, Morton D, Rogers P, Wooldrage K, et al. Design of a multicentre randomized trial to evaluate CT colonography versus colonoscopy or barium enema for diagnosis of colonic cancer in older symptomatic patients: the SIGGAR study. Trials 2007;8:32. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Barish MA, Soto JA, Ferrucci JT. Consensus on current clinical practice of virtual colonoscopy. AJR Am J Roentgenol 2005;184:786–9 [DOI] [PubMed] [Google Scholar]
- 20.Näppi JJ, Frimmel H, Dachman AH, Yoshida H. Computerized detection of colorectal masses in CT colonography based on fuzzy merging and wall-thickening analysis. Med Phys 2004;31:860–7 [DOI] [PubMed] [Google Scholar]
- 21.Halligan S, Taylor SA, Dehmeshki J, Amin H, Ye X, Tsang J, et al. Computer-assisted detection for CT colonography: external validation. Clin Radiol 2006;61:758–63 [DOI] [PubMed] [Google Scholar]
- 22.Taylor SA, Iinuma G, Saito Y, Zhang J, Halligan S. CT colonography: computer-aided detection of morphologically flat T1 colonic carcinoma. Eur Radiol 2008;18:1666–73 [DOI] [PubMed] [Google Scholar]
- 23.Taylor SA, Greenhalgh R, Ilangovan R, Tam E, Sahni VA, Burling D, et al. CT colonography and computer-aided detection: effect of false-positive results on reader specificity and reading efficiency in a low-prevalence screening population. Radiology 2008;247:133–40 [DOI] [PubMed] [Google Scholar]
- 24.Yee J, Kumar NN, Hung RK, Akerkar GA, Kumar PR, Wall SD. Comparison of supine and prone scanning separately and in combination at CT colonography. Radiology 2003;226:653–61 [DOI] [PubMed] [Google Scholar]
- 25.Schwartz LM, Woloshin S, Sox HC, Fischhoff B, Welch HG. US women's attitudes to false-positive mammography results and detection of ductal carcinoma in situ: cross-sectional survey. West J Med 2000;173:307–12 [DOI] [PMC free article] [PubMed] [Google Scholar]