Abstract
Objective
To assess whether the performance of a computer-assisted detection (CAD) algorithm for acute pulmonary embolism (PE) differs in pulmonary CT angiographies acquired at various institutions.
Methods
In this retrospective study, we included 40 consecutive scans with and 40 without PE from 3 institutions (n=240) using 64-slice scanners made by different manufacturers (General Electric; Philips; Siemens). CAD markers were classified as true or false positive (FP) using independent evaluation by two readers and consultation of a third chest radiologist in discordant cases. Image quality parameters were subjectively scored using 4/5-point scales. Image noise and vascular enhancement were measured. Statistical analysis was done to correlate image quality of the three institutions with CAD stand-alone performance.
Results
Patient groups were comparable with respect to age (p=0.22), accompanying lung disease (p=0.12) and inpatient/outpatient ratio (p=0.67). The sensitivity was 100% (34/34), 97% (37/38) and 92% (33/36), and the specificity was 18% (8/44), 15% (6/41) and 13% (5/39). Neither significantly differed between the institutions (p=0.21 and p=0.820, respectively). The mean number of FP findings (4.5, 6.2 and 3.7) significantly varied (p=0.02 and p=0.03), but median numbers (2, 3 and 3) were comparable. Image quality parameters were significantly associated with the number of FP findings (p<0.05) but not with sensitivity. After correcting for noise and vascular enhancement, the number of FPs did not significantly differ between the three institutions (p=0.43).
Conclusions
CAD stand-alone performance is independent of scanner type but strongly related to image quality and thus scanning protocols.
Multidetector-row CT (MDCT) has become the first-line diagnostic imaging modality for pulmonary embolism (PE) at most institutions [1-4]. With the technical evolution of MDCT, an increasing number of ever-thinner sections are generated that have to be systematically scrutinised for the presence of emboli in pulmonary arteries. This is a challenging task, especially for smaller and more peripheral pulmonary vessels. As a result, the detection of subsegmental small emboli has a variable interobserver agreement. One study using a 16-slice MDCT scanner reported κ-values ranging from 0.56 to 0.85 [5].
The aims of automated computer-assisted detection (CAD) software for pulmonary emboli are to decrease perception errors, to reduce the workload and speed up evaluation, and to make reader performance less dependent on their level of skill or training. The majority of the previously published studies have tested the stand-alone performance of various CAD algorithms [6-13]. Up to now there have been four studies that tested the influence of CAD on radiologists' detection performance [8,14-16]. However, all studies so far tested the CAD on CT pulmonary angiograms (CTPAs) from a single institution.
It is likely that the quality of the CTPA influences the performance of the CAD algorithm. Poorly timed contrast, motion artefacts, noise and the presence of accompanying lung diseases lead to higher numbers of false-positive (FP) findings [6,7]. Because image quality depends on CT protocol parameters, the CT manufacturer and patient instructions, it can be expected that the performance of a CAD algorithm might differ between institutions. The purpose of this study was therefore to assess whether the performance of a CAD algorithm for acute pulmonary embolism differs in CTPAs acquired at various institutions.
Methods and materials
Patient selection
In this institutional review board-approved study we retrospectively included 240 consecutive 64-slice CTPA scans from 3 institutions that use CT scanners by different manufacturers (Brilliance-64, Philips Medical Systems, Cleveland, OH; GE LightSpeed Volume CT 64, Waukesha, WI; Somatom Sensation 64, Siemens Medical Solutions, Forchheim, Germany). The first 40 consecutive scans with PE and 40 consecutive scans without PE were selected per institution between January and October 2008. Presence or absence of PE was decided based on their original radiology reports. All CTPA studies were obtained with the use of manufacturer-specific dose modulation software and local CT protocols (Table 1).
Table 1. CT protocols.
| Scan parameters | Site A | Site B | Site C |
| Scan direction | Craniocaudal | Caudocranial | Craniocaudal |
| kVp | 120 | 120 | 100 |
| Slice collimation (mm) | 64×0.625 | 64×0.625 | 24×1.2 |
| Rotation time (s) | 0.4 | 0.4 | 0.37 |
| Pitch | 0.984 | 1.172 | 0.75 |
| mAs | 136 | 100 | 135 |
| Contrast volume (ml) | 70 | 90 | 40–70a |
| Concentration (iodine ml–1) | 320 | 300 | 300 |
| Flow (ml s–1) | 5 | 5 | 6 |
| NaCl chaser (ml) | 20 | 30 | 30 |
| Delay(s)/threshold to trigger (HU) | 3 | 8/150 | 5–12 |
| Reconstructed slice thickness (mm) | 0.625 | 0.9 | 1.5 |
| Reconstruction interval (mm) | 0.5 | 0.45 | 1.0 |
aWeight adapted.
In total, eight patients had to be excluded: in four patients the lung segmentation was misled by air in abnormal locations, leading to failure of the CAD algorithm; in two patients there were massive streak artefacts due to improper arm positions; one patient had chronic PE; and one patient had a thrombus solely in the main artery, which is not included in the CAD analysis. This resulted in inclusion of 38, 39 and 38 positive CTPAs and 40, 40 and 37 negative CTPAs per site, respectively, according to the original reports.
Data analysis
All examinations were analysed by prototype CAD software (Philips Healthcare, Best, The Netherlands). After fully automatic segmentation of the lung and vessels [17,18], the algorithm looks for contrast differences in both pulmonary arteries and veins >2 mm and marks the difference if it exceeds a threshold of 150 HU. The main pulmonary arteries are not included in the CAD analysis. Candidate lesions detected by the CAD are indicated by a region of interest (ROI) and presented to the observer on demand. Processing time takes about 30 s per examination.
The reference standard was established by consensus of at least two of three readings. All CT scans were independently evaluated by a researcher specially trained in reading PE studies (>300 exams; RW) and a chest radiologist (>15 years of experience; CMS-P). The readers first evaluated all data sets without CAD and subsequently the CAD results. They reported the presence and anatomical locations of thrombi independently from each other without knowledge of the original reports. In case of discordant findings between these two readers, and/or with the original report, a third experienced chest radiologist (>15 years of experience; MP) was consulted as an arbiter. The anatomical level of the thrombus was annotated as central, lobar, segmental or subsegmental according to its proximal end.
The CAD findings were compared with this reference standard and classified as true positive or FP. We differentiated central locations in main and lobar arteries from peripheral locations in segmental and subsegmental arteries.
Underlying reasons for FP CAD markers were classified as related to anatomical structures (veins, lymphatic tissue and intrapulmonary opacities) or to motion artefacts and low vascular enhancement (Figures 1 and 2).
Figure 1.
False-positive finding found by computer-assisted detection located in a vein.
Figure 2.
False-positive finding found by computer-assisted detection due to low vascular enhancement.
Image noise was measured in the descending aorta at the level of the bifurcation of the trachea as the standard deviation of CT numbers using a standardised ROI of 1 cm2. Noise per scan was calculated as the average over three measurements.
To evaluate the vascular enhancement, we measured the mean CT number in Hounsfield units (HU) using individual ROIs at the level of the central pulmonary artery (ROI with 2 cm diameter) and the segmental (3 mm) and the subsegmental arteries (1 mm) in the left upper and lower lobe. The enhancement in the subsegmental artery in the upper lobe was measured at the level of the aortic arch, and in the lower lobe between the inferior pulmonary veins and just above the diaphragm. In each artery we measured the enhancement three times and calculated the average. If for any reason (e.g. embolus, underlying lung disease, motion artefacts) the enhancement in the left lung could not be measured, equivalent arteries in the right lung were selected.
The quality of the CT examinations was subjectively scored by a researcher (RW) using a 4- and 5-point scale with respect to the overall quality, the presence of motion artefacts and the effects of accompanying lung disease (Table 2).
Table 2. Subjective scores of image quality characteristics.
| Scale and score | Description |
| Overall image quality | |
| 1 | Inadequate, no diagnosis of PE possible |
| 2 | Low, diagnosis possible until the lobar level |
| 3 | Sufficient, diagnosis possible until the segmental level |
| 4 | Good, diagnosis possible until the subsegmental level |
| 5 | Excellent |
| Motion artefacts | |
| 1 | Massive, no diagnosis possible |
| 2 | Definite, establishment of diagnosis impeded |
| 3 | Moderate, but image sufficient for diagnosis of PE |
| 4 | Minor |
| 5 | None |
| Accompanying lung disease | |
| 1 | Present, lung disease disturbing for CAD and radiologist |
| 2 | Present, lung disease disturbing for CAD |
| 3 | Present, but no influence |
| 4 | None |
CAD, computer-assisted detection; PE, pulmonary embolism.
Statistical analysis
Statistical analysis was performed with SPSS (version 15.0; SPSS Inc., Chicago, IL). For all tests a p-value <0.05 was considered significant.
Differences in patient groups were assessed using a χ2 test with respect to sex and inpatient/outpatient ratio, and an analysis of variance (ANOVA) test with respect to patient age.
Sensitivity and specificity for the presence of PE were calculated on per-patient basis separately for each institution. Furthermore, the sensitivity was calculated on a per-lesion basis. A Fisher–Freeman–Halton test was used to assess differences between the three institutions.
We used factor analysis and Cronbach's α statistics to assess the variability between the vascular enhancement measured at the central, lobar, segmental and subsegmental level. An ANOVA followed by a Hochberg's GT2 post hoc test were used to test for significance of difference with respect to the vascular enhancement and noise between the three institutions.
A Kruskal–Wallis test followed by a post hoc Shaffer-corrected Mann–Whitney U-test was used to test for significance of difference between the three institutions with respect to overall quality, motion artefacts, presence of accompanying lung disease and number of FP findings. An analysis of covariance (ANCOVA) with noise and vascular enhancement as covariates was performed to establish the relation between scanner type and number of FP findings.
To assess the correlation between sensitivity per scan and noise or between vascular enhancement and overall image quality, a Pearson's correlation or Spearman's rank correlation test was used, respectively. To assess the correlation between the various image quality parameters and the number of FPs, a multiple linear regression analysis was applied.
Results
Study groups
The three patient groups did not significantly differ with respect to age (p=0.220) and inpatient/outpatient ratio (p=0.674; Table 3). The reference standard differed from the original reports in nine patients: in one patient, originally reported as negative, the standard determined as positive; and in eight patients, originally reported as positive, the standard determined as negative. Thus, the reference standard for the 3 institutions rated 34, 38 and 36 scans positive for PE, and 44, 41 and 39 scans negative for PE, respectively.
Table 3. Results: patient group characteristics.
| Group characteristics | Site A | Site B | Site C |
| Mean age (range) (years) | 58 (18–88) | 61 (27–93) | 63 (11–91) |
| Inpatients (%) | 51 | 58 | 56 |
| Sex, female:male (%) | 68:32 | 48:52 | 47:53 |
| Number of positive scans | 34 | 38 | 36 |
| Number of negative scans | 44 | 41 | 39 |
There were on average 6 (range 1–18) thrombi per patient in the first institution (hereafter Site A), 5 (range 1–18) thrombi per patient in the second institution (hereafter Site B) and 4 (range 1–17) thrombi per patient in the third institution (hereafter Site C).
Sensitivity and specificity
The sensitivity on a per-patient basis was not significantly different between the three institutions, with 100% (34/34), 97% (37/38) and 92% (33/36), respectively (p=0.21).
The sensitivity on per-lesion basis was significantly different, with 76% (165/216), 75% (146/194) and 64% (84/132), respectively (p=0.025). CAD found in total 16 out of the 17 patients (94%) with only isolated subsegmental emboli.
The specificity of CAD on a per-patient basis was not significantly different between the three institutions, with 18% (8/44), 15% (6/41) and 13% (5/39), respectively (p=0.820; Table 4).
Table 4. Results: computer-assisted detection performance for the three different sites.
| Computer-assisted performance | Site A | Site B | Site C |
| Sensitivity on a per-patient basis (%) | 100 | 97 | 92 |
| Sensitivity on a per-lesion basis (%) | 76 | 75 | 64 |
| Specificity | 18 | 15 | 13 |
Analysis of false positives
The mean number of FP CAD findings per patient was 4.5 (median 2, range 0–29), 3.7 (median 3, range 0–20) and 6.2 (median 3, range 0–23), respectively, with the last being significantly different from the other 2 (p=0.021 and p=0.03). In most scans (63–75%) CAD found 5 or fewer FP candidates.
After correcting for differences of noise and vascular enhancement using an ANCOVA analysis, the mean number of FP findings per patient did not significantly differ between the three institutions (p=0.425).
In all institutions most of the FP findings were located in veins or intrapulmonary opacities (Table 5).
Table 5. Results: analysis of false-positive computer-assisted detection findings.
| Findings | Site A | Site B | Site C |
| Mean number of FPs | 4.5 | 6.2 | 3.7 |
| Median number of FPs | 2 | 3 | 3 |
| ≤5 CAD findings per scan (%) | 72 | 63 | 75 |
| >10 CAD findings per scan (%) | 17 | 22 | 8 |
| Veins as FP finding (% of total FP) | 27 | 34 | 37 |
| Intrapulmonary opacifications as FP finding (% of total FP) | 26 | 17 | 29 |
FP, false positive; CAD, computer-assisted detection.
Vascular enhancement and noise
A factor analysis, performed for each hospital separately, revealed a high correlation between the mean vascular enhancement of the central, segmental and subsegmental arteries allowing for calculation of a single average enhancement measure per institution. This amounted to 384 HU, 266 HU and 429 HU, respectively (Table 6; Figure 2), with all differences being significant at pairwise comparisons (p<0.001 to p=0.039).
Table 6. Results: quality parameters for the three different sites.
| Quality parameters | Site A | Site B | Site C |
| Vascular enhancement (HU) | 384 | 266 | 429 |
| Noise (HU) | 34.4 | 30.2 | 34 |
| Overall quality (scale 1–5a) | 4.3 | 3.9 | 4.4 |
| Accompanying lung disease (scale 1–4a) | 3 | 2.8 | 2.8 |
| Motion artefacts (scale 1–4a) | 3.1 | 3 | 2.9 |
aThe higher the score, the better the image quality and the lower the presence and influence of overall quality, motion artefacts and accompanying lung disease.
There was a significant negative correlation between the number of FPs and vascular enhancement in two sites (p=0.023 and p=0.046), but no significant correlation between vascular enhancement and the sensitivity per scan (p=0.872).
The mean noise amounted to 34.4 HU, 30.2 HU and 34 HU, respectively (Table 6), with the differences between the lowest noise level and the remaining two being significant at p=0.001 and p=0.004.
There was a significant positive correlation between the number of FPs and the noise in two sites (p=0.029 and p=0.012), but no significant difference between the noise and the sensitivity per scan (p=0.726).
Overall quality, motion artefacts and accompanying lung disease
There were no significant differences between the three sites with respect to the presence and effects of accompanying lung diseases (p=0.123) and the severity of motion artefacts (p=0.356). There was a significant positive correlation between the number of FPs and the severity of motion artefacts in two sites (multiple regression analysis, p=0.047 and p=0.002). The overall quality was significantly lower at Site B than at Sites A (p=0.011) and C (p=0.002; Table 6). There was a significant correlation between the number of FPs and the overall quality in two sites (p=0.001 and p=0.001), but no significant difference between the overall quality and the sensitivity per scan (p=0.647).
Discussion
There is little known about the performance of CAD software when used in different institutions and whether results made in one site can be transferred to another site. In addition to patient-related factors such as concomitant lung disease or motion artefacts, image quality is influenced by the CT data acquisition protocol, which may vary per institution. We therefore investigated the stand-alone performance of a CAD prototype in three institutions with respect to sensitivity, specificity and a number of subjectively and objectively scored image parameters. Both performance measures (sensitivity and specificity) were assessed on a per-patient basis and a per-lesion basis. We consider the first the more important measure from a clinical point of view, given the fact that detection of an isolated embolus is sufficient to call a study positive for the presence of pulmonary emboli, and thus to indicate need for treatment. The latter, however, we consider more suited to describing the level of stand-alone performance of this prototype software.
For CAD software to be beneficial in clinical practice as second reader, a high sensitivity appears warranted. In our study we calculated the sensitivity both on a per-lesion basis, as a criterion of the software's performance, and on a per-patient basis, which appears to be clinically more relevant given the current concept that, in haemodynamically stable patients, treatment is based on a yes/no decision for the presence of PE.
In our study, the sensitivity on a per-patient basis varied between 92% and 100%, thus yielding a performance in all three institutions that compares favourably with published data so far, reporting a sensitivity of 53%, 86% and 94%, respectively [8,10,11]. Statistically, the difference did not reach significance, which does not necessarily mean that the observed difference is not noteworthy. The lower sensitivity in 1 of the 3 sites was due to false-negative findings in 3 of the 36 patients found positive for PE. Two patients had emboli in only one location and one patient in three locations. The missed intravascular defects were of various sizes and on various anatomical levels (one lobar, one segmental and three subsegmental). Underlying reasons for the failures included low vascular enhancement in two patients and perivascular pulmonary consolidations in one patient.
On a per-lesion basis the sensitivity differed significantly between the three institutions. However, as opposed to a significant correlation between the number of FP findings and parameters of image quality, we could not find a significant correlation between the sensitivity and parameters of image quality. This was further underlined by the finding that the institution with the lowest sensitivity did not show a lower mean vascular enhancement or an increased noise. We concluded from these findings that the relationship between sensitivity and image quality seems to be more complex, also involving factors relating to the locations and multitude of the emboli. This finding conforms with previously reported results by Dewailly et al [13], who also could not find an effect of overall image quality or different scanning conditions on the detection rate of peripheral clots of a prototype CAD that was different from the one we tested.
In the entire study group, CAD found 16 out of 17 patients (94%) with isolated subsegmental emboli. This is noteworthy, because usually radiologists securely detect the rather obvious central or lobar emboli, but may miss isolated subsegmental emboli. It has to be noted, however, that from a clinical point of view the significance of isolated subsegmental emboli is still uncertain. In the literature it is suggested that the clinical relevance of subsegmental emboli is larger in individuals with cardiopulmonary restriction and that subsegmental emboli may predict a more severe embolic disease in the future or the development of pulmonary hypertension [19].
On a patient basis, we found generally low specificities of 18%, 15% and 13%. For two of the three institutions we found a significant correlation between the number of FP findings and the overall quality, vascular enhancement, noise and motion artefacts. For the other institution, the numbers of examinations with low vascular enhancement or high image noise were too small to prove a statistically significant correlation with the number of FP findings. However, this does not mean that a correlation exists also in this institution.
On average, CAD showed 4.5, 6.2 and 3.7 FP findings per patient, with the difference being significant for the site with the highest mean number of FPs. Although the mean numbers of FP findings differed, the median numbers of FPs were comparable and amounted to 2, 3 and 3. This suggests that a subgroup of examinations with exceedingly high numbers of FP candidates were responsible for the significant increase of the mean FP findings and not a generally lower image quality of all examinations. If we considered a maximum of five FP candidates per scan an acceptable threshold for clinical application, this criterion was fulfilled in 72%, 63% and 75% of examinations. In all three institutions, most FP candidates were localised in veins. It has to be noted that all FP candidates located in veins were classified as such based on the misinterpretation of the anatomical location and irrespective of other aspects such as inhomogeneous vascular enhancement, a contrast difference against a surrounding pulmonary consolidation or a motion artefact contributing to the misinterpretation of the CAD algorithm. It is therefore likely that multiple conditions have contributed to the high number of FP CAD candidates in veins. Simply adjusting the protocol to achieve high venous contrast would only partially solve the problem. Teaching the algorithm to differentiate between veins and arteries would seem to be a more promising approach. This is an important aspect considering that too high a number of FP findings will result in prolongation of reading time and may decrease confidence in true-positive findings of the CAD. Therefore further lowering the number of FPs, especially in examinations of lower image quality, seems warranted before applying CAD into clinical routine.
It is important to note that after ANCOVA analysis and correcting for noise and vascular enhancement, we did not find a significant difference between the institutions with respect to the number of FP candidates. We concluded from this that only image quality parameters (and thus CT protocols) are responsible for the differences in the number of FP CAD candidates between the institutions, and not the scanner type used.
Our study has the following limitations. Per institution, we consecutively included the first 40 scans reported positive for PE and the first 40 scans reported negative for PE. As intended, by this procedure scans were included irrespective of image quality, time of acquisition or patient condition. We chose almost equal numbers of positive and negative scans to get a realistic estimation of the performance of the CAD software in both groups of examinations. The stand-alone performance of the CAD was determined by a reference that had been established by independent readings of two radiologists, and in cases of discordance by assessment of a third experienced radiologist. Although this reference has to be considered as not optimal, it reflects clinical applicability.
In summary, the CAD prototype yields a comparable stand-alone performance with different scanner types and various CTPA protocols if image quality matches.
We found a significant correlation between image quality parameters and the number of FPs, suggesting that an optimisation of the protocol with respect to vascular enhancement, noise and the avoidance of breathing artefacts is useful to increase the specificity of the CAD. After correcting for image quality parameters, the scanner type had no significant influence on the number of FP candidates. Although high on patient basis in all three institutions, with numbers between 92% and 100%, significant differences in sensitivity existed between the three institutions on a per-lesion basis. These differences appeared to be influenced not only by image quality but also by a combination of patient-related aspects, for which further analysis is needed to determine magnitude and clinical relevance.
Acknowledgment
Research grant by Philips Healthcare, Best, the Netherlands.
References
- 1.British Thoracic Society guidelines for the management of suspected acute pulmonary embolism. Thorax 2003;58:470–83 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Cronin P, Weg JG, Kazerooni EA. The role of multidetector computed tomography angiography for the diagnosis of pulmonary embolism. Semin Nucl Med 2008;38:418–31 [DOI] [PubMed] [Google Scholar]
- 3.Remy-Jardin M, Pistolesi M, Goodman LR, Gefter WB, Gottschalk A, Mayo JR, et al. Management of suspected acute pulmonary embolism in the era of CT angiography: a statement from the Fleischner Society. Radiology 2007;245:315–29 [DOI] [PubMed] [Google Scholar]
- 4.Schaefer-Prokop C, Prokop M. MDCT for the diagnosis of acute pulmonary embolism. Eur Radiol 2005;15(Suppl 4):D37–41 [DOI] [PubMed] [Google Scholar]
- 5.Brunot S, Corneloup O, Latrabe V, Montaudon M, Laurent F. Reproducibility of multi-detector spiral computed tomography in detection of sub-segmental acute pulmonary embolism. Eur Radiol 2005;15:2057–63 [DOI] [PubMed] [Google Scholar]
- 6.Buhmann S, Herzog P, Liang J, Wolf M, Salganicoff M, Kirchhoff C, et al. Clinical evaluation of a computer-aided diagnosis (CAD) prototype for the detection of pulmonary embolism. Acad Radiol 2007;14:651–8 [DOI] [PubMed] [Google Scholar]
- 7.Schoepf UJ, Schneider AC, Das M, Wood SA, Cheema JI, Costello P. Pulmonary embolism: computer-aided detection at multidetector row spiral computed tomography. J Thorac Imaging 2007;22:319–23 [DOI] [PubMed] [Google Scholar]
- 8.Walsham AC, Roberts HC, Kashani HM, Mongiardi CN, Ng YL, Patsios DA. The use of computer-aided detection for the assessment of pulmonary arterial filling defects at computed tomographic angiography. J Comput Assist Tomogr 2008;32:913–18 [DOI] [PubMed] [Google Scholar]
- 9.Zhou C, Chan HP, Patel S, Cascade PN, Sahiner B, Hadjiiski LM, et al. Preliminary investigation of computer-aided detection of pulmonary embolism in three-dimensional computed tomography pulmonary angiography images. Acad Radiol 2005;12:782–92 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Maizlin ZV, Vos PM, Godoy MC, Cooperberg PL. Computer-aided detection of pulmonary embolism on CT angiography: initial experience. J Thorac Imaging 2007;22:324–9 [DOI] [PubMed] [Google Scholar]
- 11.Wittenberg R, Peters JF, Sonnemans JJ, Prokop M, Schaefer-Prokop CM. Computer-assisted detection of pulmonary embolism: evaluation of pulmonary CT angiograms performed in an on-call setting. Eur Radiol 2009;20:801–6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Masutani Y, MacMahon H, Doi K. Computerized detection of pulmonary embolism in spiral CT angiography based on volumetric image analysis. IEEE Trans Med Imaging 2002;21:1517–23 [DOI] [PubMed] [Google Scholar]
- 13.Dewailly M, Remy-Jardin M, Duhamel A, Faivre JB, Pontana F, Deken V, et al. Computer-aided detection of acute pulmonary embolism with 64-slice multi-detector row computed tomography: impact of the scanning conditions and overall image quality in the detection of peripheral clots. J Comput Assist Tomogr 2010;34:23–30 [DOI] [PubMed] [Google Scholar]
- 14.Das M, Mühlenbruch G, Helm A, Bakai A, Salganicoff M, Stanzel S, et al. Computer-aided detection of pulmonary embolism: influence on radiologists' detection performance with respect to vessel segments. Eur Radiol 2008;18:1350–5 [DOI] [PubMed] [Google Scholar]
- 15.Engelke C, Schmidt S, Bakai A, Auer F, Marten K. Computer-assisted detection of pulmonary embolism: performance evaluation in consensus with experienced and inexperienced chest radiologists. Eur Radiol 2008;18:298–307 [DOI] [PubMed] [Google Scholar]
- 16.Engelke C, Schmidt S, Auer F, Rummeny EJ, Marten K. Does computer-assisted detection of pulmonary emboli enhance severity assessment and risk stratification in acute pulmonary embolism? Clin Radiol 2010;65:137–44 [DOI] [PubMed] [Google Scholar]
- 17.Buelow T, Wiemker R, Blaffert T, Lorenz C, Renisch S. Automatic extraction of the pulmonary artery tree from multi-slice CT data. Spie 2005;5746:730–40 [Google Scholar]
- 18.Bouma H, Sonnemans JJ, Vilanova A, Gerritsen FA. Automatic detection of pulmonary embolism in CTA images. IEEE Trans Med Imaging 2009;28:1223–30 [DOI] [PubMed] [Google Scholar]
- 19.Schoepf UJ, Costello P. CT angiography for diagnosis of pulmonary embolism: state of the art. Radiology 2004;230:329–37 [DOI] [PubMed] [Google Scholar]