Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 May 1.
Published in final edited form as: Eur Radiol. 2016 Sep 5;27(5):2055–2066. doi: 10.1007/s00330-016-4571-4

Prospective Evaluation of Reduced Dose Computed Tomography for the Detection of Low-Contrast Liver Lesions

Direct comparison with concurrent standard dose imaging

B Dustin Pooler 1, Meghan G Lubner 1, David H Kim 1, Oliver T Chen 1, Ke Li 1,2, Guang-Hong Chen 1,2, Perry J Pickhardt 1
PMCID: PMC5337451  NIHMSID: NIHMS814799  PMID: 27595834

Abstract

Objectives

To prospectively compare diagnostic performance of reduced-dose (RD) contrast-enhanced CT (CECT) with standard-dose (SD) CECT for detection of low-contrast liver lesions.

Methods

70 adults with non-liver primary malignancies underwent abdominal SD-CECT immediately followed by RD-CECT, aggressively targeted at 60–70% dose reduction. SD series were reconstructed using FBP. RD series were reconstructed with FBP, ASIR, and MBIR (Veo). Three readers—blinded to clinical history and comparison studies—reviewed all series, identifying liver lesions ≥ 4 mm. Non-blinded review by two experienced abdominal radiologists—assessing SD against available clinical and radiologic information—established the reference standard.

Results

RD-CECT mean effective dose was 2.01±1.36 mSv (median, 1.71), a 64.1±8.8% reduction. Pooled per-patient performance data were (sensitivity/specificity/PPV/NPV/accuracy) 0.91/0.78/0.60/0.96/0.81 for SD-FBP, compared with RD-FBP 0.79/0.75/0.54/0.91/0.76; RD-ASIR 0.84/0.75/0.56/0.93/0.78; and RD-MBIR 0.84/0.68/0.49/0.92/0.72. ROC AUC values were 0.896/0.834/0.858/0.854 for SD-FBP/RD-FBP/RD-ASIR/RD-MBIR, respectively. RD-FBP (P=0.002) and RD-MBIR (P=0.032) AUC were significantly lower than SD-FBP; RD-ASIR was not (P=0.052). Reader confidence was lower for all RD series (P<0.001) compared with SD-FBP, especially when calling patients entirely negative.

Conclusions

Aggressive CT dose reduction resulted in inferior diagnostic performance and reader confidence for detection of low-contrast liver lesions compared to SD. Relative to RD-ASIR, RD-FBP showed decreased sensitivity and RD-MBIR showed decreased specificity.

Keywords: Diagnostic Imaging, Multi-detector Computed Tomography, Radiation Dosage, Liver, Metastases

Introduction

Contrast-enhanced computed tomography (CECT) is routinely used for oncologic staging and surveillance of metastatic disease in the chest, abdomen, and pelvis for a wide variety of primary malignancies.[1] Although there was a transient decline in CT utilization related to concerns over radiation dose exposures,[2] CT volumes appear to be trending higher again.[3] Nonetheless, scrutiny surrounding the small theoretical increased risk of malignancy from medical radiation at the population level remains high.[4] The impact of this effect, while controversial in terms of actual harm, has led to the prioritization of CT dose reduction while maintaining diagnostic image quality. Recent research has focused on iterative algorithms designed to reconstruct CT data acquired at reduced doses. Increased availability of commercially-available iterative reconstruction methods that operate in image space, projection (raw data) space, or both, has allowed for moderate dose reductions beyond classic filtered back-projection (FBP).[5] Specific algorithms operating primarily operating in image space include adaptive statistical iterative reconstruction (ASIR, GE Healthcare), adaptive iterative dose reconstruction (AIDR, Toshiba), iDose4 (Phillips), and sinogram affirmed iterative reconstruction and iterative reconstruction in image space (SAFIRE and IRIS, Siemens).[5–9] For ASIR, studies have demonstrated the potential to reduce dose an estimated 25–40%.[6, 9, 10] The emergence of higher-order model-based image reconstruction (MBIR) algorithms, which may operate throughout the system with multiple forward and backward reconstructions, show promise for even more aggressive dose reduction.[5] MBIR algorithms include Veo (GE), advanced model-based iterative reconstruction (ADMIRE, Siemens), iterative model reconstruction (IMR, Philips), and forward projected model-based iterative reconstruction solution (FIRST, Toshiba).[5, 11, 12] Of note, ADMIRE operates in both image and raw-data space, whereas MBIR, IMR, and FIRST primarily operate in raw-data space.

The utility of iterative reconstruction algorithms has previously been demonstrated for specific low-dose abdominal CT applications including CT colonography (CTC),[6, 13, 14] detection of urolithiasis,[15, 16] and colonic diverticulitis.[17] For these clinical indications, aggressive dose reductions up to 90% relative to FBP may be feasible. However, in these contexts, the findings of interest are inherently high-contrast relative to their surroundings. For the liver, detection of low-contrast soft tissue lesions against a background of hepatic parenchyma is a much more difficult imaging task. Accurate imaging evaluation of the liver for focal metastatic lesions is critical in oncology for staging and therapeutic decisions, as well as for determining treatment response or progression.[1, 18]

Clinical performance data for detection of low-contrast liver lesions at reduced dose CT are lacking, especially with regard to newer iterative reconstruction algorithms. Recent phantom-based studies have shown little or no improvement in the performance of iterative reconstruction techniques compared with traditional FBP reconstruction at highly reduced doses.[19–21] Recent human subject research suggested a possible role for iterative reconstruction techniques for assessing liver lesions at reduced-dose CT, but these studies either relied on retrospective artificial noise insertion[22] or did not compare directly against standard-dose CT.[12] Preliminary data by our own research group suggested superior subjective image quality for MBIR (Veo) over FBP and ASIR at reduced-dose CT in human subjects , but also that further study of actual diagnostic performance was warranted.[11]

The purpose of this prospective trial was to directly compare the diagnostic performance of reduced-dose (RD) CECT using three different reconstruction algorithms (FBP, ASIR, and MBIR(Veo)) with concurrent standard-dose (SD) CECT for the detection of low-contrast liver lesions.

Materials and Methods

This HIPAA-compliant prospective study was approved by our institutional review board. All subjects provided signed informed consent.

Patient Population

Between April 2011 and April 2014, 70 adult patients (mean age 59.4±12.8 years, M:F 31:39) were enrolled. Mean patient BMI was 27.7±5.2 kg/m2 (Appendix Table 1). Inclusion criteria included adult men and non-pregnant women in outpatient care with a known diagnosis of non-liver primary malignancy presenting for CECT of the abdomen for staging or surveillance of metastatic disease. Primary malignancies included colorectal (n=18), pancreatic (n=14), neuroendocrine (n=9), breast (n=9), lung (n=4), esophageal (n=3), gastrointestinal stromal tumor (n=3), lymphoma (n=2), gallbladder (n=1), cervical (n=1), sarcoma (n=1), prostate (n=1), small bowel adenocarcinoma (n=1), anal squamous cell (n=1), testicular (n=1), and gastric (n=1).

Appendix Table 1.

Patient dose and demographic data

Patient No. Age (yrs) Sex BMI (kg/m2) CTDIvol (mGy) SSDE (mGy) Effective Dose (mSv)
Standard Dose Reduced Dose Standard Dose Reduced Dose Standard Dose Reduced Dose Dose Reduction
1 82 M 28.4 14.51 3.78 17.23 4.49 5.71 1.49 74%
2 55 F 39.8 32.09 3.94 33.72 4.14 15.63 1.92 88%
3 63 F 38.8 21.26 8.48 21.38 8.53 9.60 3.83 60%
4 34 M 28.0 7.17 4.61 9.30 5.98 3.08 1.98 36%
5 35 F 23.1 4.54 1.29 6.22 1.77 1.73 0.49 72%
6 85 F 33.8 35.30 15.05 39.92 17.02 10.11 4.31 57%
7 77 F 24.0 4.85 1.53 6.92 2.18 1.42 0.45 68%
8 72 F 35.9 25.71 10.25 28.65 11.42 8.81 3.51 60%
9 58 F 19.4 4.56 1.09 6.85 1.64 1.70 0.41 76%
10 43 F 21.5 4.85 1.37 7.54 2.13 1.98 0.56 72%
11 42 M 28.1 11.59 4.56 14.03 5.52 4.99 1.96 61%
12 60 F 23.6 5.92 2.16 8.54 3.12 1.95 0.71 64%
13 49 M 24.3 7.29 2.88 9.47 3.74 2.70 1.07 60%
14 76 M 27.9 41.48 10.08 46.47 11.29 14.84 3.61 76%
15 52 F 30.3 17.88 3.23 21.33 3.85 7.40 1.34 82%
16 42 M 32.4 64.22 4.14 35.36 4.28 13.78 1.67 88%
17 69 M 25.9 7.10 2.77 9.09 3.55 2.74 1.07 61%
18 54 M 31.8 10.93 4.35 13.28 5.29 4.38 1.74 60%
19 85 M 24.0 15.49 1.51 21.00 5.05 4.87 0.48 90%
20 52 M 22.0 4.70 1.41 6.97 2.09 1.72 0.52 70%
21 57 F 39.6 15.80 6.31 17.340 6.95 5.71 2.28 60%
22 70 M 28.7 10.67 4.18 12.28 4.81 4.76 1.86 61%
23 63 F 43.4 27.50 11.06 26.53 10.67 12.11 4.87 60%
24 70 M 25.4 13.86 1.57 17.53 1.99 4.83 0.55 89%
25 63 M 29.5 7.83 3.08 9.97 3.92 2.74 1.08 61%
26 52 F 30.3 10.53 3.99 12.12 4.59 3.73 1.41 62%
27 85 M 23.0 5.50 2.11 7.51 2.88 1.53 0.59 62%
28 74 F 24.5 5.20 1.94 7.39 2.76 1.78 0.66 63%
29 59 F 26.2 6.18 1.39 8.76 1.97 1.99 0.45 78%
30 66 F 29.6 7.34 2.71 9.80 3.62 2.50 0.92 63%
31 77 M 23.1 8.98 3.50 11.83 4.61 2.99 1.17 61%
32 59 M 30.1 26.32 10.47 27.07 10.77 11.54 4.59 60%
33 52 F 31.5 11.74 4.61 14.05 5.52 4.00 1.57 61%
34 61 M 29.3 17.59 6.88 19.48 7.62 6.82 2.67 61%
35 77 F 22.5 7.92 3.10 11.20 4.38 2.36 0.92 61%
36 48 F 25.8 12.79 4.97 16.15 6.28 5.20 2.02 61%
37 72 M 28.5 10.29 4.09 13.16 5.23 3.89 1.55 60%
38 49 F 23.3 7.04 2.74 9.60 3.74 3.02 1.18 61%
39 59 F 29.7 15.58 6.17 19.55 7.74 5.19 2.06 60%
40 55 F 28.2 14.59 5.81 17.47 6.96 6.31 2.51 60%
41 47 F 17.7 7.71 3.01 11.11 4.34 2.70 1.05 61%
42 62 M 25.7 8.99 3.50 11.79 4.59 3.44 1.34 61%
43 63 F 23.0 14.51 5.72 17.94 7.07 5.68 2.24 61%
44 66 F 36.9 28.84 11.55 30.21 12.10 12.70 5.08 60%
45 59 M 24.2 6.91 2.66 9.74 3.75 2.42 0.93 62%
46 58 M 23.2 9.31 3.62 12.45 4.84 3.80 1.48 61%
47 57 M 31.8 26.92 10.59 27.06 10.65 12.46 4.390 61%
48 42 M 32.4 31.85 7.52 33.86 8.00 12.29 2.90 76%
49 70 F 26.1 13.52 5.34 16.70 6.60 5.29 2.09 61%
50 57 F 25.5 10.05 3.92 12.60 4.91 3.97 1.55 61%
51 47 F 16.3 3.25 1.05 5.46 1.76 1.12 0.36 68%
52 59 M 26.3 13.96 5.52 16.82 6.65 5.02 1.99 60%
53 74 F 24.1 14.07 5.65 17.33 6.96 4.93 1.98 60%
54 61 F 30.6 12.42 4.84 15.51 6.05 4.86 1.89 61%
55 60 M 41.5 41.02 13.10 37.28 11.91 19.06 6.09 68%
56 58 F 26.5 13.48 5.35 16.39 6.51 4.85 1.92 60%
57 65 F 31.1 21.09 8.41 25.14 10.02 8.93 3.56 60%
58 36 M 28.4 15.57 6.21 18.70 7.46 7.09 2.83 60%
59 51 F 29.3 15.94 6.35 20.86 8.31 7.38 2.94 60%
60 55 F 22.0 5.15 2.00 7.44 2.89 1.79 0.70 61%
61 70 F 25.4 8.93 3.53 11.98 4.74 3.04 1.20 60%
62 51 F 25.5 8.44 3.31 10.85 4.26 3.72 1.46 61%
63 57 F 25.5 9.23 3.65 12.24 4.84 3.41 1.35 60%
64 33 M 26.6 10.14 4.01 13.75 5.44 4.79 1.89 60%
65 70 M 33.6 27.13 10.70 28.06 11.07 12.71 5.01 61%
66 43 F 34.4 27.19 10.82 30.61 12.18 11.05 4.40 60%
67 50 M 26.0 15.05 5.95 18.32 7.24 6.43 2.54 60%
68 45 M 20.7 9.47 3.71 12.27 4.81 4.47 1.75 61%
69 53 M 28.5 20.02 8.09 21.74 8.79 9.34 3.77 60%
70 85 F 26.7 9.40 3.72 12.43 4.92 3.22 1.28 60%
Mean ± StDev 59.4 ± 12.8 M 31: F 29 27.7 ± 5.2 14.49 ± 9.29 5.04 ± 3.18 16.98 ± 9.04 5.92 ± 3.16 5.77 ± 4.02 2.01 ± 1.36 64% ± 9%
Open in a new tab

CT Protocol

All studies were performed on a 64-slice multi-detector CT scanner (Discovery CT750 HD, GE Healthcare) with a detector configuration of 64 × 0.625 mm, 120 kV, and tube current modulation (Auto-mA and Smart-mA, GE Healthcare). All patients first underwent SD CECT (Appendix Table 2) of the abdomen in the portal venous phase with automatic bolus tracking (SmartPrep), followed immediately by matched-coverage RD acquisition within the same breath hold to minimize any differences in phase of parenchymal contrast enhancement. The target range for dose reduction was deliberately set at 60–70%, which is intermediate between typical reduction levels for CECT with ASIR (25–40%) and unenhanced indications such as CTC and urolithiasis (80–90%). This level of dose reduction (approximate CTDIvol of 5 mGy) was based on prior studies suggesting levels that would maintain diagnostic quality.[23, 24] Although lower levels down to CTDIvol of 3–4 mGy have been reported,[11] this aggressive reduction is better suited for abdominal applications such as CTC[25] and urolithiasis.[15] Given the inherent low-contrast nature of most liver metastases, the decision was made to reduce dose to a more intermediate degree.

Appendix Table 2.

Standard dose CT parameters

Parameter Value
Scanner GE Discovery CT750 HD
Scan type Helical
Helical pitch 0.516
Rotation time (s) 0.5
Beam collimation (mm) 40
No. of detector rows 64
Detector configuration 64 × 0.625
Scan field of view Large body
kV 120
Smart mA min/max mA Range 60–660
Slice thickness for noise index (mm) 5.0
Noise index 16.3
Reconstruction (filtered back projection)
 Display field of view (cm) 36–50
 Reconstruction type Standard
 Slice thickness (mm) 2.5
 Slice interval (mm) 1.25
 Window width/level (HU) 400/50
 Reconstruction option Plus
Open in a new tab

Individual dose reductions were achieved as needed by adjustments in tube current range and noise index. CT doses are reported both as volume CT dose index (CTDIvol, mGy) as well as size specific dose estimate (SSDE, mGy). Effective dose measurements (mSv) were calculated from the dose-length product (DLP), multiplied by the conversion factor of 0.015 mSv/mGy × cm) recommended by the AAPM[26] and verified by Deak et al.[27]

CT Image Reconstruction and Analysis

Following acquisition, CT examinations were anonymized. The SD data set for each patient was reconstructed using FBP. The RD data set for each patient was reconstructed using three different reconstruction algorithms equipped with the CT scanner: FBP, ASIR, and MBIR. For the ASIR series, a 40% blend was used to optimize subjective image quality, as previously described. [9, 11, 28] All SD and RD series were reconstructed in the axial and coronal planes using 2.5 mm slice thickness at 1.25 mm intervals. Following reconstruction, the four series (SD-FBP, RD-FBP, RD-ASIR, and RD-MBIR) were randomized into separate experimental groups (one series per patient per group) for review. In this way, no experimental group contained more than one series from any given patient, preventing experimental readers from making direct comparisons among reconstructions.

Each of the experimental groups was independently interpreted by three different reviewers, including a senior radiology resident, an abdominal imaging fellow, and an experienced abdominal radiologist with over 15 years of experience. Reviewers were aware the study was performed to evaluate for metastatic disease in the liver, but were blinded to the specific indication and reconstruction algorithm for each series, as well as any prior or subsequent comparison CT examinations and all patient clinical information. Reviewers were instructed to identify up to five distinct liver lesions ≥ 4 mm in diameter, ignoring the remainder of the abdomen and prioritizing the identification of malignant lesions, only listing benign lesions if there were fewer than five total lesions believed to be malignant. Use of axial versus coronal reconstructions as well as window and level settings was left to the discretion of each reviewer.

The location, diameter, Hounsfield unit (HU) density, subjective classification as malignant or benign (5-point scale: 1=definitely benign, 2=likely benign, 3=malignancy not excluded, 4=likely malignant, 5=definitely malignant), and confidence that the lesion was a true finding (3-point scale: 1=low-confidence, 2=moderate-confidence, 3=high-confidence) were recorded by each reader for each lesion identified. Overall by-patient confidence was also recorded for all studies interpreted as negative for liver lesions. Each reader reviewed all randomized experimental groups such that all SD and all RD series were independently reviewed by all three experimental reviewers, with a minimum of one week washout period between groups.

Reference Standard

Two experienced abdominal radiologists (over 15 and 7 years in practice, respectively) served as tandem reviewers for establishment of the reference standard. All SD and RD series were reviewed with attention to the lesions identified by the experimental group reviewers. For the reference standard, comparison was made with available past and subsequent cross-sectional imaging examinations (CT, MR, and PET/CT), as well as all available clinical information. All potential lesions identified by experimental group reviewers on the SD and RD series were correlated to findings on both the index and comparison studies, and were classified as metastases, benign lesions, or false positive lesions. Additionally, any malignant lesions ≥ 4 mm found on reference standard review but not identified by the experimental group reviewers were identified as false negatives. For patients with five or more metastatic lesions by the reference standard, the experimental readers were only scored with false negatives if they failed to identify five malignant lesions. Experimental group findings for both SD and RD series were compared with this enhanced reference standard and performance metrics calculated on a per-patient basis. For calculation of performance metrics, benign or false positive lesions scored 3 or higher on the malignancy scale were considered falsely positive against the reference standard. Truly malignant lesions on the reference standard either not identified by reviewers or scored 2 or lower on the malignancy scale were considered falsely negative.

With regard to prior and follow-up cross-sectional imaging examinations (CT, MR, and PET/CT) used in the enhanced reference standard, patients underwent a mean [± SD] of 5.3±4.4 scans (range, 0–27) over 2.1±2.4 years (range, 0.1–10.1 years) prior to the study and 4.3±4.2 scans (range, 0–17) over an average of 1.6±1.1 years (range, 0.1–4.4 years) for follow-up after the study CT scan.

Statistical Analysis

The modified Wald method was used to calculate confidence intervals for proportions. The t-test was used to assess differences in continuous variables and chi-squared or Fisher’s exact test was used, where appropriate, to assess differences in categorical variables. A two-tailed P-value <0.05 was used as the criterion for statistical significance. Receiver operating characteristic (ROC) curves were constructed using Microsoft Excel (Microsoft Excel 2010, Microsoft Corporation, Redmond, WA) and areas under the curve (AUC) were calculated using the trapezoidal rule with comparison of ROC curves performed using the bivariate analysis model described by Hanley and McNeil.[29, 30]

Results

The mean [±SD] effective dose for SD and RD CECT series were 5.77±4.02 mSv (median, 4.81 mSv) and 2.01±1.36 mSv (median, 1.71 mSv), respectively, representing a mean [±SD] reduction of 3.77±2.94 mSv, or 64.1±8.8% (Appendix Table 1). The mean [±SD] SSDE for SD and RD CECT series were 16.98±9.04 mGy (median, 14.04 mGy) and 5.92±3.16 mGy (median, 4.92mGy). The mean [±SD] CTDIvol for the SD and RD CECT series were 14.49±9.29 mGy (median, 11.67 mGy) and 5.04±3.18 mGy (median, 4.05 mGy).

Of the 70 patients in the study group, 19/70 (27%) had at least one verified metastatic liver lesion ≥ 4 mm identified by the reference standard, with 12/19 (63%) of those patients having at least 5 metastatic lesions. Compared with the reference standard, pooled SD-FBP among the experimental readers demonstrated sensitivity/specificity/PPV/NPV/accuracy of 0.91/0.78/0.60/0.96/0.81, respectively (Table 1). Pooled performance metrics for RD series (sensitivity/specificity/PPV/NPV/accuracy) were: 0.79/0.75/0.54/0.91/0.76 for RD-FBP; 0.84/0.75/0.56/0.93/0.78 for RD-ASIR; and 0.84/0.68/0.49/0.92/0.72 for RD-MBIR (Figure 1). Although none of the 95% confidence intervals for the RD series fall entirely below that of SD, relatively decreased sensitivity for RD-FBP and decreased specificity for RD-MBIR is seen. In comparison, RD-ASIR appears to maintain sensitivity relative to RD-FBP and specificity relative to RD-MBIR, respectively. Performance metrics for individual readers are provided in Appendix Table 3.

Table 1.

Per-patient pooled performance data by reconstruction algorithm

Performance Measure % [95% CI] (N/total N) Reconstruction Algorithm
SD-FBP RD-FBP RD-ASIR RD-MBIR
Sensitivity 0.91 [0.84–0.99] (52/57) 0.79 [0.68–0.90] (45/57) 0.84 [0.75–0.94] (48/57) 0.84 [0.75–0.94] (48/57)
Specificity 0.78 [0.71–0.84] (119/153) 0.75 [0.68–0.82] (115/153) 0.75 [0.68–0.82] (115/153) 0.68 [0.61–0.75] (104/153)
Positive Predictive Value 0.60 [0.50–0.71] (52/86) 0.54 [0.43–0.65] (45/83) 0.56 [0.45–0.66] (48/86) 0.49 [0.40–0.59] (48/97)
Negative Predictive Value 0.96 [0.93–0.99] (119/124) 0.91 [0.85–0.95] (115/127) 0.93 [0.88–0.97] (115/124) 0.92 [0.87–0.97] (104/113)
Accuracy 0.81 [0.76–0.87] (171/210) 0.76 [0.70–0.82] (160/210) 0.78 [0.72–0.83] (163/210) 0.72 [0.66–0.78] (152/210)
Open in a new tab

Figure 1.

Figure 1

Open in a new tab

Pooled per-patient performance data by reconstruction algorithm

Appendix Table 3.

Per-patient performance data by reconstruction algorithm for individual readers

Reader Performance Measure % [95% CI] (N/total N) Reconstruction Algorithm
SD-FBP RD-FBP RD-ASIR RD-MBIR
Reader 1 Sensitivity 0.89 [0.76–1.00] (17/19) 0.74 [0.54–0.93] (14/19) 0.79 [0.61–0.97] (15/19) 0.74 [0.54–0.93] (14/19)
Specificity 0.90 [0.82–0.98] (46/51) 0.82 [0.72–0.93] (42/51) 0.80 [0.69–0.91] (41/51) 0.80 [0.69–0.91] (41/51)
Positive Predictive Value 0.77 [0.60–0.95] (17/22) 0.61 [0.41–0.81] (14/23) 0.60 [0.41–0.79] (15/25) 0.58 [0.39–0.78] (14/24)
Negative Predictive Value 0.96 [0.90–1.00] (46/48) 0.89 [0.81–0.98] (42/47) 0.91 [0.83–0.99] (41/45) 0.89 [0.80–0.98] (41/46)
Accuracy 0.90 [0.83–0.97] (63/70) 0.80 [0.71–0.89] (56/70) 0.80 [0.71–0.89] (56/70) 0.79 [0.69–0.88] (55/70)
Reader 2 Sensitivity 0.95 [0.85–1.00] (18/19) 0.84 [0.68–1.00] (16/19) 0.89 [0.76–1.00] (17/19) 0.89 [0.76–1.00] (17/19)
Specificity 0.55 [0.41–0.69] (28/51) 0.61 [0.47–0.74] (31/51) 0.67 [0.54–0.80] (34/51) 0.49 [0.35–0.63] (25/51)
Positive Predictive Value 0.44 [0.29–0.59] (18/41) 0.44 [0.28–0.61] (16/36) 0.50 [0.33–0.67] (17/34) 0.40 [0.25–0.54] (17/43)
Negative Predictive Value 0.97 [0.90–1.00] (28/29) 0.91 [0.82–1.00] (31/34) 0.94 [0.87–1.00] (34/36) 0.93 [0.83–1.00] (25/27)
Accuracy 0.66 [0.55–0.77] (46/70) 0.67 [0.56–0.78] (47/70) 0.73 [0.62–0.83] (51/70) 0.67 [0.49–0.71] (42/70)
Reader 3 Sensitivity 0.89 [0.76–1.00] (17/19) 0.79 [0.61–0.97] (15/19) 0.84 [0.68–1.00] (16/19) 0.89 [0.76–1.00] (17/19)
Specificity 0.88 [0.79–0.97] (45/51) 0.82 [0.72–0.93] (42/51) 0.78 [0.67–0.90] (40/51) 0.75 [0.63–0.86] (38/51)
Positive Predictive Value 0.74 [0.56–0.92] (17/23) 0.63 [0.43–0.82] (15/24) 0.59 [0.41–0.78] (16/27) 0.57 [0.39–0.74] (17/30)
Negative Predictive Value 0.96 [0.90–1.00] (45/47) 0.91 [0.83–0.99] (42/46) 0.93 [0.85–1.00] (40/43) 0.95 [0.88–1.00] (38/40)
Accuracy 0.89 [0.81–0.96] (62/70) 0.81 [0.72–0.91] (57/70) 0.80 [0.71–0.89] (56/70) 0.79 [0.69–0.88] (55/70)
Open in a new tab

ROC curves for per-patient performance data were calculated for SD-FBP and each of the reduced dose reconstruction algorithms (Figure 2). Corresponding AUC values were 0.896 for SD-FBP, 0.834 for RD-FBP, 0.858 for RD-ASIR, and 0.854 for RD-MBIR. Statistically significant differences were seen between the AUC values for SD-FBP and RD-FBP (P=0.002), and for SD-FBP and RD-MBIR (P=0.032), but not quite for SD-FBP and RD-ASIR (P=0.052); no statistically significant difference was observed among the AUC curves of the reduced dose reconstructions (RD-FBP vs RD-ASIR, P=0.251; RD-FBP vs RD-MBIR, P=0.344; RD-ASIR vs RD-MBIR, P=0.842). Examples of false negative and false positive cases at RD are shown in Figures 3 and 4, respectively.

Figure 2.

Figure 2

Open in a new tab

Receiver operating curve for reduced dose reconstructions versus standard dose

Figure 3. False negative lesions at reduced dose reconstructions.

Figure 3

Open in a new tab

Red arrows indicate metastatic lesions within the right lobe of the liver on standard dose FBP imaging [top left] of an 82-year-old male with metastatic neuroendocrine tumor. These lesions were not identified at reduced dose MBIR [top right], FBP [bottom left] or ASIR [bottom right]. Dose reduction was 74% on the reduced dose series, with an effective dose of 1.49 mSv. Additional lesions in the left lobe are partially seen on the standard dose image but not clearly seen on the reduced dose series.

Figure 4. False positive lesion at reduced dose MBIR.

Figure 4

Open in a new tab

The red arrow indicates a false positive lesion within the right lobe of the liver on reduced dose MBIR [top right] in a 52-year-old female with a primary diagnosis of pancreatic neuroendocrine tumor. This lesion persisted on adjacent axial slices. No lesion is seen at standard dose FBP [top left], reduced dose FBP [bottom left], or reduced dose ASIR [bottom right]. Comparison to subsequent CT scans confirmed absence of metastatic disease. Dose reduction was 82% on the reduced dose series, with an effective dose of 1.34 mSv.

Reader confidence scores are summarized in Table 2. Notably, compared to SD-FBP, patients truly negative for metastatic disease were called true negative with significantly less confidence on all RD reconstructions with mean confidence scores of 2.70±0.55 for SD-FBP and 1.98±0.63, 1.98±0.74, and 1.62±0.64 for RD-FBP, RD-ASIR, and RD-MBIR, respectively (P<0.001 in all cases, with RD-MBIR also statistically significantly lower than RD-FBP, P<0.001, and RD-ASIR, P=0.001). Compared to SD-FBP, clinically small, but statistically significant, differences in reader confidence were noted for true positive malignant lesions for RD-FBP (P=0.016) and RD-ASIR (P=0.038), but not for RD-MBIR, with reduced dose series demonstrating lower reader confidence. Differences in lesion diameter among reconstruction algorithms are summarized in Table 3. Of note, false positive lesions detected at RD-MBIR, in addition to being greater in number, were statistically significantly smaller when compared with SD-FBP (P=0.004). Differences in lesion density among reconstruction algorithms are summarized in Table 4. Notably, true positive malignancies were noted to be of significantly lower density at RD-MBIR when compared to SD-FBP (P=0.042); no other statistically significant differences were noted among SD or RD reconstructions.

Table 2.

Per-patient pooled reader confidence by reconstruction algorithm

Lesion Category Mean ± SD (Total N) SD-FBP RD-FBP RD-ASIR RD-MBIR
Mean diagnostic confidence* with P-values†
True positive malignant lesions 2.89 ± 0.40 (199) 2.77 ± 0.55 (174) P=0.016 2.79 ± 0.56 (177) P=0.038 2.87 ± 0.39 (182) P=0.596
True positive benign lesions 2.96 ± 0.24 (103) 2.96 ± 0.19 (84) P=0.922 2.95 ± 0.28 (75) P=0.711 2.91 ± 0.33 (76) P=0.215
False positive lesions 2.43 ± 0.85 (14) 1.64 ± 0.60 (14) P=0.009 2.00 ± 0.51 (12) P=0.141 1.69 ± 0.54 (32) P<0.001
False negative lesions (mets mischaracterized as benign) 3.00 ± 0.00 (2) 2.33 ± 1.15 (3) P=0.495 2.50 ± 0.71 (2) P=0.423 3.00 ± N/A (1) P=N/A
True negative patients 2.70 ± 0.55 (90) 1.98 ± 0.63 (97) P<0.001 1.98 ± 0.74 (97) P<0.001 1.62 ± 0.64 (82) P<0.001
Open in a new tab
*

Three point scale: 1 = low confidence, 2 = somewhat confident, 3 = high confidence

†

P-values compared to SD-FBP

Table 3.

Lesion diameter by reconstruction algorithm

Lesion Category Mean ± SD (Total N) SD-FBP RD-FBP RD-ASIR RD-MBIR
Mean lesion diameter (mm) with P-values*
True positive malignant lesions 21.6 ± 16.5 (199) 22.7 ± 18.1 (174) P=0.537 22.0 ± 17.8 (177) P=0.812 22.8 ± 18.4 (182) P=0.484
True positive benign lesions 12.6 ± 10.2 (103) 11.8 ± 8.5 (84) P=0.509 11.9 ± 8.1 (75) P=0.610 11.6 ± 9.1 (76) P=0.481
False positive lesions 18.0 ± 14.4 (14) 12.9 ± 8.1 (14) P=0.254 12.0 ± 8.1 (12) P=0.214 8.6 ± 6.6 (32) P=0.004
False negative lesions (mets mischaracterized as benign) 6.0 ± 0.0 (2) 7.0 ± 1.0 (3) P=0.272 7.5 ± 3.5 (2) P=0.609 9.0 ± N/A (1) P=N/A
Open in a new tab
*

P-values compared to SD-FBP

Table 4.

Lesion density by reconstruction algorithm

Lesion Category Mean ± SD (Total N) SD-FBP RD-FBP RD-ASIR RD-MBIR
Mean lesion density (HU) with P-values*
True positive malignant lesions 62.9 ± 19.1 (199) 60.6 ± 15.7 (174) P=0.214 60.1 ± 16.6 (177) P=0.136 59.0 ± 18.0 (182) P=0.042
True positive benign lesions 23.2 ± 26.1 (103) 19.0 ± 17.7 (84) P=0.200 19.0 ± 18.5 (75) P=0.224 17.7 ± 20.1 (76) P=0.123
False positive lesions 80.7 ± 15.3 (14) 67.8 ± 19.2 (14) P=0.060 77.3 ± 18.5 (12) P=0.607 75.0 ± 21.3 (32) P=0.369
False negative lesions (mets mischaracterized as benign) 73.5 ± 27.6 (2) 43.7 ± 15.4 (3) P=0.205 57.0 ± 29.7 (2) P=0.623 87.0 ± N/A (1) P=N/A
Open in a new tab
*

P-values compared to SD-FBP

Discussion

CECT remains a mainstay in the diagnosis, staging, and surveillance of metastatic disease within the chest, abdomen, and pelvis.[1] Consequently, the continued development of methods to lower CT radiation dose without significantly impacting performance is an important goal, especially given that many patients will undergo multiple CT scans over the course of their diagnosis and treatment. Although the actual health risks related to medical radiation remain a topic of considerable debate,[4] it is nevertheless reasonable to ensure exposures are as low as reasonably achievable for accurate diagnosis. To this end, considerable effort has been directed toward the development of more sophisticated iterative reconstruction algorithms, such as MBIR, with the promise of substantial dose reduction. While there are data in the literature to support subjective improvements in image quality with these newer iterative reconstruction techniques (e.g. MBIR) compared with older techniques such as ASIR, there remains some question as to whether there will also be meaningful differences in diagnostic performance.

CT dose reduction strategies have been explored and largely validated for some specific abdominopelvic indications, including CTC[6, 13, 14] and urolithiasis.[15, 16] However, these tasks feature greater inherent contrast between the target lesion and background tissues (e.g. soft tissue colorectal polyps against gas or contrast-tagged fluid and high-attenuation calculi against renal parenchyma). Comparatively, low-attenuation liver lesions against a background of liver parenchyma provide less contrast, and pose a much greater diagnostic challenge. Several previous studies have evaluated this key area of reduced-dose CT for the detection of liver lesions.[12, 19–22] However, while very informative, these studies all have some methodological issues that may limit their applicability to clinical practice, including use of static non-contrast phantoms instead of actual patients,[19–21] the artificial introduction of noise,[22] and lack of direct comparison to a SD acquisition.[12] Our study builds upon these prior efforts and, to our knowledge, represents the first attempt at aggressive dose reduction (60–70%) in a clinical patient population where both RD and SD series are obtained in immediate succession, allowing for direct meaningful comparison.

The methodology applied in this study allows for comparisons between RD and SD CECT not possible in prior studies. Rather than simply compare the experimental low dose series to a SD reference standard, we directed our experimental readers to also interpret the SD images under the same blinded and isolated conditions as with each RD series. This methodology demonstrates that even the SD series is not infallible when interpretation is performed in a vacuum without clinical history or comparison imaging. In comparison, the reference standard readers interpreted the SD images with the benefit of full access to the medical record and comparison studies (both prior and follow-up) for each patient. This paradigm permits better direct comparison of RD and SD CECT under the same conditions and tempers the lower accuracy values found with the former.

Interpreting the SD-FBP images in isolation, our experimental readers were able to achieve reasonable overall per-patient sensitivity (91%) with modest specificity (78%) and accuracy (81%) for metastatic liver lesions ≥ 4 mm when compared to the non-blinded reference standard. For the RD series, we aggressively reduced dose by over 60% on average, with a mean effective dose of approximately 2 mSv. At these doses, we observed decreased diagnostic performance for all RD reconstructions when compared to SD-FBP, with the performance values for the RD series falling at or below the 95% confidence interval for the SD-FBP series. Although at no point did the 95% confidence intervals for the RD series fall entirely below that of SD, relative outliers include decreased sensitivity for RD-FBP and decreased specificity for RD-MBIR, whereas RD-ASIR maintained a more balanced performance, matching RD-MBIR for sensitivity and RD-FBP for specificity. We also noted that the area under the ROC curves were lower for RD-FBP and RD-MBIR, with statistical significance, and nearly so for RD-ASIR. Practically, this translated into both false negatives (both missed and misclassified lesions) (Figure 3) as well as false positive lesions (Figure 4) at RD imaging, which resulted in complete misclassification of the metastatic disease status of some patients. Additionally, we noted that the readers had lower confidence in the RD reconstruction algorithms when compared to SD, most notably when declaring patients free of disease, where there was a relatively large (nearly a full point on a 3-point scale) and statistically significant difference for all three RD reconstruction algorithms. Differences in lesion size and density between SD and RD CECT were considerably less pronounced.

Perhaps more importantly, unlike prior studies evaluating more modest dose reductions there was no improvement seen with MBIR over ASIR or FBP at aggressive dose reduction in our study. In fact, RD-MBIR demonstrated lower specificity compared with both RD-ASIR and even RD-FBP. As we and others have noted anecdotally, RD-MBIR seems more prone to generating a somewhat smudgy liver texture with occasional pseudolesions (Figure 4), which were frequently labeled in our study as concerning for metastatic disease and drove down specificity. Of note, this appearance is not isolated to single slices; in fact, this phenomenon seems to persist (and even appear accentuated) while scrolling through these pseudolesions in a cine mode. While further investigation is clearly warranted to explain this phenomenon, we hypothesize that it may stem from MBIR’s attempts to constrain noise within a subtly photopenic region of liver, perhaps resulting in the appearance of a smoothly marginated low-density focus. This phenomenon is also evident in the confidence data as readers were significantly less confident in labeling patients truly negative on RD-MBIR when compared to RD-FBP and RD-ASIR. Given decreased performance and reader confidence at the radiation doses evaluated in this study—in some cases resulting in the misclassification of patients as falsely positive or negative for liver metastatic disease—it may be prudent to question the perceived benefit of sparing radiation dose in this patient population versus the risk of potential misdiagnosis.

We acknowledge several study limitations. Given that dose reduction in CT is an ongoing area of research, there is no broadly accepted standard protocol for reduced dose imaging; consequently, centers whose protocols differ substantially from our own may experience different results. For this prospective trial, we were confined to the dose reduction level set at the outset of the study, which was based on the best available data. For the purposes of this study, our experimental readers were blinded to patient clinical history as well as prior and follow-up imaging, which may result in some degree of underestimation of diagnostic performance when compared to clinical practice (as seen with the reduced performance with the SD series). Additionally, while our SD and RD scans were obtained in immediate succession, there is an unavoidable associated slight delay in phase of contrast, which could theoretically affect lesion detection in some instances. Our patient population was obese on average (mean BMI of 28 kg/m2), which likely affected lesion detection, even if the resulting effective doses for both the SD and RD series were higher relative to thinner patients. Despite the use of at least a one-week washout period between series, the possibility of recall bias exists, particularly for patients who made have had memorable or characteristic findings. There was heterogeneity in the type, number, and interval of prior and follow-up cross-sectional examinations used to establish the reference standard, based on the particular clinical course of each individual patient; however, patients averaged multiple scans both before and after the scan for the current study (nearly 10 total scans per patient on average), allowing for establishment of a robust reference standard. Finally, this study assessed the performance of a specific model-based reconstruction algorithm from one vendor (Veo). As such, our results may not necessarily generalize to iterative reconstruction algorithms offered by other vendors, and additional specific studies are clearly needed to confirm diagnostic performance for those related techniques.

In conclusion, using aggressive CT dose reduction (ie, >60% reduction from our standard imaging protocol; effective doses around 2 mSv) to evaluate low contrast liver lesions ≥ 4 mm, we observed lower diagnostic performance and reader confidence for all RD CECT reconstructions (FBP, ASIR, and MBIR) when compared to SD CECT, sometimes resulting in the misclassification of the metastatic disease status of patients. Furthermore, unlike prior experiences with less aggressive dose reduction, there was no clear advantage seen with MBIR over ASIR (in fact, specificity was decreased for MBIR). Consequently, the role of aggressive CECT dose reduction in the clinical oncology setting for detecting low-contrast liver lesions should be questioned given the potential for misdiagnosis in cancer patients.

Key Points.

  • Reduced-dose CECT demonstrates inferior diagnostic performance for detecting low-contrast liver lesions.

  • Reader confidence is lower with reduced dose CECT compared to standard dose CECT.

  • Overly aggressive dose reduction may result in misdiagnosis, regardless of reconstruction algorithm.

  • Careful consideration of perceived risks versus benefits of dose reduction is crucial.

Acknowledgments

The scientific guarantor of this publication is PJ Pickhardt. This research was supported by the National Institutes of Health, grant 1R01 CA169331. Dr. Pickhardt is co-founder of VirtuoCTC and shareholder in Cellectar Biosciences, SHINE, and Elucent. Dr. Kim is co-founder of VirtuoCTC, consultant for Viatronix, and on the medical advisory board for Digital Artforms. All other authors declare no relevant disclosures. No complex statistical methods were necessary for this paper. Institutional Review Board approval was obtained. Written informed consent was obtained from all subjects (patients) in this study. Methodology: prospective, case-control study, performed at one institution.

Footnotes

Dr. Pickhardt is co-founder of VirtuoCTC and shareholder in Cellectar Biosciences andSHINE. Dr. Kim is co-founder of VirtuoCTC, consultant for Viatronix, and on the medical advisory board for Digital Artforms. The other authors declare no disclosures.

Reference List

  • 1.Eisenhauer EA, et al. New response evaluation criteria in solid tumours: Revised RECIST guideline (version 1.1) European Journal of Cancer. 2009;45(2):228–247. doi: 10.1016/j.ejca.2008.10.026. [DOI] [PubMed] [Google Scholar]
  • 2.Levin DC, V, Rao M, Parker L. The Recent Downturn in Utilization of CT: The Start of a New Trend? Journal of the American College of Radiology. 2012;9(11):795–798. doi: 10.1016/j.jacr.2012.05.023. [DOI] [PubMed] [Google Scholar]
  • 3.Moreno CC, et al. Changing Abdominal Imaging Utilization Patterns: Perspectives from Medicare Beneficiaries Over Two Decades. J Am Coll Radiol. 2016 doi: 10.1016/j.jacr.2016.02.031. [DOI] [PubMed] [Google Scholar]
  • 4.Hall EJ, Brenner DJ. Cancer risks from diagnostic radiology. British Journal of Radiology. 2008;81(965):362–378. doi: 10.1259/bjr/01948454. [DOI] [PubMed] [Google Scholar]
  • 5.Patino M, et al. Iterative Reconstruction Techniques in Abdominopelvic CT: Technical Concepts and Clinical Implementation. AJR Am J Roentgenol. 2015;205(1):W19–31. doi: 10.2214/AJR.14.13402. [DOI] [PubMed] [Google Scholar]
  • 6.Flicek KT, et al. Reducing the Radiation Dose for CT Colonography Using Adaptive Statistical Iterative Reconstruction: A Pilot Study. American Journal of Roentgenology. 2010;195(1):126–131. doi: 10.2214/AJR.09.3855. [DOI] [PubMed] [Google Scholar]
  • 7.Gervaise A, et al. CT image quality improvement using adaptive iterative dose reduction with wide-volume acquisition on 320-detector CT. European Radiology. 2012;22(2):295–301. doi: 10.1007/s00330-011-2271-7. [DOI] [PubMed] [Google Scholar]
  • 8.Lee SJ, et al. A Prospective Comparison of Standard-Dose CT Enterography and 50% Reduced-Dose CT Enterography With and Without Noise Reduction for Evaluating Crohn Disease. American Journal of Roentgenology. 2011;197(1):50–57. doi: 10.2214/AJR.11.6582. [DOI] [PubMed] [Google Scholar]
  • 9.Sagara Y, et al. Abdominal CT: Comparison of Low-Dose CT With Adaptive Statistical Iterative Reconstruction and Routine-Dose CT With Filtered Back Projection in 53 Patients. American Journal of Roentgenology. 2010;195(3):713–719. doi: 10.2214/AJR.09.2989. [DOI] [PubMed] [Google Scholar]
  • 10.Singh S, et al. Abdominal CT: Comparison of Adaptive Statistical Iterative and Filtered Back Projection Reconstruction Techniques. Radiology. 2010;257(2):373–383. doi: 10.1148/radiol.10092212. [DOI] [PubMed] [Google Scholar]
  • 11.Pickhardt PJ, et al. Abdominal CT With Model-Based Iterative Reconstruction (MBIR): Initial Results of a Prospective Trial Comparing Ultralow-Dose With Standard-Dose Imaging. American Journal of Roentgenology. 2012;199(6):1266–1274. doi: 10.2214/AJR.12.9382. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Volders D, et al. Model-based Iterative Reconstruction and Adaptive Statistical Iterative Reconstruction Techniques in Abdominal CT: Comparison of Image Quality in the Detection of Colorectal Liver Metastases. Radiology. 2013;269(2):468–473. doi: 10.1148/radiol.13130002. [DOI] [PubMed] [Google Scholar]
  • 13.Chang KJ, Yee J. Dose reduction methods for CT colonography. Abdominal Imaging. 2013;38(2):224–232. doi: 10.1007/s00261-012-9968-1. [DOI] [PubMed] [Google Scholar]
  • 14.Lubner MG, et al. Sub-milliSievert (sub-mSv) CT colonography: a prospective comparison of image quality and polyp conspicuity at reduced-dose versus standard-dose imaging. European Radiology. 2015;25(7):2089–2102. doi: 10.1007/s00330-015-3603-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Pooler BD, et al. Prospective Trial of the Detection of Urolithiasis on Ultralow Dose (Sub mSv) Noncontrast Computerized Tomography: Direct Comparison against Routine Low Dose Reference Standard. Journal of Urology. 2014;192(5):1433–1439. doi: 10.1016/j.juro.2014.05.089. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Zilberman DE, et al. Low Dose Computerized Tomography for Detection of Urolithiasis-Its Effectiveness in the Setting of the Urology Clinic. Journal of Urology. 2011;185(3):910–914. doi: 10.1016/j.juro.2010.10.052. [DOI] [PubMed] [Google Scholar]
  • 17.Laqmani A, et al. Reduced-dose abdominopelvic CT using hybrid iterative reconstruction in suspected left-sided colonic diverticulitis. European Radiology. 2016;26(1):216–224. doi: 10.1007/s00330-015-3810-4. [DOI] [PubMed] [Google Scholar]
  • 18.Zacharia TT, et al. CT of colon cancer metastases to the liver using modified RECIST criteria: Determining the ideal number of target lesions to measure. American Journal of Roentgenology. 2006;186(4):1067–1070. doi: 10.2214/AJR.05.0038. [DOI] [PubMed] [Google Scholar]
  • 19.Dobeli KL, et al. Noise-reducing algorithms do not necessarily provide superior dose optimisation for hepatic lesion detection with multidetector CT. British Journal of Radiology. 2013;86(1023) doi: 10.1259/bjr.20120500. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Schindera ST, et al. Iterative Reconstruction Algorithm for CT: Can Radiation Dose Be Decreased While Low-Contrast Detectability Is Preserved? Radiology. 2013;269(2):510–517. doi: 10.1148/radiol.13122349. [DOI] [PubMed] [Google Scholar]
  • 21.Baker ME, et al. Contrast-to-Noise Ratio and Low-Contrast Object Resolution on Full- and Low-Dose MDCT: SAFIRE Versus Filtered Back Projection in a Low-Contrast Object Phantom and in the Liver. American Journal of Roentgenology. 2012;199(1):8–18. doi: 10.2214/AJR.11.7421. [DOI] [PubMed] [Google Scholar]
  • 22.Fletcher JG, et al. Observer Performance in the Detection and Classification of Malignant Hepatic Nodules and Masses with CT Image-Space Denoising and Iterative Reconstruction. Radiology. 2015;276(2):465–478. doi: 10.1148/radiol.2015141991. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Deak Z, et al. Filtered Back Projection, Adaptive Statistical Iterative Reconstruction, and a Model-based Iterative Reconstruction in Abdominal CT: An Experimental Clinical Study. Radiology. 2013;266(1):197–206. doi: 10.1148/radiol.12112707. [DOI] [PubMed] [Google Scholar]
  • 24.Martinsen ACT, et al. Iterative reconstruction reduces abdominal CT dose. European Journal of Radiology. 2012;81(7):1483–1487. doi: 10.1016/j.ejrad.2011.04.021. [DOI] [PubMed] [Google Scholar]
  • 25.Lubner MG, et al. Sub-milliSievert (sub-mSv) CT colonography: a prospective comparison of image quality and polyp conspicuity at reduced-dose versus standard-dose imaging. European Radiology. 2015;25(7):2089–2102. doi: 10.1007/s00330-015-3603-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.AAPM. The Measurement, Reporting, and Management of Radiation Dose in CT. American Association of Physicists in Medicine; College Park, MD: 2008. [Google Scholar]
  • 27.Deak PD, Smal Y, Kalender WA. Multisection CT Protocols: Sex- and Age-specific Conversion Factors Used to Determine Effective Dose from Dose-Length Product. Radiology. 2010;257(1):158–166. doi: 10.1148/radiol.10100047. [DOI] [PubMed] [Google Scholar]
  • 28.Prakash P, et al. Reducing abdominal CT radiation dose with adaptive statistical iterative reconstruction technique. Invest Radiol. 2010;45(4):202–10. doi: 10.1097/RLI.ob013e3181dzfeec. [DOI] [PubMed] [Google Scholar]
  • 29.Hanley JA, McNeil BJ. The Meaning and Use of the Area Under a Receiver Operating Characteristic (ROC) Curve. Radiology. 1982;143(1):29–36. doi: 10.1148/radiology.143.1.7063747. [DOI] [PubMed] [Google Scholar]
  • 30.McNeil BJ, Hanley JA. Statistical Approaches to the Analysis of Receiver Operating Characteristic (ROC) Curves. Medical Decision Making. 1984;4(2):137–150. doi: 10.1177/0272989X8400400203. [DOI] [PubMed] [Google Scholar]

RESOURCES