Automated Identification of Optimal Portal Venous Phase Timing with Convolutional Neural Networks

Abstract

Objectives:

To develop a deep learning-based algorithm to automatically identify optimal portal venous phase timing (PVP-timing) so that image analysis techniques can be accurately performed on post contrast studies.

Methods:

681 CT-scans (training: 479 CT-scans; validation: 202 CT-scans) from a multicenter clinical trial in patients with liver metastases from colorectal cancer were retrospectively analyzed for algorithm development and validation. An additional external validation was performed on a cohort of 228 CT-scans from gastroenteropancreatic neuroendocrine cancer patients. Image acquisition was performed according to each centers’ standard CT protocol for single portal venous phase, portal venous acquisition. The reference gold standard for the classification of PVP-timing as either optimal or non-optimal was based on experienced radiologists’ consensus opinion. The algorithm performed automated localization (on axial slices) of the portal vein and aorta upon which a novel dual input Convolutional Neural Network calculated a probability of the optimal PVP-timing.

Results:

The algorithm automatically computed a PVP-timing score in 3 seconds and reached AUC of 0.837 (95% CI: 0.765, 0.890) in validation set and 0.844 (95% CI: 0.786, 0.889) in external validation set.

Conclusions:

A fully automated, deep-learning derived PVP-timing algorithm was developed to classify scans’ contrast enhancement timing and identify scans with optimal PVP-timing. The rapid identification of such scans will aid in the analysis of quantitative (radiomics) features used to characterize tumors and changes in enhancement with treatment in a multitude of settings including quantitative response criteria such as Choi and MASS which rely on reproducible measurement of enhancement.

Introduction

Most abdominal CT-scans are acquired after contrast enhancement at the “portal venous phase” (PVP). The PVP acquisition usually uses a fixed delay time after contrast injection. This parameter is set based upon the CT scanner characteristics and is not tailored for a patient’s body habitus or cardiovascular system (1). This leads to a variability in timing and enhancement of the PVP acquisition. The variability is especially apparent in multicenter clinical trials where different vendors’ equipment is used and potentially different image acquisition protocols are employed. Additionally, in many of these clinical trials as well as in clinical practice, the acquisition of serial scans in the same patient to assess for change over time requires an understanding of the variability and its impact on measurements of response and progression. There is no reliable way to quantitatively and automatically recognize the quality of the timing of acquired images. This study aimed to design a deep learning algorithm to automatically recognize optimal PVP acquisitions on CT-scans to ascertain image quality so that the assessment of density-based biomarkers will be more reliable (2).

Optimal PVP timing is crucial in oncology for the automatic detection and characterization of lesions (3), as well as the estimation of tumor enhancement or vascularity, which is increasingly being used to predict treatment response, as well as outcome and recurrence (2, 4–6). In clinical trials and routine practice, anti-cancer treatment efficacy may be evaluated using change in tumor size (response evaluation criteria in solid tumors: RECIST) (7, 8), or change in tumor size and/or density (Choi and MASS criteria) (9–12). RECIST criteria define treatment response by tumor shrinkage, while Choi criteria specify that a 15% decrease in tumor density on CT-scan acquired at the PVP could be an additional surrogate marker of treatment efficacy (10–12). The incremental value of Choi criteria as compared to RECIST is in the detection of additional patients with clinical benefit, and the response rate can double (from 46% using RECIST to 83% using Choi criteria) (13).

Previously, a visual assessment quality control (QC) tool, based on a quality control scoring system differentiating early, optimal, and late PVP, was demonstrated to greatly improved the estimation of cancer drugs’ efficacy using Choi criteria (14). The application of this QC for the evaluation of anticancer treatment efficacy demonstrated that a non-optimal PVP timing (too early or too late) significantly altered response evaluation when using the Choi criteria. A non-optimal PVP-timing induced an apparent, false 15% decrease in the measurement of tumor density. This study demonstrated the actual clinical utility of such QC tools. These previously designed QC tools used two different methodologies, both of which had limitations: (1) radiologists’ visual assessment or (2) a semi-automatic classification software (14). The major limitation of the visual assessment is that it is highly reader dependent and subjective even after appropriate reader training. The ability to manually perform this type of QC depends on readers’ skills and experience. Additionally, both visual assessment by a radiologist and the semi-automatic classification software can be time intensive due to the multitude of tasks that must be performed including optimizing viewing window width/level, and visually estimating density relationships between the relevant organs (or putting region of interests (ROIs) on the organs to calculate density ratios).

In the new era of radiomics and deep learning, convolutional neural networks (CNN)s are highly effective for automated detection and classification tasks such as lung nodule detection (15, 16), differentiation of liver masses (17), assessing skeletal maturity (18), and anatomical recognition (19). Our hypothesis was that a deep learning method can automatically process the information embedded in CT images without the need for visual assessments and the manual selections of ROIs used in the previous machine learning algorithm (14). The purpose of this study was to design a deep learning algorithm to recognize optimal PVP acquisitions on CT-scans, PETAL (Portal vEin & aorTa based Automatic quaLity control), to evaluate if a PVP-timing acquisition is optimal or non-optimal, so that other sensitive algorithms and techniques can be evaluated with optimal quality scans that require accurate assessment of tumor enhancement over time.

Materials and Methods

Patient image data

This retrospective study was waived by our institutional review board because only de-identified CT-scan images were utilized. We selected liver metastasis from colorectal cancer (LM-CRC) to train and validate our PETAL algorithm, as colorectal cancer is the second leading cause of cancer death and liver metastasis involve greater than half of the patients with colorectal cancer (20). To study our algorithm’s generalizability, we chose liver metastasis from gastroenteropancreatic neuroendocrine tumors (LM-NET) as external validation dataset. The gastroenteropancreatic NET is indeed associated with prolonged survival and relatively high prevalence (21–23). In the training and validation sets, we included all LM-CRC patients. No patients had previous hepatectomy in the LM-CRC datasets: all patients were non-resectable. In the external validation dataset, we excluded 25 out of 99 patients with hepatectomy or any severe mass effect with portal vein anatomy changed, because several LM-NET patients had performed previously severe cytoreduction surgery (including hepatectomy) that are very specific to LM-NET and could affect the detection of portal vein. Additionally, the size and anatomic distribution of lymphadenopathy of LM-NET patients was different from LM-CRC patients. In colorectal cancer, the average size of the enlarged lymph node caused no mass effect or portal vein distortion. In contrast, with neuroendocrine tumors, several patients initially had such large portocaval lymph node masses that there was compression and inflow/outflow vascular phenomena which dramatically change following therapeutic cytoreduction surgery to the liver.

Both data sets were obtained sequentially from previous, already completed, multicenter clinical studies of colorectal and neuroendocrine tumors. Patients’ CT images were acquired with standard-of-care abdominal imaging protocols at the PVP after intravenous injection of an iodinated contrast-enhancement product. The PVP acquisition used fixed delay time after contrast injection. The CT imaging settings are shown in Table 1.

TABLE 1:

CT Imaging Settings Statistics of LM-CRC and LM-NET Dataset.

Dataset	LM-CRC	LM-NET	p-value
Scanner	GE, Philips, SIEMENS, etc.	GE, Philips, SIEMENS, etc.
Tube voltage Median (Q1, Q2)	120 (120, 120) kVp	120 (120, 120) kVp	0.11
Tube current time Median (Q1, Q2)	156 (121, 200) mAs	169 (114, 212) mAs	0.24
Recon kernel	smooth	smooth
Slice thickness Median (Q1, Q2)	5 (5, 5) mm	3 (2, 5) mm	6.1*10−31
Pixel spacing Median (Q1, Q2)	0.74 (0.68, 0.79) mm	0.71 (0.66, 0.77) mm	0.0035

Q1 is lower (first) quartile and Q2 is upper (third) quartile.

We randomly divided LM-CRC dataset into training and validation sets on a ratio of 7:3 for the deep learning algorithm development. CT-scans from the same patients were assigned to the same group: assigned either to the training set or to the validation set.

Reference standard

For each patient in both LM-CRC and LM-NET datasets, only one pair of radiologists reviewed the images and provided their PVP timing score to create radiologists’ initial visual consensus. Four radiologists interpreted LM-CRC dataset due to a large amount of CTs. Their experience were 3, 5, 8, and 15 years respectively. In the external validation stage, two radiologists with 8 and 12 years of experience interpreted the LM-NET dataset. The reference standard was the three-phase consensus process shown as follow.

Visual Scoring of PVP timing.

We categorized the PVP timing as optimal or non-optimal. Non-optimal PVP timing included either early or late PVP timing. The definition was based on the relative contrast enhancement within vessels and tissues (24–28). Optimal PVP demonstrated peak enhancement of the liver parenchyma and portal vein, as well as some enhancement of the hepatic veins. Early PVP was defined by contrast still predominantly in the arterial supply as compared with the portal vein. Late PVP was associated with a washout of the hepatic contrast enhancement and approached the nephrogenic phase with more enhancement of the renal medulla. Six examples from three patients are shown in Figure 1.

Fig. 1— Clinical implication of intrapatient variability in Portal Venous Phases timing using CT images acquired at baseline and after treatment initiation.

Optimal PVP timing (A left and C right) was defined by peak enhancement of the liver parenchyma and portal vein, as well as some enhancement of the hepatic veins. Patient A was treated for a NET. At baseline (A, left), the biodistribution of the contrast enhancement product was optimal and significantly differed from the subsequent follow-up (A, right) which was non-optimal as demonstrated by the contrast still being predominantly in the arteries. This might mimic treatment effect if tumor density is used as a response criterion (e.g., Choi, MASS, mRECIST). Patient B was treated for a CRC, at baseline (B, left), the PVP timing was non-optimal because the contrast enhancement in the liver parenchyma and portal vein was low due to the washout of the contrast enhancement. The acquisition after treatment initiation (B, right) was non-optimal as explained by the fact that the contrast agent was still predominantly in the arteries. Patient C was treated for a NET. The PVP timing was non-optimal at baseline(C, left) and optimal during the follow-up (C, right). This will lead to an increase in liver and tumor density at follow-up due to acquisition protocol and might mask treatment effect if tumor density is used as a response criterion.

(1). Initial Visual Consensus.

Each CT scan was initially scored by two radiologists. The radiologists independently read their CT scans and scored the PVP timing. If the two radiologists’ scores agreed, then the score was finalized. If the two radiologists disagreed, then a joint consensus reading was performed between both radiologists after a time interval to avoid memory bias. If the two radiologists could not reach an agreement during the consensus reading, then a third radiologist adjudicated to reach a majority consensus for the score.

(2). Semi-automatic classification.

Each CT scan was also scored by a semi-automatic classifier. We used a previously described semi-automatic classification of PVP timing in (14) to ensure the quality and reproducibility of our PVP timing classification. To this end, we used a previously described probability map that the PVP timing was optimal based on the average density (in Hounsfield Unit) of the image pixels inside a manually drawn ROI in the portal vein and in the aorta as well (14). The details of semi-automatic classification method were shown in the supplemental materials S1.

(3). Final (including 1 and 2) Reference standard Consensus.

For any semi-automatic classification that differed from the initial visual consensus, an additional consensus reading was performed by the pair of radiologists to determine whether the optimal PVP timing classification, after the semi-automatic classification scores were shown to the pair of radiologists. The goal was to obtain the most accurate and optimal reference-standard PVP-timing consensus for subsequent analyses. In summary, this was derived from (1) a visual consensus between a pair or trio of radiologists, (2) a semi-automatic classification of all scans to find discrepant scans of which computer based classifications were different from radiologists’ visual consensus, and (3) a final visual consensus by the radiologists for any discrepant scan.

Algorithm building

The contrast enhancement in portal vein and aorta was demonstrated to be the dominant information that contributed to classify the optimal PVP-timing as the ROIs in liver, spleen, inferior vena cava, psoas muscle, and kidney did not provide incremental value (14). With this knowledge, an automated algorithm for deriving a probability of optimal PVP-timing by analyzing the portal vein and aorta was developed with the following 6 steps (Fig. 2).

Fig. 2— This graph illustrates the pipeline of our proposed PETAL algorithm.

Step 1.

Input data for the algorithm was the volumetric data (voxel size is N*512*512, where N is the number of slices) from a contrast enhanced abdominal CT scan.

Step 2.

Identification of CT slices containing portal vein was determined using the Alex-net (29) which pushed classification accuracy of the ImageNet Challenge (30) by a significant stride in comparison to traditional methodologies. By training the first CNN, Alex-net, our algorithm predicted the probability of appearance of portal vein in every CT slice. Consecutive CT slices with probabilities greater than 0.5 were determined as the axial slices containing portal vein and then fed in step 3 and step 4. These axial slices also contained aorta according to the anatomy. An example is shown in Figure 3b. The rest slices were excluded for the subsequent steps to reduce false positives detections of portal vein and aorta.

Fig. 3— An example case predicted by PETAL.

(a) All the image series of the input CT. The AlexNet identified the slices containing portal vein shown in the red boxes. (b) the axial slices containing portal vein with AlexNet. Faster R-CNN detects the bounding-boxes of portal vein (green boxes) and aorta (blue boxes), which are shown in (c) and (d) respectively. Finally, PVP score is predicted based on the detected portal vein and aorta bounding-boxes with our proposed dual input CNN.

Step 3.

Detection of portal vein’s bounding-boxes was performed next with another CNN, the Faster R-CNN (31) which is a deep-learning based object detection system with high performance to run at near real-time frame rates. Figure 3c illustrates the detection of the portal vein. The bounding-box of portal vein contained portal vein and its adjacent tissues, such as a part of the liver. At most one bounding-box was created for each slice identified by the step 2. For each cases, if the number of the bounding-boxes was less or equal to five, all the bounding-boxes were retained. If the number is greater than five, top five bounding-boxes with highest probability values were retained and the rest bounding-boxes were excluded. A case is shown in Figure 4. These retained bounding-boxes were used to feed in step 5.

Fig. 4— An example of the automatically portal vein and aorta detections.

The images show a case where PETAL detected top five highest probabilities bounding-boxes from five slices for both portal vein and aorta. The bounding-boxes in the same column were in the same pair to feed in dual input CNN.

Step 4.

Detection of aorta’s bounding-boxes was identical to the portal vein localization in the previous step. Figure 3d showed a case to illustrate the bounding-box detections of the aorta.

Step 5.

The optimal PVP-timing was automatically classified based on the pair of portal vein’s bounding-box and aorta’s bounding-box attained in step 3 and 4. Portal vein bounding-box paired with aorta bounding-box according to the probability order. If the number of portal vein bounding-box was less than the number of aorta bounding-boxes, the portal vein bounding-box with the highest probability would be replicated multiple times to equal the number, and vice versa. Here, we developed the third CNN, a new dual input CNN as shown in Figure 5, to process the image patches of portal vein and aorta. Each score can be obtained by analyzing each pair of portal vein bounding-box and aorta bounding-box.

Fig. 5— Proposed dual input CNN architecture.

Input patches are processed by convolutional and max-pooling layers. The output PVP score is processed with the concatenation of portal vein and aorta information and fully connected layers. In this network, two blocks of two consecutive convolutional layers with 32 and 64 filters were used. Each filter had 3-by-3 pixels convolutional kernels with ReLU activation, zero padding on the edge. Each block was followed by a 2-by-2 max-pooling layer and a dropout layer of dropout rate 50%. Finally, the fully connected layer of 512 nodes with ReLU activation combined the information about the portal vein and the aorta. The final 1 node layer was connected with the previous layers with sigmoid activation.

Step 6.

Output of final PVP probability was the average of five scores resulted from five pairs of portal vein’s and aorta’s bounding-boxes.

Implementation details of PETAL were shown in the supplement materials S2.

Performance Evaluation

We used Free-Response ROC (FROC) to evaluate the detection performance of the portal vein and aorta. All the bounding-boxes of PETAL were inspected by one radiologist and were classified into true positives or false positives. A true positive detection (hit) was defined by the bounding-box being centered on the target (portal vein or aorta). Otherwise, the bounding-box was considered as a false positive detection. The sensitivity and false positive per scan of the FROC were defined as following:

$$\text{Sensitivity}=\frac{\text{\#}casesthatatleastoneboundingboxhitthetarget}{\text{\#}totalcases}$$

$$\text{False positive per scan}=\frac{\text{\#}boundingboxesthatdidnothitthetarget}{\text{\#}totalcases}$$

The performance of PETAL was evaluated in terms of area under the curve of ROC curve (AUC). The ROC curve was calculated by comparing the output probabilities with the reference standards. The real positive case was a case of which the reference standard was optimal PVP and the real negative case was a case of which the reference standard was non-optimal PVP. The sensitivity and specificity in the ROC were defined as following:

$$\text{Sensitivity}=\frac{\text{\#}truepositivecases}{\text{\#}realpositivecases}$$

$$=\frac{\text{\#}casesthatbothPETALoutputsandreferencestandardswereoptimalPVP}{\text{\#}totalcasesthatreferencestandardswereoptimalPVP}$$

$$\text{Specificity}=\frac{\text{\#}truenegativecases}{\text{\#}realnegativecases}$$

$$=\frac{\text{\#}casesthatbothPETALoutputsandreferencestandardswerenon-optimalPVP}{\text{\#}totalcasesthatreferencestandardswerenon-optimalPVP}$$

The 95% confidence interval of AUC was estimated by using bootstrapping with 1000 bootstraps of prediction scores. We collected two cohorts of data independently. The LM-CRC cohort was used for training and validating PETAL and the LM-NET was used for testing PETAL blindly.

Results

In total, 479 PVP CT-scans (300 optimal and 179 non-optimal) from LM-CRC dataset were used for training and 202 PVP CT-scans (137 optimal and 65 non-optimal) from LM-CRC dataset were used for validation. The external validation set utilized LM-NET dataset with 228 PVP CT-scans (111 optimal and 117 non-optimal).

Of note, PETAL algorithm consisted with six steps. The errors in the previous steps would affect to the subsequent steps, so all results contained the propagating errors from the prior steps.

Overall optimal PVP classification

The optimal PVP probability of PETAL outputs ranged from 0 to 1. In the external validation set, the optimal threshold of the PVP probability was 0.84. At this operating point, the sensitivity, specificity, positive predictive, and negative predictive values were 82%(91/111), 74%(87/117), 75%(91/121), and 81%(87/107), respectively. At the operating point of 0.91, the optimal threshold on the validation set, the sensitivity, specificity, positive predictive, and negative predictive values of external validation set were 78%(87/111), 77%(90/117), 76%(87/114), 79%(90/114), respectively. Good performances were also observed in the validation set, the LM-CRC dataset, with an AUC of 0.837 (95% CI: 0.765, 0.890) and in the external validation set, the LM-NET dataset that was a different tumor type than the training set, with an AUC of 0.844 (95% CI: 0.786, 0.889) (Fig. 6). There was no significant difference between colorectal cancer and neuroendocrine cancer in terms of AUC. It is likely that these two different cancer types do not vary the pharmacokinetics of contrast media in the vessels we used in this study. Additionally, PETAL algorithm was robust to the CT slice thickness and pixel spacing which were statistically significant different between the LM-CRC and LM-NET datasets as shown in Table 1.

Fig. 6— Performance evaluations with FROC and ROC curves.

(a) FROC of portal vein detection that shows PETAL achieved 89.5% sensitivity with 1.08 false positive result per scan in the external validation set. (b) The aorta detection was almost perfect since it achieved 98.2% sensitivity with 0.0132 false positive results per scan. (c) The ROC curve of optimal PVP classification of validation set, AUC of 0.837 (95% CI: 0.765, 0.890) and the ROC in the external validation set, AUC of 0.844 (95% CI: 0.786, 0.889).

Intermediate results

(1). Portal vein slices classification.

In the external validation set, four cases failed to locate the axial slices containing portal vein (1.75%) due to poor contrast in the PV (n=1), displaced portal vein caused by mass effect (n=2) and systematic failure of the algorithm (n=1, contrast enhancement was unexpectedly high in the PV with 360 HU, which is above our abdominal CT window of −160~240 HU).

(2). Portal vein detection.

PETAL algorithm extracted totally 1,046 bounding-boxes of portal veins from 228 cases in the external validation set and received 89.5% sensitivity with 1.08 false positive per scan. The FROC curve is shown in Figure 6a. One radiologist analyzed the detections of 1,046 bounding-boxes visually. There were some errors due to selection of multiple vessels (especially in the coeliac trunk 93 cases, 8.9%), detections of bone vertebra or rib (71 cases, 6.8%), detection of structures with oral contrast enhancement (opacification digestive tract, 18 cases, 1.7%), and selection of normal structures such as liver, kidney, stomach (65 case, 6.2%).

(3). Aorta detection.

The aorta detection was almost perfect; it achieved 98.2% sensitivity with 0.0132 false positives per scan. We analyzed the detections of all 1,046 bounding-boxes in the external validation set. There were three mistakes out of 1,046 cases, two of which were caused by oral contrast enhancement and one was caused by selection of multiple vessels in the coeliac trunk.

Duration of data processing

PETAL algorithm can evaluate a CT scan in 3 seconds on average by a computer with a GTX 1060 GPU and an Intel i7–7700HQ 2.8GHz CPU. The algorithm was fully automatic. We applied PETAL algorithm to the LM-NET dataset twice and got exactly the same probability of being optimal PVP for every case, because the algorithm did not contain human interactions or random factors.

Discussion

In this study, we developed a deep learning method to identify CT-scan acquisitions with an optimal PVP-timing, based on a fully automated analysis of CT scan images. This method first screens the entire abdominal CT scan, proposes five bounding-boxes of the aorta and five bounding-boxes of the portal vein, and finally outputs a probability of the CT scan being optimal PVP; all automatically in 3 seconds. This algorithm was developed and validated with CT scans from two multi-center clinical studies and involving two different primary tumor types, colorectal metastases which are generally hypointense on the PVP phase and neuroendocrine metastases which are generally hyperintense. The performance of the algorithm was comparable with the radiologists for the visual assessment of the PVP timing, which was reported to be 81.7% accuracy in (14). It achieved an AUC of 0.837 (95% CI: 0.765, 0.890) in the validation set and an AUC of 0.844 (95% CI: 0.786, 0.889) in the external validation set. Thus, the deep learning method was able to differentiate an optimal PVP-timing vs. a non-optimal PVP-timing (early or late PVP) acquisition. This algorithm can assist in determining whether an intended PVP acquisition is optimal and can also help researchers by identifying non-optimal PVP acquisitions on retrospective studies, which they may consider evaluating separately. In this study, 43%(87/202) LM-CRC cases and 50%(114/228) LM-NET cases were recognized as non-optimal PVP by this algorithm. However, the patients with non-optimal PVP scans may have heart failure or other conditions that negatively affect studies. It still need to be studied in the future by using the quality of the PVP as a new factor in statistical analysis or as a new variable to integrate into radiomics model (normalizing features or model building).

This algorithm potentially provides additional and incremental value to the current bolus tracking technique used in CT imaging. Bolus tracking is used during the imaging acquisition to control phase timing so that the likelihood of the PVP-timing being optimal is increased. However, it does not consider the individual patient’s biological variation and thus cannot ensure if the optimal PVP-timing was successfully reached in a given patient (24, 32–35). In bolus tracking, a small bolus of radio-opaque contrast media is injected into a patient via a peripheral intravenous cannula. The flow of contrast to a specific vessel can be tracked using an ROI. When the contrast in the vessel reaches a certain threshold, the acquisition timing is considered optimal and the CT-scan acquisition is triggered. Additionally, bolus tracking is not used in all center and most centers still rely on image acquisition on the historical and empirical 60-second delay between contrast-agent product injection and image acquisitions. The problem is that the individual optimal hepatic enhancement timing cannot be assumed a priori, because it is associated with multiple variables as well as the anthropomorphic characteristics and hemodynamic status of the patient (36–38). Consequently, an optimal PVP timing is only reached in two out of three patients in multicenter clinical trials (14). Hence, the need for such quality control techniques is critical.

This algorithm may help to ensure an optimal extraction of imaging biomarkers that could be useful to classify new patterns of progression (39–41) and response (42–44) in novel anticancer agents by automatically identifying optimal PVP timing and potentially excluding imaging data sets with improper acquisition timing. It could also be applied to improve radiomics signatures by selecting CT images of optimal quality (45–50). The algorithm is capable of ensuring that conditions are optimal to appraise tumor vascularity and density-based response criteria such as CHOI (9–11). A previous study (14) demonstrated that a 14.8% variability in tumor density measurement was directly caused by variation in PVP acquisition timing which is deleterious to the use of density as a biomarker, because tumor density reduction would not be robust as its variability could be due to acquisition timing (rather than tumor lesion biology) and would therefore be poor at predicting overall survival and progression-free survival (9–12). Increasing the reproducibility of tumor density measurements is essential to acceptance of density-based biomarkers as the basis for imaging endpoints in clinical trials and response in new anticancer agents. Identifying optimal PVP enhancement may also benefit some organ segmentations due to increasing the contrast between the target organ and the background. For the patients studied over time, the consistency of PVP enhancement can make the assessments more reliable.

There are some limitations in this work. First, the gold standard of optimal PVP-timing for each patient’s CT scan required the creation of a consensus reference standard based on the subjective visual assessment of trained radiologists and the objective computer-aided classification. Second, PETAL cannot be applied to the patients with major hepatic surgery or any severe mass effect with portal vein anatomy changed. Finally, the overall performance was limited by the accuracy of portal vein localization. Since the algorithm consists of multiple parts, an error in any one part will propagate thru the entire algorithm. The greatest challenge in the algorithm was in the portal vein localization, for which we will continue to optimize.

Conclusion

Our work demonstrates that a fully automatic, deep-learning derived PVP-timing recognition system can reliably and rapidly identify the optimal PVP-timing based on CT images. This technique could be invaluable for any type of image analysis, such as radiomics, which rely on features that might be affected by the phase of contrast administration and scanning.

Abbreviations:

AUC

area under the curve

CI

confidence interval

CNN

convolutional neural network

CT

computed tomography

LM-CRC

liver metastases from colorectal cancer

LM-NET

liver metastases from neuroendocrine tumor

PVP

portal venous phase