Computer-aided detection of mantle cell lymphoma on ¹⁸F-FDG PET/CT using a deep learning convolutional neural network

Abstract

¹⁸F-FDG PET/CT can provide quantitative characterization with prognostic value for mantle cell lymphoma (MCL). However, detection of MCL is performed manually, which is labor intensive and not a part of the routine clinical practice. This study investigates a deep learning convolutional neural network (DLCNN) for computer-aided detection of MCL on ¹⁸F-FDG PET/CT. We retrospectively analyzed 142 baseline ¹⁸F-FDG PET/CT scans of biopsy-confirmed MCL acquired between May 2007 and October 2018. Of the 142 scans, 110 were from our institution and 32 were from outside institutions. An Xception-based U-Net was constructed to classify each pixel of the PET/CT images as MCL or not. The network was first trained and tested on the within-institution scans by applying five-fold cross-validation. Sensitivity and false positives (FPs) per patient were calculated for network evaluation. The network was then tested on the outside-institution scans, which were excluded from network training. For the 110 within-institution patients (85 male; median age, 58 [range: 39-84] years), the network achieved an overall median sensitivity of 88% (interquartile range [IQR]: 25%) with 15 (IQR: 12) FPs/patient. Sensitivity was dependent on lesion size and SUV_max but not on lesion location. For the 32 outside-institution patients (24 male; median age, 59 [range: 40-67] years), the network achieved a median sensitivity of 84% (IQR: 24%) with 14 (IQR: 10) FPs/patient. No significant performance difference was found between the within and outside institution scans. Therefore, DLCNN can potentially help with MCL detection on ¹⁸F-FDG PET/CT with high sensitivity and limited FPs.

Introduction

Mantle cell lymphoma (MCL) is a subtype of non-Hodgkin lymphoma with distinct histological and diagnostic characteristics [1]. It can have a range of aggressiveness and frequent extranodal involvement, especially in the gastrointestinal tract or bone marrow [2]. ¹⁸F-FDG PET/CT is widely used for MCL treatment response assessment [3-7]. Increasing evidence shows that quantitative metrics from PET/CT images, including maximum standardized uptake value (SUV_max) and metabolic tumor volume of the lesion with the highest SUV_max, can provide detailed characterization of MCL and are of strong prognostic value [8]. However, such quantitative analysis of PET/CT requires detecting all MCL lesions, which can be labor-intensive, time-consuming, and subject to inter-reader variability. The wide range of the lesion aggressiveness, size, and frequent extranodal involvement present additional challenges for manual detection of MCL [9].

Recently, a machine learning approach was reported for detecting bone marrow involvement of MCL [10]. However, this study was based on pelvic PET/CT images only. A deep learning approach has been developed and applied to classify positive FDG uptake regions on PET/CT images as lymphoma or lung cancer [11]. This approach reached a sensitivity of 75.4% for lymphoma detection, and image cropping and foci atlas positions were required during its application. Several deep learning approaches have been recently implemented for lymphoma detection and segmentation in whole-body PET/CT images [12-14]. However, these studies included mostly diffuse large B-cell lymphoma, which typically has larger lesion sizes and more intense signals on PET images compared with MCL. Therefore, detection of MCL is more challenging. To our knowledge, applying deep learning for MCL detection in whole-body PET/CT images has not been reported.

Among the known deep learning network structures, U-Net has been widely adapted for object detection or segmentation because of its robust performance [15]. Typically, U-Net and its adapted versions contain two convolutional pathways: a convolutional encoder and a decoder. The encoder converts input images into latent space feature maps, and the decoder translates the feature maps back to image space. By concatenating the feature maps of the same resolution from the encoding and decoding pathways, the network can be trained to map the input images to pixel-wise classification masks, with each pixel having an associated probability of belonging to a certain tissue type or the background. If a group of pixels in a lesion are correctly classified, the lesion can then be considered detected. In this study, we constructed a U-Net–based deep learning convolutional neural network (DLCNN) for computer-aided detection of MCL in ¹⁸F-FDG PET/CT images.

Materials and methods

Institutional Review Board approval was received, and written informed consent was waived for this retrospective study. This study was compliant with the Health Insurance Portability and Accountability Act.

Study participants

Baseline ¹⁸F-FDG PET/CT scans of 165 patients with biopsy-confirmed MCL from May 2007 to October 2018 were selected, and 142 of these were included for our study. Scans were excluded if (1) the PET and CT slice numbers were different, (2) either the PET or CT images could not be extracted from the picture archiving and communication system, or (3) the reference standard contour cannot be established (Figure 1).

Figure 1.

Flowchart of patient inclusion and exclusion of the study.

The scans included were acquired at multiple centers: 110 at our institution and 32 at outside institutions. The 32 outside-institution scans were acquired at 32 institutions in the United States, with one scan from each institution. Of the 142 scans, 117 were acquired on GE PET/CT systems (GE Healthcare, Waukesha, WI), and 25 were acquired on Siemens PET/CT systems (Siemens Healthineers, Erlangen, Germany). The detailed scanner models were listed in Table 1. Typical CT scan parameters were: kVp = 120 V, slice thickness = 3.27 mm, field of view = 50 × 50 cm², and matrix size = 512 × 512. The images were reconstructed using soft tissue or body contrast filters, with display window center = 40 HU and window width = 400 HU. Typical PET scan parameters were: field of view = 70 × 70 cm², matrix size = 192 × 192 or 128 × 128. The ¹⁸F-FDG dose was 370 to 555 mBq for patients with a body mass index under 40.

Table 1.

Characteristics of patients whose scans were taken within and outside our institution

Characteristic	Within-institution group	Outside-institution group	P
Patients	110	32	-
Age, median (range), y	58 (39-84)	59 (40-67)	0.89
Sex			0.79
Male	85	24
Female	25	8
Body mass index, median (IQR)	27.8 (6.8)	27 (6)	0.24
Scanner vendor			0.85
GE	91	26
Discovery 600		1
Discovery 690		1
Discovery 710	46	2
Discovery IQ		5
Discovery MI	4	1
Discovery RX	16
Discovery ST	1	10
Discovery STE	24	5
Optima 560		1
Siemens	19	6
Biograph 6 TruePoint		2
Biograph 16		1
Biograph 40 mCT	1
Biograph 40 TruePoint		1
Biograph 64 mCT	18	1
SOMATOM Definition AS mCT		1
Reference SUV, median (IQR)	1.51 (0.41)	1.43 (0.41)	0.23
Lesions	3473	625
Lesions per patient, median (IQR)	22 (48)	13 (23)	0.11
Lesion SUVmax, median (IQR)	2.76 (1.99)	2.46 (1.21)	< 0.01
< Reference	387	57
≥ Reference	3086	568
Lesion size, median (IQR), mm	18 (14)	23 (16)	< 0.01
< 10 mm	736	36
≥ 10 mm	2737	589
Lesion location			< 0.01
Nodal	3367	573
Non-nodal	106	52
GI tract	86	14
Bone	19	38
Other	1	0
Splenomegaly	37	8

Abbreviations: SUV, standardized uptake value; IQR, interquartile range; GI: gastrointestinal.

PET/CT image analyses

A total of 4098 MCL lesions of the 142 patients were manually identified and contoured by three board-certified nuclear medicine physicians (G.X., Y.L., and H.M., each with more than 10 years of experience). After retrospectively examining the PET/CT images and prior reports, lesion contours were drawn on baseline PET images as the reference standard for DLCNN training and evaluation. All physicians used MIM software (v6.6, Beachwood, OH) to delineate lesions on PET. Specifically, each lymphoma lesion was first identified by physicians using the PET Edge tool of MIM, which was a gradient-based technique that detected the steepest SUV drop to create initial contour boundaries. Based on the co-registered CT images, the physicians then revised the contours on each 2D axial slices, forming a 3D contouring volume for each lesion. Although only the baseline PET/CT scans were used for DLCNN training, the physicians also referred to post-therapy scans and pathological reports to confirm the diagnosis: suspicious loci seen on the baseline scans that had resolved on the post-therapy scans or been biopsied with MCL pathological findings were confirmed as lesions. A consensus read was made between the physicians if any disagreement of lesions occurred.

Data curation

The DLCNN was trained and first tested on images from within our institution. The outside-institution images were excluded from network training and were reserved for additional testing of the trained network. Because of the relatively small patient sample, we chose to use five-fold cross-validation for the training and initial testing. The 110 within-institution scans were randomly split into five folds, with each fold containing 17 male and 5 female patients. While one fold was selected for independent testing, the other four folds were combined and randomly split as the training and internal validation datasets with a ratio of 80%:20%. The training and internal validation datasets were used for network training and hyperparameter tuning, respectively.

To use the interslice information of the imaging volumes, we constructed our network to simultaneously take three successive axial slices of both PET and CT as input. The output was a lesion/non-lesion binary classification map for the center PET/CT slice, with the associated classification probability for each pixel. We adopted the following sliding-window strategy for data curation. First, the PET and CT images of each patient were aligned, cropped, and resized to the same matrix size of 128 × 128, with a field of view of 50 × 50 cm². Second, three-slice sliding windows were applied for the PET and CT images along the axial dimension, forming three-slice image slabs. Finally, the PET and CT slabs were separately normalized between [0, 1].

Network constructions

The DLCNN was constructed using Keras Python Library [16]. The encoder structure used in our study was an Xception-based network [17]. While other network structures such as VGG, ResNet, and DenseNet are also widely used, Xception has been reported to have 8%, 2%, and 2% improvement in image classification performance over VGG16, ResNet152, and DenseNet201, respectively [18-20]. Therefore, we decided to construct the encoder based on Xception, where the depth-wise separable convolution and residual connection were critical for its improved performance.

Our constructed encoder had 124 layers in total, including 34 depth-wise separable convolution layers and 12 residue addition layers. Different from the original Xception network, we modified the first convolutional layer with a stride size of 1. This modification permits the network to generate a shallow feature map with the same resolution of the input image, which can be directly used for mask prediction. Because the data contained PET and CT, we constructed two encoder channels to simultaneously process the PET and CT images. For the decoder pathway, instead of simply passing the PET and CT feature maps individually back to the top, the feature maps were also combined along the pathway. Detailed structures of the DLCNN is shown in Figure 2.

Figure 2.

Framework of the constructed DLCNN (left) and a convolutional block of the Xception encoder (right). The inputs are the curated PET and CT images. For each input channel, resolutions of the feature maps after the ReLU activation layers are 128 × 128, 64 × 64, 32 × 32, 16 × 16, and 8 × 8. Starting from the bottom layer, the feature maps are first resized, convoluted, and concatenated with the upper level feature maps from each channel. The concatenated feature maps from the PET and CT channels are then joined to form the PET/CT feature map, which was resized, convoluted, and connected with the next level feature maps.

Training configurations

Image augmentation using random affine transformations was applied during network training. Transformations included horizontal flip, translational move, and image zoom in or out with a scale of 0.8 to 1.2. The training dataset was augmented by three folds. Because the negative to positive voxel ratio was around 250:1, weighted binary cross-entropy was used as the loss function for training. By applying the weights, extra loss was added if the network miss-predicted the positive voxel. The loss was optimized using an Adam optimizer with the initial learning rate set to 2 × 10^-4 [21]. The rate was adaptively decreased by 20% if the validation loss did not improve for three consecutive training epochs. The training procedure was terminated if the validation loss did not improve for seven consecutive epochs.

Network evaluation

Performance of the DLCNN was evaluated by computing sensitivity and false positives (FPs) per patient. Because the network was intended to assist physicians detecting as many lesions as possible or draw attention to suspicious regions, the lesion was considered true positive (TP) if at least one of its voxels was detected [22]. The predictions were FPs if they had no overlap with any lesions. Sensitivity was defined as the ratio between the number of TPs and all lesions of the patient. To address potential pitfalls of the one-voxel metric, for each detected lesion, we also examined the actual percentage of its detected voxels, and the Sørensen-Dice similarity coefficient (DSC) of the prediction generated by the network.

On the contrary, voxel-based specificity would be inflated and misleading, because of the large imbalance between negative and positive voxels. Therefore, specificity of the network was evaluated on the anatomic level. Organs had physiologic FDG avidities were considered true negatives (TNs) if there were no associated lesion predictions. Included organs were the brain, heart, liver, kidneys, and bladder. For each organ, specificity was defined as the ratio between the number of TNs and the number of testing patients. Finally, after each fold’s training, the DLCNN was also tested on all outside-institution scans to verify its robustness and generalization.

Statistical analysis

Categorical characteristics of the within- and outside-institution patients were compared using chi-square tests. Because of the small sample size of the outside-institution patients, continuous variables were compared using non-parametric Wilcoxon rank sum test. For the network performance, because the voxel classification output was binary (lesion vs. non-lesion), we first chose a probability threshold of 50% to assess the detection. The median and interquartile range (IQR) were used to evaluate the network’s performance for each fold. Again, due to the small sample size of each testing fold and the outside-institution group, performance of the network between the within- and outside-institution scans were compared using Wilcoxon rank sum test.

For the within-institution scans, sensitivity of the network was also compared between lesion size (< 10 mm vs. ≥ 10 mm), SUV_max (below vs. above the reference SUV), and lesion location (nodal vs. extranodal) using independent t tests. The reference SUV of each patient was the average SUV of a round patch at the upper right lobe of the liver with physiological activities. Finally, free-response receiver operating characteristic was plotted to inspect the network’s performance at probability thresholds other than 50%.

Ablation study

Besides using both PET and CT images for network training and testing, we also conducted experiments using the curated PET images only. Using the same training and testing configurations, the study was performed on the same partition of patients. The overall performance was compared to using both PET and CT images. The results were also compared between the within- and outside-institution patients.

Results

Patient characteristics

The 142 patients included in the study had a median age of 58 years (range, 39-84 years), and 109 were male. Detailed characteristics of the patients and lesions, according to the image source, are listed in Table 1. Extranodal lesions were mostly found in the gastrointestinal tract and bone marrow. Splenomegaly was also present in approximately 30% of the patients.

Network performance

The DLCNN training time was around 16 hours for each fold on an Nvidia DGX-1 workstation. The inference time per patient for the trained network was approximately 6 seconds. With the 50% probability threshold, the network detected on average 27 of 32 MCL lesions for each within-institution scan, achieving a median sensitivity of 88% (IQR: 25%) with 15 (IQR: 12) FPs/patient. For the outside-institution scans, the median sensitivity was 84% (IQR: 24%) with 14 (IQR: 10) FPs/patient. Wilcoxon rank sum tests showed that the two metrics were not significantly different between the within- and outside-institution scans, with P-values of 0.20 and 0.56, respectively. Detailed detection results are summarized in Table 2.

Table 2.

Detailed detection results of each fold for the within- and outside-institution scans

Fold	Sensitivity			FPs/patient
	Within institution	Outside institution	P	Within institution	Outside institution	P
Fold 1	88% (12%)	86% (15%)	0.66	15 (17)	16 (12)	0.89
Fold 2	91% (20%)	84% (30%)	0.44	18 (18)	13 (9)	0.08
Fold 3	91% (33%)	84% (21%)	0.38	13 (12)	14 (11)	0.70
Fold 4	88% (25%)	86% (19%)	0.58	12 (10)	14 (13)	0.44
Fold 5	87% (29%)	85% (29%)	0.92	16 (11)	14 (13)	0.17
Overall	88% (25%)	84% (24%)	0.20	15 (12)	14 (10)	0.56

Note: All numbers are medians with interquartile ranges in parentheses. For within-institution scans, the overall median was calculated based on all the 110 scans. For outside-institution scans, the overall median was calculated based on each patient’s five-fold average.

Additionally, sensitivity to detect the top SUV_max lesion in each patient was 93% (102/110) for the within-institution scans, and 88% (28/32) for the outside-institution scans. Two detection examples, one from a study within our institution and one from outside our institution, are shown in Figures 3 and 4, respectively. The free-response receiver operating characteristic of the DLCNN is shown in Figure 5.

Figure 3.

Mantle cell lymphoma detection of a 45-year-old male patient. The scan was acquired within our institution. From left to right, the columns are ¹⁸F-FDG PET image, PET/CT images, reference standard contours, and prediction maps. White arrows on the reference standard images indicate false negatives, orange arrows on prediction maps indicate false positives, and blue arrows indicate over-prediction covering small lesions. The scale bars on the left represent SUV, and those on the right (ranging between 0 and 1) represent pixel-wise probabilities of being lesions. On the coronal slice, there is a false negative in the abdomen. The false positives are believed to be results of elevated physiologic and inflammatory activities.

Figure 4.

Mantle cell lymphoma detection of a 56-year-old male patient. The scan was acquired from an outside institution. From left to right, the columns are ¹⁸F-FDG PET image, PET/CT images, the reference standard contours, and prediction maps. Orange arrows on prediction maps indicate false positives, and blue arrows indicate over-prediction covering small lesions. The scale bars on the left represent SUV, and those on the right (ranging between 0 and 1) represent pixel-wise probabilities of being lesions. All lesions were detected, and a false positive was found near the neck.

Figure 5.

Free-response receiver operating characteristic of the DLCNN. The sensitivity was plotted against the false positives per patient as the confidence threshold decreased from 90% to 10% with a step of 10%. As the threshold decreased, the sensitivity increased at a cost of increased false positives per patient. The error bars in both directions are 95% confidence intervals.

Regarding the one-voxel metric, for each detected lesion of the within-institution scans, the median percentage of the lesion’s detected voxels was 99% (IQR: 17%), and the median DSC of the prediction was 0.42 (IQR: 0.35). These two metrics for the outside-institution scans were 95% (IQR: 29%) and 0.46 (IQR: 0.33). Each lesion’s voxel detection percentage and each prediction’s DSC were plotted in Figures S1 and S2. The relatively limited prediction DSC indicates that the network still requires physicians’ attention. However, the high lesion voxel identification percentage indicates that the network can reliably detect MCL lesions.

For the specificity, several patients had organs with normal physiologic activities partially identified as FPs. On average, each fold had no patients with FPs in the brain, two in the heart, two in the liver, five in the kidney, and one in the bladder. Therefore, specificity for these organs with substantial background activities was 100% (22/22), 91% (20/22), 91% (20/22), 77% (17/22), and 95% (21/22), respectively.

Independent t tests showed that sensitivity of the network was significantly different between lesion size and SUV_max (both P < 0.01). As expected, lesions with larger sizes and higher SUV_max were easier to be detected. The sensitivity for lesions smaller than 10 mm was 71±32%, while it was 84±19% for those greater than 10 mm. The sensitivity for lesions with SUV_max below the reference was 62±34%, while it was 85±18% for those with SUV_max above the reference. The sensitivity was not dependent on lesion location: it was 82±20% for nodal lesions, and 79±37% for extranodal lesions (P = 0.5).

In the ablation study, using PET images only for network training and testing yielded a median sensitivity of 92% (IQR: 18%) with 21 (IQR: 14) FPs/patient for the within-institution scans, and a median sensitivity of 86% (IQR: 17%) with 19 (IQR: 11) FPs/patient for the outside-institution scans. Compared to using both PET and CT images, the within- and outside-institution scans showed similar results: no significant difference was found for the overall sensitivities (Wilcoxon rank sum test, P = 0.20 and 0.74, respectively), however, the FPs/patient were significantly higher if only the PET images were used (both P < 0.01). Detailed results of the ablation study are summarized in Table S1.

Discussion

Using a modified Xception network and U-Net, we constructed a DLCNN that showed promising performance for computer-aided detection of MCL in whole-body ¹⁸F-FDG PET/CT images. The network achieved high sensitivity with limited FPs/patient. It also had high specificity for the brain, heart, liver, kidney, and bladder, despite these organs’ high FDG avidities. Sensitivity of the network was dependent on lesion size and SUV_max, but not on lesion location.

Performance on outside-institution images revealed that the network was robust and applicable to independent datasets. The patient characteristics were similar between within- and outside-institutions, including the referen-ce SUV and lesions/patient. However, the lesion characteristics were significantly different: the outside-institution lesions had slightly lower SUV_max, larger size, and more extranodal occasions. Additionally, the outside-institution scans were acquired from 32 institutions and on a plethora of scanner models, which significantly increased the heterogeneity of these images. It is reassuring that even though the outside-institution images were excluded from the network training, the network achieved similar sensitivity and FPs/patient as those achieved for the within-institution scans. In the future, the network’s robustness can be further validated by collecting and curating more images from different institutions.

No significant difference was found for the sensitivity of the network when it is trained on PET/CT or on PET only. However, training on PET only introduced significantly more FPs/patient. Although the reference standard contours were drawn on PET, CT is deemed necessary for network training by providing images of different contrast and reducing FPs. This was also consistent with the manual diagnosis process: besides PET, physicians also refer to CT for lesion identification and anatomic orientation, especially for the small nodal lesions.

As illustrated by the free response receiver operating characteristic plot, performance of the network can be adjusted with different probability thresholds. A higher threshold (such as 90%) can decrease the FPs/patient to approximately 13, although the sensitivity will decrease to approximately 77%; a lower threshold (such as 10%) can increase the sensitivity up to 90%, but the FPs/patient will also increase to approximately 25. The adjustable performance of the network provides radiologists the flexibility to choose preferred predictions according to their specific clinical needs.

The high sensitivity to detect the top SUV_max lesions in each patient is a promising feature when the network is applied for clinical MCL assessment. According to a recent study [8], evaluation of the highest SUV_max lesions is of strong prognostic values for MCL patients. Since our network can automatically detect these lesions in approximately 90% of the patients, the DLCNN can potentially be implemented for computer-aided MCL prognosis. One limitation of our current network is the relatively low prediction DSC, which was found as a result of the network over-predicting regions around the lesion. Such over-prediction can be caused by the partial-volume effect of PET imaging [23]. The limited DSC can lead to inaccurate metabolic tumor volume calculation and may also affect the SUV_max report. Nonetheless, the network reported exactly the same top SUV_max as the reference standard for 65% within-institution and 68% outside-institution scans without human interference.

Another limitation of our study was the relatively high FPs/patient, which would need to be corrected through physicians’ inspection. Fortunately, most FPs were found to be limited to locations with inflammatory or physiologic signals. For example, diffuse FDG uptake within bilateral thyroid glands in one patient represented possible thyroiditis. The FPs in the stomach and colon from several patients who had negative endoscopy and colonoscopy findings likely represented physiologic gastrointestinal tract activities. Additionally, sensitivity of the network was relatively low for lesions with SUV_max below the liver reference. Such lesions may be indolent MCL and might be overlooked by the DLCNN.

In the future, we plan to accrue and include more ¹⁸F-FDG PET/CT images for network training and performance improving [24]. In addition to the baseline images used in this study, the network can also be investigated on post-treatment PET/CT to detect residual MCL for treatment response assessment. Finally, further improvement of the prediction DSC should be explored for the network to be used to accurately report the top SUV_max and metabolic tumor volume. A possible approach is to include a second network that is designed for lesion segmentation after detection [25,26], whereby the partial-volume effect might be remediated.

In summary, we developed a DLCNN for computer-aided detection of MCL on baseline whole-body ¹⁸F-FDG PET/CT images. The network achieved high sensitivity, limited FPs, and high specificity for physiologic FDG avidities. The network is robust in its ability to locate lesions and applicability for images acquired at various institutions. By including data from more patients, we expect that the network can be further improved and become practical in assisting the clinical assessment of MCL.

Disclosure of conflict of interest

None.