Synthetic PET from CT improves diagnosis and prognosis for lung cancer: Proof of concept

Summary

[¹⁸F]Fluorodeoxyglucose positron emission tomography (FDG-PET) and computed tomography (CT) are indispensable components in modern medicine. Although PET can provide additional diagnostic value, it is costly and not universally accessible, particularly in low-income countries. To bridge this gap, we have developed a conditional generative adversarial network pipeline that can produce FDG-PET from diagnostic CT scans based on multi-center multi-modal lung cancer datasets (n = 1,478). Synthetic PET images are validated across imaging, biological, and clinical aspects. Radiologists confirm comparable imaging quality and tumor contrast between synthetic and actual PET scans. Radiogenomics analysis further proves that the dysregulated cancer hallmark pathways of synthetic PET are consistent with actual PET. We also demonstrate the clinical values of synthetic PET in improving lung cancer diagnosis, staging, risk prediction, and prognosis. Taken together, this proof-of-concept study testifies to the feasibility of applying deep learning to obtain high-fidelity PET translated from CT.

Graphical abstract

Highlights

• •Deep learning can generate high-fidelity synthetic PET from CT
• •Radiologists confirm high imaging fidelity between synthetic and actual PET
• •Radiogenomics proves the biological fidelity of synthetic PET
• •Synthetic PET improves cancer diagnosis, risk prediction, and prognosis

Salehjahromi et al. develop a GAN-based CT-to-PET translation framework and validate it through thoracic radiologists and radiogenomics analysis. The synthetic PET scans demonstrate potential in enhancing early lung cancer detection, stratification of high-risk populations, and prognostication. Early assessments indicate promising applications in clinical oncology practice.

Introduction

[¹⁸F]Fluorodeoxyglucose positron emission tomography (FDG-PET) is widely used to image patients with cancer.¹ FDG-PET measures glucose consumption, offering a valuable functional view that is complementary to computed tomography (CT) in multiple clinical settings. For example, PET can improve the accuracy in diagnosing high-risk indeterminate pulmonary nodules detected by low-dose CT screening.² It has shown superior value over CT in determining the nodal and distant metastasis for cancer staging.³ For treatment response, PET offers a new view to assessment of metabolic tumor response to systemic therapy and beyond as proposed in PERCIST.⁴ As such, it has become an indispensable component in oncology practice.

Although PET has demonstrated clinical value, it is still not universally applicable compared to CT.⁵ PET-CT has approximately double the radiation exposure than a standard CT scan, raising concerns about increasing cancer risk,⁶ especially in those at high risk, including pregnant women, children, and cancer patients with repeated scans. PET is not considered appropriate for lung cancer screening at current dose levels, as the radiation risk outweighs the benefits. This might change with the development of low-dose PET-CT.⁷ Moreover, PET scanners are expensive and complex to acquire, run, and maintain, which contributes to stark global inequities, with only five PET scanners existing for roughly 50 low-income countries covering a total population of 706 million, based on recent data from the International Atomic Energy Agency.⁵

One strategy to address these limitations is by using cross-modality imaging synthesis as an alternative way to obtain PET. Deep learning has reshaped the landscape of medical image synthesis.⁸ Pilot studies have proven the feasibility of machine-learning models to learn inter- and intra-modality mapping functions, with potentially broad applications in radiology. Imaging synthesis may help streamline the clinical workflow and bypass certain imaging procedures, such as synthetic CT, to potentially replace extra CT acquisition for pelvis PET/MR attenuation correction⁹ and for MRI-based prostate radiation dose planning.¹⁰ Currently, cross-modality synthesis is focused on mapping the anatomical findings between CT and MRI,⁸ where there is a large amount of overlapping information. However, machine-learning-based synthesis remains underexplored for bridging anatomical to functional mapping, given that each modality offers a different view of the underlying physiology. Interestingly, a recent study demonstrated that CT-based lung perfusion images synthesized by a transfer learning framework achieved a strong voxel-wise correlation with single-photon emission CT (SPECT) perfusion images.¹¹ However, most previous studies have assessed synthetic image quality without expert radiologists’ input and have lacked biological and clinical validation.

We hypothesized that deep learning can effectively learn the anatomical-to-metabolic mapping based on paired diagnostic CT and FDG-PET scans, whereby the synthetic PET can preserve biological correlates and add clinical value in lung cancer diagnosis and prognosis. To test this, we performed a comprehensive study on multi-center multi-modal data (imaging, genomics, clinical, and longitudinal data) to develop a computational framework and systematically validate its fidelity based on imaging signal, radiologists’ assessments, and radiogenomics correlates, as well as evaluation of biological and clinical values.

Results

High fidelity of PET images produced by applying conditional GAN on CT with confirmation by thoracic radiologists

We developed a conditional generative adversarial network (cGAN) model by hybridizing pix2pix and ResUNet++ based on multi-center PET and CT scans (for details see STAR Methods and Figure 1). The model was trained on the MDA-TRAIN cohort and locked for external testing on the MDA-TEST and TCIA-STANFORD cohorts with a head-to-head comparison with acquired PET as ground truth. We displayed the synthetic PET scans from the two testing cohorts to compare them side by side with ground-truth PET and visualize the model output (Figure 2A).

Figure 1.

Overview of the study design

(A) Training the cGAN to predict PET image from CT. The input of the generator is a CT slice along with its six neighboring slices while its output is the synthetic PET image. The discriminator tries to classify between the synthetic and ground-truth PET/CT pairs.

(B) In the imaging validation, similarity metrics including SSIM and RMSE were employed for comparing the synthetic and ground-truth PET images. A group of two thoracic radiologists was enrolled blindly to visually assess the quality of synthetic PET images. Next, they conducted a Turing test on synthetic and ground-truth PET images. Moreover, we analyzed the pairwise similarity between the synthetic and ground-truth PET features.

(C) In the biological validation, we applied radiogenomic analysis using the GSEA method to find the association of cancer hallmarks with extracted features from ground-truth and synthetic PET scans.

(D) In the subsection “synthetic PET complements CT for early lung cancer diagnosis,” we investigated whether the performance of indeterminate pulmonary nodule classification using only main CT-based features can be further improved by adding the extracted features from the synthetic PET. In cancer prediction, we validated the clinical value of synthetic PET in prediction of the development of lung cancer and in staging prediction. In staging prediction, two radiologists assess the synthetic PET capability for accurately staging the lung cancer patients. In the subsection “synthetic PET predicts prognosis after standard of care,” we showed that the extracted nodule features from the synthetic PET are capable of stratifying patients into good and bad survival groups.

(E) The cohorts used in different sections.

Figure 2.

Validation of imaging signal fidelity and cancer staging performance by radiologists

(A) Presentation of synthetic PET images with ground-truth PET in MDA-TEST and TCIA-STANFORD testing cohorts. The first three columns from left to right for each cohort correspond to CT, ground-truth, and synthetic PET images. The PET images are shown inversely with the normalized window of SUV in $\left[0,3\right]$. Therefore, the completely black color in tumors indicates that the tumor had uptake with maximum SUV value of at least 3.

(B) The radiologists’ score on quality and relative uptake of lung region in task 1 of the imaging quality test.

(C) Alluvial plot shows the radiologists’ scoring on imaging quality difference using paired PET scans.

(D) Alluvial plot shows the radiologists’ scoring on tumor contrast difference using paired PET scans.

(E) Alluvial plot shows the radiologists’ reading of ground-truth vs. synthetic using paired PET scans.

(F) Comparison matrix of staging between radiologists reading CT and ground-truth PET and pathological stage.

(G) Comparison matrix of staging between radiologists reading CT and synthetic PET and pathological stage.

(H) Consensus matrix of staging between the two radiologists when one radiologist reads true PET and CT compared to another reading synthetic PET and CT.

Two experienced radiologists evaluated the imaging quality and did a Turing test (see STAR Methods for details). They rated equivalent quality of true and synthetic PET for subjective imaging quality (Figure 2B), with an average score of 4 for true PET and 3.6 for synthetic PET using 5-point system (with 1 as poor, 3 as adequate, and 5 as excellent), as well as for tumor contrast, with an average score of 4.5 for true PET and 4.1 for synthetic PET using a 5-point system (with 1 as low, 3 as equal, and 5 as high). For imaging quality rating and within one score tolerance, radiologists rated 75% cases as equivalent and 19% cases as decreased when comparing synthetic PET to its ground-truth counterpart (Figures 2C and S1A). For lesion contrast evaluation, 83% cases stayed unchanged, and 13% cases decreased within one score tolerance (Figures 2D and S1B). For identification of the synthetic scans in the Turing test, the radiologists achieved overall 75% accuracy and misclassified 7% of synthetic cases and 40% of real cases (Figures 2E and S1C). Also, two radiologists had high agreement regarding their consensus for imaging quality (87%) and lesion contrast evaluation (99%) tolerant within one score difference, while they had 77% agreement in identifying the synthetic PET scans (Figure S1D, with individual radiologists’ performance detailed in Figure S1E).

Quantitative comparison confirmed the high fidelity of the synthetic PET

The generated PET scans were compared with ground-truth PET images quantitatively. As seen in Figure 2A, the model showed good performance (Figures S2A and S2B) for ten presented cases from MDA-TEST (mean structural similarity index measure [SSIM] = 0.88) and TCIA-STANFORD (mean SSIM = 0.81), where the SSIM is between 0 and 1 with a higher value indicating more similarity. We further computed the SSIM and root-mean-square error (RMSE) for a large sample of cases from an internal validation set in MDA-TRAIN as well as in the external testing sets (i.e., MDA-TEST and TCIA-STANFORD). As shown in Figure S2C, the cGAN model had high SSIM values, with a median value in MDA-TRAIN and MDA-TEST around 0.9 and with a narrower deviation for MDA-TRAIN than MDA-TEST. For the TCIA-STANFORD cohort, the median of SSIM was around 0.8. Together, these data indicate high fidelity of synthetic PET with ground-truth PET during validation. The RMSE had a similar trend, with the MDA-TRAIN cohort showing the lowest median and MDA-TEST a lower RMSE compared to TCIA-STANFORD (Figure S2D).

Next, we calculated the pairwise Pearson correlation of imaging metrics (metabolic tumor volume [MTV], total lesion glycolysis [TLG], and mean and maximum standardized uptake values [SUV_mean and SUV_max]) between true and generated PET in the MDA-TRAIN, MDA-TEST, and TCIA-STANFORD cohorts (Figure 3A). Relatively high positive correlations were observed for MTV and TLG features consistently in all the cohorts. The highest PET feature correlations were MTV_1.5 (ρ = 0.71) for MDA-TRAIN, MTV_1.5 (ρ = 0.85) for MDA-TEST, and MTV_2.5 (ρ = 0.55) for TCIA-STANFORD. Low correlations were observed for SUV_max and SUV_mean features, which may be attributed to the long tail effects of SUV values (refer to STAR Methods). Further, we studied the agreement between predicted and true PET features using Bland-Altman plots (Figure S3). Reasonable agreements were observed across the predicted and true PET consistently in three cohorts for MTV.

Figure 3.

Validation of imaging signal fidelity by PET feature correlation

(A) Pearson correlation for evaluating the pairwise similarity between the synthetic (rows) and ground-truth (columns) PET features in MDA-TRAIN, MDA-TEST, and TCIA-STANFORD cohorts.

(B) Threshold-based confusion matrix for synthetic SUV_max and the ground-truth SUV_max for four different thresholds ($\alpha =1.5,2.5$) in MDA-TRAIN, MDA-TEST, and TCIA-STANFORD cohorts.

Given the observation that synthetic SUV_max had reduced correlation to ground -truth, we computed a threshold-based confusion matrix to evaluate the progressive correlation at different SUV threshold values. As shown in Figure 3B, when the threshold was set at 1.5, the synthetic SUV_max aligned closely with the ground-truth SUV_max. With increasing threshold, SUV_max values of more synthetic PET scans did not pass the threshold, leading to an accuracy of 89%–91% in MDA-TRAIN or MDA-TEST and 66% in TCIA-STANFORD at a high cutoff of 2.5. Taken together, these results systematically validated that a reasonable fidelity of SUV_max exists between the predicted and the ground-truth PET slices.

Furthermore, we assured that the synthetic SUV_max produced by the cGAN model was not solely driven by the tumor/nodule tumor volume. In particular, we computed the correlation between the SUV_max and tumor/nodule volume, with a small correlation for both the ground-truth PET (${\mathrm{R}}^2=0.039$) and synthetic PET (${\mathrm{R}}^2=0.101$) (Figures S4A and S4B). A similar trend was also observed for subgroup analysis of patients with SUV_max < 7 (Figures S4C and S4D). Additionally, we presented a scatterplot illustrating the relationship between ground-truth and synthetic SUV_max in the combined train and test cohorts of MDA-TRAIN, MDA-TEST, and TCIA-STANFORD (Figures S4E and S4F).

Biological inferences are consistent between synthetic and ground-truth PET

We next performed gene set enrichment analysis (GSEA) to explore radiogenomics correlation¹² of an imaging feature (MTV_1.5) extracted from true and synthetic PET scans with hallmark pathways. Given the fact that MTV has several advantages over SUV_max, including its robustness to access the spatial extent of metabolically active tumor tissue rather than providing a fragile single value based on the highest activity in a small region,^2,13,14 we chose to concentrate our efforts on exploring its radiogenomic correlates. Specifically, we studied MTV_1.5, since it shows a higher correlation between synthetic and ground-truth PET scans (Figure 3A). We observed that several cancer-associated hallmark pathways consistently have significant associations with true and predicted MTV_1.5, although the enriched pathway lists vary across different cohorts (Figure 4A), which may be attributable to different genes covered in tumors of different cohorts. E2F targets, MYC targets, and G₂M checkpoint were the top dysregulated pathways in the MDA-TRAIN cohort; epithelial-mesenchymal transition (EMT), glycolysis, and MYC targets were the top enriched pathways in the MDA-TEST cohort; and EMT, mitotic spindle, and transforming growth factor-β signaling were the top dysregulated pathways in the TCIA-STANFORD cohort. High concordance was observed between synthetic PET and true PET scans regarding the normalized enrichment score of all cancer hallmark pathways calculated, with Pearson correlation of 0.88 (p = 1.3e−16) in MDA-TRAIN for training, 1.0 (p < 1e−16) in MDA-TEST, and 0.74 (p = 1.9e−9) in TCIA-STANFORD, suggesting consistent image-to-genomics alignment.

Figure 4.

Validation of biological fidelity by radiogenomics analysis

(A) Significant hallmark gene sets associated with MTV extracted from ground-truth and synthetic PET for MDA-TRAIN, TCIA-STANFORD, and MDA-TEST cohorts.

(B) The first column shows the unsupervised hierarchical clustering heatmap of hallmark pathways normalized enrichment score for correlation of each pathway with MTV feature from ground-truth and synthetic PET across MDA-TRAIN, TCIA-STANFORD, and MDA-TEST cohorts, where the asterisks represent the significant false discovery rate q value >0.25.

Further unsupervised hierarchical clustering of the pathway enrichment scores from all three cohorts revealed hallmark pathways with consistent positive/negative correlation with MTV_1.5 across the cohorts (Figures 4B and S5A). Increased activity reportedly associated with cancer aggressiveness such as EMT, hypoxia, angiogenesis, and mitotic spindle pathways were consistently associated with greater MTV_1.5, while biological processes implicated in cancer suppression, such as peroxisome, adipogenesis, heme metabolism, fatty acid metabolism, and bile acid metabolism, were consistently associated with lower MTV_1.5 values. Among processes linked to cancer progression, EMT and angiogenesis were the top pathways significantly positively correlated with MTV_1.5 values (Figure S5A). Highly similar enrichment plots were observed for the association of each pathway with MTV_1.5 from true or synthetic PET in each studied cohort (Figure S6). Interestingly, glycolysis was found to be significantly positively associated with true and synthetic MTV_1.5 only in two cohorts, MDA-TRAIN and MDA-TEST (q values from <0.0001 to 0.02). Considering the known association of PET features with glycolysis-related genes in non-small cell lung cancer (NSCLC),¹⁵ we further evaluated associations between the true and synthetic MTV_1.5 with a glycolytic score representing overall expression of glycolysis-related genes in each cohort. Combining the three cohorts and dichotomizing tumors into glycolysis-high and glycolysis-low groups, we found that the former group has significantly greater MTV_1.5 values extracted from either ground-truth or synthetic PET images (Wilcoxon rank-sum test p value 0.03 and 0.05, respectively) (Figure S5B).

Synthetic PET complements CT for early lung cancer diagnosis

Next, we applied the cGAN model and generated the corresponding PET from the CT scans in the LIDC-IDRI cohort (n = 1,048 for training and n = 350 for testing) to assess whether the synthetic PET could provide additional information to classify the benign vs. malignant nodules. A cutoff value was set at 1.5 for MTV, which is highly correlated with MTV_2.5 and TLG features (Figure 3A). We also chose SUV_max, which was less correlated with MTV and TLG-based features (Figure 3A). We first evaluated the accuracy of individual CT features (tumor_size and tumor_max-d) or PET features (SUV or MTV), where CT features have demonstrated higher accuracy (training in Figure S7, testing in Figure 5A). Next, we integrated PET and CT features to assess whether PET provides additional value on top of CT and improves the accuracy of lung cancer diagnosis. For the three machine-learning models, we observed consistent augmentation when adding PET features (Figures 5A and S7). For instance, using the XGBoost classifier, the top-ranked models were obtained  by adding SUV_max and MTV to tumor_max-d and tumor_size, respectively, resulting in training accuracies of 98.1% and 90.8%, and testing accuracies of 84.1% and 83.5%, respectively. These results outperformed the best CT-based prediction, which had a training accuracy of 89.8% and a testing accuracy of 82.3%, when using tumor_size (training accuracy = 89.8%, testing accuracy = 82.3%). Similar patterns were observed for random forest and support vector machine (SVM) models, where PET improved the training accuracy by 3%–7% and testing accuracy by 2%–5%, respectively.

Figure 5.

Validation of clinical value by diagnosing malignant vs. benign from indeterminate pulmonary nodules

(A) Model accuracy in the test cohort (n = 350) obtained from synthetic PET univariate features (MTV, SUV_max), CT univariate features (tumor_size, tumor_max-d), and CT and PET bivariate features (tumor_size & MTV, tumor_size & SUV_max, tumor_max-d & MTV, tumor_max-d & SUV_max) in the LIDC-IDRI cohort.

(B) The first column corresponds to the confusion matrix of the best performance obtained using either of single CT features tumor_size or tumor_max-d. The second column corresponds to the confusion matrix of the best performance when adding one synthetic PET feature, SUV_max or MTV.

(C) Threshold-based confusion matrix for synthetic SUV_max and the nodule malignancy at four different thresholds ($\alpha =1.5,2.5$) in the LIDC-IDRI cohort.

Further, we presented the lesion level classification performance by head-to-head comparison of the optimal CT model with the optimal PET-CT model (Figure 5B). In general, during testing the synthetic PET improved the F1 score in lesion detection by 3%, 5%, and 12% in XGBoost, SVM, and random forest, respectively. Also, we observed a 5% and 13% sensitivity increase in SVM and random forest, respectively.

We next explored the relationship of nodule likelihood of malignancy with the synthetic SUV_max (Figure 5C). With the increase of threshold $\alpha$, an increase in the specificity and decrease in the sensitivity were observed as expected. Of note, when we used the cutoff of 1.5, the specificity reached 99% with sensitivity at 27%.

Synthetic PET improves the performance of state-of-the-art CT deep-learning model for predicting lung cancer risks

We next sought to test whether synthetic PET scans will add additional value to the CT deep-learning model that has demonstrated high accuracy in predicting lung cancer risks using low-dose CT scan images for lung cancer screening.¹⁶ We applied the cGAN model to predict the risk of developing lung cancer in the MDA-SCREENING set (n = 122). We divided this dataset into two subsets. The first subset includes CT scans from patients who developed lung cancer within 1 year vs. patients who were diagnosed with lung cancer after >3 years, consisting of 355 CT scans. The second subset consists of CT scans from patients who were diagnosed with cancer within 1 year vs. patients who were diagnosed with lung cancer after >5 years, totaling 259 CT scans (Figure 6A). We randomly divided the patients into discovery (n = 90) and testing (n = 32). We then obtained the synthetic PETs.

Figure 6.

Validation of clinical value of synthetic PET by predicting the risk of developing lung cancer

(A) Distribution of CT scans in training, validation, and test sets in both group 1 and group 2.

(B) Two box plots were used to compare the synthetic SUV_max values in group 1 (lung cancer diagnosed within 1 year vs. lung cancer diagnosed at >3 years) and group 2 (lung cancer diagnosed within 1 year vs. lung cancer diagnosed at >5 years).

(C) Receiver-operating characteristic curves for CT and CT plus synthetic PET analysis along with their respective AUC values for both group 1 and group 2.

We observed that the SUV_max measured on synthetic PET were significantly different between the low-risk vs. high-risk groups (Figure 6B). Next, we employed a DenseNET-based autoencoder to extract its latent features from synthetic PETs. In addition, we obtained the CT deep-learning-based risk prediction based on a pre-trained model (i.e., SYBIL¹⁶) risk. By combining the PET features with SYBIL risk, we trained a joint CT and PET model for risk prediction. We observed higher area-under-the-curve (AUC) values when adding the synthetic PET features. Specifically, for predicting cancer development within 1 year vs. >3 years, the AUC increased from 0.67 to 0.70; and for the 1-year vs. >5-year group, the AUC increased from 0.73 to 0.77 (Figure 6C).

Radiologist staging on synthetic PET scans achieves accuracy similar to that of staging on true PET with pathological staging as gold standard

To further assess the potential clinical utility of synthetic PET, two thoracic radiologists directly staged the lung cancer on synthetic PET to compare head-to-head with ground-truth PET based on 30 lung cancer cases randomly selected from the testing group (for details see STAR Methods and Table S1B). The final pathological staging was used as the gold standard. The radiologists reading CT paired with ground-truth PET achieved an overall accuracy of 70% compared to the pathological stage, with 13.3% downstaged and 16.7% upstaged (Figure 2F). When reading CT paired with synthetic PET, their accuracy was 66.7%, with 16.7% downstaged and 16.7% upstaged (Figure 2G). Interestingly, compared to ground-truth PET, radiologists’ reading from synthetic PET is better aligned with pathological stage I and II patients, which dropped for stage III patients. Furthermore, two radiologists exhibited an 87% consensus in staging (Figure 2H) when one read ground-truth PET and another read synthetic PET.

Synthetic PET predicts prognosis after standard of care

Given the correlation of imaging features obtained from synthetic PET and ground-truth PET (Figure 3) and prior clinical evidence,^14,17,18 we next assessed the prognostic performance of MTV_1.5 and MTV_2.5 to stratify patients’ overall survival (OS) using Kaplan-Meier analysis (Figure 7). We observed that the synthetic MTV_1.5 and MTV_2.5 can stratify patients’ OS in both MDA and non-MDA cohorts: for MTV_1.5, MDA cohort hazard ratio (HR) = 1.78, 95% confidence interval (CI) 1.11–2.84 $(\mathrm{p}=0.00051$) and non-MDA cohort HR = 1.51, 95% CI 1.26–1.81 $(\mathrm{p}<1\mathrm{e}-4$); for MTV_2.5, MDA cohort HR = 1.42, 95% CI 1.07–2.04 $(\mathrm{p}=0.015$) and non-MDA cohort HR = 1.37, 95% CI 1.18–1.60 $(\mathrm{p}<1\mathrm{e}-4$).

Figure 7.

Validation of clinical value of synthetic PET by predicting overall survival

Kaplan-Meier curves of patients’ overall survival (OS) stratified by MTV_1.5 and MTV_2.5 features obtained from the synthetic PET images on MDA cohorts (MDA-TRAIN and MDA-TEST) and non-MDA cohorts (TCIA-STANFORD and NSCLC-RT).

Furthermore, we compared the prognostic value of individual features in a head-to-head fashion between synthetic PET and ground-truth PET when it was available (Figure S8A). We observed that the prediction capacity of synthetic MTV declined as the SUV cutoff threshold was raised from $1.5$ to $2.5$, possibly due to decreasing correlation of the MTV feature extracted from synthetic and true PET for higher SUV cutoff thresholds (Figure 3). While it was expected that the synthetic PET would achieve an inferior performance as measured by concordance index compared to ground-truth PET, it did achieve statistically meaningful prediction in the majority of cases. For SUV_max, we observed a relatively lower prognosis compared to MTV and TLG, especially the cases with low fidelity of ground truth (Figures S8B and S8C). More interestingly, the synthetic PET offered meaningful prediction for stratifying a patient’s prognosis for the NSCLC-RT cohort when true PET data were not available.

Discussion

In this study, we developed and validated a conditional GAN-based pipeline to synthesize PET of high fidelity from diagnostic CT scans from multi-center multi-modal lung cancer datasets (n = 1,478). Our proposed computational framework demonstrates robust performance during external validation assessed by thoracic radiologists, measuring image quality and a Turing test, whereby the synthetic PET scans were found to be of equivalent quality and tumor avidity when compared side by side to the ground-truth FDG-PET scans. The synthetic PET images were also consistent with true PET in radiogenomics validation by demonstrating significant correlation of imaging features with dysregulated biological processes. More importantly, the synthetic PET demonstrated additional diagnostic value in distinguishing malignant from benign lung nodules compared to CT nodule size and maximum diameter measurements, in accurately staging patients compared to pathological staging, in predicting risk of developing lung cancer during follow-up, and in predicting survival of lung cancer patients. Taken together, our proof-of-concept study proves the feasibility of applying deep learning to obtain high-fidelity functional imaging translated from the anatomical imaging. With further tuning and validation, this pipeline may potentially add value in cancer screening, staging, diagnosis, and prognosis.

It is worth clarifying that synthesizing PET is not intended to replace PET scanning but rather to offer complementary value to CT data, which are more popular, especially when PET images are difficult to acquire. A few pilot studies have been carried out to show the feasibility of CT-to-PET conversion in some clinical applications, such as reducing the false-positive rate in detecting malignant liver lesions¹⁹ and improving the differentiation between malignant lymph nodes from thyroid tissue in head and neck cancer.²⁰ However, such attempts at clinical translation have been missing in lung cancer, the leading cause of cancer death, where PET-CT is part of the standard of care across different clinical settings. To the best of our knowledge, this is the largest reported CT-to-PET mapping study based on multi-center multi-modality datasets of clinical, radiological, and molecular information. The details of our algorithm development and evaluation according to the newly published guidelines to develop and evaluate artificial intelligence (AI) specifically in nuclear medicine^21,22 are detailed in Tables S2 and S3. Our study has made important contributions in the following ways.

At the technical level, we leveraged the 2.5D convolutional neural network that combines neighboring slice information to augment performance and to balance computational cost. Moreover, in contrast to generation of a fake image, which utilizes a fixed-length random vector to generate a photorealistic sample, we used ResUNet++ as backbone for the generator without adding a random vector to learn the definitive CT-to-PET mapping that increases the reproducibility. Collectively, this has also led to our cGAN model outperforming the diffusion model UNSB (unpaired neural Schrödinger bridge),²³ which holds a distinct advantage over classical diffusion models by bypassing the limitation imposed by the Gaussian prior assumption (Figure S9).

Beyond the classical imaging signal validation, we have pioneered a systemic validation incorporating different fronts, including image quality and Turing test evaluated by thoracic radiologists and biological validation by radiogenomics design. Contrasting with prior studies, which utilized radiologists to qualitatively assess the image in limited small tumor regions,²⁴ we asked them to quantitatively evaluate the quality and contrast of synthetic PET in the whole field. In addition, we directly fed these synthetic PET scans for cancer staging to radiologists—a common clinical task performed on PET. With pathological staging as ground truth, it is encouraging to see that staging on synthetic PET has achieved an equivalent accuracy with ground-truth PET scans when pairing with CT scans.

Furthermore, we validated synthetic PET at the biological level in three independent cohorts to prove radiogenomics fidelity of cancer-related hallmark pathways between true and synthetic PET. Both true and synthetic PET avid tumors demonstrated upregulated pathways involving EMT, hypoxia, and angiogenesis, corroborating the previous observations of high ¹⁸F-labeled FDG activities in cancer cells undergoing EMT, particularly in hypoxic regions.²⁵ In addition, we observed that aggressive tumors with higher MTV, as stratified by true or synthetic PET, were consistently associated with elevated glycolytic activity and downregulated metabolism of fatty acid, bile acid, and heme. This is consistent with a recent study on The Cancer Genome Atlas lung adenocarcinomas showing that high glycolytic and low lipid metabolism exhibited high metastatic potency and poor survival.²⁶

A robust CT-to-PET translation framework would bring several potential impacts on oncology practice. Our data suggested that synthetic PET can confer additional value on top of CT scans to distinguish benign from malignant lung nodules, a clinically unmet need for lung cancer screening and early diagnosis. In addition, it can augment the CT deep-learning model to accurately identify patients at high risk of developing lung cancer.²⁷ We also demonstrated a similar prognostic capacity of synthetic PET scans compared to true PET. With sufficient training data for individual tasks to further improve the model, this technology can be of great potential from a patient safety perspective and financial standpoint. Because of concerns regarding higher radiation doses compared to routine CT scans and radiotracer exposure, PET is used with caution for certain patient populations, including young children and pregnant women. Recently, a strong association between cumulative radiation dose from CT scans and the risk of hematological malignancies was reported for young people.⁶ Our models can be potentially applied to routine CT scans to extract additional clinically relevant data from these patients and reduce their exposure to radiation. Also, it may reduce the frequency of repetitive PET scans, and the substantially lower cost from routine CT scans may relieve the increasing healthcare cost in western countries. More importantly, the optimized model could be swiftly deployed in low-income countries to fill the gap and improve cancer staging and management. The encouraging results from our study, along with other structure-to-function transfer studies such as hyperpolarized gas MRI ventilation scans derived from CT²⁸ and SPECT perfusion scans based on CT,¹¹ underscore the need for future research investigations. These studies hold the potential to facilitate the development of an all-in-one multi-modality scanner designed to optimize the diagnosis and prognosis prediction of lung cancer. Finally, as PET scans are also widely used in managing other medical conditions such as cardiac and neurological diseases, our proposed tool and pipeline may have a broader impact beyond cancer care.

It is worth clarifying that cross-modality medical imaging synthesis is conceptually different from the fake image generation in computer vision,²⁹ although they share some common technical foundations such as using a GAN algorithm. Fake image generation (e.g., deepfake³⁰) aims to produce photorealistic images to fool people and can be viewed as an interpolation problem to produce non-existing samples. By contrast, cross-modality imaging synthesis, such as the CT-to-PET translation presented here, can be mathematically formulated as a regression problem, which aims to learn a latent transmodality mapping function. This is where deep learning outperforms conventional algorithms, as it efficiently learns any complex non-linear function according to the universal approximation theorem.³¹ Our study adds important evidence to support the growing utility of deep learning for cross-modal/platform synthesis in radiographic scans,^8,11,32 digital pathology for microscopy-based drug discovery,³³ and immunohistochemical image quantification.³⁴ When a certain data type is missing or difficult to access, synthetic data will potentially help to fill the coverage gaps in order to build trustworthy AI models.³⁵

Limitations of the study

Our study has several important limitations. First, although we included large multi-center multi-modal datasets, the results need to be interpreted as proof-of-concept and hypothesis-generating research. The primary aim of this pilot study is to establish the feasibility of generating synthetic PET images from CT scans as well as to assess the clinical and biological values of synthetic PET images through radiologist evaluations, radiogenomics correlations, radiomics analyses, and deep-learning experiments. It is important to highlight that while our proof-of-concept study holds promise, it is not intended to supplant conventional PET imaging or alter current clinical practices. Instead, it serves as a critical stepping stone, warranting further investigation using a prospective design with fine-grained lung cancer subtypes to bring this to clinical translation. Second, PET imaging has limited value in lesions with predominant ground-glass opacity³⁶ and nodal immune flare after immunotherapy.³⁷ Unfortunately, a synthetic PET approach would be expected to inherit these intrinsic limitations. Here, we have leveraged limited imaging metrics from synthetic PET scans that have been extensively examined in prior studies, including MTV, SUV, and TLG, to prove the added value. Future efforts are needed to develop next-generation imaging biomarkers beyond conventional metrics to overcome these challenges through radiomics,^38,39,40,41 habitat imaging,^42,43 and deep learning.^27,44 Moreover, with the rapid evolution of generative AI, more sophisticated pipelines⁴⁵ can be leveraged to improve the quality of synthetic scans. In the end, it remains challenging to integrate complex AI models into clinical workflows, which is beyond the scope of the current study, and future efforts are critical to overcoming these hurdles.⁴⁶

In conclusion, we have developed and validated a cGAN-based CT-to-PET translation framework based on multi-center PET and CT scans. The synthetic images were extensively validated by clinical thoracic radiologists and biological correlation using a novel radiogenomics analysis. More importantly, the synthetic PET scans demonstrated promising diagnostic values in cancer staging, improving early detection of lung cancer, stratifying lung cancer development in a high-risk population, and cancer prognostication. All things considered, future studies in a prospective setting are warranted to validate and translate these intriguing findings to clinical oncology practice.

STAR★Methods

Key resources table

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Deposited data
MDA-TRAIN/TEST/SCREENING	This paper	N/A
TCIA-STANFORD	Bakr, S. et al.47	https://wiki.cancerimagingarchive.net/display/Public/NSCLC+Radiogenomics
LIDC-IDRI	Armato III, S. G. et al.48	https://wiki.cancerimagingarchive.net/pages/viewpage.action?pageId=1966254
NSCLC-RT	Aerts, H. J. et al.49	https://www.cancerimagingarchive.net/
Software and algorithms
R 3.6.1.	The R Project for Statistical Computing	https://www.r-project.org
Python version 3.8	Python software	https://www.python.org/
Source code for the deep learning model	This paper	https://github.com/WuLabMDA/Synthetic-PET-from-CT/

Resource availability

Lead contact

Further information and requests for resources should be directed to and will be fulfilled by the lead contact, Jia Wu (JWu11@mdanderson.org).

Materials availability

This study did not generate new unique reagents.

Data and code availability

• •In our study, the internal datasets from MD Anderson, which contain sensitive patient information, are not publicly available due to privacy and institutional policy. However, the publicly available datasets used in our research can be accessed online, with detailed instructions and links provided in the “patient cohort” section of this paper.
• •Source code for the deep learning model is available at: https://github.com/WuLabMDA/Synthetic-PET-from-CT/
• •Any additional information required to reanalyze the data reported in this work paper is available from the lead contact upon request.

Experimental model and subject details

Patient cohorts

This study was approved by the Institutional Review Board (IRB) of The University of Texas MD Anderson Cancer Center. We collected 5 multi-center multi-modal datasets from a total of 1478 lung cancer patients. The details of individual datasets and their roles are summarized in Figure 1E and elaborated in the following.

• •MDA-TRAIN cohort(n = 132): contains lung cancer patients who underwent surgical resection at MD Anderson (see details elsewhere⁵⁰). We collected the diagnostic CT, PET/CT, gene expression, and clinicopathological and follow-up data. This cohort was used to train the deep learning model and used for biological and clinical validation. Clinical characteristics of this cohort is presented in Table S1A.
• •MDA-TEST cohort(n = 75)& TCIA-STANFORD cohort(n = 125): MDA-TEST contains lung cancer patients who underwent surgical resection at MD Anderson (see details elsewhere^51,52), and TCIA-STANFORD cohort contains lung cancer patients treated at Stanford,⁴⁷ which is publicly shared as NSCLC-Radiogenomics through the TCIA website: https://wiki.cancerimagingarchive.net/display/Public/NSCLC+Radiogenomics. The diagnostic CT, PET/CT, gene expression, and clinicopathological and follow-up data were complied. These cohorts were used to validate the deep learning model, and used for imaging, biological, and clinical validation. Clinical characteristics of these two cohorts are presented in Table S1A.
• •LIDC-IDRI cohort(n = 665): LIDC-IDRI contains lung cancer CT screening scans from Lung Image Database Consortium with a total of 1398 nodules with detailed radiologists’ annotation and diagnosis⁴⁸ (see details at https://wiki.cancerimagingarchive.net/pages/viewpage.action?pageId=1966254). This dataset contains the lesion annotations and their segmentations for nodule classification, including the nodule information such as CT slice number, malignancy rating, texture information, and the coordinate of the center of the nodules. Four experienced thoracic radiologists independently assigned the malignancy rating from $‘1’$ to $‘5’$ for all the nodules, where a higher value indicates a higher likelihood of malignancy. To minimize ambiguity, only nodules with malignancy rating of $‘1’,’2’$ or $‘4’,’5’$ were selected, while the undifferentiated nodules with malignancy rating of $‘3’$ were discarded. This resulted in a total of $1398$ nodules for further analyses. We have downloaded the screening CT scans, and diagnosis readings. This cohort was used to validate the deep learning model in lung cancer diagnosis.
• •NSCLC-RT cohort(n = 359): contains lung cancer patients who received radiation treatment.⁴⁹ This dataset is publicly shared as NSCLC-Radiomics through the TCIA website (https://www.cancerimagingarchive.net/collection/nsclc-radiomics/). CT scans, and clinical and follow up data were obtained. This cohort was used to validate the prognostic value of the deep learning model.
• •MDA-SCREENING (n = 122): comprises longitudinal CT scans obtained from 122 patients who underwent lung cancer screening at MD Anderson, totaling 355 CT scans. This cohort was used to test the predictive value of developing lung cancer based on the deep learning model.

Method details

Study design

Based on the high-quality multimodal data (including FDG-PET and CT, clinicopathological, genomic, and survival information) from different centers, we developed a CT-to-PET deep learning model and evaluated its fidelity as well as value in the clinical context (Figure 1). We first developed a conditional generative adversarial network (cGAN) based model that can generate FDG-PET from CT scans trained from paired CT and PET scans in the MDA-TRAIN cohort. Then, we externally validated the performance of the CT-to-PET model on MDA-TEST and TCIA-STANFORD cohorts by comparing the synthetic PET with acquired PET scans as the ground-truth through the lens of radiologists and quantitative metrics. Next, we biologically validated the model by assessing the degree of consistency of radiogenomics correlates⁵³ with the true PET and synthetic PET scans, leveraging the paired gene expression data in MDA-TRAIN, MDA-TEST and TCIA-STANFORD. Finally, we tested the added clinical value of models for both lung cancer diagnosis in LIDC-IDRI and prognosis in NSCLC-RT.

Imaging data preprocessing

We collected the ¹⁸F-FDG PET/CT scans and separate diagnostic CTs. The PET images were reconstructed with an ordered-subset expectation maximization (OSEM) algorithm, using the co-acquired CT data for attenuation correction. We computed standardized uptake value (SUV) maps for the FDG-PET images. The PET SUV map was registered to the diagnostic CT scans using elastix toolbox.⁵⁴ The registration results were visually checked and manually corrected to mitigate the uncertainties due to respiratory motion and positioning differences when necessary. For MDA-TRAIN and MDA-TEST, radiologist collaborators manually contoured the primary tumor on diagnostic CT scans. For TCIA-STANFORD, LIDC-IDRI, and NSCLC-RT, the tumor contours were provided along with the original imaging data. In addition, we extracted the lung masks based on pre-trained U-net⁵⁵ to help identify the slices covering lung regions from the original CT images. The CT scans were displayed using lung window/level settings.

Conditional generative adversarial network (cGAN) pixel by pixel maps CT to FDG-PET

A cGAN model was adopted to learn a non-linear mapping function from input CT images in order to output PET images, where it was extended on top of the original pix2pix translation algorithm.⁵⁶ The objective function in our cGAN is defined as:

$$G^{\ast }=\lambda {\mathbb{E}}_{CT,PET}{\Vert G\left(CT\right)-PET\Vert }_1+\arg \underset{G}{\min }\underset{D}{\max }{\mathcal{L}}_{cGAN}\left(G,D\right)$$ (Equation 1)

where $G$ and $D$ are the generator and discriminator, respectively, $G^{\ast }$ is the optimized generator, ${\Vert .\Vert }_1$ denotes the L1-norm. The first term in Equation 1 is the L1-norm of the pixel-wise difference between the learned PET and ground-truth PET and its goal is to make them closer. $\lambda$ is a regularization parameter that balances between the two terms; $CT$ is the input CT slide(s) and $PET$ is the ground-truth PET slice. The second term can be extended as follows:

$$\arg \underset{G}{\min }\underset{D}{\max }{\mathcal{L}}_{cGAN}\left(G,D\right)=\arg \underset{G}{\min }\underset{D}{\max }\left[{\mathbb{E}}_{CT,PET}\left[\log D\left(CT,PET\right)\right]+{\mathbb{E}}_{\mathit{C}\mathit{T}}\left[\log \left(1-D\left(CT,G(CT\right))\right)\right]\right]$$ (Equation 2)

Essentially, Equation 2 calculates the binary cross entropy (BCE) loss where the discriminator gives higher loss value if it cannot classify the generator’s output as fake and the ground-truth PET output as real. In contrast, the generator learns to generate an output to fool the discriminator with a lower loss value if the discriminator cannot catch its output as fake.

Similar to the pix2pix translation algorithm,⁵⁶ we did not train our cGAN using a random noise vector $\mathit{z}$ as input for the generator $G$. The main reason is that we aim to produce deterministic PET images as the output of generator. Instead, the ResUNet++ architecture was utilized for the generator, which has outperformed U-Net and ResUNet in several image-to-image applications.⁵⁷ For the discriminator, we employed the original structure used in the image-to-image translation algorithm,⁵⁶ with a total of five convolution layers with a kernel size of four.

cGAN model configuration

The MDA-TRAIN cohort was utilized for training and internal validation of the deep learning model using a 5-fold cross-validation approach. A total of 120 patients were employed for training, and the remaining 12 patients were used for validation and fine-tuning of the generated PET image quality. Registered diagnostic CT and FDG-PET slices from the lung region were used to train the model. Specifically, the cGAN model was configured with a 2.5-dimentional (2.5D) scheme, which took seven consecutive axial slices of CT ($512\times 512\times 7$) as input to predict the central PET slice ($512\times 512$) (Figure 1A). Of note, we focused on the lung regions in the current study to mitigate the computational cost. In total, the training dataset was comprised of $15,291$ unique sets of data arrays of seven CT slices and one PET slice.

One challenging aspect of CT to PET translation is related to accurately predicting the dynamic ranges of SUV. As we observed in the training cohort, the SUV_max of primary tumors fluctuated in the range of $\left[1.4,56.6\right]$ with a median of $11.5$. The distribution of SUV scores demonstrated a long tail, with fewer and fewer voxels associated with higher SUV values. Two strategies were used to mitigate the long tail effect. First, we introduced a maximum cutoff (SUV = 7) to clip and normalize the SUV map. Since SUV 2.5 is clinically accepted to distinguish between malignant and benign tumors and also predict patient’s survival,¹⁷ the selected cutoff balanced between computational challenges and clinical value. Second, we applied gamma compression with $\gamma =0.5$ on the SUV intensity maps. Thus, the distribution range for lower SUV values becomes wider to increase its resolution, with a narrower range for the less frequent higher SUV values.

cGAN model training

We applied the common strategy to train the cGAN model, alternating between one step of optimizing $D$ and one step of optimizing $G$. The Adam optimizer with ${\beta }_1=0.5$ and ${\beta }_2=0.999$ was used in optimizing both $D$ and $G$. The initial learning rate was set to $0.0002$ for $200$ epochs and batch size of 5. For data augmentation, we used on-the-fly augmentation: affine transformation with rotation (−π/4, +π/4), translation $\left(0.1,0.1\right)$, scaling $\left(0.85,1.15\right)$, shearing (−8, +8).

Imaging signal level validation by thoracic radiologists

We systematically evaluated the fidelity of synthetic PET scans on testing cohorts. First, visual comparison was employed for validation, where we carried out a 2-step imaging Turing test. Two experienced thoracic radiologists qualitatively assessed the scans independently. In task 1, they were blinded to the information that any PET images were synthetic and asked to assess PET scans paired with corresponding CT (randomly sampled from 60 patients’ from the MDA-TEST and TCIA-STANFORD cohorts) for: 1) subjective quality score (5-point system, with 1 as poor, 3 as adequate, 5 as great); 2) uptake of dominant lung lesions relative to mediastinal blood pool (5-point system, with 1 as low, 3 as equal, 5 as high). In task 2, the radiologists were informed of the presence of synthetic PET images and asked to identify whether the PET images (randomly sampled from 30 patients’ from the MDA-TEST and MDA-TRAIN cohorts) were real or synthesized. The questionnaires of tasks 1 and 2 are available online at (https://drive.google.com/drive/folders/13qlGhYc5jl9DrlINPmzAxxRiW8RYBFmW?usp=sharing).

Second, we quantitatively compared the synthetic PET images with the ground-truth PET scans. In particular, we computed the structural similarity index measure (SSIM)⁵⁸ and the root-mean-square error (RMSE) for both the training cohort (MDA-TRAIN) and two external validation cohorts (MDA-TEST and TCIA-STANFORD), by randomly sampling 195 PET slices from a subset of 13 patients for each cohort. The SSIM index provides a fractional value between 0 and 1 where a higher value indicates more similarity between two images. This index is calculated by weighted multiplications of three global component: luminance, contrast, and structure. While SSIM captures global components, RMSE is calculated based on pixel-to-pixel difference, which makes it very sensitive to misalignment between the two images. Of note, the ground-truth PET scans cannot be perfectly registered to the diagnostic CT images, which were taken on a different date, due to inevitable respiratory motion and positioning differences. Therefore, the RMSE is not expected to be zero even for the ground-truth PET. Nevertheless, RMSE can be used for comparing the model’s performance of training and testing.

Third, we evaluated the consistency of conventional imaging metrics extracted from ground-truth PET and synthetic PET, including SUV_max, SUV_mean, metabolic tumor volume (MTV), and total lesion glycolysis (TLG). Beyond directly assessing the clinical value of synthetic PET in clinical workflow by the radiology team, we extend the evaluation of how it would assist cancer diagnosis and prognosis in the context of radiomics analysis. Specifically, we focus on the putative PET metrics, including metabolic tumor volume (MTV), total lesion glycolysis (TLG), in conjunction with SUV_max to provide a comprehensive characterization of tumors, which have been extensively validated in prior studies.^2,59 The Pearson’s correlation was used for evaluating the pairwise similarity. Also, Bland–Altman plots used in analyzing the agreement between two different types of PET features. In particular, we assessed the consistency of the SUV_max in the synthetic PET slices. To do so, we introduced a threshold-based confusion matrix (CM) to further examine the change of SUV_max. The elements of the matrix represent the number of cases in which the SUV_max exceeds or falls below a given threshold in the generated and ground-truth PET slices. We calculated this matrix with different SUV thresholds ($\alpha =1.5,2.5$) in MDA-TRAIN, MDA-TEST and TCIA-STANFORD cohorts.

Biological validation using gene expression profiles

We used the gene set enrichment analysis (GSEA, version 4.2.3, default parameters)⁶⁰ of RNA sequencing data of the MDA-TRAIN and TCIA-STANFORD cohorts and microarray data of the MDA-TEST cohort to identify biologic processes and pathways associated with imaging features extracted from true and synthetic PET scans.

Hallmark gene sets from the Molecular Signatures Database (MSigDB) were tested for associations with ground-truth and synthetic PET features from metabolic tumor volume and total lesion glycolysis in the 3 cohorts. Only known genes, found in all samples in the cohorts, were used for the analysis. The Pearson correlation of log-transformed PET feature values with the gene expression profile across samples was used for ranking genes in each cohort. Gene sets significantly correlated with ground-truth or synthetic PET features were then compared using the normalized enrichment score (NES) and the false discovery rate (FDR) q-value. To discover biological processes that were consistently positively or negatively correlated with PET features across three cohorts, we performed unsupervised hierarchical clustering of all identified NES followed by meta-analysis. Namely, the Fisher’s method implemented in R library “metap” was used to combine the q-value separately for ground-truth PET and synthetic PET. Biological processes with combined q-values less than 0.25 were considered as significantly associated with PET features.

Associations of true and synthetic PET imaging features with the tumor glycolytic activity were evaluated using a glycolytic score¹⁵ representing overall expression of glycolysis related genes in the tumor and is calculated as the geometric mean of two genes GPI and GAM4 (log2-transformed). The scores were initially computed separately for each cohort then normalized using the standard score and merged. For tumors having high or low glycolytic activity, the joined set of the glycolytic scores was dichotomized according to the median. Then the Mann-Whitney test was used to determine significance of the difference in values of ground-truth and synthetic PET features, between groups with low versus high glycolytic activity.

Clinical validation: Cancer diagnosis of indeterminant pulmonary nodules

We evaluated the clinical value of the synthetic PET for improving cancer diagnosis in lung cancer screening in LIDC-IDRI cohort. Based on the extracted imaging features from synthetic PET scans, we built different machine learning models (including XGBoost, random forest, and SVM) to classify malignant versus benign nodules, where we randomly split the $1398$ indeterminate pulmonary nodules (IPNs) into training (n = 1048) and testing (n = 350) sets. To address the added value of synthetic PET on top of CT, we used parsimonious models with limited features to mitigate model overfitting risk. In addition, we also tested the impact of dynamic cutoff values of SUV_max on the prediction accuracy of benign or malignant.

Clinical validation: High-risk patient identification for developing lung cancer

Next, we also applied the deep learning approach to synthetic PET for predicting individual patients’ risk of developing lung cancer. This analysis was carried out on MDA-SCREENING set. DenseNET-based autoencoder was applied to extract PET latent features. Also, we a pre-trained CT deep learning model (i.e., SYBIL¹⁶) to obtain the risk of developing lung cancer. For combining PET features and CT-based SYBIL risk, random forest classifier was trained. In order to explore the association between SUV_max from synthetic PET images and the risk of cancer development, we enlisted the assistance of our radiologists, who performed segmentation on the lung lesions of each CT scan for SUV_max extraction.

Clinical validation: Cancer staging by radiologists on synthetic PET with pathology as gold standard

Two radiologists assess the synthetic PET capability for accurately staging the lung cancer patients when replacing the ground truth PET. We randomly selected 30 patients, with 10 patients assigned to each stage of lung cancer (stages I, II, and III) from the testing group (MDA-TEST cohort). For all 30 patients, we collected their pathological staging after surgery, which was used as the ground truth label for evaluating the imaging staging. After obtaining their synthetic PET images, we conducted the experiment in a blind way in which two in-house radiologists were blind of PET image types (synthetic or real). For each lung cancer cases, one radiologist provided the stage reading based on the CT paired with ground-truth PET, and another radiologist independently provided the stage based on the CT paired with the synthetic PET. To mitigate the bias, each radiologist randomly read 50% cases with read PET and 50% cases with synthetic PET (Table S1B).

Clinical validation: Lung cancer survival prediction after standard treatment

We performed survival analysis using the tumor features obtained from the synthetic PET images, and independently tested them on MDA-TRAIN, MDA-TEST, TCIA-STANFORD, and NSCLC-RT cohorts to test whether the synthetic PET can provide clinically relevant information comparable to ground-truth PET scan. In particular, we evaluated the individual feature’s prognostic value to compare between synthetic PET and ground-truth PET.

Quantification and statistical analysis

Statistical analysis

We reported the diagnostic model’s F1 score, sensitivity, specificity, and accuracy when classifying an indeterminate lung nodule into benign versus malignant. Receiver operating characteristic (ROC) curve and area under the curve (AUC) analysis were employed as evaluation metrics to assess the performance and effectiveness in predicting the lung cancer development. For prognostic evaluation, Harrell’s C statistics (C-index) was used to measure the goodness of fit between a PET feature and overall survival (OS) time. The Kaplan-Meier (KM) curves of patients’ OS along with their hazard ratio (HR) and p value were used to show the performance of the tumor features extracted from the individual PET features. The statistical tests were double-sided, with p values less than 0.05 or FDR less than 0.25 assumed statistically significant. All statistical analyses were performed in R 3.6.1.

Published: March 11, 2024