Radiomics of ¹⁸F Fluorodeoxyglucose PET/CT Images Predicts Severe Immune-related Adverse Events in Patients with NSCLC

Abstract

Purpose

To investigate the performance of pretreatment fluorine 18 (¹⁸F) fluorodeoxyglucose PET/CT radiomics in predicting severe immune-related adverse events (irSAEs) among patients with advanced non–small cell lung cancer (NSCLC) treated with immunotherapy, which is important in optimizing treatment plans and alleviating future complications with early interventions.

Materials and Methods

The retrospective arm of this study included 146 patients with histologically confirmed stage IIIB-IV NSCLC who were treated with immune checkpoint blockade between June 2011 and December 2017 and who were split into training (n = 97) and test (n = 49) cohorts. A prospective validation arm enrolled 48 patients before initiation of immunotherapy between January 2018 and June 2019 as an independent test cohort. Radiomics features extracted from baseline (preimmunotherapy treatment) PET, CT, and PET/CT fusion images were used to generate a radiomics score (RS) to quantify patient risk for developing irSAEs by an improved least absolute shrinkage and selection operator method. Weighted multivariable logistic regression analysis was then used to develop a nomogram model to predict irSAEs, which was assessed by its calibration, discrimination, and clinical usefulness.

Results

The radiomics nomogram, incorporating the RS, type of immune checkpoint blockade, and dosing schedule, was able to predict patients with and without irSAEs with area under the receiver operating characteristic curve of 0.92 (95% confidence interval [CI]: 0.86, 0.98), 0.92 (95% CI: 0.86, 0.99), and 0.88 (95% CI: 0.78, 0.97) in the training, test, and prospective validation cohorts, respectively. Decision curve analysis showed that the radiomics nomogram model had the highest overall net benefit.

Conclusion

A high RS is a significant risk factor for development of irSAEs, demonstrating the value of PET/CT images in predicting irSAEs. By the identification, at baseline, of patients with NSCLC most likely to have irSAEs, treatment plans can be optimized before initiation of immunotherapy.

Supplemental material is available for this article.

© RSNA, 2020

See also the commentary by Yousefi.

Summary

A nomogram model consisting of a radiomics score obtained from pretreatment PET/CT images, immunotherapy type, and dosage has the potential to identify patients at higher risk of having severe immune-related adverse events, which is important to optimize patient management and treatment plans.

Key Points

• ■ A radiomics score based on PET/CT is capable of estimating the risk of patients developing severe immune-related adverse events (irSAEs) and reveals a potential biologic association between programmed death ligand 1 expression and development of irSAEs.

• ■ The immunotherapy type and dosage were independent clinical risk factors for irSAEs.

• ■ A nomogram consisting of the radiomics score and clinical risk factors achieved significantly higher area under the receiver operating characteristic curve (AUC) of 0.92, 0.92, and 0.88 compared with clinical risk factors alone, which yielded AUCs of 0.74 (P = .002), 0.76 (P = .090), and 0.68 (P = .020) in the training cohort and two test cohorts, respectively.

Introduction

In the United States, lung cancer is the second most commonly diagnosed cancer and the leading cause of cancer-related deaths. There have been only small improvements in 5-year survival among patients with lung cancer, primarily due to high rates of late-stage detection, for which the survival rates are dismal (1). Until the advent of immunotherapy, there were limited treatment options for patients with late-stage non–small cell lung cancer (NSCLC). Among the many immunotherapeutic strategies, immune checkpoint blockade agents targeting the immunosuppressive molecules cytotoxic T-lymphocyte–associated protein 4 (CTLA-4), programmed cell death protein (PD-1), and its ligand PD-L1 have shown significant clinical benefit in treatment of late-stage NSCLC (2,3).

However, checkpoint blockade can lead to development of autoimmune manifestations (4), leading to immune-related adverse events (irAEs). irAEs are diverse, have different clinical implications, and can affect any organ system. Although deaths from irAEs are rare, deaths due to myocarditis, pneumonitis, colitis, and neurologic events can occur (5). According to Common Terminology Criteria for Adverse Events, irAEs are graded from 1 to 5 according to their severity (6). For patients with severe irAEs (irSAEs) (grade 3 or higher), treatment with immune checkpoint blockades should be permanently discontinued. According to recent studies, the rate of irSAEs is estimated to be between 7% and 43% (4,7–12). Therefore, identifying patients at risk for irSAEs before the start of immunotherapy is a critical unmet need that could optimize patient management and treatment planning and alleviate future complications with early interventions.

Currently, risk factors for irSAEs are ill defined (5) and directed studies are limited. Genetics influence risk of autoimmune disease in the absence of immune checkpoint blockade (13). Prior studies have shown that CD177 and CEACAM1 have promise as biomarkers for gastrointestinal toxic effects due to anti–CTLA-4 antibodies (14). Serum levels of sCD163 and CXCL5 may serve as possible prognostic biomarkers for irAEs in patients with advanced melanoma treated with nivolumab (15), and the increase from baseline in eosinophils and interleukin 17 after anti–CTLA-4 antibody treatment has also been shown to be associated with irAEs (16,17). Interestingly, the gastrointestinal microbiota has also been investigated, which suggests that patients with a predominance of bacteria from the Bacteroidetes phylum have reduced rates of immune-related colitis (18). Notably, none of these tests are routinely performed and must be specifically requested if there is suspicion of an irAE.

In contrast, imaging studies such as fluorine 18 (¹⁸F) fluorodeoxyglucose (FDG) PET/CT for NSCLC staging are routinely performed as part of the standard of care. We have well-established methods to rapidly extract a large suite of quantitative image features from standard-of-care imaging, including PET/CT. Conversion of digital medical images into mineable high-dimensional data is a process known as radiomics (19). Radiomics is motivated by the premise that standard biomedical images contain information that reflects the underlying pathophysiology within the volumes of interest, such as tumors, and that these relationships can be revealed by conversion of images to structured data and subsequent statistical analyses.

Radiomic analyses have been extensively performed on CT images in patients with NSCLC (20,21). Recent studies have revealed that radiomics of PET/CT images can be useful for prognosis or therapeutic response assessment (22–24). Thus, we hypothesized that radiomics of pretreatment PET/CT images might identify patients at risk for irSAEs. The goal of this study was to investigate clinical risk factors for irSAEs and assess whether PET/CT images can predict irSAEs in patients with advanced NSCLC treated with immune checkpoint blockade using radiomics analysis.

Materials and Methods

Patients

This study was approved by the institutional review board at the University of South Florida, and the need for informed consent was waived to ensure anonymity. For the retrospective arm of this study, inclusion criteria included patients with histologically confirmed advanced stage (IIIB and IV) NSCLC who were treated with immune checkpoint blockade between June 2011 and December 2017 at H. Lee Moffitt Cancer Center & Research Institute. To enroll more data for analysis and compare the differences between different immune checkpoint blockades in generating irSAEs, patients from different clinical trials treated with four different treatments were enrolled: (a) single anti–PD-(L)1 agent, (b) combined anti–PD-(L)1 agent and anti–CTLA-4 agent, (c) combined anti–PD-(L)1 agent and gefitinib, and (d) combined anti–PD-(L)1 agent and chemotherapeutic agent. A detailed description of the dosage schedule, dosing discretization, and treatment discretization is provided in Appendices E1–E3 (supplement).

Generally, the onset of toxic effects occurs within the first 4 months in 85% of patients (25), so the minimum follow-up period in our study was 6 months after initiation of immunotherapy. An additional inclusion criterion was that all patients had to have undergone PET/CT within 6 months of therapy initiation with no intervening therapies. The detailed inclusion and exclusion criteria are shown in Figure 1. On the basis of these criteria, we identified 146 patients, who were randomly divided into a training cohort (97 patients) and an independent test cohort (49 patients) on the only condition that the two cohorts have similar distributions of FDG uptake, measured as maximum standardized uptake value (SUVmax), to ensure that the two cohorts had similar distributions of metabolic characteristics. Furthermore, 48 patients who had received immunotherapy from H. Lee Moffitt Cancer Center & Research Institute between January 2018 and June 2019 were enrolled in a prospective study with the same inclusion and exclusion criteria as the cohort used to train the model.

Figure 1:

Diagram of study participant enrollment. AE = adverse event, G3 = grade 3, irSAE = severe immune-related AE, NSCLC = non–small cell lung cancer.

Clinical characteristics were obtained from the medical records, including age at diagnosis, sex, body mass index, historical treatments, smoking status, history of chronic obstructive pulmonary disease, Eastern Cooperative Oncology Group performance status, and sites of distant metastasis. Patients with irSAEs (grade 3 or worse) who were permanently discontinued from receiving immune checkpoint blockade treatments were used as positive cases for our study; patients without irSAEs were randomly selected as control patients. Progression-free survival (PFS) was chosen as the end point of the study, which was defined as the time from the start date of immunotherapy to progression (defined according to Response Evaluation Criteria in Solid Tumors [RECIST 1.1]). Patients who were free of progression or lost to follow-up were censored at the time of last confirmed contact.

PET/CT

The details of PET/CT are presented in Appendix E4 (supplement). Briefly, all PET images were converted into SUV units by normalizing the activity concentration to the dosage of ¹⁸F FDG injected and the patient body weight after decay correction.

Radiomic Feature Extraction

The primary lung tumors on PET and CT images were semiautomatically segmented with an improved level-set method based on gradient fields (26) and were further reviewed and corrected by a chest radiologist (J.Q.) with 16 years of experience who was blinded to the outcome label. After spatial registration using a rigid transformation by maximizing the Dice similarity coefficients on the condition that the maximal axial cross sections of the nodules were aligned, 1092 radiomic features were extracted from the PET images, CT images, and PET/CT fusion images on the basis of a minimum Kullback-Leibler divergence (KLD) criterion for these segmented tumors, as described in Appendices E5–E7 (supplement).

Statistical Analysis

Student t and Fisher exact tests were used to compare training and test cohorts for continuous variables and categorical variables, respectively. For PFS comparison, Kaplan-Meier analysis and a log-rank test were used. A P value less than .05 was regarded as significant, and statistical analyses were conducted with R (version 3.5.1) and Matlab (R2016b; MathWorks, Natick, Mass).

Feature Selection and Radiomic Signature Building

Because there is a low incidence of irSAEs, to correct for an imbalance between the two classes (irSAEs vs no irSAEs), data augmentation was used to balance and enlarge the training cohort (further described in Appendix E8 [supplement]). The robust features were first selected on the basis of resampling classification ability, and these were further reduced by Pearson grouping to reduce redundancy, as described in Appendix E9 (supplement). Subsequently, the least absolute shrinkage and selection operator (LASSO) method was used to select the useful features from the selected robust and nonredundant features. The penalty parameter of LASSO was selected using 10-fold cross validation via minimum mean cross-validated error. On the basis of these selected features, we computed a radiomics score (RS) for each patient through a linear combination weighted by the corresponding LASSO regression coefficients.

The stability of the RS was first investigated with analysis of variance (ANOVA) to compare the distribution of the radiomics signatures among the different reconstruction parameters. Second, different RS values of the prospective cohort were calculated using different segmented boundaries obtained through automated dilation and shrinkage of the tumor boundaries by one voxel and two voxels on PET images and three to six voxels (according to the ratio of PET axial resolution and CT axial resolution) on CT images, which were called dilation1, dilation2, shrinkage1, and shrinkage2. The effect of the segmentation could be evaluated through ANOVA analysis on different RS values and comparison of the corresponding discrimination performances using the Z test.

Development of Individualized Radiomics Nomogram Model

Univariable logistical regression analysis was performed on the training cohort with the RS and clinical risk factors, along with the commonly used PET/CT-derived metrics including SUVmax, metabolic tumor volume (27), and volume (from CT images) as input variables. Only factors with statistically significant odds ratio were used for developing the irSAE prediction model with multivariable logistical regression analysis. Because of the imbalance in the number of patients with different treatment methods, weighted multivariable logistical regression analysis was used, and the weight for each patient was determined by the product of the number of patients who received another two treatments. A backward stepwise selection was applied by using the likelihood ratio (LR) test with Akaike information criterion as the stopping rule (28). On the basis of this analysis, a radiomics nomogram model for irSAE identification was obtained for clinical use.

Qualitatively, calibration curves were plotted to determine the agreement between the estimated probability and the actual irSAE rate on the basis of the retrospective and prospective cohorts, respectively. Additionally, to compare the clinical usefulness of the different models, a decision curve analysis was performed by quantifying the net benefits at different threshold probabilities (29).

Quantitatively, the goodness of fit of the models was evaluated with Akaike information criterion, and the Hosmer-Lemeshow test (30), an insignificant test statistic implied that the model was well calibrated. The area under the receiver operating characteristic (ROC) curve (AUC) and classification accuracy were calculated to evaluate the discrimination performances of different models in the training, test, and prospective test cohorts. Furthermore, the positive likelihood ratio (+LR) and negative likelihood ratio (−LR) were also calculated for the diagnosis evaluation. To demonstrate the significantly incremental value of the RS with the clinical characteristics, total net reclassification improvement (NRI) was calculated.

Results

Clinical Characteristics

Among the 146 patients, there were 88 men and 58 women with an overall mean age of 65.72 years ± 12.88 (standard deviation) and a median PFS of 6.85 months. Five different autoimmune-related events were observed in this cohort, including six colitis/diarrhea, eight pneumonitis, two Guillain-Barré syndrome, three hepatitis, one myalgia, and one rash. The median (range) interval between the start of immunotherapy and occurrence of severe autoimmune disease was 3.55 months (range = 0.47–24.17 months). For the 48 retrospective patients, who consisted of 23 men and 25 women with a mean age of 66.44 years ± 9.77 and median PFS of 6.78 months, three different autoimmune-related events (four colitis/diarrhea, four pneumonitis, and one hepatitis) were observed within 1.43 months (range = 0.93–14.63 months) since the start of immunotherapy. The demographic characteristics are shown in Table 1.

Table 1:

Demographic and Clinical Characteristics of Patients

There were no significant differences between the retrospective training and test cohorts for all the clinical characteristics or follow-up data, and the two cohorts had identical distributions of SUVmax (Fig E1, A in Appendix E1 [supplement]) and a statistically insignificant difference in PFS (P = .38; Fig E1, B in Appendix E1 [supplement]).

Feature Selection and Radiomics Signature Building

According to each feature’s resampling classification ability, 12 PET features, 41 CT features, and 12 KLD features were selected for their resampling classification ability. After Pearson grouping, 21 features with the largest resampling classification ability in each group were fed into the LASSO method. From these analyses, five features were selected to construct the RS, which was incorporated into the following calculation formula:

Note that the features are named as f_type, where “f “ represents the feature name and “type” represents the type of the image (PET, CT, or KLD images) used to calculate f. Coarseness is the coarseness calculated from the neighborhood gray-level difference matrix; GLN is gray-level nonuniformity calculated from the gray-level size zone matrix; GS is geometric symmetry calculated from the texture spectrum matrix; ZP is zone percentage calculated from the gray-level size zone matrix; SZLGE is short-zone gray-level emphasis calculated from the gray-level size zone matrix; and SRLGE is short-run low gray-level emphasis calculated from the run-length matrix. The distributions of the RS and the irSAE status for each patient in the training, test, and prospective cohorts are shown in Figure 2.

Figure 2:

A–C, The radiomics score (RS) and, D–F, the radiomics nomogram predicted the probability for every patient in the training cohort, test cohort, and prospective test cohort. The irSAE status is marked with different colors. irAE = immune-related adverse event, irSAE = severe irAE.

To perform ANOVA analysis, we regarded the scanners that had less than five cases as one type, obtaining eight different types all together. The P value of the ANOVA analysis between groups for RS was .42, and the P value of the pairwise posthoc least significant difference test was in the range of .80–1, indicating that the RS was stable among different scanners and reconstruction parameters, which is also shown by the box plot (Fig E2, A in Appendix E2 [supplement]). The prospective study also showed the stability of RS with a P value of .22 with ANOVA analysis, and the Student t test showed no significant difference between the retrospective and prospective cohorts with the same scanners and reconstruction parameter (P > .05) (details shown in Fig E2, B in Appendix E2 [supplement]).

The mean RSs and the relative standard deviations obtained over different boundaries are shown in Figure E3 in Appendix E3 (supplement) together with the RS calculated from the reader-corrected segmentation results. There was no significant difference between the RS obtained with the dilation1, shrinkage1, and shrinkage2 boundaries (P = .21, .15, and .14). Because of the inclusion of pleura or other soft tissues, there was a significant difference between the RS obtained with dilation2 and the accurate boundaries (P = .022). The AUC values for these four RSs were 0.81–0.85, showing no significant difference compared with the accurate segmentation (P = .27–.46, Z test), indicating that RS was stable across small variations of segmentation boundaries.

Diagnostic Validation of Radiomics Signature

There was a significant difference in RS between patients with irSAEs and those with non-irSAEs in the training cohort (P < .001), which was validated in an independent test cohort (P < .001) and the prospective cohort (P < .001). This signature yielded AUC values of 0.88 (95% confidence interval [CI]: 0.81, 0.95), 0.90 (95% CI: 0.82, 0.98), and 0.86 (95% CI: 0.76, 0.96) in the training, test, and prospective cohorts, respectively. Detailed information on radiomics signature performance is shown in Table 2, and the corresponding ROC curves are shown in Figure 3.

Table 2:

Performance of Different Models in irSAE Prediction

Figure 3:

A, Receiver operating characteristic (ROC) curves for the radiomics score (RS) in the augmented training, training, test, and prospective test cohorts. B, C, ROC curves for the RS in patients who received anti–programmed cell death ligand 1 (PD-L1) immunotherapy and anti–programmed cell death protein 1 (PD-1) immunotherapy. D, ROC curves for the radiomics nomogram in the augmented training, training, test, and prospective test cohorts. E, F, ROC curves for the radiomics nomogram in patients who received anti–PD-L1 immunotherapy and anti–PD-1 immunotherapy. Given that no severe immune-related adverse events occurred in the prospective test cohorts who received anti–PD-L1 immunotherapy, no ROC curves for the prospective test cohorts are given in B and D. ACC = accuracy, AUC = area under the ROC curve.

Given that the PET/CT scans of the retrospective cohort were acquired at different time points before the start of immunotherapy, the temporal relationship between the PET/CT scan time and the output of the model was further investigated. AUC values were calculated for the subgroups with various interval times of 1 month or less (G1), 1–2 months (G2), 2–3 months (G3), and 3–6 months (G4): These were 0.93 (95% CI: 0.84, 1.00), 0.89 (95% CI: 0.77, 1.00), 0.85 (95% CI: 0.62, 1.00), and 0.82 (95% CI: 0.57, 1.00), respectively. Although the AUC appeared to decrease with increasing time, the differences were not significant (P > .05). Nonetheless, all the patients in the prospective cohort underwent PET/CT scans within 1 month before the start of immunotherapy.

Development of Individualized Radiomics Nomogram Model for irSAEs Prediction

Univariable logistical regression analysis identified the RS, immunotherapy type, and dosage as strong predictors for irSAEs (P < .001, P = .032, and P = .029, respectively) (Table 3). The further weighted multivariate logistic regression analysis identified them as independent risk factors (RS: P < .001, hazard ratio [HR]: 4.20 × 10³, 95% CI: 176.37, 9.98 × 10⁴; immunotherapy type: P = .005, HR: 2.40, 95% CI: 1.30, 4.43; dosage: P = .074, HR: 2.15, 95% CI: 0.93, 4.97), presented as a nomogram, shown in Figure 4, A. The distributions of the nomogram predicted probability and irSAE status for each patient in the training, test, and prospective test cohorts and are shown in Figure 2.

Table 3:

Logistic Regression Analysis of Risk Factors for irSAE Prediction

Figure 4:

Radiomics nomogram to estimate the risk of severe immune-related adverse effect (irSAE) and performance evaluation of its results. A, Detailed radiomics nomogram constructed with the radiomics score (RS), treatment type, and dosage. For example, in a patient with RS of 0.6 who received anti–programmed death ligand 1 (PD[L]1) plus chemotherapy (Chemo) with a dosage level of 2, the total points would be 70 (RS of 0.6 = 55 points, anti–PD[L]1 plus Chemo = 10 points, dosage level of 2 = 5 points), which corresponds to a risk of developing irSAE of 0.9. CTLA-4 = cytotoxic T-lymphocyte–associated protein 4. B, Assessment of the model calibration in the retrospective and prospective cohorts. C, Scatterplot of the nomogram-predicted irSAE probability in individuals. Blue = training cohort, red = test cohort, orange = prospective cohort. irAE = immune-related adverse event. D, Decision curves for the RS, the complete radiomics nomogram (RS + clinical), and the clinical-only nomogram.

The calibration curves of the radiomics nomogram predicted that irSAE probability showed good agreement in the retrospective and prospective cohort (Fig 4, B). The Hosmer-Lemeshow test showed a good logistic fit in the training (P = .79), test (P = .83), and prospective (P = .84) cohorts. The nomogram yielded AUC values of 0.92 (95% CI: 0.86, 0.98), 0.92 (95% CI: 0.86, 0.99), and 0.88 (95% CI: 0.78, 0.97) in the training, test, and prospective cohorts, respectively (Table 2). +LR (12.00) and −LR (0.15) in the test cohort showed that the radiomics nomogram had a large effect on increasing the probability of irSAEs and a moderate effect on decreasing the probability of irSAEs. The corresponding ROC curves are shown in Figure 3.

When compared with the RS, inclusion of the treatment type and dosage yielded a total NRI of 0.42 (95% CI: 0.15, 0.70; P = .0027) in the training cohort and 0.62 (95% CI: 0.020, 1.22; P = .043) in the test cohort, showing significantly improved classification accuracy for irSAE prediction. When compared with the clinical nomogram (immunotherapy type: P < .001, HR: 2.55, 95% CI: 1.51, 4.29; and dosage: P < .001, HR: 3.36, 95% CI: 1.72, 6.55), inclusion of the RS yielded a total NRI of 1.07 (95% CI: 0.81, 1.32; P < .001) in the training cohort, 1.43 (95% CI: 0.87, 1.99; P < .001) in the test cohort, and 1.09 (95% CI: 0.49, 1.70; P < .001) in the prospective cohort, also showing significantly improved classification accuracy for irSAE prediction.

Results of the decision curve analysis (Fig 4, D) revealed the performance of the RS, clinical nomogram model, and radiomics nomogram model in clinical application. All three models showed advantages compared with the extreme schema, wherein all or no patients are assumed to develop irSAE. When the three models were compared, the RS + type + dosage nomogram model had the highest overall net benefit across the range of reasonable threshold probabilities in both retrospective and prospective cohorts. Thus, this nomogram model provided the optimal decision-making strategy to maximize the net benefit (ie, the difference between the expected benefit and expected harm), regardless of the determination of the cut point.

Patients in the retrospective cohorts were further clustered into high-risk and low-risk groups, according to the cutoff point determined by the Youden index of the augmented training cohort. The median PFS of high-risk patients was 9.43 months, significantly longer than that of low-risk patients, with a median PFS of 6.23 months (P = .042) (Fig 5, A). Further stratification of patients in the prospective cohort showed that the high-risk patients had a significantly longer median PFS of 11.47 months versus the 6.23 months of the low-risk patients (P = .049) (Fig 5, B).

Figure 5:

Kaplan-Meier survival curves for progression-free survival according to the radiomics nomogram–predicted probability in the, A, retrospective cohorts and the, B, prospective cohort. GH = high risk, GL = low risk.

Discussion

In this study, we observed that a high RS combined with type and dosage of immunotherapy were risk factors for development of irSAEs. A radiomics nomogram model that incorporated these risk factors was validated to predict onset of irSAEs, and its performance in the prospective cohort demonstrated the potential generalizability of the model. Early intervention for the identified patients with a high risk of irSAEs will help those patients obtain more benefit from immunotherapy with less risk of toxic effects.

Radiomics provides a way to leverage PET/CT images to predict metastasis (31) and the prognosis for patients with NSCLC receiving new therapy (32,33). This analysis connects the metabolic-level information with the macroscopic-level representation. Because any organ system could be affected by an unbalanced immune system, determination of the organs at risk as regions of interest can be challenging. Our radiomic analysis of the irSAEs was confined to the primary nodule, given our assumption that the primary nodule provides pertinent genetic and microenvironmental information (20,21).

When we investigated the informative components of the RS, the most important feature with the highest weight was short-zone gray-level emphasis (SZLGE) from the KLD images. This can be interpreted to mean that patients with tumors consisting of many small, lowly metabolic and attenuation-connected regions were more likely to have an irSAE. In addition, smaller CT zone percentage (CT_ZP) (strongly linear region of interest volumes), smaller CT geometric symmetry (CT_GS) (irregular shape), and smaller CT gray-level nonuniformity (CT_GLN) were also related to a high risk of developing irSAEs. According to Saeed-Vafa et al (34), more linear and irregularly shaped tumors (ie, those that are growing around blood vessels) express less PD-L1, suggesting that an irSAE may be associated with PD-L1 expression; however, this will require further study with larger cohorts.

Radiomic analyses of diagnostic CT images have shown value in predicting response to checkpoint blockade (35). However, in the current study, when a similar model was trained using only CT images, the nomogram constructed with the corresponding RS achieved significantly lower AUCs of 0.84 (P = .08), 0.80 (P = .17), and 0.72 (P = .04) in the training, test, and prospective cohorts, respectively. A possible explanation for the low predictive power of CT in the current study is that the CT images in PET/CT have lower resolution and are not contrast-enhanced, unlike diagnostic CT images. In addition, some studies have shown that the metabolic modifications seen at PET are more predictive than the morphologic modifications seen at CT, especially in prediction of early immunotherapy response (36,37). Therefore, the proposed RS containing both metabolic and anatomic information could be better used in predicting irSAEs.

When the relationship between clinical characteristics and irSAEs was investigated, only the larger dosage and the combination of different antibodies significantly increased the risk of irSAEs, consistent with the results of studies by Larkin et al (7) and by Simeone and Ascierto (38). The target of the checkpoint blockade (PD-L1 or PD-1) was not related to irSAEs in this synchronization analysis of results from different clinical trials, which was not consistent with observations in other trials (8,9). This may be a function of our particular patient population. Notably, the RS and radiomics nomogram had higher predictive power in patients who received anti–PD-1 antibodies (Fig 3) than in those who received anti–PD-L1 antibodies. A subsequent prognostic investigation of the current nomogram model showed that patients with a higher risk of irSAEs may have a better prognosis, consistent with the results of Dovedi et al (39).

Another strength of this study was that PET/CT images were collected from different centers, which had scanners from different manufacturers, models, and reconstruction parameters. Through downsampling and upsampling, as well as application of resampling classification in feature selecting, we obtained a stable RS validated by an ANOVA test (Fig E2 in Appendix E2 [supplement]). A moderate concern was that the AUC values of the test cohorts were consistently higher than those of the training cohorts, although these differences were not significant (P = .59). Although this may have been a product of overfitting, it may also have been due to slight differences between the training and testing populations that were not captured demographically (Fig E1A in Appendix E1 [supplement]) or though SUV matching. Further independent validation on the PET/CT images of the prospective cohort, which were obtained with different equipment, showed no significant differences in RS between the prospective and retrospective cohorts (P = .38, Wilcoxon test), and the AUC values of the RS and the radiomics nomogram in the prospective cohort were 0.86 (95% CI: 0.76, 0.96) and 0.88 (95% CI: 0.78, 0.97), mitigating the concern about overfitting.

However, the present study did have some limitations. First, the sample size of patients with irSAEs was small relative to the entire cohort. However, we used the augmented data to maintain a balance between groups with and without irSAEs. Second, the proportions of the three different treatments were imbalanced, but we used weighted logistic regression to construct the nomogram model. Third, genomic data, which can be significantly associated with the outcome of immunotherapy (40), were unavailable in this cohort. With the collection of the genomic data, their association with irSAEs and the added value to the RS in predicting irSAEs will be investigated in our future work. Fourth, although it contained independent training and testing cohorts, this study was based on limited data from a single institution. The findings warrant further study in larger prospective multi-institution populations.

Data from pretreatment PET/CT images have the potential to identify patients with a greater probability of experiencing irSAEs after the start of immunotherapy through the radiomics nomogram model. This model could contribute to patient management and treatment planning as well as reduce future complications with early interventions after initiation of immunotherapy, pending further external validation with larger cohorts.